Support grows for PM Albanese's luxury home buy as critics question timing and context