He made the charge at his cabinet's first session on October 31.
In recent weeks, there has been an increase the occupation of abandoned buildings by refugees from Georgia's breakaway regions of Abkhazia and South Ossetia along with other homeless people.
Those numbers increased after Ivanishvili's Georgian Dream political coalition defeated the ruling United National Movement party in parliamentary elections on October 1.
Ivanishvili recommended that the government investigate the situation.
He added that authorities should begin monitoring the squatters.