Immigration Minister Tony Burke says he has spoken "face-to-face" with asylum seekers on Manus Island who have decided to return home because of Labor's new offshore settlement policy.
Last month, and if found to be refugees, resettled there as well.
A similar arrangement has since been struck with Nauru.
Mr Burke says he spoke with two groups of men from Iran at the canteen in the offshore processing facility in PNG.
"I had the opportunity, for the first time, to have very detailed conversations - (the) first time face-to-face for people who were there for processing," he said.
"I did not allow the department or anyone to select in advance who it would be - I simply turned up at the cafeteria area and sat down during breakfast."
Mr Burke said one group told him they had already decided to go back home and were in discussions with the International Organisation for Migration about returning.
He said the other group were still grappling with the decision, including one man who was on a boat at the time Labor decided to adopt its offshore resettlement policy.
"He indicated to me directly that had he known about the policy he would not have boarded the vessel," Mr Burke said.
"He's still trying to work out a bit more about Papua New Guinea before he makes a decision as to whether he would rather go home."
Meanwhile, Mr Burke says he is very close to sending family groups from Christmas Island to Nauru, adding that the PNG government has specifically requested Australia start sending family groups to Manus Island soon.
So far 276 men have been sent to Manus Island, but Mr Burke says women and children will be transferred to Nauru within weeks because the facilities will be ready there earlier.
"I know that some people have tried to anticipate that maybe we won't send family groups or we won't send unaccompanied minors," he said.
Opposition Leader Tony Abbott has raised the prospect of a new people smuggling trade in the Torres Strait, after two men from Somalia were found on a remote island in the area.
"If you're going to have a PNG plan, you also need to have a Torres Strait plan," he said.
Mr Burke says the Coalition's reaction is "hysterical".
"There have been two vessels each containing two people - each has been intercepted," he said.
"So I'm yet to see an argument other than a hysterical reaction from the opposition."
One boat was intercepted today north-west of Ashmore Island, carrying 121 passengers and five crew.
Two boats were intercepted yesterday, one near Christmas Island carrying an estimated 50 passengers and two crew and one near Ashmore Island carrying 31 passengers and three crew.