It comes as the government tries to find out how a group of workers were able to depart for New Zealand for seasonal work, despite the temporary ban on the regional seasonal employer scheme.
RNZ Pacific reports Tuala Iosefo Ponifasio told the Parliament in Apia that the Australian Government made the announcement on new visas without consulting the Samoan Government.
He said this would hurt the Samoan labour workforce and lead to the loss of more skilled workers and their families to Australia permanently.
The 3000 visas would be allocated annually through a ballot process across Pacific countries and Timor Leste.
Those selected from the ballot will be invited to apply for permanent residence and the new Pacific Engagement visa (PEV) will start in July to boost permanent Pacific migration to Australia.
"There has been no communications with our Government on this. They will start by taking our workers for three years and I believe they will not return," said Tuala.
He said they would take workers in the tourism and hospitality industries plus teachers and police officers.
Tuala said Cabinet had to stand firm and ensure it is properly prepared to deal with any fallout.
It is envisaged that the programme will benefit participating countries through increased remittances, opportunities for skills and knowledge transfer, the provision of a new migration pathway, and enhanced people-to-people links.
Discussions revealed that the PEV will be broadly modelled on New Zealand's Pacific Access Category (PAC) Resident visa which has been available since 2002.