Youth custodial officers have walked off the job, accusing the Corrective Services Minister of deliberately portraying them as lazy thugs.

In the past week Joe Francis has publicly criticised guards from the Banksia Hill detention centre claiming half of them had called in sick at the weekend and not turned up to work.

Toni Walkington from the Community and Public Sector Union says only one youth custodial staff member called in sick on the Saturday.

"Our members are very disappointed that Mr Francis continues to make false accusations about their dedication to the role and we have written to the Premier stating that we want an apology and for Corrective Services to be removed from Mr Francis's portfolio," Ms Walkington said.

Around 60 officers attended a stop work meeting at Hakea Prison today and passed a motion of no confidence in Minister Francis.

Banksia Hill is the state's only correctional facility for juveniles but it was severely damaged in January when detainees rioted at the centre resulting in a damage bill of more than $3-million.

The detainees - most of which have pleaded guilty to charges stemming from the riot - have been housed at Hakea adult prison while repairs to Banksia Hill are carried out.