Glasgow City Council is consulting on a move to close three of its seven centres which cater for around 400 people each day.
It says people with severe learning difficulties would be found places in other centres while those who are more able would move to mainstream centres.
But carers are outraged at the plan and packed out a meeting in the Unison offices in Merchant City.
Tommy Gorman, who lives in the north of the city, cares for his 21-year-old daughter Patsy who attends Summerston day centre, which is threatened with the axe.
He told the meeting: "The council is making a vicious attack on the learning disabled community and we intend to fight it.
"The council needs a massive budget cut and the first door they have knocked on is the weakest group of people in Glasgow.
"If the council carries through the policy it will put very vulnerable people in a position were they are open to being the victim of predators."
Another carer said: "My son attends Summerston centre - take him out of it and he will think he has done something wrong."
Ian Hood, co-ordinator for the Learning Disability Alliance Scotland, said: "There has been no assessment of the people who use day centres in Glasgow, so how can anyone make any judgment about their needs.
"For some, the alternative to a day centre will be a £5 lunch in a church hall - is that what we have come to?"
Carers agreed unanimously they had no confidence in the city council consultation and called for it to be taken over by the Care Inspectorate.
They also decided to investigate legal action against the council and to lobby a meeting of the council next month which will discuss the axe plan.
A city council spokesman said: "A consultation is now under way and we are working to a detailed plan that aims to bring forward views from the widest possible range of stakeholders.
"We see input from service users, carers, trades unions and others as being is an essential part of the consultation process.
"The results of the consultation exercise will be included in a paper with clear proposals for the way forward on learning disability day services, which it is hoped will go to the executive committee in January when a decision will be taken."