Chris Boardman made his claims as he was commentating for the BBC during yesterday's event.
As the bikes were passing through Muirhead in North Lanarkshire, Boardman said: "It's too subtle to pick up with our cameras but some of these surfaces really are quite rough."
Boardman, who won gold in the individual pursuit in the 1992 Olympics, was backed up by fellow commentator Simon Brotherton.
Brotherton, who has reported on several Tour de France races, said at one point: "See the bike there visibly bouncing off one of the potholes in the middle of the road."
A spokeswoman for the Games said: "Glasgow 2014 has worked closely with local authorities and was pleased with the course and its condition."
A North Lanarkshire Council spokesman hit back at the criticism.
He said: "We have inves-ted heavily in resurfacing and repairs to the roads in North Lanarkshire for the Glasgow 2014 Common-wealth Games Time Trial.
"The appropriate tech-nical experts at Glasgow 2014 have expressed their pleasure with the quality of the route and we utterly refute any suggestion that any potholes exist on the route."
Neil Bisland, owner of Saltmarket cycle shop Billy Bilsland, said: "Good roads are key for cyclists in general and cycle safety."
Neil, whose father Billy is a former Commonwealth and Olympic cyclist, said: "With the time trial the guys are tucked in an aggressive position, they are not looking up the road, if they hit a pot hole at the speeds they were going, you could kill yourself.
"A smooth surface is essential for the guys to be safe."
Bike mechanic Barry Wilson travels regularly on the time trial route, and said: "There are a few bits here and there on that route - the road surface is mixed. One minute it is smooth and the next it can be rough."
Hundreds lined the routes for the men and women's time trials, which travelled through Glasgow, East Dunbartonshire and North Lanarkshire.
England's Alex Dowsett and New Zealand's Linda Villumsen claimed the golds.
The women's time trial at 10am yesterday saw Ms Villumsen take first place at 42 minutes 25 seconds.
She was followed by Emma Pooley, of England, and Katrin Garfoot, of Australia.
Team Scotland athlete Katie Archibald, of Clydebank, came in fifth at 43 minutes 30 seconds while Lucy Coldwell, of Clydebank, made eighth place at 44 minutes 3 seconds.
In the afternoon, Dowsett finished in 47 minutes 41 seconds to take gold by 9.3seconds.