Glasgow City Council's Planning Committee rejected an application by St Enoch Centre Investors and Caffe Nero to open a coffee house on the ground floor of the Buck's Head building on Argyle Street.
Councillor Graeme Hendry, leader of the opposition SNP group, said the decision could drive away future potential investment from the city.
He said: "Argyle Street is being strangled by the insistence of the council in enforcing planning guidance designed for the 70s rather than the current day.
"All successful shopping areas now require a mixed offer of retail and leisure, so the refusal to allow even a simple coffee shop in this building does nothing to benefit our city.
"Instead, it runs the risk of driving away much needed investment in this area.
"The council should have reviewed this policy decades ago and it must get it done as a matter of priority.
"It's time to wake up and smell the coffee."
The Buck's Head building was designed by Alexander 'Greek' Thomson in 1862 and gets its name from John Mossman's deer statue that sits proudly on its roof.
It originally housed The Glasgow & London Clothing Company, but the upper floors have been unoccupied for years.
The ground floor was most recently occupied by sports chain JJB, which went under last year, and the American Candy Co.
A spokeswoman for Glasgow City Council said it rejected the application as it was "contrary to planning polices." She said: "We have rejected other advances in this area to turn retail units to other class uses such as fast food restaurants, hot food takeaways, bookmakers and pawnbrokers.
"These have been rejected as they are contrary to our planning policies which have been put in place to protect, maintain and grow the offering in our city centre.
"In its simplest terms, a change from a class one retailing to class three food and drink would mean in the future that a unit with this classification could be used for any business selling food or drink to eat on the premises as there is no difference, for example, between a coffee shop or fast food restaurant.
"We have these policies in place to project our city centre retailing and we would not wish to open the flood gates to other calls for reviews of changes of uses at other sites in our city centre which have previous been rejected.
"We will be reviewing our retail planning policies in the coming months to ensure that we maintain a strong retail heart in the centre of Glasgow, which is the best retail offering outside of London.
"However, this decision is in line with our current policies."
Caffe Nero already has six city centre outlets and one in Partick.