Phillip Hammond said a 'yes' vote could cut Scotland off from UK Ministry of Defence orders for warships.
He told an audience in Edinburgh, at defence firm, Selex-Es, that as well as the shipbuilding industry the level of jobs at Faslane and Coulport would not be sustainable without UK support.
Mr Hammond said the current contracts for new aircraft carriers was responsible for maintaining 4000 Scots jobs.
He said: "Yet the spectre of Scottish independence threatens to put those shipbuilding jobs in jeopardy.
"Outside the two world wars, the UK has not had a warship built abroad since the start of the 20th century.
"There is absolutely no plan to change that posture.
"So these jobs, the very existence of those yards, would become dependent upon orders placed by an independent Scottish government."
Union leaders at the Glasgow yards backed his claims.
Duncan McPhee Unite convenor at BAE Systems' Scotstoun yard, said: "The reality is that if we left the UK, then it would be the end for shipbuilding in Scotland.
Angus Roberson, SNP defence spokesman, said the military would be better managed in an independent Scotland.
He said: "Instead of coming to Scotland to peddle the 'no' campaign's Project Fear, he should stay and debate the reality of Scotland's defence abilities as an independent country and Westminster's woeful record.
"Westminster's record is cutting personnel numbers to an historic low, closing bases despite promises to invest here, and sacking troops by posting out p45s."