Fraser Rae, 28, entered Glasgow's Central Mosque on September 23 and urged those inside to run away, prosecutors said.
Rae, from Johnstone in Renfrewshire, also shouted that he had a gun and threatened to shoot those near him, the court was told.
Following his arrest, Rae's backpack was searched and found to contain a number of items including a 'Scream' mask, gloves and scissors. He was also shouting racist and threatening remarks.
At Glasgow Sheriff Court yesterday, he was jailed for two years and four months after earlier admitting the offences.
He pleaded guilty to two charges of threatening and abusive behaviour, and another of carrying a knife.
Speaking after sentencing, John Dunn, procurator fiscal for the West of Scotland, said: "Everyone has the right to live free from violence, threats, intimidation or the fear of harassment or abuse stemming from the prejudice of another person.
"To be targeted because of the colour of skin or faith is totally unacceptable.
"Hopefully the conviction and sentencing of Rae will encourage the public to report all hate crimes.
"They can have the confidence that all such crimes will be investigated carefully and prosecuted robustly."
Detective Inspector Joe McKerns, of Strathclyde Police, said: "The actions of this man led to many people feeling frightened and alarmed.
"We will not tolerate acts targeted against minority communities and will always take robust and prompt action, no matter how minor the offence.
"No person should be subject to any form of victimisation and such cases will be dealt with swiftly."