Kenny MacAskill joined representatives from Police Scotland, Trading Standards and the Home Office for discussions on how to deal with the increasing problems posed by new psychoactive substances (NPS).
The summit addressed issues such as product safety and consumer protection powers, enforcement and prevention, and looked at powers available to local government.
Legal highs were involved in 47 deaths in 2012 and hundreds more people across Scotland have been admitted to hospital after taking them in the past five years.
Substances sold as legal highs are produced to have similar effects to drugs such as Ecstasy, but they fall outside the UK Government's misuse of drugs laws.
Mr MacAskill warned people to avoid them ahead of the summer music festival season.
He said: "Just because they are legal doesn't mean they are safe. Just because they are not illegal doesn't mean they can't cost you your life or cause other serious harm.
"We have to address supply and do that through law enforcement, but we also have to address demand. Young people have been trying this and some have sadly lost their lives at all too young an age.
"That's why we're trying to raise awareness and drive home the message that we're not here as middle-aged party poopers, we're trying to save their lives."
He added: "They are not legal highs, what you are getting is a chemical compound, a drug the precise contents of which we do not know and which are more likely to cause you harm or injury than give you a good time."
Some of the discussions at the event will contribute to the Scottish Government's response to the Home Office review of NPS legislation.
In 2012, NPS were implicated in 47 deaths in Scotland and in five of those cases they were the only drug present, according to official statistics, published by the General Register Office for Scotland.
The UN and the EU recorded 73 new drugs in 2012, with 693 online shops selling legal highs across Europe in the same year.
Police said it was an increasing problem and urged people not to use the substances.
Detective Chief Inspector Garry Mitchell of the Police Scotland specialist crime division said: "We would encourage people not to take these substances. Nobody knows what's in them, nobody knows what the health implications are of these substances.
"Through unfortunate circumstances, NPS have contributed to deaths and serious health problems."
In June last year, the UK Government placed banning orders on four types of the legal high "N-Bomb" and the legal substance "Benzo Fury" after they were linked to deaths.
Legal highs are often labelled and sold as plant food or bath salts, or marked as not fit for human consumption, as a tactic to avoid the law, Community Safety Minister Roseanna Cunningham told MSPs earlier this year.
They are easily accessible online, and, according to research, have also been found to be sold at petrol stations, newsagents and takeaways.
Dave Liddell, director of the Scottish Drugs Forum, said it is not just young people who use NPS, and that existing drug users are increasingly using the substances as they are cheaper and more easily accessible than banned drugs.
Mr Liddell, who attended the summit, said a wide ranging approach is needed to tackle the problem.
He said: "The notion that we can deal with this through enforcement alone is clearly not the case. We need to be looking at credible messages from credible sources to young people."