The Swede has decided to look for new challenges in football and the process of appointing a replacement will now begin.
Mjallby told the Celtic website: "I have enormous affection for Celtic Football Club and it is with sadness that the time to leave has arrived. However, I now leave to seek some different opportunities.
"The club has been part of my life for almost two decades and I have been dedicated to them in every way, first of all as a player and then in management.
"As a player, it was a time when we achieved great success and I had the privilege to work with football people and players who were first-class and real quality. We had some great moments and memories I will always cherish.
"It was an honour to return to the club and work in management with Neil, Garry and the players, and we have worked hard over the last few years to once again bring the club success.
"I hope Neil, the staff and the players can deliver many more trophies and titles for our fans in the future. I would like to sincerely thank all our supporters for all they have given me and everything they have given to the club. They are special fans, the best I have ever seen in football."
Celtic manager Neil Lennon said: "I will be very sad to see Johan leave the Club. He has given us such great service over a number of years and he will always be a big part of Celtic.
"When I was given the chance to be in charge of the team, Johan was the obvious choice to join me in tackling the challenges ahead and from day one, he has been a very important part of our success.
"He was always respected by players and our supporters, who knew how committed he was to Celtic. Johan is a great man and will always be a great friend.
"I know whatever challenge Johan chooses next, he will be a great success and I wish him and his family nothing but good fortune for the future."
Peter Lawwell, Celtic chief executive, said: "We wish Johan absolute success. Johan is a man who has given Celtic so much, he has shown dedication to the Club and has demonstrated absolute professionalism throughout his association with Celtic.
"As a player and in management he has given us his total commitment, in recent years working closely with Neil to achieve great things for the club.
"On behalf of the board of Celtic we thank Johan for all he has given Celtic and wish him and his family the very best for the future."