The group's Rise Ye Sunken Ships album may have been a hit with critics and rock fans alike, but it took its toll on the New York native.
Many of the lyrics dealt with the suicide of Billy's brother, James, and so the frontman decided to refresh himself by going back to his old primary school to start writing new songs.
"The first album dealt with such difficult subject matter, that I needed to erase the chalkboard," says Billy, who brings Augustines to the Arches tonight.
"When I was a boy at school playing flute and saxophone there was this room we used.
"I'd been thinking about it, and that if we could find a room like that again then I could feel at peace. Eventually I thought, why not see if it's there?
"I was a difficult student, so this was going full circle - it was heartwarming for me to see the old faculty again."
That kick-started the process for Augustines, the band's sophomore album that delivers more heart-on-the-sleeve rock that's earned them comparisons to the likes of Bruce Springsteen, U2 and fellow New York outfit The National.
There's a more hopeful streak running through the record, after the darkness that covered Rise Ye Sunken Ships.
"As the last album dealt with suicide there was a lot of discussions that came out of it, and we did shows with the Civil Liberties Union and lots of great organisations," he recalls.
"But so much had been said on the album and we'd really gone into these depths, and seen homeless people and prisoners, all these people who'd been lost in the system, that I just felt spent.
"It was like there was sick on me, and I needed to clean myself up. So the connection to childhood was a very pure thing."
There had been other turbulent times for the band, too. Originally formed out of the ashes of the band Pela, Augustines were forced to call themselves We Are Augustines for their first album, after it emerged another band shared the same name.
Just getting a record out was an achievement in itself, and that led Billy to write with such bluntness.
"On the first record we were in a bubble and I didn't think we'd get out of it, so I just wrote," he explains.
"It was like coming out the closet in a way, because I didn't care how emotional it was.
"And then we survived, and there's always hope with survival - we'd had some pretty intense stuff, but then we ended up at T In The Park, where people were rejoicing and celebrating with us."
The group's Scottish connections run deeper than just gigs - they're pals with Frightened Rabbit too.
"We've just come off a long tour with Frightened Rabbit, and they were really gracious guys, so we're definitely in Camp Scotland," chuckles Billy.
WE were having drinks and they were teaching us some Scottish slang, and we were having a really good time.
"I don't want to sound like a total idiot onstage on Friday though, so I might not try and say what I know…"
Another similarity that Augustines have with the Selkirk group is that they've both worked with producer Peter Katis.
Yet Augustines also insisted on co-producing the album with him, such is their determination not to lose any creative control.
"The way we made the first record was therefore very hodge-podge, in bedrooms and that - it was a Frankenstein type of thing," adds Billy.
"We did an incredible amount of production on Ships and again on this, and at this point it would take a wizard on a magic carpet coming in to produce us on his own, because no-one knows our stuff better than we do."
Finally, it seems Billy can see a future for his band in the long-term.
"Scott (Hutchison, of Frightened Rabbit) and I had a lot of talks about how long can a band last, how can we keep this fresh and keep going on?
"We're really lucky that our fan base has come from the heart.
"We're not skinny pop guys with big hair from some flamboyant scene, but I can see down the road and it looks pretty good."
l Augustines, tonight, Arches, £12.50, 7pm