Of course Jesse McCartney knew posting a photo of himself with his head resting against a pregnant woman's belly would get people talking.

"We were playing into that quite a bit," the singer exclusively told E! News about getting fans hyped for "Make a Baby," the second single off of his upcoming EP All's Well, featuring Yung Gravy. "We knew it would elicit the reaction that it did, cause some commotion on the internet."

The pair really leaned into the daddy imagery to promote the labor-of-love track, cradling dolls and toasting with baby bottles for the 'gram. Which begged the question: Does McCartney, who's been married to Katie Peterson since October 2021, have a case of baby fever?

"That's the path we're traveling down at this moment," the 36-year-old acknowledged. "To be clear, we're not pregnant. But it's something we've discussed."

Yet while parenthood is part of their "broad plan," he admittedly has the most understandable of fears about becoming a father.

"The general idea of having to keep something alive is very scary to me," McCartney explained. "We've done a pretty good job with our dog"—their pup Bailey is thriving—"but it pales in comparison to the responsibility of taking care of a human child."

At the same time, the Dream Street alum can envision all the good stuff that comes with having a child, too.

"I look at a lot of my close friends now that have kids," he said, "and there's so many special moments that they have. Eventually, that's something I definitely want to have."

And McCartney, whose last album was 2021's New Stage, is grateful that he found the right partner to make a baby with in the first place. Left to his own devices, "I'd be raising a cave baby," he quipped, "so it's a good thing that she's the one that will be raising the child with me."

But the former teen sensation, whose signature hit "Beautiful Soul" came out almost 20 years ago, may be selling himself short in the grown-up department.

Thinking about his own relatively undramatic experience coming of age in the spotlight, "I'm lucky I didn't really have any major public meltdowns or crazy, stressful moments, at least in front of the camera," McCartney noted. "At least not yet. I've been able to maintain a relatively sane and normal life considering all that has gone on, particularly at that age."

He attributed his less dramatic arc to his "great family and friends," he continued, "and just a solid foundation of people that I had around me at that time. Of course there's things I went through that I think any normal teenager would go through, just in a public way, but I don't have any regrets."

Missing school dances or breaking out before an appearance on The Tonight Show may have felt like the end of the world at the time, but McCartney looks back on even the pitfalls of adolescence with a certain fondness.

"I wouldn't trade any of it," he said. "And it was well worth it, considering this is many people's dreams as a kid and was definitely mine. So, my dreams came true. It's one of those crazy clichés, but they really did."

And though there was a period of time where McCartney got a little tired of "Beautiful Soul," from his 2004 album of the same name, "it's more special now than ever before," he said. It's "obviously in every set list. And if I get bored with singing it, I let the audience sing it because they know every word."

Which, as he gears up to head back out on tour for All's Well, feels pretty amazing.

"It seems to be a song that has crossed over into a new generation," McCartney reflected. "I have a very strong sense of gratitude for all that I've been given, and for the fans who have followed me for so many years and continue to do so. They're still showing up, which is kind of incredible to me. And so long as they keep showing up, I'm going to show up."

Speaking of which, it's not out of the question that concertgoers might get a glimpse of McCartney's former Dream Street mates at some point. While he didn't join Matt Ballinger, Frankie Galasso and Greg Raposo when they got the band back together last year (fellow original member Chris Trousdale died in 2020), McCartney is hoping they can figure something out for the fans at one of his shows.

"I've kept in touch with all of them," McCartney shared, noting that they went on a ski trip to Big Sky, Mont., with their significant others in January 2023.

"It was amazing," he said. "We rented a big house and just hung out and talked for hours by the fire and drank wine and went out to dinner for three, four nights. It was just crazy to see how much all of us have grown and what we've been up to in our lives, because it literally had been over a decade since we had last seen each other. So now we have a group text chat and we talk every week or so."

It's the "classic friend chat," McCartney said of the memes and jokes that make up the meat of the conversation. "I think they're going to come to one of my shows on this tour, maybe the New York show. We don't have any plans to do any reunion of any kind, at least not yet. But you know, never say never."

And in the meantime, everyone can dream.

—Reporting by Dayn Nanda

Sign up for E! Insider! Unlock exclusive content, custom alerts & more!