David Duchovny Talks 'Failure of Friendship' with Gillian Anderson During “The X-Files”: 'Could've Handled Myself Better' (Exclusive)

On Duchovny's 'Fail Better' podcast, the costars reunited and revealed that there were times when they wouldn't speak off-camera for weeks

The X-Files costars David Duchovny and Gillian Anderson are looking back at their long friendship and working relationship — and the missteps they’ve made over the years.

Anderson joins Duchovny for an intimate conversation on the upcoming Nov. 12 episode of his Lemonada Media podcast Fail Better. The actress and author, 56, whose book Want was released in September, has been a fan of Duchovny's show for a while now. In July, she posted a video message for Duchovny on her social media channels, praising Fail Better and describing his interviews with people like Sean Penn and Gabor Maté as “intimate and vulnerable and very smart.”

“When I first started listening and had reached out to you, I wasn’t thinking about it necessarily in terms of me or talking to you about the book,” Anderson told Duchovny on the new episode. “It was just more of just really enjoying it and listening to the depth of your conversations that you were getting into with people and appreciating that I felt like I was learning more about you than I knew, or than I ever knew.”

Related: David Duchovny Recalls the 'Immediate Connection' with Gillian Anderson Before Being Cast on The X-Files (Exclusive)

“We know each other very deeply and yet we don’t know each other either in some weird way,” Duchovny, 64, said.

Over the past 30-plus years, in which the duo have starred together in a total of 11 seasons of The X-Files as well as two spin-off films, Duchovny said, “I don’t know that we ever sat down and said, ‘Hey, what was your childhood like?’ ”

“We didn’t. And why would we? We were busy,” Anderson agreed. “We have a closeness that we don’t have with probably many other people, and went through something that we didn’t go through with any other people. I mean, yes, there were crew and etcetera, but in terms of our experience as actors. And so, I thought it would be a curious investigation.”

Duchovny admitted that he thought the “trickiest” part of their conversation would be addressing what he called his “failure of friendship.”

The actor and podcast host told PEOPLE in June that he and Anderson felt a connection as early as their audition process for The X-Files.

“I think immediately, when we were reading together on the stairs to go in to audition, we knew that we could work together,” he said. “There was an immediate connection through the work that we could do, and that lasted for a long, long time.”

Related: X-Files' Gillian Anderson Reminisces About Her Time in the FBI with Taylor Swift Meme

But as he noted to Anderson on Fail Better, at some point during the show’s original nine-season run from 1993 to 2002, the costars drifted apart.

“There was a long time, working on the show, where we were just not even dealing with one another off-camera,” Duchovny recalled. “And there was a lot of tension. Which didn’t matter, apparently, for the work cause we’re both f------ crazy, I guess. We could just go out there and do what we needed to do.”

“That is kinda crazy,” Anderson agreed. “I mean, it’s crazy that we were able to present on camera, you know, the various feelings and emotions and attraction and all that kind of stuff, but then not speak to each other for weeks at a time.”

Duchovny suggested that it may have been a smart choice. “Cause we’re, like, savin’ it up. I don’t know,” he said. “But I could've handled myself better, you know? And as you know, we went through a crazy-making kind of a process with this thing. We went from — I mean, I was pretty inexperienced. You were really inexperienced. And all of a sudden … It was like a global phenomenon before the Internet. And we’re just scurrying, trying to figure out who we are.”

Noting how young both actors were at the time, Duchovny said the experience of the show’s success turned their world upside-down completely.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories. 

Despite those tensions, the two have remained friends over the past three decades. A photo Anderson posted of the two of them with her dog in 2021 went viral, and earlier this year, Duchovny told PEOPLE the two are like “family.”

“When you share a seminal kind of experience in your life — this huge success with the show that we had — only we know what it's like to be in the center of that,” he said. “It's almost like being in the same family. We know what it was like to be raised in that time. There's a certain kind of shared experience, knowledge, and past that never goes away.”