The Mirror reports that an insider has claimed the father and son's relationship is "near rock bottom" and Harry will decline Charles' invitation ahead of the release of his memoir on January 10.
Harry's bombshell memoir, which is titled Spare, references Prince William and the Duke's role as 'the heir and the spare', where the Prince of Wales is the heir to the throne and his bother is his spare.
A source told the publication that the royals were "completely caught by surprise" when the title was announced.
"This book could spell the end for any relationship Harry wants to have with his family, it’s desperately sad," another insider shared.
"Everyone is bracing themselves for yet another fraught and frantic period at the mercy of the Sussexes."
It comes after a royal insider shared the real reason behind why Harry decided to write the "truth bomb" memoir that promises "raw, unflinching honesty".
The royals are bracing for more of Harry's "truth bombs" in the book and worry that Harry will go "fully rogue", which would spark "drastic" retaliation from King Charles.
Now, sources have told UK's Sunday Telegraph, Harry wrote the book because he can't spend the rest of his life trying to please the royal family.
"You can't kind of always live your life trying to make your family or your siblings happy," one insider said. "You have to choose your own happiness."
They added that the book will "take aim" at members of the royal family, including his father, with Harry addressing "genetic pain and suffering".
It comes following claims by Tom Bower that Charles may strip Harry and Meghan of their titles, with the book being "make or break" for the family.
"I imagine that King Charles has been warned that any retaliation won't be pretty.
"All that King Charles can really do is to withhold the titles for his grandchildren Archie and Lilibet. Ultimately, I suppose, he could take away Harry and Meghan's titles as well, but that's pretty drastic."
It's believed Harry has barely spoken to his brother Prince William since news of the memoir first broke, with the belief their rift will only be worsened by the book.
Never miss a thing. Sign up to Yahoo Lifestyle’s daily newsletter.
Or if you have a story idea, email us at email@example.com.