It might not permanently decapitate it but it would be absolutely devastating. If Manning leaves then we are automatically a bottom 5 or so team in the NFL. That's how poor this roster is. It would take years to be respectable again.
Our winning history is 100% tied to Manning. We have no tradition here aside from the miracle 95 run.
People know Peyton can't play forever, but they expect him to the leave the Colts when he can't play football anymore, not because he wants to play for another team. Manning playing somewhere else would be catastrophic for the franchise. Luckily for Irsay, he was handed that gift of a stadium when the popularity was at its peak.