Yes that's usually the case when someone loses the SB they are the biggest loser and since its Manning even more so.
Sucks but that's how it is. Is it fair? No but nothing really is.
I'm sure over time it will be looked back as rather impressive he made it back to the SB when two years ago nobody thought this was even possible.
However not right now.