Why your Favorite Character is always Killed Off

Okay, just to warn you, I am a movie nerd. I watch movies, critique movies, look into the behind-the-scenes of the movies, analyze the movies, etc. And there is one theme that I find is often repeated (and when it's not, it's either because (A) it's a children's film (B) I don't like the character everyone else does) is that the favorite character DIES.

Example: Horatio Hornblower, episode 1.
For those of you who don't know (and I don't blame you if you don't), the Horatio Hornblower series centers around a young ship's officer named: you can guess. In the first episode, he is assigned to a ship that has been mulling in a port city for over a year. Along with him arrives a surly, tyrannical officer vying for the same promotion as Horatio. They fight, Horatio silently suffers, they fight some more, Horatio finally snaps and the antagonist challenges him to a fatal duel. As Horatio is below decks preparing himself, a fellow officer (who earlier saved Horatio from being beaten to death by the villain (it was awesome)) comes to give him some final advice. At the moment Horatio's back is turned, the officer knocks him unconscious and goes to the duel in his place.
And guess what?
He DIED.

At first I was stuck glaring at the screen and letting out words of agony, but after that I started to wonder: why do the favorite characters always die?

The answer might actually not be as heartless as we thought.
And the summary of this answer is: Phil Coulson

(Sorry if you haven't seen the Avengers, but this post centers upon a spoiler for that film so....
yeah, sorry)

Anyway, Phil Coulson could be said to be one of the most interesting humans in Marvel Cinematic History. He talks passive-aggressively to Tony Stark and yet helps with his science project (notably by picking up a piece of equipment that looks a lot like Cap's shield), he threatens Russian terrorists, chills with Natasha on the phone, he gets pumped up over Captain America, and he negotiates with Thor and Loki without breaking a sweat. He shows both the human side of humanity, having interests and enthusiasm, and the side we would like to have, being fearless and cool.
Then they killed him.
Now, that hurt. And of course Nick Fury only rubs it in by tossing his blood-stained, unsigned Captain America trading cards on the table in front of Steve. It's that final loss that gets the Avengers going, that get them organized and united, to stop the alien invasion and avenge Phil Coulson by beating the snot out of Loki.
But then Maria Hill points out that the trading cards were in Coulson's locker. And Nick Fury's response?
"They needed a push."

That's why all our favorite characters die. Because there is something about them that inspires us and the hero, some element they add to the effort, something that we don't fully recognize until they're gone. And when they're gone, we want them back. We want them back so bad, but we know there is no way they'll come back (except in Marvel films (unless you're Pietro)).
So what do we do?
We/the hero try to preserve their memory and honor them by being what they were, by actually becoming what we wanted to be but they were. And sometimes we don't notice until they're gone. Sometimes it takes tragedy to make us see what we're capable of.
Sometimes we need that push.