Postseason Probability Added - An MVP Award Metric

[sbrylski]

I felt like playing around with some numbers today, and with it being a hot topic around this time of year, I decided to model my own MVP award metric. I don't necessarily think this is the hands down, best way to name an MVP, however the logic behind it is more than fair. I call it: Postseason Probability Added (PPA).

 

Here's the premise: I wanted to find the player that brought his team closer to the ultimate goal than any other. Now, what is the ultimate goal? Many say it is simply to win ballgames, and seek the player's that provided the most production in terms of win value. Others refuse to consider any players on non-playoff teams, because the playoffs are the ultimate goal. I wanted to try and combine those two sentiments and find which player increased his team's chances of making the postseason the most, by way of producing wins.

 

I believe that a player on a non-playoff team should be eligible win the MVP Award. But I also believe that all wins are not of equal value, and therefore each win provided by a player to his team is not of equal value. For a team that is 61-100, their final game is meaningless. For a team that is 89-72, if they win their final game it happens to increase their chances of making the playoffs by around 11% in the average season. Big difference.

 

I started by compiling every team's record since 1998, the last expansion year, and noted whether or not they made the playoffs. I then ran a logistic regression to find the odds of a team making the playoffs in an average season given their win total. Results:

 

Postseason Odds = e ^ (0.4591 * W - 41.1605) / (1 + e ^ (0.4591 * W - 41.1605))

 

http://i35.photobucket.com/albums/d175/sbrylski06/PostseasonOdds.jpg

 

*The blue dots are the probability of the sample statistics, when I had multiple teams at the same win total. For example, there were five teams with 87 wins since 1998, only one made the playoffs, so a mark was added at (87,0.20).

 

 

Now that I know each team's postseason odds in an average season, I can see how much a player increased his team's chances by his production expressed in wins above replacement (WAR)...

 

The metric starts simply by calculating the team's odds of making the postseason in an average year with the player on the team, by using the team's current real life win total. Then it calculates the team's odds of making the postseason in an average year if they were forced to use a replacement level player instead, by subtracting the player's WAR from the team's real life win total. The difference between those two postseason odds is a players value in terms of postseason probability, or the odds of his team reaching the ultimate goal.

 

Because the season isn't over yet, I don't know each team's final, real life win total. Instead, I used their average expected win totals from Baseball Prospectus. For the players, I calculated the metric for everyone with 250+ PA or 100+ IP. Here's the results:

 

 Name Team Wins WAR With Without PPA ---------------- --------- ---- --- ----- ------- ----- 1 Albert Pujols Cardinals 93.2 8.3 0.836 0.101 0.735 2 Chase Utley Phillies 94.3 7.7 0.894 0.197 0.697 3 Ubaldo Jimenez Rockies 91.2 5.2 0.670 0.157 0.513 4 Chris Carpenter Cardinals 93.2 5.1 0.836 0.329 0.507 5 Joel Pineiro Cardinals 93.2 5.1 0.836 0.329 0.507 6 Adam Wainwright Cardinals 93.2 4.9 0.836 0.349 0.487 7 Troy Tulowitzki Rockies 91.2 4.5 0.670 0.205 0.465 8 Chone Figgins Angels 95.0 5.7 0.921 0.459 0.461 9 Jon Lester Red Sox 95.6 6.1 0.939 0.482 0.456 10 Jason Marquis Rockies 91.2 3.8 0.670 0.262 0.408 11 Jason Hammel Rockies 91.2 3.6 0.670 0.280 0.390 12 Jorge de la Rosa Rockies 91.2 3.5 0.670 0.290 0.381 13 Kevin Youkilis Red Sox 95.6 5.1 0.939 0.596 0.343 14 Jayson Werth Phillies 94.3 4.1 0.894 0.562 0.332 15 Seth Smith Rockies 91.2 3.0 0.670 0.339 0.331 16 Josh Beckett Red Sox 95.6 4.9 0.939 0.618 0.321 17 Todd Helton Rockies 91.2 2.7 0.670 0.370 0.300 18 Dustin Pedroia Red Sox 95.6 4.7 0.939 0.639 0.300 19 Ryan Howard Phillies 94.3 3.8 0.894 0.596 0.298 20 Raul Ibanez Phillies 94.3 3.8 0.894 0.596 0.298 

 

As you might have guessed, popular candidates for the award like Joe Mauer (139th) and Zach Greinke (294th) are nowhere to be found. That's because they only added 3.2% and 0.0015% to their team's chance of making the playoffs, respectively. And from the perspective used to design this metric, that makes sense. Zach Greinke had a tremendous season - without him the Royals would have the worst record in the AL. But with him, they have the, um, well they still have the worst record in the AL. Did Greinke help the Royals win a bunch of games? Absolutely. Did it matter? Nope. If your team's chances of reaching the ultimate goal are about equal whether you signed the player or not, his signing was of zero value in that given season.

 

Same thing on the other end of the spectrum. Derek Jeter has the 12th best WAR in the league at 6.4. But even without him, the Yankees would have had a 95.7% chance of making the playoffs. Jeter only added 4.1% postseason odds. Cut Jeter and the Yankees still make the playoffs, easy. Can Jeter really be an MVP candidate if that's the case?

 

So what we are really isolating here is which player truly put his team over the top, in an average season. And that is why you see so many Cardinals, Rockies, and Phillies on that top 20 list. Those three teams rank 5th thru 7th in wins this season. Their postseason odds in an average year range between 89.4% and 67.0%. If you cut a star player off of one of these teams, they will really feel it. Those players are, by definition, the most valuable to their respective teams.

 

So by this metric, your 2009 National League MVP is Albert Pujols. If the Cardinals instead had to play a replacement level player at first base, their odds of making the postseason would drop 73.5%. Now that is what I call a valuable player. And in the American League, your MVP is Chone Figgins. Again, it is hard to argue with an increase of 46.1% in the Angels postseason odds with him on the team.

 

Lastly, if you were wondering, I give you your Least Valuable Players* of 2009:

*Subject to having 250+ PA or 100+ IP

 

 Name Team Wins WAR With Without PPA ---------------- --------- ---- ---- ----- ------- ------ Chris Davis Rangers 88.5 -1.1 0.370 0.494 -0.123 Garrett Atkins Rockies 91.2 -0.6 0.670 0.728 -0.058 Gary Matthews Jr. Angels 95.0 -2.0 0.921 0.967 -0.046 Delmon Young Twins 82.3 -1.7 0.033 0.069 -0.036 Alexi Casilla Twins 82.3 -1.5 0.033 0.064 -0.031 Alfonso Soriano Cubs 83.9 -0.8 0.066 0.093 -0.027 Garret Anderson Braves 87.2 -0.3 0.245 0.271 -0.026 Chris Duncan Cardinals 93.2 -0.3 0.836 0.854 -0.018 Emilio Bonifacio Marlins 85.5 -0.3 0.129 0.146 -0.016 Carlos Quentin White Sox 79.9 -1.1 0.011 0.018 -0.007 

 

Phew, no Brewers (not that its necessarily a good thing), but we can all laugh at the Cubs with Alphonso Soriano making the list.

 

The Brewers' worst were Looper (-.002) and Suppan (-.001). Prince was our best at .005.

[sbrylski]

One last note: This analysis also excludes players that changed teams mid-season. It can be done, but I'd need split WAR stats for those players to see how much value in postseason odds they added to each of their teams. I'm not sure the WAR splits are available though. Cliff Lee and Matt Holiday would be the two that might score well, but I highly doubt they would be near the top of the leaderboards given they spent half a season earning a PPA around zero.

 

Anyway, comments definitely welcome! I want to hear why this metric sucks! (Or how it can be bettered, of course.)

[rluzinski]

It would be kind of neat to just use a player's WPA for each game, weighted by how much that win changed their playoff odds. Say a player hit a home run in a game that increased the odds of winning by 25% and that game raised his teams chances of making the post season by 4%, He'd get .01 PPA. I assume that's the way you'd like to do it if it was easy to automate?

[logan82]

That would be interesting to see. Sounds like quite a bit of work even if you only do it for 2 players per team. WPA for each game is readily available.

[sbrylski]

It would be kind of neat to just use a player's WPA for each game, weighted by how much that win changed their playoff odds. Say a player hit a home run in a game that increased the odds of winning by 25% and that game raised his teams chances of making the post season by 4%, He'd get .01 PPA. I assume that's the way you'd like to do it if it was easy to automate?

You would suggest using real-time postseason odds as well then?

 

What if a player adds .25 WPA, but the team loses and drops 4%?

 

What we'd almost need is a kind of LI designed for the importance of each game, rather than each AB. For example, a team enters a game and if they win they get +4% PPA, and if they lose they are expected to get -6%. So the "LI" for the game is the spread of 10%. If a player nets a .25 WPA in the game, he gets .25 times the spread of 10%, for .25 PPA, regardless if they win or lose.

[rluzinski]

You would suggest using real-time postseason odds as well then?

 

Yes. That's what i was trying to suggest, although I see I didn't make that clear.

 

What if a player adds .25 WPA, but the team loses and drops 4%?

 

Like WPA, he would still get credited with whatever percent gain a win would have raised his team's playoff odds.

What we'd almost need is a kind of LI designed for the importance of each game, rather than each AB.

 

I am suggesting weighing WPA by the change in playoff odds per win or loss. I thought coolstandings.com used to give that, although I couldn't find it with a quick glance. In addition to +WPA, You'd subtract any negative WPA ABs as well, proportional to how much a loss drop's the team's odds.

For example, a team enters a game and if they win they get +4% PPA, and if they lose they are expected to get -6%. So the "LI" for the game is the spread of 10%. If a player nets a .25 WPA in the game, he gets .25 times the spread of 10%, for .25 PPA, regardless if they win or lose.

 

The concept of LI is supposed to adjust for opportunity, so I don't think that's exactly analogous (definitely not an expert on the subject, however). Regardless, the MVP is the exact opposite of caring about adjusting for opportunity. The best batter in the league has no chance to win the MVP on a 60 win team, so clearly it's very dependent on things outside of the control of the batter.

 

If coolstandings does provide the delta playoff odds, fangraphs provides WPA by PA. It would just be matter of combining the two.

[rluzinski]

OK, I got that mixed up. WPA/LI neutralizes context. What I'm proposing is for each game, find how much a win or loss drops a teams playoff odds (call it PPA and -PPA). Like I said, I thought coolstandingss.com had those. Then just use this:

 

MVP= (WPA x PPA) - (-WPA x -PPA)

 

Would that work?

[pitchleague]

I know what your numbers say and I completely understand it.

 

However, it still boggles my mind that Prince Fielder would only change the Brewers playoff odds 0.5 percent. I understand that you did this research (good job, by the way) based on the standings right now, so with the Brewers "out of it", Prince doesn't make much difference.

 

You kind of need to do this same thing a couple of times during the season (Opening Day [based on what? I don't know], Memorial Day, All-Star Break, and maybe August 31st) to see how it changes throughout the season. Because, again, in our minds (those of us that watched the games) we all know that Prince Fielder was pretty darn important to the team in 2009 but you metric seems to show that he was pretty replaceable. And this is only shown because the metric was done with the current standings.

 

Obviously, all the teams that are in your "most valuable" top 10 are going to be playoff contenders.

 

It's the only "flaw (if you even want to call it that) that I can see. But that's just what I would look for, probably not others.

[TooLiveBrew]

Because, again, in our minds (those of us that watched the games) we all know that Prince Fielder was pretty darn important to the team in 2009 but you metric seems to show that he was pretty replaceable. And this is only shown because the metric was done with the current standings.

 

It's not that he's replaceable, it's merely the fact that his team is out of the race. He's 'replaceable' if your team is going to miss the playoffs anyway.

 

Imo Sam's approach here is one that accepts that many (or even most) MVP voters are going to demand their candidate be on a contending team. Like he said in his OP, this isn't necessarily the best way to figure out who's most valuable... but it's a good way to find out whose production was most valuable on the better teams in the league.

[pitchleague]

yep, I completely understand the logic behind the metric. Completely.

 

I'm just don't feel that just because a metric says that X player has Y value as most valuable to his team at the end of the season is an end all be all metric for most valuable to a team at any point in the season. Sure, the metric has complete value to "voters" that determine that an MVP candidate must be a playoff contender. What about the "voter" who doesn't think this way. Robin Yount won his 2nd MVP on a non-contending team. So did Andre Dawson. According to the metric, who should have won instead of those guys? I'm guessing Ruben Sierra in Yount's case (don't remember off the top of my head in Dawson's).

 

I think the metric has good, but not great, value to a "voter."

 

In the Brewers case, Prince was pretty darn valuable than 0.5% to the playoff odds. In my opinion, without a Prince Fielder on this year's team, the Brewers would be lucky to win 70 games. That even includes having a Ryan Braun on the team and a "Lyle Overbay" or "Sean Casey" type of player as the replacement first baseman.

 

I understand that Prince's "Most Valuable-ness" at the end of the season only adds 0.5% to the playoff odds. I just don't believe that that number is a constant throughout the season. And that's what I think the metric needs to improve on.

 

I'd like to say, "Well, duh, no kidding that Prince adds only 0.5% to the playoff odds at the end of the season but Pujols adds almost 75%." And I would say this because my 5th grade students that are into baseball could figure that out here at the end of the season and they don't need a metric to tell them that. But give them a metric that charts and tracks this "most valuable" metric over the course of the season and they will begin to understand how, over the course of an entire season, that Pujols is more valuable than a Fielder.

 

And metric or no metric, Prince would most definitely have a much higher share of the actual MVP vote if his teammates played up to expections (or, in the case of those that did play up to them, better than expectations). Prince played at an MVP caliber-level the entire season. There were hardly any cold streaks (the only one that really comes to mind was when he was "stuck" at 126 RBI a few weeks ago.)

Pujols also had just a handful of cold streaks. But Pujols' teammates (particularly the pitchers--again) played at or above their expectations.

 

Anyways, I'm not selling the metric short. I think it is a good tool . . . . for the end of the season. Just not all year. Which is really what the voters are supposed to be looking at in the first place.

 

(yeah, yeah, I know. Several writers place special emphasis on the month of September for their MVP vote.)

[TooLiveBrew]

yep, I completely understand the logic behind the metric. Completely.

 

Sorry http://forum.brewerfan.net/images/smilies/embarassed.gif

[rluzinski]

Since there's no actual definition of what an MVP is, there can't be a metric designed to satisfy everyone. Like TooLiveBrew pointed out, I think sam's metric does a reasonable job to try and quantify the most

[pitchleague]

yep, I know Russ.

 

Sam asked for feedback of the metric, quoting: "Anyway, comments definitely welcome! I want to hear why this metric sucks! (Or how it can be bettered, of course.)"

 

So I gave my comments. I am absolutely not saying that I am correct. I'm just giving my 2 pennies worth of feedback and stating how I think it could be bettered.

 

Frankly, I thought I would be ripped to shreds in this thread for stating my opinions and not stating facts and statistics. It's probably the number one reason that I rarely post in this sub-forum.

[sbrylski]

Russ, I can't find PPA+ and PPA- anywhere.

 

----

 

Pitch, I know you go on to make other valid criticisms (with a nice concise response to those thoughts by Russ), but I'll focus on this comment because its directly concerning the construction of the metric, rather than the use or its validity.

 

I understand that Prince's "Most Valuable-ness" at the end of the season only adds 0.5% to the playoff odds. I just don't believe that that number is a constant throughout the season. And that's what I think the metric needs to improve on.

 

No, it wouldn't be a constant value throughout the season. But isn't the final tally the one that's most important? Do MVP voters care about who was projected to make the playoffs in mid-June? Neither does the system for deciding who moves on to the postseason - all that matters is the final standings.

 

In the Brewers case, Prince was pretty darn valuable than 0.5% to the playoff odds. In my opinion, without a Prince Fielder on this year's team, the Brewers would be lucky to win 70 games. That even includes having a Ryan Braun on the team and a "Lyle Overbay" or "Sean Casey" type of player as the replacement first baseman.

 

Yeah, but we can't assume we would have a Lyle Overbay ready to go. If Prince would have broken his leg in May, we probably would have had Brad Nelson or Adam Heether up with the club and getting starts.

 

And really, Prince added 0% to the playoff odds, because we didn't make the playoffs with him. There's two methods to get a meaningful number: what I tried above, using the average number of wins it takes to make the playoffs, and Russ's idea to use a real time aggregate, which in the end will probably be pretty close given that most seasons will be about an average season.

[sbrylski]

A note to self about the original metric - if I'm trying to model the average season, I should probably use a team's third order wins or something, rather than their actual number of wins, to remove team luck.

[sbrylski]

An interesting effect of Russ' method - the sum of the individual's PPA's on a playoff team would equal one, while the sum of the individual's PPA's on a non-playoff team would equal zero. Kind of neat.

[rluzinski]

An interesting effect of Russ' method - the sum of the individual's PPA's on a playoff team would equal one, while the sum of the individual's PPA's on a non-playoff team would equal zero. Kind of neat.

 

Actually, we have to take into effect the odds of the team making the playoffs before the season started, right? If we assume every team has the same odds at the beginning of the season, an NL team has 4/16= 25% of making the playoffs. A team that made the playoffs collectively added 75% to their playoff odds over the course of 162 games.