WAR - What is it good for?

[thebruce44]

I have seen a lot of people question some of the more advanced metrics on here, so I thought this was an interesting blog post on Fangraphs. Basically, Dave Cameron admits that WAR has its short comings (base running and catcher defense) but shows that it does a very good job of predicting wins and losses. It is not perfect by any means, but is probably the best metric we have right now to accurately predict how a trade or free agent addition will help or hurt the team this offseason.

(Mods, please feel free to move this to the statistical analysis forum if need be but I thought it might pertain to some off season discussions such as Hoffman's signing or any Prince trade).

WAR: It Works

by Dave Cameron - October 7, 2009

We use Wins Above Replacement around here a lot, as one of the focuses of the site is to accurately quantify the value each player produces, and WAR is the best tool we have to do that. However, it faces a decent amount of skepticism from people who don't trust various components for a variety of reasons - they don't like the numbers that UZR spits out for defense, they don't believe in replacement level, or they believe that pitchers do have control over their BABIP rates.

So, the question is, does WAR work? If it's designed well, there should be a pretty strong correlation between a team's total WAR and their actual record. Fans of WAR rejoice - there is.

For 2009, the correlation between a team's projected record based on their WAR total and their actual record was .83. This is a robust number, especially considering that WAR is almost completely context independent and currently includes some notable omissions - base running (besides SB/CS, which are included in wOBA) and catcher defense are both ignored in the calculations. We also don't have an adjustment for differences in leagues, so we're not accounting for the fact that the AL is better than the NL.

Despite these imperfections, WAR still performs extremely well. One standard deviation of the difference between WAR and actual record is 6.4 wins, and every single team is within two standard deviations. Only four teams were more than 10 wins away from their projected total by WAR, with Tampa Bay ending up the furthest away from our expectation (96.6 projected wins, 84 actual wins), and 18 of the 30 teams were within six wins of their projected WAR total.

For comparison, the correlation between pythagorean expected record and actual record is .91, and pythag includes some aspects of context (performance with men on base, for instance) that impact runs scored and allowed, so we would expect it to predict actual record somewhat better than a context independent metric like WAR. The fact that WAR is even close to somewhat-context-included pythag is impressive in its own right.

WAR isn't perfect. But given the known limitations and the variations in how contextual situations impact final record, it does an awfully impressive job of projecting wins and losses.

[orca1963]

I am not much of a stats person, but I will say that your title is the best I have seen. Well played. Now if I can only get the song out of my head. http://forum.brewerfan.net/images/smilies/smile.gif

[Patrick425]

I'm not much of a stat person either...at least not as much as many of the posters here. However, I'll never understand these stats that the "experts" call predictive. When you take a stat or a set of stats from a certain year and it somehow correlates to the team's wins in that same year, that's not predictive, that's reflective.

[joepepsi]

Great title thread! So let me get this straight: every team is within 2 standard deviations where one standard deviation is 6.4 wins? Doesn't seem all that remarkable to me....

[Baldkin]

I'm not much of a stat person either...at least not as much as many of the posters here. However, I'll never understand these stats that the "experts" call predictive. When you take a stat or a set of stats from a certain year and it somehow correlates to the team's wins in that same year, that's not predictive, that's reflective.

If you sum the WAR for a teams players, which is determined independently of their record (without factoring in wins and losses at all) it predicts, fairly well, their record for the season. Its not as though they took the teams wins, then divided them up amongst the players.

 

So, going forward, if you *can* use WAR and WAR predictions to correlate to a teams record the next year.

[MJLiverock]

Great title thread! So let me get this straight: every team is within 2 standard deviations where one standard deviation is 6.4 wins? Doesn't seem all that remarkable to me....

That kind of stood out to me as well, six games is meaningful and 12.8 to be within 2 standard deviations isn't that impressive. Youcould say any team with a vowel in in its name will have 81 wins and be within 12.8 games on either side and hit 75% of the teams. The fact that it can identify a team with some better players like the Yankees vs. the Nationals gives it a little better predictive nature but I still wouldn't buy into player X adding 2.5 wins over player Y to predict a club's record. It has it's place comparing the past season's of players versus one another but still has plenty of questionable inputs.

[Patrick425]

I'm not much of a stat person either...at least not as much as many of the posters here. However, I'll never understand these stats that the "experts" call predictive. When you take a stat or a set of stats from a certain year and it somehow correlates to the team's wins in that same year, that's not predictive, that's reflective.

If you sum the WAR for a teams players, which is determined independently of their record (without factoring in wins and losses at all) it predicts, fairly well, their record for the season. Its not as though they took the teams wins, then divided them up amongst the players.

 

So, going forward, if you *can* use WAR and WAR predictions to correlate to a teams record the next year.

I think if you put a teams stats in front of me at the end of the year (excluding pitcher's wins), I could give you a pretty good idea of how that team did without doing a whole lot of calculations. I don't see why this is so amazing. Sure, if I knew each players 2010 WAR before the season started, it might be useful, but I don't. Regardless of the systems that claim to be good at player projections, most of them are really not that accurate from what I can tell, and even if they are, they can't predict injuries, trades, season long slumps (Zito, Andrew Jones, etc), or other in season roster changes. It's about as predictive as the stock market.

[joepepsi]

I think if you put a teams stats in front of me at the end of the year (excluding pitcher's wins), I could give you a pretty good idea of how that team did without doing a whole lot of calculations. I don't see why this is so amazing.

I bet you'd be within 12.8 wins too.

[thebruce44]

Thanks for the praise on the thread title. I know I could go for a little Edwin Starr right now.

 

I don't pretend to know what Dave Cameron is talking about all the time, but if you follow the link there is a lot of good discussion going on in the comments on the blog post. I would start there. Basically, it sounds as though any metric that produces a correlation around .8 is pretty good. I think the reason it is not more accurate is that it doesn't take situational hitting and other aspects into account. Really I should stop talking here and wait for someone with more knowledge to chime in.

 

The one thing I do really like WAR for is to compare players. For example, a Prince trade doesn't seem so bad on the offense if you think he will produce around 5 WAR next year and a stopgap 1b of 4 WAR can be signed to a short contract. Then if you can bring back 3 or 4 WAR through pitching it becomes a now brainer. I am just throwing these numbers out there as an example, but I hope it makes some sense.

[trwi7]

Absolutely nothing.

 

High five anyone?

[Patrick425]

Of course, one wonders if "War and Peace" would have been as succesful under this alternate original Title?

[Ennder]

Hitting WAR seems to be pretty good, pitching WAR is pretty flawed though. Would be interesting if they broke the two aspects apart and looked at the correlation of each separately somehow.

[TooLiveBrew]

Absolutely nothing.

 

High five anyone?

High five for sure. I'll just be another one to add that I think this is one of the best thread titles ever.

[owbc]

It is true that all of us could look at a basic stat sheet and predict wins/losses for next year. In fact, I could just take every team's W/L record from 2009 and add 0 or 5 wins to it and come fairly close for 2010. If every GM can do that also, then there is no way to gain a competitive advantage if everyone knows the value of basic stats. However, if I am looking for free agents to sign and I only have a small budget, it might be useful to find a stat like WAR to combine with more traditional player evaluation in order to gain a competitive advantage. Maybe WAR is not the answer, but it is important to continue developing/researching statistical methods.

[endaround]

People are missing what this was. Its not tuse WAR to predict records, we know how to do that for a team with better correlations. Its to show that WAR is constructed in such a way that even with it not taking into account lots of thins you'd like if you were predicting how many games a team won with base stats it comes close meaning it does a decent job of representing the talent level present.