Stats Are Evil

[FVBrewerFan]

My oversimpistic view is that stats are more important in baseball than many people want to admit. But with that said, there are certainly intangibles that are difficult or impossible to measure,

 

The Brewers, as an example, have that intangible struggle to breal .500. Of course stats can back up why they are not an above .500 club. But you will never convince me there isn't something psychological involved with the struggle.

 

When you have a player perfectly capable of laying down a bunt, but doesn't in a critical situation stats can't explain it. When you have a guy fall asleep and get picked off base. Needing that one more hit in a close game to get the win. Some teams make those plays, others don't.

[H0FFY]

This is a weird thread. I never saw so many people with differing opinions have such an intelligent, well thought out, gentle (but inspiring) discussion.

 

For that reason, I will refrain from posting in this thread.

 

But is sure is a good read.

[gregmag]

I knew Hardy would hit dispite the fact that he was hitting .180 for a long time. there wasnt a statistic to be found that would indicate that Hardy would hit and hit a lot. Low and behold, post all star break OPS .878. Find me a stat that would have predicted that.

 

How about his minor league stats from last year or the year before?

 

BP's PECOTA system, which is just a sophisticated predictor based on past performace, projected before the season that Hardy would hit .268 / .328 / .437 this year. Stats are exactly why a lot of us knew Hardy would hit.

 

I'm not trying to dump on your post, scoop; I think you make some good points. But I do think this is an example of how people sometimes assume things are unquantifiable, when they really are. We're talking about baseball. It's wonderfully complex, but it isn't love, death, or poetry. An amazing amount of what goes on in a baseball game is quantifiable.

 

There are indeed some things that can't be reduced to numbers. But there are a lot of things, like defense, that I'm quite sure can be reduced to numbers, but we haven't quite figured out how to do it yet.

 

Greg.

[bjamesw22]

BP's PECOTA system, which is just a sophisticated predictor based on past performace, projected before the season that Hardy would hit .268 / .328 / .437 this year.

 

But those numbers were not accurate. I don't see what they predicted. It is an estimated number...a best guess...not what will actually happen.

[gregmag]

Right, but they predicted he would be a solid major league hitter this year. In June, a lot of people -- including Jim Callis, IIRC -- were saying he couldn't hit. I'm not saying BP's projections were spot on; I'm saying I saw exactly what scoop saw, but not because of a hunch -- I saw it because I had numbers that told me Hardy would hit.

 

Greg.

[bjamesw22]

Right, but they predicted he would be a solid major league hitter this year.

 

But if you look at his overall stats, those numbers do not show a hitter who is a solid major league hitter this year. A .684 OPS is much different than the .765 that was predicted.

[whizkid]

Can you point us to the study that analyzes how accurate PECOTA projections have been for all rookies since BP began these projections?

[1992casey]

Part of the beauty of the Baseball Prospectus system is its objectivity. But when the book comes out, there's also a personal touch. In the player commentary, a writer will occasionally say that he feels that the PECOTA system has been too kind to or too harsh on the player.

[DougJones43]

i have a question. i realize stats are very important towards determining what kind of success a player may or may not have. the biggest limitation in my eyes, however, is the fact that stats can really only be fairly applied to players who have put in some time at the major league level, or at least the high minors.

 

i'm not sure if this has been discussed before, but where does that leave stats in regards to drafting players, particularly out of high school? i know that stats are recorded for high school and college players, but a lot of those stats aren't always going to be relevant for a number of reasons (huge difference in competition level from player to player, putting up good numbers in HS or even college doesn't necessarily mean a guy has what it takes to be a good major league player). getting quality players throughout your farm system is crucial to future success. you can't really base your drafts off of ops, oxs, or whip when evaluating prep players, so it seems as though that only leaves room for the leathery-faced, skoal-spitting scouts with their stopwatches, radar guns, and gut-feelings.

[EdgarDiazRocks]

Alright. I have been letting this one simmer and stew for a bit before I jumped in. Since this is an amicable discussion (surprise, surprise), I am going to kick in my #1 pet peeve with stats and SABRites.

 

The everybody fits into the same hole mentality.

 

It seems to me that by using tools like VORP, WS, WARP and even OXS by position you are measuring every single player against the same type and not allowing them to be individuals with different games. And of course by every single player we mean only hitters, which is another problem I have.

 

Ichiro doesn?t walk, he doesn?t get on base enough. Branyan strikes out too much. Bill Hall isn?t patient enough and a bust but Nick Swisher is going to be a ?great player?. Compared to who I ask? The mythical ?replacement player?? The ?league average? for their position? The least paid player that produces ?equivalent? numbers? Poppycock.

 

Players have different games. They do different things. Boiling every thing they do down to one number dehumanizes everything about them, yet it is where stats boys go every single time to prove their point.

 

Why therefore is not pitching the same?

 

Jamie Moyer?s Baseball Reference page lists the player most similar (before 2005) to him this way . . .

 

1 Frank Viola (921)

2 Mike Flanagan (912)

3 John Burkett (909)

4 David Wells (903)

5 Kenny Rogers (902)

 

Now nobody would ever get any of those pitchers confused with Moyer having ?watched? them, yet statistically they are ?similar?, yet we take the differences in skills into consideration when we interpret the numbers. Why aren?t hitters judged the same way?

 

Bill Hall?s (before 2005) peers?

 

1 Omar Infante (964)

2 Brent Butler (959)

3 Jay Canizaro (957)

4 Marco Scutaro (956)

5 Buddy Blattner (956)

 

Therefore he must suck, because he?s just like Omar Infante. And this year, he?s above ?league average? for a shortstop, but as a 3B? Ohh, he still sucks.

 

It always kinda bugs me when hitters numbers are simply tallied up into one easy to digest ?end all? figure (take your pick) and similar players are simply lumped together in ways that no one would ever do with a pitcher. Tell me you don?t look at minor league numbers differently. That when you see a hitter dominating a league, you may look up his age and simply be happy with that, but with a pitcher you have to know his age, what his velocity is, what pitches and movement he has along with control and ?projectability.? We accept that pitching is a lot harder to define, yet we fervently believe that hitting is simply everything that ends up in the boxscore. Why?

 

24 year-old JDLR could be a stud, because we have seen him pitch. His numbers don?t back it up, but knowing his velocity and the fact that he could ?find it? gives us a sense of hope. And for Christ?s sake, he?s left-handed!

 

Yet with a 25 year-old Bill Hall, ?He is what he is.? Numbers don?t lie. Despite the fact that he continually improves at the plate.

 

Why the double standard? Simply because as of now Hall?s numbers are considerably lower than what we think they should be? That they compare unfavorably to great players or a made up one that changes at every position he plays at?

 

But in the end, what I?m trying to get at here is Stats, for me anyway, are tools to greater appreciate what sets a player apart from his peers, what makes their individuality greater rather than what players they are ?better? , ?worse? or "equal" to.

 

I could get lost for days in Tony Gwynn?s Retrosheet page. Diving into the splits and pitcher vs batter numbers, amazed at what he did. I don?t care that Ted Williams was worth more ?Win Shares? nor am I going to spend any time trying to figure out who was more valuable, Gwynn, Rod Carew or Wade Boggs. They are totally different players with different skillsets, and that?s why I love baseball and the religion of stats. Because I can remember each of those players and recall how different they were and then go look up the numbers and find the intricacies in those differences.

[rluzinski]

Anyone who ranks all players by their OPS and declares that anyone who's above anyone else is better; well that guy just doesn't get it. That said, everything a player does in offense is aimed at one goal; scoring runs for the team. For that very reason I don't feel it's misguided to ask the question, "How many runs IS that player worth?"

 

The validity of isolating one player from a team can be questioned, for sure. The environment a player is placed in has a huge effect on that player's ability to score runs. If we place all players in a "league average" environment, I think it's fair to make some relative comparisons between players. I also think it's fair to estimate the average run production of any baseball event (HR, double, steal) and sum them up for each player.

 

As long as people remember they are ESTIMATIONS based on average situations I think the numbers are very useful for comparative examination.

[gregmag]

But if you look at his overall stats, those numbers do not show a hitter who is a solid major league hitter this year. A .684 OPS is much different than the .765 that was predicted.

 

The thesis here -- the one scoop was talking about and the one I'm talking about -- is that Hardy's first half was not the "real" J.J. Hardy, because he got hurt last year and wasn't ready to come back, and that once Hardy got over that initial freakout, the "real" J.J. Hardy emerged.

 

That's a thesis. I can't prove it conclusively; maybe he'll hit .230 again next year. But the circumstances provide a solid basis for the thesis. BP's numbers -- which, by the way, don't consider the injury-recovery angle, just his raw performance in the minors when he was healthy -- show a player that looks an awful lot like J.J.'s line would look if his first half had been merely poor, rather than world historically bad.

 

This is part of the problem with arguments about stats, especially predictive stats. It's not a crystal ball. Statistics vary for a variety of systematic and random reasons. Too many people look at that problem and say it proves stats aren't worth anything in a given situation, like this one. That's dead wrong, IMHO; it just means we have to be critical and sophisticated in how we read and crunch the numbers.

 

That's all I got. For me, it's self-evident: BP's numbers (which is to say, J.J.'s minor league performance) made me confident he would hit. He is now hitting.

 

Can you point us to the study that analyzes how accurate PECOTA projections have been for all rookies since BP began these projections?

 

I can't. I don't know whether such a study exists. I do recall that Bill James, when he first developed major league equivalencies back in the late 1980s / early 1990s, claimed that MLEs had totally failed to predict a minor league hitter's approximate value in only one case: Glenn Braggs. Presumably PECOTA has improved over James' initial methods. But that's anecdotal; maybe it's all smoke and mirrors.

 

Greg.

[brettac1]

Nick Swisher is going to be a ?great player?.

 

Not to nitpick, as I agree with you on the all-encompassing stats for the most part, but I don't think anyone said Swisher would be "great", maybe "good", but not great. When I see him, I see Brad Wilkerson as a switch hitter, and that's a good player. A much better one than Bill Hall anyway.

[rluzinski]

A huge limitation of most any model in baseball is the natural variance in the stats themselves. If you took a guy who ALWAYS has a 25% chance to get a hit, he only actually has a small percent chance of actually hitting .250 over the course of the year. In fact, he has almost a 60% chance to hit .010 points above or below that, based strictly on normal distribution of coin flip events. He has an 8% chance to bat 25 points over his "actual" batting average:

 

http://i14.photobucket.com/albums/a345/rluzinski/bavar.gif

 

Just because a statistical prediction is off by 20 points on a batting average doesn't make it wrong. Even if you knew the EXACT ability of the player you wouldn't be exactly right. I think people need to keep that in mind.

[bjamesw22]

The thesis here -- the one scoop was talking about and the one I'm talking about -- is that Hardy's first half was not the "real" J.J. Hardy, because he got hurt last year and wasn't ready to come back, and that once Hardy got over that initial freakout, the "real" J.J. Hardy emerged.

 

So if he finishes the year with a .680 OPS and a .240 average are you really going to look back and say, well, that year he was really a .270 hitter?

 

Just because a statistical prediction is off by 20 points on a batting average doesn't make it wrong. Even if you knew the EXACT ability of the player you wouldn't be exactly right. I think people need to keep that in mind.

 

But again, you're assuming that a .250 hitter is actually a .250 hitter. How do you know? And we've already talked about this before...but that model TOTALLY falls apart at the extremes. That chart tells you nothing because of the non-certainty of the actual ability of a player and the way that it falls apart at the extremes.

[pogokat]

you guys can comment civilly, but if this turns into a one on one match there will be warnings issued

[EdgarDiazRocks]

Not to nitpick, as I agree with you on the all-encompassing stats for the most part, but I don't think anyone said Swisher would be "great", maybe "good", but not great. When I see him, I see Brad Wilkerson as a switch hitter, and that's a good player. A much better one than Bill Hall anyway.

 

I picked swisher because as of right now, in 2005, he and Hall are nearly identical statistically.

 

It's scary.

 

Yet somehow because of Swisher's higer walks (14 as of yesterday) against Hall higher Hit total (18 as of yesterday) Nick is determined to be the better prospect.

 

The two guys are 11 months apart in age so I'm kinda at a loss at why one is regarded so much higher than the other. Hall's plate discipline improves every season, Swisher's BA for the most part is unchanged. What leads to the optomism on him and not Hall?

 

Swisher's minor league numbers that Hall didn't get to have at that age because Taylor rushed him to the bigs?

 

That said, everything a player does in offense is aimed at one goal; scoring runs for the team.

 

You see, we differ in opinon there too. I think at least 1/3 of a player job at the plate is to entertain the fans. And people don't pay $20 to watch somebody NOT swing. No matter how much that may or may not help a team win. How many bases loaded walks to win a game do you think are in people's top 10 baseball memories?

[Brian the Automator]

[ I think at least 1/3 of a player job at the plate is to entertain the fans. ]

 

Maybe they can sign Ben Grieve under the condition that he learn to juggle while walking to 1st base in the offseason.

[DougJones43]

Quote:

---

Hardy's first half was not the "real" J.J. Hardy, because he got hurt last year and wasn't ready to come back

---

isn't that a prime example of how unreliable stats can be? because they don't take things like this into account? that's an entire half-season worth of baseball that stats would have never predicted due to the "human element."

 

to me, this quote is taking one of the big flaws of statistical analysis, and just disregarding it.

[brettac1]

I think at least 1/3 of a player job at the plate is to entertain the fans. And people don't pay $20 to watch somebody NOT swing. No matter how much that may or may not help a team win. How many bases loaded walks to win a game do you think are in people's top 10 baseball memories?

 

I like to see guys not swing, if the pitches aren't strikes. I know you like Ichiro and all, but he's a rare breed, and it sure would help him to take a walk this year since hit luck seems to have taken a downturn. If nothing else, he's a posterboy for the fluctuation of batting average in comparison to walk rate.

 

Ichiro year-by-year AVG

.350

.321

.312

.372

.297

 

Ichiro year-by-year Iso-OB

.031

.066

.040

.042

.050

[bjamesw22]

But again, you're assuming that a .250 hitter is actually a .250 hitter. How do you know?

 

I want to throw an example out there with this statement. We'll use JJ Hardy as the focus. He will probably finish the year around .240. The percentages of him batting worse than that are much lower than that. The chance of him batting better than that, much better in fact, is much higher than that chart lists.

 

It's just another stat plugged into a computer that is supposed to tell us something that it really dosen't. I would consider it more accurate if it was used for players strictly in their prime years...but JJ Hardy is not in his prime years.

[Brian the Automator]

The thing that bothers me about this entire argument is when people point to the exceptions and say "See! Stats are totally unreliable".

 

The point of statistical analysis isn't to predict everything with absolute certainty. It's too look at trends and say "what is the most likely thing to happen".

 

If I were to say something like "8 of 10 people who smoke a pack a day or more over the course of X many years are likely to die of lung cancer or a smoking-related disease" people would make the argument that since other 2 people got hit by a bus, statistics on smoking are irrelevant.

 

People use statistics and probability to model scenarios for a reason, not just because they like numbers and arguments. While it's not going to give you guaranteed results, the chances that it will aid in making an informed decision is quite high.

 

It's the blend of scouting and analysis that leads to success... not purely one over another.

[EdgarDiazRocks]

Maybe they can sign Ben Grieve under the condition that he learn to juggle while walking to 1st base in the offseason.

 

Juggling is entertaining? Who knew?

 

These guys must be huge with the ladies.

 

http://www.delafont.com/Jugglers/Juggler_Images/fxbrothers-96.jpg

 

I like to see guys not swing, if the pitches aren't strikes. I know you like Ichiro and all, but he's a rare breed, and it sure would help him to take a walk this year since hit luck seems to have taken a downturn. If nothing else, he's a posterboy for the fluctuation of batting average in comparison to walk rate.

 

I understand the point entirely. I just like to see guys at the top of the order get on base and guys in the middle of the order drive them home. It's a very traditionalist stance, but as I have stated before the argument that "winning is the only thing that get's fans in the seats" doesn't hold water.

 

I like watching different guys do different things.

 

Jeff Francoeur is also a posterboy. One for riding an exciting player out for all he's worth to higher ticket sales. Can fans rally their kids up by saying lets all go watch Kevin Youkalis not take the bat off his shoulder?

 

And in my Roto league I still traded him for Jeremy Hermida because of the no walk thing.

 

And none of this answers my "why are pitchers held to a different standard" question.

[bjamesw22]

I agree with that...but the model of %'s that a player will perform above or below their "ability" is a very flawed chart. It doesn't take some major things into equation.

 

Age

Injury

How a player ability changes over the course of their careers

 

You can't make an informed decision based on a chart where the variable is a floating variable.

 

Again, I agree that many of them do aid in decision making but it depends what you're using it for and strictly you adhere to their rules.

[rluzinski]

You see, we differ in opinon there too. I think at least 1/3 of a player job at the plate is to entertain the fans.

 

Fans are far more entertained, in the end, not by an AB but by the result of the game. If their team wins, they are happy. If their team wins enough to win the World Series, the fans are very happy.

 

I know you are trying to say that Ichiro is worth more to a team because the fans love him and you are probably right. While that type of worth may show up on an accountant's spreadsheet, SABR fella's are concentrated on a player's worth in winning games.

 

He will probably finish the year around .240. The percentages of him batting worse than that are much lower than that. The chance of him batting better than that, much better in fact, is much higher than that chart lists.

 

The whole point of my post was to show that even if you took a robot hitter that had a 25% chance of getting a hit per AB, he wouldn't bat .250 as a result of the natural distribution of probabilistic events. Therefore, the very best simulator in the whole world that could somehow magically KNOW what the exact abilities of a player will be for the next year STILL will be off as often and as much as my chart shows.

 

I agree with that...but the model of %'s that a player will perform above or below their "ability" is a very flawed chart. It doesn't take some major things into equation

 

It's not SUPPOSED to. It's only showing that no matter WHAT, all samples of probabilistic events will have some variance. Flip a coin 10 times. You should flip 5 heads but you normally won't. Google "binomial distribution" for more information.