Judging a Starting Pitcher's Performance: "Stuff" vs. Results

[rluzinski]

Very interesting look at AJ Burnett's best and worst starts in 2009.

 

http://www.hardballtimes.com/main/article/what-do-stats-tell-us/

[greeg35]

The biggest thing that jumped out to me is that what the hitter is expecting vs what the pitcher throws will have an effect on the outcome. If the pitcher is predictable you will have a lot worse results. I'm sure there's alot more to it; but there's my 5 minute read, one minute react reaction. Fun read.

[Funketown]

That's a very interesting read. I think that further supports the notion that a lot of what we believe we see, in terms of pitch quality, when watching a game is based on the outcomes of those pitches. Kind of like how a Gallardo fastball down the pipe on a 3-2 count is a great pitch if it's a called/swinging strike, but a mistake if it gets hammered. Same pitch, different result, yet we hold each pitch in a different regard. That said, this whole idea of context and situational pitching is so difficult to understand. Wouldn't surprise me to see a lot of research done on this topic in the near future.

[rluzinski]

I'm am convinced that if us fans saw every pitch that a pitcher threw in the zone for a year but not their outcome, we would struggle to identify the good pitches from the bad. 99.9% of the fan's observations are results based. I'd like to think the professionals would do a much better job but looking at this makes me wonder....

 

Of course, this is just one case study.

[Funketown]

My one pet peeve with announcers has always been "Pitcher X is really doing a good job keeping the ball down in the zone" during good starts, and "Pitcher Y is leaving the ball too far up in the zone" during bad starts. Another one of my favorites is watching games on TBS with their ball tracker (is this thing as accurate as PitchFX?), and hearing them criticize a pitcher for leaving a ball over the plate when it's in the corner of the strikezone.

[DuWayne Steurer]

Context is everything here. A 3-1 curveball in the dirt is a bad pitch. An 0-2 curveball in the same location is most likely a good pitch.

 

Just taking a pitch without context and looking at the break, velocity, and location tells us nothing. Consider what the pitcher is trying to accomplish on that pitch, because it's not as simple as saying, 'well obviously it's to get the batter to swing and miss'. What was the pitch location in relation to the targeted delivery spot? What was the pitch location in relation to the current batter's strong or weak areas of the strike zone?

 

I'm not trying to dismiss the author's intent here, I'm simply saying there's a lot of context into each and every pitch that has to be considered, not just the measurable data such as location, break, and velocity.

 

I do think to some degree, a reasonable baseball fan can, with context, judge weather most pitches are 'good' or not. I know if Yo fires a 2-0 fastball right down the middle and the batter swings and misses, I don't say 'great pitch!' just because the batter missed. I'm sure most others don't either.

[logan82]

RoCoBrewfan wrote:

I know if Yo fires a 2-0 fastball right down the middle and the batter swings and misses, I don't say 'great pitch!' just because the batter missed. I'm sure most others don't either.

I am not so sure about that. I think most fans are results driven. They would probably say "Wow, Yo just blew that one by that guy."

[greeg35]

Most fans or most 'educated' fans? I know I have plenty of instances where someone (Vladimir!) golfs a good pitch out of the park and I know it was a good pitch despite the result. Not that I'm educated http://forum.brewerfan.net/images/smilies/smile.gif

[dlk9s]

Context is everything here. A 3-1 curveball in the dirt is a bad pitch. An 0-2 curveball in the same location is most likely a good pitch.

 

Just taking a pitch without context and looking at the break, velocity, and location tells us nothing. Consider what the pitcher is trying to accomplish on that pitch, because it's not as simple as saying, 'well obviously it's to get the batter to swing and miss'. What was the pitch location in relation to the targeted delivery spot? What was the pitch location in relation to the current batter's strong or weak areas of the strike zone?

Additionally, sometimes pitches are used to setup other pitches later in the game or at least get the batter thinking too much. If the batter happens to swing and miss, all the better.

 

I remember in high school, even though I was very successful, one of my coaches taught me a knuckleball. I asked him why he wanted me to learn it and he said that I should just throw it a couple times during first inning warmups or maybe once to a batter in that inning. That way, the other team sees I have it and will start thinking about it. Even if I never throw it the rest of the game, it will be in their head, decreasing the chances of them guessing my pitches correctly. I know there's a big difference between high school and the pros, but I thought that was a fun example.

[Thurston Fluff]

It appears to me the article suggests luck has a lot more to do with success than I would think possible. So either the article missed some vital differences or talent really doesn't matter all that much.

[rluzinski]

I agree that context matters and I also agree that it would be very helpful to know what the intended location was. But to say that knowing the speed and location of a fastball/changeup or break and location of a curveball tells us nothing is quite the overstatement, IMO.

 

If you knew nothing of a pitcher and were given the choice of either seeing the data presented in this study (20 starts) or just the context, pitch type and location data (20 starts), which one would do you think could give you a better idea of a pitcher's future expected performance? I think I must be taking your "tells us nothing" comment too literally?

 

Game theory is only going to take a pitcher so far. Yeh, a pitcher should throw more balls out of the zone when the batter has 2 strikes but how do you decide what is the "right" pitch in a specific situation?

[JohnBriggs12]

It's an analysis of one pitcher so I don't put much stock into it. Obviously, what the count was when a pitch is thrown matters. It also can't measure some things, like whether on a particular day a pitcher is slowing his arm motion and giving away his changeup.

 

If you want a better analysis, I suggest going on SI website's vault, and downloading the July 8, 1968 issue that has cover story on Ted Williams Science of Hitting.

[rluzinski]

It appears to me the article suggests luck has a lot more to do with success than I would think possible. So either the article missed some vital differences or talent really doesn't matter all that much.

 

What it's showing you is that a pitcher's performance might be very consistent but the results can still vary significantly, from game to game. Fans are seeing binomial variance (luck of the dice roll) and assuming performance variance. With enough starts, much of that randomness cancels itself out, which reduces the binomial variance and let's the performance shine through. Over 30 starts, good performances has a decent chance of rsulting in results. Over 90 starts, its almost guaranteed. It's the signal (talent) to noise (luck) ratio.

[TooLiveBrew]

How close would anyone estimate us being to having a stat something like FIP, except it would also take out the batters' component? In other words, something like FIP that focused on velocity, location, break, pitch f/x, etc... analyzing more strictly just the pitcher instead of also including what batters do with his pitches.

[Thurston Fluff]

What it's showing you is that a pitcher's performance might be very consistent but the results can still vary significantly, from game to game.

 

I understand that is what it is trying to show,. I just question if it did show that. This isn't directed at you, you know I like your stuff and certainly respect your point of view, but I think sometimes stat people write off things in their analysis as luck or random variance a little too often. I think if luck or randomness seems to play a large role then maybe a more in depth look is warranted instead of just writing it off as noise or randomness.

If there is a large amount of luck in performance one would think there would be a larger variance in players performance and team results would routinely vary without much relationship to talent. Yet that doesn't seem to be the case. So I think something is going on related to actual ability more than just noise or luck.

What that is I'll leave to those smarter than I.

Good article that does make a point of how small the differences in performance may be but not convinced it goes deep enough to say the same pitcher can pitch the same and have drastically different results.

[rluzinski]

The Yankees can crush the Red Sox one day and lose to the Royals the next, so there's obviously a huge amount of uncertainty there. Sure personel changes and maybe a guy is more or less hurt but sometimes balls just find holes or liners get caught. There is a tremendous amount of luck involved.

 

And It's not like we can't estimate the size of the luck compnent. If a batter is a true .300 hitter, we can estimate how often he is expected to have an O-fer game. Fans will say the batter just didn't play well but that isn't necessarily true. He might have played exactly how he always does. Of course, he might have actually not been a true .300 hitter for that day, for whatever reason (injured, pitcher strength, etc...).

 

Observed variance = true variance + luck

 

I agree that sometimes studies are too quick to attribute variance solely to luck. We could assume that the difference between Burnett's results is just luck but that would be wrong. There are many other factors not being considered. I don't think this author even suggests that he's captured everything, however. He's just quantified what we typically refer to as the pitchers "stuff".

[DuWayne Steurer]

I agree that context matters and I also agree that it would be very helpful to know what the intended location was. But to say that knowing the speed and location of a fastball/changeup or break and location of a curveball tells us nothing is quite the overstatement, IMO.

 

If you knew nothing of a pitcher and were given the choice of either seeing the data presented in this study (20 starts) or just the context, pitch type and location data (20 starts), which one would do you think could give you a better idea of a pitcher's future expected performance? I think I must be taking your "tells us nothing" comment too literally?

 

Game theory is only going to take a pitcher so far. Yeh, a pitcher should throw more balls out of the zone when the batter has 2 strikes but how do you decide what is the "right" pitch in a specific situation?

Yeah, I worded that poorly. Obviously velocity and break are indicators of physical ability. You can't teach someone to throw 98 mph. They either can, or they can't. I think luck is involved to some degree, but there's variances here that aren't being taken into context.

 

Why is Trevor Hoffman's changeup so good, while other's are mediocre with it? Is it because his release point, arm angle, etc, are all the same as his fastball, while other's have difficulty throwing off speed with the same body motion as their fastball? I'd have to say it's at least worth considering. That again is something that isn't being brought up here.

 

Again, I'm not trying to be dismissive of the subject material, it's interesting to be sure. It just seems like the author is ignoring a lot of contextual information and just randomly calling it 'luck'.

[endaround]

There is a difference between "luck" and a bunch of variables that are unobserved or too small individually to matter. For example we know lighting has a large effect and isn't really "luck" as in the sun sets at a given time. However it is beyond the pitcher's control.

[Ennder]

Over a single game there is a ton of luck in baseball, over time the skill wins out though. It is kind of like Poker, you can lose money over a session to players who are worse than yourself very easily but if you play them long enough they'll eventually end up broke.

 

I don't know that his article proves this but it seems so obvious to me that it always surprises me when someone disagrees with it.

[logan82]

Over a single game there is a ton of luck in baseball, over time the skill wins out though.

 

Over time isn't necessarily one year.

[topper09er]

I think the conclusion that just because all the average pitch speeds and breaks are similar he had the smae stuff in both samples must be wrong. The difference has to be that over 1000 or so pitches in both sets a few hangers will not change the average, but in the bad outings he must have hung the wrong pitch at the wrong time. I think the biggest thing this article indicates is the worst fear of any super stats person, that how the player performs based on the situation is the most important aspect of baseball, or any sport. Stats guys want to think that as long as you try to walk or hit a HR every at bat no matter the situation you are doing the best you can or as long as you have the same velocity and break on your pitches no matter the situation you are performing at a maximum level and anything that shows that is not true is just based on luck.

[Ennder]

Stats guys want to think that as long as you try to walk or hit a HR every at bat no matter the situation you are doing the best you can or as long as you have the same velocity and break on your pitches no matter the situation you are performing at a maximum level and anything that shows that is not true is just based on luck.

 

In general stats guys just want to look at how the game works and find better ways to understand it. Anti-stats people seem to try to make up elaborate examples as to why stats don't mean anything from what I can tell. I don't really get it but oh well.

 

A pitcher can make an absolutely perfect pitch in the perfect situation and have it go over the fence, he can throw a total meatball with the bases loaded and get a pop out. When those things hold true how on earth can there not be some luck in the results over a single game.

 

If you make a perfect pitch to Albert Pujols it is still a hit some non 0% of the time. If you make the worst pitch you can throw to him it is still an out some non 0% of the time. When part of the outcome is completely out of your control which is the case here, then there is some elemental of variance or luck or whatever you want to call it. The larger your sample of the results the less the variance or luck involved. A single game just is not a large sample at all. Sometimes you throw the ball good and have a bad game, sometimes you make a lot of mistakes and have a good game. That is baseball.

[endaround]

People have a hard time dealing with the fact that much in life is random and outside of an individual's control. Baseball isn't any different.

[rluzinski]

It just seems like the author is ignoring a lot of contextual information and just randomly calling it 'luck'.

 

I think the author could have been more clear when he said, " I think I've established that there was practically no difference in how he pitched in his good starts compared to his bad starts." There are many other factors he missed or didn't quite capture perfectly. But I hope we can all agree that what he has captured plays a HUGE role in what makes a pitcher successful. He's captured velocity and movement perfectly. He's also at made at least a crude check on location (down the middle, on the edge strike, on the edge ball). What percent of what makes a pitcher successful do people think he has captured? Less than 50%?

 

About "luck", I think some fans truly believe that if you did know the true odds associated with every AB, you could perfectly predict the outcome of not only that AB but the game. Not even close! For whatever reason, luck is concept that many sports fans are still very resistent to. Some even take it almost personally (see Topper's elequent response above). And even if someone is open to the idea, it's a hard thing to explain.

 

Anything with odds has luck. IT IS ALWAYS THERE, IT CAN NEVER BE ELIMINATED. The odds of a head is 50% but you can roll 7+ heads in 10 flips pretty easily. People know this intuitively when they are rolling dice or flipping a coin. When they are dealing with something with unknown odds (like people), however, they invariably want to attribute too much of uncertainty to the odds. You are still dealing with the same sample error you are with the coin flip PLUS the unceratinty of the actual odds. If Burnett gives up an .800 BA in 10 AB, do we know it was because of only luck? Of course not. What we know is that even a "good" Burnett will still give up 8 hits in 10 AB some of the time through no fault of his own. We can even estimate how often we think he should, based on our estimateof his true talent. If, in fact, he's doing it more often than we think he should, THAT's when we should question whether we've really correctly estimated his true talent.

 

I suggest to anyone who wants to understand the role of luck in sports, you really need to do some research. I'm not suggesting that you should be able to perform the associated calculations. I just thing you would get a better appreciation for what the hell I'm rambling on about if you went online and started reading. Specifically, learn about binomial distributions and bernouli trials. And if you spend 2 minutes googling those phrases and feel compelled to come back here to post, "These are human beings, not coin flips!!!!", you didn't spend enough time looking into it. http://forum.brewerfan.net/images/smilies/smile.gif

[TheCrew07]

I think the article missed the 2 or 3 biggest components of the game. Intended location vs actual location, the context of the pitch, and the mythical "what the hitter was looking for". I think you'd get a much more accurate representation looking at pitchf/x data for the pitches put in play in both kind of starts. The pitch, velocity, relative location, the count, and so on. The hitters only need to get 1 pitch when they are looking for it for something good to happen for them.

 

I think it's pretty easy, especially on television to see where the catcher sets the target to where the ball actually ends up, and that's the first thing I look for when the game starts, which pitcher is hitting their spots. Of course every pitcher is going to miss a spot and sometimes hitters will swing through pitches they should mash... Prince has a classic stare at his bat when he does it that makes me chuckle every time. However what the data the author looked at actually supports rather than luck is that the mind game between pitcher and hitter is much more important than many statistically inclined people are willing to admit, which I guess can easily be dismissed as luck or random variance. Of course there is another side, if you're pitcher who knows a certain kind of pitch induces ground outs and pop ups X% of the time, then you just feature that pitch and don't worry about studying hitters and their tendancies any more. I forget where I read the article, I think it was a link out of this site or BCB and I forget who the pitcher was but he analyzed which pitch he threw was most like Webb's best pitch and started to feature that pitch more. If I wasn't at work I'd take the time to go find the exact article because it was a great read.

 

At any rate I've always believe that it's not what you throw as much how you throw it and when you throw it, I think this quote below from the comments pretty much sums up my feelings.

 

I believe that the data may support the following conclusion: When pitchers throw a predictable pitch (i.e. a fastball in a fastball count) in a hitter's count, they lose the huge advantage they have over hitters, and in turn, they are more prone to the vagaries of chance.