Judging a Starting Pitcher's Performance: "Stuff" vs. Results

[DuWayne Steurer]

It just seems like the author is ignoring a lot of contextual information and just randomly calling it 'luck'.

 

I think the author could have been more clear when he said, " I think I've established that there was practically no difference in how he pitched in his good starts compared to his bad starts." There are many other factors he missed or didn't quite capture perfectly. But I hope we can all agree that what he has captured plays a HUGE role in what makes a pitcher successful. He's captured velocity and movement perfectly. He's also at made at least a crude check on location (down the middle, on the edge strike, on the edge ball). What percent of what makes a pitcher successful do people think he has captured? Less than 50%?

 

About "luck", I think some fans truly believe that if you did know the true odds associated with every AB, you could perfectly predict the outcome of not only that AB but the game. Not even close! For whatever reason, luck is concept that many sports fans are still very resistent to. Some even take it almost personally (see Topper's elequent response above). And even if someone is open to the idea, it's a hard thing to explain.

 

Anything with odds has luck. IT IS ALWAYS THERE, IT CAN NEVER BE ELIMINATED. The odds of a head is 50% but you can roll 7+ heads in 10 flips pretty easily. People know this intuitively when they are rolling dice or flipping a coin. When they are dealing with something with unknown odds (like people), however, they invariably want to attribute too much of uncertainty to the odds. You are still dealing with the same sample error you are with the coin flip PLUS the unceratinty of the actual odds. If Burnett gives up an .800 BA in 10 AB, do we know it was because of only luck? Of course not. What we know is that even a "good" Burnett will still give up 8 hits in 10 AB some of the time through no fault of his own. We can even estimate how often we think he should, based on our estimateof his true talent. If, in fact, he's doing it more often than we think he should, THAT's when we should question whether we've really correctly estimated his true talent.

 

I suggest to anyone who wants to understand the role of luck in sports, you really need to do some research. I'm not suggesting that you should be able to perform the associated calculations. I just thing you would get a better appreciation for what the hell I'm rambling on about if you went online and started reading. Specifically, learn about binomial distributions and bernouli trials. And if you spend 2 minutes googling those phrases and feel compelled to come back here to post, "These are human beings, not coin flips!!!!", you didn't spend enough time looking into it. http://forum.brewerfan.net/images/smilies/smile.gif

Russ, I'm smellin' what you're cookin here, believe me. I'm not arguing the point you're trying to make. I guess I'm saying if the guy was going to do a study, he should have/could have went all the way with it. There's too much contextual data that was left out, *in my opinion*. I'm not saying at all that the addition of that contextual data would push luck out of the equation, not at all. I'm saying the data that he left out has meaning, that's all. We're on the same page as far as what the data means, I just think there could have been more of it.

 

You're saying potato and I'm saying potahto, and I think we can leave it at that.

[stalton]

After reading the article and all the posts, I am very intrigued by the stats and the analysis the author provided. I know pitcher's BABIP has been said here and other places that luck plays some roll into this stat. I'm guessing Burnetts 10 worst starts would have a higher BABIP simply because of the increase in number of hits/runners than the his best 10 starts, but who knows.

 

Otherwise, I thought the article was an interesting read, and would really like to see somebody do more analysis into this.

[Thurston Fluff]

There is a difference between "luck" and a bunch of variables that are unobserved or too small individually to matter. For example we know lighting has a large effect and isn't really "luck" as in the sun sets at a given time. However it is beyond the pitcher's control.

 

I think it depends how luck is defined. For instance it is luck that the lights are a certain way but it isn't what most stats people would call luck in a calculation. It's an uncalculated variable. It is my personal belief (nothing more than that) there are a lot more unobserved variables going on that generally gets lumped into the luck category.

 

A pitcher can make an absolutely perfect pitch in the perfect situation and have it go over the fence, he can throw a total meatball with the bases loaded and get a pop out. When those things hold true how on earth can there not be some luck in the results over a single game.

 

this is where I disagree. A pitcher who throws the perfect pitch in the right situation would rarely if ever have it hit over the fence. Unless you think the hitter could hit the ball with his eyes closed over the fence as often as he can hit the perfect pitch over the fence you have to dig deeper for the explanation.

There is a difference between being anti stats and questioning how well the current stats tell what happened. I am not anti stats. In fact I believe stats can tell you almost everything and predict to a very great degree if we have the right ones. Until we have the perfect ones I am of the opinion that stats should be questioned so they can be improved. Thus I ask questions about the ones being used. IF we end up at a point where we have the perfect stat for the occasion I tend to think we will find very little luck is involved. Again with luck being defined in the sense that it is completely random with no variables that can account for it. Closing your eyes and hitting a baseball would be complete and random luck. Having your eyes open and hitting it should be assumed to have some amount of skill involved. It is the job of the stats to uncover what skill that was and predict how often it will happen and when. I believe stats can do that to a very great degree. Just disagree the current ones do that at a deep enough level. Thus to beleive in current ones is to beleive in more luck than perhaps is really there.

[TooLiveBrew]

A pitcher who throws the perfect pitch in the right situation would rarely if ever have it hit over the fence.

 

Rare, but it happens. You're actually agreeing with this point.

[Thurston Fluff]

Rare if ever is not agreeing. The "if ever" was put there for a reason. It is possible it is random I just am not as sure we can assign luck to all the things some seem willing to designate as such. One would think those who rely on statistical evidence would want more than it's just luck as an answer. I would also think to assign it as luck or to say it really does happen multiple times dureing any given game is a huge assumption that should be backed up. Thus why I question if it is just random luck or are the stats missing something and need to be refined.

Since we know there are skills involved it makes sense to me we should assume it took some skill to make an event happen. To just assign it as luck just because the current calculation doesn't show the skill seem contrary to what those very calculations are attempting to do.

Really I'm not slamming the stats or those who go by them. I'm endorsing them. Just think they need to sometimes dig deeper instead of just writing it off as random lucky events.

Random events that are out of control are still events that had a cause. In baseball those causes should not be called luck until every factor is exhausted. Then we can make areal determination of how much luck is involved. I tend to suspect there is less of it than some others may. If an event has a cause we must find that cause to know if it is controllable or just plain old luck.

[TooLiveBrew]

No, "rare if ever" means that it can happen even if a pitcher makes the perfect pitch. It's exactly the point people have been trying to make. If you are honestly arguing that if a pitcher makes a perfect pitch it will never be hit well, there's not much point in continuing the conversation.

[Ennder]

this is where I disagree. A pitcher who throws the perfect pitch in the right situation would rarely if ever have it hit over the fence. Unless you think the hitter could hit the ball with his eyes closed over the fence as often as he can hit the perfect pitch over the fence you have to dig deeper for the explanation.

 

If he hits it over the fence with his eyes closed just 1 out of 1000 times the 1 time he actually does it is going to be variance or luck. Eventually at some point he is going to make that perfect pitch and still give up a HR. The opposite example is probably better though. Pitchers throw meatballs all the time that don't go for hits, you hear the announcer say 'I bet he wishes he could get that swing back', because the pitcher made a clear cut mistake and the hitter did not make him pay. This stuff is going to happen and there is no way it evens out over a single game.

 

You are never going to be able to isolate exactly what is or isn't luck of course, but most of the more advanced stats are trying to regress things to try to give a better picture of talent over a small sample than more traditional stats can hope to show. Just using old fashioned stats you need a good 3+ years of data to really learn much about a player.

[Baldkin]

http://losangeles.dodgers.mlb.com/media/video.jsp?mid=200810013580503&c_id=la

 

Perfect pitch, perfect location (it's gonna bounce on the back of the plate!). Hit 420 feet.

It happens.

 

Ben Sheets actually got hits. He might've done better if his eyes were closed.

 

I think you're misconstruing the use of the word "luck" to mean more than most here want it to mean. In this context, all luck is, is random variance. Good luck, being variance in favor of the player (3 black jacks in a row) and bad luck being against the player (an ace after doubling down on 11).

[jerichoholicninja]

It seems to me the best way to use the data about location and velocity would be to judge it against the results. If a player is leaving balls over the plate but still is getting outs he very well could just be very lucky and might be come back down to earth eventually. A player can put up terrible numbers but be painting the corners and throwing breaking balls at the ankles and comparing the two pieces of data will show that he is just very unlucky.

 

We will never have a "be all end all" set of data to judge players so we have to use all the data we have to make decisions.

[Thurston Fluff]

If he hits it over the fence with his eyes closed just 1 out of 1000 times the 1 time he actually does it is going to be variance or luck.

 

Sure if it ever happens. since there has never been a reported case where that happened I don't know how you can use it to show luck is involved. Yes there is luck involved to some degree. I question to what degree. I just happen to think there is a lot more stuff written off as pure luck than is really involved.

 

 

I think you're misconstruing the use of the word "luck" to mean more than most here want it to mean. In this context, all luck is, is random variance. Good luck, being variance in favor of the player (3 black jacks in a row) and bad luck being against the player (an ace after doubling down on 11).

 

Random variance is not the same in coin flips as it is in baseball. Treating them as such is a mistake. In flipping a coin there is no amount of skill involved to change the odds. IF you flip the coin five times and it comes up heads five times it is pure luck. In baseball there is a huge amount of skill involved which means a player who normally hits the ball 2 out of 5 times suddenly hits it 5 out of 5 cannot just be called luck. It could as easily be he's seeing the ball well or he faced some pitchers whose repertoire is something he's good at hitting. AS time goes on it usually evens out. Sometimes it doesn't. That still doesn't mean the player had no variance in his skill set that could account for it. It may not even be something he knows is different. That still doesn't mean it isn't a skill based measurement. For analysis as a fan this is all academic. Most times those variables, luck or otherwise, go back to the norm. AS a GM, coach or player I would be all over any variance to find out what changed. I would never just say it was luck and leave it at that. I'd try to find the reason why so I could control it. Many times you might find out there is some measurable and correctable asset that made those numbers change. When Yost found a hitch in Jason Kendall's throwing motion was causing him to be a split second late on stolen bases attempts and corrected it for example. To have just written that off as luck or declining skill set would have been wrong but the problem could have been easily misclassified as uncontrollable.

To say luck is why something happened when so many controllable variables are in play just seems like too broad an assumption for evidence based statistical analysis. Not without exhausting every other possibility anyway.

 

Pitchers throw meatballs all the time that don't go for hits, you hear the announcer say 'I bet he wishes he could get that swing back', because the pitcher made a clear cut mistake and the hitter did not make him pay.

 

To say that is luck goes to show how easily things get called luck that isn't necessarily luck. Yes the pitch was one that should have been hit. But to say it was pure luck it wasn't neglects the fact that the pitch previous to that was different enough that it caught the player by surprise. Or that the players timing was off in that ab which caused him to miss the meatball. Again easy to say it was luck but not necessarily correct. When all the variables are added up it may be he missed because of a combination of surprise, excellent pitches prior to that meatball and the inadequate ability of the hitter at that particular time. Granted that amounts to luck but that type of "luck" is not pure random luck it is lucky in that all the things necessary for that outcome came together. That is not statistical random variance luck.

[Ennder]

We obviously just look at variance in a very different way so this isn't going to go anywhere. Nobody is saying that the game just comes down to random luck, that should be very obvious. Skill plays a huge part in baseball but when a hitter gets 5 hits in a day it doesn't just mean he was seeing the ball great that day, there is most certainly an elemental of variance in there as well. Skill wins out over a huge sample but over a small sample you aren't going to be able to separate the skill from the variance, but you can look at some stats like BABIP that can help you come closer to separating it.

[Thurston Fluff]

Skill wins out over a huge sample but over a small sample you aren't going to be able to separate the skill from the variance,

 

This goes back to my original point. Just because it is difficult to separate doesn't mean it should automatically be assigned to luck or randomness. It very well could be a trait that players work to achieve and hold when they do achieve it. Whether they are in good place or bad place at the moment is going to look like they just got lucky or unlucky but luck has way less to do with it than how well they are actually playing at the moment. My original contention was it seems to be designated as just a matter of luck a little too often.

 

I think it's mostly semantics. Randomness is different within the context of baseball than it is in things like flipping a coin. Coin flips are pure random happening. No amount of skill can possibly change the outcome. Baseball is an active exercise in decreasing the amount of luck necessary to achieve a successful outcome. The amount of luck that goes into it is a matter of degree. I happen to think there is far less luck than sometimes gets attributed to it. Changes in bapip to me is one of the stats that often gets relegated to luck when perhaps it shouldn't. It often seems like luck but it may be as simple as the hitter's stance changing ever so slightly thus his swing is later or earlier than usual so the ball gets hit were it usually doesn't. IF that happens it has a better chance of falling for a hit because the defense is positioned to where he used to generally hit the balls. I think that sort of subtle changes go on all the time in baseball. That isn't luck or randomness. It's a variance in skill set

[logan82]

It is very similar to a coin flip. It just has more variables. No matter what a player does, they are still only going to get a hit within a certain range of odds.

[Ennder]

Changes in bapip to me is one of the stats that often gets relegated to luck when perhaps it shouldn't. It often seems like luck but it may be as simple as the hitter's stance changing ever so slightly thus his swing is later or earlier than usual so the ball gets hit were it usually doesn't.

 

If it isn't sustainable then it doesn't really matter whether it was a hot streak, skill or whatever, it ends up being variance. When a hitter puts up a .370 BABIP I don't care why he did it, he isn't going to repeat it and it isn't indicative of his real long term skill level.

[topper09er]

A pitcher can make an absolutely perfect pitch in the perfect situation and have it go over the fence, he can throw a total meatball with the bases loaded and get a pop out.

I would say by definition this is impossible because a perfect pitch would result in the indended result of the pitcher, strikeout or double play ball etc. If the the ball is hit for a HR it was not a perfect pitch obviously. You cant ignore the mental aspect of hitting and the fact that a batter may be looking for a certain pitch in a certain situation. If he guess exactly what you are going to throw then you threw the wrong pitch.

[TooLiveBrew]

Or he could hit a lazy flyball that carries out with the wind blowing out at Wrigley.

[Ennder]

A pitcher can make an absolutely perfect pitch in the perfect situation and have it go over the fence, he can throw a total meatball with the bases loaded and get a pop out.

I would say by definition this is impossible because a perfect pitch would result in the indended result of the pitcher, strikeout or double play ball etc. If the the ball is hit for a HR it was not a perfect pitch obviously. You cant ignore the mental aspect of hitting and the fact that a batter may be looking for a certain pitch in a certain situation. If he guess exactly what you are going to throw then you threw the wrong pitch.

That is called results oriented thinking and is exactly the type of thing you want to avoid if you ever want to understand a subject. I don't mean just baseball, pretty much anything you ever study. I have seen a hitter chase a slider in the dirt and still manage to hit it over the fence, there was nothing wrong with the pitch. Sometimes you are going to do the right thing and get a bad result.

[Funketown]

A pitcher can make an absolutely perfect pitch in the perfect situation and have it go over the fence, he can throw a total meatball with the bases loaded and get a pop out.

I would say by definition this is impossible because a perfect pitch would result in the indended result of the pitcher, strikeout or double play ball etc. If the the ball is hit for a HR it was not a perfect pitch obviously. You cant ignore the mental aspect of hitting and the fact that a batter may be looking for a certain pitch in a certain situation. If he guess exactly what you are going to throw then you threw the wrong pitch.

That is called results oriented thinking and is exactly the type of thing you want to avoid if you ever want to understand a subject. I don't mean just baseball, pretty much anything you ever study. I have seen a hitter chase a slider in the dirt and still manage to hit it over the fence, there was nothing wrong with the pitch. Sometimes you are going to do the right thing and get a bad result.

Agreed. This is similar to thinking that because a drunk driver made it home safely through a blizzard while driving on the wrong side of the interstate, his driving must have been good.

[Thurston Fluff]

It is very similar to a coin flip. It just has more variables. No matter what a player does, they are still only going to get a hit within a certain range of odds.

 

No a coin flip has no controllable variables. There is no need to measure the results to find out who is the better coin flipper. The results have nothing to do with level of ability. When someone flips the coin fifteen times the results are purely random. All the variance is purely based on uncontrollable luck. In baseball the opposite is true. The vast majority of the outcomes are based on measurable skills with SOME luck involved. The differences we seem to be having is the amount of luck involved. I am not saying there is no luck involved. I am saying less should be attributed to luck than many times is.

 

In a game of skill it should be assumed some skill was involved in the outcome. If there is a measurable skill involved then the assumption shouldn't always be random variance beyond a players control. There needs to be a question asked about the measuring system. I think the stats do tell us everything but in order to tell if it was truly random luck or a measurable skill we may need more sophisticated calculations than we are currently using. To just buy into the stats at hand as tell all and ignore the outliers as random variance is the mistake that I feel is sometimes made.

 

Sometimes you are going to do the right thing and get a bad result.

 

That is an assumption that is hard to know is true to any statistical significance. IT may be on occasion something like that does happen. But we know there are bad ball hitters. That is a skill. So to look at that pitch we also have to look at the hitters skill at hitting balls in the dirt. Again difference of opinion you think it happens more often than I do. I would think the burden of proof is on the person who says it was random luck. After all we all know it takes a good amount of skill to do those things. To say it wasn't skill based should come with evidence.

 

To assume outliers in the norm are just random variance and thus not skill based in the thing I think we need to rethink. Outliers are something most people tend to want to ignore because it doesn't fit the theory they are viewing the subject through. It's a lot easier to just write it off as random variance than it is to find out why. That happens all the time in many fields. Then someone comes along with a study that can account for those outliers and we suddenly change the paradigm we view the subject through. I think there is a lot of that going on in baseball. If it wasn't there would be no Bill James types out there refining the stats we hold as gospel. Come to think of it if the stats we have now are perfectly suited Bill James' work would be done know. Yet his continued work shows he must not be happy with what he has.

[Ennder]

In a game of skill it should be assumed some skill was involved in the outcome. If there is a measurable skill involved then the assumption shouldn't always be random variance beyond a players control.

 

The problem with this is the skill of at least 2 and usually as many as 3 or 4 players determines the outcome of every play in baseball. The player rarely has over 50% control of the outcome, 50% of the outcome is largely out of his control by definition. Even the 50% at most control he has is subject to a lot of variance as seen by how erratic hits are in batting practice when they are just grooving balls in for you. Over a huge sample you can mostly ignore this stuff but over a small sample it makes the level of variance way too high to tell pretty much anything about a players skill. Nobody is saying skill isn't a huge part of baseball, they are just saying when you look at a small sample there is a lot of luck too and there are stats out there that can be identified as more variance based than others. BABIP, LOB%, HR/FB%(for pitchers at least) all are heavily influenced by year to year variance.

[Thurston Fluff]

I get what you are saying Ennder. What I'm not so eloquently trying to say is even if there are 3 or 4 players who determine the outcome that doesn't mean the variance is luck. It is the combined skill of the players involved more than luck. In one sense it is lucky to have good players helping make plays but that still amounts to skill based measurable. I know it's splitting hairs but when it comes to measurable skills vs pure luck based results those split hairs matter.

 

 

I guess I just am not willing to say variance in year to year stats like bapip are mostly due to pure random luck. I think mostly we disagree on why the variance is there not that there is variance.

 

Maybe some of the differences are based on what objective we have in looking at the stats. I tend to view them as how to improve or maintain performance. I look at how they performed and try to see if the stats/studies can tell me where they need to improve or why a certain outcome happened so it can be changed/maintained. Some others may view them purely as predictive tools. Neither are wrong just different uses for the same thing.

[Ennder]

I guess I just am not willing to say variance in year to year stats like bapip are mostly due to pure random luck. I think mostly we disagree on why the variance is there not that there is variance.

 

I personally could care less about why it is there because it is variance and not sustainable. I look at stats as a way to judge the talent level and future talent level of a player. If he does something that isn't sustainable it is variance and that data set is 'tainted' as far as looking at his talent level. A mediocre talent player can be hot all year and put up a great season, doesn't change the fact he is mediocre talent. You want to worry about why he was hot, was the BABIP spike a hot streak or luck or whatever. All I care about is it isn't sustainable so it isn't a true reflection of his talent.

[logan82]

I am not saying there is no luck involved. I am saying less should be attributed to luck than many times is.

 

I am not sure what you mean here. Nobody is saying very much luck is involved.

 

To assume outliers in the norm are just random variance and thus not skill based in the thing I think we need to rethink. Outliers are something most people tend to want to ignore because it doesn't fit the theory they are viewing the subject through

 

If it isn't repeatable, then it can't be given much weight. In general a guy with a BABIP outside of the "normal" range isn't going to repeat that year after year. Since that is the case, you can't give to much weight to it.

[Thurston Fluff]

I personally could care less about why it is there because it is variance and not sustainable. I look at stats as a way to judge the talent level and future talent level of a player. If he does something that isn't sustainable it is variance and that data set is 'tainted' as far as looking at his talent level. A mediocre talent player can be hot all year and put up a great season, doesn't change the fact he is mediocre talent. You want to worry about why he was hot, was the BABIP spike a hot streak or luck or whatever. All I care about is it isn't sustainable so it isn't a true reflection of his talent.

 

Or that same player found something that made him better. If someone can find what that is they can help the average player become above average.

 

 

I am not sure what you mean here. Nobody is saying very much luck is involved.

 

We are talking about how much luck in involved. What your definition of much is can very well be different than mine. We are now discussing what that level is. This all started by my assertion that luck often gets attributed in areas that it not warranted.

 

 

If it isn't repeatable, then it can't be given much weight. In general a guy with a BABIP outside of the "normal" range isn't going to repeat that year after year. Since that is the case, you can't give to much weight to it.

 

As a predictive tool your are right. That doesn't mean it shouldn't be evaluated as something that could possible be repeated if we found out what the reason for that bump was. If we find out there was a solid controllable factor then it should be addressed so it can be repeated. All this falls into the category of different uses for stats. I can certainly see both yours and Ennder's view that it is likely not going to be repeated so it can safely be viewed as irrelevant. Maybe it's the old coaching instincts coming through but I tend to want to know if there is something that can be controlled and repeated. Just a different way to use the same stats.

[Ennder]

Or that same player found something that made him better. If someone can find what that is they can help the average player become above average.

 

If he improved it wouldn't just be in BABIP though, that is the rub. A player doesn't just improve his BABIP and nothing else, I can't find a single player who has improved like that.