HBO Real Sports - Man vs Machine (Umpires vs computers for balls and strikes)

[trwi7]

 

I mentioned this in another thread a few weeks ago but hadn't seen the entire segment. I have watched it now and think it might deserve its own thread.

 

Some notes.

 

It starts out with Eric Byrnes talking about how he watches more games now as an analyst than he ever did as a player and constantly seeing games that are affected by wrong ball and strike calls. They then run through a series of clips of the most egregious calls they could find.

 

They go on to explain "High tech cameras are now in use in every major league ballpark capable of mapping the precise path of every pitch in real time. Which means that today that everyone watching a game, from broadcasters, to TV viewers, to fans watching online can see for sure whether a pitch either hits the strike zone or misses it. Everyone that is but the guy who gets to decide."

 

They go back to the Byrnes interview. He randomly pulls up a Texas/Anaheim game on his phone on the MLB app and shows gameday and whether or not every pitch is a ball or a strike. Byrnes goes on to say "So here we are, 3,000 miles away from Anaheim and I have this information (snaps fingers) like that and the person who's responsible for that call doesn't have the same information? Why do millions of people watching at home get to know if it's a ball or a strike, yet the poor dude behind home plate is the one left in the dark?"

 

Jon Frankel then goes on to interview former umpire Jerry Crawford. Get rid to get angry as you start reading more from Crawford.

 

Frankel: Could you ever see a situation where you might want technology?

 

Crawford: Never. It's it's it's ridiculous.

 

Frankel: But progress is a good thing, isn't it?

 

Crawford: Not in the game of baseball.

 

Frankel: Why do you think the umpire can't be replaced?

 

Crawford: Traditions make the game of baseball. You bring your kids to the game to watch the players, the managers, the umpires, eat hot dogs. That's what you do. It's all about tradition. I know the direction everyone seems to be going. Something mechanical can do the job better or won't get things wrong. The machine does get things wrong and umpires are doing an absolutely tremendous job umpiring balls and strikes.

 

Frankel wanted to test that so he went to Yale University and asked a leading economist for help. Toby Moskowitz agreed to analyze every pitch thrown in MLB in the last 3.5 years, about 1 million pitches total.

 

MLB claims umpires get 97% of the calls right, however, Moskowitz found that since 2013 that umpires are only 88% right. That equates to 30,000 mistakes per year and that includes the easy calls.

 

When the pitches are anywhere near the border of the strike zone, Moskowitz found that they miss at a much higher rate. They went on to show a strike zone and then a yellow region that is the equivalent of 4 inches wide extending 2 inches in each direction around the border of the strike zone. In a typical game, around 32 pitches get thrown in this region. Moskowitz found that umpires get pitches thrown in that area wrong 31.7% of the time

 

Moskowitz was surprised by that high of a number but after thinking about it and how fast every pitch is, it just shows the limitations of what humans can do, despite being trained.

 

They then expanded the yellow region to 3 inches to include pitches that should be even more obvious balls or strikes but even then, the error rate only went down to 25.9%.

 

To make matters worse, Moskowitz found that umpires aren't only missing calls but are doing so in a biased way. In an average game, they make 8 more mistakes in favor of the home team.

 

Moskowitz: Do they want to see the home team win? No, they're professionals, they want to be unbiased but it's hard to be unbiased as a human. They've got to make that judgment in a split second and what they're hearing is 50,000 fans who are quite sure of how that should go.

 

Frankel then goes on to say that hours after the interview with Moskowitz, that his favorite team, the Cubs won a game on a walk off walk on a pitch that was actually a strike.

 

Okay, now get ready to be angry.

 

Frankel: In the decisive game 7 of the 2011 World Series, the data shows that the home plate umpire missed 14 calls in favor of the Cardinals against only 3 for the Rangers. (During him saying this they show a pitch way outside to Josh Hamilton that was called a strike and Yadier Molina drawing a walk on a 3-2 pitch with the bases loaded and 2 outs in the 5th inning when it was still a 3-2 game, a pitch that was actually a strike.) Guess who won?

 

Frankel: As a baseball fan, is this disappointing, unacceptable to you?

 

Moskowitz: If there were no other alternative you would say, well, this is the best humans can do and that's what we gotta live with. But, if you care about getting it right, there's an alternative that's so much better.

 

They then go back to Jerry Crawford.

 

Crawford: I don't care what the guy from Yale's looked at to be honest with you. It's not even feasible. He's absolutely incorrect.

 

Frankel says that Crawford doesn't trust the math or the technology, which he says is no match for guys like him.

 

Crawford: These guys have gotten so good, so good at calling balls and strikes. You got guys that are not missing any pitches.

 

Frankel: (shocked, taken aback) You think there are guys calling perfect games?

 

Crawford: Yes, they're calling perfect games.

 

Frankel (voiceover): Major League Baseball appears to disagree. The league has not only installed this technology in every park but for the last several years has actually been using it to try to tutor their own umpires. Yes, after every game, umpires are shown exactly where each pitch actually was, so they can review what they got right and what they got wrong. At least, that's what the league thinks is happening.

 

Crawford: About 20 minutes after the game was over there would be a knock on the door, there would be a guy standing there, he'd have a disk, he'd say umpire Crawford, I'd say me, he'd hand me the disk.

 

Frankel: And you would usually do what?

 

Crawford: I threw it in the trash.

 

Frankel: Are you telling me you never sat down and used it as a learning tool to improve the way you called a game?

 

Crawford: No, I didn't. I never did.

 

Frankel then goes on to visit SportVision in Silicon Valley, creator of Pitch f/x and the same company who created the computer generated 1st down line in football.

 

Frankel: How does it work for you to actually track the pitches?

 

Zander (SportVision's GM of baseball products): So you see we have two tracking cameras, one mid 3rd and one mid 1st and each camera is tracking the pitch about 20 times, so we get about 40 samples of the pitch from when the pitcher releases it to when it crosses home plate.

 

He says you do need a human to oversee the program, pitch f/x is easy to use and accurate to within a half inch.

 

Frankel: How easy is it to establish the strike zone?

 

Zander: It's quite easy. So we have an operator, that will establish a strike zone for a player the first time we actually see that player and then we'll save that strike zone information so the next time that player comes up it's already loaded in.

 

Frankel: If a player comes up and decides to change his batting stance, how quickly can you adjust the strike zone for him?

 

Zander: The operator will do that instantly. As you can see here, I can drag the bottom of the zone and adjust it to where it needs to be.

 

They then went on to show the game that was called entirely by pitch f/x in the video at the beginning of this post. The players and coaches were skeptical. Afterwards, however, they seemed converted.

 

Matt Kavanaugh (former Indy League player, current manager of San Rafael Pacifics): The game wasn't left in someone else's hands, it was left in the players hands and I think that's what this game is meant to be.

 

Frankel: Did you feel like it interrupted the flow of the game at all?

 

Kavanaugh: Had great flow to the game. I think we had one of our shorter games of the year.

 

Frankel: If a big league manager came to you or an executive from MLB came to you and said, what do you think, should we implement this? What would you say?

 

Kavanaugh: Right now, today, I'd say do it for a full season and see what happens.

 

Back to the interview with Crawford.

 

Frankel: How would you feel if you were sitting behind the plate with an earpiece and someone telling you whether the machine saw it as a ball or strike?

 

Crawford: I would certainly feel inadequate. Let's face it, you're not doing what you were trained to do.

 

Frankel: Would you feel like you were just a puppet if the machine was making the calls?

 

Crawford: No question. You're not really doing anything.

 

Frankel wanted to ask MLB how it would feel about such a change, as well as what they think of the rates of umpire error the Yale professor found and why the league is willing to stop games, often for minutes on end to correct the rare missed call on the base paths. While on average they allow more than a dozen mistakes at home plate every game, which could've been gotten right in real time.

 

The league however would not talk nor would any of the players, not one of whom would agree to speak on the record.

 

They go back to Byrnes.

 

Byrnes: These guys aren't going to speak out on it!

 

Frankel: Why aren't the players speaking out about it?

 

Byrnes: The umpires! You think they want to be the guys to speak out against this? No way! There's no way I would've done it when I played either!

 

So what does everyone think about this? The argument was very compelling and made only stronger by having Crawford act as pompously as he did. Just a completely smug, arrogant interview. Me typing what he says doesn't do justice to how smug and arrogant he was.

[jerichoholicninja]

I have lots of thoughts on this and will probably post something bigger tomorrow but I would just like to point out that the system talked about still involves human judgement with the bottom and the top of the zone. I know there are specific rules that define the top and the bottom but it's still going to be a judgement on whomever is inputting the data as to what is the batters regular stance. Also, I've never seen check swings addressed. They don't happen all that often but it still needs to be considered. I guess they first would need to actually define what a swing is.

[Ennder]

I prefer a completely standardized strike zone that does not change game to game. I assume this means I prefer the robot ump. Pitchers adjusting to the days umps strike zone just means a flaw in the system in my mind.

 

I can't speak to whether machines can call strikes better than a human just yet, but if they can get it working to the point where it is clearly the case I totally embrace the change.

[Oxy]

I keep going back and forth on this one. I kind of enjoy it when the catcher sets up for a ball a half an inch off the outside corner, doesn't move his mitt the width of a glove lace and the Ump calls it a strike. When the pitcher makes his pitch like that, there are certain times when I think it should be rewarded.

[MrTPlush]

Can anyone imagine how the game would be with a robot pitch caller? What do we do? All stare up at the scoreboard waiting for the call? Pitcher whips around to see the call? Weird I tell you weird. Honestly such a change would make the game more boring and hurt ratings.

[sveumrules]

The call is relayed to the home plate umpire in real time via ear piece so there would not be any significant change to the pace or rhythm of the game in that regard.

[igor67]

Why not just let the robots hit? What are we going to do if someone uncovers good film and does an analysis indicating errors in numerous historic World Series games?

[stoutdude04]

Can anyone imagine how the game would be with a robot pitch caller? What do we do? All stare up at the scoreboard waiting for the call? Pitcher whips around to see the call? Weird I tell you weird. Honestly such a change would make the game more boring and hurt ratings.

 

Did you watch the clip above?

[MilwaukeesBest]

Crawford talks about the human element being taken out of the game but it continues to happen with the growth of replay and the lack of manager/umpire arguments just to name a few. I do think this is inevitable, as well. I like the idea of having a standard, concrete strike zone that is intrinsic to each batter. Eliminating the "umpire show" every few nights where umps blatantly get a few calls wrong, someone argues back, and they're immediately tossed for showing them up would make the game cleaner and more honest.

[Cool Hand Lucroy]

FanGraphs did a series on umpire zones a few years ago, and I thought it was fascinating. Based on my subjective viewing of the Cubs-Dodgers series, Ted Barrett's Game 6 strike zone seemed much more fair and consistent than Alfonso Marquez's the previous game. For me, that means there are some umpires who can do a really nice job, and the troubling thing about the clip is that it shows some of the people who might benefit most are refusing to do so (I hope Real Sports didn't just choose Crawford because he was the most extreme--I'd like to think lots of umps can take a self-improvement approach).

 

My take is that a certain amount of subjectivity is okay in baseball. It's a human endeavor, and it's subject to human fallibility. The goal is to use technology to eliminate egregious bias and unfairness.

 

For me, a strike zone has never really been a precise thing. It's defined, but the definition is subject to variables that change (height, stance, etc.). That means that the same exact pitch could be a strike or a ball depending on the hitter. In that sense, "precision" is a myth. There's no such thing as a pure strike or pure ball. It depends.

 

What does that mean? For me, it means MLB should reward the best-rated umpires with whatever incentives they can. They should perhaps even appoint the best home plate umps to ump home plate more often rather than arbitrarily rotating. We can use the technology to get the human error rate down, save umpire jobs, get the call right, and keep the human element that traditionalists care about.

 

I guess what I'm saying is I don't see this as a thorny issue. You can have human umps that use technology to get better. Fire guys who act like Crawford and reward behavior geared toward improvement. Hire the right skills but also the right personalities. If you do that and use the technology right, this is something that can be fixed while keeping every group happy. I see no reason the baseball community can't get together on this. It ain't that hard, and the fact that the debate quickly bifurcates into entrenched positions when there are nuanced solutions that would be agreeable to most gets at problems that may be culturally deeper than just baseball.

[logan82]

There are fair strike zones and unfair strike zones. The worst part is the unfair strike ones seem to be with the really touchy umps like Joe West who is a total douche.

[BuckyBrewer61]

I am generally against instant replay and automation, but this I can get behind.

 

Yes, getting the calls right is great and I am all for, but not if it is at the expense of the rhythm, drama, and fun of the game. Instant replay in multiple sports is currently ruining that IMO. But if balls and strikes can be called instantaneously like this, and you still have an umpire there calling them out to preserve the tradition of the game, I'm totally for it.

 

Also, in the first post, there's mention of the Cubs winning on a walk-off walk on a pitch that should've been a strike. I believe that was against the Brewers, wasn't it? I vaguely remember losing on a pitch that was right over the middle of the plate, not even a borderline strike.

 

EDIT: Nevermind looks like it was a Cubs-Cardinals game this year. But I swear something like that happened at Wrigley to the Brewers like 5 years ago...

[I Heart Brewers]

I care about the man vs. machine part but even more; Crawford. That guy is just a vile human being. He sounds like the most unhappy person on the planet. How does someone like him ascend to the post of MLB umpire?

 

I truly think his kind are going to be extinct soon.

 

Anyway, traditions change. Who knows what the game will look like in 20 years but I do think there will be more algorithms in scouting, replays, and machines calling balls and strikes. Inefficiencies work their way out of most systems eventually. It's inevitable.

 

Finally, nobody goes to a game because ''X' is umping today'!

[dlk9s]

I'll guarantee Crawford votes for Trump.

[hurricanecrush]

I'll guarantee Crawford votes for Trump.

Keep the political garbage in the political forum.

[Fear The Chorizo]

I think I'm on board with the automated ball/strike calls - however I'm left wondering how that new system can also be potentially gamed or not be as good for the game as what most think...

 

Right now I would guess the umps call more strikes than what are actually pitched, meaning they err on the side of pitchers by giving an extra few inches on the corners or sticking with a low strike. I feel these pitches get call strikes more frequently than high strikes get called balls, since most pitchers tend to want to live on the corners and low in the zone. If things go automated, I would forsee an uptick in walks, more runs scored, and in general longer games.

 

I also think about the games being played under threat of rain or generally miserable weather, where the umpire is obviously giving a larger strike zone than normal to try and speed up the game. There would be no flexibility with the automated system and while that's good on paper, maybe dealing with unnecessary rain delays or 4 hour games played in 35-degree weather is actually worse.

 

Finally, I can see a Cardinals-type team messing with the system in their home park, recalibrating it to their advantage by a millimeter or two depending on whether they are hitting or in the field. I'm sure human bias exists for umpires depending on teams playing, but I'd rather deal with human bias than potential tampering with technology by teams.

 

All in all, I think it would be a great idea for MLB to give it a trial run for a small but analyzable sampling of games for a season (maybe 10%) to see how those games are played compared to human element homeplate umpiring. I think that study would be even more significant if the only people who knew what games were being called via the automated technology were the homeplate umpires.

[brewcrewdue80]

Id imagine guys like Crawford would give a big fu to MLB if this were implemented, still calling the strike zone as they wanted. Hearing a ball in ear piece, and calling a strike, to fit "their" zone.

 

There are a lot of concerns. Like the above mention of home team adjustments. Say opening the strike zone 1/2inch and narrowing 1/2inch. Or 1/4. What of the man in the room blurting what the zone dictates? But doesnt call a strike or ball properly to the ump. 2-2s ball just outside but as close to box line and says strike 3. But in truth its a ball. Or 3-1s ball paints the corner but he says ball 4? It cant be 100% correctly called out. The outline of the strike zone and outline of the ball. Itll have instances every game its indistinguishable. Is the ball touching that outline? Is there the smallest of millimeter of space between the outlines?

[hurricanecrush]

Id imagine guys like Crawford would give a big fu to MLB if this were implemented, still calling the strike zone as they wanted. Hearing a ball in ear piece, and calling a strike, to fit "their" zone.

In the video, it said that the call was flashed on the scoreboard as well, so everyone in the stands would know what the correct call was.

 

There are a lot of concerns. Like the above mention of home team adjustments. Say opening the strike zone 1/2inch and narrowing 1/2inch. Or 1/4. What of the man in the room blurting what the zone dictates? But doesnt call a strike or ball properly to the ump. 2-2s ball just outside but as close to box line and says strike 3. But in truth its a ball. Or 3-1s ball paints the corner but he says ball 4? It cant be 100% correctly called out. The outline of the strike zone and outline of the ball. Itll have instances every game its indistinguishable. Is the ball touching that outline? Is there the smallest of millimeter of space between the outlines?

Regarding home team adjustments, have the technical person be an employee of MLB and part of the umpiring crew. More ump jobs may be a way to keep the umpires happy.

 

Regarding the close calls, tennis is able to figure it out. I'm sure they can, or do use, similar technology with the strike zone. Even if it's slightly off, it will be off consistently which is a huge improvement over what they have now.

[MKE81]

I'll guarantee Crawford votes for Trump.

Keep the political garbage in the political forum.

I laughed at his line. Mainly because its true. Its ok to throw some funny remarks around on occasion given it directly applies to a MLB ump and his illogical reasoning and blatant disregard of facts

[owbc]

Tennis-style replays are the way to go. Allow < 10 replays per team per game. If the pitcher or hitter doesn't like the call, the home plate umpire signals for a replay. The signal has to be made immediately or the game continues. No waiting for the dugout. The pitch is immediately replayed on the scoreboard with the call. The whole thing takes 20 seconds at most.

 

The only argument against it might be that it takes away the "art" of catchers framing pitches. Otherwise it should be a no-brainer.

[MrTPlush]

I'll guarantee Crawford votes for Trump.

Keep the political garbage in the political forum.

I laughed at his line. Mainly because its true. Its ok to throw some funny remarks around on occasion given it directly applies to a MLB ump and his illogical reasoning and blatant disregard of facts

 

No seriously...take the political garbage elsewhere.

 

On another note the article is borderline annoying because they found a extreme bias in Crawford to give an opinion. Ruins the entire article in my mind. I'm sure they could have found someone a little more thoughtful, but then again that wouldn't fit their intent to push the robot idea.

[trwi7]

It wasn't an article, it was a video segment. They had Eric Byrnes who was extremely in favor and Crawford who was extremely against. They did an entire game under the system and everyone came out in favor of it.

[Fear The Chorizo]

After watching last night's WS game, I am even more in favor of automated ball strike calls....Hendricks was getting extra inches in each corner while Tomlin was getting squeezed on the outer corner and low. GREAT game that the Indians actually could have had better opportunities for runs had Hendricks actually been forced to throw strikes.

 

Inconsistent zone depending on which team is pitching raises questions on intentional bias trying to sway outcomes...the fact the automated graphics on broadcasts now can confirm one team is getting squeezed and the other is getting breaks should help drive that point home.

[logan82]

Inconsistent zone depending on which team is pitching raises questions on intentional bias trying to sway outcomes...the fact the automated graphics on broadcasts now can confirm one team is getting squeezed and the other is getting breaks should help drive that point home.

One would hope so but we have had this information for a while and still nothing.

[3and2Fastball]

Even a traditionalist like me is in favor of Robot Umps calling balls & strikes after this World Series