“Quiz”: Guess that WPA!

This isn’t really a quiz, just a wry chuckle about the limitations of Win Probability Added (WPA) for measuring a player’s game impact.

In Thursday’s 5-2 win, Miguel Cabrera went 4 for 5 with a HR and a double. His only out was a line drive; he was never picked off or caught stealing; and he made no errors.

So what was his WPA?

-0.018.

Yes, that’s a minus sign. The WPA method says that Cabrera’s total contributions reduced Detroit’s chance of winning by about 2%.

Almost all of Cabrera’s ABs were discounted, in a WPA sense, through no fault of his own:

— His double in the 1st came with 2 out and none on, so it barely moved the WPA needle — about a 2% positive impact on Detroit’s chance of winning.

— With 2 on, no outs and no score in the 3rd, his liner was turned into a brilliant leaping catch and DP, for a -13% impact. WPA doesn’t see Austin Jackson breaking too aggressively from 2nd base with no outs; it just sees the team’s probability of winning before and after the entire play.

— In the 4th, Cabrera singled with 2 on and 1 out, but Jackson ran into another out — this time at the plate, on a great throw and (probably phantom) tag — making the hit a negative event for Cabrera, -2%. WPA doesn’t care that Jackson pulled a Jeremy Giambi, standing when he should have slid, nor that on-deck man Prince Fielder was standing directly behind the umpire while gesturing for Jackson to get down, nor even that the ump called an out that he could not possibly have seen. (Watch every angle as often as you like, you’ll never know for sure if the tag was made — but in any case, it was physically impossible for the home plate ump to have seen the alleged tag on Jackson’s backside.) WPA’s motto is: “You hit it; you bought it.”

— In the 6th, Cabrera finally broke into the RBI column, singling home Jackson from 3rd (safe at last!) with 2 out. But they already led by 2 runs, so WPA rates it a modest 8%. (The way things were going, it’s a miracle the ball didn’t hit the pitching slab and land in Jeff Keppinger’s glove.)

— And lastly, his HR in the 9th restored a 3-run lead, but WPA saw that as just a 4% boost in win probability, or 1/3 as much absolute value as the line-drive DP.

This was the 282nd regulation games this century with 4 hits in 5 PAs, 1 HR and 1 double (all figures exact). Cabrera rated the worst WPA, and only one other was in the red. The other 280 games got higher WPA than Cabrera, even though:

  • 91% of those hitters’ teams scored more than the 5 runs Detroit had; and
  • 69% of the 230 wins were by larger margins than Detroit’s.

None of this is meant as a criticism of Win Probability Added. Every measurement has shortcomings; every record of an event contains some noise. We know that WPA simplifies things by ascribing all the credit or blame for every change of situation to just one offensive player and one defensive player. The negative assessment of Cabrera’s game was a fluke confluence of events, a perfect storm of sinister circumstances. It doesn’t mean WPA is a bad stat.

But if you want to introduce someone to the WPA concept, do yourself a favor: Keep them away from games like this for as long as you can.

 

0 0 votes
Article Rating
Subscribe
Notify of
guest

41 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Yippeeyappee
Yippeeyappee
12 years ago

I guess it’s a great stat, but I’m pretty good at math and, if I can’t figure it out in my head, I’m not interested…

Lawrence Azrin
Lawrence Azrin
12 years ago
Reply to  Yippeeyappee

Discuss:

More useless stat: Game-Winning RBI or WPA?

vincent
vincent
12 years ago

Good column. Horrible stat.

Richard Chester
Richard Chester
12 years ago

John, I liked that article. I have almost always been doubtful about WPA. Particularly annoying is the event of game 6 of the 1986 WS between the Mets and the Red Sox. Mookie Wilson got credit for a WPA of 0.400 because of somebody else’s error, giving the false impression of a game-winning clutch hit.

Doug
Doug
12 years ago

Yes Richard, but Mookie somehow avoided being hit on a pitch right at him and two feet off the plate. His agility allowed the winning run to move into scoring position, before Mookie delivered the “slow roller heard round the world”. 🙂

nightfly
12 years ago

If, in 282 instances such as this, we have only two instances of a negative WPA, then it’s actually fair to call the stat generally reliable. Not perfect, obviously, but things rarely are.

BryanM
BryanM
12 years ago

Great post . I have been lured out of retirement to applaud your exposé of the complete silliness of WPA . This artifact attempts to measure the likely impact of events by means of counterfactuals., not what happened in the game that actually happened , but what might have happened on average had the same events been embedded in a large number of hypothetical games that weren’t played. The gaping hole in the number is of course that it makes no sense to talk about probabilities without at the same time talking about confidence in the probability distribution. It would… Read more »

BryanM
BryanM
12 years ago
Reply to  John Autin

John , I hope I am not bashing anything; perhaps my language was intemperate , and no, I recognize that all stats have their limitations, and that we have to make mental reservations about batting average ( insert expletive Al Oliver expletive comment) and the others; my issue with WPA is more technical as expressed in @31

MikeD
MikeD
12 years ago

I’ve been mostly agnostic on WPA, mostly because I haven’t spent enough time reviewing it to have an opinion either way. Unfortunately, after reading this, I’m now against it! 🙂

DavidJ
DavidJ
12 years ago

I agree with JA’s next-to-last paragraph and nightfly #5. Virtually every stat (whether “traditional” or “advanced”) has the limitation of simplistically attributing outcomes to just one or two players, so you will always be able to find quirky, fluky examples like this where the stat looks absurdly misleading. Any stat can look horrible in a one at-bat or one game sample. It’s also worth noting that WPA doesn’t necessarily ask you to believe that Miguel Cabrera had a bad game–it simply points out to you that Miguel Cabrera’s at-bats didn’t end up having a huge impact on the outcome of… Read more »

Jim Bouldin
Jim Bouldin
12 years ago
Reply to  DavidJ

Agree fully. There’s noise in any quantitative system. You don’t throw out the regression line of p < .05 model just because not every point falls on the line. Indeed, you expect very few points to actually fall on the line.

BryanM
BryanM
12 years ago
Reply to  DavidJ

Sorry, but i think this misses the point. The value of summarizing complex events in a single number is that enables one to intuitively grasp something that would elude us if we didn’t have the number. While it is true that all statistics are estimates, and thus subject to variation, most others are not as badly constructed as this one; it is the flawed design of piling counterfactuals on top of each other that leads to absurdity. In this case, WPA does not ask us to think that Cabrera had a bad game , but since the number is negative… Read more »

Jim Bouldin
Jim Bouldin
12 years ago
Reply to  BryanM

With due respect, I think you are missing the larger point, one that John recognizes. The fact that, as you say, this Cabrera number is a “flukey” result is exactly the point; it’s flukey–it doesn’t explain anything, doesn’t represent the larger value of the statistic in explaining player contributions to wins. It’s like taking the one point that lands furthest from a linear regression line and saying “Aha, faulty relationship”. Not the case. You can’t judge the validity of any metric in explaining some other metric, based on a single value. The single number doesn’t “ask us to believe” (or… Read more »

BryanM
BryanM
12 years ago
Reply to  John Autin

John; My use of the word efforts was of course sloppy; I should have said “outcomes”, I did not mean to imply that we were somehow trying to measure how hard he tried. I think I understand , basically , the premise of WPA, and , when used to measure individual events , I can see that it conveys information; striking out with 2 out and the bases loaded is more negative for your team than striking out with 2 out and the bases empty. Because events in a baseball game are not independent of one another, adding them together… Read more »

Lawrence Azrin
Lawrence Azrin
12 years ago
Reply to  DavidJ

Great point; you can take _any_ single stat, either mainstream or advanced, and come up with counteruntuitive results: – Walter Dropo (144) had more RBI’s in a year than Willie Mays (141) or Frank Thomas (143) – Al Wingo (.370) with 493 career MLB games played, had a higher season BA than Wade Boggs (.368). -Ron Bryant (24) with 57 career wins, had more wins in a season than Warren Spahn (23 twice) of 363 career wins -Here are several players that had a higher/same seasonal WAR than Roberto Clemente (8.7): – Rico Petrocelli, 9.5 – Terry Turner,9.2 – Lenny… Read more »

BryanM
BryanM
12 years ago
Reply to  Lawrence Azrin

I think we are talking about 2 different things . in your examples you have great players with great stats, and somewhat lesser players with “surprisingly” better numbers – but 8.7 seasonal WAR still says that Clemente was pretty good — and there is no dispute about the numbers – with WPA we have something that attempts to estimate counterfactuals, treats interdependent events in the same game as if they were independent and can lead to results that are not just counterintuitive , but absurd. The events in which Cabrera participated, in my opinion, did not, when taken as a… Read more »

Lawrence Azrin
Lawrence Azrin
12 years ago
Reply to  BryanM

BryanM –

I understand your point; I guess that I was “comparing apples and oranges” by contrasting the single-game stat of WPA with a player’s seasonal stats.

What I was trying to say was that using any ONE stat ONLY, to evaluate a particular player, will sometimes produce results that don’t make a lot of sense.

This would be similar to a casual fan who compares pitchers _only_ by Wins, or batters _only_ by BA.

BryanM
BryanM
12 years ago
Reply to  Lawrence Azrin

Agreed — we need to look at a lot of different numbers to truly evaluate a player , and every one has its limitations, some are better than others

Lawrence Azrin
Lawrence Azrin
12 years ago
Reply to  Lawrence Azrin

Actually, I think I’m more disappointed that no one has gotten my “Willie Horton” reference in #27 (yet).

BryanM
BryanM
12 years ago
Reply to  Lawrence Azrin

“ignoring” is not the same as “Not Getting” wasn’t it Olivia Dukakis, anyway?

BryanM
BryanM
12 years ago

JA. Of the 282 times in your study , can you determine how many times the player’s team won? If it was say. 60 percent. Then we could say with reasonable statistical validity that Cabrera’s feat added a win probability of about. 10 percent , all thing being equal…..

Richard Chester
Richard Chester
12 years ago
Reply to  BryanM

I tried to duplicate John’s PI runs. My results were that from 2001-2012 there were 276 games that matched those stats, of which there were 221 winners. From 2000-2012 there were 303 such games of which 242 were winners. I don’t know why I did not come up with John’s number of 282 games.

no statistician but
no statistician but
12 years ago

BryanM: Your protests reminded me of something. Summary courtesy of Wikipedia: A vain Emperor who cares for nothing but his appearance and attire hires two tailors who are really swindlers that promise him the finest, best suit of clothes from a fabric invisible to anyone who is unfit for his position or “just hopelessly stupid”. The Emperor cannot see the cloth himself, but pretends that he can for fear of appearing unfit for his position; his ministers do the same. When the swindlers report that the suit is finished, they mime dressing him and the Emperor then marches in procession… Read more »

Nash Bruce
Nash Bruce
12 years ago

You know, I could never understand how the Emperor could not _feel_ that he was not wearing any clothes…..?

DavidJ
DavidJ
12 years ago

I think the major limitation in WPA that this example shows is one that can very easily be tweaked, and that’s that it gives Cabrera the full penalty for the two double plays. While I think it’s reasonable to give the hitter the full penalty for a GIDP, for double plays where a runner gets doubled off or gets thrown out trying to advance, I think it makes sense to give that portion of the negative WPA to the baserunner. So, in Cabrera’s third-inning at-bat, charge him with the game-state change from runner-on-second/no-outs to runner-on-second/one-out. Then charge Jackson with the… Read more »

BryanM
BryanM
12 years ago
Reply to  DavidJ

DavidJ, I think your last sentence captures the situation well – WPA as it is constructed is mostly just trivia, and as long as we don’t expect it to be more we can have fun with it. I accept the limitations of RBI , but think it is vastly more useful than WPA as presently constructed – as presently constructed, for example , it is easy to find examples of WPA for both the winning team and the losing team to be positive (Cle-Bal june 29) someone hits a 3 run HR in the top of the first — this… Read more »

BryanM
BryanM
12 years ago

Apologies to everyone. I just reread JA’s original post . He says that 230 of the 282 instances of Cabrera’s feat were wins — 81.5% , so going 4 for 5 with a double and homer seems to add about 31% to the team’s probability of winning.

brp
brp
12 years ago

Obligatory Art Shamsky reference in a WPA-related thread.

Richard Chester
Richard Chester
12 years ago
Reply to  John Autin

Details of that play can be found in the Charlton Chronology on baseballlibrary.com. That play ended up with three runners on third base leading to a famous joke about the Dodgers.

Person A; Hey, the Dodgers have three men on base.
Person B: Oh really, which base?

Lawrence Azrin
Lawrence Azrin
12 years ago

It was really Dazzy Vance’s fault; he should have _ easily_ scored from second, on the long hit to the outfield by Herman, which was at least a double,or maybe a triple(he DID end on third base…). C’mon, how can you not score from second on a double?

Ed
Ed
12 years ago
Reply to  John Autin

Well since the winning run scored on that play (in the bottom of the 7th) Herman’s WPA would probably be pretty good.

Ed
Ed
12 years ago
Reply to  Ed

Odd joke seeing as how the team was called the Robins and didn’t become the Dodgers till 1932, six years after Herman’s gaffe.

Ed
Ed
12 years ago
Reply to  John Autin

Interesting, never knew that! Course, it is a few years before my time… 🙂

Richard Chester
Richard Chester
12 years ago
Reply to  John Autin

John: I tried my hand at calculating that WPA. On May 9th birtelcom provided a link to the WPA calculator. It applies to 1957-2005 but it’s all I had to go with. Herman came to bat in the bottom of the 7th with the bases loaded and one out with the scored tied at one. At that time the Robins’ WPA was 0.773. Then when the Braves came to bat in the 8th with no one on and no outs and behind in the score 2-1 their WPA was 0.227. That meant that the Robins’ WPA at that time was… Read more »

Lawrence Azrin
Lawrence Azrin
12 years ago

#37/ I was referring ironically to the role a “Willie Horton” (NOT the ballplayer) played in the 1988 presidential campaign. And I don’t know how “Olivia” (do you mean Olympia, Mike’s actress {“Moonstruck”} cousin?) plays in this.

BryanM
BryanM
12 years ago
Reply to  Lawrence Azrin

I meant Olympia , and Yes, I got your reference, I guess mine was a weak attempt at suggesting how forgotten the Dukakis name is — didn’t work