The folks at Fangraphs have developed, and are advocating the use of, stats that seem to me to represent a major improvement, in evaluation accuracy, over the “save” and the “blown save” as methods for ranking relief pitcher performance.   The Fangraphs concept of the “Shutdown”, as a replacement for the save, is very simple in concept if you  understand the idea of “Win Probability Added” (WPA), which itself has been much discussed over the years at the baseball-reference blog that spun off into High Heat Stats.

WPA measures the degree to which the outcome of each plate appearance changes the standard probability of the hitter’s or pitcher’s team winning the game, compared to where that probability stood immediately before the plate appearance.  The resulting change in probability, positive or negative, is then assigned to the pitcher and hitter who participated in that plate appearance.  For example, a  PA with the bases loaded, one out, tie score in the late innings that results in a double play and no runs scored will mean a very large dollop of positive WPA for the pitcher and a big WPA deduction for the hitter. The results of this calculation for every plate appearances in a game are then  added up, with each pitcher and hitter in the game receving a WPA number for the game that is the sum of all his plate appearances WPAs.  Both b-ref and fangraphs calculate WPAs for every pitcher and hitter for every game (b-ref’s WPAs go back to 1948).

A “Shutdown” for a relief pitcher is defined at Fangraphs as any game by a relief pitcher in which his WPA for the game is plus 6% (+.06) or more.  This standard has a number of advantages over the traditional save statistic.  First, it eliminates credit for the very easiest saves.  For example, in the second game of the 2012 season, over in Japan, Grant Balfour picked up a save coming  in for the A’s at the start of the bottom of ninth inning with Oakland holding a three-run lead.  Teams win about 97% of games that they lead at the start of the bottom of the ninth, so Balfour closing out that game contributed only about 3% in Win Probability Added.  So although Balfour gets a save, he does not get a Shutdown for that game becasue his WPA was less than the 6% stnadard for a Shutdown.  Only a save that had a meaningful amount of risk to it qualifies as a Shutdown.  In addition, any sort of relief appearance, no matter when it happens in the game, can qualify as a Shutdown, if the reliever pitches successfully under enough pressure.  Indeed, multiple Shutdowns can be awarded in a single game if multiple relief pitchers meet the Shutdown standard.  In Atlanta’s win over Houston on Tuesday (April 10), Eric O’Flaherty, Jonny Venters and Craig Kimbrel all earned Shutdowns for Atlanta, having contributed similarly to Atlanta’s chances of winning, while only Kimbrel gets the save.

In short, the Shutdown gives credit whenever, but only when, a reliever actually pitches successfully in a reasonably high-risk situation.  

In 2011, the top five pitchers in the majors in saves were:
Jose Valverde 49
Craig Kimbrel and John Axford 46
J.J. Putz 45
Mariano Rivera 44

In 2011, the top five pitchers in Shutdowns (using b-ref’s Play Index):
Jonny Venters 45
John Axford 44
Heath Bell 43
Craig Kimbrel and Tyler Clippard 41

Jose Valverde, the major league saves leader in 2011, had 38 b-ref Shutdowns, good for eighth in the majors. Two Yankees, Mariano Rviera and David Robertson, tied for 10th in the majors in Shutdowns. 

Most Career Saves, All-Time:
Mariano Rivera 604
Trevor Hoffman 601
Lee Smith 478
John Franco 424
Billy Wagner 422

Most Career Shutdowns, All-Time:
Mariano Rivera 539
Trevor Hoffman 530
Lee Smith and John Franco 474
Rollie Fingers 434

Fangraphs and Baseball-reference use slightly different formulas to calculate WPA, so the Shutdown numbers using the b-ref Play Index will be a bit different than those that Fangraphs lists. But the concept is the same, and in most cases the results will be comparable.

Just as there is a negative version of the save in the form of the blown save, Shutdowns have a negative flip side that Fangraphs calls Meltdowns. I’ll discuss those in a separate post.