Crooked Numbers: The Bull in the Pen

In his discussion of the likely NL MVP race shaping up, Joe Sheehan pointed out that if you look at the most prominent contenders–Derrek Lee, Albert Pujols, and Andruw Jones–Lee distances himself from the field in WARP by a vast margin. Looking at the traditional stats, Lee leads the league in batting average, Jones leads in home runs and RBI, and Pujols is second, third and second in those categories, respectively. It’s easy to see why those three would be the favorites headed into the final month of the season.

Debates about the NL MVP aside, this discussion highlights the fact that much of the time–in this case with Lee and Pujols–mainstream stats and more advanced metrics like WARP agree in their assessment of players, but some of the time–with Jones–they do not. No one’s arguing that Lee isn’t having a great season, but there are many players on whom the two categories of stats disagree. Usually these are players not on the margins–Lee is good, Cristian Guzman is bad–but towards the middle of the pack where middling RBI totals can either reveal a talented player who doesn’t get many RBI opportunities or a terrible player granted too much leeway in the middle of the lineup. Discerning between the two makes for good fantasy players and better front offices, though a diatribe on the faults of RBIs likely won’t win many bar arguments.

In that vein, let’s take a look at a group of players who the mainstream fails to value properly: relievers. For example, let’s look at the current leaders on each team in saves, the main metric dedicated to those who ply their trade in the late innings. But rather than simply looking at saves, let’s see how those relievers perform in two of BP’s metrics: Expected wins added over a replacement level pitcher (WXRL) and Adjusted Runs Prevented (ARP). WXRL shows how well a pitcher did given the context of his performance; striking out the side up by one in the ninth is given greater emphasis than turning the same trick in the fifth. However, because it is context-based, WXRL can be altered by usage patterns as well as ability, so we’ll also include ARP, the number of runs a reliever prevented over a replacement level reliever, adjusted for other factors such as inherited and bequeathed runners.

Both metrics have their uses depending on what you’re trying to measure, but ARP is a better measure of performance regardless of context, so if you’re running a team and want to know how a reliever will do if moved to higher-leverage situations, his ARP can stay the same while his WXRL will likely rise or fall more dramatically. Thus, if we’re looking to find undervalued relievers who could step into big roles next season, ARP is an excellent place to start.

Here’s how baseball’s elite firemen stack up, sorted by their current save totals:

PITCHER              TEAM  SV   WXRL    ARP
Chad Cordero         WAS   43   5.636   23.0
Bob Wickman          CLE   36   3.506   10.2
Joe Nathan           MIN   35   3.434   11.5
Trevor Hoffman       SDN   35   2.974    5.6
Todd Jones           FLO   35   4.657   27.0
Mariano Rivera       NYA   35   3.301   22.5
Brad Lidge           HOU   34   3.342   11.0
Danny Baez           TBA   34   3.819   19.8
Jason Isringhausen   SLN   34   3.104   13.0
Francisco Rodriguez  ANA   34   3.885   10.9
Dustin Hermanson     CHA   33   4.178   17.6
Billy Wagner         PHI   32   3.489   22.0
Francisco Cordero    TEX   31   2.956    9.8
Eddie Guardado       SEA   31   3.789   11.1
B.J. Ryan            BAL   29   2.014   14.3
Derrick Turnbow      MIL   29   4.151   20.2
Braden Looper        NYN   28   0.968    3.4
Jose Mesa            PIT   27  -0.221    2.3
Brian Fuentes        COL   26   5.225   18.0
Miguel Batista       TOR   25   1.411    1.9
Ryan Dempster        CHN   23   3.242   10.4
Tyler Walker         SFN   22  -0.172   -6.0
Yhency Brazoban      LAN   21   0.699  -19.3
Huston Street        OAK   18   3.360   34.2
Mike MacDougal       KCA   17   0.796    7.6
Chris Reitsma        ATL   15   1.071    6.3
Keith Foulke         BOS   15  -0.953   -3.4
Brandon Lyon         ARI   13   1.117   -6.2
Dave Weathers        CIN   11   3.432   14.9
Ugueth Urbina        DET    9   1.777    7.2

Three of these relievers–all likely considered the best reliever on their team since they’re the one being used in the closer role the most often–have negative totals in WXRL: they’ve cost their teams more games than a replacement-level reliever would have. And it’s not just that they’ve been unfortunate in tight situations: four of them have negative ARP totals, most notably Yhency Brazoban who is last in the major leagues in ARP. That’s right: the Dodgers used the worst reliever in baseball as their closer for much of the year.

So which teams are the most egregious offenders of misallocation of their resources? Here are the team leaders in saves, RA (minimum 30 games), WXRL and ARP:

TEAM  SV                RA               WXRL             ARP
ANA   F. Rodriguez      F. Rodriguez     F. Rodriguez     Scot Shields
ARI   Brandon Lyon      Jose Valverde    Lance Cormier    Jose Valverde
ATL   Chris Reitsma     Blaine Boyer     Kyle Farnsworth  John Foster
BAL   B.J. Ryan         B.J. Ryan        B.J. Ryan        Todd Williams
BOS   Keith Foulke      Mike Timlin      Mike Timlin      Mike Timlin
CHA   D. Hermanson      D. Hermanson     D. Hermanson     Cliff Politte
CHN   Ryan Dempster     Will Ohman       Ryan Dempster    Ryan Dempster
CIN   Dave Weathers     Matt Belisle     Dave Weathers    Dave Weathers
CLE   Bob Wickman       Arthur Rhodes    Bob Wickman      David Riske
COL   Brian Fuentes     Brian Fuentes    Brian Fuentes    Brian Fuentes
DET   Ugueth Urbina     Kyle Farnsworth  Kyle Farnsworth  Kyle Farnsworth
FLO   Todd Jones        Todd Jones       Todd Jones       Todd Jones
HOU   Brad Lidge        Dan Wheeler      Dan Wheeler      Dan Wheeler
KCA   Mike MacDougal    Andrew Sisco     Mike MacDougal   Andrew Sisco
LAN   Yhency Brazoban   Duaner Sanchez   Duaner Sanchez   Duaner Sanchez
MIL   Derrick Turnbow   Derrick Turnbow  Derrick Turnbow  Derrick Turnbow
MIN   Joe Nathan        Jesse Crain      Jesse Crain      Jesse Crain
NYA   Mariano Rivera    Mariano Rivera   Mariano Rivera   Mariano Rivera
NYN   Braden Looper     R. Hernandez     R. Hernandez     R. Hernandez
OAK   Huston Street     Huston Street    Huston Street    Huston Street
PHI   Billy Wagner      Billy Wagner     Billy Wagner     Billy Wagner
PIT   Jose Mesa         Salomon Torres   Salomon Torres   John Grabow
SDN   Trevor Hoffman    Scott Linebrink  Trevor Hoffman   Scott Linebrink
SEA   Eddie Guardado    Eddie Guardado   Eddie Guardado   Julio Mateo
SFN   Tyler Walker      Scott Eyre       Scott Eyre       Scott Eyre
SLN   J. Isringhausen   J. Isringhausen  J. Isringhausen  Al Reyes
TBA   Danny Baez        Danny Baez       Danny Baez       Danny Baez
TEX   F. Cordero        F. Cordero       F. Cordero       John Wasdin
TOR   Miguel Batista    Pete Walker      Jason Frasor     Justin Speier
WAS   Chad Cordero      Hector Carrasco  Chad Cordero     Chad Cordero

Before we dive into some notes about the above lists, let’s clarify some of the save situation debate quickly. By stating that teams are misallocating resources by not using their best relievers as their closer, we’re assuming that teams view their closer as their preeminent reliever. With some organizations this is not always the case, but with the majority of the league, it is. (When criticizing letting the save rule dictate decisions, we’re not saying that pitching the ninth inning isn’t more important than pitching the seventh, but rather keeping your best reliever seated in the bullpen in a tight situation in the seventh so that he can protect a three-run lead in the ninth isn’t the best way to deploy your resources.) Furthermore, those relievers who find themselves leading their team in WXRL or ARP are likely in line for a closer position in the near future, but not necessarily with their current team. Abusing those differences between performance as measured by WXRL/ARP and saves or ERA–a particularly poor stat when measuring relievers because of the inherited and bequeathed runner scoring rules–is a way for teams to pluck undervalued talent from their competitors.

Now, those notes:

• Only seven teams are led in all four categories by the same player: Rockies (Brian Fuentes), Marlins (Todd Jones), Brewers (Derrick Turnbow), Yankees (Mariano Rivera), Athletics (Huston Street), Phillies (Billy Wagner), and Devil Rays (Danny Baez).
• Kyle Farnsworth leads two different teams in WXRL: the Braves and Tigers. In fact, Farnsworth still leads the Tigers in every reliever category except saves despite being shipped to Atlanta on July 31. In a little over a month, he’s already contributed more wins than any other Brave reliever.
• Eleven teams are led in saves by a player who fails to lead the team in RA, WXRL or ARP: Diamondbacks (Brandon Lyon), Braves (Chris Reitsma), Red Sox (Keith Foulke), Tigers (Ugueth Urbina), Astros (Brad Lidge), Dodgers (Brazoban), Twins (Joe Nathan), Mets (Braden Looper), Pirates (Jose Mesa), Giants (Tyler Walker) and Blue Jays (Miguel Batista). Of those, six teams (excluding Detroit) have a player who leads the team in RA, WXRL and ARP and yet is not the saves leader: the Mets, Boston, Houston, Los Angeles, Minnesota and San Francisco. Roberto Hernandez, Mike Timlin, Dan Wheeler, Duaner Sanchez, Jesse Crain and Scott Eyre are having great seasons for their squads and are still in the shadows of other players. Obviously there are extenuating circumstances in many of those situations–Lidge and Nathan are very good pitchers even if they’re trailing their teammates in certain stats–but the performances of those five relievers may easily be overlooked.
• Yes, Brian Fuentes and Derrick Turnbow are two of the best relievers in baseball this year.
• Cliff Politte has been the third-best reliever in baseball in ARP, behind only Street and Jones, and he leads teammate Dustin Hermanson by 8.0 runs in ARP. However, Politte trails Hermanson by more than a win in WXRL because Hermanson has one of the highest Leverage indicators in the league. Assuming Politte can maintain his performance in higher leverage situations, his WXRL would likely rise towards the top of the league.
• Only the Braves and Blue Jays are led by four different players in the four categories. As the NFL has demonstrated, the line between mediocrity and balance is a fine one.

• Among closers leading their teams in saves, those with the lowest leverage–meaning those closers used in the least tense situations–are Dave Weathers, Foulke, Street, Ryan Dempster and Lyon. It would be tempting to say that those players are overvalued by their save totals, but Street leads the league by a huge margin in ARP–likely a result of being behind Octavio Dotel to start the year and the A’s desire to ease their rookie into the big leagues. The highest leverage: Francisco Rodriguez, Urbina, Bob Wickman, Nathan, and Trevor Hoffman.

When it comes to bullpens, identifying those pitchers who are performing well is the first step, but identifying which are over- or under-valued in the market is the next. By comparing mainstream stats like saves to more descriptive stats like WXRL and ARP, it’s easy to highlight those relievers whose performance is vastly different than the public perception. And if you know that your data is better than that of your competitors, it’s easy to snag a few bullpen gems out from under their noses.