Printer-friendly

December 22, 2005

Crooked Numbers

Value Over Replacement Column

by James Click

So far this winter we’ve seen a few teams acquiring players despite not having openings at those positions; we've seen this particularly with pitchers. The first major instances of acquiring surplus talent were the Red Sox’s acquisition of Josh Beckett and the Toronto Blue Jays' signing of Beckett’s former teammate, A.J. Burnett. At the time they traded for Beckett, the Red Sox already had a full five man rotation under contract for 2006 (Curt Schilling, Matt Clement, David Wells, Tim Wakefield, and Bronson Arroyo) to say nothing of the presence of Jon Papelbon and several other throwers in the high minors. Likewise, the Blue Jays were at the time sitting on five starting pitchers (Roy Halladay, Ted Lilly, Gustavo Chacin, Josh Towers, and the newly traded David Bush) along with several other options (Scott Downs and a multitude of arms in the high-minors or bullpen including Brandon League, Dustin McGowan, Zach Jackson, and Chad Gaudin).

When judging the value of these acquisitions, it’s very easy to fall back on metrics like VORP or WARP to estimate how much a player will add to a team in terms of runs or wins. Then, by comparing those totals to estimated economic incentives for making the playoffs and the increased revenue of more wins, an estimate of the player’s economic value can be made as well. For example, those of you who are subscribers to Sports Illustrated may have noticed that Daniel G. Habib called upon PECOTA to estimate the value of the players signed to some of the more recent contracts this winter. Though it was not published, PECOTA’s estimate for Burnett was $8.3 million per year for a five year deal. By that number, the Blue Jays appear to have overpaid by $2.7 million per season.

However, the problems with relying on numbers like these to value players are two-fold. First, those numbers do not consider some of the more recent valuation analysis that accounts for the varying value of victories and the odds of reaching the playoffs. As highlighted by Nate Silver, the same player is going to be worth vastly different amounts to teams in different markets and stages of competition. For example, if the Royals determine that Burnett is a five win player and enter the bidding, their return on those five extra wins--going from 56 to 61--isn’t anywhere close to the Blue Jays’ increase in expected revenue both because they’ve increased from 80 to 85 wins and because they’ve helped their chances to secure a playoff spot.

The second reason is the one I want to delve into today: the application of replacement level. It’s hard to overstate the value of the work done by Keith Woolner and others to establish a numerical value for replacement level and to develop metrics that value a player’s contribution over that level rather than league average or zero. But the theoretical replacement level involved in metrics like VORP and WARP is just that: theoretical. It’s extremely handy when comparing players across teams or leagues, but when it comes to specific player acquisitions, it’s possible to better analyze the situation by using the actual options available to the team than the theoretical threshold, at least in the short term.

Getting back to the Red Sox for a minute, let’s take a simple example and say that they simply remove Arroyo from the rotation and replace him with Beckett. Arroyo posted SNLVARs of 4.0 and 3.0 in the last two seasons, respectively while Beckett was good for 3.8 and 5.3. Now, rather than acquiring an extra 5.3 wins--Beckett’s value over replacement--the Sox have only added 2.3, assuming both players repeat their performances from 2005. If Beckett can stay healthy and pitch more than his 178.2 innings from last year, perhaps the Sox might net a little more in the deal. But rather than netting Josh Beckett, the Red Sox have effectively netted Brett Tomko or Woody Williams.

Likewise, in Toronto, Burnett has been worth 3.5 and 4.6 wins over replacement the last two seasons. However, the Jays’ worst starter last year was Lilly, who posted a 1.7 SNLVAR in 126.3 innings (after a 5.6 in 2004). Likewise, Bush posted a 2.6 last year in 134.0 innings, meaning the Jays have only increased their team total by about 3.1 wins, again assuming a consistent performance from all parties. In the bullpen, things aren’t quite as bad. B.J. Ryan notched 3.4 WXRL last year to Miguel Batista’s 0.9 or Pete Walker’s 0.5. Once again, the improvement to the team isn’t above the theoretical replacement level, but rather a group of players slightly above our baseline. As such, basing valuations of these players is overvaluing their likely contributions to the team going forward.