So far this winter we’ve seen a few teams acquiring players despite not having openings at those positions; we’ve seen this particularly with pitchers. The first major instances of acquiring surplus talent were the Red Sox’s acquisition of Josh Beckett and the Toronto Blue Jays’ signing of Beckett’s former teammate, A.J. Burnett. At the time they traded for Beckett, the Red Sox already had a full five man rotation under contract for 2006 (Curt Schilling, Matt Clement, David Wells, Tim Wakefield, and Bronson Arroyo) to say nothing of the presence of Jon Papelbon and several other throwers in the high minors. Likewise, the Blue Jays were at the time sitting on five starting pitchers (Roy Halladay, Ted Lilly, Gustavo Chacin, Josh Towers, and the newly traded David Bush) along with several other options (Scott Downs and a multitude of arms in the high-minors or bullpen including Brandon League, Dustin McGowan, Zach Jackson, and Chad Gaudin).
When judging the value of these acquisitions, it’s very easy to fall back on metrics like VORP or WARP to estimate how much a player will add to a team in terms of runs or wins. Then, by comparing those totals to estimated economic incentives for making the playoffs and the increased revenue of more wins, an estimate of the player’s economic value can be made as well. For example, those of you who are subscribers to Sports Illustrated may have noticed that Daniel G. Habib called upon PECOTA to estimate the value of the players signed to some of the more recent contracts this winter. Though it was not published, PECOTA’s estimate for Burnett was $8.3 million per year for a five year deal. By that number, the Blue Jays appear to have overpaid by $2.7 million per season.
However, the problems with relying on numbers like these to value players are two-fold. First, those numbers do not consider some of the more recent valuation analysis that accounts for the varying value of victories and the odds of reaching the playoffs. As highlighted by Nate Silver, the same player is going to be worth vastly different amounts to teams in different markets and stages of competition. For example, if the Royals determine that Burnett is a five win player and enter the bidding, their return on those five extra wins–going from 56 to 61–isn’t anywhere close to the Blue Jays’ increase in expected revenue both because they’ve increased from 80 to 85 wins and because they’ve helped their chances to secure a playoff spot.
The second reason is the one I want to delve into today: the application of replacement level. It’s hard to overstate the value of the work done by Keith Woolner and others to establish a numerical value for replacement level and to develop metrics that value a player’s contribution over that level rather than league average or zero. But the theoretical replacement level involved in metrics like VORP and WARP is just that: theoretical. It’s extremely handy when comparing players across teams or leagues, but when it comes to specific player acquisitions, it’s possible to better analyze the situation by using the actual options available to the team than the theoretical threshold, at least in the short term.
Getting back to the Red Sox for a minute, let’s take a simple example and say that they simply remove Arroyo from the rotation and replace him with Beckett. Arroyo posted SNLVARs of 4.0 and 3.0 in the last two seasons, respectively while Beckett was good for 3.8 and 5.3. Now, rather than acquiring an extra 5.3 wins–Beckett’s value over replacement–the Sox have only added 2.3, assuming both players repeat their performances from 2005. If Beckett can stay healthy and pitch more than his 178.2 innings from last year, perhaps the Sox might net a little more in the deal. But rather than netting Josh Beckett, the Red Sox have effectively netted Brett Tomko or Woody Williams.
Likewise, in Toronto, Burnett has been worth 3.5 and 4.6 wins over replacement the last two seasons. However, the Jays’ worst starter last year was Lilly, who posted a 1.7 SNLVAR in 126.3 innings (after a 5.6 in 2004). Likewise, Bush posted a 2.6 last year in 134.0 innings, meaning the Jays have only increased their team total by about 3.1 wins, again assuming a consistent performance from all parties. In the bullpen, things aren’t quite as bad. B.J. Ryan notched 3.4 WXRL last year to Miguel Batista‘s 0.9 or Pete Walker‘s 0.5. Once again, the improvement to the team isn’t above the theoretical replacement level, but rather a group of players slightly above our baseline. As such, basing valuations of these players is overvaluing their likely contributions to the team going forward.
Of course, the Jays largely solved their rotation overload with the trade for Lyle Overbay, a player worth 33.3 VORP last year, a total that would have led the Jays. Here again, Overbay doesn’t fill a position of need for the Jays at the moment. He would largely be replacing the production of Eric Hinske or Shea Hillenbrand, players who totaled 17.1 and 32.5 VORP last year, respectively. Assuming that Overbay eventually replaces Hinske, and disregarding a slight discrepancy in playing time, the Jays have gained 16.2 runs rather than 33.3. Add that up and swap out Bush and Hinske for Burnett and Overbay, the Jays have gained themselves only about 4-5 wins while blocking Hinske and shipping out one of their better pitching prospects.
Admittedly, viewing these decisions solely based on 2005 performances and disregarding injuries leaves several other factors to be considered, but the application of a team-specific replacement level to decision-making can change the value of players as much as a team’s situation with regards to the competition and the number of games they’re likely to win with and without the player.
At the bottom of this piece, there are three charts. The first is the average VORP of all bench players by team last year, an attempt to estimate each team’s replacement level. Note that most teams are in the positive in this regard, not necessarily because the theoretical replacement level is too low, but because platoon players and players like backup catchers–who get significant playing time–are included. But regardless of the crudeness of this estimate, it’s still easy to see that the same player with the same production is going to be significantly more valuable to the Yankees than the Brewers–to the tune of nine runs or nearly one win–assuming they are both looking at him to fill a starting role.
The next two charts are the estimations for replacement level in both the rotation and the bullpen. The sample groups for these charts are all players who ranked sixth or lower in GS for the rotation or seventh or lower in IP (max 3 GS) for the bullpen. A few quick notes on these two charts: 1) Nearly every team requires at least 10 starts from a pitcher outside its top five, so the value of having quality pitchers past the rotation is key. 2) Most teams are going to require at least 100 innings from bullpen pitchers outside their top six, or more than any individual reliever. 3) The Cubs–a team that has so aggressively spent money to acquire setup men for its bullpen–actually had the highest estimated replacement level in the pen last year. This finding indicates that of all 30 major league teams, the Cubs are the team that should value additional relief help the least because their marginal returns are the lowest.
It should be obvious that the charts below fail to compensate for player movement both during and since the 2005 season. More importantly, they can largely overlook teams that may have a number of capable backups who combine to contribute respectable value. The ability to combine equal value into a single roster spot is both overlooked and highlighted by this analysis: the charts below will overvalue teams who have a strong bench top to bottom, but they also emphasize the fact that if a team can sign one player to replace two, the newly available roster spot can likely be filled by a player who is above theoretical replacement level, thereby adding value to the team above and beyond the actual value contributed by the newly acquired superstar. Furthermore, a team that is able to raise its replacement level will find less value in new players than those who neglect their bottom line. This can be both a blessing (the ability to let a team’s free agents walk because their value would be less to their current team than they will demand on the open market) and a curse (a capable bench may encourage teams to underbid on free agents, and while the bench may provide more bang for the buck, the only playoffs they’ll reach will be of the Triple-A variety).
On a micro level, this kind of analysis can be very enlightening, but by no means does it indicate a need to reevaluate the league-wide replacement-level thresholds currently in use. Those levels of performance established for replacement level are accurate on a macro level and across seasons and are still very valuable when comparing players. But when determining a player’s value to a specific team, comparing the player to the available options can reveal what opposing teams will be willing to spend. Of course, that assumes that teams are all valuing player performance accurately and similarly which, if it were true, wouldn’t leave the rest of us very much to write about.
Batter Replacement Level
Year Team Batters PA VORP VORP/Batter
2005 MIL 9 1603 59.9 6.7
2005 CIN 12 1618 60.8 5.1
2005 NYN 11 1566 49.6 4.5
2005 CHN 8 1226 34.1 4.3
2005 PHI 7 1086 29.8 4.3
2005 SDN 14 1999 47.9 3.4
2005 ATL 9 1564 28.8 3.2
2005 PIT 16 1820 48.7 3.0
2005 LAN 14 2200 40.5 2.9
2005 TBA 12 1393 33.3 2.8
2005 SLN 9 1617 24.9 2.8
2005 SFN 13 1415 35.4 2.7
2005 WAS 17 1604 36.6 2.2
2005 COL 17 2269 36.1 2.1
2005 TOR 10 1398 19.8 2.0
2005 DET 13 2119 24.0 1.8
2005 BOS 17 1183 29.6 1.7
2005 ANA 14 1644 22.9 1.6
2005 MIN 13 1960 17.8 1.4
2005 FLO 9 681 8.7 1.0
2005 ARI 7 1150 6.3 0.9
2005 OAK 11 1486 8.3 0.8
2005 HOU 9 1138 0.0 0.0
2005 TEX 11 984 -0.2 0.0
2005 BAL 17 1655 -1.6 -0.1
2005 CHA 11 1033 -8.6 -0.8
2005 KCA 15 1571 -12.7 -0.8
2005 CLE 12 859 -12.6 -1.1
2005 SEA 16 1327 -20.1 -1.3
2005 NYA 14 1003 -32.6 -2.3
Rotation Replacement Level
Year Team Pitchers GS SNLVAR
2005 PIT 4 26 5.9
2005 NYN 3 27 5.1
2005 ATL 4 31 4.6
2005 NYA 8 38 4.6
2005 COL 6 44 3.9
2005 SEA 5 23 3.8
2005 WAS 10 32 3.3
2005 MIL 3 26 3.2
2005 SFN 7 31 3.1
2005 ARI 3 27 2.7
2005 FLO 5 26 2.4
2005 MIN 3 15 1.8
2005 SDN 6 35 1.8
2005 CHN 6 31 1.6
2005 TEX 9 57 1.6
2005 CHA 1 10 1.5
2005 TOR 4 27 1.2
2005 ANA 4 13 1.0
2005 TBA 5 29 0.8
2005 PHI 2 9 0.8
2005 KCA 6 33 0.6
2005 BOS 5 19 0.5
2005 LAN 5 26 0.5
2005 CLE 1 4 0.4
2005 SLN 2 2 0.4
2005 BAL 3 19 0.3
2005 OAK 3 14 0.3
2005 HOU 2 16 0.0
2005 DET 5 18 -0.4
2005 CIN 4 22 -1.0
Bullpen Replacement Level
Year Team Pitchers IP WXRL
2005 CHN 13 197.3 4.6
2005 ATL 15 164.3 2.8
2005 PIT 7 82.3 2.7
2005 WAS 12 177.0 2.3
2005 ARI 14 158.7 2.1
2005 CLE 6 91.7 1.7
2005 LAN 10 149.0 1.5
2005 DET 11 163.3 1.5
2005 TEX 19 272.0 1.4
2005 NYN 11 150.0 1.4
2005 ANA 6 67.7 1.3
2005 BAL 9 152.0 1.3
2005 SEA 10 94.0 1.2
2005 TOR 8 118.3 0.9
2005 SFN 14 226.0 0.9
2005 TBA 11 174.0 0.6
2005 CIN 11 209.3 0.4
2005 OAK 10 101.0 0.3
2005 CHA 7 62.0 0.2
2005 SLN 10 73.7 -0.2
2005 SDN 14 163.0 -0.2
2005 KCA 10 190.7 -0.2
2005 HOU 8 62.0 -0.3
2005 MIN 5 24.3 -0.3
2005 MIL 10 154.0 -0.4
2005 FLO 17 189.0 -0.9
2005 BOS 16 144.7 -0.9
2005 PHI 8 83.3 -1.5
2005 NYA 15 128.0 -1.8
2005 COL 16 202.7 -3.4
Thank you for reading
This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.
Subscribe now