Pitching Backward: Best. Reliever Season. Ever.

Relievers are endlessly fascinating to me, which is good because our fearless leader also has some sort of affinity for them as well. In the past two months I’ve talked about Andrew Miller’s emergence as a relief ace, discussed how awesome Jeremy Affeldt secretly is, critiqued the bullpen management of a few managers, and shown why critiquing them isn’t really fair after all. We can add this post to my growing list of bullpen reflection pieces, though this one has a much larger focus than any of the previous pieces.

The question posed by, and hopefully answered in, this post is which relief pitcher had the best single season in recent history. The first hurdle to all of this is figuring out the best way to measure reliever performance. In the past I’ve hung my hat on either WPA (especially for closers) or RE24, because they capture the unique responsibilities of relief pitchers better than something like ERA or WARP.

WPA is telling, though partially flawed, because to a large degree it includes a component of trust. That is to say, WPA weights high-leverage events more than low-leverage events. For this reason WPA gives a sense of how well a pitcher performed, but also to some degree how comfortable their manager was using them. WPA is important, but it might cause us to miss a standout performance simply because the pitcher’s manager didn’t use them in high-leverage situations very often, or until too late.

For a better context-neutral guide to reliever performance we can examine RE24. This stat simply gives a pitcher credit for the run expectancy of the base states they encounter upon entering and leaving a game. Simplified, this means that a pitcher who enters an inning with two runners on base and one out will have a positive RE24 if he gets an inning-ending double play. It also means that if that same pitcher walked the first batter he saw before being pulled, he’d get a negative RE24 score for that game. It doesn’t matter if the appearance came in the fifth inning or the ninth, as context doesn’t matter with RE24.

The Data
I’ve restricted the timeline for this study to 1988-2014. The reason for this is that it mostly encapsulates the most recent eras of baseball (the steroid era and the modern era) and provides a decent sample size of reliever seasons that readers might remember. It’s totally arbitrary, and this study could easily be re-run with a larger set of years if desired.

From 1988 through 2014 there are 1,002 relief pitchers who threw 3,495 qualifying relief seasons. Nearly all of these relievers threw at least 49 innings in the seasons included for the study, though shortened seasons (like 1994’s strike-shortened season) will have some qualifying relievers with fewer innings thrown. On average, the 3,495 seasons had the relievers in question throwing 66.2 innings while posting a 3.59 ERA.

WPA and WPA/G
Win Probability Added helps tell us exactly how much each pitcher added to his club’s likelihood to win over the course of a season. Our list of 1,002 pitchers posted an average WPA of 0.56 with a standard deviation of 1.42. The best season by WPA since 1988 belongs to Keith Foulke, who posted a 6.62 WPA during his 2000 season with the Chicago White Sox. For reference, that’s 4.3 standard deviations better than the mean. The only problem with Foulke’s 2000 season is that he pitched in 72 games and threw 88 innings. He’s a compiler. Sure, throwing a lot of innings is valuable, but we’re looking for sheer dominance here. Not really good-ness over a lot of innings.

For that reason we’ll be taking these counting stats (WPA and RE24) and dividing them by the number of innings each reliever threw that season. This essentially gives us an idea of how effective each pitcher was each time he took the mound. Those who threw a lot of innings no longer have such an advantage, and the impact that the managers had on the seasons for each of these pitchers in lessened. Since the pitcher can’t control how many games or innings he throws, it’s unfair to penalize those who were utilized less frequently despite being elite. After all, managers probably can’t be expected to make perfect decisions in these cases.

Dividing WPA by the number of innings thrown by each pitcher (WPA/IP) gives us a new leaderboard. The mean WPA/IP for this sample is 0.01 and the standard deviation is 0.02. More importantly, a new leader has emerged: Jose Mesa’s 1995 season. Mesa averaged a 0.09 WPA in each inning he pitched during the 1995 season, which is almost exactly four standard deviations above the mean. Here’s a pseudo-relevant video of Jose Mesa pitching in 1995 that includes an Indians squad of Sandy Alomar Jr., Will Clark, Kenny Lofton, and Manny Ramirez against sad Ken Griffey Jr. and the Mariners:

Mesa’s 1995 at the top of the WPA leaderboard probably shouldn’t surprise anyone. He threw 64 excellent innings, notched 46 saves, and posted a 1.13 ERA. His peripherals might suggest he wasn’t actually that good, but for this one season the stars aligned and Mesa was superb. The rest of the top 10 looks like this:

For what it’s worth, Keith Foulke’s 2000 season was the 12^th best once we took into account the number of opportunities he had to accumulate WPA, with 0.075 WPA/IP. Below is the distribution for WPA/IP, which has a handful of interesting takeaways:

That dot at the bottom is 2009 Brad Lidge. Yay Phillies.

Generally speaking, all of the pitchers fall on the continuum, with just a few elite and terrible pitchers beginning to have any meaningful separation at the ends. The group as a whole skews positive, which we’d expect with the mean being 0.01. This is likely a result of the 49 inning minimum to qualify for the study. Managers are unlikely to stick with a truly horrible reliever over 50+ innings, except in rare cases.

RE24
The top pitcher for RE24 is Mariano Rivera and his 1996 season. Rivera might have had better seasons by ERA or peripheral stats, but 1996 sticks out when it comes to RE24. He posted a 40.48 RE24 season over 107.2 IP while helping lead the Yankees to yet another World Series title. Unfortunately for Rivera, his high innings total meant he’s left out of the leaderboard when we average out his RE24 over all of innings. A new leader rises to the top.

That leader is B.J. Ryan, 2006. When we divide RE24 by the number of innings each reliever pitched, we can see that the average reliever posted an RE24 of 0.07 for every inning he threw. The standard deviation for the sample is 0.14, which means that the 0.49 RE24/IP that Ryan posted in 2006 is almost exactly three standard deviations better than the mean. Ryan’s 2006 campaign was, much like Mesa’s 1995 effort, a very good one even by traditional measures. He posted a 1.37 ERA over 72.1 IP and solidified himself as one of baseball’s elite relievers in his first season in Toronto.

The rest of the top 10 looks like this:

Note that the runner up to Ryan was also in 2006: Jonathan Papelbon’s electric first full season in the Red Sox bullpen. The pitchers on this list include many of those who distanced themselves from the pack when looking at the distribution, which is included below:

That dot at the bottom is, shockingly, not Brad Lidge. It’s 1998 John Davis of the Chicago White Sox

RE24/IP has a similar distribution to WPA/IP with nearly all of the pitchers falling on the main band. Once again there are some stragglers and some elite pitchers, but the majority fall in line where you’d expect them too. There’s also a positive skew here, which is to be expected for the same reason WPA/IP was positive. Mariano Rivera’s excellent 1996 season finished 24^th on the leaderboard; his elite RE24 score is somewhat mitigated by the sheer number of innings he threw that season.

There you have it. Those are the top 10 relief pitcher seasons by either WPA/IP or RE24/IP since 1988. There are a lot of interesting points that can be pulled out of this data, like which team-seasons were the best/worst, but those will have to be saved for another day. Personally, I feel a little strange saying “These are the best reliever seasons of the last 25 years” while not having Mariano Rivera on either leaderboard, but the numbers are what they are. One explanation for Rivera’s omission is that there are many more factors that go into valuing relief pitcher performance than just WPA or RE24, but that too is a conversation for another day. For now, let’s just marvel at the impressive ability (or inability) of the gentlemen highlighted above.