Breaking Balls: Optimal Bullpen Usage, Continued

I wrote a piece in the Baseball Prospectus Basics last week (“How to Run a Bullpen.”) I got a lot of feedback that ran like this:

Hey, that table you ran shows that it’s good to generally use the best reliever in tighter situations, rather than to protect three-run leads in the ninth…Facing a tie game in the eighth, wouldn’t it make sense that a manager would save his best pitcher for the ninth, which would be even more important?

This is a fine question, and one I think deserves to be answered in some depth. Here’s that chart again, for ease-of-reference:

    Home                                 Away
1) Top 9th, lead by 1 (.170)     1)  Bottom 9th, lead by 1 (.223)
2) Top 9th, tied      (.160)     2)  Bottom 8th, lead by 1 (.158)
3) Top 8th, lead by 1 (.123)     3)  Bottom 9th, tied      (.155)
4) Top 8th, tied      (.115)     4)  Bottom 8th, tied      (.122)
5) Top 7th, lead by 1 (.096)     5)  Bottom 9th, lead by 2 (.113)
6) Top 7th, tied      (.092)     6)  Bottom 7th, lead by 1 (.111)
7) Top 9th, lead by 2 (.080)     7)  Bottom 8th, lead by 2 (.108)

There’s no simple answer to the question of whether it’s better to pitch early or later. If you’re the home team, faced with a tie game in the seventh, there’s a possibility that if you can hold the other team down but are unable to score yourself, you’ll have to fill in three innings: the seventh tied, the eighth tied, and the ninth tied. And each inning grows in leverage, so if you had to order your bullpen of one-inning-only relievers according to that chart, you’d want the best to pitch the ninth, the second-best to pitch the eighth, and so on.

The problem with that logic is twofold. First, you don’t know and can’t know in advance what the score will be in future innings, and suddenly we’re into the kind of prediction theory that’s better suited to meteorology and the advanced combinational work of game theorists. A tie game in the seventh is a high-leverage inning, no matter what happens later.

Baseball’s not a game of vague theory, though. No team features nine league-average hitters, and that’s what makes baseball interesting. Let’s take this tied-in-the-seventh situation and look at it in detail from the home team’s dugout. The manager should know:

• Who is due up for the visiting team in the top of the seventh, eighth, and ninth innings
• Who is available in his bullpen to pitch that night
• Who is due up for his team in those innings
• Who is probably available in the visiting bullpen to pitch that night
• About how likely his team is to score in any given inning, given the last two pieces of information

The advantages we see in the chart are significant, but small compared to the normal variation in a baseball lineup. In our example, say the game’s been pretty normal up to that point: it’s 3-3 after six, and both teams have had six hits–one a home run–and two walks. That’s 26 plate appearances, so the team will see…

In the seventh: #9, #1, #2, maybe #3
In the eighth: #3 and then definitely #4, #5, maybe #6 and #7
In the ninth: the bottom of the lineup again, which could mean pinch-hitters (who, generally speaking, don’t hit all that well, but may also mean platoon starters, etc).

Here’s last year’s AL chart of hitting by lineup position

Lineup Position  AVG     OBP     SLG
1               .275    .330    .405
2               .278    .338    .424
3               .282    .361    .484
4               .287    .368    .480
5               .265    .338    .439
6               .258    .323    .420
7               .261    .325    .413
8               .244    .304    .372
9               .244    .295    .350

Meanwhile, for the sake of simplicity, say the home team has an average offense in exactly the same situation: their #9 man will lead off the seventh inning.

Looking it over, you want your best man to pitch the eighth and try and get the big bats out. You throw your second-best guy at the 7th, it’s pick ’em against the bottom of the lineup, and hopefully you’ve scored by then so you don’t go into extra innings.

Except here’s the rub: It’s not really much better to have your best guy pitch to the best hitters and the worst guy to the worst hitters. What you want, ideally, is for the best guy to pitch the two big innings coming up, the 7th and 8th. But there are real differences you can look out for. In a tie game you’d want to play every advantage you can get, trying to get lefty-lefty matchups. Or putting extreme groundballers in against power hitters, even if they’re more likely to give up the single.

And if at any point you score seven, it’s Ramiro Mendoza mop-up time.

The differences to be made in finding the best matchups for a bullpen within a game far outweigh the marginal season-long advantages of having one guy pitch the eighth or the ninth optimally, even given our lovely chart.

I have always argued that within reason, every manager should try and win the game they’re in and, to do that, do whatever they have to do in that inning. As Leo Durocher said, “You don’t save your best pitcher for tomorrow. Tomorrow it might rain.” In the seventh, tied up, facing the heart of the other team’s order, you don’t know whether or not you’ll score one run or seven, but no matter what happens on your side, you’ll be more likely to win if these guys get shut down.

There’s a second interesting question in this discussion. Is the implied statement that teams have one good reliever and then it’s a steep drop true? Or do teams generally have more options? I decided to take a look at the “shape” of major league bullpens. Using Michael Wolverton’s Reliever Run Evaluation report I took each team’s bullpen and stacked them up by value.

So what you see there is the best reliever (the ‘1’ tick) was occupied by relievers ranging from awesome–Eric Gagne–to not-that-hot–Matt Roney, who was the best reliever on the Tigers, worth 3.7 runs above an average reliever. It then trails off pretty quick–the back end of everyone’s bullpen got worse.

This isn’t a perfect picture. I’d have liked to have run something a little more sophisticated that figured out a way to weight by spot, but it gets really complicated, and you know that to be good or bad requires a lot of innings pitched–you can already see that closing up the middle wouldn’t reveal anything new.

What we see is that teams rarely have only one good reliever. There are two obvious things going on: First, as at all positions, scarcity means that great relievers are hard to find, and also that teams generally have bullpens that are strong or weak in the same way their lineups, or starting rotations, can be strong or weak. It’s hard to field a bullpen composed entirely of fine relief aces, but you can still have Gagne, Guillermo Mota and Paul Quantrill, or whoever, and put together a nice unit.

So what did I learn writing this?

Research into the value of closers and bullpen usage shows us that the best places to use your best relievers is in close games, especially games that are tied, or where you have a one-run lead. The difference in quality between the first and the third man out of the pen isn’t as great as is generally perceived, so worrying about saving the best pitcher for the highest-leverage inning in a tight game doesn’t make much sense. For all this, though, how to use your best relievers in a game will almost never be quite as clear as choosing a .080 advantage over a .059 advantage in spotting your second-best reliever in the seventh or eighth, because the game situation will never allow you enough future information to use relievers in a way that will appear optimal in retrospect. And if it did, you should move to Las Vegas immediately, and I’ll spot you the startup capital.