Manufactured Runs: What We Really Know About the Shift

Last week, we examined the effects of fielding shifts on fielding metrics. For those who missed out, I’d advise you to go read it, but the short version is that location-based fielding metrics can overstate the importance of fielding shifts to a team’s defense and thus overrate players who are shifted in such an arrangement.

But if the fielding shifts are throwing defensive metrics off, that doesn’t mean that they aren’t contributing to team defense, right? And we are in what some people might term a shifting renaissance. John Dewan of Baseball Info Solutions says:

Recently our focus has been on evaluating The Shift.  And for the first time ever, we’re seeing major league teams immediately adopt a major strategy that is in line with the new analytics.

This is so cool.

In each of the last two years, there have been about 1,900 shifts in MLB.  This year we’re on pace for twice that, about 3,800. …

I will say that the research is not yet 100% conclusive that The Shift is effective.  In our book, The Fielding Bible—Volume III, we showed that about 50 points are knocked off the batting average on grounders and short liners for the most commonly shifted hitters in the last two years.  It’s a good sample size, but it’s not a conclusive sample size.  Baseball Info Solutions is continuing to track this and provide research updates to team clients.

Having said that, I believe the teams that are ramping this up are doing the right thing.  In The Fielding Bible—Volume II, that came out three years ago, we were suggesting more shifts to more hitters.  In our newest book we provide more evidence.  I expect that as we continue to dig deeper, we will continue to uncover more evidence in favor of the Shift Defense.

Major media outlets like the New York Times, Sports Illustrated, and others are getting on the shift bandwagon as well. The common thread through those media articles is info supplied by Baseball Info Solutions. Let’s first discuss that info briefly, and then let’s consider how to proceed when we don’t have it.

Baseball Info Solutions utilizes “video scouts” to watch commercial feeds of baseball games (the same baseball broadcasts you or I get on cable, satellite, Extra Innings or MLB.tv, in other words). In the case of the shift, they describe their data collection thusly (from The Fielding Bible, Volume III, page 45):

At Baseball Info Solutions we code The Shift and the shift. Anytime a team moves their infielders out of the normal alignment we call it a shift. That includes the Ted Williams Shift, also known as simply The Shift, where three infielders are on one side of the second base bag. But it also includes other similar shifts such as these:

[images of shift types]

These we code as a generic shift, but we also code The Shift.

In other words, it’s a binary shift/no-shift designation, with a special designation for the Ted Williams “wishbone” shift. It’s unclear exactly what criteria they use to determine whether something is truly a shift— teams will alter their defensive alignment based upon the number of outs and the bunting tendencies of the batter at the plate, for instance, but presumably that sort of routine defensive positioning does not qualify as a shift of any sort.

Now, for the bad news. Let’s watch a very nice Ian Desmond play, please. I’ll wait. Watch it a few times. It’s pretty good.

Once you’re done marveling at what Ian Desmond does, we should take a second to consider what that clip doesn’t show. It doesn’t show where Desmond set up, simply where he was at when he fielded the ball. It doesn’t show the rest of the infield defense at all until Desmond makes the throw over to first. And that’s on a play where the batter:

• Put the ball into play, and
• Hit it on the ground so that it made it to an infielder.

In other words, that’s about the best you can hope for in determining what the defensive alignment was, and frankly it’s not great—the fielders have quite a bit of opportunity to move around before the camera gets to them. If your official demarcation of a Ted Williams shift is what side of second base the shortstop is positioned on prior to the start of the play, whichever side of second base the ball is hit to is going to determine where the shortstop actually is when the camera is finally pointed at him. And on non-BIP plays, good luck getting any recording of what the defensive alignment was at all. (You may be asking why you would care about shift plays where the ball was never hit into the shift at all. There are questions you might want to answer, like whether a pitcher changes his approach with the shift on and how that affects home run and strikeout rates, where you would need to know this.)

The other thing to note about the claims that the incidents of shifts have increased dramatically this season—it is important to note that there are many times where BIS stringers have recorded dramatic changes in their data from year to year, and very frequently it seems as if measurement error, not a change in the underlying thing being measured, is behind them. Their line drive rates, for instance, have experienced a far more dramatic change over the past several years than either MLB Advanced Media’s recording or BABIP. Their estimation of the number of pitches located in the strike zone has shifted dramatically over time, without a corresponding change in walk or strikeout rates. They’ve observed sizable increases in cutter usage over time that line up neither with what PITCHf/x tells us, nor with common sense (Mariano Rivera was not throwing his cutter less than half of the time in 2004).

To answer your first question, I’ll quit beating this horse when it quits moving around. To answer your second question, while none of this tells us that there isn’t a massive increase in shifts this year, it is a cautionary tale as to why we should be skeptical about the proposition. Extraordinary claims require extraordinary evidence, as they say, and the track record strongly suggests that changes in what BIS’s video scouts record from year to year are not particularly strong evidence about actual changes in what’s going on in baseball.

That said, we are unlikely to do better than BIS at the task, were we to set ourselves to it. If I wanted to repeat the Lawrie video experiment from last week over a larger sample, I would quickly run into the same limitations of working from video that I detailed earlier. There are efforts to track actual fielder position at point of contact, from Matt Thomas’ work with photogrammetry to Sportvision’s work on FIELDf/x. But none of them is being conducted in the league-wide fashion that would enable us to fully analyze the shift, and FIELDf/x is not routinely available to outside analysts at any rate.

So what’s left? We may not be able to readily examine whether the shift is more common, but we can determine whether or not there’s an increase in plays made that could be readily attributed to the shift. First, we need to determine who is a likely candidate for an extreme defensive shift. There seem to be three common traits of frequent shift victims:

• They are left-handed,
• They hit for power, and
• They are slow.

So I determined that a player was a shift candidate if his ISO was more than 1.25 times the league average and his triples per extra-base hit rate was below the league average. This is admittedly a crude definition, but if the breathless claims about the shift are really true, then a crude measure should be able to pick up on the difference.

What I did next was to look at the BABIP (or rather, I looked at a variation on BABIP that counted reaching on an error as a hit) of these shift candidates, as well as the BABIP of players who were not in the shift candidate pool. I subtracted the BABIP of shift candidates from the BABIP of the rest of the league (minus pitchers hitting). This is that difference, year by year, over the so-called “steroid era”:

We do see a decrease in the advantage our shift candidates enjoy in BABIP this year relative to 2011, but on the whole what we see is actually the opposite trend of what we would expect if there was an epidemic of defensive shifts robbing hits from left-handed hitters (although it should be noted that we don’t see much of a trend at all, with an r-squared of only .005). Again, this is a crude method, so this doesn’t tell us that shifts aren’t on the rise or that if shifts are indeed on the rise that they are ineffectual. But it certainly doesn’t corroborate the idea that defensive shifts are having a significant impact relative to past seasons.

Is it possible that the lack of impact isn’t because of a lack of shifts, but because hitters have figure out how to beat the shift? A left-handed hitter facing an extreme defensive shift in fact has a way to beat the shift—laying down a bunt along the third-base line. Could a rise in bunt hits (which presumably do less damage than other kinds of BIP hits) be masking the rise of the shift? Probably not. Looking at bunt hit rates from left-handed hitters, excluding pitchers:

There is a slight uptick this season, but the difference between 2008’s .0049 and 2012’s 0.0071 is a paltry -0.0021, paltry compared to the changes we saw in overall BABIP over that same time period. And again, the overall trend line is down, albeit insignificantly.

The notion that a rise in smart, sabermetrics-minded teams is changing the face of the sport is an alluring narrative—but the premise of sabermetrics is holding onto cold, hard evidence in the face of alluring narratives. If we allow the narrative of the shift to overwhelm the evidence at hand, then we won’t be seeing an example of sabermetrics becoming part of mainstream baseball analysis, but an example of sabermetrics becoming like the sorts of mainstream baseball analysis that it originally rebelled against. There is much more to be learned about defensive shifts, but it should come from a position where people are asking tough questions and seeking evidence that can answer them.
 

Dan Turkenkopf provided research assistance for this article.