Framing and Blocking Pitches: A Regressed, Probabilistic Model: A New Method for Measuring Catcher Defense

[T]he expected runs produced from each plate appearance starting with a strike decreases by .029 runs and increases by .040 for every ball thrown on a first pitch. In other words, having as many of those 0-0 'striballs' called strikes can greatly impact the outcome of the game.

—Jerry Weinstein, Catching Coach, Colorado Rockies
The Complete Handbook of Coaching Catchers
Coaches Choice, 2014

Introduction
The mechanics of framing pitches are simple enough to explain: Quiet, mechanically sound catchers with a knack for good receiving help their pitchers by getting favorable calls from the home plate umpire. This effect has been known ever since umpires started calling balls and strikes. Although it wasn't always called framing, it has long been a source of speculation and commentary about prominent catchers.

Since the beginning of the PITCHf/x era, researchers have calculated framing in several different ways. We are presenting a new method that we will call the "Regressed Probabilistic Model" of framing (RPM for short). In brief, RPM works by calculating the combined probability (and associated run value) that each pitch will be called a strike; summing those probabilities (and run values) across opportunities; attributing those values to a player (catcher or pitcher); and regressing "career" values to the mean.

We will freely admit: If you haven't seen the results of previous framing studies, it can be tough to wrap your mind around the size of the impact of a good or bad framing catcher. These effect sizes are not out of line with what has been reported in the past, but they're still obscenely large. Everyone agrees that Mike Trout was either a deserving MVP or a deserving runner-up in each of the past two seasons, which the stats say were worth close to 10 wins apiece. Our data suggest that over the past five years, the teams that have employed good framers like Jonathan Lucroy, Brian McCann, and Jose Molina have received essentially "free" MVP-caliber seasons from framing alone. (Each of those catchers has been worth about two extra wins per season over that span). This is a staggering amount of value. Add in the fact that these wins are almost assuredly not properly priced into the free agent market, and the difference between having a good framing catcher or a bad framing catcher can make or break a cost-conscious team.

Method
Calculating Probability for Each Pitch

Rather than identifying a single strike zone and giving binary credit for each pitch relative to that strike zone's borders (i.e., strike or no strike), our model gives partial credit for each pitch based on that pitch's likelihood of being called a ball or a strike. To determine that, we created a probability map of likely calls.

To create this map from the raw data, we used a generalized additive model (using the mgcv package in R), which creates a smoothed "surface." Although there are other alternatives for creating smoothed surfaces (Dave Allen popularized the LOESS method), Brian Mills, Carson Sievert, and others have recently adopted the GAM alternative, which has the benefit of empirically identifying the correct smoothing parameter rather than setting it by hand (as in LOESS). The package also has a special function (BAM) specifically for running large models. And crucially, it supports multi-core processing, without which the processing would have taken so long that we'd be writing this article next year.

To reflect what is best known about the way the size and position of the strike zone shifts from count to count and batter to batter, we ran individual models for each set of batter and pitcher handedness as well as "pitch group" (see Table 1). The smoothing parameters of each model were allowed to vary by count, so that while the general shape of the strike zone derived for each variable combination did not change, the width and height of it did (reflecting, for example, a larger strike zone on 3-0 counts than on 1-2 or 0-2 counts). We also accounted for the changing size of the strike zone from season to season (although these yearly changes are much smaller than the other changes we measured).

Table 1: Pitch Groups

We also corrected the data in several ways before running these models. First, all pitch classifications were hand-labeled by Pitch Info to eliminate variability in pitch labels. (This is the same improved dataset that powers the BrooksBaseball.net player card pages). To account for batter height differences, we normalized the height of each pitch by the batter's height using what is now the standard formula (first published by Mike Fast). We also used the correction scheme that Mike published at BP for correcting the X and Y location of each pitch based on the likely distribution of pitch locations that each pitcher would use against left-handed hitters and right-handed hitters (the one difference here was using the LOESS smoothing algorithm rather than a moving average, which we tuned to more aggressively correct for outliers).

After we created these probability maps, we assigned strike probabilities for each pitch at each half-inch location.

Run values

Rather than simply give a single credit for each pitch (~.14 runs) as has been done in many previous models, we looked at the count in which each pitch was framed and gave credit equal to the difference in runs between framing or not framing that pitch. For example, a frame in an 0-2 count was counted as more valuable than a frame in an 0-0 count, because a frame in an 0-2 count can result in a large change in run expectancy while a frame in an 0-0 count does not have quite the same impact.

To be clear, both the positive and negative frames were calculated with that increased difference in an 0-2 count (or in any other count, depending on the run value for that count). We should note that this decision may be somewhat controversial because it's possible that counts will be unequally distributed based on the catcher's team pitching talent. However, see the following section on pitcher adjustments and note that we provide uncalculated calls above average, which should allow the interested reader to create unbiased estimates if they wish.

The run value for a framed pitch is the run value differential for that count (see Table 2) multiplied by the residual of the probability—in other words, if an 0-0 pitch is called a strike in a spot where it's normally called a strike just 80 percent of the time, the catcher will get 20 percent of the available value (.08) for a total of .0004 runs credited (which will later be adjusted based on the pitcher and umpire impact). Failing to get a strike on the same pitch would result in a .0016 run deduction.

Table 2: Framing Run Value Matrix

As you can see, a framed pitch on a 3-2 take is worth a lot. How is that number derived? Think of it this way—a strikeout costs the batting team -0.28 expected runs, while a walk earns them 0.31 expected runs. The difference is .59 runs. These 3-2 takes don't happen often (0-0 pitches contribute the most, which is not surprising), and when they do, the catcher just gets credit proportionally as described above. RPM isn't doling out half-runs in a single shot with any kind of regularity

Player Attribution

Because catching necessarily involves pitching, and because pitching talent is not equally distributed across the league, it can be difficult to correctly assign credit for each catcher's contribution to a framing total. For example, if Mariano Rivera, Brian Wilson, or Derek Lowe is your batterymate, you are likely to get more favorable calls than if your batterymate is Andrew Miller, Brandon League, or Micah Owings.

We empirically determined each pitcher's value—to isolate it from each catcher's value—by performing a WOWY ("With or Without You") analysis. We note that we also compared these values to a linear regression model that included pitcher and catcher as separate factors; the high correlation between these measures suggested a good degree of ability to correctly assign credit (or blame) to individual players. The WOWY adjustments provide a viable and modular means of assessing the impact of pitchers on framing.

The adjustments derived from the WOWY analysis reflect two aspects of our approach. First, pitchers who throw a pitch that may not fit the norm for a given pitch group may show some difference in the WOWY results (such as hard cutters in the slider/cutter group). Second, pitchers with better command of a pitch than their peers (or the unqualified respect of the umpire) will seem easier to frame.

The WOWY analysis created adjustments ranging from +/- .1 called strikes per opportunity and from +/- .01 runs per opportunity. The largest gross beneficiary of easy-to-frame pitchers was—Yadier Molina. The perennial gold glove winner started the analysis with 127 runs added before giving 60 back to his pitchers. This reflects the command contributions of teammates of the class of Chris Carpenter and Adam Wainwright and is no knock on Molina, who still ranks high overall.

Table 3a: Largest and Smallest Pitcher Impact: Total Framing Runs Added 2008-2013

Table 3b: Largest and Smallest Pitcher Impact: Framing Runs Added per 2000 opportunities 2008-2013 (min. 500)

The difficulty of framing a particular pitch—the difference between a fastball and a knuckleball—is already accounted for in the probabilistic model. R.A. Dickey's catchers may earn an adjustment above and beyond the credit already given to them for handling a knuckleball if Dickey is harder to catch than his peer group. Dickey actually outperforms the model by a bit, so his catchers get a small deduction.

According to the RPM method, Tom Glavine was a wizard at getting extra strikes, which supports his reputation.

Umpire Adjustments

We also made systematic but small changes to the data based on the umpire who was calling each game. Because umpires are randomly distributed throughout the data, they tend to have a very small effect on a measure of framing, although they might seem to have a large effect within any individual game. For example, if a particularly generous umpire calls a Jose Molina game on Monday, and then a particularly conservative umpire calls a Jose Molina game on Thursday, although the umpire will have exerted effects within an individual game, Jose Molina's skill will come through in the aggregate.

From our earlier example of a (not) randomly selected Molina, Yadier lost just three runs to the umpire adjustment from his post-pitcher WOWY adjusted tally.

Regression to the Mean

Like other skills, catching involves not only some amount of talent, but also some amount of luck. We've dealt with some of that luck by attempting to correctly attribute runs to catchers (who don't ordinarily get to choose their batterymates), but there are also other sources of luck and inexplicable variability.

To control for this luck, we have regressed career totals to the league average. The amount that we regressed each catcher was based on a measurement of stability for both framing calls and framing runs determined by the intraclass correlation ("ICC") of each measurement. See the "Results" section below for a description of how these correlations were computed and the determination of stability. Because seasonal variability is different from career variability, we also regressed seasonal totals to career totals based on a similar formula.

ICC consistency and agreement both showed that a 50/50 point (where a player's regressed data would consist of 50 percent his own and 50 percent of the mean values) occurred after ~290 framing opportunities (a pitch that isn't swung at and has a called strike probability >0 in our model) for the number of called strikes, and ~430 opportunities for the associated run values. Even the busiest catchers in our sample were regressed to at least .06 percent and .09 percent of the mean for their called strikes added and run values, respectively.

Results
Framing Runs

The big winners in total framing runs are the good receivers who provide enough offense, combined with durability, to pile up innings behind the plate. But you'll also find the likes of Jose Molina, a catcher who wasn't a perennial no. 1 guy until he met a team that properly valued what his glove could do.

Table 4a: Top 10 RPM Framing Runs Earned 2008-2013

On the trailing end you have players who had to contribute in other ways behind the dish and at the plate to accrue any value.

Table 4b: Bottom 10 RPM Framing Runs Earned 2008-2013

Slicing runs into a rate stat, we can see some less-used catchers who stand out, even with their numbers regressed more the everyday guys. Seven thousand opportunities is roughly a full season's workload (some catchers handle over 8000, but those are unusual seasons), so we've set that as our standard for comparison. Jose Molina distinguishes himself further in this view, and Yasmani Grandal also appears among the elite.

Table 5a: Top 10 RPM Framing Runs Earned per 7000 Opportunities 2008-2013

And then there's the list of the worst-rated receivers on a rate basis, which includes more than a few Mariners.

Table 5b: Bottom 10 RPM Framing Runs Earned per 7000 Opportunities 2008-2013

This table of yearly total framing runs shows that Lucroy and McCann have combined for five of the six titles. Again, Mr. Grandal makes an appearance.

Table 6a: Yearly Leaders and Trailers, RPM Total Framing Runs

Table 6b: Yearly Leaders and Trailers, RPM Framing Runs per 7000 chances

In 2011, Jonathan Lucroy contributed a net difference of 39 framing runs to his team compared to Carlos Santana. That's somewhere in the area of four wins if we simply assume that "average" is a sufficient proxy for "replacement level" when it comes to this skill (a true analysis of that question waits for another day). If we include blocking, the difference grows.

With the combined tools of RPM framing and blocking (see sidebar), we begin to get a more complete picture of a catcher's value. Carlos Santana has an elite bat, one that's worth getting into the lineup. But beware the hidden costs: factor Santana's receiving and blocking into the equation, and Lucroy looks like the more valuable player, despite his somewhat weaker bat.

Table 7: Lucroy vs. Santana With and Without RPM (Framing + Blocking)

Santana's value is not found behind the plate. He is an extreme case where even his loud offensive skills are nearly washed out by his receiving deficiency. Reducing Santana's time behind the dish (see 2013 and his conversion to third base this offseason) brings him back into the value range you're used to seeing next to his name.

New Site Features

You can find all of this new framing and blocking information in a couple place on the Baseball Prospectus site. Since our information is based on PITCHf/x, you'll find it only back to 2008, when the system was installed in all parks.

For any catcher who has played since 2008, you'll find a new tab on his player card called "Catching."

Under this new "Catching" section you'll find numbers for framing, blocking, and a combined value for both.

More detailed numbers can be found in a new section in our Sortable Statistics called "Advanced Catching Metrics." Mouse over the "Statistics" tab on the navbar at the top of any BP page, then click on "Sortable Statistics."

Then select the proper report. The advanced catching stats in the sortables and on the player cards will be updated daily during the 2014 season.

You can also view the new catcher landing pages at brooksbaseball.net—there's rising receiving star Yasmani Grandal.