Notice: Trying to get property 'display_name' of non-object in /var/www/html/wp-content/plugins/wordpress-seo/src/generators/schema/article.php on line 52
keyboard_arrow_uptop

I Was Framed
Catchers play a central role in the game of baseball through their involvement with every pitch that their pitchers throw. One of their key tasks is receiving borderline pitches without discouraging the umpire from calling strikes.

Dan Turkenkopf was one of the first analysts to investigate this phenomenon using detailed pitch location data. He found that catchers differed significantly in their performance at getting strike calls from umpires. Matthew Carruth and Bill Letson followed up with additional research and similar conclusions. The size of the effect that Dan and Bill found was so large as to be almost unbelievable.

Earlier this year, I observed that where the catcher set the target relative to the edge of the strike zone and whether the pitcher hit the target had a large impact on the likelihood of a strike call. Some pitchers, such as Tom Glavine and Livan Hernandez, were consistently able to expand the edges of the strike zone by several inches. The fact that catchers are paired with a limited number of pitchers in a season affects our ability to properly assign credit for extra strike calls to catchers. Adjusting for pitcher-catcher pairing reduces the apparent size of the catcher responsibility for the effect to more reasonable levels.

When I reported this observation, I applied a pitcher adjustment to Bill Letson’s catcher receiving numbers from 2008-2009. Now, having added catcher defensive information to my own pitch database, I am in position to report my own measurements of catcher performance at getting extra strikes for their pitchers.

Catcher Performance Results
To calculate the catcher performance, I first established a baseline for each pitcher over the period 2007-2011. I used the strike zone definition described here and counted the number of extra strikes and subtracted the number of extra balls tallied by each pitcher. I also applied a small correction to the pitch location data as described here. I divided the net number of extra strikes by the total number of called pitches for that pitcher to arrive at an expected net extra strike rate for each pitcher. (The term “extra strikes” will be used in this article to refer to the net of extra strikes minus extra balls.)

Next, I applied the same procedure for each pitcher-catcher pair and subtracted the pitcher baseline from the result. Then, I summed the results by catcher. I also calculated an approximate run value for the extra strikes saved or lost by each catcher using Dan Turkenkopf’s finding that switching the call from a ball to a strike on a close pitch was worth about 0.13 runs on average.

Here is how catchers have done over the past five seasons, according to this method, at saving runs for their team by getting extra strike calls at the edge of the zone.

Catcher

Called Pitches

Total Runs

R/120G

2011

2010

2009

2008

2007

Jose Molina

18788

73

35

10

15

15

26

7

Russell Martin

42186

71

15

15

10

20

14

11

Yorvit Torrealba

26306

41

14

5

14

3

11

7

Jonathan Lucroy

14205

38

24

17

21

0

0

0

Yadier Molina

39184

37

8

7

7

1

16

6

Gregg Zaun

17376

36

19

0

6

16

6

9

Miguel Montero

27053

33

11

6

8

17

2

1

Alex Avila

17023

25

13

16

9

0

0

0

Ryan Hanigan

18016

25

12

7

5

8

4

0

Geovany Soto

32705

24

7

14

11

7

-9

1

Jarrod Saltalamacchia

20065

23

10

15

0

1

5

3

Brian McCann

42098

20

4

1

-3

11

9

3

Jeff Mathis

25731

19

7

5

1

4

2

7

David Ross

14315

16

10

0

1

6

6

2

A.J. Pierzynski

40828

12

3

3

3

3

2

1

Chris Snyder

23394

10

4

7

8

5

-1

-8

Humberto Quintero

18991

10

5

-2

6

5

3

-1

Ronny Paulino

20239

10

4

-9

1

7

7

3

Brian Schneider

18685

8

4

2

-3

-2

9

2

Joe Mauer

30260

7

2

9

-7

4

1

0

Matt Wieters

25008

6

2

2

3

1

0

0

Ivan Rodriguez

30558

5

2

1

5

-6

0

5

Kelly Shoppach

21402

3

1

0

-1

-2

5

0

Kurt Suzuki

43225

2

0

-3

5

-9

11

-2

Josh Bard

18275

1

0

-1

5

-3

-2

1

Miguel Olivo

30104

-2

-1

6

0

-3

-1

-3

Victor Martinez

22864

-8

-3

-1

-2

-9

1

2

Bengie Molina

30096

-14

-4

0

-3

1

-8

-4

Rod Barajas

26858

-15

-5

-10

-3

4

-6

0

Jason Varitek

27075

-17

-6

-10

-3

-4

5

-5

Dioner Navarro

24980

-19

-7

-1

1

1

-15

-5

Carlos Ruiz

32761

-23

-6

-13

-2

-6

-5

3

Ramon Hernandez

27917

-23

-7

-8

-9

-5

2

-3

Matt Treanor

16276

-24

-13

-17

-1

1

-5

-2

Mike Napoli

23939

-24

-9

-6

-3

-8

-10

2

John Buck

31314

-25

-7

-11

-17

0

0

3

Koyie Hill

14399

-25

-15

-10

-4

-9

-1

-1

Rob Johnson

14285

-25

-16

-7

-6

-12

0

0

Nick Hundley

20827

-27

-12

-12

-2

-3

-11

0

Chris Iannetta

25308

-30

-10

-11

5

-3

-19

-1

Kenji Johjima

19588

-33

-15

0

0

-9

-9

-15

Jason Kendall

35772

-37

-9

0

-10

-19

-5

-3

Jorge Posada

17942

-50

-25

0

-23

-11

-5

-10

Gerald Laird

30298

-52

-15

-2

-12

-16

-11

-11

Ryan Doumit

22861

-66

-26

-9

-3

-16

-36

-1

Data for all catchers are available here. These numbers are for the observed performance, and they do not incorporate any regression toward the league average in order to estimate the persistent skill level going forward.

We do not distinguish here how catchers may be getting the extra strike calls for their pitchers. It may be that they have superior mechanics that influence the umpire to call more strikes, whether by action or lack of action on the catcher’s part. Or, it may be that they are particularly adept at setting the target for the pitcher in such way that he delivers the ball on a trajectory that is more likely to get a strike call. It may even be that some catchers exert a verbal influence over umpires or develop friendships that sway calls in their favor. Any effect that gains strike calls and is related to the catcher should be captured here.

Do These Numbers Make Sense?
If we divide the catchers into thirds based upon their performance in getting extra strike calls from 2007-2011, we find that better performance translated into increased strikeouts and decreased walks for their pitchers.

Catcher Group

Extra Strikes/Pitch

Delta BB/PA

Delta K/PA

Best Third

+.081

-.004

+.002

Middle Third

-.001

-.004

-.001

Worst Third

-.083

+.002

-.005

The run impact of the walk and strikeout changes for the pitchers is about one third of the total run impact we would expect based upon the changes in ball-strike calls. Since walks and strikeouts comprise about a third of the total run impact at the team pitching level, we can presume that the remainder of the run impact from extra strike calls would be observed if we calculated the run impact of changes in results of balls in play. However, it would be worthwhile to verify that presumption.

In The Hardball Times Baseball Annual 2011, Sean Smith introduced a new method for computing catcher ERA. He looked at multi-year samples of pitcher-catcher pairs from 2003-2009 and computed estimated runs based upon defense independent pitching statistics and batted ball classifications. He found that catcher skill varied by about +/- 15 runs per season between the best and worst catchers, after applying significant regression toward the mean. (He added about three seasons of league-average performance to each catcher’s numbers.) The unregressed observed performance varied by about +/- 40 runs per season.

Sean’s numbers include the catcher’s performance at getting extra strike calls, along with pitch calling and any coaching or interaction between catcher and pitcher that affected the pitcher’s performance against the batter. Catchers that are particularly good at pitch calling or coaching their pitchers might rate better in Sean’s method than in my method here, which is restricted to measuring the effect of extra strikes.

Sean reported results for 20 catchers who also played at least one full season in my sample. His regressed results for those catchers correlate well with my results here, at r=0.59. Some of the differences may plausibly be attributed to pitch calling and coaching skills. For example, Jason Kendall rates poorly in my metric and above average in Sean’s, and Ivan Rodriguez rates about average in my metric and fairly high in Sean’s.

The catchers who are best and worst in getting extra strike calls repeat pretty well from one season to another. We can group the 2008 and 2010 seasons in one pool and the 2009 and 2011 seasons in the other pool and compare the extra strike rates between pools for all 39 catchers who caught at least 6000 called pitches in each pool.

We may not be measuring the extra strike effect perfectly, but it does seem to stand up to basic scrutiny. The correlation for catcher ratings between the two pools is around r=0.7, depending on where exactly the playing time cutoff is set. That means that we should add about half a season’s worth, or 4500 called pitches, of league-average performance to the observed performance for each catcher in order to get a better estimate of the catcher’s actual skill level.

If these numbers accurately represent catcher performance at getting extra strike calls, catchers have the ability to gain or cost their team a win or two over the course of the season in this department. That would make this skill as important as all the other facets of catcher defense that we are currently able to measure (preventing stolen bases, blocking pitches, and fielding balls) put together.

My method controls for the effect of pitcher bias in each catcher’s sample. Max Marchi used a mixed-level regression model to control for the effect of bias from pitchers, umpires, batters, ball-strike count, pitch location, and pitch type. His results show a similar magnitude for the variation in framing performance among catchers and have a good correlation with my results for individual catchers. (Correlation coefficient was r=0.86 between Max’s results and mine for the 80 catcher-seasons for which Max reported results.)

Problems with Catcher Comparisons
One concern I have with my method is the extent to which catchers are compared either to themselves or to the very limited number of catchers who played on the same team. I attempted to minimize this by comparing catchers to pitcher baselines established over a five-year period, in order to encourage pitcher and catcher switching between teams. Nonetheless, some teams have maintained the same starting catcher for the whole period under investigation. If a team has also tended to promote the bulk of their pitchers from within their own farm system, the degree of pitcher and catcher cross-pollination may be limited.

For example, Joe Mauer has been the starting catcher for the Minnesota Twins during all five seasons 2007-2011, and the Twins have primarily developed their pitching staff from within. As a result, Mauer himself has caught 50 percent of the pitches by the pitchers who made up his comparison baseline. The Twins backup catchers during that period make up another 30 percent of the pitcher baseline, led by 11 percent from Drew Butera and nine percent from Mike Redmond. The rest of the catchers in the league make up only 20 percent of the pitcher baseline for Mauer. As a result, this method probably understates Mauer’s performance, and in addition, the performance of Butera and Redmond may have unduly affected his rating. Similarly, Mauer’s performance may unduly affect Butera and Redmond’s ratings.

Jeff Mathis and Mike Napoli were a long-time pairing at catcher for the Angels. Mathis himself caught 36 percent of the pitches by the pitchers who made up his comparison baseline. Napoli caught 24 percent of the pitches in the pitcher baseline for Mathis, and other Angel backup catchers caught another 14 percent. The rest of the catchers in the league made up 25 percent of the pitcher baseline for Mathis. However, as can be seen in the previous table of extra strike results, neither Mathis nor Napoli saw a large change in their performance after Napoli changed teams during the 2010-2011 offseason.

On the other hand, John Buck changed teams twice during the sample period, going from Kansas City in 2007-2009 to Toronto in 2010 and to Florida in 2011. Buck caught 28 percent of the pitches in his own pitcher baseline. The other top ten leading catchers represented in Buck’s pitcher baseline were Olivo (10 percent), Kendall (five percent), Paulino (five percent), John Baker (five percent), Barajas (four percent), J.P. Arencibia (four percent), Brayan Pena (three percent), Jose Molina (three percent), Zaun (two percent), and Treanor (two percent). These catchers represent a good mix of varying performance in getting extra strike calls, and as such probably make for a fairly accurate comparison. Buck’s performance dropped significantly from 2009 to 2010; however, the changes in comparison baseline do not seem to explain the drop. I am unsure whether the change represents a real change in performance for Buck or a measurement artifact of unknown nature.

Between Mauer at one extreme and Buck at the other, catchers are compared to varying sets of other catchers in the league. Max Marchi’s mixed-level modeling is designed to address this concern at least to some extent. Max noted with his method that 14 catchers who switched teams had an extra-strike rate in the new season that correlated at r=0.67 with their previous season’s performance, whereas 53 catchers who stayed on the same team had a season-to-season correlation of r=0.82. I observed a correlation at r=0.61 using my method for the same 14 team-switching catchers.

Catcher Receiving Techniques
Having covered the derivation and accuracy of the measurement, let us turn to the question of what catchers are doing to gain or lose a strike call. Are there mechanical differences in technique between the best and worst catchers?

I selected two of the best catchers—Jose Molina and Jonathan Lucroy—and three of the worst catchers—Ryan Doumit, Jorge Posada, and Jason Varitek—and reviewed video of their borderline pitch calls. I observed two primary differences in technique between the good and bad catchers.

Both Molina and Lucroy displayed stable, quiet mechanics when receiving borderline pitches. Posada was the opposite, often jumping, bouncing, and shifting his stance and lunging and swiping to catch pitches. It was almost painful to watch him behind the plate. Varitek and Doumit had more stable stances, but each displayed a negative behavior that appeared to cost them strike calls. (Posada displayed both of these negative traits, and more.)

It is worth mentioning that one should not compare receiving mechanics on curveballs to mechanics on other pitch types, as was done in this analysis of Lucroy. Catchers typically set up with a different stance on curveballs to prepare to block a possible pitch in the dirt. Curveballs also drop much faster from the front of the plate to the catcher’s glove than other pitch types do. (The numbers reported above include all pitch types, but one potential improvement would be to look at receiving numbers for curveballs separately, though the smaller sample size could be a challenge.)

Let’s compare Varitek to Lucroy. According to my metric, Varitek cost the Red Sox about 16 runs by losing 101 expected strike calls in part time duty over the last two seasons. Lucroy took over the starting job for the Brewers in the middle of the 2010 season and has gained about 41 runs by getting 284 extra strike calls since then.

Lucroy got extra strikes for his pitchers both on the bottom and outside edges of the strike zone. Varitek did not get these same calls for his pitchers. Let’s see how they compared in 2011 on calls to right-handed batters.

I reviewed video of seven pitches caught by Lucroy along the bottom boundary of the zone and six pitches caught by Varitek along the bottom of the zone. Lucroy and Varitek both had stable, quiet stances, but Lucroy’s glove was also very quiet in receiving all seven pitches, whereas Varitek stabbed at the ball with a much bigger arm motion for all six pitches.

Compare Lucroy and Varitek’s glove movements in the following pitch animations. Both pitches are fastballs that crossed the front of home plate about 22 inches above the ground.

 

Lucroy’s glove moved a few inches at most, but Varitek’s glove traveled about two feet down and then back up.

I also reviewed video of 10 pitches caught by Jose Molina along the outside edge of the zone to left-handed batters and 12 pitches caught by Doumit in similar locations. Eight of ten pitches received by Molina were called strikes, and 10 of 12 pitches received by Doumit were called balls. Let’s see how they compared in 2011 on calls to left-handed batters:

Molina got quite a few more strike calls on the outside edge with left-handed batters at the plate than Doumit did.

Compare Molina and Doumit’s movements in the following pitch animations. Despite Doumit’s solid stance and subtle glove movement, he dropped his head to follow the pitch into his glove and he hunched down slightly, as if he were trying to coax the ball carefully into his glove. Molina’s head stayed mostly stable as he received the pitch.

Doumit dropped his head on 11 of the 12 pitches I reviewed on video. On the one pitch where he did not do that, he got a strike call. Molina dropped his head to follow the ball into the glove on two of the 10 pitches I reviewed on video, and both of those pitches were called balls.

Lucroy’s head was stable on all seven pitches I reviewed, and he got seven strike calls. Varitek’s head was stable on all six pitches I reviewed, all called balls, but his exaggerated glove movement may have cost him those strike calls.

I also reviewed five of Posada’s borderline pitches on video. In two cases, his arm and body movements were very exaggerated and potentially distracting to the umpires. In the other three cases, his body and glove were fairly stable (though not perfectly so like Lucroy), but he dropped his head noticeably to follow the ball into the glove. All five pitches were called balls.

Exaggerated glove and body movements are well known to be distracting to umpires. As Brent Mayne wrote in The Art of Catching:

Simply catch the ball firmly. When the pitch and glove meet, that’s where the action should stop. The catcher should have enough strength to stop the momentum of the ball so that strikes don’t turn into balls. Think of a gymnast “sticking” a landing. Just “stick” the ball, hold it for a brief second, then throw it back.

Lucroy and Molina have that technique perfected. Varitek, on the other hand, appears to have lost dozens of strike calls every year because he does not.

I am not aware of any previous finding that the catcher dropping his head to follow a borderline pitch into his glove will cost him a strike call. However, I am convinced that it is an important factor making Doumit one of the worst receiving catchers in the majors. I have rarely, if ever, been able to identify such a striking mechanical difference between major-league players that has correlated so directly to performance differences. For the borderline pitches I reviewed where the catcher had otherwise quiet and stable mechanics, the presence or absence of the catcher head drop predicted the umpire’s strike call 31 of 32 times.

Jeff Zimmerman and Kurt Romeiser reviewed video for strike calls lost by Zack Greinke in 2010 with Kendall and Pena behind the plate. This is the only other extensive review of catcher technique in conjunction with PITCHf/x data of which I am aware. They noted that some of the strikes were lost due to Greinke missing the catcher target or due to glove movement by the catcher. However, some of the calls they attributed simply to a blown call by the umpire. In at least a few of those cases, I note that the catcher dropped his head to follow the pitch into the glove.

I welcome feedback on my findings from catchers, umpires, coaches, or other interested parties. There may be other mechanical factors that I did not observe that influence umpire strike calls. It is very helpful to be able to review the video in conjunction with using PITCHf/x pitch location data to identify borderline calls that went for or against the catcher.

Catchers appear to have a substantial impact on the success of their pitchers through their ability to gain extra strike calls from the umpire. This is an important factor to consider when valuing the contributions of catchers to a team. We have identified at least two specific techniques that affect catcher performance in this arena. Given the important impact on major-league teams, further research along that line is warranted.

Thank you for reading

This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.

Subscribe now
You need to be logged in to comment. Login or Subscribe
AlexHoefer
9/21
This is the amazing stuff that make this site so great.
lucasjthompson
9/21
It seems there's a problem with the fact that the "baseline" for each pitcher is going to be a function of which catchers have caught the pitcher most over their career. Even pitchers with considerable track records will often have a big part of their total innings caught by a single catcher or two. So, e.g., if a catcher is terrible at framing pitches, but he's compared against the career baselines of the pitchers he's caught, he's not going to look so bad because those pitchers' baselines were largely created by him.
mikefast
9/21
Yes, it is a problem, for some catchers moreso than others. I discussed it in the section "Problems with Catcher Comparisons."
SaberTJ
9/21
Good Point
SaberTJ
9/21
Mike - This is fascinating stuff. Can't wait for more like it. One of the best research pieces I've read on here in a while.
mikefast
9/21
Thank you, Alex and TJ.
jjaffe
9/21
Outstanding stuff, Mike. Very interesting discussion & quantification of Posada's shortcomings, among other things. And wow on Martin.
dduncan
9/21
Really interesting read, as usual. Learned a lot, especially with Varitek as I watch him more often than other catchers.
bmmillsy
9/21
Really enjoyed this. I think there is a lot of work to do here both from the perspective of the sabermetrician as well as applications of this in catcher training and practices. As usual, Mike presents stuff that is really pertinent to those both in the front office as well as on the field. Keep them coming.
redsoxin2004
9/21
Great article. I've shared this with two high school coaches that have taken a good deal of interest in it.
philly604
9/21
Great read. One thing that you should address on the video clips is the game context. Varitek certainly looks awful in that one video compared to all of the other catcher videos. But, for example, if that pitch is late in a 8-1 game that would partially explain his lazily stabbing at the ball.

It won't matter in the bigger sample of pitches presented in the tables, but you're such a careful researcher that I think it's a nit pick worth making. Those anecodotal videos make powerful impressions. I think it's worthwhile making sure that they also represent similar game states.
mikefast
9/21
I looked at many more pitches than the four clips I presented here. What I presented here is representative of what I saw across all the pitches I viewed.

Other things like ball-strike count and pitch type can also have an effect on the size of umpire zones, but I looked at a mix of counts, and some mix of pitch types, though mostly fastballs.

The specific pitches shown above are as follows:
Lucroy: May 4, 2001, 6th inning, 2-2 pitch to Alex Gonzalez, Milwaukee trailing Atlanta 4-2
Varitek: July 17, 2011, 12th inning, 1-1 pitch to Ben Zobrist, Boston and Tampa Bay tied 0-0
Molina: May 21, 2011, 8th inning, 1-1 pitch to Brian Bogusevic, Toronto leading Houston 6-4
Doumit: May 21, 2011, 2nd inning, 0-0 pitch to Alex Avila, Pittsburgh and Detroit tied 0-0
davidpom50
9/21
In the context of the article, does it really matter though? Whether it's in a close game or a blow out or when his legs must be exhausted from catching 12 innings, his actions seem to be robbing hi pitcher of called strikes. Yeah, it might make Varitek look worse than he really is, but it still demonstrates the habits that Mike is discussing.
Richie
9/21
Another "Wow!" vote.
delorean
9/21
awe·some Adjective /ˈôsəm/
1. Extremely impressive or daunting; inspiring great admiration, apprehension, or fear.
Timcarvin
9/21
I need to read this 5 more times but WOW
brownsugar
9/21
I'd like to echo the other commenters. Brilliant stuff.

The article actually makes me question whether or not I make the same mental adjustments while watching a game at home. If the catcher's head drops, am I more inclined to think the pitch was a ball? When I grumble about an umpire's strike zone, should I really have been grumbling about my home team's catcher all along?!?

Now I am questioning myself and having an existential crisis. Damn you, Mike Fast! I've seen too much!
mfalcon
9/21
Really impressive article. I haven't read something that wowed me from an analysis standpoint like this article did in a while. Bravo!
EricMeeker
9/21
Great stuff Mike. Now can we let the computers call balls and strikes so we can eliminate all this fudging.
davidpom50
9/21
I can't decide if this article more firmly entrenches me in the "let the robots call the balls and strikes" camp, or changes my mind because now I can see that, instead of close pitches being vagaries of luck, there's a skill set a team can employ to gain an advantage.
EricMeeker
9/21
While I respect the hard work that goes into refining any skill, such as being able to effectively frame pitches, my problem with it is this; the hitters are being cheated by those who are good at this, and the pitchers are being squeezed (at least relatively) by those catchers who are less proficient.

I don't want to detract from any of the outstanding research Mike Fast has done on this topic, but to me there is too much of an unfair element in allowing pitch framing when we have more accurate ways of calling balls & strikes.
jim7white
9/21
Another framing technique that should be investigated: catching the ball in a different part of the glove. For example, assuming a right handed batter, if a catcher can catch a borderline outside pitch in the webbing of his glove, the pitch appears closer to the strikezone. If it's inside, he wants to catch it near the heel of his glove.

The research from this article might explain why Bob Boone stuck around so long.
MichavdB
9/21
Unless you have hands of steel, that's physically impossible. A 95 mile per hour fastball in the palm of your hand (while trying to keep that hand still) may be fine once or twice. Do it 20 times in a game and your hand will swell up like you won't believe. Also, almost every ball these guys throw has a tremendous amount of movement on it. You try to catch the ball in the web every time. Plain and simple.
thegeneral13
9/21
This is true - nobody is catching the ball with their palm on purpose. The difference from one side to the other is just the framing technique, which catchers work on a ton.
jim7white
9/21
It's not impossible. The "sweet spot" on a catcher's mitt is big enough to fit almost two entire baseballs - I'm not talking about catching it on the absolute heel of the glove or at the complete end of the web, but vaiances within the sweet spot of one to three inches. See Brian Downing's quote on Bob Boone: "He (Boone) catches a lot of balls on the edge of his glove so he has his glove looking like a strike all the way. He's not really moving that much. He catches the ball in the web of his glove and makes it look like a better pitch."


http://goo.gl/X6YN4 [shortened url - ed]
jrmayne
9/21
As I read the start of the article, I was concerned with the comparison pool and with the ability to show causation. I find that often - too often - analysts come up with statistical conclusions without looking at possible artifacts.

By the end, I was not concerned. I join the chorus of "Awesome!" that this article richly deserves. Might I suggest that those articles which clearly and substantially move forward analysis on a key point get placed in a special section? Two weeks or four years from now, some interested person might not know how to find this, and this deserves to be found. This reminds me of some of the best long analytical articles from the earlier books.

There's clearly more to go on this issue, and there's been great work by others indicating an effect. But this robust effort seems to me to have significantly moved knowledge forward. And the video clips are fantastic.

Awesome.

Awesome.

Awesome!

(One aside: I think one of the possible statistical artifacts is that umpires call different pitches strikes on different counts; if you had a team with a lot of 3-0 counts, you'll end up with more fake strikes called. However, measuring for that is likely to befoul your sample sizes sufficiently as to make the noise overwhelm the likely very slight effect. But that's just my opinion. I could be wrong.)

--JRM
mikefast
9/21
Thank you. Max Marchi's work addressed the effect of the ball-strike count. I've tried to learn more about the mixed-model regression technique he used, but that's something I don't yet know how to replicate.

I think that as a general rule, the things like batters and umpires, that a catcher is paired with fairly randomly, will tend to wash out with larger samples. Things like pitchers and parks (and the specific PITCHf/x camera system in a park) are tied more closely to the catcher's identity, and we have to take more care in adjusting for them.

Things like ball-strike counts and pitch types probably fall somewhere in between. There shouldn't be a wide variation among catchers in what counts they're in or what pitch types they see, but the small variation that is there could be tied to the team, and thus a persistent bias for the catcher.

I don't know any way to adjust for it simply without killing the sample size, though Max's mixed-model approach does appear to be a potential candidate if you want to adjust for these additional factors.
thegeneral13
9/21
My favorite part of this article is that it bridges the gap between old school (qualitative) and new school (quantitative) analytics, which is desperately needed. For baseball insiders who already know how important framing is, quantification of that value allows it to be compared to the value of other skills (hitting, etc.) in determining a catcher's overall value. For baseball outsiders who don't believe a skill exists unless it has been quantitatively proven to exist, this is hopefully enlightening, as it highlights that though this is a subtle skill, it is real and exceptionally valuable.
hotstatrat
9/21
Excellent work! So, can we conclude that we'll generally call pitch framing accounts for about half the differences in catcher's ERA? That's my rough estimate using your description of Sean Smith's un-regressed numbers and throwing out your outlier Jose Molina.
mikefast
9/21
That's about my feel for the situation, too, but I would have more confidence in the size of the pitch/game-calling portion of catcher ERA if I had a way to measure it directly like we do with catcher framing.
Mooser
9/21
Wow. So if you incorporate this into a WARP like stat for catchers, I guess you have to deduct these runs saved from the pitcher's runs / WARP or you would be double counting the catcher's framing skills on a team level.
bornyank1
9/21
We're already kicking the possibility of that kind of adjustment around internally.
hotstatrat
9/21
Let’s go back to the old Angels catching controversy. If the best catchers save 40 runs a year and the worse add on that many, is it possible Steve Scioscia was correct to have played Mathis at Napoli’s expense all those years? Let’s say Smith’s 80 run difference is based on 120 games per year – the number Mike Fast used for comparing catcher’s runs saved by pitch framing. Let’s say 120 games produce 480 PA. Using career Runs Created as tallied in Baseball-Reference today (Sept. 21, ’11), over 480 PA Jeff Mathis has on average produced 35.25 runs. Over the same number of plate appearances, Mike Napoli has produced an average of 77.75 runs. The difference is just over 40 runs – half of Smith’s un-regressed numbers, but more that his regressed differences of +/- 15 runs (30). Keep in mind that is assuming Mathis is one of the best catchers at saving runs and Napoli is one of the worse. An article written by Sam Miller in the Orange County Register January 23, 2009 directly compared how the Angels starters pitched under Mathis and Napoli. While, Mathis did have a slight edge in K:BB%, but not HR%, if you take out Ervin Santana – who had Mathis as his personal catcher, while the other starters about evenly shared the two catchers – Napoli comes out slightly ahead – defensively!

In June of this year, Miller guested in Baseball Prospectus with an article on the same subject. Like Miller and the 24 BP readers who commented, this is a subject we can’t let go, despite Napoli having left Anaheim last winter. It is a subject worth looking into on any team; the Napoli/Mathis debate is just the most obvious extreme case. Perhaps, we should be crying that the John Farrell should have caught Jose Molina instead of J.P. Arencibia? It seems we only really notice this debate when the defensive catcher plays over the offensive catcher – no doubt because good hitting is more obvious thanks to the stats we see than good defense.

In Miller’s BP article, in an indirect flippant way, he makes the case based on all those preceding studies, that pitch framing may account for just about all the significant differences in catcher defense. Certainly the work that Mike Fast and Dan Turkenkopf did have solid evidence that Mathis did save an average of 16 runs per 120 games over Mike Napoli’s catching. That is still a long way from the 42.5 runs Napoli hit better in 480 PA.

Sorry, I have no further fancy concluding statement.
TheBunk
9/21
Great stuff. In addition to Molina's subtle framing, he has a great rapport with umpires which i'm sure has aided in many borderline calls.
rbrianc
9/21
Since this is a new look at an all too familiar "debate" (actualy Angel fans spitting venom about having Mathis instead of Napoli), I too would be interested in your analysis, Mike, overall, who the Angels would be better off having to use, including how Hank Conger ist starting off and projected to perform offensively.

I can think of no better test case than this team, especially how how Napoli's Rangers look to be taking the AL West this year by a slight margin - did the choice of catching in your view make the difference as most all of Halodom assumes?
mikefast
9/21
Well, there is a difference of 30-40 runs, conservatively, between Mathis and Napoli in batting performance. So that's a pretty big hill to climb with fielding performance.

I have Mathis making up about 16 runs per season over Napoli with framing performance. We need to regress that a bit toward average if we're going to estimate actual skill, but then we inflate again a little bit because Mathis and Napoli are being compared partially against one another.

Jeff Mathis does not grade out particularly well at stopping the running game or blocking pitches, so he doesn't gain much on Napoli there, if anything.

If Mathis gains on pitch calling, I haven't tried (and don't know how) to measure that directly, but Sean Smith's study suggests it wouldn't be enough to make up the rest of the offensive difference between Mathis and Napoli.

If you're really generous about leaning toward Mathis as far as possible in all the areas of uncertainty in our fielding measurements, you can get them close to comparable territory, but I wouldn't necessarily recommend that viewpoint.
Clemente
9/21
Just wanted to add: Great article! I think this is a major effort that will lead to lots more work and discoveries, both analytic and coaching. I bet you see alot of 'frozen heads' in the playoffs.
rawagman
9/21
Brilliant! This will change the way I watch the game. I can't give a more fawning compliment than that. Thank you.
jplcarl
9/21
Tremendous work!

Is there any accounting for variances between individual umpires? Are there some umpires that are more easily "fooled" by a well framed pitch in certain areas of the strike zone?
mikefast
9/22
I did not account for the variations between umpires in this analysis. My belief is that over a season-size sample, that catchers would be paired with enough different umpires that it shouldn't make a big difference.

I would definitely, though, at some point like to investigate the second part of your question, which is whether some umpires are more susceptible to these sorts of techniques than others.
markpadden
9/22
That would be great. Ideally, for each given catcher/ump pair, you'd like to be able to estimate the net expected impact on called strikes -- looking at independent catcher impact, independent umpire impact, and also any possible catcher/umpire interaction effect.
nfkrueger
9/21
I'm with those who think it is clearly time to eliminate umpires' calling balls & strikes.

Barring that, umpires should be alerted as to the framing tendencies of C's (at least it's not accusing them of personal biases).

If I get called out on strikes on clearly ball 4... but the C scams the ump... I would raise holy hell. (And "scam" is not too strong a word. BP & Mike Fast just proved that.)
lesmash
9/22
I wonder if any teams out there are on this already . . . or if the wonderful insight offered by Mike will see him offered a job with a MLB team. This is an absolutely awesome read. Thank you.

I would be curious to have Mike go back and study Charlie O'Brien in the ATL days and TOR days when he caught something like six or eight Cy Young performances over the span of a decade. Dude was helping Maddux et al. get calls that were a foot off the black.
mikefast
9/22
The earliest the PITCHf/x data exists for this analysis is back to the 2007 season.

Maddux was not particularly good at getting calls off the plate in 2007-2008. It's possible he was better at that further back toward his prime, but we don't have the detailed data available before 2007.

Glavine, on the other hand, is at the top of the charts.
nosybrian
9/22
Just joining the chorus: this is a fabulous piece of research.
sam19041
9/22
Mike, nice work!! Thanks for posting all of the data, too. If I read it right, Josh Thole was VERY effective in 2010, then very ineffective in 2011. Is that a correct interpretation? If so, does it make sense to have swings that big?
mikefast
9/22
According to this measure, Thole saved 29 extra strike calls in a half-season of 2010 and lost 18 extra strike calls in 2011, though he wasn't playing quite full time in either 2010 or 2011.

If you simply look at it as a rate stat, it's a pretty big contrast, but as a counting stat, it doesn't seem so big to me.

I suggested an amount that the observed data could be regressed toward league average to find the persistent skill, and if you apply that to Thole, you get him at +4 runs / 120 games in 2010 and -2 runs / 120 games in 2011. That doesn't seem unreasonable to me, but I have not looked into any details for Thole about the actual pitch locations where he gained or lost strikes.

I also don't have as firm a handle as I would ultimately like to have on quantifying the uncertainty level on these numbers. I certainly have a decent idea about the uncertainty from the year-to-year correlations, but I'd like to get a better sense of the impact of catcher teammates, among other things.

In general, I view that we are in the middle of the investigation on this topic, with the work that Max and I have done this summer. We're no longer at the beginning, as we were with Dan and Bill's work, where the findings looked promising but didn't make baseball sense. But we're also not in the end game yet where we've sewed up all the important knowledge and are only making tiny incremental changes. We've gained a significant handle on this effect, but there is significant work yet to be done, as there is with much of baseball analysis.

So, that was a good question, and this was my long-winded way of saying, "Maybe, I think so, but we'll know better as we refine this."
alangreene
9/22
Great article.
koopbri
9/22
Let me join the chorus of praise. This is why I subscribe to BP. I learned some new and important things about baseball from this article and the linked articles that it builds on. Now I know why Zaun talks about this so often as a broadcaster. Many thanks.
mikefast
9/22
Thank you, and I'm pleased you took the time to read the linked articles, too. I'm "standing on the shoulders of giants" in all of this.
TheRedsMan
9/23
Excellent article, Mike. I absolutely love that you've joined the stats and scouting angles here. One excels at identifying what and the other at the why. I think BP would greatly benefit from hiring or featuring a scout or player development guru who can speak directly to issues like these.
Richie
9/24
Looks like the thread has died out. But still figure I'll point out that The Economist now has an article all about this article. http://www.economist.com/blogs/gametheory/2011/09/statistical-research-baseball
mikefast
9/24
Thanks, Richie.

People are welcome to continue to post comments and questions here. I'll check in every day or two. Also, I'll be chatting here at BP on Monday at 1pm ET, so bring your questions for me about catcher defense or other topics.
russellj90
9/25
Fantastic work Mike. I guess my question from this deals with how GM's should view catchers. Would it be worth it to start someone like Jose Molina every day as opposed to someone like John Buck or Nick Hundley? How much would you value this in determining who to use at catcher? This opens up so many new discussions, and for that I am extremely grateful.
mikefast
9/26
Thanks, Russell. It's a big enough effect to make a difference in who should play. It's not big enough to play a bad bat over a good bat, generally speaking.

I want to refine the run estimates that I gave in this article with further work, but they should give you an idea of how much value should be ascribed to this skill.
oakwcj
9/25
Len Kasper and Bob Brenly discussed this study at length during the Cubs-Cardinals game on Friday. Kasper summarized the article really well and Brenly elaborated on the technigue of setting up on the outside of the plate and moving the glove slightly in as the catch is made to give the illusion of the pitch being closer to the middle of the plate. Brenly said that the best catchers will catch the ball in the pocket or tip of the glove to further the illusion, by having most of the glove over the plate, but only with pitchers who have very fine control. There is no way that umpires can be "alerted" to this behavior, because it largely take place at a subconscious level. A great article and an excellent discussion by Kasper and Brenly, who deserve a better team to cover.
Oleoay
9/25
It's a bit ironic too. From these numbers, Soto's a superior defensive catcher to Koyie Hill. Yet, reputation-wise, Hill is the better defensive catcher.
mikefast
9/26
Soto and Hill both rate well in blocking pitches, according to Matt Klaassen's numbers.
mikefast
9/26
I, too, really enjoyed the discussion by Len Kasper and Bob Brenly. They talked about it in the top of the fourth inning on Friday.
Oleoay
9/25
I wonder, if framing is a skill that can be coached, does that show up at an organizational level? Do some organizations develop catchers that tend to frame pitchers better than other organizations?
mikefast
9/26
I know anecdotally that some organizations do coach framing-- the Rays, for instance--but whether their principles line up with my findings is a different question. At least one scout indicated to me that my findings were quite different than how catching is currently scouted and taught.
Oleoay
9/26
Perhaps the teaching will/should change. Intuitively the findings makes a bit of sense. If something moves that is near me, my eyes, as a reflex, tend to focus on that and lose focus on things that are farther away. The movement of a catcher's head or hand, especially a dramatic movement, is more likely to reflexively shift an umpire's focus.
Oleoay
9/27
Something else too, just based on those the four clips, Varitek and even Doumit with his "quiet glove" tilt their shoulders/shift their balance. Kind of like a crowded room when you can "feel" someone near you, the umpires might react to that as well.
BillJohnson
9/26
Nothing about park effects in here? I assume you've done some analyses like the odd-year/even-year graph to convince yourself there aren't any, or something like that?
mikefast
9/27
The PITCHf/x calibration adjustment is a park-related effect, though it doesn't operate strictly like most traditional park corrections do (i.e., it isn't constant across the season).

I can't think of any other park-related effect that would be significant. If you have ideas about some other park-related effect that might be in operation, please share.
Oleoay
9/27
Btw anyone notice how stoic/still the umpire in the Molina clip is? It almost looks like a freeze frame.

I also don't see many umpires "stand" upright like him.
mikefast
9/27
Are you talking about the Lucroy pitch? (Braves-Brewers, solid blue background on the wall behind the umpire)

If so, that's Gary Cederstrom.
Oleoay
9/27
Yeah I meant the Lucroy pitch. That's what I get for commenting from work.
mikefast
9/27
I looked at another Gary Cederstom pitch about a month ago, here:
Framing Ball Four to Cano
Oleoay
9/27
Looks like Cederstorm has that same "still" stance. Note that Suzuki also drops his head, though maybe not as dramatic, similar to the Doumit example. You can tell from the two still shots before the "replay frame". Note how Suzuki's head at first blocks Cederstorm's elbow but moves enough so that by the last frame of the replay, you can see Cederstorm's elbow.
KerryHofmeister
2/02
I am using this information in private catching instruction and coaching that I do. As a former catcher I am biased, but this is my favorite article that I have ever read on BP.