The genesis of this article was a simple reader question, the kind of question that fantasy writers get almost every day and answer by rote.
What’s wrong with Adam Wainwright?
I wrote something in the neighborhood of 1,700 words on the subject, read what I wrote, and realized that it was utter garbage. This isn’t because I wrote something substandard but rather because there is no industry standard, or if there is an industry standard I am unaware of its existence. This complaint of mine is not directed at the baseball community on the whole but specifically at the fantasy community. We are all trying our best but many of us are simply stumbling around in the dark.
There are problems throughout our entire process, and identifying every one of these problems on a granular level could fill a small book. What is listed below is not intended as a comprehensive listing of what’s wrong but rather a high-level overview.
To simply this explanation, let us return again to the question I attempted to answer.
What’s wrong with Adam Wainwright?
The most obvious problem is one that we are well aware of: we are reacting to a small sample size. Our response to this is often to note that the sample size we are reviewing is small but then to go ahead and answer the question anyway! Think about this for a moment. We know on one level that what we are doing is “bad” yet we go ahead and do it anyway. The argument in favor of this approach is that we “have” to, but this in and of itself is a poor response. Answering a question like this to generate content in April is a bad reason for answering, but is often what we do.
Let us assume that we are approaching these questions from the genuine perspective that we are trying to help. Beyond the perils of a small or useless sample size, there are other problems with this common approach.
The Data Are Wrong (Or Frequently Doesn’t Prove What We Say It Does)
Frequently, a corollary is presented between A and B as if this corollary has definitively been proven, is established in fact, and backed by multiple data points. In many cases, the link between A and B is uncertain at best. In a handful of cases, the link between the two data points doesn’t exist at all and cannot be proven using the data that we have available.
Below is an example of two years of data for pitchers, the expected outcome based on the data presented, and the actual results.
Table 1: Expected Strikeout Risers and Actual Results 2014-2015
Year |
Pitcher |
April K% |
xK% |
Actual K% |
Predictive? |
2014 |
14.3% |
21.7% |
25.4% |
Yes |
|
2014 |
14.3% |
20.7% |
11.5% |
No |
|
2014 |
16.3% |
21.8% |
15.1% |
No |
|
2014 |
16.3% |
21.5% |
20.6% |
Yes |
|
2014 |
21.4% |
26.5% |
22.5% |
No |
|
2014 |
16.0% |
20.6% |
15.4% |
No |
|
2014 |
19.2% |
23.3% |
21.9% |
Yes |
|
2014 |
25.4% |
29.5% |
23.5% |
No |
|
2014 |
17.3% |
21.4% |
18.9% |
Maybe |
|
2015 |
21.6% |
27.1% |
32.1% |
Yes |
|
2015 |
19.4% |
24.4% |
20.3% |
Maybe |
|
2015 |
16.9% |
21.3% |
14.3% |
No |
|
2015 |
11.4% |
15.8% |
13.8% |
Maybe |
|
2015 |
11.1% |
15.5% |
8.4% |
No |
|
2015 |
12.2% |
16.3% |
15.2% |
Yes |
|
2015 |
11.5% |
15.5% |
14.1% |
Yes |
|
2015 |
17.3% |
20.9% |
22.3% |
Yes |
|
2015 |
19.3% |
22.7% |
17.9% |
No |
|
2015 |
11.8% |
14.9% |
13.4% |
Maybe |
|
2015 |
14.8% |
17.8% |
17.9% |
Yes |
Table 1 is a list of 20 pitchers from two separate articles on a fantasy website – from April 2014 and April 2015 – listing expected strikeout rate risers based on swinging strike rates versus actual strikeout rates year-to-date. The last two columns are mine. The actual K% is self explanatory while the final column is my “scoring” of the result.
Are swinging-strike rates predictive of future strikeout rates? Perhaps, but the scorecard from these two articles shows eight clear cases where it was, eight clear cases where it wasn’t, and four cases where the data could be viewed either way (the “maybes”). There is no way to sugarcoat these results, this isn’t good. I would need more information to ascertain whether swinging strike rates work or not as far as predicting future strikeout rates, but in terms of fantasy advice, this was not helpful if readers acted on this information. Or if it did help, it’s because you got lucky and picked the right pitcher.
This problem is not limited solely to one type of data. This is a frequent challenge when analysts use data in an attempt to prove a point. This initially happened when FIP came to the fore as a popular metric. FIP was interesting to look at and is a superior to ERA in terms of measuring a pitcher’s contributions to the game but a few years ago it was a new, shiny tool that became a lazy catch-all. I studied this back in my blogging days and discovered that while FIP was better than ERA at predicting future performance it wasn’t all that much better than ERA at doing so. Batted-ball distance is another example of a dataset that is interesting to look at but not nearly as predictive as you would expect based on how glowing its proponents are when discussing it.
The Data Here Prove Tha…OH HEY LOOK AT THESE PRETTY CHARTS AND GRAPHS
One of the most positive developments in baseball analysis has been the addition of charts, graphs, and video to baseball articles. What once was a dry and turgid slog through data has been become fun and entertaining thanks to the addition of visual aids. As a writer, this is an excellent way to keep your readers entertained and interested in a story. It is also a way to present a misleading picture of your data, whether you intend to do so or not.
Let’s take a look at some pitch zone data.
Chart A: Pitcher Zone Data, 2014
Chart B: Pitcher Zone Data, 2015, Opening Day through May 13th
This pitch data was used to display that the pitcher in question was due to improve his home run rate in 2015 based on the data available through mid-May. Despite allowing a similar home run percentage in 2014, this pitcher’s groundball rate jumped from 34.4 percent in 2014 to 43 percent through the date on the chart. The conclusion presented was that “his current home run rate is a career high but being distorted by a 20 percent home run/fly ball rate that won’t hold.”
The pitcher is Danny Salazar. The results turned out as predicted, but the reason they were correct had little if anything to do with where Salazar was locating his pitches. The leader among 2015 qualifiers in HR:FB was James Shields with a 17.6 percent rate. Since HR:FB have been tracked in 2002, no qualifying pitcher has allowed home runs at a 20 percent rate; Odalis Perez at 19.7 percent in 2003 came the closest. Shields finished with the same 34.4 percent fly ball rate Salazar had in mid-May, but in his case the HR:FB rate did not drop.
Another reason it was unlikely that Salazar had experienced some kind of significant breakout was that his 37 percent strikeout rate during his first five starts of 2015 would have been unsustainable for any starting pitcher. Randy Johnson is the all time leader (minimum 1500 innings pitched) at 28.6 percent. The active leader is Clayton Kershaw, at 27.5 percent. Salazar’s 23.2 percent strikeout rate indicate a pitcher with a strong strikeout profile, but there was nothing in his numbers that indicated an historic strikeout spike was coming.
Sure enough, the batted ball data for the rest of the 2015 season did not indicate any kind of fundamental change in Salazar’s pitch location.
Chart C: Danny Salazar Zone Data, 2015, May 14th Forward
For the remainder of 2015, Salazar’s zone profile reverted back to what he did in 2014. While he did work a little bit lower in the lower right-hand part of the zone, the difference wasn’t significant enough to merit notice over the course of a full season. As one might expect, Salazar’s zone profile normalized.
Charts and graphs often reinforce the perception that something is accurate even if the data presented do not necessarily support this. Our senses react positively to the colors within graphics and have the ability to make us more inclined to believe that we are getting good information. This phenomenon combined with the lack of context can convince us that something is true or accurate even if there is no basis in fact.
Both of the issues highlighted above dovetail into a third issue, which is the most significant challenge when it comes to the application and usage of data.
Descriptive Is Not Predictive
In the 1998 Coen Brothers film The Big Lebowski, two characters try to solve a mystery, despite the fact that neither one has the training or temperament to be a private detective. In addition to a confusing series of clues and intended misdirection by the other actors in this fiction, both Jeff “The Dude” Lebowski and Walter Sobchak struggle with their lack of ability in this area. In an attempt to recover a lost briefcase containing one million dollars, The Dude and Walter find themselves in the house of Larry Sellers, a 15-year-old who stole Lebowski’s car, which contained the briefcase.
An agitated Walter screams repeatedly “is this your homework, Larry?” The Dude yells at Walter “just ask him if he…ask him about the car, man!” and later “we know it’s his fucking homework, Walter!” Where’s the fucking money, you little brat!” Where Walter is agitated about the homework, The Dude recognizes that while the homework led them to Larry’s house, it is useless as an additional data point to discover what they are looking for: the briefcase with the money.
In data science, prior events can lead us to what we are looking for but we have to know what we should keep and we should remove from our study. Too often, all descriptive data is simply presented as evidence that will have an impact on future results without any proof that these data are indeed predictive.
One reason that we in the fantasy community are tempted to follow this blueprint is because amazing writers like Sam Miller and Rian Watt do such a damn good job at using descriptive data to make their already strong writing even better. This week at Baseball Prospectus, Miller wrote about Clayton Kershaw while Watt wrote about Aaron Nola. Both writers adroitly used charts, still photographs, and video to augment their wonderful words on each pitcher. Neither author used descriptive data to suggest that past results would be a definitive indicator of future success. Their job is to write pieces that describe to us what a pitcher is doing and how he has improved or declined. Our job is to write pieces that describe if a pitcher can get better or worse based on what has happened to date.
Where we get stuck in the fantasy community is that we are constantly asked to predict how well someone will do in the future. The vast amount of data available is both a blessing and a curse. It is great to have such a mass amount of data to work with, but the drawback to having copious amounts of data is that it is easy to go down the wrong path and either apply the wrong data to your study or apply the correct data improperly.
There are two areas where I can see room for immediate improvement.
Approach Your Problem Correctly
Let us return for a moment to one of the data sets presented above but look at it in a different manner.
Table 2: Top 10 Actual Strikeout Risers: 2014-2015
Year |
Pitcher |
April K% |
K% Total |
Diff |
2014 |
20.8% |
28.3% |
7.5% |
|
2014 |
9.7% |
16.6% |
6.9% |
|
2014 |
Julio Teheran |
15.9% |
21.0% |
5.1% |
2014 |
25.0% |
30.1% |
5.1% |
|
2014 |
Jeff Samardzija |
18.5% |
23.0% |
4.5% |
2014 |
13.4% |
17.3% |
3.9% |
|
2014 |
14.4% |
18.2% |
3.8% |
|
2014 |
20.4% |
24.0% |
3.6% |
|
2014 |
15.4% |
18.9% |
3.5% |
|
2015 |
14.4% |
17.8% |
3.4% |
|
2015 |
5.9% |
17.7% |
11.8% |
|
2015 |
Chris Sale |
21.1% |
32.1% |
11.0% |
2015 |
Jacob deGrom |
18.3% |
27.3% |
9.0% |
2015 |
16.3% |
23.7% |
7.4% |
|
2015 |
19.7% |
26.9% |
7.2% |
|
2015 |
22.6% |
29.6% |
7.0% |
|
2015 |
13.1% |
19.5% |
6.4% |
|
2015 |
15.0% |
20.1% |
5.1% |
|
2015 |
25.7% |
30.7% |
5.0% |
|
2015 |
17.7% |
22.5% |
4.8% |
Table 2 approaches the notion of in-season strikeout risers from a different perspective, searching retrospectively for pitchers who actually did see their strikeout rate jump from the first month of 2014 or 2015 throughout the rest of the season. Viewing the data through this lens, the idea of using early swinging strike rates as a predictor of future success is even more problematic. None of the Top 10 predicted strikeout risers from Table 1 appear in Table 2 for either 2014 or 2015. While there may be some success using this method, the most prominent examples of strikeout risers could not be identified over the last two years using swinging strike rates as a predictive method.
This in and of itself is not a conclusion or an end to a definitive study on this matter. Rather, this is a jumping-off point to a longer conversation about why strikeout rates increased for these pitchers. This leads directly to my second suggested area for improvement:
You Are a Data Scientist
For some, sabermetrics and data science give this impression of someone writing thousands of lines of code, having knowledge of a programming language, and most importantly someone who works with data. These are all extremely useful assets to have if you are going to pursue a career in data science. However, knowledge of your subject matter is also a useful asset that cannot be underestimated. If you are a student of the game, you have the ability to ask the questions that can lead you down the path of discovery.
When I started writing about baseball, I never would have considered myself a “scientist” or an “analyst.” But the more I learned about the game, the more I understood the data that was presented about the game. The more access I had to data – even to “rudimentary” public data on websites like Baseball Prospectus or Fangraphs – the more I figured out how to ask the questions that led me to a greater understanding of the data.
Asking questions about the numbers you see in an article is an excellent start. If you find out that a conclusion presented doesn’t wash, the next step is asking yourself why the information is wrong. Finally, you want to figure out how you can get there from here. It is also entirely impossible that you cannot do so. Some of the greatest experiments ever conducted have negated their hypothesis without presenting an alternate conclusion.
If you see something happening on the field that you believe is sustainable, ask yourself why this is. Find out if there are similar, past examples of this phenomenon. Examine if there is a corollary between Player A and Players B-J. The ability to mine data is great, but if you don’t know what you are looking for all of your skills in this area are ultimately useless.
Finally, it is better to say “I don’t know” than it is to present faulty data or an unsustainable conclusion. Copious amounts of research that end in no greater understanding may seem fruitless but are in fact another way to increase your knowledge base. It may not be the answer that a reader is looking for, but if written well, your conclusions can still help his or her ability to understand, to comprehend, and to learn more about the game beyond “what is wrong with this player on my fantasy team?”
Thank you for reading
This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.
Subscribe now