Checking the Numbers: Gauging True Talent

Last week, we rolled out post-season PECOTA projections that were used to generate anticipated results for each game of every playoff matchup. The goal was simple: provide the readers with the most accurate information capable of helping guide their expectations for a particular matchup. The projections were never meant to be treated as the gospel, but rather as more accurate tool of evaluating the outcomes of certain games than simply eyeballing the lineups or strictly using numbers from this past season. However, in the process of writing the daily summaries and projected results for every lineup, it became clear that not only was the entire process in need of further explanation, but so too was the concept that a true talent level is best-suited for this type of exercise.

What is a true talent level? In short, the true talent level comprises what we expect a player to be capable of doing today, based on a variety of factors such as his production over the last few years, his age, his regression to the league average, perhaps his injury history, or even the way in which he will be utilized in a lineup. For instance, Lance Berkman’s overall projection, in both raw numbers and rates, would look different in Houston where he was an everyday player than they would in New York, where the switch-hitter is essentially a designated hitter against righties for the Yankees. The longer answer involves delving deeper into each of the aforementioned components to tailor the true talent level to the specifications of each one. And this is before even mentioning that very few of the factors are clear-cut propositions, as those who study aging curves and regression will attest.

But realistically, the true talent level helps craft expectations of a player at a particular point in time. Berkman’s true talent slash line is going to be much higher in 2004 than in 2010, simply because he was younger and in the prime of his career. Now, at 34 years old, he is expected to slow down and be less productive. The true talent level also helps in using the right basis for comparison and it can be modified based on matchups. Berkman’s true talent level in a vacuum is vastly different than his expected line of production against David Price. This is an incredibly important concept that is too often forgotten, or else many articles written on various sites over the last month wouldn’t have been as glaringly incorrect in their playoff previews.

As important of a concept as it is, the true talent level of a player is equally tough to grasp from the standpoint of developing expectations, because it feels right to simply use the current season numbers as a proxy for performance. Josh Hamilton may have hit .359/.411/.633 this past season, but that was only 571 plate appearances of production. Over his last 1,640 PAs, his slash line is an impressive but notably lesser .315/.372/.543. When trying to figure out what he might hit in the postseason, or next season, it is easy to fall into the trap of expecting the current success to continue, but more accurate to use the latter line as the pre-adjustment projection. After all, something like regression doesn’t exactly happen with the flick of a switch so it can be very difficult for fans to look past what they just saw for six months and expect that their player will decline or improve based on the past.

This issue has surfaced a few times in the articles written last week, with the most obvious example being between Jason Heyward and Nate McLouth. Both Braves were projected to hit about the same off of Tim Lincecum, which confused many. McLouth absolutely stunk this season while Heyward put up somewhat unprecedented numbers given his age. How, then, could they both be expected to hit the same off Lincecum, or anyone for that matter? The answer lies in their true talent levels. We don’t know much about Heyward, to be perfectly honest. He had a great season, and the odds are that he will have a wonderful career, but we just don’t know, and that would constitute a mighty big assumption to program into the system. On the other hand, we do know a lot about McLouth.

We know that from 2007-09 he hit .265/.353/.467, and while Braves fans haven’t gotten to see that version of McLouth, the version still existed in the past and cannot be ignored. His true talent level entering a matchup against Lincecum isn’t going to be solely derived by his putrid line this season, even if fans will swear qualitatively that something looks off in his mechanics causing the decline. Instead, we will craft his true talent level based off all the pertinent information produced over the last several seasons, which still sees him as a capable hitter. Maybe he isn’t anymore, but it wouldn’t be in our best interest to just follow that line of thinking because of one poor year.

The next question then becomes: how does PECOTA utilize true talent levels relative to its playoff projections? The process is twofold. First, generate the PECOTA projections in a vacuum, which refers to projections sans context of who the player is facing, using the normal aging adjustments and phenotypic attributes. The PECOTAs are run, a spreadsheet is exported, and suddenly we have the true talent level of all players, though for the purposes of these playoff summaries, the 2010 season is weighted very similarly to the 2009 campaign, which will change when the projections are rerun prior to the 2011 season. The next step is tailoring that output to a playoff matchup, which involves a process written about in this space before, called the odds ratio.

The odds ratio is the ratio of a certain event happening in one group to the same event happening in another group. Relative to baseball, it would be the likelihood that Ryan Howard strikes out compared to the rate with which Edinson Volquez generates strikeouts. The odds ratio is used in simulations and, to an extent, games like Strat-o-Matic, because it gets to the root of the batter-pitcher matchup. The odds ratio answers the question of what will happen when a certain batter faces a certain pitcher. Some events will have a higher rate of expected occurrence because we can tailor the numbers to a specific matchup as opposed to using numbers resulting from matchups across several different types of pitchers. Using a specific example with fabricated numbers, here is how the odds ratio would work for the likelihood a strikeout occurs if Volquez strikes out 22 percent of his batters faced while Howard whiffs 32 percent of the time, in a league with an average of 15 percent:

Odds(Hitter): 0.32 / (1-0.32) = 0.471
Odds(Pitcher): 0.22 / (1-0.22) = 0.282
Odds(League): 0.15 / (1-0.15) = 0.176
Odds(H) * (Odds(P)/Odds(L)) = 0.471 * 1.602 = 0.755
Rate of an SO: 0.755 / (1+0.755) = 0.430

So in this situation, with an above-average strikeout pitcher facing a batter that strikes out at a rate well above the league average, a strikeout is expected to occur 43 percent of the time. This process is repeated for every matchup that could occur in the playoffs, and for every event that could occur in a plate appearance. The only difference is that, instead of regular-season numbers being used, PECOTA provides the rates that are plugged into the formulas above.

The lines are also adjusted for the park in which the game will be played and whether or not the players in question are at home or on the road. From there, the results projected over 650 PA are prorated to the appropriate number of PAs against the starter and top relievers, and linear weights run values are used to determine the expected score of the matchup. But everything boils down to an understanding that the true talent level of a player is what must fuel an exercise like this, as players simply should not be expected to just pick up where they left off in the regular season.