The BP Wayback Machine: PECOTA Takes on Pitching Prospects and Left-Handed Pitchers

While looking toward the future with our comprehensive slate of current content, we'd also like to recognize our rich past by drawing upon our extensive (and mostly free) online archive of work dating back to 1997. In an effort to highlight the best of what's gone before, we'll be bringing you a weekly blast from BP's past, introducing or re-introducing you to some of the most informative and entertaining authors who have passed through our virtual halls. If you have fond recollections of a BP piece that you'd like to nominate for re-exposure to a wider audience, send us your suggestion.

Last week, the Rays signed young lefty Matt Moore to an extension that should prove to be team-friendly if he stays healthy, but as Nate discussed in an article which originally ran as a "Lies, Damned Lies" column on April 12, 2007, it's never safe to assume that a young pitcher's arm will remain intact.

Maybe I'm too harsh on PECOTA–my default assumption is that it tends to be a lot less useful for pitching prospects than it does for hitting prospects. But truth be told, it had a pretty good year with the pitchers last season. Here were the top ten pitching prospects in the game last season according to both PECOTA and Baseball America, respectively:

PECOTA                  Baseball America
-------------------------------------------------
1.    Francisco Liriano        Francisco Liriano
2.    Yusmeiro Petit*          Chad Billingsley*
3.    Philip Hughes*           Justin Verlander
4.    Justin Verlander         Matt Cain
5.    Anthony Reyes*           Jon Lester*
6.    Matt Cain                Scott Olsen*
7.    Jeremy Sowers*           Joel Zumaya
8.    Joel Zumaya              Mike Pelfrey*
9.    Cole Hamels*             Jonathan Papelbon*
10.   Chuck James*             Homer Bailey*

(NOTE: The Baseball America rankings exclude Bobby Jenks, who only made their list on a technicality.)

Both systems had remarkably good years. Then again, that's easy to accomplish in a season in which there was so much young pitching talent moving up into the league. The asterisks indicate pitchers who were unique to their respective lists. We can re-arrange those pitchers into six-man "rotations" based on the current cachet of the pitchers involved:

Philip Hughes             Homer Bailey
      Cole Hamels               Jonathan Papelbon
      Anthony Reyes             Scott Olsen
      Jeremy Sowers             Chad Billingsley
      Chuck James               Jon Lester
      Yusmeiro Petit            Mike Pelfrey

With another year of performance to analyze, I'd probably take Hughes over Bailey, and Hamels over Papelbon, but I'd probably now agree with BA and take Olsen over Reyes; I'd definitely take Pelfrey over Petit. The other two are too close to call. In any event, when it comes to identifying elite pitching prospects, PECOTA holds up very well for itself–or at least it did in 2006.

PECOTA did have a few issues if you dig under the surface, however. Firstly, when it misses–as in the case of Yusmeiro Petit–it tends to miss badly. Secondly, it sort of gives up at some point and puts all pitching prospects in one undifferentiated lump, at which point the scouting-based systems have a real edge. If we look at the 11-20 names from last year, for example:

PECOTA                   Baseball America
-----------------------------------------------------
11.    Brandon Erbe               Philip Hughes
12.    Chad Billingsley           Anibal Sanchez
13.    Shaun Marcum               Anthony Reyes
14.    Fausto Carmona             Mark Rogers
15.    Anibal Sanchez             Adam Loewen
16.    Paul Maholm                Adam Miller
17.    Troy Patton                Dustin McGowan
18.    Dallas Braden              Jason Hirsh
19.    Andrew Sonnantstine        Jeremy Sowers
20.    Cesar Carrillo             Craig Hansen

It's not a rout, but the Baseball America list is pretty clearly better. They make up for lost ground by nabbing pitchers like Hughes, Reyes, and Sowers, whereas guys like Papelbon and Bailey remained completely off PECOTA's radar screen. While each list had its share of misses (Dallas Braden, Dustin McGowan), BA's "hits" (like Adam Loewen and Adam Miller) tended to be a lot better.

More importantly, when you combine the rankings for pitchers and hitters, Baseball America had the pitchers ranked relatively higher, and that worked out pretty well for them last year. Francisco Liriano, for example, was #6 overall on the Baseball America list, and #15 overall on the PECOTA list. Matt Cain was #10 on BA's list, and #33 on ours.

This was perhaps the most noticeable weakness of the PECOTA prospect lists last season; it was too bearish on elite pitching prospects. Ironically, I was convinced that this was going to be its greatest strength. Baseball America had a very poor track record at projecting pitching prospects in the '90s, and I suspected that they were systematically underestimating the attrition rates associated with young pitchers.

So what happened? I suspect that all of the following are true to some degree:

• Young pitchers, as a group, had a flukishly good season in 2006.
• Baseball America has gotten quite a bit better at ranking pitchers (in large part because they pay more attention to statistical analysis and workloads).
• It can be deceptive to look at prospect lists just one year out. Liriano looks like a brilliant pick right now in spite of his injury, but will the same be true if he never gets back on track and the rest of his career resembles Mark Prior's? Remember, the argument about pitching prospects is not necessarily that they are greater risks in the near-term, but that they are greater risks in the medium-to-long term.
• Young pitchers are handled a lot more carefully than they used to be, and are therefore more likely to develop in accordance with expectations.

The last of these contentions–that pitching prospects are a lot less risky than they were a decade or so ago–is absolutely fascinating, and something that I hope to devote an entire column to at some point later this season. For the time being, let's take a look at the list of all pitchers who were ranked in Baseball America's overall Top 10 at some point during the 1990s.

Ryan Anderson
Rick Ankiel
Steve Avery
James Baldwin
Alan Benes
Kris Benson
Jason Bere
Bruce Chen
Matt Clement
Livan Hernandez
Tyrone Hill
Kiki Jones
Pedro Martinez
Ben McDonald
Carl Pavano
Brad Penny
Arthur Rhodes
Frankie Rodriguez
Roger Salkeld
Jose Silva
Brien Taylor
Todd Van Poppel
Allen Watson
Matt White
Paul Wilson
Kerry Wood

Literally something like three-quarters of those pitchers suffered a catastrophic injury that ended or significantly interrupted their careers. In some cases (Pedro Martinez) the injury came after tremendous success, but more often than not it occurred early, and precluded the pitcher from having much of a career at all. It is this lost generation of pitching prospects that motivated concepts like TINSTAAPP and Pitcher Abuse Points. If the dynamic has changed to the point where pitcher attrition rates are much lower than they used to be, this has enormous implications for both analysts and organizations.

PECOTA's minor league comparables database goes back to 1997. If there was a Lost Generation of pitching talent during the early and mid '90s–and I suspect that there is–it may be catching the tail end of it. Interestingly, however, the pitchers fare quite a bit better in PECOTA this time around, with two pitchers included in the system's overall top five prospects. Some of this is because of methodological changes–we now give starting pitchers slightly more credit toward their Upside score, for example, and relief pitchers slightly less. But the performances of guys like Liriano and Cole Hamels and Jered Weaver are starting to flow into the system, and I suspect that's also making some difference.

I still suspect that PECOTA's pitching prospect ratings need to be taken with something of a grain of salt. Particularly in the low minors, it's possible for two different pitchers to post similar statistics while taking radically different approaches. This is generally not the case with young hitters; it's hard to hit 20 home runs in the minor leagues, for example, unless you have some legitimate kind of power stroke.

Guys who pitch to contact, for example, almost invariably post low walk rates in the minor leagues, and they can also maintain decent strikeout rates, since minor league hitters aren't always good enough to catch up with mediocre stuff. Sometimes it's possible to red-flag these guys by means of their high BABIPs or home run ratios, but other times it's not. There may also be solutions like looking at how well a guy's numbers hold up as he advances levels, or evaluating various kinds of split data, which haven't been fully fleshed out yet. In the meantime, we need to make at least some allowance for a guy's scouting reports.

On to the left-handed starters. We're going to go through these guys a bit more quickly than we did for the position player prospects, since there's generally a bit less to distinguish them based on the statistical record alone. Note that relief pitchers are not included here; I will probably settle up on those guys in an Unfiltered post.

Excellent Prospects

1. Clayton Kershaw, Dodgers (20)      156.7
2. Brent Fisher, Royals (19)          104.0
xx. Kei Igawa, Yankees (27)           102.4

The Dodgers really couldn't have asked for anything more from Clayton Kershaw last year. Some 37.5% of his plate appearances last year ended with a strikeout; just 3.5% ended with a walk, and none with a home run. All the usual caveats about small sample sizes and rookie ball performances apply, but you just don't see a pitcher break out of the gates so strongly very often, as evidenced by Kershaw's extremely low Similarity Score. He is by no means a sure thing–his top two comparables are Petit and Justin Jones–but the scouting reports match the stuff.

Brent Fisher will join Kershaw in the Midwest League this year, but the gap between the two prospects is probably wider than PECOTA indicates. Fisher throws 3-4 MPH slower, is not as adept at keeping the ball down, and the talk about his deceptive delivery raises some recollection of Yusmeiro Petit. I do think the upside here is higher than advertised–Fisher is young enough and has a good enough pitcher's body to add a couple more ticks to his fastball–but the risk is higher than PECOTA is making it out to be too.

I don't find Kei Igawa terribly interesting on his own merits, but PECOTA is treating him more as an established commodity, and he forms a benchmark of sorts for what a reasonably young #3 starter looks like. In general, a pitching prospect needs to be very good indeed before you would prefer him to an established #3, cost considerations aside.

Very Good Prospects

3. Scott Lewis, Indians (23)            76.9

Scott Lewis won something called the Most Spectacular Pitcher Award last season by virtue of having the lowest qualifying ERA in the minor leagues–I am not making this up. It's not surprising, then, that a statistically-based forecasting system finds him at least somewhat intriguing. He was old for his level and doesn't throw very hard; on the other hand, he's been handled carefully after undergoing Tommy John surgery following his 2003 season at Ohio State, and there's an argument that he's only now getting stretched out and coming into his own, particularly having come from a cold-weather state. I'm not ready to endorse PECOTA's forecast yet, but will be willing to do so fairly quickly if his numbers hold up at Double-A. It would also be interesting to see if he can add some velocity if the Indians move him to the bullpen.

Good Prospects

4. Scott Elbert, Dodgers (21)        41.6
5. John Danks, White Sox (22)        41.5
6. Jacob McGee, Devil Rays (20)      41.3
7. Garrett Olson, Orioles (23)       40.1
8. Jaime Garcia, Cardinals (20)      40.0
9. Anthony Butler, Mariners (19)     39.6
10. Franklin Morales, Rockies (21)   34.8
12. Daniel Haigwood, Phillies (23)   33.8
13. Charles Lofgren, Indians (21)    32.7
14. Patrick Misch, Giants (25)       32.4
15. Donald Veal, Cubs (22)           32.1
16. Troy Patton, Astros (21)         31.7
17. Kristofer Johnson, Red Sox (22)  31.1
18. Glen Perkins, Twins (24)         30.6
19. Ryan Feierabend, Mariners (21)   29.6
20. Gio Gonzalez, White Sox (21)     29.5
21. Jeffrey Locke, Braves (19)       28.6
22. Sean Henn, Yankees (26)          25.6

Although PECOTA has gotten a bit more aggressive with its ratings for elite pitching prospects, it still has a tendency to bunch run-of-the-mill pitching prospects together. This is not entirely bad news–the flip side of fewer pitching prospects being sure things is that there are also fewer pitching prospects that count as sure failures; it's much more likely for a pitching prospect with middling statistics to make a Great Leap Forward than a hitting prospect with comparable performance. This dovetails with the sabermetric conventional wisdom that while the objective with hitting prospects is to compile quality, with pitching prospects it may be more worthwhile to collect quantity, and let them sort themselves out.

The interesting question, of course, is whether those pitching prospects with better scouting reports are more likely to make that leap forward. This is complicated by the fact that the scouting reports for left-handed pitching prospects tend to be less differentiated than those for right-handed pitching prospects. A great number of these prospects throw between 88-92 (but not harder), with a decent secondary offering (but not a great one). The only left-handed pitching prospect in the game who can probably be described as having plus-plus stuff is the Tigers' Andrew Miller, who does not qualify for this list because he didn't pitch enough innings.

I do think that some of the more recognizable names on this last are slightly overrated. Scott Elbert has had some tremendous problems with his command and his home run rates, and while there are certainly some precedents for that getting turned around–Scott Kazmir and Rich Harden appear prominently on his comparables list–it is by no means a sure thing. I support the White Sox' decision to gamble on John Danks in the back end of their rotation, but both the numbers and the stuff are good rather than great. The pitcher I'm most intrigued by on this list is Jacob McGee, who is perhaps the only pitcher on this list who can hit the mid-90s with some regularity, and is young enough to have a couple of extra years to sort his command issues out.

On the flip side, there aren't too many totally goofy selections on this list, but Olson, Haigwood, Misch and Henn all deserve an asterisk for not bringing it in very hard.

Average and Marginal Prospects
(Players Ranked in Kevin Goldstein's Positional Top Ten or other noteworthy names with Upside scores below 25)

Travis Wood, Reds (20)           23.2
Sean West, Marlins (21)          22.6
Mark Pawelek, Cubs (20)          12.4

Wood and West just barely missed the 25 Upside score required to make it to the Good Prospect category, so I wouldn't read too much into their placement here. Pawelek's low score is a bit more troubling, since it comes coupled with reports that his mechanics and velocity were off at Boise. Keep in mind as you're perusing the PECOTA cards that that Clay's adjustments to minor league pitching statistics tend to be harsh. For example, the entire Northwest League struck out more than 7.5 batters per nine innings last season, so Pawelek's 7.6 K/9 was no better than average. Minor league pitching statistics should not be read the same way as major league pitching statistics; striking out a batter per inning is really just a starting point to qualify as a good pitching prospect, particularly at the lower rungs of the system.

The Big Picture: Rankings Combined With Non-Rookies 25 Years Old Or Younger

1. Francisco Liriano, Twins (23)      279.5
2. Scott Kazmir, Devil Rays (23)      204.3
3. Cole Hamels, Phillies (23)         184.3
4. Clayton Kershaw, Dodgers (19)      156.7
5. Dontrelle Willis, Marlins (25)     140.0
6. Zach Duke, Pirates (24)            119.0
7. Scott Olsen, Marlins (23)          116.3
8. Adam Loewen, Orioles (23)          113.8
9. Jeremy Sowers, Indians (24)        112.9
10. Brent Fisher, Royals (19)         104.0

PECOTA is not aware of the extent of Liriano's injury; perhaps eventually we'll be able to build a module into PECOTA to handle Tommy John surgery, but we haven't incorporated that yet. Injury and all, I'd take Liriano before any pitcher on this list but Kazmir and probably Hamels, and I think it's very close between those three.