Pebble Hunting: The Importance of Top Prospect Trajectories

If you’re the sort of fan with only enough mental storage space to keep track of the top 101 or so prospects, you may officially quit paying attention to Mike Olt, Trevor Story, Rymer Liriano, Kaleb Cowart, Taylor Guerrieri, Bubba Starling, Danny Hultzen, Mason Williams, Luis Heredia, Arodys Vizcaino, J.R. Graham, Alen Hanson, Allen Webster, Jake Marisnick, Justin Nicolino, Hak-Ju Lee, Casey Kelly, Tyrell Jenkins, Clayton Blackburn, Dorssys Paulino, Sean Nolin, Michael Fulmer, Lewis Brinson, Delino DeShields. All were top 101 prospects last year, but things happen to prospects (they break your heart) and none is this year. You can just rewrite over those brain cells.

If you’re the sort of fan, though, who keeps a few post-hype sleepers in the back of your mind, those are your boys. Each has already demonstrated some combination of tools and traits—and, usually, performance—that was worth getting excited about, so, you figure, even after a bad year, or a bad injury, there’s at least something exciting buried under each player’s recent disappointment. To the collector of post-hype sleepers, the only thing better than a prospect is a guy who used to be a prospect.

So what this will attempt to answer is whether that’s true. Or, because it’s the question that got me wondering: Is Gary Brown still worth paying attention to?

Brown, you’ll recall, was a top 101 prospect two years ago, and after a torrid season in High-A we considered his arrival imminent: in a perfect world “he’s a dynamic, All-Star center fielder,” according Kevin Goldstein at the time. A year later, he was off the top 101—though still one of the top prospects in a (relatively weak) Giants system. Jason Parks’ write-up of him that year: “The adjustment issue and lack of growth is a red flag, but the overall athleticism and glove-work give him a high floor. Even if the bat is only average, the defensive profile will have use on a major-league bench (eventually).” This year, Brown tumbled further, off the Giants’ top 10 entirely. Parks again: “It’s highly unlikely that his bat improves to the point where it plays as a major-league regular. But the defensive profile and speed should be enough to carry him to the highest level… however, when you have the physical tools of Brown—a former first round pick—the bar for success is set much higher than most players.”

So, Gary Brown. Worth the brain cells or not?

In 2007, Kevin Goldstein wrote his first top 101 for this site. In 2008, he wrote his second, predictably. Twenty-six players disappeared in the meantime, not counting players who became ineligible for prospect lists due to service time. The 26:

Jeff Niemann, Philip Humber, Brandon Erbe, Chuck Lofgren, Donnie Veal, Billy Rowell, Troy Patton, Trevor Crowe, Sean West, Humberto Sanchez, Ryan Tucker, Ryan Sweeney, Will Inman, Matt Harrison, Glen Perkins, Sean Gallagher, Brad Lincoln, Sean Rodriguez, Cesar Carrillo, Chris Parmelee, Dellin Betances, Brett Sinkbeil, Cedric Hunter, Pedro Beato, Javier Herrera, Kyle Drabek.

What do you think of that list? Nobody on there who is a star; one closer (Perkins) and one Opening Day starter (Harrison), some solid role players, but if you’d ignored these 26 in your keeper league you’d be okay right now. Of the 26, only 10 have produced a positive career WARP (two of them with just 0.1). Nine are inactive, just six years later. Overall, they produced a cumulative 28.3 WARP, about one win per player. Ryan Sweeney, by WARP, has had the best career in the group, at 8.3. Ryan Sweeney, by WARP, has had the best career in the group. Ryan Sweeney, by WARP, has had the best…

On the other hand, relative to the nearly 6,000 minor leaguers outside the top 101, these guys did pretty well. The point of this exercise is to figure out whether players who drop on prospect lists are underrated assets, but to know whether something is underrated you have to know how he is rated. And once a player drops off the top 101, it’s hard for the uninformed among us to know exactly how he was rated—was he the 102nd best prospect in the game by consensus, or the 1,500th best?

So step away from the guys who fell off the list entirely and look at players who merely dropped lower on the list—we still know, after all, how they were rated. I divided the list into tiers, admittedly arbitrary tiers but something to let me clearly define whether a player truly dropped in the eyes of the assessor or was merely shuffled about. Tiers:

• 1 to 5
• 6 to 15
• 16 to 24
• 25 to 39
• 40 to 59
• 60 to 79
• 80 to 101

And I looked only for players who dropped two tiers or more. (Going from 15 to 16 would drop the player a tier but wouldn’t signify any real change. Hence two tiers or more.) In Goldstein’s first two lists, the following players dropped two tiers or more:

• Brandon Wood (fell three tiers)
• Reid Brignac (two tiers)
• Fernando Martinez (two tiers)
• Adam Miller (two tiers)
• Luke Hochevar (three tiers)
• Scott Elbert (two tiers)
• Carlos Gomez (two tiers)
• Carlos Carrasco (two tiers)
• Dexter Fowler (three tiers)

You’ll note that a lot of these players, save Fowler and Gomez, have something in common: disappointment. But are they disappointing relative to the other bottom-half prospects they were ranked alongside? Let’s compare their career WARPs to the prospects who were most similarly ranked in 2008, the post-drop year for each of these players. Pitchers are compared to pitchers only; hitters to hitters. Each is compared to the two prospects directly above him and the two directly below:

On average, the prospect who had dropped produced about two fewer WARP over the next six years than he “should” have produced based on his new prospect ranking. There is so much noise here, though, so let’s try another year. Moving further back, and switching to Baseball America rankings, we’ll repeat with three more seasons’ dropped prospects:

2003 to 2004 fallers: Jason Stokes, Adam Wainwright, Brad Nelson, Sean Burnett, Jose Lopez, Bryan Bullington, Mike Jones.
Produced: 4.7 WARP per player
Comparable prospects: 5.7 WARP per player

2000 to 2001 fallers: Dee Brown, Chin-Feng Chen, Michael Cuddyer, Eric Munson, Abraham Nunez, Danys Baez.
Produced: 2.6 WARP per player
Comparable prospects: 8.9 WARP per player

1995 to 1996 fallers: Ben Grieve, Todd Hollandsworth, Dustin Hermanson, Doug Million, Josh Booty, LaTroy Hawkins, Julio Santana, Trey Beamon, John Wasdin
Produced: 4.1 WARP per player
Comparable prospects: 11.1 WARP per player

There’s nothing conclusive about this very imprecise and very arbitrary way of looking at things, but what we can say is this: In all four years I looked at, the players who dropped down prospect lists produced fewer WARP than similar ranked players. In the four years overall, the difference is large—falling prospects produce around half of what non-falling prospects do. (You might keep that in mind for this year's two-tier fallers: Billy Hamilton, ​​Travis d'Arnaud, Matt Barnes, David Dahl, Gary Sanchez.) If we were to hypothesize anything from this, for further research, it is that prospects who fall are actually overrated, not underrated.

In which case, the idea of the post-hype sleeper becomes less compelling overall. That’s not to say everybody in the opening paragraph is equally hopeless; Jake Marisnick might still have a case to be on a top 101 prospect list, while Mike Olt took the Gary Brown path all the way off his team’s top 10. Jason Parks frequently notes that prospect rankings are a snapshot of a particular moment in time. Snapshots are amazing: they stop the world for an instant and let us study them in detail, to really explore a scene in a way that we never can when life is moving forward, as it does. Snapshots give us insight that we could otherwise never have. If there is one thing a snapshot struggles to depict, though, it’s momentum. Is this man walking backward, forward, or standing still? It’s hard to know. Two snapshots, though. Two snapshots often do the trick. When we’re talking about prospect rankings, the two snapshots might be telling us something about the third.