Before the winter Ben Lindbergh asked me to create a list of the top-50 free agents. Today let's revisit that list with an eye toward improvement.
In dissecting the list we have to begin with the two unemployed players that were ranked: Kyle Lohse (ninth) and Jose Valverde (43rd). Two missteps on the list's part, or unfortunate victims of the marketplace? How about one apiece. Lohse has not signed because of the draft-pick compensation requirement rather than his talent (he's fine as a middle-of-the-rotation starter). Were I redoing the list, Lohse would remain at nine. The same is not true of Valverde. He would lose his spot to a more-deserving player. Perhaps Lance Berkman, who went unranked because of the trepidation surrounding his future.
To figure out who else I misplaced, either by over- or under-ranking, I relied upon each player's average annual value of the guaranteed portion of their contracts. (The reason I favored AAV to total value is because of the intent of the list. I ranked players based on their present talent level as opposed to their future value; hence Hiroki Kuroda ranked in the top 10 despite being a short-term investment.) I found the correlation between my rankings and AAV and then compared the expected AAV to the actual AAV. From there I could use the deltas to figure out my most overrated (players who signed for less than their ranks suggested) and underrated (players who signed for more than anticipated). On the whole I was satisfied with how things turned out:
There were, however, a few rankings I'd like back. Let's start with the players I overrated: Mike Napoli (ranked 12th but came in 35th in AAV), Shaun Marcum (24, 38), and Ryan Madson (30, 41). Each of these players boasts an injury history. Not to make an excuse for overzealous rankings but this is one of the big issues with internal versus external valuations of players. Both the pitchers have undergone Tommy John surgery in the past, with Madson still on the recovery road. Napoli's hip issues seemed to drop his price only after he agreed to a lucrative deal with the Red Sox. If it took a team-issued physical to find the flaw then how should any of us know about it in advance?
As for the underrated players: Andy Pettitte ranked 32nd initially but finished 13th in AAV. Pettitte's choices seemed limited to the Yankees or retirement so it seems the team simply met his price. Jonathan Broxton might be the worst ranking of the bunch, seeing as how he improved 18 spots after signing a three-year deal with the Reds. Broxton began throwing a cutter after joining the team (an observation later confirmed by Ryan Hanigan). Perhaps Walt Jocketty and company fell victim to the endowment effect and overvalued what the new pitch means to his odds of succeeding heading forward, but had I known about the pitch at the time he would've slotted somewhat higher.
Zack Greinke and Josh Hamilton were also underrated. Yes, those were the top-two players on my list, but the limitations of the format understated their superiority relative to the rest of the class. Hamilton and Greinke received the highest AAV of the class by $8 million. While they both netted AAV of $24.5 million or more only three others topped $15 million. This may have been a weak class but the market gave the two elite players their due.
The team prediction portion of my list went about as poorly as expected. I got six out of 48 right but you can call it six out of 50 without an argument. In theory you have a one-in-30 shot at getting one player right, which means random chance should see you guess about two players correctly. In reality you should get more than two right. That's because not every player carries a one-in-30 chance; we all knew Greinke would sign with one of three or four teams, and we all knew which teams had a specific need for a certain position. Six out of 50 represents about the bare minimum given that the guesses weren't completely random.
If there are lessons to take away from this exercise, they involve two oft-repeated truths regarding the information divide between teams and outsiders: personal attributes and the endowment effect. Teams know players better than we do—easy enough to acknowledge as true. The second one is sticky. We see teams like the Reds re-sign Broxton or the Dodgers re-sign Brandon League and we snark. But in reality these teams are simply doubling-down on their own investments. Is there a degree of irrationality to it all? Maybe sometimes. But irrationality is part of all marketplaces, and part of the decision-making process. The key is separating true irrationality from hidden rationality. In most cases we don't have enough information to tell the difference, whether we'd like to admit it or not.
Thank you for reading
This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.
Subscribe now
I agree. Does this make transaction analysis useless? If commentary about team decisions has to bear a caveat like this, what is the point of even commenting on them at all? If all 30 teams know everything we do, plus additional information, then the only way we can assess their decisions is by looking at their records at the end of the year.
Speaking from experience I think there types of transaction analysis that are more useful than others. For instance, analyzing the player in question and (in some cases) how he could improve provides more value to me than trying to evaluate a team's spending habits (in part because nobody knows the budget, and because budgets can change with owner exceptions and the like).
I suppose the key is acknowledging you don't know all the factors that go into making a deal. Work with what you have access to and don't try to fake the stuff that nobody on the outside could know about (save the instances where you have a good source, then the dynamics change).