I don’t believe in open source or the wisdom of crowds. The only peer reviews I need are the ones I get from editors and fact checkers. All that said, I do believe in explanation and kaizen. Since I started at BP, I’ve been publishing Team Health Reports that use a simple red/yellow/green coding system, one that I thought was a simple front end for everyone who isn’t colorblind. I have also realized that there are a vocal few of you that like looking under the hood and who want to know all twelve of the factors that I use, and there are more than a few of you who are trying to reverse engineer the whole thing. Maybe being a Mac guy ever since the days of OS 6 and my Mac Classic has steered me in the other direction; I just want things to work. So let’s meet in the middle and take a look at how the Health Report system is put together, how it has evolved, and how we might figure out how to make it even better.
The base of the system is an actuarial table. You can’t get more boring than that, and unfortunately, it’s the part I can’t share with you. The table is put together by an outside entity, and is… well, let’s just say it’s very similar to the one used to calculate the premium on insuring player contracts and setting workman’s comp payments. Like any actuarial table, it’s simply a matter of presenting risk based on various categories. The most basic categories here are age and position; a 26-year-old pitcher could have a 40 percent risk for injury (these aren’t the real numbers), while a 32-year-old pitcher might have a 38 percent risk. These injury risks are calculated to account for a severe injury-one that would put a player at risk of passing the elimination period of the policy, which is usually 90 days. The injury risk is also based on a three-year period. For a 29-year-old pitcher, it doesn’t care if the pitcher is CC Sabathia or Jeremy Affeldt; it’s a baseline risk.
Because that baseline risk is so broad, it requires adjustment. Sabathia does not carry the same risk as Affeldt, and because of that, I make eleven separate adjustments to the baseline figure. Most of these are very incremental, which is a bit counterintuitive. The baseline risk is broad, yes, but it’s also accurate in most cases. It’s not perfect-outliers like Tim Wakefield and Jamie Moyer throw things off some years, but when you consider that their health status over the past couple of seasons has been roughly a matter of a coin flip, these isolated cases don’t really indicate any faults in the adjustments themselves.
The first major adjustment from the baseline comes from PECOTA. The Attrition Rate isn’t necessarily one that’s predictive of injury, but it is an accurate predictor of playing time that may often be limited due to injury. I add half the attrition rate to the baseline risk before making the smaller fine adjustments. Those adjustments come from ten factors, including team, body mass (instead of height/weight), position change, injury history, recovery time, role change, conditioning, and a small subjective number that I only use when a player is “on the edge,” allowing me to use subjective information to push a player from a “low red” to a yellow (for example) when I can find evidence for doing so. For pitchers, I also factor in workload and a mechanical adjustment based on discussions with scouts and pitching coaches. Because it’s subjective, it’s very small, but it’s my hope that I’ve come up with some kind of consistent number, and so far it’s proven to be very accurate.
This year, as in every year, there will be a new underlying table, and I also have a five-year table with data that shows how quickly players tend to return from injuries-this allows me to make the recovery-time adjustment more accurate. We’ll also be presenting the data to you in three different ways. First, we’ll have the normal Team Health Reports in the format that people are used to and seem to like. Secondly, we’ll be transferring all of the data to a positional health report for those that like that format-each position will have a document, and once a Team Health Report is published, we’ll update the Positional List as well. Finally, we’ll have a spreadsheet showing the ratings for those of you that have early drafts or who just want a quick guide. As usual, I’ll remind you that the colors and the commentary don’t always agree. I may look at the system’s take on a player and flat out disagree, as I did with last year’s comment on Manny Ramirez. In the end, I think both were right.
I’m always looking to improve, so if you have suggestions, you know how to reach me. There’s still some time before you’ll start seeing the Team Health Reports published, but the work is already happening behind the scenes. There’s not only no other system like it out there, there’s nothing else close to it. For the BP subscriber, it might be second only to PECOTA as a reason why they win their fantasy leagues.
Thank you for reading
This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.
Subscribe now
http://www.baseballprospectus.com/unfiltered/?p=1140
I appreciate that Will\'s success depends heavily on his ability to access closed source information, but open source knowledge - properly used and understood - is very powerful.
Hopefully he doesn\'t entirely dismiss the \"peer reviews\" provided by his readers! :)
http://uptime.netcraft.com/up/graph?site=http%3A%2F%2Fwww.baseballprospectus.com
That\'s perfectly fine, and doesn\'t mean you can\'t believe in open source. Open source works fine for some things -- BP included. It just means you\'re not interested in letting everyone know the formula for THR, and if you aren\'t, then that\'s that. There\'s really no need to defend that position and, in my opinion, people who cite \"open source\" as a reason to share the formula are in the wrong.
After all, they\'re forgetting another favorite mantra used in open source -- if you don\'t the way things are done, fork it and do it yourself. ;)
I will say I don\'t believe in the wisdom of crowds, but I\'m a BIG believer in crowdsourcing. I think people conflate the two.
The expression perfectly fits a lot of bureaucratic situations -- people are perfectly willing to make stupid decisions if they can evade or diffuse culpability. My friend started using this as a .sig online, and not long after she did, she was accosted by a Niven fan. This guy was one of those pestiferous jerks who claimed to have read everything that Larry had ever written (even scribblings on cocktail napkins, I suppose), and *he* wasn\'t able to find those words, and therefore *she was lying*. Rather than explain that, yes, she was friends with Larry, and yes, she had heard him say those words first-hand, she yanked her .sig, and that was that. I\'ve used it myself a few times (a Google search will catch some of my usages), yet it has not caught on. Wisdom of crowds? Well, maybe.
I love you, but you sound like an idiot today. OS X is loaded with open source technologies.
Safari in particular traces it\'s origins to the open source KHTML and to this day depends on the open source WebKit to render web pages (as does Chrome).
I\'d also suggest that open source software is more like crowd sourcing than the \"wisdom of crowds\" since it\'s about contributing effort to a project and not just preferences.
And if you\'re against the wisdom of crowds does that mean that you are against democracy, (free agent) markets and the awesomely helpful Amazon.com star ratings?
---
Mac User since 1991.
But I think what you\'re saying is that you don\'t want to distribute the THR recipe (correct me). Which is fine, you own it. I think you were very open about the ingredients, just not how much of each goes in.
You couldn\'t possibly be against open source, because if you purchased a web server, database, application server and operating system my subscription price would go up and I\'d rather pay for the content, not the delivery charge.
So for those of us who don\'t win, are THR yet more evidence at how bad at fantasy we really are?
(prior post was supposed to have content)
:)
A \"wisdom of the crowds\" approach to THR would be more akin, for example, to conducting a secret poll of BP readers, who would each weigh in with a vote as to the injury risk presented by a certain player, and then aggregating those votes into a single value (though such a crowd may not be truly wise, in that it may lack what Surowiecki terms \"Independence\" since each of us, in theory, would be influenced by the same information we consume from BP).
Come to think of it, I wonder whether such crowd-based approach would \"beat\" your secret formula... :)
Especially when one claims to have an amazing recipe, but doesn\'t share the ingredients.