Here's a quick roundup of PECOTA news for the week:
-
Clay has posted the 3/19 depth chart/PFM update. The 3/16 update was accompanied by a tour of the first half of the major leagues through spring training; if you missed it, please have a read. We got enough positive feedback that we wanted to make sure the second half was seen by as many people as possible, and since the blogs system is still a work in progress its going to run as a full article. It'll be published soon.
-
The PECOTA beta hitter cards have had another build which is undergoing review by our newly formed beta test squad. We're hoping to release them in their final form this weekend.
- We are hoping to have a build of the PECOTA pitcher cards released to the beta testers this weekend as well–we're finishing them up right now. Fun fact: we're going to end up with full cards on nearly 6000 players this year.
Thanks for reading, and have a great weekend.
Thank you for reading
This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.
Subscribe now
But really - REALLY - it is MARCH 20th. Almost all of us now have drafts this weekend. That's what I'm paying for - I can't speak for anyone else.
You're STARTING to create pitcher cards? Really? On March 20th? Really? And you think this is acceptable?
Just credit back my credit card and we will call ourselves even for 2010...otherwise....what the hell are you guys doing?
This has nothing to do with the QUALITY of your product.....
You owe me my subscriction fee back because the TIMINING is not acceptable.
If the one-year projections are not enough for your draft prep, or if you have any other questions or concerns, please send me an email at dpease@baseballprospectus.com anytime.
For example, let's say I don't believe Nate Mclouth's weighted mean PECOTA projection. In this case, I might be inclined to use his 30th percentile projection for my own valuation. Without the player cards, there is no way for me to do this. The player cards are important even for those of us that are not looking for multi-year forecasts.
Let me put it this way: if most of the value of the fantasy or premium subscription for our subscribers who use our stats for draft prep is rolled up in the PECOTA cards, we surely need to work on an additional product offering, because I can't imagine that being efficient.
In other words, PECOTA was worthless weeks ago, and for us -- the subscribers -- to have any sense
of it having been reconciled, it would be nice if you guys would actually fess up to what in the world you did to screw these numbers up so badly and what you did to fix them. Sweeping it under the proverbial rug would not be a good first step, I would think.
I think it's telling that in 2003, when I was first introduced to the site, Baseball Prospectus told me everything I needed to know about statistics and value and the great world of sabermetrics in baseball. Now? I only come here for prospect reports phoned in by scouts. I think the message has been lost somewhere, gentlemen.
As far as A-Rod slugging .700, well, stranger things have happened. People forget that Alex Rodriguez is truly a once in a generation talent. Mickey Mantle, Hank Aaron, Barry Bonds, Babe Ruth, Honus Wagner level talent.
We're all paying for this stuff, Dave. I understand that BP is going through growing pains trying to make the system better and more efficient. And I'm confident that once this is over, we're going to end up with a better PECOTA. I get that.
But in the interim, some of us are entering fantasy drafts with hundreds of dollars in entry fees. And we pay BP for a quality product. I appreciate the updates, but, at minimum, an acknowledgment of what needs to be fixed would be appreciated. We've all figured out on our own that it has to do with outlier projections and multiyear projections, but an acknowledgment from BP as to what on Earth is going on would help out your paying customers.
One of the problems with this process has been the perceived lack of transparency. You come onto the blog and tell us you're working on it, but you don't say what the problems are. You owe it to paying customers to give us a little more detail.
Thanks,
Chris
Second thing. The reason the PECOTA cards have gone to the Beta testers is that, so far as BP knows they are now ready to go. First they'll see whatever (hopefully minor, data misentry type) things might be wrong with them.
If you're not a keeper-league player or someone who's drafting minor leaguers, there's nearly nothing you're going to get from the new hitter or pitcher cards that you don't already have access to.
I think the transition issue has had less to do with platform and more to do with the intricacy of the system that used to be maintained by one person who designed it (Nate Silver) and now not only has to be ported, but had to be learned in the process by the people doing the porting.
It certainly sucks to feel like PECOTA is less than the most reliable rock you have available to lean on that it's been in the past, but I do think the histrionics are excessive: if you have access to the book projections, only a significant minority of fantasy players do not have access to the full data they need to draft this weekend.
Those of you who are pushing should keep in mind that the situation is as it is: an early release is just more likely to include bugs or problems or simple miss-links or accessing of old databases instead of the newer versions. Would you take a release today in exchange for a potential major database error, or a monday release without one?
Also, most fantasy publications I know had their very rock solid and very usable projections out in December (some were even ready in November!) That BP, in mid-to-late March, is still fixing (not tinkering, FIXING) their 2010 projections to approach even moderately believable levels of accuracy is beyond incomprehensible. Fantasy drafts are over. They're done. There is no point in owning BP's fantasy membership at all in 2010 any longer. It's a sunk cost.
I have loved Baseball Prospectus since the very first day I ever visited the site in 2003. But the quality control in the past two years, and most especially this year, in terms of their fantasy product has been absurd. If I had paid for a fantasy subscription, I would be absolutely livid right now. The fantasy product's time for usefulness has come and gone. Period.
As far as I'm concerned, BP has had well over a year to iron out the kinks in PECOTA's transition for Nate Silver's Excel files to the new program format. It's a year that they've apparently spent twiddling their thumbs instead of paying attention to glaring, outrageous errors in their system's output. And the damage control has been almost as outrageous, too.
The cost of PECOTA's performance and greater information flow has always been that it takes longer to produce, and while it is clearly a problem this year - no doubt about it.
Equally, while the issues this year are clearly a loss, its quite apparent to me (admittedly, not a serious fantasy player) that the fantasy content is recieving a huge influx of attention, staff and time.
If you feel this time has been spent on thumb-twiddling, it is my belief you have no personal first-hand experience with excel-based programming and converting it into a real programming language. PEOCTA took Nate years to create, and it essentially had to be re-written from scratch into another language and every calculation (which, as was mentioned, used to take over 72 hours to process) verified to be working identically to its excel counterpart.
It was clearly a mistake that BP expected that they could do this over the winter with part-time work. That they addressed that too-late doesn't mean they didn't make the mistake - but it does mean that the mistake was already addressed over a month ago through Clay's full time employment to handle this specific task.
My point was more that there really is nothing to gain by complaining and pointing fingers over a series of problems that were already acknowledged and have already recieved attention. Unless you believe that the responses will prove to be inadequate (and it's really too early to tell if they're going to work for the rest of the year or not), all you're really doing is telling them to be ashamed of themselves for making mistakes in December and January and trying to correct them in February and March. Clearly, they are, but you're not really contributing anything constructive by doing so, since they've already taken steps to address pretty much every one of those issues.
People need to vent.
The comments here and elsewhere are serving as a forum for venting frustration and disappointment. BP sets a pretty high standard for itself and has clearly fallen short of expectations. I have worked many years in customer service and sometimes an organization just screws up, drops the ball, chokes, ignores a problems, lets things spin out of control, gets overwhelmed etc. And when this happens you try to make things right with your customers, but you sometimes just have to take your medicine and let them vent their frustration.
Then hopefully you have another chance to earn back their trust and confidence. If so, you both move forward together. If not, you both move forward separately.
Mike, it's clear that you have no interest in fantasy content and are not affected by the lack thereof at all. But for the thousands of subscribers that BP has that do look to BP for fantasy projections and content, the lack of any semblance of a timeline or reliability in the services offered has presented a very real issue that has not been voiced loudly enough in my estimation. That you are content with an ever-drawn-out delivery of the site's product doesn't make it okay. Your lack of fantasy interest has nothing to do with the thousands of people who paid for subscriptions to receive a product that has been virtually worthless since it's unveiling.
I'm not trying to shame BP for the errors they've made. I'm trying to make the point that their efforts are now essentially useless for a fairly large number of people who paid for their services. That you think BP has successfully addressed these issues with their opaque transparency and their expanding timeline is in many ways more preposterous than some of the numbers PECOTA has been spitting out for the past couple months. The only step that could possibly address the problems at this late in the fantasy game is either a refund or an upgrade.
We're not done talking about the PECOTA process and the lessons we've learned from it yet, but our top priority is getting the PECOTA package completely released.
PECOTA has never strictly been a fantasy tool and in fact wasn't initially introduced as one. It's a forecasting tool and one that I, as a fan, spend a ton of time with in order to better inform my baseball watching experience.
While my use is a bit less deadline-sensitive than yours, I've certainly lost out on a big part of why I pay for a subscription.
Nonetheless, as someone who's coded many an excel calculator, I empathize with what they're going through right now because I appreciate how difficult it is to pick up someone else's excel sheet and modify it without breaking it.
Do many of us love and support BP, and rely upon its PFM and PECOTA for our drafting? Yes
Do we sometimes look at a projection and think, "that doesn't look right" (either good or bad), and discount it from our draft sheet? Yes
Do we also seek out other fantasy draft projections at the same time? Yes
Is $40 for a year of all services a good deal, or $20 for fantasy features only a good deal? Yes
Has BP botched the normally smooth rollout of the PECOTA? Yes
Is this the first major glitch most of us have experienced in our dealings with BP? Yes
Do customers have a right to be disappointed/angry about this? Yes
BUT .... let's realize that there has been a lot of personnel turnover at BP during the last year, and they're trying to automate an Excel process featuring tons of data and inter-related algorithms.
Just my opinion here, but if you are playing fantasy sports for "thousands" of potential dollars, perhaps you've lost sight of the "fun" that is supposed to be involved.
Fantasy sports are supposed to ADD to one's enjoyment of the game. Spewing vitriol at a supplier of information used in fantasy gaming may be cathartic for the venter, but after a certain point, it only turns into static.
Like some of my favorite players, I'm willing to give BP an "off-year", and I'm willing to pony up another $40 to give them another chance.
Comeback player of the year, 2011 perhaps?
Look back over my recent comments:
1) I lobbied for, and got, a customer-based "beta-testing" group for PECOTA.
2) I've lobbied for, and have been promised, a unique ID system for much if not all fantasy content, which will enable easier merging of disparate data such as THRs, PECOTA, PFM, and the Weighted Means Spreadsheet.
3) I've lobbied for a downloadable form of the Annual. (Though various members of BP staff agree with me on this matter, its out of BP's hands. Its a publisher's decision, and they haven't been able to convince a publisher to go that route yet.)
I simply believe in the mission and performance of BP more than any other independent baseball research site.
I'm not on BP's payroll. I'm not kissing their asses. They appreciate my input.
What do you bring to the improvement of this site?
"Like some of my favorite players, I'm willing to give BP an "off-year", and I'm willing to pony up another $40 to give them another chance."
Felt like you were directly stumping for subscriptions, which seems a little strong to me. Why frame it in that context when defending BP? I guess it's the form that a lot of these complaints take...
I totally agree that BP delivers great content overall. For fantasy this year, though, they've been subpar, and they should face the music for that.
This leads to a tendency to defend the difficulties inherent in it because we have a little more perspective on how it had evolved and how difficult it must have been to transition it from Nate's private excel sheets to a real program run by multiple people.
Everth Cabrera no longer appears to be a Hall of Fame caliber player (although his #6 comp is still Hanley Ramirez, which is kinda of fun to think about).
Matt Weiters' comps, sadly, are still a bunch of guys who never really had career as MLB starters. Would love to know why PECOTA "thinks" this.
When the release is done, could you guys offer some commentary on the more controversial projections/comps?
Wieters is an uber-prospect who had great minors numbers but who's MLB abs didn't nearly live up to the potential. There are, frankly, a lot of guys like that who never panned out, and its not entirely within PECOTA's capabilities to weed out this uber-prospect from all the others who looked like it and fell flat the second they hit real major league pitching. Another season will probably differentiate Wieters from those guys, but his Baltimore numbers were not really those of the kind of player he's suspected of being and are, frankly, quite in line with a lot of people who we'll all hope he isn't.
How many big-bodied, great-bat catching prospects wash out and turn out to be significantly inferior hitters to what people talk about? Piazza types are the exception by a long shot, and for every guy who looks like he could be one in the minors and hits the majors and is great, there are literally dozens of guys who would be lucky to get Dioner Navarro's career to date. Remember when he was an uber-touted catching prospect? Not quite Wieters level hype, but he was expected to be all-star caliber, remember?
Regarding PECOTA, I understand it will always be subject to improvement, and fully support that it continue to be improved. In my opinion, though, it's simply timing. That said, here is my expectation / proposal:
- Continue to work on improvements during the year, but with a clear "stop tinkering/improving" date that is WELL before season's end (e.g. Stop tinkering in October). And for gosh sakes, take full advantage of those volunteer beta test guys. A group that I would be more than please to be part of.
- Once the 2010 MLB season is over, simply add the 2010 data to PECOTA, knock out the FINAL version of the cards, and know that BP customers everywhere will rejoice.
- Repeat the cycle annually.
Although the above is clearly simplified, I think it makes the point.
I understand that PECOTA will never be "good enough" as long as there are brilliant minds with ideas that will improve upon it. But a very good product that is timely is much more valuable than an "even better" product that doesn't reach the shelf in time.
My guess is that the poorness of second basemen makes utley the best. Its far easier to get a player like wright at 3b than it is to get performance of that quality at second base.