Measurement Error

Over the summer, I developed this dumb joke with one of my friends. We started saying, “Oh, you can’t measure that,” like a color commentator in response to almost anything. Initially it sprung out of vaguely appropriate situations – I think it started during pickup basketball games — but it slowly spread outside of the context of sports. The waitress repeats the order back to make sure it’s correct – you can’t measure that. One of our friends shows up five minutes before we do – you can’t measure that. Even things you can measure, like how fast I would be driving, someone would say it – you can’t measure that.

Look, I said it was a dumb joke.

Not to deconstruct this too much, but as many of you might know, “You can’t measure that” is one of those phrases sports broadcasters seem to love. Football broadcasters, baseball announcers, basketball, anyone who talks about sports on TV has probably said it. You can’t measure heart. You can’t measure that hustle. You can’t measure clubhouse chemistry, all the Tostitos, how many licks it takes to get to the center of a Tootsie Roll Pop, all those things. It’s cliched beyond the point of meaning, and half the time it seems to be used about things people actually can measure. (I believe last season an SNY broadcast mentioned the threat of Jeff Francoeur’s throwing arm in right field as something you couldn’t measure. The effects of a outfielder’s throwing arm are actually one of the easiest things about fielding to measure.) So whenever I hear that you can’t measure something, it tends to elicit an eye roll from me, as I imagine it does from others.

But as cliched as a phrase may be, it is true that there are some things can’t be measured. It’s not because these things are inherently beyond quantification, but rather because they’re beyond our current abilities for quantification. The world is an immensely complicated place, well beyond the abilities of our monkey brains to understand in any real way. We have enough trouble figuring out what the weather is going to be like in three days, nevermind the world as a whole. Doctors washing their hands is a relatively new development. We’re really not that smart. But there are answers out there for everything. There is a concrete value to everything. Whether or not we’ve figured out how to measure that value is another story. But the answers are out there if we can find them.

So, for example, hustling. There is a concrete value to hustling in baseball. I don’t know if we can measure it quite yet, but we can start and get closer than you might think.

Jason Bay, as much as he’s struggled this season, has run out every ground ball he’s hit. I think everyone would agree that Jason Bay hustles. Maybe we can’t measure all of that hustle, but we can measure some of it. Baseball-Reference tells me Bay has reached base on seven errors this season. Some of those are from a shortstop throwing the ball into the seats and might have happened with any runner. But maybe some of it is from Bay forcing hurried throws across the infield while sprinting down the line, so some of it could be the effect of hustle. And in their calculations for wins above replacement, Baseball-Reference compares the number of times Bay has reached via an error with all the number of times other runners in the league have reached via error, and says that those seven times Bay has reached base are worth about one run compared to an average player (who probably reaches five times). So there’s one run just from hustle.

But Bay has also hit into just seven double plays in 95 opportunities, a low number. Again, maybe some is dumb luck and some is just the type of hitter Bay is. But maybe running hard down the line played a roll. It’s easier to turn two with a jogging runner compared with turning two on a runner sprinting down the line. But Baseball-Reference again measures how many double plays a batter hits into, compares it with everyone else, and says that Bay has been two runs better than an average player in terms of avoiding double plays.

So right there, between reaching on errors and staying out of double plays, Jason Bay has created three runs from running down the line every play. Maybe it’s not a complete measurement of hustle, but it’s a start. It’s possible to measure some of the effects of an individual player hustling in his statistics. Some of it is, in fact, quantifiable right now.

But our ability to measure any effects of Jason Bay’s hustle break down from there. Let’s say, for example, some of the younger Mets players copy Bay’s hustle, and now they bust it down the line, too. Monkey see, monkey hustle. The kids beat out a few more ground balls, stay out of some double plays, things like that. It’s going to show up in their statistics, and in the team’s statistics — but it’s not going to show up in Bay’s statistics. So if that were happening (and I don’t know that it is, but if it were) that would be an effect of Jason Bay we can’t properly measure and attribute to him. It very well could be happening, but it would be difficult to isolate and measure objectively.

(I will point out though, as we saw above, Bay’s hustle is probably worth three runs. So even if we say all the Mets’ hitters hustle because, and wouldn’t have done so without, Jason Bay, we can realistically credit him with 30-35 runs at the most. That’s three games in the standing, a sizable jump. But it’s not the difference between an 80 win team and a 95 win team, as announcers and fans sometimes imply. My educated guess is that contagious hustling, as a team-wide event, is the difference between an 80-win team and an 83-win team at the absolute extremes, and is normally something much less than that. So it matters, but you probably can’t hustle your way from mediocrity to excellence.)

Basically, we are very, very good at measuring the actions of a single player when he’s on his own. This is why hitting statistics are so accurate: It’s just one dude at the plate. He hits a home run, you credit him for hitting a home run. We are nowhere near as good at separating and measuring the actions of members of a group. Fielding and pitching statistics are getting better, but they’re still a mess in certain areas; advanced basketball and football metrics face this problem for just about everything, so they’re similarly dicey. We’re not just not very good at separating out the individual contributions of teammates when they’re interacting. It’s trying to learn objective information from really, really messy experiments, like trying to dissect a frog while the other kids in your freshman year biology class throw frog parts at each other.* It’s difficult and often times impossible.

*I went to an all-boys high school, and that is a true story. Then when the teacher left the room for a moment to get more frogs, one kid hid a partially-dissected frog in another kid’s bookbag.

So there really are things we can’t measure right now. Maybe a more relevant and real example: Bobby Parnell has started throwing a knuckle-curve, Jason Isringhausen’s signature pitch. Parnell threw seven during Wednesday’s game against the Cardinals, four for strikes, three of them swing-and-miss strikes — Parnell struck out Adron Chambers with one in the dirt to end the ninth. I imagine it’s easy to surprise hitters and get swings-and-misses when you bring out a brand new pitch they don’t know you have. But, hey, at the very least Parnell looked like a useful pitcher again. Little steps. Maybe it’ll make a difference for him.

According to the SNY broadcast, Jason Isringhausen has been working with Parnell on the knuckle-curve. It seems reasonable that Isringhausen’s experience and knowledge has real value here, though I have no idea how much. And we can’t measure that value, because it’s really difficult to separate Parnell’s ability to throw the pitch from Isringhausen’s ability to teach it. It seems obvious that some credit should go to Isringhausen, but how much is the question. I have absolutely no idea, but it seems like there should be some. But that’s something beyond our abilities to measure: The Bee Gees can’t measure how deep your love is, and we can’t measure Jason Isringhausen helping out Bobby Parnell.

That is the real limit of statistical analysis. The issue with sabermetrics is not that it takes the mystery out of the game — being well-informed is always a good thing. Saying otherwise is anti-intellectualism, and I feel quite strongly that we shouldn’t tolerate such arguments in any area of our society. The option of more information, whether one choose to use that information or not, is always good. Plus, I can’t imagine anyone has ever said, “Well, I understand FIP now. That’s it, I completely understand the game of baseball. There are no more things about baseball for me ever to learn. All the wonder is gone.” That’s not the problem with statistics.

But what seems to happen is this: Statistically-inclined writers and fans sometimes dismiss things with a value that can’t be measured as having no value. Almost saying, “if we can’t measure it, that means it must not exist, or is at the very least unimportant.” And that just strikes me as wrong. There is some value in running out every ground ball, even if we can’t objectively measure it. There is some value in having an older player show the younger players new pitches, even if we can’t objectively measure that value. The manager of a baseball team has value, even if we can’t figure out how to measure objectively their effects on the team. Just because a thing is beyond our ability to measure doesn’t mean that thing has no value. It’s not all waiting on us to figure it out.

Statistics are not lies. Numbers don’t lie. That’s simply not true. A person can misunderstand a statistic, misunderstand the predictive power it might have. If a hitter has three hits in four at-bats against a particular pitcher, that’s not a good indicator he’s going to hit .750 against him forever. The statistics don’t mislead, for they’re only inanimate numbers. But a person can misunderstand them and misuse them. It’s important to know what the numbers are measuring, and it’s just as important to know what the numbers are not measuring. Because while there are things we aren’t capable of measuring, at least not yet, that doesn’t make those things any less real. Baseball is a whole big game, and limiting one’s viewpoint to just the quantifiable or the thus-far-unquantifiable is missing half the picture. And half the picture – you can’t measure that.

About these ads

7 Comments

Filed under Columns, Mets, Words

7 responses to “Measurement Error

  1. We’ll never be able to measure everything while the players still think and don’t behave like robots because we’ll never have all the inputs.

    Take the effect of Francoeur’s arm. Over the long haul, you can get a sense of it, But we don’t have as much data on how long it takes the ball to get to Francoeur, and where he’s positioned. Maybe his pitching staff sucks, and gives up more line drives. hits to RF get to Francoeur faster than they do to other RFers, therefore the third base holds up more runners trying to score from second because he usually has the ball sooner. That’s a measurable statistic, hit speed, but not one I’ve seen around. Maybe Jeff plays 150ft from home plate with a runner on second, whereas someone else plays 154ft. Marginal difference maybe, but sometimes it’s a real close call atthe plate.

  2. “being well-informed is always a good thing. Saying otherwise is anti-intellectualism, and I feel quite strongly that we shouldn’t tolerate such arguments in any area of our society. The option of more information, whether one choose to use that information or not, is always good.”

    As this statement relates to entertainment (including sports) and the arts, I don’t think it’s remotely true. Aesthetic enjoyment hinges on balances. Any hard maxim like this will never be borne out. Someone will find a Latin mass more beautiful than one in his own tongue. Most will prefer to know nothing of the preparation of his sausage. The overheated backstage of a ballet recital is KEPT backstage for considered reasons, and the stage is lighted to create a break from reality and its prosaic information.

    Likewise, some people find baseball more enjoyable AS AN ENTERTAINMENTwith sentimental narratives of heart, clutch performances and gritty play, as opposed to realist narratives of random variation, regression to the mean, etc.

    Every GM should be 100% saber, there’s no doubt. But I think we’re very wrong to assume people should enjoy baseball the way we do.

    • “being well-informed is always a good thing. Saying otherwise is anti-intellectualism, and I feel quite strongly that we shouldn’t tolerate such arguments in any area of our society. The option of more information, whether one choose to use that information or not, is always good.”

      What about the Things That Man Was Not Meant To Know? What about Curiosity Killing The Cat? What about Eating The Fruit From The Tree Of Knowledge? Dude, when man delves too deeply into forbidden knowledge, you never know what might happen — Rlyeh rises, frogs rain from the sky, dogs and cats start living together, pandemonium reigns, the four horsemen rise… in short, you have a ghastly mess.

      Don’t be that messy guy.

    • Patrick Flood

      Curtis, Pack — I think you’re both right and I’m wrong here, because I agree with everything you just wrote. Speaking as someone who has been in the clubhouse on occasion, and has different “how the sausage is made” knowledge from that — I don’t know if that has made me enjoy the game less, but I enjoy the game differently now. Knowledge isn’t always a good thing.

      At the same time, the more I learn about baseball, I become more and more overwhelmed by the number of things I don’t know. I don’t think it’s possible to ever drive the wonder out of anything.

      I guess what I meant to say, or should have said is . . . everyone should enjoy the game however they please, and no one should tell them otherwise. BUT I also think, and maybe I’m wrong, but I think we have a society-wide problem of people arguing for positions that can easily be refuted by facts. I don’t mean about subjective things, like art and beauty and whatever, where it’s all subjective. I mean things like if Derek Jeter is a good fielding shortstop, or important things like climate change and basic economic issues. I feel we should have less tolerance for the rejection of facts and science than we do now, particularly when it comes to important issues. That’s what I meant to say.

      • My apologies, Patrick. I had intended my reply to Pack to be so over-the-top that it would be obvious I was poking fun at him, but apparently it came off as agreement instead. Next time I’ll just say what I mean. For my true feelings see my post below.

      • Patrick Flood

        I thought it was over the top, but I didn’t want to assume and be wrong . . .

  3. (should be ‘the four horsemen ride’)

    Back to Bay, it’s possible your three runs created still sells him short.

    What if his hustle resulted in some extra bases being taken — stretching singles or doubles — that an average player wouldn’t’ve gotten? (I’ve seen maybe half of Bay’s plate appearances this season, and I’m doubting that one, though I’ve absolutely seen other Mets lose bases due to lack of hustle.)

    What if his hustle caused him to earn an extra infield hit or two? I can’t swear that he did that, either, but I know I’ve seen him make it photo-close on numerous occasions.

    What if his hustle has resulted in him moving up a base in situations when the average runner wouldn’t? Advancing on groundouts or flyballs, steals (which he’s nearly perfect at) advancing on overthrows or rundowns or throws to other bases — now here he must be ahead of the norm.

    I agree that the whole thing probably doesn’t amount to a hill of beans, and might not even if everyone on the team did (which they most emphatically DO NOT), but… it’s the principal of the thing. Trying hard isn’t worth $16.5 million a year, but at least I don’t feel he’s insulting me.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s