Fixing IRP

So, I decided to see if the numbers I was compiling for IRP were actually were making sense. Since the inherent values for the offensive plays were determined using the league's 2009 Run Expectancy Matrix data, I figured that I could use the league's hit (and walk) totals to estimate the number of runs that should have been scored. If the numbers correspond closely with the actual runs scored, it means that we've got something that really does relate to the most important part of the game: touching all four.

IRP runs scored projection: 20394

Actual MLB 2009 run tally: 22419

Error: 9.03%

Well, that's ballpark - Coors Field-sized, but ballpark. I'd feel much better, though if we could get the margin of error under 4 or 5%. So, where's the error source? My first decision is to suck it up and include hit-by pitch (which some players, such as Ron Hunt, Craig Biggio, and Chase Utley, do have a knack for). This immediately brings the error down to 7.29%. That's certainly better, but it does leave a lot to be desired. Then, after making sure I was using Excel correctly, I start the logical thinking process over again. I quickly latch onto the problem: home runs.

You'll recall I spent a paragraph justifying why a home run was worth a run and a run alone. Well, that paragraph was lousy, and I'm embarrassed that such garbage came from head. I completely ignored the central idea behind IRP (and my own stern warning to you that you should remember it): the batter is only responsible for the situation the next guy up faces. The batter who hits a home run absolutely deserves the 1 run currently assigned to him, but he also deserves credit for the runs that will score as a result of not making an out. As I see it, the most important thing a batter can do is not make an out, and I wasn't crediting home run hitters for staving off the next half-inning a bit longer.

So, let's add the run expectancy result for each out situation to the one run already assigned. A home run with nobody out, in 2009 at least, is worth 1.52 runs. Averaged out over the three out situations, a home run goes from being worth 1 run in IRP v.1 to be worthing about 1.3 runs in IRP v.2. What difference does that make?

IRP Runs Scored Projection: 22302

Actual MLB 2009 runs tally: 22419

Error: 0.52%

Now THAT's what I'm talking about. A half-percentage point? Statistically speaking, that's absolutely nothing. Using a season's run expectancy matrix and the hit, walk, and hit by pitch totals, we're able to nail the number of runs scored league-wide. Just multiply the various events by the proper value, add 'em up, and you've got the answer. Just as a refresher, in 2009 those values were:

One-Base - 0.2457

Double - 0.4164

Triple - 0.5825

Homer - 1.3009

(For ease of use, just use 0.25, 0.4, 0.6, and 1.3. It actually comes a tiny bit closer, with the error for league totals being only 0.22% using those approximations, but I suspect that's a random quirk more than anything.)

Now, the question is whether these inherent values remain consistent from year-to-year, and if we can use inherent value data cross-years (using the above 2009 values in 2008, for instance) without significant statistical effect. I tried the latter first, using the 2009 values for each season from 2005 to 2008. Using the exact values rather than the approximations, I got these errors in each year:

2005 - 1.03%

2006 - 2.62%

2007 - 3.30%

2008 - 1.64%

Those are still very good estimates - I'm an engineering major, and we operate under the "5% is good enough" rule of thumb - but they do indicate there is a bit of fluctuation between the exact inherent values over time. That makes sense, of course, since the run expectancy matrixes are similar but not exact from year-to-year, and the trends in the matrixes are over decades rather than single seasons. We can expect the same to be true for the inherent values.

I was curious about those approximations we came up with earlier, for ease of use. Applying them to those same years, we got these errors:

2005 - 0.75%

2006 - 2.36%

2007 - 3.03%

2008 - 1.35%

They all get a little better, and that suggests to me that the numbers bounce around a core value, which the approximations might be closer to than the exact numbers in 2009. With that in mind, I decided to look at the inherent values over the five-year period, 2005 to 2009, using the run expectancy matrix from each season. Here are the results:

Type - 2005 / 2006 / 2007 / 2008 / 2009 / Average / Deviation

One-Base - 0.2596 / 0.2602 / 0.2585 / 0.2495 / 0.2457 / 0.2547 / 0.0066

Double - 0.4254 / 0.4346 / 0.4478 / 0.4239 / 0.4164 / 0.4300 / 0.0120

Triple - 0.6532 / 0.6109 / 0.6207 / 0.6352 / 0.5825 / 0.6205 / 0.0266

Homer - 1.3012 / 1.3171 / 1.3094 / 1.3025 / 1.3009 / 1.3062 / 0.0070

The triple varies the most from year-to-year - not surprising, as the runner on third situation is the most rare of those studied, so there's sample size induced fluctuation. Still, one can see that the numbers are nice and consistent, all plays having relatively small standard deviations. Now I'm going to apply those averages to each individual year and also the numbers for that specific year and look at the percentage error. This should tell us whether we can just declare an average (and make approximations) or if we would be better off using exact, year-specific data. The results:

Year - Specific / Average

2005 - 2.49% / 1.62%

2006 - 1.46% / 0.04%

2007 - 0.87% / 0.68%

2008 - 0.32% / 1.03%

2009 - 0.52% / 2.15%

2005 and 2009 were the biggest outlier years according to that big table earlier, so it's unsurprising that the error for the averages there is higher. In 2006, the average projected a total off by all of ten runs. That's just flat out cool. I think the 2005 specific is off by as much as it because it was a weird year. The triple inherent value is very high, but there weren't that many triples hit (it was virtually tied with 2008 for the fewest in the time interval, and those two years rank far behind the other three), and the one-base value is very low, but there were much fewer one-base plays than in any other year. This means that the real situations were just a bit out of the ordinary, so it becomes a little trickier to predict. I'm not sure, but that seems reasonable. Big picture, though, is that those errors range from "small" to "negligible" to "EUREKA!", so there's nothing to be worried about.

I'm working on providing individual data using these numbers. I plan on using both 5-year and single-season data. For now, though, I'm happy that IRP projects very close to the real value. It is pretty compelling evidence that we can measure the independent value of each play, which is an important step to identifying individual performance. It can also tell us how well each team is taking advantage of its opportunities, or if a player or team is getting unlucky. I see plenty of possibilities. Mostly I'm just happy that the math works out. I'll wrap up by showing you the Independent Run Production and Independent Run Production Average values for the average 2009 big league player.

Average 2009 Major Leaguer (600 Plate Appearances)

1-year IRP: 71.07 runs

1-year IRPA: 0.117 runs/plate appearance

5-year IRP: 73.00

5-year IRPA: 0.120