How are Runs Really Created - Second Installment
by Tangotiger
Understanding the impact of the out
In the world of sabermetrics, I think there is nothing more misunderstood, misapplied, misinformed, mis-everything, than how to handle the run impact of the out. You have your Pete Palmer fans on one side, where they consider the run value of the out to be about -.27 runs. And you have the Bill James fans on the other side, where they consider the (effective) run value of the out to be -.10 runs. Who's right? Would you believe both?
Recap plus something new
Remember what we said about how runs are created: the value of the event is the marginal impact on a given specific run environment. The values I presented therefore are perfect, under the above condition. The out therefore must have a value of -.27 runs. A perfect game however does not have negative runs scored, though.
The other thing that we learned is that we can break up the various events into two parts: (1) value of getting on base, and (2) the value of moving runners over. There is actually a third component that only applies to the out. The value of keeping the inning alive. When a player makes an out, he is reducing the run potential of all future batters. He's giving his team 2 outs to work with instead of 3. Let me present part of the table from the last article which showed the run value of all the hitting events, but broken up into the 3 components above.
Run value of major hitting events, broken up by its building blocks, 1974-1990
Actual | "Theoretical breakup"
|
Event Total |Getting on Moving over Inning killer
--------------------------------------------------------------
single .46 | .25 .21 ---
double .75 | .41 .34 ---
triple 1.03 | .61 .42 ---
homerun 1.40 | 1.00 .40 ---
walk .30 | .24 .06 ---
steal .19 | --- .19 ---
CS -.44 | -.26 -.02 -.16
out -.27 | -.01 -.10 -.16
The building blocks of run creation
Let's take these one at a time. We know that a random single will add an average of +.46 runs to a game. We also know that about 25% of the time, a player who reaches first on a single (or is replaced there by a pinch runner or a force play) will score. That leaves .21 of run potential that a single adds to the runners on base. (This can also be calculated exactly, as we have shown with the walk. It's a much more cumbersome calculation to be sure.) The double follows a similar pattern. The triple and HR are interesting in their differences. While we know for certain the getting on value of a HR (1 run), why would the "moving runners over" value of the HR and triple be different? Again, this relates to these values representing the weighted average of their moving over value. While they have the exact same values in each of the 24 base-out states, because the average HR occurs slightly more frequently with bases empty than the triple does, the triple occurs more often with men on base. The walk was discussed in the previous article.
The stolen base (and all of its brothers like balks, passed balls, etc) are interesting in that they have no "getting on" value. Their value lies entirely in moving runners over.
Outs and caught stealing
Here is where it gets interesting. The out. Let's study the caught stealing. Three very important things happen when a runner is caught stealing: (1) a runner is removed from the base path, (2) the run potential of all existing runners is reduced (i.e., a runner on 3B has a much greater chance of scoring with 1 out rather than 2 outs, which is what would happen if a runner is caught with 1 out), and (3) it gives the team 2 outs instead of 3, thereby reducing the potential number of future batters coming to the plate. Let's take these things one at a time.
A runner prior to attempting to steal, will have a 26% chance of scoring. If he is unsuccessful, his chances of scoring are reduced to zero. Therefore, he erases a runner from the bases (the "getting on" part) and that costs his team -.26 runs. Sometimes, a runner steals when there are other runners on base. A runner at 3B with 1 out has an excellent chance of scoring. With 2 outs, his chances of scoring are reduced greatly. While the impact of this is high, the frequency at which a CS occurs with runners on base is low, hence the small -.02 run value impact of "moving runners over".
The last part is the reduction in run potential of all future batters. Since the average game in 1974-1990 had 4.3 RPG, this means that each inning produced .48 runs per inning. Therefore, each out reduces the run potential of all future batters by .16 runs. That's the "inning killer" value. Add up all three values, and we get -.44 runs. What is very important to realize is that -.44 value can be calculated in 2 independent ways (through a simulator, or looking at the difference in run expectancy). We now have a third independent way (loosely related to the second way). We now have the building blocks of run creation.
A regular out will sometimes remove a runner from the bases (like a GIDP), and in hindsight, I should have split up the out event into "out 1", "out 2", "out 3" to denote the type of out. In any case, the "getting on" value is a -.01. For the "moving runners over", while the out will sometimes move a runner from 2b to 3b, most of the time, the out will reduce the chances of the existing runners from scoring. That effect, which is probably the most complex calculation we have, is -.10 runs. The "inning killer" value has been discussed with the CS, and it is -.16 runs. The total comes in at -.27 runs.
Important note to remember: all of these values apply ONLY to the 4.3 RPG environment of 1974-1990. Every run environment (whether at an era, league, team, or batting spot level) will have its own run values for the hitting events.
Reconciling Linear Weights and Runs Created
If you take the "total" run values for each event and multiply them by how often each event occurred, what would you get? Zero. (Partial innings actually prevents this from being totally true, but it is virtually zero.) This is exactly what would have been expected, since we did say that each offensive event is the marginal impact on a given specific run environment. Since the run environment was 4.3 rpg, then we would expect that all the "good" and "bad" offensive events to have a marginal impact that would cancel each other out overall.
Now, what if you just want to figure out the actual runs created (absolute), and not the marginal runs over average (relative)? If you just take the "getting on" and "moving over" values, and multiply that in the same way by how often each event occurs, you get actual total runs. This is again expected, since we know that we have a -.16 run value for the "inning killer" component, and with 27 outs, that gives us -4.3 runs. Since we are removing this third component from the calculation, our overall total will give us 4.3 runs.
So, the entire difference between absolute runs and relative runs is the -.16 value for the out.
Another interlude
At this point, the detractors of Linear Weights come in. They claim that a slightly below average full-time player, who by definition will have a negative linear weights run value, will "have less value" than a part-time slightly above average player, who by definition will have a positive run value.
First of all, we are not talking about "value". Secondly, linear weights is a relative scale, like Celsius is a relative temperature scale. 40 celsius is not "twice as hot" as 20 celsius. Relative scales can not be used in these types of computations. All that linear weights does is say what the run value is relative to the given specific run environment. This is how every offensive event gets its value. A walk is only worth what it is, because of the given environment.
If you need to extend this methodology to individual players, where playing time is another variable, simply present the player's "getting on + moving over" value (absolute runs created), and his "inning killer" value, and his "total" value (relative runs created, aka linear weights). While the slightly-above average part-time player did in fact contribute more in a relative sense, the full-time player contributed more in an absolute sense, without accounting for his "inning killer" value. This would be wrong to do, since those outs actually cost the team potential runs.
The concept that is required is something called "replacement level" and is outside the scope of this article. I also do not propose that players be valued, listed, ranked, or judged by their linear weights (or runs created) values. I only propose that we present the values with an understanding of what they mean. The basis of valuing individual players is also outside the scope of this article, but the concepts presented here (along with replacement level or playing time) would be an integral part of any valuation scheme.
Bringing it all together
You either love math or hate math. After I'm through, you will feel more strongly about it, either way. Let's focus on what we know, in a series of true equations.
plate appearances = batter reaches base + batter hits home run + batter makes out batter reaches base = runner left on base + runner scores + runner makes out
Given only these two equations, how can you represent the number of runs scored? Even without those equations, you would apply a little common sense and come up with:
runs scored = batter reaches base x runner scores rate + batter hits home run which we can expand to R = (1B+2B+3B+BB+HBP+CI+RBOE+OtherSafe) x score rate + HR which we will reduce for ease of explanation for the balance of this article to R = (H-HR+BB) x score rate + HR
Now, how do we determine the "score rate"? That is, how can we figure out the percentage of times that our runners on base will score? A rate should always return a value between 0 and 1. A rate can be defined for our terms as:
rate = successes / (successes + failures) which I will use as shorthand as rate = B / (B + C) which we will apply to the Run formula as R = (H-HR+BB) x B/(B+C) + HR
We also know that the building blocks of run creation is made up of: (1) getting runners on, (2) moving runners over, (3) inning killers (the out). In the above formula, the first building block is represented by the first and last terms. Therefore, the score rate must be made up of the other 2 building blocks: moving runners over, and the out.
The "B" is the successes, or some combination of hits, HR, walks, etc, and the "C" component, the failures, is undoubtedly the out.
David Smyth's BaseRuns
David Smyth comes into the picture, about 10 ago, and reasoned essentially all this in his head. He then tried to weight the various components in the "B" to come up with something reasonable. He called his equation BaseRuns. Let's flash forward back to today, and here are the weights that I assign based on my testing:
B = .8 x 1B + 2.1 x 2B + 3.4 x 3B + 1.8 x HR + .1 x BB
When I first saw David's "B", I was skeptical. Could it be that easy, and could the value of the HR be less than the 3B in this part of the equation, yet still maintain that the overall value of the HR is greater than the 3B? The short answer is yes.
Let's look for the long answer. If you recall from the first article, I mentioned that we can determine the impact of a single event by having a controlled system with known inputs and output, and then introduce this single event into the system and see the result. Let's call this methodology the "plus 1 method".
Using the 1974-1990 as the controlled environment, and applying the above "B" equation to the true known Runs equation, we get an estimate of runs scored that is exactly equal to the actual number of runs scored. This was fixed though. Through a slight process of trial and error, I changed all the above constants until I got my answer. However, what would happen if instead of using the 53,874 HR that were actually hit, I plug in a value of 53,875 HR (the plus 1 method)? What estimate of runs scored would BaseRuns give? The result is that the difference in the output between the new environment and the known controlled environment is +1.40 runs. Look familiar? That is also the actual known marginal value of a HR, using a simulator, the RE method, and the building block method.
The trial and error method however also used this knowledge. I wrote a program that actually calculated the coefficients in the "B" equation using the established run values of the offensive events of 1974-1990, such that the overall runs scored of that time period matched my estimate using BaseRuns.
Assessing where we stand
Okay, so where do we stand? We know that what we are after is the rate at which runners score, given all the input variables. We know that when using the plus 1 method, the marginal impact of each event in the 1974-1990 environment must correspond to the actual run values, previously published. If we accept BaseRuns' theory on how the rate at which runners score is calculated (B/(B+C)), then we have to provide the coefficients in the "B" equation which corresponds to an actual known environment (1974-1990), and is verified against the plus 1 method. We've done this above.
What we don't know, yet, is if we can apply this equation to other environments, or to even extreme environments, like a team of Pedros or a team of Bonds or even weird profiles like Rob Deer. All we have to do is run our simulator. We will create various run environments, apply the formula that was fixed for the 1974-1990 environment, and see if it holds up against these other run environments.
To be continued...
Note: David's concept of the "B" equation is simple, elegant, accurate. It would be interesting if other analysts come along who can come up with different variations of the B equation (or the score rate component). While it is possible that a more precise version can be formulated, it is probable that the simple and elegance of the linear version would be lost. I have already reduced its elegance when I will introduce a more complex version that includes RBOE, PO, WP, BK, etc.
August 20, 2002 - Patriot
(www)
Hmm...this article has been posted for more than a day and there is no response. How come everyone yawns when Base Runs is introduced?
August 20, 2002 - tangotiger
(www)
(e-mail)
I've gotten a few response emails with some nice remarks, but I guess there's not much controversy in what I'm saying.
August 24, 2002 - David Smyth
Tango, don't let the lack of response deter you from completing this project. It is important to get all this down on paper in one place. Combined, it will become an important source for proper RC understanding.
August 25, 2002 - Chris Dial
(e-mail)
Tango, trust me, it's not yawning -- it's digesting. I am thoroughly enjoying the work.
August 25, 2002 - Anonymous
Tango, if you recall the discussions on that primer-methods forum, it was the position of several well-qualified analysts that models don't matter--all that matters is accuracy in the range of interest. To them, the range of interest was the 3 to 6 R/G range of real MLB teams. To me, and to you, the range of interest is 0 to infinity R/G.
In your next installment (or the one after that :)), I hope you delve into the subject of how to apply a successful team run formula to individuals. I believe that this area has NOT to this point been analyzed properly, and I am curious as to what you have come up with.
August 25, 2002 - tangotiger
(www)
(e-mail)
Tango, don't let the lack of response deter you from completing this project. It is important to get all this down on paper in one place. Combined, it will become an important source for proper RC understanding.
David, I agree with your sentiment. I have a half-dozen other projects that I don't yet know the answer to, and therefore have more interest to me (things like when to actually walk Barry Bonds specifically, based on score, inning, men on base, outs, and batting order). However, I've read too many run evaluator articles that I hope to put a stop to the gobbledygook type approaches, and steer the search in the right direction.
Your BaseRuns is using the only right approach, by definition. As I've mentioned, the only thing left in understanding run creation is the score rate, and how to calculate it.
Tango, trust me, it's not yawning -- it's digesting. I am thoroughly enjoying the work.
Thanks, and hopefully the next article in the series will be as satisfying.
...it was the position of several well-qualified analysts that models don't matter--all that matters is accuracy in the range of interest. To them, the range of interest was the 3 to 6 R/G range of real MLB teams. To me, and to you, the range of interest is 0 to infinity R/G.
Yes, the shortcut way, while easier, and sometimes even more accurate, does not lend itself to extrapolating beyond what it was designed for. Since we are living in the time of Pedro and Bonds, maybe we care more about the extreme types.
I hope you delve into the subject of how to apply a successful team run formula to individuals. I believe that this area has NOT to this point been analyzed properly, and I am curious as to what you have come up with.
This area also needs alot of work, but I will present what I have nonetheless (probably in the 4th installment).
*** A few others have responded by email, and I appreciate any feedback (positive or negative).
August 26, 2002 - Ben
Excellent article!
Does the fact that an out happens to be a strikeout change it's value? It seems to me that a strikeout is more costly than any old out because runners are not advanced at all. (I know WPs and PBs can cause advance or even the batter to reach, but that's part of the WP/PB event not the strikeout event IMHO.)
Ben
August 26, 2002 - tangotiger
(www)
(e-mail)
Thanks Ben. To answer your question, there are 24 base-out states to consider. Depending when the K occurs, it's value would be different than a regular out.
You can check out a chart that breaks down the various run values by the 24 base-out states here: Run values by the 24 base-out states .
The K also has an extra wrinkle in that the batter can be safe after a K, or other events can happen afterwards. I've chosen to include them as part of the K, but in reality the effect is very tiny.
Related Web Pages and Articles