Reliever Usage Pattern, 1999-2002 (June 24, 2003)
I break out my LI to show how all relievers have been used the last 4 years. The chart may be pretty, but the situation is not.
--posted by TangoTiger at 12:16 PM EDT
Posted 2:28 p.m.,
June 24, 2003
(#1) -
Rally Monkey
The pitchers with the top leverage index are the ones used as traditional closers. I was a little surprised Dotel wasn't higher, as he seems to get all the high leverage, non save opportunities.
Why does even Percival get 23% of his time in low leverage situations? There are times when you just need to get a guy some work. It would be nice to see him used more, like Sutter was, but in Percival's case he just might not be able to handle it.
Posted 2:39 p.m.,
June 24, 2003
(#2) -
tangotiger
Well, I'm not so surprised that Percy gets 1/4 of his batters when they basically don't count, but that that's the best figure, and that the top relievers get 1/3. That's seems a bit high to me. I'm guesssing 15-20% would be a better target?
I could understand if say Mariano comes into the 8th in a high-leverage situation and gets out of the jam, and then the Yanks take a big lead, making the 9th inning a mop-up job for Mo (though you could consider taking him out if that happens). But, this is not what usually happens with the current fireman.
Give me the usage pattern of the late 70s to mid 80s.
Posted 3:38 p.m.,
June 24, 2003
(#3) -
Rally Monkey
Was Sutter in the 15-20% range?
Posted 4:25 p.m.,
June 24, 2003
(#5) -
tangotiger
Rally: I looked at Gossage from 82-86. His line was: 20,9,14,27,31. That's probably what we should shoot for.
Tribe: Yes, you are accurate. Remember what we are NOT measuring. We are NOT measuring the performance level of a pitcher. What we ARE measuring is the level of fire a pitcher finds himself in. If you are John Franco and you have a propensity to get men on base, and then work yourself out of a jam, well, that's a big fire you had to put out, even though you were the arsonist.
As well, the manager is making the call to leaving in Franco after every batter. I can't say that "If Franco stays in, the LI is 1, but if Strickland comes in to bail him out, the LI is 2", can I?
To measure performance, you need a "win probability added" type of measure (which I have, and have shown some results in the past). What this does is combine the performance of the pitcher with the leverage of the situation. So, say you are at LI 1.0, and you give up a hit. That brings you to LI 2.0, and you give up a walk. That walk will count as "2 walks". Now you are at 3.0, and you give up a HR. That counts as "3 HR with 2 men on each". At this point, the game may be out of reach, at which point your LI is now 0.3.
Leveraged Index is NOT this measure. It's one-half of what you need to measure the performance of a reliever. Eventually, I'll combine the two for a complete measure. For the moment, take my LI and multiply it by Wolverton's ARP. That'll get you close enough.
Posted 4:41 p.m.,
June 24, 2003
(#6) -
tribefan08
Thanks Tango. Great response. Now I understand better. I'll look forward to that complete measure someday soon. In the meantime could you point me to some of your win probability added type measures you mentioned?
Posted 5:11 p.m.,
June 24, 2003
(#7) -
FJM
It is often said that closers don't pitch well in low leverage situations. Can you confirm or deny that? Does performance really improve with LI? Can you post OBA or OOBP for each pitcher in each LI Class?
Posted 5:31 p.m.,
June 24, 2003
(#8) -
Nick S
LI is amongst your best work and is the correct way to evaluate the current issue of reliever usage. It's meaning is very clear and cuts right to the real difference between 'starter' and 'reliever', that is, starters have an LI of 1 (more or less, if I recall correctly, and that should be the value as starter's innings are, more or less, randomly distributed into close games and blowouts) whereas relievers do not.
This is work that could actually crossover to the mainstream. BaseRuns and DIPS just don't stand a chance right now (ever?) of being cared about by anyone who isnt' a 'stathead', they are too convoluted in their design, purpose, and meaning. Even if someone can't see quite where an LI number comes from, they can certainly understand it ("See son, at the bottom of the "Fox Box" that flashing, red 2.0, that means that this at bat is twice as important as a normal one.) And, best of all, it is a macho stat that would appeal to players. Relief pitchers seem to want the respect that they are the player to count on in tough situations, currently they associate that with saves, but I'd love to see the day that a pitcher bitches about being sent out for the ninth with a three-run lead because he needs to keep his LI up for salary arbitration :)
Posted 6:14 p.m.,
June 24, 2003
(#9) -
Jim
Can you give an example of how you determined what the LI is for a particular situation? Also, how did you determine the cutoff points?
Posted 6:37 p.m.,
June 24, 2003
(#10) -
tangotiger
Nick, maybe you should sell someone at FOX on the idea! I love the story!
Jim, the example is based on figuring out what the win probability is at each situation, and the possible distribution of win probabilities based on each possible outcome. That "variance" gives you the leverage. Check out Phil Birnbaum's article that I posted elsewhere on this STUDIES section of Primer as he gets into it as well. I figured this mathematically, for all possible inning/score/base/out, with the score +/- 18 runs.
As for the cutoffs, they are arbitrary, but I tried to group them so you get 50% in the first grouping, and then about 10-15% in each of the other 4.
FJM: I started to do that a little while ago, with Urbina, Shuey, Stanton, and Benitez. Eventually I'll do it, but I'm just trying to find the best way to present it. My feeling is that I will not find a difference.
Posted 7:54 p.m.,
June 24, 2003
(#11) -
David Smyth
I agree, the LI concept is good, creative analysis by Tango.
But, although it is technically correct that the manager makes a 'decision' to leave Franco in batter by batter, this does not really seem to be what actually happens. The primary decision is when to bring a certain reliever in; whether to leave him in is a secondary decision. Therefore, it *may* be that the more informative way to look at reliever usage is what was the situation when he was brought into the game.
Posted 8:06 p.m.,
June 24, 2003
(#12) -
tangotiger
I agree, when he was brought in would ALSO be a good metric. Heck, when he was REMOVED would also be a good metric too. Both are a snap to calculate as well. A manager therefore can do 3 things:
1 - when does he put him into the fire, and how big is that fire
2 - how long does he let him stay in the fire, and how big is that fire while he's battling it
3 - when does he take him out, and how big is the fire when he leaves
You can look at 1 and 3 and say that the difference in fire is his performance level, but
a - if he pitches multiple innings, then his offense can help/hurt
b - his fielders always help/hurt
Drinen handles #1/#3 by taking him "out" at the end of the inning, and inserting him back "in" at the start of the next inning. It's a good way to do it.
Like I said, there's alot of descriptions you can generate using the LI concept. For the moment, I've only done #2. Drinen has done #1 and #3.
Posted 4:12 p.m.,
June 25, 2003
(#13) -
aaron
Great stuff Tango.
I'm confused by the concept of "average game state". When you use "average game state" as a baseline, are you referring to the average state for the league or the particular team in question? Do teams that play a lot of close games have a higher LI over the course of the season than teams involved in many blowouts?
What might be interesting for this chart is an indication of each reliever's percentage of the team's total reliever innings at each leverage category.
For example. Troy Percival had 29% of his innings at 2.5+ leverage. What percentage of the team's total relief innings at 2.5+ leverage did Troy Percival pitch? Many teams have far fewer of these high leverage relief situations to go around, presumably meaning that their ace relievers (and all their other relievers as well, of course) will also get a smaller percentage of these high leverage innings.
It seems like you would need this calculation to determine how a team was actually using their relievers, or am i missing something?
Posted 4:41 p.m.,
June 25, 2003
(#14) -
tangotiger
(homepage)
What might be interesting for this chart is an indication of each reliever's percentage of the team's total reliever innings at each leverage category.
You are absolutely correct. I actually did this for the Yankees (see homepage link), and I should do this for all the teams.
Posted 5:04 p.m.,
June 25, 2003
(#15) -
Walt Davis
I think one thing that would help in terms of presentation is to give some examples or "ideal types" of the different leverage ranges. What's a "typical" 0-.5 leverage situation? It will help the non-statistical get a better idea of what this really means.
Obviously at this point it would be great to look at changes in these percentages over time (from Sutter to today, e.g.).
I'm curious as to why the .5-1 range has such low percentages. Any idea why these situations aren't very common?
Posted 5:11 p.m.,
June 25, 2003
(#16) -
tangotiger
Walt, good idea! I can try to pick out say the "typical" LI for each class.
As for the % of distribution, I really wasn't sure what to expect. 45% for 0-0.5, 15% for 0.5-1, 10% for 1 to 1.5, 8% for 1.5 to 2.0, 7% for 2.0 to 2.5 among our group of relievers here. I guess that's an ok distribution, though I was surprised by the big dropoff from 45% to 15%.
Essentially, half the PAs in MLB have very little value.
Posted 5:13 p.m.,
June 25, 2003
(#17) -
Jason
Quick question is LI biased against good relievers on bad teams? I can see that on worse teams and exceedingly good teams a larger percentage of the outs available to the pen will be of the high leverage variety and I'm wondering how that impacts the stat?
Posted 6:24 p.m.,
June 25, 2003
(#18) -
Walt Davis
I think Jason makes a good point -- it would be interesting to see how leverage differs by team based on record. I'm not sure there will be big differences. Leverage comes in close games and it seems that all teams play about the same number of close games. The difference between good and bad teams is often that good teams win lots of blowouts and bad teams lose lots of blowouts.
So I guess another way to look at is what, if any, is the relationship between the distribution of leverage and the distribution of game run differential? If 50% of ML PA have little leverage that, kinda, suggests that 50% of ML games are blowouts.
Posted 7:29 p.m.,
June 25, 2003
(#19) -
FJM
Walt: I don't think you can conclude that, just because most situations are low leverage, there must be a lot of blowouts. Consider all 2 out, nobody on, score tied situations. Unless it's late in the ballgame, these are low leverage situations. Hitting a homerun is about the only thing you can do to significantly alter the win probability, and even that wouldn't make a big difference early in the game.
Posted 7:48 p.m.,
June 25, 2003
(#20) -
Zen Bitz
re: FJM - that is what I am thinking. Leverage should automatically decline as outs go from 0-1-2. And even a a 9th inning 2-run lead with 2 outs can't be that high leverage.
Posted 8:03 p.m.,
June 25, 2003
(#21) -
Philip Nut
This is a really fun stat.
I was wondering if LI is normalized so that
Sum(LI) over all atbats = Number of at bats
(over a very large sample of at bats)
Also, Can someone refer me to where I can find:
the avg LI for starting pitchers vs avg LI for relievers
different subject----
Tangotiger,
have you tried the following yet?
(W/L)= [ (RS) / (RA_SP * LI_SP + RA_relievers * LI_relievers) ] ^2
as a sort of improved pythagorean relationship for a team....
I think this only works assuming the above normalization- if so, I can almost guarantee that this would work better than pythag
(RA_SP= runs allowed by starting pitchers, RA_relievers= runs allowed by relievers)
I don't think it spoils the spirit of the pythag formula since there is prob just as much correlation between the strength of a team's offense and defense, as there is between the strength of a teams's starting pitching and relief pitching. I also don't think it'd take away from its predictive power.
Philip Nut
bah, LaTeX, bah
Posted 8:52 p.m.,
June 25, 2003
(#22) -
Philip Nut
strike part of the end of my comment-- the spirit of the pythag formula has little to do with the weakness of correlation between the strength of a team's offense and defense (who knew what I was thinking)
Posted 9:04 p.m.,
June 25, 2003
(#23) -
tangotiger
LI *is* normalized to 1.0, so your equation holds for PA, by definition.
From 1974-1990, the LI for starters was 1.01, and relievers was .98. I haven't checked the 99-02 period, yet.
Posted 2:05 a.m.,
June 26, 2003
(#24) -
Doug
It occurs to me that the distribution of LI will very quite a bit by team. If you pitch on a juggernaut team with great pitching/great hitting, a great proportion of the PA will be low LI, because there'll be lots of game where you're up 5-0 in third or something and just coast in (or run up the score some more). Similarly, if you're on a weak hitting/weak pitching team, then most of your PA will be low LI, for the opposite reason. The rest of the teams should be in more close games so should have more PA with higher LI.
So..., if we're looking at how managers use relievers, why not look at each team's number of PAs in each LI range, and then calculate what percent of those PAs each reliever had. So instead of saying Percival had 23% of PAs in low LI situations, you might instead say that 10% (or whatever the number is) of the Angel's relievers' low LI PAs were pitched by Percival - which I think is telling you something a bit more interesting.
Bottom line is, if a team has many low LI PAs or many high LI PAs, they still need to get pitched, and the work needs to get spread around.
Posted 6:37 a.m.,
June 26, 2003
(#25) -
tangotiger
Doug, your concern was also expressed by post #13, and I addressed part of that in #14.
Btw, the overall LI, by team, varies by about 0.1. I don't think you will find quite the distribution differences that you might be looking for. But the concern is definitely valid.
Posted 9:04 a.m.,
June 26, 2003
(#26) -
Marc Stone
Tango-
From my understanding of what you've done you don't need to combine LI and ARP or some other performance measure to get a measure of the value provided. LI is derived from win probability so change in win probability for each inning pitched can be calculated directly.
Posted 9:40 a.m.,
June 26, 2003
(#27) -
tangotiger
You know, I spoke too quickly. ARP includes base/out, but not inning/score. So, I can't use LI and multiply it by ARP, even for a basic verison.
Yes, all you need to do is calculate the change in WE (win expectancy). This is what Drinen did in the other article I posted (WE before pitcher came in, after he left, and give difference to pitcher. In multi-innings, assume the pitcher left the game, and came back in, so that the change in offense won't affect him). I've actually also done this, and posted the results somewhere.
But, I cannot use LI directly to get into my change in WE.
The issue comes with "crediting" the fielders. Right now, I give it all to the pitcher (which is why I'm not too crazy about publishing my current results). Same with hitters, and I give all the credit to the hitter, and none to the runners.
Posted 1:54 p.m.,
June 26, 2003
(#28) -
Jim Detry
Leveraged plate appearances must also apply to batters. This may be a better measure of "clutch" situations than is normally used. Pick a figure of merit for a batter (simple OPS or something more complicated, penalizing for hitting into double plays, etc.) and plot figure of merit vs. Leverage. A positive slope means he's a clutch hitter. A negative slope means he chokes. If a large enough sample shows everyone is basically flat, then there's no such thing as a clutch hitter.
Posted 2:21 p.m.,
June 26, 2003
(#29) -
tangotiger
Jim, check out the other link I added, called "Win Probability Added".
What I intennd on doing (eventually) is
1 - win probability added using inning/score/base/out
2 - win probability added using no context
3 - calculate hitter's LI
Take #3 and multiply by #2. This gives you his win probability, if he were to hit the same regardless of the leveraged situation.
Compare the the result of this to #1. The difference is the player's "clutch" performance.
To measure his underlying clutch skill, you'd check year-to-year correlation.
Now, if you read the Hidden Game, I believe they did this kind of thing with the Mills' Brothers PWA (2 years only). We're in a position to do it from 20+ years.
Not now though.
Posted 11:02 p.m.,
June 26, 2003
(#30) -
tangotiger
I wrote this on Jan 2, in one of the Primer articles, and it bears repeating:
==========================
What we are after is *not* to maximize a pitcher's LI, but rather to maximize their leveraged-innings (LI x IP). LI of 1.00 with 120 IP will have the same win impact as 1.50 LI with 80 IP to a reliever. Of course, it's not that simple, as you have to take the totality of your starters and relievers, and maximize the leveraged innings for the good pitchers, and minimize the leveraged innings for the bad pitchers, such that all innings are accounted for. You have other constraints as well, with respect to the tiredness of a pitcher's arm, etc.
Mark Eichorn, for example, had 200 leveraged innings (LI of about 1.3) in his great year. That is an excellent total
Posted 1:35 p.m.,
June 27, 2003
(#31) -
Withrow
This is very interesting.
It would be intersting to know how managers use their relievers in different LI situations.
Also, is there a way to adjust for situations where a manager may be giving his closer some work after the closer hasn't pitched in a while? Do you think this is worth adjusting for?
Posted 1:48 p.m.,
June 27, 2003
(#32) -
tangotiger
THat would be good to know, but I'm not sure what you mean about "adjusting".
LI is a reflection of the game state. It is dependent only on the inning/score/base/out.
Certainly, if Bonds were at plate, the LI would be different. Certainly, if Clemens is throwing 160 pitches already, the LI would be different. But, I'm trying to keep the player's identity out of the picture. (For this purpose anyway.)
Posted 4:22 p.m.,
July 11, 2003
(#33) -
Phil Birnbaum(e-mail)
Tom,
This is very interesting stuff ... thanks!
A couple of thoughts:
1. In "Moneyball", Billy Beane (or maybe Depodesta) is quoted as saying it's good to make the starter throw a lot of pitches, so you get into the bullpen earlier. This is supposed to be a good thing because relief pitchers are generally not as good as starters. But if managers use better relievers when the game is tied, that strategy may no longer make sense, since the starter would be replaced by a mediocre reliever only when the game was already decided. This data has the potential to help us find that out.
2. Suppose the average team ERA is 4.5. That tells us something. But it would also be good to know what the average team ERA is *weighted by importance*. Some teams may use bad pitchers when the game is 15-1, but then their pitching staffs aren't as bad as they seem, because who cares how bad your reliever is when it's 15-1? Some picthing staffs may therefore look worse (or better) than they actually are.
Again, nice work. If I have some time, I might try a couple of these studies someday.
Phil
Posted 4:42 p.m.,
July 11, 2003
(#34) -
tangotiger
(homepage)
Phil,
See above link for a discussion on wearing out the pitcher. There's also a cool chart by MGL in post #25 I think it was. I'd ignore the back/forth between Steve and Ross... they no like each other.
As for your other comment, I think this definitely has value. I've been meaning to break down the performance of all relievers by classes of leverage. It would be interesting to see at the team level this difference, as you propose. Something tells me that we shouldn't find much difference.