Livan Hernandez and Scouting (September 10, 2003)
Randy St. Claire... "Livan dropped his arm angle on some breaking balls. He'd done it just on a few pitches, but during our next side (throwing session) he asked me what I thought about it. I told him I liked the action on his breaking ball and I liked the reaction by the hitters. I told him, 'All your pitches are coming out of the same spot.' "
Previously when Hernandez pitched, his different pitches looked different before he threw them. "He was higher on his curveball than he was on his slider," St. Claire says, "and he was lower on his two-seamer than on his four-seamer."
This is the kind of stuff that results-based performance analysis would NOT pick up. The sample size is too small, and we can't get the confidence to make a statement that Livan has achieved a new level of performance.
But, visual-based scouting analysis DOES pick it up. While we are using the performance numbers to prop up the scouting (in the article anyway), it really isn't necessary. A good scout or pitching coach would tell you that even if the results weren't there after the change in his pitching mechanics, that the results would come.
I believe that in terms of scouting and performance, you have something like:
100 scouted PA = 300 performance hitting PA
100 scouted PA = 600 performance pitching PA
100 scouted PA = 900 performance fielding PA
(I don't know what the numbers are).
However, at some point, the second 100 scouting PAs give you less information that the first 100, etc, etc far faster than the second 300 hitting PAs. So, once you get to 10,000 PAs, scouting tells you basically nothing.
Unless of course, something mechanical has changed, like here.
--posted by TangoTiger at 10:48 AM EDT
Posted 1:43 p.m.,
September 10, 2003
(#1) -
studes
(homepage)
A good scout or pitching coach would tell you that even if the results weren't there after the change in his pitching mechanics, that the results would come.
But isn't that questionable? Don't athletes and coaches try things all the time, until they find something that works? I agree with your sentiment, Tango, but it seems to me that current performance is best judged by the results, to the degree we can quantify them accurately.
Posted 2:33 p.m.,
September 10, 2003
(#2) -
tangotiger
Suppose that two pitchers named Orlando and Livan have been pitching rather so-so (to the eye and performance-wise).
The pitching coach comes up to both of them and exclaims "I know what you are doing wrong!" They practice the two new pitching mechanics, and the pitching coach is satisfied that they both have adjusted properly, and should be more effective.
For the next 7 starts each facing a total of 200 batters, Orlando shows no change in his performance numbers (always striking out 15% of his batters), but Livan does improve his numbers (say from striking out 15% of his batters to striking out 25% of his batters).
You also have a third pitcher, say Pasqual, who always strikes out 15% of batters, but, without changing anything, struck out 25% of his next 200 batters.
Question to the Primate statisticians:
1 - what is your best guess as to Livan's true K rate (assume lg of 15% if you need that)?
2 - What is your best guess to Orlando's true K rate?
3 - What is your best guess to Pasqual's true K rate?
Please provide confidence levels and margin of error.
Posted 3:08 p.m.,
September 10, 2003
(#3) -
Warren
FYI - I'm not statistician, as my comments will undoubtably show...
I guess the first questions to ask is if a scout guess at a player's true rate is exactly analogous to a certain number of batters faced (or innings or plate appearances) of a sample. So, taking the example above, if the scout believes both Livan and Orlando now have a true rate of 25%, can we treat that as equivalent to having some extra data suggesting that they had a rate of 25% in (say) 500 batters?
If so, then the issue of margin of error goes back to however you determine that based on your sample size when you have no scouting data. If the error is +/- 5% at 200 batters, and +/- 3% at 700 batters, then it would also be +/- 3% at 200 batters with scouting information. You could then "rate" scouts according to their "effective sample size". You would also regress the sample towards the mean as you would normally do.
Time for some educated guessing to Tango's questions...
1. Livan has 200 batters at 25% and 500 "fake" batters at 25%. So we have, in effect, 700 batters at 25%. Let's say we regress 50% of the way toward the mean at a sample size of 700 batters. So Livan's true rate is 20% +/- 3% (the 3% being from the fake numbers above).
2. Orlando has 200 batters at 15% and 500 "fake" batters at 25%, giving him 700 batters at 22%. Regressing halfway, gives us 18.5% +/- 3%.
3. Pasqual has 200 batters at 25%. If we regress 75% of the way at a sample of 200 batters, then his true rate is 17.5% +/- 5%.
The real question, of course, is how many batters is a scout's opinion worth? 0? 500? 5000?
Posted 3:52 p.m.,
September 10, 2003
(#4) -
tangotiger
Interesting approach. I'll give you my thoughts tomorrow, as I'd like to see how others would approach this.
I do agree with your last statement, and it is for this reason that I *do* believe that scouts serve a purpose, the extent of which is yet to be established by the public (though may been established privately).
Posted 8:15 p.m.,
September 10, 2003
(#5) -
David Smyth
I, like Warren, am not a statistician (not even close :)) But at the major league level, and for an 'experienced' pitcher like Livan, I would put zero stock in a scouting report about his 'arm angle' before the fact--that is, before any improvement in performance has been manifested. After the fact, I wouldn't pay much attention until about now--after a couple months of consistently improved pitching. At that point it is reasonable to start wondering. But I am watching Livan vs the Cubs as I am writing, and all I see is a pitcher with poor velocity who is trying to make up for it with control and guile. Livan's main problem (as I recall) has been a huge number of HR allowed. As soon as the batters adjust, I expect him to decline. Another pitcher who has shown sudden significant improvement is E Loaiza. For some reason, I see his breakout as more likely to be 'real'.
Posted 8:10 a.m.,
September 11, 2003
(#6) -
studes
(homepage)
I don't know the answer to your question either, Tango, but I think David hits the point on the head. Talking about a change in the arm angle, without actual "proof" of improvement (representing some appropriate statistical likelihood -- and I certainly agree that the scouts' comments reduce the threshold for statistical likelihood) is what some sportswriters do all the time. (Gammons: "Scouts say Joe Shlabotnik has experienced great success in Winter Ball with a new delivery.") And then you never hear of the player again, and the writer never mentions it again.
Together, scouts' reports and players' results tell a great story. I wouldn't separate them.