This is a follow up to my article on baseballprospectus.com about starting pitcher times through the order penalties (TTOP).
Several readers wondered whether pitchers who throw lots of fastballs (or one type of pitch) have a particularly large penalty as opposed to pitchers who throw more of a variety of pitches. The speculation was that it would be harder or take longer for a batter to acclimate himself to a pitcher who has lots of different pitches in his arsenal. As well, since most starters tend to throw more fastballs the first time through the order, those pitchers who follow that up with more off-speed pitches for the remainder of the game would have an advantage over those pitchers who continue to throw mostly fastballs.
First I split all the starters up into 3 groups: One, over 75% fastballs, two, under 50% fastballs, and three, all the rest. The data is from 2002-2012. I downloaded pitcher pitch type data from fangraphs.com. The results will amaze you.
FB % |
N (Pitcher Seasons) |
Overall |
First Time |
Second Time |
Third Time |
Fourth Time |
Second Minus First |
Third Minus Second |
Fourth Minus Third |
> 75% |
159 |
.357 |
.341 |
.363 |
.376 |
.348 |
.027 |
.020 |
-.013 |
< 50% |
359 |
.352 |
.346 |
.349 |
.360 |
.361 |
.003 |
.015 |
.010 |
All others |
2632 |
.359 |
.346 |
.361 |
.370 |
.371 |
.015 |
.015 |
.013 |
…
Pitchers who throw mostly fastballs lose 35 points in wOBA against by the third time facing the lineup. Those with a much lower fastball frequency only lose 24 points. Interestingly, the former group reverts back to better than normal levels the fourth time (I don’t know why that is, but I’ll return to that issue later), but the latter group continues to suffer a penalty as do all the others. Keep in mind that the fourth time numbers are small samples for the first two groups, and that fourth time TBF are only around 15% of first time TBF (i.e., starters don’t often make it past the third time through the order) .
The takeaway here is that a starter’s pitch repertoire is extremely important in terms of how long he should be left in the game and whether he should start or relieve (we already knew the latter, right?). If we look at columns three and four, we can get an idea as to the difference between a pitcher as a starter and as a reliever, at least as far as times through the order is concerned (there are other considerations, such as velocity – e.g., when a pitcher is a short reliever, he can usually throw harder). The mostly fastball group is 16 points (around .5 runs per 9 innings) more effective the first time through the order than overall, while the low frequency fastball group only has a 6 point (.20 RA9) advantage. Keep in mind that some of that first time through the order advantage for all groups is due to the “first inning” effect (see my original article on BP).
Next I split the pitchers into four groups based on the number of pitches they threw at least 10% of the time. The categories of pitches (from the FG database) were fast balls, sliders, cutters, curve balls, change ups, splitters, and knuckle balls.
# Pitches in Repertoire (> 10%) |
N (Pitcher Seasons) |
Overall |
First Time |
Second Time |
Third Time |
Fourth Time |
Second Minus First |
Third Minus Second |
Fourth Minus Third |
1 |
41 |
.359 |
.344 |
.370 |
.375 |
.303 |
.027 |
.009 |
-.061 |
2 |
1000 |
.358 |
.343 |
.359 |
.371 |
.366 |
.016 |
.018 |
.007 |
3 |
1712 |
.361 |
.349 |
.362 |
.371 |
.372 |
.013 |
.015 |
.014 |
4 |
378 |
.351 |
.340 |
.351 |
.360 |
.368 |
.011 |
.013 |
.019 |
…
This is even more interesting. It appears that the fewer pitches you have in your repertoire, the more that batters become quickly familiar with you, we we might expect. One-pitch pitchers lose 36 points by the third time through the order, while four-pitch pitchers lose only 24 points. The fourth time through the order is exactly the opposite. Against one-pitch pitchers, pitchers gain 61 points (small sample size warning – 639 PA). Again, I have no idea why. Maybe fastball pitchers are able to ramp it up in the later innings, or maybe they start throwing more off-speed pitches. A pitch f/x analysis would shed some more light on this issue. Against the four-pitch pitchers, batters gain 19 points the fourth time around compared to the third. If we weight and combine the third and fourth times in order to increase our sample sizes, we get this:
# Pitches in Repertoire (> 10%) |
N (Pitcher Seasons) |
Overall |
First Time |
Second Time |
Third and Fourth Times |
Second Minus First |
Third+ Minus Second |
1 |
41 |
.359 |
.344 |
.370 |
.364 |
.027 |
-.001 |
2 |
1000 |
.358 |
.343 |
.359 |
.370 |
.016 |
.017 |
3 |
1712 |
.361 |
.349 |
.362 |
.371 |
.013 |
.015 |
4 |
378 |
.351 |
.340 |
.351 |
.361 |
.011 |
.015 |
…
Again, we see the largest, by far, second time penalty for the one-pitch pitchers (27 points), and a gradually decreasing penalty for two, three, and four-pitch pitchers (16, 13, and 11). Interestingly, they all have around the same penalty the third time and later, other than the one-pitch pitchers, who essentially retain their quality or even get a bit better, although this is driven by their large fourth time advantage, as you saw in the previous table.
It is not clear that you should take your one-pitch starters out early and leave in those who have multiple pitches in their weaponry. In fact, it may be the opposite. While the one-pitch pitchers would do well if they only face the order one time (and so would the two-pitch starters actually), once you allow them to stay in the game for the second go around, you might as well keep them in there as long as they are not fatigued, at least as compared to the multiple-pitch starters. Starters with more than one pitch appear to get 10-15 points worse each time through the order even though they don’t have the large penalty between the first and second time, as the one-pitch pitchers do. Remember, for the last two tables, a pitch is considered part of a starter’s repertoire if he throws it at least 10% of the time.
I’ll now split the pitchers into four groups again based on how many pitches they throw, but this time, the cutoff for a “pitch” will be 15% rather than 10%. The number of pitchers who throw four pitches at least 15% of the time each are too few for the their numbers to be meaningful, so I’ll throw them in with the three pitch pitchers. I’ll also combine the third and fourth times through the order again.
# Pitches in Repertoire (> 15%) |
N (Pitcher Seasons) |
Overall |
First Time |
Second Time |
Third and Fourth Times |
Second Minus First |
Third+ Minus Second |
1 |
447 |
.358 |
.342 |
.362 |
.364 |
.027 |
-.001 |
2 |
1954 |
.359 |
.346 |
.361 |
.370 |
.016 |
.017 |
3+ |
742 |
.355 |
.347 |
.352 |
.371 |
.013 |
.015 |
…
The three and four-pitch starters are better overall by three or four points of wOBA (.11 RA9). The first time through the order, however, the one-pitch starters are better by 5 points or so (.15 RA9). The second time around, the one-pitch pitchers fare the worst, but by the third and fourth times through the order, they are once again the best (by 6 or 7 points, or .22 RA9). It is difficult to say what the optimal use of these starters would look like. At the very least, these numbers should give a manager/team more information in terms of estimating a starter’s penalty at various points in the game, based on his pitch repertoire.
I’ll try one more thing: Two groups. The first group are pitchers who throw at least 80% of one type of pitch, excluding knuckleballers. These are truly one-pitch pitchers. The second group throw three (or more) pitches at least 20% of the time each. These are truly three-pitch pitchers. Let’s see the contrast.
# Pitches in Repertoire |
N (Pitcher Seasons) |
Overall |
First Time |
Second Time |
Third and Fourth Times |
Second Minus First |
Third+ Minus Second |
1 (> 80%) |
47 |
.360 |
.343 |
.367 |
.370 |
.025 |
.009 |
3+ (> 20%) |
104 |
.353 |
.350 |
.357 |
.357 |
.008 |
.009 |
…
It certainly looks like the 42 one-pitch pitchers (47 is the number of pitcher seasons) would be much better off as relievers, facing each batter only one time. They are not very good overall, and after only one go around, they are 25 points (.85 RA9) worse than the first time facing the lineup! The three-pitch pitchers suffer only a small (8 point) penalty after the first time through the order. Both groups actually suffer the same penalty from the second to the third (and more) time through the order (9 points).
So who are these 42 pitchers who are ill-suited to being a starter? Perhaps they are swingmen or emergency starters. I looked at all pitchers who started at least one game – not just regular starters. Here is the complete list from 2002 to 2012. The numbers after the names are the number of TBF faced as starters and as relievers.
Mike Timlin 20, 352
Kevin Brown 206, 68
Ben Diggins 114, 0
Jarrod Wahburn 847, 0
Mike Crudale 9, 199
Grant Balfour 17, 94
Shane Loux 69, 69
Jimmy Anderson 180, 3
Kirk Reuter 620, 0
Jaret Wright 768, 0
Logan Kensing 55, 11
Tanyon Sturze 57, 277
Chris Young 156, 0
Nate Bump 33, 286
Bartolo Colon 2683, 49
Carlos Silva 876, 10
Aaron Cook 3337, 0
Cal Eldred 12, 141
Rick Bauer 21, 281
Mike Smith 18, 0
Shawn Estes 27, 0
Troy Percival 4, 146
Andrew Miller 306, 0
Luke Hochevar 12, 41
Luke Hudson 13, 0
Dana Eveland 15, 13
Denny Bautista 9, 38
Dennis Safarte 81, 274
Roberto Hernandez 548, 0
Mike Pelfrey 812, 0
Daniel Cabrera 881, 0
Frankie de la Cruz 15, 37
Mark Mulder 3, 9
Ty Taubenheim 27, 0
Brad Kilby 7, 58
Darren Oliver 17, 264
Justin Masterson 1794, 4
Luis Mendosa 60, 0
Ross Detwiler 627, 51
Cesar Ramos 11, 109
Josh Stinson 17, 21
Ross Detwiler 627, 51
Many of these pitchers barely had a cup of coffee in the majors. Others were emergency starters, swingmen, or they changed roles at some point in their careers. Others were simply mediocre or poor starting pitchers, like Kirk Reuter, Jarrod Washburn, Mike Pelfrey, Carlos Silva, and Daniel Cabrera, while others were good or even excellent starters, like Kevin Brown, Mark Mulder, and Bartolo Colon.
I think the lesson is clear. Unless a team has a compelling reason to make a one-pitch pitcher a starter (perhaps they are an extreme sinker-baller, like Brown, Cook, and Masterson), they should probably only relieve. If a team is going to use a swingman for an occasional start or a reliever for an emergency start, they would do well to use a two or three-pitch pitcher or limit him to one time through the order.
If we remove the swingmen and emergency starters as well as those pitchers who faced fewer than 50 batters in a season, we get this:
# Pitches in Repertoire |
N (Pitcher Seasons) |
Overall |
First Time |
Second Time |
Third and Fourth Times |
Second Minus First |
Third+ Minus Second |
1 (> 80%) |
28 |
.353 |
.336 |
.364 |
.365 |
.028 |
.004 |
3+ (> 20%) |
104 |
.353 |
.350 |
.357 |
.357 |
.008 |
.009 |
…
Even if we only look at regular starters with one primary pitch other than a knuckleball, we still see a huge penalty after the first time facing the order. In fact, the second time penalty (compared to the first) is worse than when we include the swingmen and emergency starters. Although these pitchers overall are as good as multiple-pitch starters, they still would have been much better off as short relievers.
Here is that updated list of starters once we remove the ones who rarely start. These guys as a whole should probably have been short relievers.
Cook
Miller
Colon
Diggins
Silva
Young
Cabrera
Wright
Washburn
Anderson
Masterson
Brown
Rueter
Kensing
Mendoza
Pelfrey
Hernandez
Detwiler
You might think that the one-pitch starters in the above list who are good or at least had one or two good seasons might not necessarily be good candidates for short relief. You would be wrong. These pitchers had huge second to first penalties and pitched much better the first time through the order than overall. Here is the same chart as before, but only including above-average starters for that season.
# Pitches in Repertoire |
N (Pitcher Seasons) |
Overall |
First Time |
Second Time |
Third and Fourth Times |
Second Minus First |
Third+ Minus Second |
1 (> 80%) |
11 |
.328 |
.307 |
.332 |
.332 |
.039 |
-.013 |
3+ (> 20%) |
35 |
.321 |
.318 |
.323 |
.323 |
.004 |
.003 |
…
Here are those pitchers who pitched very well overall, but were lights out the first time facing the lineup. Remember that these pitchers were above average in the season or seasons that they went into this bucket – they were not necessarily good pitchers throughout their careers or even in any other season.
Kevin Brown
Jarrod Washburn
Jaret Wright
Chris Young
Bartolo Colon
Carlos Silva
Justin Masterson
Ross Detwiler
Interestingly, the very good multiple-pitch pitchers had very small penalties each time through the order. These are probably the only kind of starters we want to go deep into games! Here is that list of starters.
Sonnanstine
B. Myers
Pavano
Sabathia
Billingsley
Carpenter
Hamels
Haren
F. Garcia
Iwakuma
Shields
J. Contreras
Beckett
Duchscherer
Gabbard
K. Rogers
Buehrle
M. Clement
Halladay
R. Hernandez
T. Hunter
Finally, in case you are interested, here are the numbers for all of the one-pitch knuckleballers that I have been omitting in some of the tables thus far:
Knuckle Ballers Only |
N |
First Time |
Second Time |
Third+ Time |
Second Minus First |
Third+ Minus Second |
20 | .321 | .354 | .345 | .034 | -.006 |
…
Where are all the knuckle ball relievers? Although we don’t have tremendous sample sizes here (3024 second time TBF), so we have to take the numbers with a grain of salt, it looks like they are brilliant the first time through the order but once a batter has seen a knuckleballer one time, he does pretty well against him thereafter (although we do see a 6 point rebound the third time and later through the order).
I think that more research, especially using the pitch f/x data, is needed. However, I think that teams can use the information above to make more informed decisions about what roles pitchers should occupy and when to take out a starter during a game.
[…] Does having a shallow repertoire actually hurt pitchers on later times through the order? MGL investigates. […]
I believe there was a part in The Book that mentioned that bringing in different type of pitchers back to back had an even bigger positive impact? For instance following up a hard throwing guy with more of a junk baller?
I’ll have to look that up – I don’t remember. A few years ago, I found that when a knuckleballer leaves the game, the relievers have a large advantage – a sort of knuckleball hangover effect.
[…] Jeff Zimmerman: http://mglbaseball.wordpress.com/2013/11/11/ttop-and-a-starting-pitchers-repetoire/ […]
Nice work! I was going to start along this path, but you’ve covered it well.
Some other questions that may or may not prove to lead to interesting answers could be:
(1) Is there a pitch type that helps lessen the TTOP? I had looked into something similar, and found curveballs to be the pitches that hitters adjusted to most quickly, followed by other offspeed pitches, with fastballs declining at the slowest rate. Actually knuckleballs looked very good too! But of course the small sample size meant I didn’t know what to make of it.
(2) Does it matter *when* these pitches are thrown, in terms of frequency? For example, if two pitchers throw sliders 20% of the time, but one throws it 20% of the time each time through the order, while the other throws is only 5% of the time the first time through, then 20% the second, then 35% (or something like that….) do they suffer different penalties?
Do you think either or both of the above might be interesting follow-up studies?
Absolutely! I think this is a fascinating subject and one that could greatly help teams who are paying attention or doing their own similar in-house studies.
One area that arouses my curiosity (why does that word have no “u”?) are the fourth time numbers for one-pitch pitchers or primarily fastball pitchers. Might be just a sample size glitch, but if it isn’t, I wonder if it because they throw almost all fastballs to start the game and then mix in some off-speed late in the game, or that mostly fastball pitchers can ramp it up late in a game.
The interesting question along the lines of what you are talking about in your penultimate paragraph, is, “What is the optimal strategy?” If a pitcher, as you say, throws sliders 5% of the time the first time through the order, and then 20% the second, etc., is he giving up a lot early in order to gain something late? I would think that there has to be a give and take as far as a pitcher’s distribution of pitches early and late in a game, if it changes a lot.
I love the idea of trying to find usage patterns that help to reduce TTOP. I was thinking about this today, and realized one would have to be careful in trying to study this, as while a strategy could look to minimize TTOP, it could be at the cost of an excessively poor performance the first time through the order. So for example, a set of pitchers who “save” a pitch type for subsequent times through the order may have lower TTOP (I might guess), but this may partly be due to their performance being worse than it could be the first time through due to using less pitches. Does that make sense? So to look at optimal strategies, one would have to bin pitchers by their overall performance (ERA or whatever) first, then look within each bin for different strategies maybe? Can you think of a better way to control for this?
When I looked at this for 2012 data several months ago, the starters who showed a particular pitch type increase of at least 10% usage between first and second times through the order were:
James Shields, curveball, 12.9%
Ervin Santana, slider, 12.9%
Justin Masterson, slider, 10.8%
Wandy Rodriguez, curveball, 10.4%
Chris Sale, slider, 10.2%
Jered Weaver, changeup, 10.2%
I either didn’t check or didn’t find anything interesting about their relative TTOP compared to other pitchers, I don’t remember. Probably never checked.
Anyway despite all of the TTOP work that mostly you have done, I suppose there is always more that could be studies and learned!
Yes, I think that much more can be done with the PITCHf/x database. Other than reporting the frequencies of various pitches throughout the game and how that relates to the various TTOP, I am not sure much else can be concluded. It is a little like trying to determine the optimal approaches for various batters – e.g., should Votto swing more or Vald swing less? That is impossible, because we don’t know what individual hitters can and can’t do with various approaches based on their talent and preferences.
About the only thing we can conclude once we look at the data is, for example, so-and-so, because of his repertoire and the way he distributes his various pitches throughout a game, is probably suited to going deep into a game (facing the lineup multiple times), whereas some other pitcher, is not. We can speculate that it might be more optimal for a certain pitcher to change the way he approaches exposing his repertoire, but that would be pure speculation. For example, perhaps pitchers who start out with mostly fastballs and then expose more of their off-speed pitches later in the game, take a while to get a feel for those pitches? Maybe if they uses them early, they would not be effective? Or maybe the pitcher who shows his entire arsenal early in a game suffers a lot the second, third, and fourth times through the order, but makes up for that loss the first time? The teams themselves would have to experiment with this sort of stuff. which is one reason why this angle of inquiry has the potential to be so valuable, I think. For example, it looks to me like there should be many more knuckleball short relievers. If you look at the TTO data that I presented, it appears that while they are overall good pitchers (think Dickey – at least the 2012 version – and Wakefield), they really are not well-suited for going deep into games even through conventional wisdom says that they are because they don’t appear to tire easily (although we don’t know that either).
[…] through, those numbers jump to .248/.316/.398. Granted, that’s another small sample size, but MGL has established that pitchers with a limited arsenal have more trouble getting through an order multiple times than […]
[…] Licthman recently published work on the penalty two-pitch pitchers paid each time through the batting order. His work discovered […]
Any chance there there is survivorship bias with respect to the drop in the wOBA for the 4TT guys? Are they the only ones who are consistently good enough to make it 4TT? Do the 3TT guys get pulled before they can get to 4TT?
Yes, there is a huge survivorship bias with respect to all the TTO, especially the 4th, a little for the 3rd. The pitchers who pitch the 4th time through the order, on the whole, are much better pitchers. That is why you have to adjust/control for the quality of the pool of pitchers in each bucket, which I did.
There is also another, mostly separate issue, which I did not address. There is a selective sampling effect which has the opposite effect of the survivorship bias, although you could call it another survivorship bias if you wanted to.
That is, if a pitcher is pitching well, he tends to make it to the next TTO. So, for example, pitchers who are in the 3 TTO bucket, regardless of their true talent, tended to get a little lucky the 2nd time through the order, because some of the rally unlucky performances the 2nd TTO are removed, because managers pull them before they face the order for the 3rd time. This has the effect of RAISING the observed penalty (the other survivorship bias LOWERS it if you don’t correct for it). It is also more prevalent in the 4th time penalty, since a lot of pitchers who have not pitched well the first 3 times don’t make it to the 4th.
However, the magnitude of this bias depends on how the penalty is computed, If it is computed by taking the difference between, say the 4th TTO and a pitcher’s overall performance for the year, then the effect is small. If it is computed by taking the difference between the 4th TTO performance and the third TTO performance, then the effect could be large. I think I use the former method. I cannot remember off the top of my head exactly.