Question of the day #2 (Revised): Should you take out your starter who is pitching a shutout?

Posted: October 17, 2013 in Pitching

Edit: The following article was edited and revised from its original version. There were some mistakes and coding errors. I take full responsibility for the errors.

It goes without saying, per conventional wisdom at least, that when your starter is pitching a shutout and he has not thrown too many pitches, say, less than 90 or even 100, you leave him in there until his pitch count is elevated, he gets into some trouble, or you have to pinch hit for him in a close game or one in which you are losing. And even this last consideration is sometimes ignored – you often see a manager let an NL pitcher hit in the 6th-8th inning in a close or losing contest, if he is pitching well.

Not only is this conventional wisdom, but it would be heresy to think otherwise. For example, in the NLCS game 4, Lynn, the Cardinals starter, came out to pitch the 5th inning in a close game. No one thought anything of it even though he was facing the batting order for the 3rd time, and he wasn’t even pitching a shutout – he had allowed 2 runs in 4 innings. In game 2 of the ALCS, Tigers manager JIm Leyland was mildly criticized for taking Scherzer out after 7 innings – he had thrown 108 pitches and allowed 2 hits and 1 run.

So what happens when we let a starter pitch another inning when he is throwing a shutout? Surely we know the answer to that – at least managers do, right? Yeah, right! Managers do 100′s of things right and wrong, when they have no idea what the numbers are (somehow they think they do, I guess). I truly find it hard to believe that after 100 some odd years of baseball no one can tell us what happens when a starter is pitching a shutout versus, say, after allowing a couple of runs, or even 5 or 6. Managers will gladly remove a starter after 5 or 6 innings when they have allowed 4 or 5 runs, but almost never do that when they are pitching a shutout. I realize that some of that has to do with not alienating your players, building confidence and stamina in young starters, etc. You can’t, I guess, be yanking pitchers left and right in the early innings when they have been pitching well.

However, there is not a manager alive, I don’t think, or most everyone for that matter, who does not think that a pitcher who is pitching well through 4, 5, 6, or 7 innings will not continue to pitch well, as long as his pitch count is reasonable (and he is a regular starter who is used to throwing 6 or 7 innings, etc.).

We addressed this issue to some extent in The Book. If you want to know what that research had to say, you’ll have to consult your copy or buy one. I also posted about this earlier in the year on The Book blog.

Here is some new research:

I looked at all games in which the starter was pitching a shutout so far and how they did in the subsequent inning. For example, if they were pitching a shutout after 1 inning, I looked at how they performed in the second inning. If they pitched a shutout through 2 innings, I looked at how they pitched in the 3rd inning. Etc.

I could not just look at innings that they completed – that would be a biased sample. Completed innings tend to be good ones and partial innings, when a pitcher is yanked min-inning, tend to be bad ones. So if a pitcher was yanked mid-inning, after facing at least one batter, I used the run expectancy of the base/out state when they left as a proxy for the number of runs that they allowed that inning. In other words, I assumed that the starter completed the inning and that he allowed a number of runs equal to the RE when he left (plus any runs he did allow). That is a little bit of a fudge, but I don’t think that it is a huge deal.

In presenting the numbers, the number of runs allowed in each inning after pitching a shutout so far, I adjusted for the quality of the batters in that inning. For example, if a pitcher pitches a shutout through one inning, he likely is facing the middle or bottom of the order in the second. If he allowed 3 or 4 runs, he is likely back to the top of the order in the second inning (and he is facing them for the second time). I also multiplied the runs per inning, typically around .5 of course, by 9, in order for it to look like a runs allowed per game.

I also adjusted for park factors. Shutouts tend to occur in pitchers’ parks and when pitchers allow lots of runs, that tends to occur in hitters’ parks. These are minor tendencies.

Column 3, next to the adjusted runs allowed, is the pitchers’ collective actual runs allowed for that season. This is the expected number of runs allowed in any inning (once you adjust for the batters in that inning). So comparing column 2 with column 3 is the key to this analysis. If there is a carryover effect to pitching a shutout, we would expect column 2, runs allowed in that inning, to be less than column 3. runs allowed per 9 for the whole season for these pitchers. If there is a carryover effect for getting shelled (allowing 4 or more runs), we expect column 2 to be greater than column 3.

Keep in mind that it is expected that after 1 or more shutout innings, subsequent innings will be slightly worse than seasonal numbers. What I mean by that is this: If a pitcher allows exactly .5 runs per inning for an entire season, in any inning other than 1 or more shutout innings, he is naturally going to allow slightly more than .5 runs. In other words, in any game where that pitcher has some shutout innings, the other innings will be slightly worse than his overall seasonal average – and vice versa for games in which a pitcher allows lots of runs through 1 or more innings. I adjusted for this by subtracting out the requisite number of shutout or shelled innings. So for example, if the seasonal RA9 were 5.00, then we might expect a starter to allow 5.15 runs per 9 after 5 shutout innings. After 5 innings of 4 runs or more, we might expect an RA9 of 4.90. These adjusted numbers are what is presented in column 3.

Anyway, here is the data:

These are pitchers who are pitching a shutout through X-1 innings, where the first column is inning X. The second column is the number of runs allowed in inning X for pitchers who allowed no runs in innings 1-X, adjusted for the park and the batters in that inning. Remember these are not quite actual runs allowed (almost). They are runs allowed adjusted for batters and park. And remember that they include the run expectancy for that inning when a starter leaves mid-inning.

The time period I studied was 1998-2012, and I limited the sample to games in AL parks only for various reasons. As I said, the runs in each inning were prorated to 9 innings (multiplied by 9) so that they look like runs scored per game per team.

The 4th column is the league-wide average number of runs scored in that inning (for all games and all pitchers), also adjusted for the batter pool in that inning, but not adjusted for the quality of all pitchers who pitch in that inning.

The last column is the percentage of batters who batted from the same side as the pitcher throws in that inning. As starters stay later in the game, they tend to face more same side batters, which makes sense. This column is more FYI only, although to some extent it can affect the numbers.

Note that the 5th column is now opponent runs allowed (the pitching team’s runs scored) through that inning, prorated to 9 innings. This gives us an idea as to the run scoring environment, although it appears to be dependent on how long the starter pitches so don’t put too much stock in it.

Shutout prior to inning X

Inning Adj RA Pitchers’ season RA9 Lg Avg runs Pitchers’ team RS Batter wOBA Park Factor Pitcher platoon adv
 2 4.68 5.07 4.83 4.88 .335 1.00 41.1%
 3 4.90 5.00 5.16 4.87 .331 1.00 40.4
 4 4.73 4.94 5.13 4.96 .346 1.00 40.0
 5 5.05 4.86 5.25 4.94 .326 1.00 41.4
 6 4.85 4.80 5.15 4.96 .340 1.00 39.5
 7 4.92 4.70 4.86 4.95 .334 .99 40.3
 8 4.80 4.61 4.61 4.69 .322 .99 39.8

I don’t know about you, but I think that those numbers in the runs allowed column are a little troubling. They should be to managers too – virtually no one thinks that you should take out a pitcher who is throwing a shutout and has a reasonable pitch count, because surely he is going to continue to pitch well, right?

In the first few innings, we see a small carry over effect from throwing a shutout thus far – maybe (it could just be a lower run environment, other than the park, for various reasons – weather, umpires, year, etc.). After 4 shutout innings, we don’t see any carryover effect at all – in fact, we see starters pitching worse than they normally do.

As it turns out, the deeper in the game they go, despite pitching very well so far, the more they face the lineup. By the 6th, 7th, and 8th innings, they are facing the lineup for the 3rd and 4th time. Look at the 7th and 8th innings. Starters pitching a shutout are allowing between .08 and .1 runs more than the average pitcher (which are mostly relievers of course) in those innings!

In the 5th and 6th innings, these pitchers are allowing .20 to .30 more runs per 9 than they typically allow for the season as a whole. By the time they pitch in the 7th and 8th, they are allowing .40 more runs than usual! Again, that is the times through the order penalty. There is no carry over from pitching a shutout that trumps that penalty.

And look how good the pitchers are (column 5) who make it deep into the game. By the time we get into the 7th and 8th innings, only aces are allowed to continue, on the average. But still, they pitch more like middle relievers.

The number of times a starter faces the batting order is way, way, way more important than how he has been pitching. I cannot emphasize that enough and it may be the single most important thing that managers (and everyone else) should get through their thick skulls!

Let’s look at the same chart for all pitchers who have allowed at least 4 runs prior to the listed inning.

 4 or more runs allowed prior to inning X 

Inning Adj RA Pitchers’ season RA9 Lg Avg runs Pitchers’ team RS Batter wOBA Park Factor Pitcher platoon adv
 2 5.90 5.53 4.90 5.01 .347 1.01 41.6%
 3 5.02 5.40 5.21 4.94 .342 1.01 40.8
 4 5.72 5.28 5.08 4.94 .335 1.01 41.4
 5 5.45 5.20 5.27 5.05 .350 1.00 41.3
 6 5.25 5.02 5.23 4.91 .332 1.00 42.6
 7 5.21 4.83 4.88 4.54 .336 1.00 43.5
 8 4.78 4.74 4.59 3.99 .346 1.00 45.8

Other than the 3rd inning, here we also see a small carry over effect in the other direction from pitching badly in the first few innings. By the 5th inning, however, as with the shutout pitchers, the times through the order penalty is evident with very little additional carry over effect. Of course allowing 4 or more runs after 4 or 5 innings is not terrible pitching.

What about pitch count? How does that play into it?

Let’s look at pitchers who are throwing a shutout, but we’ll only look at those innings in which he starts the inning with fewer than 100 pitches:

Shutout so far, under 100 pitches going into inning

Inning Adj RA Pitchers’ season RA9  Lg Avg runs Pitchers’ team RS Batter wOBA Park Factor Pitcher platoon adv
 2 4.68 5.07 4.83 4.88 .335 1.00 41.1%
 3 4.90 5.00 5.16 4.87 .331 1.00 40.4
 4 4.73 4.94 5.13 4.96 .346 1.00 40.0
 5 5.05 4.86 5.25 4.94 .326 1.00 41.4
 6 4.85 4.80 5.15 4.97 .340 1.00 39.5
 7 4.95 4.69 4.86 4.98 .334 .99 40.2
 8 4.71 4.65 4.61 4.78 .322 .99 39.7

So even at fewer than 100 pitches going into the mid to late innings of a shutout game, you are going to give up lots of runs. Pitch count, and I presume fatigue, appears to have little to do with why pitchers, even when throwing well, give up lots of runs late in the game. Times through the order, times through the order, times through the order!

Pitching a shutout, but over 100 pitches going into the 7th or 8th inning

Inning Adj RA Pitchers’ season RA9  Lg Avg runs Pitchers’ team RS Batter wOBA Park Factor Pitcher platoon adv
 7 4.39 4.78 4.86 4.61 .326 .99 43.6
 8 5.19 4.38 4.61 4.27 .332 .99 40.4

If you do have a high pitch count while throwing a shutout, you are worse in the 8th, but better in the 7th, overall about the same. The sample sizes are small (205 and 333 innings, respectively) so these numbers are not particularly reliable. Notice that only aces are allowed to pitch into the 8th inning with a high pitch count. Nevertheless they allow a lot more runs than they normally do – .81 runs per 9 .

What if you have given up 4 or more runs, but still have a low pitch count (< 100)?

Allowing 4 runs or more – fewer than 100 pitches going into the 6th- 8th inning.

Inning Adj RA Pitchers’ season RA9  Lg Avg runs Pitchers’ team RS Batter wOBA Park Factor Pitcher platoon adv
 6 5.21 5.03 5.23 4.91 .331 1.00 42.7%
 7 5.15 4.86 4.88 4.56 .333 1.00 43.5
 8 4.42 4.69 4.59 3.96 .344 1.00 48.6

And finally, here is what happens when pitching badly (4 or more runs allowed) and you have a high pitch count:

Allowing 4 runs or more – more than 100 pitches going into the 6th- 8th inning.

Inning Adj RA Pitchers’ season RA9 Lg Avg runs Pitchers’ team RS Batter wOBA Park Factor Pitcher platoon adv
 6 5.69 4.96 5.23 4.96 .335 1.00 41.7%
 7 5.33 4.75 4.88 4.49 .344 1.00 43.6
 8 5.18 4.80 4.59 4.03 .348 1.00 42.6

As you can see, if you have allowed lots of (4 or more) runs going into the middle and late innings, it matters how many pitches you have thrown. If you have thrown more than 100 pitches, you will allow anywhere from .03 to .07 more runs (in the next inning) than if you have thrown fewer than 100. Again, small sample size warning for these numbers!

Summary

To summarize these results, a starter who is throwing a shutout does not appear to allow runs in any subsequent inning much less than what he normally allows. His runs allowed in innings 2-4 are slightly lower than league average as well as what he normally allows based on that season’s stats, but that could be due to an overall depressed run environment in these games. Once he gets into the middle and late innings, where he is facing the order for the 3rd or 4th time, he experiences the normal “times through the order” penalty despite the fact that he has not yet allowed any runs to score. In other words, even a very good pitcher throwing a shutout is not so good starting in the 5th or 6th inning. Don’t expect a pitcher who is pitching a shutout to continue to pitch well into the middle and late innings even with a low or moderate pitch count. There is little carry over effect and the times through the order penalty is too powerful.

Pitch count does not seem to be a factor for pitchers throwing a shutout. If they are under or over 100 pitches going into the middle to late innings, they pitch about the same – in a very mediocre fashion.

Pitchers who allow 4 or more runs through X innings don’t really continue to pitch badly other than innings 2 and 4 where they pitch a little worse than they usually do. For some reason in inning 3, they pitch very well – I have no idea why that might be. Maybe batters the second time through the order are acting sub-optimally since they scored 4 or more runs in the first 2 innings or maybe the starters are trying to make up for a poor first 2 innings. Or perhaps it is just a random anomaly (although it is a pretty large sample size – 2031 innings).

Pitch count does seem to be a factor for pitchers who have allowed 4 or more runs. Under 100 and you actually pitch better in the later innings than pitchers who are throwing a shutout, relative to how you pitch normally! Over 100 and it is time for you to hit the showers, although your performance in the late innings is not terrible.

About these ads
Comments
  1. Baltar says:

    What a great post! This is shocking and revolutionary!

  2. Jon Roegele says:

    Very interesting! I like the idea of assigning the RE at time of exit to a pitcher for partial innings. There certainly looks to be a jump in the 5th and 7th innings, perhaps tied to the 3rd and 4th times through the lineup. I recall you were interested in a study I did previously looking at what pitchers throw and how they fare each time through the order. I remember you citing a flattening out of performance in general from the 3rd to 4th time through the order, which was partly due to cooler temperatures late in night games and the fact that offense dies when teams are down by a lot in the final innings.

    May I ask how you calculated the batter quality adjustments that you used per inning? I could stand to improve some of my studies by controlling this sort of thing.

    Thanks, and enjoyed the study!

    • MGL says:

      Thanks, Joe. Here is how I do quality of batter (and pitcher, if needed) adjustments. It is not perfect.

      I use wOBA for that year and keep track of total wOBA for the pool of batters I am interested in. So, for this study, in each inning I was compiling data, for each PA, I looked up that batter’s wOBA for the season and kept a running tally for that inning.

      For example, say that Pujols, Carpenter, and Adams batted in a certain inning. If Pujols had a .360 wOBA for that year, Carp .350, and Adams, .340, then the average batter wOBA for that inning was the average of those 3 numbers, or .360.

      If the league average (including pitchers in using the NL and you are including pitcher hitting in the data) wOBA for that season were .330, then for that inning, the batter pool was 30 points better than average. I use .00087 runs per PA for every point of wOBA. So 30 points in wOBA is equivalent to a .0261 run adjustment per PA. If there were 4 PA in the inning, then the runs adjustment would be .104 by virtue of the batters in that inning.

      Same thing for pitching pool adjustments, if I am doing that. I use that season’s pitcher wOBA against. You can use RA or FIP also, or whatever you want.

      So basically I am using a batter’s or pitcher’s single season stats as a proxy for their talent, which is OK as long as I have a large sample of players and as long as my study isn’t biased in some way that relates to their season stats.

  3. Cliff Blau says:

    Phil Birnbaum had a similar study in the November 2000 By The Numbers, and reached somewhat different conclusions. See http://www.philbirnbaum.com/btn2000-11.pdf

  4. MGL says:

    Thanks, I’ll check it out and post some comments.

  5. MGL says:

    Good study by Phil from 2000! I wonder if he even remembers it. He looked at similar things that I was looking at in my study herein.

    He looked at pitchers who allowed 3+ runs (and more in some other samples) in the first inning and then how they did in the rest of the game, compared to their seasonal averages.

    He also looked at pitchers who walked a bunch in the first inning and how they did for the rest of the game.

    Also pitchers who allowed a few base runners in the middle and late innings.

    And pitchers who were pitching no-hitters through 3, 4, and 5 innings.

    He didn’t control for the quality of the batters faced, parks, weather, umpires, etc. He mentions some of that.

    He also doesn’t talk about the times through the order penalty because I don’t think researchers really knew about that back then, or at least they didn’t know how to quantify it (until The Book), although baseball insiders have always had some vague notion that it existed (then again, they have notions about lots of things that turn out to NOT be true). That is an issue when he looks at results in the middle and later innings only and compares them to a pitcher’s seasonal stats. Remember that a starter’s season stats are an average of all times through the order. If you look at a starter’s early innings, you will see better results than the late innings, after controlling for batter quality if necessary, because of the times through the order effect (the magnitude of the difference also depends on whether it is a day or night game because of temperature changes).

    Anyway, Phil concluded this:

    “Going into this study, I expected to find reasonably hefty stuff effects. I thought that average pitchers who give up lots of walks in the first inning are showing they have no control that day, and the results would continue to be disastrous. I was expecting the RC27s to show a big jump, perhaps from 4.5 to 6.5.

    But, as we have seen, they did not. There were no big effects either way, and, in general, we can conclude that there’s no evidence that the results a pitcher has obtained so far in the game should affect our estimate of what we should expect later in the game. Go figure.”

    Here are some quotes from my conclusion:

    “Don’t expect a pitcher who is pitching a shutout to continue to pitch well into the middle and late innings even with a low or moderate pitch count. There is little carry over effect…

    “Pitchers who allow 4 or more runs through X innings don’t really continue to pitch badly other than innings 2 and 4…”

    So, his results are very similar to mine. I found little effect for pitchers pitching well (shutouts) and pitchers not pitching well (4 or more runs). And what little effect I found might entirely be due to environmental factors that I did not control for, like weather and umpires (again, Phil mentions these too).

    I’m not sure why you say that we reached somewhat different conclusions, although “somewhat different” can also be construed as “somewhat similar!”

    • Phil says:

      Of course I remember that study! It’s one of my favorites, actually … it completely changed the way I look at evaluating pitchers during a game. Now, when I see a manager pulling his starter in the 2nd inning after giving up 5 runs, I think … really? Are you sure he isn’t just unlucky? Do you really want to waste your bullpen like that?

      Of course, there are other reasons for pulling the starter than just expecting him to pitch badly the rest of the game. You don’t want to leave him in for 15 runs and ruin his stats and have him mad at you … and, there might be times where the manager actually sees something about the pitcher. Them managers know their stuff, some of the time.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s