Mid-season projections part II – Pitchers

Posted: June 13, 2014 in Pitching, Projections

Yesterday I looked at how and whether a hitter’s mid-season-to-date stats can help us to inform his rest-of-season performance, over and above a credible up-to-date mid-season projection. Obviously the answer to that depends on the quality of the projection – specifically how well it incorporates the season-to-date data in the projection model.

For players who were having dismal performances after the first, second, third, all the way through the fifth month of the season, the projection accurately predicted the last month’s performance and the first 5 months of data added nothing to the equation. In fact, those players who were having dismal seasons so far, even into the last month of the season, performed fairly admirably the rest of the way – nowhere near the level of their season-to-date stats. I concluded that the answer to the question, “When should we worry about a player’s especially poor performance?” was, “Never. It is irrelevant other than how it influences our projection for that player, which is not much, apparently.” For example, full-time players who had a .277 wOBA after the first month of the season, were still projected to be .342 hitters, and in fact, they hit .343 for the remainder of the season. Even halfway through the season, players who hit .283 for 3 solid months were still projected at .334 and hit .335 from then on. So, ignore bad performances and simply look at a player’s projection if you want to estimate his likely performance tomorrow, tonight, next week, or for the rest of the season.

On the other hand, players who have been hitting well-above their mid-season projections (crafted after and including the hot hitting) actually outhit their projections by anywhere from 4 to 16 points, still nowhere near the level of their “hotness,” however. This suggests that the projection algorithm is not handling recent “hot” hitting properly – at least my projection algorithm. Then again, when I looked at hitters who were projected at well-above average 2 months into the season, around .353, the hot ones and the cold ones each hit almost exactly the same over the rest of the season, equivalent to their respective projections. In that case, how they performed over those 3 months gave us no useful information beyond the mid-season projection. In one group, the “cold” group, players hit .303 for the first 2 months of the season, and they were still projected at .352. Indeed, they hit .349 for the rest of the season. The “hot” batters hit .403 for the first 2 months, they were projected to hit .352 after that and they did indeed hit exactly .352. So there would be no reason to treat these hot and cold above-average hitters any differently from one another in terms of playing time or slot in the batting order.

Today, I am going to look at pitchers. I think the perception is that because pitchers get injured more easily than position players, learn and experiment with new and different pitches, often lose velocity, their mechanics can break down, and their performance can be affected by psychological and emotional factors more easily than hitters, that early or mid-season “trends” are important in terms of future performance. Let’s see to what extent that might be true.

After one month, there were 256 pitchers or around 1/3 of all qualified pitchers (at least 50 TBF) who pitched terribly, to the tune of a normalized ERA (NERA) of 5.80 (league average is defined as 4.00). I included all pitchers whose NERA was at least 1/2 run worse than their projection. What was their projection after that poor first month? 4.08. How did they pitch over the next 5 months? 4.10. They faced 531 more batters over the last 5 months of the season.

What about the “hot” pitchers? They were projected after one month at 3.86 and they pitched at 2.56 for that first month. Their performance over the next 5 months was 3.85. So for the “hot” and “cold” pitchers after one month, their updated projection accurately told us what to expect for the remainder of the season and their performance to-date was irrelevant.

In fact, if we look at pitchers who had good projections after one month and divide those into two groups: One that pitches terribly for the first month, and one that pitches brilliantly for the first month, here is what we get:

Good pitchers who were cold for 1 month

First month: 5.38
Projection after that month: 3.79
Performance over the last 5 months: 3.75

Good pitchers who were hot for 1 month

First month: 2.49
Projection after that month: 3.78
Performance over the last 5 months: 3.78

So, and this is critical, one month into the season if you are projected to pitch above average, at, say 3.78, it makes no difference whether you have pitched great or terribly thus far. You are going to pitch at exactly your projection for the remainder of the season!

Yet the cold group faced 587 more batters and the hot group 630. Managers again are putting too much emphasis in those first month’s stats.

What if you are projected after one month as a mediocre pitcher but you have pitched brilliantly or poorly over the first month?

Bad pitchers who were cold for 1 month

First month: 6.24
Projection after that month: 4.39
Performance over the last 5 months: 4.40

Bad pitchers who were hot for 1 month

First month: 3.06
Projection after that month: 4.39
Performance over the last 5 months: 4.47

Same thing. It makes no difference whether a poor or mediocre pitcher had pitched well or poorly over the first month of the season. If you want to know how he is likely to pitch for the remainder of the season, simply look at his projection and ignore the first month. Those stats give you no more useful information. Again, the “hot” but mediocre pitchers got 44 more TBF over the final 5 months of the season, despite pitching exactly the same as the “cold” group over that 5 month period.

What about halfway into the season? Do pitchers with the same mid-season projection but one group was “hot” over the first 3 months and the other group was “cold,” pitch the same for the remaining 3 months? The projection algorithm does not handle the 3-month anomalous performances very well. Here are the numbers:

Good pitchers who were cold for 3 months

First month: 4.60
Projection after 3 months: 3.67
Performance over the last 3 months: 3.84

Good pitchers who were hot for 3 months

First month: 2.74
Projection after 3 months: 3.64
Performance over the last 3 months: 3.46

So for the hot pitchers the projection is undershooting them by around .18 runs per 9 IP and for the cold ones, it is over-shooting them by .17 runs per 9. Then again the actual performance is much closer to the projection than to the season-to-date performance. As you can see, mid-season pitcher stats halfway through the season are a terrible proxy for true talent/future performance. These “hot” and “cold” pitchers whose first half performance and second half projections were divergent by at least .5 runs per 9, performed in the second half around .75 runs per 9 better or worse than in the first half. You are much better off using the mid-season projection than the actual first-half performance.

For poorer pitchers who were “hot” and “cold” for 3 months, we get these numbers:

Poor pitchers who were cold for 3 months

First month: 5.51
Projection after 3 months: 4.41
Performance over the last 3 months: 4.64

Poor pitchers who were hot for 3 months

First month: 3.53
Projection after 3 months: 4.43
Performance over the last 3 months: 4.33

The projection model is still not giving enough weight to the recent performance, apparently. That is especially true of the “cold” pitchers. It over values them by .23 runs per 9. It is likely that these pitchers are suffering some kind of injury or velocity decline and the projection algorithm is not properly accounting for that. For the “hot” pitchers, the model only undervalues these mediocre pitchers by .1 runs per 9. Again, if you try and use the actual 3-month performance as a proxy for true talent or to project their future performance, you would be making a much bigger mistake – to the tune of around .8 runs per 9.

What about 5 months into the season? If the projection and the 5 month performance is divergent, which is better? Is using those 5 month stats a bad idea?

Yes, it still is. In fact, it is a terrible idea. For some reason, the projection does a lot better after 5 months than after 3 months. Perhaps some of those injured pitchers are selected out. Even though the projection slightly under and over values the hot and cold pitchers, using their 5 month performance as a harbinger of the last month is a terrible idea. Look at these numbers:

Poor pitchers who were cold for 5 months

First month: 5.45
Projection after 5 months: 4.41
Performance over the last month: 4.40

Poor pitchers who were hot for 5 months

First month: 3.59
Projection after 5 months: 4.39
Performance over the last month: 4.31

For the mediocre pitchers, the projection almost nails both groups, despite it being nowhere near the level of the first 5 months of the season. I cannot emphasize this enough: Even 5 months into the season, using a pitcher’s season-to-date stats as a predictor of future performance or a proxy for true talent (which is pretty much the same thing) is a terrible idea!

Look at the mistakes you would be making. You would be thinking that the hot group were comprised of 3.59 pitchers when in fact they were 4.40 pitchers who performed as such. That is a difference of .71 runs per 9. For your cold pitchers, you would undervalue them by more than a run per 9! What do managers do after 5 months of “hot” and “cold” pitching, despite the fact that both groups pitched almost the same for the last month of the season? They gave the hot group an average of 13 more TBF per pitcher. That is around a 3 inning difference in one month.

Here are the good pitchers who were hot and cold over the first 5 months of the season:

Good pitchers who were cold for 5 months

First month: 4.62
Projection after 5 months: 3.72
Performance over the last month: 3.54

Good pitchers who were hot for 5 months

First month: 2.88
Projection after 5 months: 3.71
Performance over the last month: 3.72

Here the “hot,” good pitchers pitched exactly at their projection despite pitching at .83 runs per 9 better over the first 5 months of the season. The “cold” group actually outperformed their projection by .18 runs and pitched better than the “hot” group! This is probably a sample size blip, but the message is clear: Even after 5 months, forget about how your favorite pitcher has been pitching, even for most of the season. The only thing that counts is his projection, which utilizes many years of performance plus a regression component, and not just 5 months worth of data. It would be a huge mistake to use those 5 month stats to predict these pitchers’ performances.

Managers can learn a huge lesson from this. The average number of batters faced in the last month of the season among the hot pitchers was 137, or around 32 IP. For the cold group, it was 108 TBF, or 25 IP. Again, the “hot” group pitched 7 more IP in only a month, yet they pitched worse than the “cold” group and both groups had the same projection!

The moral of the story here is that for the most part, and especially at the beginning and end of the season, ignore actual pitching performance to-date and use credible mid-season projections if you want to predict how your favorite or not-so favorite pitcher is likely to pitch tonight or over the remainder of the season. If you don’t, and that actual performance is significantly different from the updated projection, you are making a sizable mistake.



About these ads
  1. Brian says:

    Good piece but I don’t like how you keep saying “ignore actual pitching performance to-date” and “their performance to-date was irrelevant”. It is not irrelevant – it influences the mid-season projection, at least a little bit.

  2. MGL says:

    Yes, of course. I also mentioned that. However, once you know the mid-season projection, which naturally, and by definition, includes the season-to-date performance, then yes, you can and should ignore the seasonal stats which are indeed irrelevant once you have the projection. Assuming that the projection properly weights and includes those stats. In the case of my projections, there are some problems in doing that, but certainly if you have the choice between one or the other, you would take the projection over the actual stats at any point in the season. My principal point is that most people think that the season-to-date stats can be used as a proxy for true talent and for decision-making purposes toward the middle or end of the season. As I’ve shown, that is a terrible mistake.

  3. […] that’s hitters. What about pitchers? Pitchers can add new pitches, tweak their mechanics, move to the other side of the mound, or […]

  4. toph says:

    I’m not familiar with NERA, how is that different from ERA? Is it like xfip or SIERA?

    • MGL says:

      It is merely a pitcher’s components, actual, projected, whatever, converted to runs allowed per 9 innings, using a Base Runs formula. It is similar to wOBA for batters. My component projections happen to be context neutral, that is I adjust for anything and everything that may differ from environment to environment like defense, catcher framing, park, weather, umpires, opponents, etc. I scale the runs allowed to a 4.00 runs per 9 environment, such that a pitcher with a true talent of 4.00 is exactly league average, although starters are a little higher and relievers are a little lower.

  5. logicliker says:

    Have you considered the human element — e.g. a manager’s decision to give a struggling hitter or pitcher more time off, actually helps them to “clear their head/ reset” and then perform to their established talent level the rest of the way?

    • MGL says:

      I am an analyst. I go with the data. You can invent any narrative you want. They all seem to make sense. But, they are just assertions or hypotheses without evidence. Yes, it could be.

  6. ecocd says:

    It seems to me now that you’ve been able to demonstrate that recency isn’t all that relevant to projecting the remaining season of performance, in general, wouldn’t it be more interesting to focus on the exceptions?

    The dull part of the analysis is establishing the baseline assumptions. The enjoyable part is digging into trying to find a way to consistently identify the outliers. Some sort of batted ball profile change, BABIP, NL-> AL change, throwing to a new catcher, facing unusually worse/better pitchers or hitters in a season, etc.

    You put in the grindy work that shows what everyone expected to see. now you get the reward of doing the fun stuff.

    • MGL says:

      Good luck trying to find “the exceptions” before the fact. That’s the point. And to some extent “the exceptions” are built into a good projection system. For example, if a pitcher loses velocity at some point during the season, and it appears to be more than just a one or two game blip, a good projection system incorporates that. Or, if a hitter is playing injured for half a season and is then healthy for the second half, a good projection system accounts for that.

      • ecocd says:

        Do most mid-season projection systems that really take into account season-over-season velocity changes explicitly or are they just assumed implicitly as part of an aging curve?

        I think I’m still missing the essence of this research. I understand the surface conclusion of “don’t jump to conclusions just because Yuniesky Betancourt had a great April 2013,” but it feels like there’s more to this than at the surface.

        I have to think on this more. Thanks for the posts!

        • MGL says:

          I don’t know about the other projection systems. I explicitly adjust for velocity changes during the season and at the beginning of each season.

          There really wasn’t anything earth shattering about the research. It was only to “shut up” everyone who thinks that how players are doing 3 or 4 months into the season fortells how they will do for the rest of the season. That kind of writing/thinking is ubiquitous in baseball. Just look at any article about any player or team where the projection differs from the season-to-date performance.

  7. TimBoyer says:

    Super stuff! I can see how it would be hard to post something every day when so much of your content takes hours and hours of work. Thanks!

  8. […] Mid-season projections part II – Pitchers by Mitchel G. Licthman […]

  9. […] I mentioned in a post a few weeks ago on pitchers outpitching their peripherals, Mitchel Lichtman recently demonstrated that an up-to-date projection system is much better for predicting future performance than portions […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s