MLB A.C.E. Model Overview


The goal of this article is to give you a small window into the behind-the-scenes math and complexity going into the models, while also making the layout more intuitive and simple to interpret.

Pitcher Model

We begin with the pitcher model. For the second half of last season, I gathered all the data I could, each day, to build the most predictive pitcher model possible.

Projection and Probabilities

Most everyone is familiar with projections in DFS, but projections come up short in many ways. For one thing, it’s a point estimate; It’s just one estimate of a million different possible outcomes. In a more predictable sport, like NBA, projections have tons of value. When projecting pitching performance, however, they lose some value because the possible range of outcomes is so wide and variant. We don’t just want to compare pitchers at their median outcome. We want to know who is the preferable choice if they both pitch well, if they both pitch poorly, etc. Enter: benchmark probabilities.

We forecast the probability that the pitcher will score at least 10, 15, 20, 25, and 30 DK points in their start. These probabilities are then instrumental in calculating the ratings.F

Percentile Ranks

Let’s say Max Scherzer is projected for 24.5 DK points. We compare this projection to all single-game projections from last year to say something like “24.5 is higher than 88% of all projections”. The percentile rank of his projection would then be 88. Each benchmark probability is also compared to all the previous probabilities for that benchmark score and assigned a percentile rank. Finally, we assign weights to the projection and benchmark probabilities for a weighted average percentile rank. After multiplying by 1.5 (so pitchers are more important than hitters when running optimized lineups), we have our Ratings.

Floor, FTA, and Ceiling Ratings

The FTA Rating is the main, median outcome rating. It weighs the probability of scoring 15 points and 20 points the most of the benchmark probabilities.

The floor rating speaks to how “safe” a pitcher is. It weighs the probability of 10 and 15 DK points the most.

The ceiling rating speaks to the pitcher’s upside. It weighs the probability of 20, 25, and 30 DK points the most.

You’ll notice that condensing and contextualizing larger amounts of information is a common theme of our A.C.E. Projection System. That’s exactly what these ratings do. In fact, the projected stats condense and contextualize the various pitcher and hitter stats, then the model condenses and contextualizes those projected stats into a DK projection and benchmark probabilities. Finally, then, the Ratings condense and contextualize the model’s forecasts. This means that you can easily trace a rating that stands out to you.

For example, if you notice Robbie Ray has a higher Ceiling Rating than those around him, but a lower floor, you know to look at the probability of 20, 25, and 30 to see why his Ceiling Rating is higher. Then, you can look at the projected stats to see why those probabilities are higher. In general, higher K rate means higher ceiling, while run prevention (wOBA & projected runs against) is more instrumental in the lower benchmarks.

How Pitcher Ratings are Calculated

The process begins at the far right of the dashboard, where you can view the pitcher’s stats and opposing team stats. A few notes here:

  • The opposing team’s stats are specific to their lineup on that day. Even the batting order will affect their projected stats.
  • We are showing the stats as “Ratios”. Take wOBA for example. The wOBA Ratio is the team’s wOBA, adjusted for park factor, divided by league average wOBA. Thus, any number greater than one means they are an above average hitting team (on that day, in that park, independent of the pitcher). The benefit to viewing these stats as Ratios instead of their raw numbers is to give them extra context. Just how good is a .340 team wOBA? How bad is .276? Is .315 slightly below or slightly above average? The ratio contextualizes it for you.

These ratios are then used to adjust the pitcher’s stats into a game-specific projection, weighing the pitcher’s stats more. We project, for the starting pitcher:

  • Innings Pitched
  • Strikeouts
  • Walks Allowed
  • Runs Allowed – two methods (I’ll talk about this next)
  • wOBA Allowed

Four of those are straightforward. Let’s discuss “Runs Allowed”. First, we use the wRC+ Ratio of the opponent and the pitcher’s xFIP and SIERA metrics to project the runs allowed ourself. Then, we do something truly unique. We decompose the Vegas IRTA to calculate the IRTA specific to the starting pitcher (using our IP projection and the bullpen’s SIERA). We average the results of the two methods for our Runs Allowed projection.

Finally, these projected stats are used to forecast the DK points and benchmark probabilities that fuel the Ratings.

Hitter Model

Hitter Projections

The process for the hitter model is very similar to that of the pitcher model. We start by converting the hitter’s and opposing pitcher’s stats into “projected stats”. There’s a key point I want to make here so let’s look at an example. Here’s a screen-grab of Mike Trout vs Mike Fiers:

Based on how wOBA is calculated and the variance in baseball for a single game, these “projections” should be interpreted as follows: if Mike Trout were to face Fiers 1,000 times in this ballpark, on this day, we would project him to finish with a .427 wOBA, .386 ISO, etc.

The idea here is that most subscription services provide you with the pitcher’s stats and hitter’s stats, and it’s up to you to figure out how they combine. By creating the “hitter projections”, we are once again contextualizing information. These projections aren’t just calculated with the pitcher and hitter stats, though. They also include park factors and a metric we’re calling “True Hard Hit Difference”.

True Hard Hit Difference

First, we identify hot and cold hitters by looking at true hard hit rate. HH% is defined as the percentage of balls hit in play that are hit hard. However, this ignores the at bats where contact is not made. True hard hit rate adjusts for strikeout percentage. We’ll use Joey Gallo as an example:

In 2018, Gallo had a 48.5% HH% and a 35.9% K rate. Defining true HH% as the percentage of at bats in which he hit the ball hard (instead of just the at bats in which he made contact), this gives him a true HH% of 31%.

Suppose then, that over a 30 day period, his HH% was 55%, but he struck out exactly half the time. Is he hot because his recent HH% is higher than his long term HH%? Well, this scenario would mean he hit the ball hard in only 27.5% of his at bats, his true HH%.

We then define True Hard Hit Difference as the recent true HH% – long term true HH%. In this Gallo example, his true hard hit difference would be -3.5%. This would bump down some of the key hitter projections for him.

Percentile Ranks

We create percentile ranks for the following stats:

  • wOBA
  • ISO
  • wRC+
  • OBP
  • wSB
  • K%
  • BB%
  • Bullpen SIERA
  • IRT
  • IRT difference (will touch more on this – very excited for it)

Creating the percentile ranks scales all of the stats evenly so that we can then choose how much to weight each stat for the given rating.

IRT Difference

IRT Difference = IRT – average runs scored for that team

Pretty much every MLB DFS player is aware of the importance of utilizing Vegas implied totals for hitters. However, we believe IRT doesn’t go far enough. Suppose Boston has an IRT of 4.5 and Miami has an IRT of 4.2. If all players are priced for their typical production, which team’s hitters should we target? Boston averaged 5.45 runs per game in 2018 to Miami’s 3.66. So in this example, Vegas believes Boston will have a below average day (-1.05 IRT dif) while Miami will have an above average day (+.54 IRT dif), despite Boston’s IRT being higher. If priced for their typical production, then, we would want to target Miami hitters.

Floor, FTA, and Ceiling Ratings

Most sites will give you fantasy point projections for hitters. We do not because we believe the non-normal range of outcomes and overall variance makes them inconsequential at best, and misleading at worst. Instead, we provide ratings.

Percentile ranks for stats like wOBA, wRC+, and the IRT metrics are weighted most heavily for the FTA Rating. ISO and wSB are weighted more for the ceiling rating, and OBP and K% get a boost for the floor rating. 

Since they are ratings and not projections, don’t be alarmed if a player’s ceiling rating is lower than his FTA or floor rating. A player’s floor rating should only be compared to other player’s floor ratings, his FTA Rating to other FTA Ratings, and his ceiling to other ceilings.

For instance, from the last Spring Training slate, Khris Davis (vs Trevor Cahill) and Aaron Judge (vs Andrew Cashner) had the following Ratings:

Davis: Floor – 62.8 | FTA – 67.4 | Ceiling – 73.1
Judge: Floor – 72.8 | FTA – 68.3 | Ceiling – 63.3

Cahill really struggled with right handed power, while Cashner did well to prevent it, despite similar wOBAs. Meanwhile, Judge had a higher OBP, but lower ISO and the two were very similar in wOBA. Thus, it makes perfect sense that Judge would have the higher floor and lower ceiling ratings.

Some Final Notes

  • Our analysts have the ability to adjust the park factors based on weather. For example, Arlington, Texas shouldn’t be treated the same in April as mid-August.

  • Our analysts have the ability to add in pitchers for teams like the Rays who use an opener

  • Our analysts can adjust the expected pitch count for pitchers returning from injury or who are just monitored closely (like everyone on LAD’s staff).

Fantasy Baseball

One Comment

Leave a Reply