Defining an Ace, Part II: Introducing the Ace Line Calendar

Introducing the Ace Line Calendar.

Over the past month or so, I’ve been slowly piecing together a pseudo-objective system for gauging whether or not an MLB starting pitcher is an “ace” or not on any given day. Ace. It’s a subjective term with an extremely loose definition. It’s also a fluid term that changes month to month, year to year.

What I’ve come up with is the Ace Line Calendar. Allow me to introduce you.

By the way, if you haven’t read my first post in this series, I recommend checking it out here.

Why create this system?

First, a story.

On July 26, 2015, the Kansas City Royals acquired Johnny Cueto from the Cincinnati Reds for a trio of lefty pitching prospects. Media outlets everywhere declared that the Royals had finally added the ace their team desperately needed. And they were right. According to Bill James Online’s Starting Pitcher Rankings on that day, Johnny Cueto ranked as the 7th best pitcher in all of baseball with a score of 547.1.

Two weeks later on August 10, Cueto dazzled in his first home start at Kauffman Stadium. He threw a complete game shutout, allowing only 4 hits, walking zero and striking out 8. When BJO’s rankings were updated the next day his score had jumped to 556, good for the 5th best pitcher in baseball.

From there, you may remember, it got rocky. Cueto got shelled numerous times during the months of August and September causing him to tumble 38 points to 518.5 by the end of the regular season.

The question of who to start in Game 1 of the ALDS for the Royals would have been obvious just two months prior, but many wondered whether or not the term “ace” still applied to Johnny Cueto. Was Cueto still an ace? Or was he merely a “good” pitcher at this point? Suddenly, instead of surrounding himself with names like Max Scherzer and David Price, he was hanging around the likes of John Lackey and R.A. Dickey. Still good pitchers. But not aces these days.

Wouldn’t it be nice to have some system – some quick metric – that could answer that annoyingly subjective terminology for you? I think it would. And even if it didn’t close the book on the discussion, it’d make the conversation that much richer.

So that’s my motivation. And the Ace Line Calendar just happens to be where I ended up.

By the way, the Royals threw Yordano Ventura in Game 1.

Using the Ace Line Calendar

Despite how the ensuing paragraphs appear, this isn’t a very complicated system. Bill James has done the hardest work for me in creating a fluid ranking system to utilize. All I’ve done is compiled it, plotted it and created a spreadsheet out of the numbers the data spit my way.

The Ace Line Calendar is a list of all 365 calendar days. It begins on April 1 and runs through March 31. For every day, it gives two numbers – the first number is the Obvious Yes Line, the second is the Definite No Line. The Obvious Yes Line declares that every starting pitcher with a score higher than that corresponding number is obviously an ace. The Definite No Line declares that anyone with a score below that corresponding number is definitely not an ace. For now, anyone in between is debatable.

Go ahead, pick any date in the last 6 baseball seasons. Okay, June 30, 2014. Or, the day some nobody named Jake Arrieta took a no hitter into the 8th inning at Fenway Park. Good choice.

From there, it’s a two step process:

Step 1: Go to the Ace Line Calendar and look up June 30.

Step 2: Compare the outputs with the BJO Starting Pitcher Rankings for 6/30/14. (You can select any date in the past 6 years on the SPR page.)

The output from the Calendar: 506.8/496.2.

The list for 6/30/14…

  1. Clayton Kershaw – 584.4
  2. Felix Hernandez – 557.7
  3. Adam Wainwright – 549.9
  4. Max Scherzer – 548.4
  5. Yu Darvish – 546.3
  6. David Price – 528.2
  7. Jon Lester – 526.4
  8. Cliff Lee – 523.3
  9. Chris Sale – 522.1
  10. Zack Greinke – 521.5
  11. Anibal Sanchez – 521.3
  12. Madison Bumgarner – 521.1
  13. Cole Hamels – 520.8
  14. Jered Weaver – 513.6
  15. Johnny Cueto – 513.1
  16. Justin Verlander – 507.8
    —> YES LINE – 506.8
  17. Kyle Lohse – 497.3
  18. James Shields – 497.0
    —> NO LINE – 496.2
  19. Jordan Zimmerman – 492.1
  20. C.J. Wilson – 491.1

The list goes on, but the takeaway is clear: according to my Calendar system, on June 30, 2014, everyone from Kershaw to Verlander (who came in just over the 506.8 Obvious Yes Line) is obviously an ace, Kyle Lohse and James Shields are questionable, but Jordan Zimmerman, C.J. Wilson and the rest of the list are definitely not aces coming in under the 496.2 Definite No Line.

Aaaaaaand, that’s pretty much it. That’s the “Ace Line.” Works for any day, all year long. Give it a try. See what you think and give me some feedback. It might not be perfect yet, but it feels pretty spot on so far. I’m excited to monitor it as the 2016 MLB season progresses.

How It Works

Okay, so this is where things get more dense, but here’s how I determined those values.

In Defining an Ace, Part I, I sifted through the past 6 MLB seasons (as far back as the BJO system goes), and broke the top pitchers into groups of Obvious Aces, Hesitations and Definite Nos based purely on my gut reaction. The number of individuals in these groups varied slightly each year, but there was a definite correlation in where the break happened between the Aces and Nos. The scores as of February 5 (when I did the initial research) were all somewhere in the 485 to 500 range.

Since the values are fluid throughout the season and offseason, my next goal is to discover a formula (y=ax + b) of best fit for the data, where “y” is the pitcher ranking score, “x” is the date in the calendar year,  “a” is the slope of the line and b is the starting point on the y-axis.

For “ace” caliber pitchers, scores generally rise throughout the season and then taper off consistently with the rest of the league during the offseason. Those who pitch in the postseason can continue to add to their scores. Postseason is included in the rankings, but it needs to be adjusted to a different slope due to the fact that the vast majority of starters are no longer pitching in November and their scores, therefore, are slowly decreasing.

To find this formula, I plotted the scores of the top 50 pitchers from each season on the 1st of every month (4/1/11, 5/1/11, 6/1/11, etc.). I ignored pitcher names entirely, and focused purely on rankings which causes each position on the graph to increase at generally the same rate throughout the season.

For example, in 2015, the 2nd best pitcher in baseball (after Clayton Kershaw) changed 4 times throughout the season. Due to his dominant 2014 postseason, Madison Bumgarner began the season ranked #2. By May 1, Felix Hernandez had passed him. By July 1, Max Scherzer – aided by a near-perfect game in which the only baserunner came when Jose Tabata leaned into a pitch in the 9th inning) – took over, but by September 1, Zack Greinke jumped him and finished the season at #2.

But when you ignore player names and focus only on the score of the individual ranked #2, the scores look like this:

  • April 1: 536.7
  • May 1: 551.7
  • June 1: 564.0
  • July 1: 584.6
  • August 1: 582.6
  • September 1: 583.2
  • October 1: 583.2
  • November 1: 592.1

The lower down the list you go, the more movement there is between pitchers day-to-day, but the scores end up creating a pretty consistent slope as the season progressed (which is what I’m looking for). But what slope – what ranking – should be used to determine what constitutes an ace?

Well, actually, it can’t be determined that easily because it varies year to year. The number of aces isn’t always consistent. It’s not just the Top 15 pitchers in baseball or the #1 guy on all 30 teams. That’s naive and lazy (which I tackled in my previous post on this topic). It’s fluid, not limited, so the number of aces one year might be 10 but the next year could be 18.

So I went back and looked at my lists of Yeses, Hesitations and Nos and plotted the corresponding individuals’ end-of-year ranking over the entire previous season. So in 2015, I thought Johnny Cueto (ranked #13 to end the season) was definitely an ace, Jacob deGrom (#14) and Stephen Strasburg (#15) caused me to hesitate and consider their cases, but I could quickly say that John Lackey (#16) was not an ace pitcher in 2015.

So I traced the 2015 Yes Line (Cueto, #13) back throughout the season as well as the 2015 No Line (Lackey, #16). I did this for all six seasons BJO has kept the Starting Pitcher Rankings, 2010-2015. Those individuals, if you don’t want to go back and look at the previous post again, were…

  • 2015: Cueto (Yes, #13), deGrom, Strasburg, Lackey (No, #16)
  • 2014: Strasburg (Yes, #14), Darvish, Weaver, Dickey, Shields, Samardzija (No, #19)
  • 2013: Sale (Yes, #15), Weaver, Dickey, Latos (No, #17)
  • 2012: Hernandez (Yes, #13), Gonzalez, Gallardo, Greinke, Latos (No, #17)
  • 2011: Shields (Yes, #16), Gallardo, Beckett, Garza, Price, Wilson (No, #21)
  • 2010: Lilly (Yes, #15) Johnson, Kershaw, J. Santana, Greinke, W. Rodriguez (No, #20)

When charted out, this is what those values look like. Again, this is not charted by individual, but by ranking. (If you’re on a mobile device, try turning it 90 degrees to better see the chart.)

2015

2014

2013

2012

2011

2010

High

Low

High

Low

High

Low

High

Low

High

Low

High

Low

4/1

490.1

475.6

487.2

473.1

481.4

475.5

489.2

479.6

486.5

470.5

487.3

468.7

5/1

488.5

472.0

488.8

480.2

491.2

483.3

498.0

489.6

496.4

476.9

490.3

482.3

6/1

499.4

487.1

504.6

496.2

493.7

491.2

501.2

493.8

495.6

485.4

501.8

495.6

7/1

504.5

499.2

513.1

492.1

503.3

496.8

509.0

499.6

502.2

492.5

513.7

498.3

8/1

511.8

504.7

509.1

503.9

511.1

503.7

509.0

502.5

502.6

498.9

519.1

508.2

9/1

525.5

510.1

511.2

503.5

530.7

514.4

529.8

514.2

524.7

506.6

528.7

515.4

10/1

525.5

510.1

531.0

516.5

523.2

520.2

526.9

523.3

526.2

519.1

531.4

518.2

11/1

518.6

509.1

522.5

510.5

518.0

512.9

528.3

513.3

524.7

512.1

524.3

511.2

Save for a few exceptions, each pair of columns begins roughly in the same place (481-490 high, 475-486 low), increases as the season progresses, reaches its height at the beginning of October before dipping down again during the postseason. If we continue these numbers into the offseason, each one decreases at a constant rate of a quarter point per day.

The one hiccup here is that the slope from April 1 to October 1 is different than from October 1 to November 1 due to postseason play. So we’ll need to find two different slopes.

When we aggregate and plot the 4/1-10/1 values, it generates the following slopes…

  • Obvious Aces… y = 7.0756x + 478.71
  • Definitely Not Aces… y = 7.2952x + 467.21

At first glance, I’m surprised that Nos have a steeper slope (7.29 vs 7.07) than the Yeses. I expected the higher ranking to have a slightly steeper slope, but the opposite is true. I’m not exactly sure why that is. I suppose the starting point is lower so the ending point is higher. Guys at the top are more established while the guys at the bottom of consideration are often up-and-coming names who and a lot further to climb.

When we do the same for the 10/1-11/1 values, we get…

  • Obvious Aces… y = -4.6333x + 532
  • Definitely Not Aces… -6.3833x + 524.28

The disparity of slopes here (-6.38 vs -4.63) appears much more drastic, and for two reasons. First, the regular season formulas above are considering 6 times the data that the postseason stats are, so the outputs are much less extreme. Second, simply put, true aces shine in the postseason while non-aces don’t.

And now that we have slopes for the entire year, we can simply “plug and chug” (as my high school algebra teachers used to say) for each date. Which yields a complete calendar of dates starting on April 1 and ending on March 31. All that’s left to do is copy/paste the results into a Google spreadsheet and share it here.

Now, none of this is perfect, and there are probably a dozen different reasons why this is off. A couple things I’m already aware of…

  1. Opening Day is not always April 1, the postseason doesn’t always begin October 1. I recognize that the better way to construct this would be around a 162 game schedule and have it begin and end exactly when the season does. Maybe if this beta version amounts to anything I can tweak it to be even more accurate?
  2. This whole thing is pretty arbitrary. Who knows how much things would change if I moved one spot up or down in the SPR data here or there. But one thing I know is the more data you average, the less variance there is over one tiny change in input. Or what if I’d chosen the 15th instead of the 1st of each month? How would the data adjust? I actually don’t think very much.
  3. What about Leap Year?! Well, I both skipped it and didn’t. I actually calculated February to be 28.25 days so I wouldn’t have to make adjustments for a 366th day. So, just read Feb 28. Or wait it out and check in March for all I care. You do you.
  4. There’s some level of error, I’m sure. And if I broke it down and plotted every single day before rather than just every 30 days, a line of best fit would be immediately more accurate. But cmon, let’s be realistic with my time here. The only place that might be necessary is between 10/1 and 11/1. More data would help there.

Besides, ultimately this is just my opinion on what makes a guy an “ace” – your threshold might be much more conservative or liberal than mine. But now that you know how I did it, you can go make your own ALC if you want. But just so everyone knows, this is the original.

-apc.

Photo cred: MLB.com accessed here.

Defining an Ace

What’s an ace?

It is, perhaps, the most subjective baseball term thrown around these days. “He’s the ace of their staff.” Or, “That guy is a true ace.” There are a lot of ways to define it, and none of them actually bring much clarity because it can’t really be done objectively. 

So the goal here might be impossible: to objectively define what makes a pitcher an ace.

The easiest (and laziest) way to define it would to be to say, “There are 30 of them – the best starter on each team.” That’s obviously bogus, and for a lot of reasons.

For some teams, there’s an obvious ace: Chris Sale and Felix Hernandez, for example. For a few teams, there could be multiple ace-calibur guys: the 2015 Dodgers (Clayton Kershaw and Zack Greinke) or Cubs (Jake Arrieta or Jon Lester). For many teams, there’s no clear ace at all: the Royals, for example, have a group of good starters but none of them are truly dominant. As much as we might call him “Ace,” Yordano Ventura is no where close to being an ace…yet.

However, every team does declare a #1 starter, and if they don’t say so explicitly, we can assume their best starter is the guy who throws on Opening Day. This yields names like Phil Hughes, Kyle Kendrick and Kyle Lohse, all Opening Day starters in 2015. We should all be able to agree these are not aces. There’s a distinct difference between an “ace” and a “number one.”

So where do we begin to create a definition amid this curious landscape?

First, we need a ranking system, which Bill James has so kindly constructed for us. Using his World’s #1 Starting Pitcher Rankings, we can see a list of all the MLB starters ranked from Clayton Kershaw to Matt Boyd.

If you want to know how the rankings work in detail, you can read more about it here. The short version is that every pitcher begins with a value of 300.0 then depending on their Game Score (which is calculated after every start they make) their overall number either goes up or down. So with each good start, a pitcher climbs the rankings as his overall score increases. If he has a poor start, or misses significant playing time, his score will decrease. The rankings are fluid. Think of them like golf or tennis rankings.

As of this post, Clayton Kershaw is currently ranked #1 with a score of 609.9. This is the third consecutive season he has begun the year as the #1 ranked starter in baseball. Prior to him it was Justin Verlander. Prior to him, Roy Halladay. Prior to him, Felix Hernandez, Halladay, Tim Lincecum, Halladay, CC Sabathia, Lincecum, Sabathia, Lincecum, etc., etc. etc. Players rise and fall. You get it.

I created a table of the Top 30 names going into each MLB season over the past 5 years as well as 2016. (If you’re on your phone, I recommend turning it sideways.) Here are the rankings…

2016

2015

2014

2013

2012

2011

Kershaw Kershaw Kershaw Verlander Halladay Halladay
Greinke Bumgarner Scherzer Kershaw Verlander Hernandez
Scherzer Hernandez Verlander Lee Lee Lee
Arrieta Scherzer Lee Price Kershaw Lincecum
Price Price Darvish Sabathia Hamels Sabathia
Bumgarner Sale Greinke Hamels Weaver Wainwright
Sale Lester Hamels Weaver Sabathia Hamels
Kluber Hamels Shields Cain Lincecum Oswalt
Keuchel Cueto Lester Shields Cain Verlander
Lester Greinke Sanchez Scherzer Hernandez Jimenez
Hamels Wainwright Wainwright Kuroda Lester Lester
Hernandez Zimmerman Price Dickey Carpenter Weaver
Cueto Kluber Hernandez Hernandez Haren Cain
deGrom Strasburg Bumgarner Gonzalez Romero Haren
Strasburg Darvish Sale Gallardo Kennedy Lilly
Lackey Weaver Weaver Greinke Shields Johnson
Zimmerman Dickey Dickey Latos Gallardo Kershaw
Dickey Shields Latos Fister Beckett J. Santana
Ross Samardzija Gonzalez Cueto Garza Greinke
Gray Fister Cain Halladay Price Rodriguez
Shields Lynn Kuroda Kennedy Wilson Arroyo
Archer Kuroda Bailey Sanchez Vazquez Sanchez
Quintana Gonzalez Lohse Lester Lilly Danks
Liriano Verlander Fister Lohse Greinke Price
Cole Tillman Wilson Bumgarner Rodriguez Carpenter
Lynn Sanchez E. Santana Harrison Santana Garza
Volquez Hughes Jimenez Arroyo Jimenez Billingsly
Teheren Quintana Strasburg Dempster Hudson Pettite
Carrasco Cobb Iwakuma Beckett Gonzalez Lackey
Chen Liriano Zimmerman Wilson Kuroda Guthrie

Two immediate thoughts. First: How did the 2011 Phillies fail? Second: What up, Jeremy Guthrie?

As I scanned the 2016 column, I began checking off the names I considered an ace entering that season. Kershaw? Yes. Greinke? Yes. Arrieta? Yes. But at a certain point, it gets hazy. For me, that certain point was Jacob deGrom. He’s the first person on the list who caused me to hesitate. My hesitation continues with Stephen Strasburg, but John Lackey is an definite “no.”

Moving over to 2015, I tried the same experiment. Adam Wainwright? Yes. Jordan Zimmerman? Yes. Corey Kluber? Won the Cy Young, yes. Stephen Strasburg? Yes. Yu Darvish? ….hesitation. I continue to hesitate on Jered Weaver, R.A. Dickey and James Shields, until I get to Jeff Samardzija and can easily say “no.”

Basically, just by using the eye test and our memories, each year can be split into three groupings: The Obvious Aces. Hesitations. Definite Nos.

As we think back to Opening Day 2014 and beyond, it gets harder to remember how we viewed each guy on the list at the time. However, I think our overall perception is better a few years later than it was in the moment. There’s no recency bias. I’m not compelled to call a guy an ace because I’ve seen his most recent body of work.

Here’s a good case: Chris Archer vs Doug Fister. There’s a chance Archer could take another step forward and be a legitimate ace in 2016. Or he could backslide. Okay, now look at Doug Fister: on the bubble from 2013 to 2015, and there was a chance for a few years he could’ve taken that last step, but he didn’t. This past season’s regression showed us he probably peaked in those years and isn’t an ace. Going into 2015, I might have been compelled to consider Fister an ace. Today, it feels silly to have ever considered it. I’m compelled to give Chris Archer the benefit of the doubt today, but a year or two from now, we’ll know the full story and will be able to look back with confidence.

As we go from column to column, here are my last “yes” all hesitations and first “no” for each year:

2016: Cueto (Yes), deGrom, Strasburg, Lackey (No)

2015
: Strasburg (Yes), Darvish, Weaver, Dickey, Shields, Samardzija (No)

2014
: Sale (Yes), Weaver, Dickey, Latos (No)

2013
: Hernandez (Yes), Gonzalez, Gallardo, Greinke, Latos (No)

2012
: Shields (Yes), Gallardo, Beckett, Garza, Price, Wilson (No)

2011
: Lilly (Yes) Johnson, Kershaw, J. Santana, Greinke, W. Rodriguez (No)

Maybe you disagree with me somewhat on where you stopped saying “yes” and started saying “no”, and that’s understandable. Each of us views these things somewhat differently – I love Zack Greinke, for example so I continued to hesitate on him in 2013 and 2011 when you may have been quick to say no. That’s fair. But generally, this is the area of the chart where, for me, I begin to question the label.

I went back to the rankings and looked at the scores, hoping to find some sort of correlation between the numbers. A trend developed. Again, these are their scores during the offseason between each season. The year above is the upcoming season.***

*** – By the way, I used the date of this research, February 5, as the date for each of these lists, only changing the year. I realize it’s an arbitrary offseason date, and I should probably choose Opening Day each year which varies up to a week each year. But for the sake of simplicity, I’ve used 2/5. Another note: these are offseason numbers, which decline steadily between the final day of the regular season and Opening Day.

2016
Cueto (495.1)
deGrom (489.5)
Strasburg (488.8)
Lackey (485.1)

2015
Strasburg (498.5)
Darvish (492.9)
Weaver (490.6)
Dickey (489.4)
Shields (488.5)
Samarzija (486.5)

2014
Sale (494.0)
Weaver (493.7)
Dickey (492.0)
Latos (488.9)

2013
Hernandez (504.3)
Gonzalez (500.0)
Gallardo (493.7)
Greinke (493.2)
Latos (489.3)

2012
Shields (500.7)
Gallardo (493.6)
Beckett (492.6)
Garza (491.0)
Price (490.4)
Wilson (488.1)

2011
Lilly (500.3)
J. Johnson (499.6)
Kershaw (498.6)
J. Santana (493.3)
Greinke (491.8)
W. Rodriguez (487.2)

Do you see the trend? Even in just glancing through the names, I somehow stumbled on a consistent set of scores. I’m actually shocked there’s some level of consistency here, but somewhere between 485 and 500 is the offseason barrier between ace and non-ace.

Now, this still isn’t totally objective because each of us varies in how strict we want to view the term, but it seems to me that somewhere in this range of scores is the answer to our question.

I have a few remaining questions though, that I’m not sure I’ve figured out still.

  1. Scores decline slowly but consistently during the offseason. Then as the season progresses, the top numbers rise through the season. The gap between the Opening Day low and Game 162 (and postseason, for those eligible) high is around 40-50 points. Is there some sliding scale we can create so that the 485-500 range that works today will be consistent in May, July, and September? Probably easy to do, but that’s for another post.
  2. What do we do with injuries? Are Matt Harvey and Adam Wainwright aces? Do they get grandfathered in somehow due to their past dominance, or do we require them to prove they still deserve the label and work back up the rankings?
  3. What do we do about a guy who has a meteoric rise one year, but hasn’t sustained it over time? Can we really call Jake Arrieta an ace, or does it take some time to establish himself?
  4. What do we do we call a guy with a 500+ score who isn’t the #1 starter on his team? He’s technically not a staff ace, but he still has all the other qualifications. Is there a name for that? Deuce? King? Off-Ace? Grasping.

Still working on the details, but it seems there is an objective way to say whether a starter is or is not an ace based on Bill James’s ranking system. I’ll have to do more research to determine what those barriers are over the course of the season. It’s just an algebra problem that needs plotting. Let me get my TI-83+ and follow up later.

Although, it’d be a lot easier if we just asked, “Is he better than Mat Latos?” and called it a day.

-apc.

 Image Cred: LA Times accessed here.