The Diff: On quality starts and the Indians bad pitching

The Diff

It’s always amazing when last week’s topic is just as timely a week later. I’d encourage everyone to check out last week’s edition of The Diff on the Cavs and their second-half failures of late. This week’s topic: Quality starts and how it relates to the Indians’ awful 2012.

The Diff

By now, you’re probably quite distressed with the Cleveland Indians season so far. You witnessed a solid start to the season in Toronto, then back-to-back shoutouts in Tampa Bay, followed now by back-to-back pitching disasters at home against the beat-up New York Yankees. It’s already getting ugly in Wahoo Town, as TD wrote earlier this morning. Yet, if you paid much attention during the 2012 Indians season, this should all seem quite familiar. Today, I’ll dive in-depth into the 2012 American League and the quality starts statistic. It should be fun1.

Obviously, the Indians starting pitching was quite bad in 2012. We’ve easily reviewed that dozens of times in the last six months since last season’s disaster came to a merciless end. But this season’s start had me thinking about placing Cleveland’s pitching atrocities in the context of a familiar term: a quality start. The reason why I was thinking about this is because the Indians began 2013 with back-to-back quality starts against the Blue Jays. Optimism was high, as everyone knew this team could only go as far as its pitching.

Last year as well, the Indians similarly began with three straight quality starts against the Jays2. But overall in 2012, en route to the 68-94 total record, how many quality starts did the Indians have? They had 73 quality starts. Which, because of its close proximity to the team’s 68-win total, had me thinking about all sorts of related theories. Maybe there’s something there about the percentage of a team’s quality starts to non-quality starts? Maybe looking at the entire American League in 2012 would show some answers?

A few caveats before I share the results: As a statistics person myself, of course I’m not a huge, huge fan of the quality start statistic. It’s fairly arbitrary: The only qualifications are that a starting pitcher must complete at least 6.0 innings pitched while allowing no more than 3 earned runs. In a way, a quality start also promotes mediocrity: Simply hitting the minimum requirement every start is a 4.50 ERA, which is worse then league average. Overall, it’s just not a great stat. But it’s simple, it relates to this starting pitching conundrum and I was intrigued by how total quality starts compared to team’s wins.

Thus, thanks to my spreadsheet of every single start by a 2012 American League starter, here is your chart as sorted by the number of quality starts:

TeamWLPctQSNQSPctStarter ERATeam ERA
AL 20121,1501,1180.5071,1341,1340.5004.394.08


Looking at the chart above, one can fairly easily see that the distribution of quality starts relatively mirrors the normal expected distribution of wins. The comparison of QS (Quality Starts) to NQS (Non-Quality Starts) does look like a typical record: In the American League in 2012, that ranged from 62-91 quality starts, while the teams ranged from 66-94 wins. Also, the entire AL exactly had 1,134 quality starts and 1,134 non-quality starts. So that’s fun3 too.

Obviously, the discrepancy between quality start success and actual wins can most logically be portrayed in two missing stats from above: Offensive production per game and bullpen ERA. Those are the two obvious statistics one might want to see next for their connection to the chart above. I’ll get to that momentarily in my next chart. This also would infer that good teams are better at converting quality starts into wins, while bad teams are worse. Correlation, eh?

Also something I want to showcase: AL teams’ records in QS compared to their records in NQS. What are the usual expected winning percentages for a quality start? How about a non-quality start? What teams are some of the notable outliers?

It might not be too shocking, but in fact, the Indians do appear to be in near-elite (i.e. bad) company for what they accomplished in 2012. Let’s go to the next chart:

AL 20127953390.7013557790.3134.453.55


Potentially thanks to having the second-worst offense and the second-worst bullpen ERA, the Indians by far were last in the league with a 0.169 winning percentage in non-quality starts. I’ll get to some more details on that momentarily. But let that marinate for a moment.

For now, I’d like to enjoy the breakdown: American League teams won about 70% of games in 2012 when their starter delivered a quality start. In non-quality starts, 2012 AL teams only won about 31% of games. That might seem potentially expected, but it’s still quite a neat statistic to write out in clear daylight. Because of this relatively simple rule, it will always be hard for a team with just 73 quality starts  (a la the ’12 Indians) to be even .500. The math just doesn’t normally work out.

Again, as I theorized above, you can easily tell how some of the better teams in the AL did better in both QS and NQS winning percentage. This is a factor of R/G and bullpen ERA, which also both have huge impacts in the overall formulation of a team’s expected winning percentage. Again, this seems pretty easy to understand.

Now, on the flip side of the Indians’ struggles, here are some of the positive outliers: Baltimore was a sensational 0.846 in quality starts in 2012. They also went 29-9 (.763) in one-run games, by far leading the AL. Those two stats, along with the fact that per their run differential, they only resembled a 82-win team, should obviously scream extreme outlier and regression for 2013.

The New York Yankees were the best in non-quality starts with a 0.405 winning percentage. They also led the AL with 17 wins when trailing after 5 innings last season. Logically, this is due to their excellent offense which ranked a close second in the league.

With two of the worst bullpens in the league, Toronto (.573) and Boston (.583) were the only two teams to finish sub-.600 in quality starts in 2012. The Blue Jays were a disaster down the stretch, much like the Indians4, while the Red Sox were fairly similar too under Bobby Valentine.

So the Indians. No other team in the AL had worse than a 0.258 winning percentage in non-quality starts. Every other team had at least 20 such wins. So the Indians were truly awful, awful, awful. But it begs further research: Why might the Indians have been so bad? When was it the worst?

Thus, before I move into the second main segment of today’s article, I wanted to share this breakdown of the Indians season. I’ll update it again at the end of this article before my final summary:

Apr-Jun (77)29110.72510270.2700.519
Jul-Oct (85)2490.7275470.0960.388


Wow. So the Indians were consistent all season long in quality start winning percentage. That winning percentage is slightly better than league average, which is quite good for such a bad team overall. In terms of a logical construction for why that might be, it’s probably because the team’s back-end of the bullpen is significantly better than its other pieces5.

Then, of course, one easily sees how the Indians were fairly reasonable in non-quality start winning percentage through the first 77 games of the season. Everything tanked after that. Again, no other AL team had worse than a 0.258 winning percentage in this category. The Indians went on an 85-game stretch with a 0.096 winning percentage in non-quality starts.

Also notable is the fact the Indians were only getting quality starts in about 39% of outings in the second half of last season. Since again, the average AL team in 2012 did that exactly 50% of the time, that’s also quite horrendous for a sustained period of time. Now, I’m going to transition into a related tangent that might break down more about this awful stretch.


I mentioned in the onset, and for good reason, that I’m not a huge fan of the quality starts statistic. My two main points above were that it is arbitrary and rewards mediocrity. But I’ve certainly been intrigued enough to write all that I have thus far. So I felt I’d push the topic one step further: Creating a new set of quality start-esque statistics that more accurately represent the ratio of bad-good starts in baseball.

After a little bit of trial-and-error6, along with some initial help from the Bill James’ Game Score algorithm for evaluating starts, here’s my chosen breakdown:

BS526894370.1697.50+ ERA
OS6082633450.433Other Starts
QS6063742320.617Rest of QS
SQS5284241040.8037+ IP, <3 ER


Thus, here are our four distinct, independent pitching statistic categories:

  • BS = Bad Start. This is defined as any start that finishes with a ERA of 7.50+. In relatable terms, this is like allowing 5 earned runs in 6.0 innings pitched, or something worse on that scale.
  • OS = Other Start. Yeah, I ran out of specific categories. This is everything that doesn’t qualify as a usual, technical quality start and a Bad Start. But fortunately, this fits the same pattern.
  • QS = Quality Start (adjusted). Instead of the typical category of quality starts, this is all quality starts that don’t define as the new statistic below.
  • SQS = Super Quality Start. As I’ve mentioned, a typical quality start promotes mediocrity. This one doesn’t: It’s any start of at least 7.0 innings pitched while allowing 2 earned runs or less. That’s an ERA of 2.57 or better. So significantly improved, as it shows in the winning percentage.

Again, dissecting the binary nature of only having quality starts and non-quality starts gives us a more refined look at expected winning percentages. Instead of only 70% vs. 31%, with these new pitching statistics, we now have 17% vs. 43% vs. 62% vs. 80%. It’s a bit clearer: Bad Starts really usually lead to losses, Super Quality Starts really usually lead to wins, while the rest are practically toss-ups.

Thus, now, here is a look at every 2012 AL team based on their number of starts in these new categories and their respective winning percentages:

AL AVG37.70.80343.30.61743.40.43337.60.169


Before I get into the Indians perspective again, here are a few league-wide notes: Again, we see the Baltimore Orioles being fluky. They went a spectacular 34-0 last season in SQS. Second-best was Texas at 39-5. … Oakland was the best in both QS (33-10) and BS (9-26). They had a helluva run last season, yet still actually were below league-average in the other two categories. … Tampa Bay led the league by far with only 19 bad starts. That’s how they led the league in starter ERA despite not having the most quality starts. Meanwhile, Minnesota’s no-name rotation only had 19 SQS. It’s tough to win that many games while doing that.

But again, the reason for all of this back-and-forth analysis is to discuss how it relates to the struggles of the 2012 Indians season. That year was awful and the starting pitching was a key reason why. Now, with these new starting pitching categories, here is how the Indians fared in those splits I shared above:

Apr-Jun (77)170.765230.696160.375210.190
Jul-Oct (85)71.000260.654240.083280.107


So this is where things get interesting. As we saw in the previous chart of all 2012 AL teams, the Indians ranked last by far in winning percentage in OS (.200). The league average is .433. Starting as of July 1, the Indians tanked to a record of 2-22 when their starter had an OS. That’s downright awful compared to the rest of the league.

In all other categories, the Indians weren’t that miserable. Obviously, they were slightly worse in both QS and BS down the stretch. But the biggest difference again, specifically from those non-quality starts, were in the mediocre ones that qualified as Other Starts.

As one final last research point before summarizing this article, I wanted to more clearly define what this OS statistic looks like. Here are the four kinds of starts you might see compiled here:

  • Barely too short to be a QS — A start that is 5.2 innings pitched of no-run baseball would actually qualify as an OS, per this definition. So Jerremy Hellickson’s season-ending such performance on Oct. 3 against Baltimore qualified as the best Game Score (71.7) for any OS all season.
  • Barely too many runs to be a QS — A start that allows more than 3 earned runs can never be a quality start. So although on Apr. 11 against Tampa Bay, Justin Verlander dominated through 8 innings, he then allowed 4 ER while failing to finish the ninth, so it’s also an OS and not a quality start.
  • The average OS — Per Game Score, the average OS had a rating of about 46.0-46.5. So that would be about a start of 5-6 innings pitched, with four earned runs. As an example, think of this classically mediocre Ubaldo Jimenez start that he had on June 16 against Pittsburgh.
  • Bad, but not quite awful enough to be a BS — My version of a Bad Start was arbitrarily defined as one that finishes with a 7.50 ERA. So obviously, there are dozens and dozens of permutations of starts that are quite awful, but not a 7.50 ERA. The worst Game Score for an OS in 2012 was Rick Porcello’s 24.0 on July 5 against Minnesota.


Executive Summary:

In lots of words and charts, this article aimed to show the context for the Indians’ starting pitching struggles in 2012. Starting from the basic, they only had 73 quality starts. Per the usual similar correlation7, that’s usually just not enough to cut it for a potentially winning team. Outside of Baltimore’s fluky season, every other winning team had at least 83+ quality starts.

Then, getting more specific, and likely because of the team’s poor offense and poor bullpen depth, the Indians were the worst AL team in non-quality start wining percentage. Most exactly, this came down to the team’s record in Other Starts in the final 85 games of the season from July through October. From pure regression and a likely improvement of the bullpen, the Indians should likely be expected to do a little bit better in this category in 2013.

But most importantly, this just goes to show scientifically how important quality starts can be. Could you imagine if the Indians had a staff like Tampa Bay’s with only 19 Bad Starts in 2012? Obviously, we’ve seen a couple of those already for the Tribe in 2013, in just eight games. Moving forward, this should be able to help us all more clearly define the awfulness of these starts and to more realistically curb our expectations based on the kind of start.

  1. in a statistical way, because the “WFNY” mentality and really bad pitching is not necessarily actually fun []
  2. This is one of my favorite seldom-known facts of that disaster of a 2012 season. The Indians only went 1-2 in that opening series, but at least the starting pitching was pretty decent. []
  3. Sorry for the tease, this is again the same kind of sadistic statistical fun I was talking about before. Not like actual fun like Cedar Point or vacations. []
  4. As much as being a Tribe fan is awful right now, we should at least feel empathetic for those Canadians. They started 51-49 last season through July 28, then with catastrophic starter injuries and more, collapsed to the tune of 22-40. Then, after a gigantic spending spree and poaching all the best talent from the Marlins, they start the 2013 season 2-5 and with serious questions about ace R.A. Dickey. Uh oh. []
  5. I covered this concept in more depth last month. The results: Joe Smith-Vinnie Pestano-Chris Perez had a combined 2.77 ERA in 2011-2012. All other Cleveland relievers had a 4.51 ERA during this span. For more context again, the average AL bullpen ERA was 3.81 and 3.55 in these seasons, respectively. []
  6. Obviously, retro-fitting some desired patterns to a similarly arbitrary set of pitching rules doesn’t perfectly solve the problem. But at least this breaks down starts a bit more succinctly — and more accurately — than just quality starts vs. non-quality starts. []
  7. For those intently curious, the correlation r-value of quality starts to wins among 2012 AL teams was 0.39337. So not that strong by any means. The average winning percentages are a bit clearer. []
  • The_Real_Shamrock

    Phew, I feel better now! ;o)

  • The_Real_Shamrock

    But this is one of those times where stats aren’t needed to tell the story, unfortunately. We all questioned the starting rotation but man eight games is entirely to early to be proven right. Btw no one check the AL Central standings, it’ll just make things worse!

  • Ryan

    Great job as always Jacob! I do appreciate all the hard work that must go into these articles.

  • dwhit110

    Fascinating! I really loved this article. From a data perspective it makes things so much more interesting than “we need better starting pitching, we need a RH bat, etc.”

    You start to understand some of what the front office was thinking this offseason. Keeping Perez could help to keep our advantage in converting QSs to wins, upgrading the offense could help us in OSs, etc.

  • mgbode

    no doubt. I had some questions I was going to pose while reading the article and he answered them all by the end (right to footnote-7).

  • mgbode

    one non-Tribe thing: I knew Seattle had good SP last season, but I didn’t realize how good. 2nd most SQS and among best QS overall. if only they had Morse last year.

  • Kildawg

    Indians already using 7th different starting pitcher tonight (Masterson, Jimenez, Myers, McAllister, Bauer, Carrasco, and now Kluber). This already leads MLB, not the category you want to be leading in, especially in April.

  • JacobWFNY

    Thanks Ryan and mgbode, that means a lot to me. I’m glad you both enjoyed.

    I do generally put in a lot of effort and time into these. My Tuesday nights are often spent writing and researching.

    For this idea, I had been researching since last week. So as opposed to normal, I had the vast majority of my background spreadsheet work done before Tuesday this time around.

  • JacobWFNY

    Agreed with you on a handful of ideas there. Really, the stats above almost are getting too overtly specific that they’re nearly becoming abstract. In the end, it just comes down to assembling a league-average pitching rotation. Obviously, that wasn’t the case last year.

  • JacobWFNY

    Certainly helps to have Felix Hernandez. He had 20 SQS in 2012. For context, Justin Verlander had 19 and David Price had 18. Then, a large group of pitchers (incl. Justin Masterson) had between 11-14 SQS.

    (AL starts only)

  • Ezzie Goldish

    Great stuff. I’ve always argued to friends in past fantasy leagues who argued that Ws were totally random that Ps who throw a lot of 7ip, 2er games will get you far more Ws than a similar net statistical P who does not. I think this somewhat proves me right. I also always argued as you do that it should be at least that to define as a QS (or SQS).

    I’d suggest coming up with better names and putting this definition/statistic out there, it’s fantastic.

    The big flag for the Indians to me is those “Other Starts”. That they won only 20% of games that most teams win 43% is crazy. That’s a 9-win swing. I’d say that points to a combination of a poor bullpen but more likely a horrible lineup offensively that can’t come back when down a couple runs. Perhaps they thought this season’s additions could net them 9 wins just from that?

  • Kildawg

    Too bad the only Major League caliber starters in the rotation are Masterson and McAllister. Jimenez is up and down, Myers is better suited for the pen, Kazmir is hurt, Carrasco apparently doesn’t have the mental makeup, and Bauer/Kluber aren’t quite ready but already pressed into early service.

  • Steve

    The park.

  • mgbode

    Chan Ho?

    yes, there definitely is a park factor there, but it was still higher than I thought it would be.

  • steve-o

    Nice work. Those stats really help show how important starting pitching is, and how badly we needed to improve that area this off season. Why the FO chose to spend so much money while not addressing our biggest weakness is beyond me. For 3M more we could have had Losh this year instead of Myers. That’s going to bug me all season.

  • Harv 21

    This is great, Jacob, stats extensive enough to a variety of scenarios and then broken down meaningfully with an eye toward what actually happens in games. Not just gobs o’ stats followed by conclusions.

    Nicely done.