Rating Strength of Schedules and Comparing those of the top Contenders

Catch 5's picture
November 21, 2013 at 2:37p

A few days ago, user Alex Pavlick wrote an entry in the forums about OSU's current win streak and the probability of it occurring.  He used Vegas odds to determine the likelihood of individual wins and used that to calculate the probability of winning them all.  It is an entirely valid exercise but I disagree with using Vegas odds for the data set.  Vegas odds are figured from the perception of those teams at that time.  Teams such as Northwestern and Penn State were perceived as better opponents at the time the game was played, and the odds were skewed toward that effect.  This is the same thing as claiming a win over a ranked opponent because they were ranked at the time - even though they have several bad wins now.

I'm writing this not to be critical of Mr. Pavlick but to offer another option as to how to find this result.  A few years ago, Matt Dover over at Roll Bama Roll wrote a piece comparing strengths of schedule for top teams.  It is a fascinating read for anyone interested.  He compiled data over 4 years and found the win % of top-12 schools against schools of various rankings.  From these percentages he produced a table giving the probability of loss for teams based on how good each opponent is.  This is almost exactly how Mr. Pavlick found his, except we can use up-to-date rankings to find our results. 

Here is the table, modified for win% (As Dover gave it in loss%)

Opponent Ranked      0-12  Opponent Ranked      13-25 Opponent Ranked       26-52 Opponent Ranked      >52   
Home Neut Away Home Neut Away Home Neut Away Home Neut Away
68.0 50.0 32.0 72.3 76.0 58.3 86.3 92.9 83.5 99.3 100 99.3

I used (as Dover did in the origional article) FEI rankings because they rank all FBS schools.  There are issues with some of these rankings, but for the most part they are pretty fair (FEI gives Wisky and Mich State higher rankings than the BCS so it is fair to OSU in this exercise even though it has OSU ranked lower).  In Pavlick's forum post, he found the probability of OSU's current win streak to be 0.805% (massaged up to 3.72%).  By using the method given here, that probability is 8.1%.  Though that is significantly higher, it is impressive nonetheless.  Furthermore, the probability of OSU finishing the season undefeated from this point is 63.5%, and the probability of a win streak that long (against those opponents) is 5.1%

Another exercise to look at and discuss is the strength of schedules.  OSU has been raked over the coals and has looked to be left on the outside because the B10 is weak and their OOC schedule did them no favors.  Looking at this year alone, OSU's probability of going undefeated is 55.5% to this point and 35.2% for the whole year (assuming Mich St in the B10CG).

How does this stack up against the other top teams?

Bama has also been criticized for a weak schedule (I was among them earlier in the season) but Ole Miss playing well and Auburn's unexpected excellence has changed that a lot.  Bama's probability of being undefeated at this point is 22.3%.  The likelihood of them winning out (including an SECCG against Mizzou or SC) is 15.9% for a season-long odds of 3.54%.  This is by far the most difficult of the 4 teams I looked at.

Florida State had a 40.0% chance of being undefeated at this point, and after the 77.0% chance of the remaining schedule (assuming Duke in the ACCCG), have a total strength of schedule of 30.8%.

Baylor had a 68.6% chance of being undefeated at this point, and face a 50.0% chance of loss coming up, for a total schedule strength of 34.3%

One last chart to sum it up:

Team                                          % to date     % remaing    % total Schedule
Alabama 22.3 15.9 3.54
Florida State 40.0 77.0 30.8
Ohio State 55.5 63.5 35.2
Baylor 68.6 50.0 34.3


Comments Show All Comments

oldbuckeyedave's picture

agggghhhh more math


AndyVance's picture

The flaw in this system, of course, is the same as with the Vegas Odds system: namely, that the rankings assigned to teams by the voters or the algorithm have an inherent subjectivity... In other words, in the case of the FEI, it is patently silly to think that a two-loss team should be ranked ahead of the other teams in the top 9 of the rankings you linked. I realize that the FEI is trying to measure something a little different - efficiency, in this case - but you still have the issue that the input has a significant effect on the output.
Your version of the system would certainly support the current narrative, of course, but kudos to you for at least underscoring what is obvious to everyone not writing at a mainstream sports site today: the probability of going 22-0 in the FBS is very, very small.

Catch 5's picture

But is it not silly to accept that a top-10 team could lose two games if it played 2 other top-10 teams?  If that same team were to have an easier schedule, they would be undefeated.  If the efficiency shows they are among the best, and their losses were to only top teams, that must be accepted as a possibility.
I agree, the FEI is flawed as well, but it at least has current data as opposed to time of game.  I would prefer to use a BCS or AP ranking, but they don't include all the teams.  I think Sagarin does with his poll, but I don't think it is up to date - but I haven't look it up either.  If anyone knows of any other current ranking of all the FBS teams, I'll gladly re-calculate these numbers.

Make their asses quit! - Nick Saban

AndyVance's picture

Sagrin is usually updated each week, and actually the FEI link from your post still has Ohio State at 9-0, so it's apparently a week behind, anyway.
I get the argument that a team losing to a pair of Top 10 teams is potentially more impressive than an undefeated Fresno State, for example, but in the final analysis, it's like the old question about transcripts: is it better to get a B in a hard class or an A in an easy class? Answer: neither. It's better to get an A in a hard class.
Until and unless the FBS goes to a true playoff system (as opposed to the mere 1-off we're getting next season), wins matter, and "good losses" are still losses. Stanford might be the best 2-loss team in the country, but the fact that they lost to a 4-loss Utah indicates that they are not, empirically speaking, better than a Wisconsin team that got hosed in its Pac-12 game, or better than an undefeated Ohio State team that defeated two ranked opponents (and will likely finish with two ranked opponents on its schedule, still).

I_Run_The_Dave's picture

This system doesn't seem to incorporate the current rank or value of the team in question, it only takes into account their opponent.  But I suppose this might be valid, as a 50th ranked team that plays #55 might be as hard as #1 playing #6, but you wouldn't argue that the 50th ranked team had a harder schedule.  You would, however, say that their probability of winning is the exact same.  
But, if you are going to boil the statistics down to that level, then the probabilities that you arrive at are relative and any team defying that probability is resorted to luck and random chance, rather than to being a good (or bad) team.  Of course, any discussion of probability does this to some extent, and I would say that looking at probabilities is not the proper way of evaluating the accomplishments of any team.  Any team, no matter how bad (or good) the team is, can string together consecutive wins (or losses) if you increase the number of games played enough.  
If an event has a 1% chance of happening, then if you try 10,000 times, it is expected to happen 100 of those times.  It is therefore possible that FAMU beats Ohio State in the first game they play, even if they would lose the next 99 straight if those games were to be played.  This does not make OSU worse nor does it make FAMU better; just that the occurrence of FAMU winning happened to be drawn from the hat first.
That said, how do you evaluate the accomplishments of a team?  I suppose if there was a definitively good answer, we wouldn't be having these types of discussions.  I have criticized some of ESPN's metrics on this board where Stanford with two losses is still posted as 3rd.  This could be a valid result if you keep a proper understanding of probability in mind.  You could say that Stanford was just unlucky.  
In the end though, as so many have said, "just win, baby!"  That is all.

Catch 5's picture

Sorry Dave, I should have expressed better that this is just for top-12 teams.  The probability of a team ranked higher would be even more difficult.  If you read the original post I linked, Matt does a much better job of explaining that, and why.

Make their asses quit! - Nick Saban

unholy bucknut's picture

This sounds like a big bang theory argument. NERD FIGHT!!!!

AndyVance's picture

One more thought that hasn't been said... At the end of the day, the difference between being BCS #3 and BCS #4 is irrelevant. If Ohio State does not finish in the top 2, it will not go to the BCSMNCG, and will go to the Rose Bowl. It does not matter if Baylor jumps the Buckeyes, as Baylor will still go to the Fiesta Bowl.
This only matters if/when FSU and/or Bama loses.

Catch 5's picture

Agreed, but it is a good discussion to have.  I have been of the mindset all along that OSU had by far the easiest route to undefeated out of the top 4.  While at the end of the day, that will be true (per the FEI rankings), they are much closer than I had thought - close enough that last year's undefeated record would begin to come into play.  I would probably keep them ahead of Baylor and maybe even move them ahead of FSU depending on how they play MSU in the B10CG.
Of course, I don't have a vote.  I'm just a Bama fan who likes to talk football so maybe you're right afterall...

Make their asses quit! - Nick Saban

Oyster's picture

You guys ready for that Chattanooga game?  Is that home or away?

"Scrolling hurts my finger"

(and FitzBuck was clearly the winner)

Haybucks's picture

Never have I seen where the sarcasm font was needed less than with those two questions.

If the world comes to an end, I want to be in Cincinnati. Everything comes there ten years later. -  Mark Twain


Whoa Nellie's picture

Maybe I don't understand this other system of calculating the odds of winning/losing v. us.
Does this other method ignore the effect of getting beat by us, and that losses' effect on subsequent games?  Maybe the perception of the opponent at the time of our game against them was absolutely correct.  Their psyche was certainly different then than after we beat them.  Maybe the teams have suffered injuries since then that have depleted them, e.g. NW. IMO, you can't change the reality that exist AT THAT TIME based on events that happened later.  The teams are not the same then v. later.  There was a reason that someone, i.e. Vegas, thought those teams were that good then. 

“Don’t fear criticism. The stands are full of critics. They play no ball. They fight no fights. They make no mistakes because they attempt nothing. Down on the field are the doers, they make mistakes because they attempt many things.”

ATXbucknut's picture

I always hit a wall at some point when thinking about math.

dumpus's picture

i'm not sure i agree with discounting the vegas odds at the time; statistically, they're relatively unbiased for the purposes of extrapolating relative team strength over the course of a season.  take for example USC; beating USC now is much more impressive now than it was two months ago - current ratings overcredit a september ass-whooping because they truly were a bad bad team.
also kinda sheds light on the fact that, statistically speaking, rankings shouldnt even be thought about until approx week 5 or so...bit thats a whole different story

WashingtonStateBuck's picture

It was my understanding that there would be no math on this blog.