Stats models and rankings

[slugboy]

Most of the stats models that get brought up (FEI, F+, SP+, Beta Rank, Sagarin, etc.) aren’t just ranking systems (like the CFP poll), but they’re also there to predict who will win a game. It’s tough to keep track of all of them if it’s not your full time job. It’s not my full time job. They kind of keep track of each other, though.

Here’s one modeler who keeps track of how all the models do week-by-week. You can see which one picked the best at
https://public.tableau.com/app/profile/andrew.percival/viz/CFBPicker/Standings.
He explains which models he tracks and what they are at


The people who do sports-reference.com (from where I post a lot of game stats) does a Simple Rating System (SRS) that picks games. It currently has the ACC 5th out of the P5, and we’re #54 https://www.sports-reference.com/cfb/years/2023-ratings.html.

There’s Jeff Sagarin, (who also doesn’t rate the ACC highly so far this year), and has us at 78th.

 

[slugboy]

I should add @ibeattetris and @GetYourBuzzOn here, to add in more info.

A lot of the models have “priors”. In the case of FEI, it’s the last 5 seasons., so it’s one of Johnson’s worst seasons followed by 4 of mostly Collins. We started that model this year with an F- average to dig out of.

The overall consensus for us (averaging out all these models), is #78. That doesn’t really mean anything, especially 4 games in, but if you have 9 models, hey why not average them?

There are the Massey Ratings. We’re #70 on the Power Ratings and #67 on the regular ones.

Some models tell more about how they work. FEI and Beta Rank and Sagarin and SP+ will tell how they’re calculated. and what the inputs are. Others are harder to sus out.

FEI and SP+ were popular among a lot of people because they were easy to find, and explained how they worked. SP+ is now in the ESPN paywall, so you either pay for ESPN+ or you turn to another model.

And, as a complete aside, SMU might be the best team we added in expansion. Even before becoming a P5 team.

There’s TeamRankings—we’re #58. I have no idea how they build the model.

There’s Doktor Entropy. He does weekly predictions, and picked Wake by 8. Not sure how his model works, but he’s been selling it for ages.

There’s ESPN’s FPI. It seems to think we’ll have between 5 and 6 wins this year. Ranks us 49th. It’s the one you know about, because it’s ESPN.

There’s Kelley Ford (KFord Ratings). He’s on Twitter and will explain his model. It holds up very well in Pick’em contests. We’re up to #57.

The last two are Brian Fremeau’s FEI (at bcftoys.com) and Bill Connelly’s SP+ (over at ESPN, paywalled)

Outside of those, there are the graphs and info at gameonpaper.com and cfb-graphs.com, plus other info at https://collegefootballdata.com/.

Unless we beat Bowling Green 222-0 (or even if we do) Miami will still be favored in game 6. However, we’re 2-2, if we beat Bowling Green, we have a winning record for the first time this season.

 

[roadkill]

Here’s an interesting ranking he doesn’t use: https://www.colleyrankings.com/rank.html

In this system, where .5 is average, GT now sits at .54, good for 67th. His ranking system was designed to converge on a valid national champ at season’s end, or to be useful in comparing playoff-eligible teams. He has a link to a paper explaining his mathematics in great detail. It's kinda like the transitive property on steroids.

Colley’s is unlike most of the others in that it just uses current season wins and losses, and thus starts with nothing. This makes it less useful early in the season (Then again, what poll is?). At the same time, it claims that as an asset since it doesn’t use preseason or historical data as a bias, which has been a frequent critique of the current system where last year’s champion always gets the benefit of the doubt, and can prevent teams that don’t start highly ranked from ever getting into the playoff, regardless of record. In the modern game, given the transfer portal and frequent coaching turnover, those biases are less useful than 10-20 years ago.

 

[roadkill]

By the way, Percival’s current rankings have some real head-scratchers that may be indicative of preseason bias. For example, I can see FSU at #14 based on their struggles at BC and almost losing at Clemson. But Clemson still at #8 with a 0-2 ACC record and no P5 wins? Give me a break. I mean, they have a lot of talent but wins and losses need to count for something.

 

[Augusta_Jacket]

..., if we beat Bowling Green, we have a winning record for the first time this season since the first game of 2020.

 

[AugustaSwarm]

The Colley ranking system makes more sense to me in today's playoff format with transfer action. It looks like it values quality wins over everything else, which is how it should be.

 

[GT33]

Thanks for starting this thread and the follow on comments. True GT Men here.

 

[slugboy]

By the way, Percival’s current rankings have some real head-scratchers that may be indicative of preseason bias. For example, I can see FSU at #14 based on their struggles at BC and almost losing at Clemson. But Clemson still at #8 with a 0-2 ACC record and no P5 wins? Give me a break. I mean, they have a lot of talent but wins and losses need to count for something.

That’s the preseason one—I linked it because it explains the blended models.

The current one doesn’t have Clemson in the top 15. FSU is still down at 12, though.

Clemson is at 18 and we’re at 64. Lots of preseason bias still in.

 

[kittysniper101]

Which models are fed by ESPN play by play data? Since the Louisville game isn’t in there our rating will be off.

 

[slugboy]

Which models are fed by ESPN play by play data? Since the Louisville game isn’t in there our rating will be off.

I don’t think any of them are.

 

[roadkill]

The Colley ranking system makes more sense to me in today's playoff format with transfer action. It looks like it values quality wins over everything else, which is how it should be.

That's basically how it works, and if you are willing to wait a few weeks into the season for it to get meaningful data, it has tended to be accurate in the sense of both predicting the final playoff teams as well as just seeming about right, even if it doesn't agree with the major polls.
For example, it ranked us 9th at the end of our 11-3 2014 season. I think we were 8th in the AP if memory serves. But in 2016, after wins over uga, VT, and Kentucky in the Gator Bowl, we ended the season unranked. I thought this was showing us disrespect. Colley had us ranked 21st, which feels about right.

 

[LongforDodd]

Most of the stats models that get brought up (FEI, F+, SP+, Beta Rank, Sagarin, etc.) aren’t just ranking systems (like the CFP poll), but they’re also there to predict who will win a game. It’s tough to keep track of all of them if it’s not your full time job. It’s not my full time job. They kind of keep track of each other, though.

Here’s one modeler who keeps track of how all the models do week-by-week. You can see which one picked the best at
https://public.tableau.com/app/profile/andrew.percival/viz/CFBPicker/Standings.
He explains which models he tracks and what they are at


The people who do sports-reference.com (from where I post a lot of game stats) does a Simple Rating System (SRS) that picks games. It currently has the ACC 5th out of the P5, and we’re #54 https://www.sports-reference.com/cfb/years/2023-ratings.html.

There’s Jeff Sagarin, (who also doesn’t rate the ACC highly so far this year), and has us at 78th.

I've followed sagarin for several years. I scratch my head with the company he keeps us with.

 

[78pike]

That’s the preseason one—I linked it because it explains the blended models.

The current one doesn’t have Clemson in the top 15. FSU is still down at 12, though.

Clemson is at 18 and we’re at 64. Lots of preseason bias still in.

This ranking seems to be the best I have seen by far. It appears to be consistent with on the field results mores than any other I have seen. It appears to be a ranking system that is done by someone actually watching the game as opposed to reading stat sheets after the game (at least the top ten is anyway).

 

[slugboy]

I've followed sagarin for several years. I scratch my head with the company he keeps us with.

That one is just ELO-chess. The predictor is based on margin.

That’s mainly on us. Win and you move up. Even stay close, and you’ll do well.

 

[yeti92]

This ranking seems to be the best I have seen by far. It appears to be consistent with on the field results mores than any other I have seen. It appears to be a ranking system that is done by someone actually watching the game as opposed to reading stat sheets after the game (at least the top ten is anyway).

Disagree, a lot of the rankings make no sense vs how the games were played.

Bama at #3 despite losing by 10 to Texas and struggling with USF, while Texas is undefeated but only #6?

Ohio State at #1 and Notre Dame at #11 after a game that came down to 6 inches and one team being short a player on the final play, but FSU and LSU right next to each other despite FSU being undefeated and beating LSU head to head in a game that really wasn't very close?

A&M rated ahead of undefeated Miami despite losing head to head by 2 scores?

 

[GetYourBuzzOn]

Thanks @slugboy - my $.02 here-

To echo the initial point, these rankings are not your traditional rankings, which should reflect a team's current season's resume. Instead, they are power rankings and try to predict who would win on a neutral site. The formula is simply (Power Rank Home Team) + (Home Field Adv Home Team) - (Power Rank Away Team) = Margin of Victory.

The power rankings all take into account priors, but weight them or phase them out differently. A big point of discussion this year among those who create the models is how quickly to phase out priors with teams that undergo drastic turnover such as Colorado or Texas State. Or, how quickly to phase out priors due to coaching staff turnover- like our beloved Jackets. (For the record I think all models are a bit low on GT, and the market money that we have taken every week so far backs that opinion up)

For my handicapping, I use a blend of three power rankings: SP+, PFF adjusted (I don't agree with their QB ratings so I'll do manual adjustments), and Massey Peabody. I then attempt to quantify injuries- both individual and cluster, weather, travel situations, any outlier positional matchups, coaching matchups, rivalries, time of the season, and any other factor that would potentially influence a team's number. In a perfect world I would create my own rankings, but I don't have the bandwidth to do that am not that good at R or Python, so I'll just use what is out there.

SP+ has consistently been the most predictive ranking system over the past five years and it factors prominently in my Regular Season Win total betting. SP+ is also baked heavily into every single Vegas line, so if you think you are going to win using solely SP+ you have another thing coming.

I like PFF rankings because of their grading system. I realize that this grading system isn't perfect, but to my knowledge there is no one else out there who is doing this and it has lead me to some actionable and profitable insights in the past. I do take issue with how they quantify QB's in their power rankings, so I'll make adjustments on the teams that I feel confident in my knowledge. For example. They have GT QB at -1.5 (our overall is -2.08 this week), assuming Pyron is the QB. I think Haynes is +.25, so my PFF adjusted number is -.33. My process here is more feel than science, so flame away at the subjectivity of the adjustments...

Massey Peabody rankings are the final piece of the puzzle. Not to be confused with Massey rankings, they can be found a paywall at Unabated.com. Developed by Cade Massey, a Wharton School of Business professor, and Rufus Peabody, who is one of the world's most successful modeler and sports bettor. You can read more about Massey Peabody rankings here.

 

[Techster]

 

[Heisman's Ghost]

Most of the stats models that get brought up (FEI, F+, SP+, Beta Rank, Sagarin, etc.) aren’t just ranking systems (like the CFP poll), but they’re also there to predict who will win a game. It’s tough to keep track of all of them if it’s not your full time job. It’s not my full time job. They kind of keep track of each other, though.

Here’s one modeler who keeps track of how all the models do week-by-week. You can see which one picked the best at
https://public.tableau.com/app/profile/andrew.percival/viz/CFBPicker/Standings.
He explains which models he tracks and what they are at


The people who do sports-reference.com (from where I post a lot of game stats) does a Simple Rating System (SRS) that picks games. It currently has the ACC 5th out of the P5, and we’re #54 https://www.sports-reference.com/cfb/years/2023-ratings.html.

There’s Jeff Sagarin, (who also doesn’t rate the ACC highly so far this year), and has us at 78th.

I was a liberal arts major, math gives me hives but I will take your word for it.

 

[slugboy]

It’s funny seeing two different stat takes in two different threads. That’s taking the raw number. Clemson is 2-2, and they’ve played FSU, Duke, FIU, and Charleston Southern.
This is the opponent-adjusted version of EPA (Defensive EPA is inverted, with a high EPA on the bottom, so scoring a lot and holding your opponent to a little is the upper-right-quadrant)


And here is Hale posting unadjusted success rate. Maybe FIU and Charleston Southern really shift everything?


I don’t know if they adjusted correctly for Duke and FSU (FSU isn’t that high on some of these charts). Clemson scored 66 against Charleston Southern and 48 against Florida Atlantic. They only scored 7 against Duke. They get moved down a lot when they’re adjusted for the opponent.

I think that they’re a talented team playing under their talent level. I’m not sure what’s going on with their offense—I think they brought in Riley but kept the rest of their offensive coaches and their old scheme. Unless he turns it around, this isn’t a good resume entry for Riley.

 

[slugboy]

Here’s the composite that I liked last year, for week 5:


I like the per-conference graph he puts together here. GT has fought their way into the middle (ending the Ole Miss game in the third quarter would have probably moved us up a lot)

We’re at #67 overall, and you can see and download the numbers here: https://github.com/natemanzo/composite_ratings/blob/main/composite_ratings_2023-09-26.csv.

When you were watching football in the 70’s and 80’s, seeing the AP and the Coach’s poll was a big thing. You’d check the paper on Tuesday or Wednesday to see if your team had made the top 20. Now, there’s a wealth of info.