CFP Discussion

[orientalnc]

Let me play devil’s advocate.

Wouldn’t the computer model correct for that by the time you are 3 games into the season? Sure, they all START at .500 but you have more data later on what those earlier wins and losses meant. Right? What am I missing?

My point is/was that every model has bias built in. How valuable is the first win over a 0.500 team? Colley assumes that every team has the same probability of winning their first game. But, we know that is not correct, even before any games are played.

But, my argument is not with Colley (or any computer model). My point is that every ranking system has its own bias. Maybe it is unintentional, but it's there. In Colley's rating system the strength of the opponent is not factored into the computer analysis. It is my contention that every win or loss should be viewed through the lens of who each team played. Not just the fact of the win or loss. Either way, that is a bias. It doesn't matter which bias you prefer.

 

[AugustaSwarm]

Seriously, ask yourself who could beat uga. There are two answers.

1. Nobody on the their current schedule.

2. FSU, UNC, Ohio State, Oklahoma, Texas, Oregon, Washington.

And, on the right day, Alabama, Louisville, Michigan, and (before Tech broke them) Miami.

Yep, that’s what I think of the current uga team. Problem is, they have a glide path to another national championship. Not having been tested, they only have to play over their heads and out of their minds once or twice, given how they would be seeded based on their record and reputation.

Wasn't the playoff supposed to get rid of this crap? I suppose it might once the playoff is expanded. If you actually judge a team based on resume, UGA is no where near the top. The might actually be good, but we don't know because they have a super soft schedule. Kudos to them for taking care of business thus far.

 

[Northeast Stinger]

A narrow point, but unimpeachable.

 

[FredJacket]

Here is where I differ. Even the computer models begin with a bias. Even if they say every team is equal before the season starts. And, the algorithms have bias. You may agree with the bias and some years it may appear to be accurate. But the bias is there.

My point is/was that every model has bias built in. How valuable is the first win over a 0.500 team? Colley assumes that every team has the same probability of winning their first game. But, we know that is not correct, even before any games are played.

But, my argument is not with Colley (or any computer model). My point is that every ranking system has its own bias. Maybe it is unintentional, but it's there. In Colley's rating system the strength of the opponent is not factored into the computer analysis. It is my contention that every win or loss should be viewed through the lens of who each team played. Not just the fact of the win or loss. Either way, that is a bias. It doesn't matter which bias you prefer.

Not sure we differ. Of course, any 'system' voters or computers have some bias. The feature(s) with computers is the calculations are repeatable using same data across those calculations and any bias (over time) that skews results can be identified and mitigated with tweaks.

The flaw with the AP (specifically) is the voters are asked to 'rank' teams. I believe they make no effort to share the same criteria among fellow voters on how to do that... I believe some voters don't pay attention enough to make justifiable decisions. Again... that is all fine with me. My issue is how much the AP poll is front and center in the 'conversation'... it really isn't that good at what it claims to be... compared to other systems.

The fact that Colorado was ranked in the top 15 by double-digit voters early in the season is evidence they are MOSTLY looking at Win-Loss (regardless of how it happens and not much care as to who the opponents were). The bias you describe is pretty egregious in the AP polling (I know you were not claiming otherwise). My point is there are plenty of pretty robust computer ranking systems that are supremely better at figuring out where teams align top-to-bottom. I would like THOSE to be what is referenced when discussing/comparing teams/resumes.

 

[orientalnc]

The fact that Colorado was ranked in the top 15 by double-digit voters early in the season is evidence they are MOSTLY looking at Win-Loss (regardless of how it happens and not much care as to who the opponents were). The bias you describe is pretty egregious in the AP polling (I know you were not claiming otherwise). My point is there are plenty of pretty robust computer ranking systems that are supremely better at figuring out where teams align top-to-bottom. I would like THOSE to be what is referenced when discussing/comparing teams/resumes.

Two thoughts about this discussion:

I wonder where Colley had Colorado "ranked" (I know his model is a "rating" system and does not directly rank teams) after their 3-0 start? The AP voters thought TCU would be a Top 25 team this year given what they did last year, so CU beating them looked damn good. The bias there was the erroneous TCU ranking. I also think there was (and still is) a lot of interest in what CU was doing vis a vis the portal and how that might change college football forever. But that was a unique situation early this season.

When I was in high school (and considered myself the smartest math guy in the world) I wrote a ranking model for Georgia high school football teams. Tweaks can help when things don't seem right, but tweaking itself is building in a new bias to "correct" for the previous bias. Some teams get better during the season while others seem to get worse. Injuries happen and extreme weather affects some teams more than others. If UCF has a road game in Miami and another in Denver, those trips are not the same. A lot of college football games have special situations. A great example is the Louisville loss to Pitt just after the great win over ND. Did UL suddenly get worse? Or, were they never that good? Or, is our view of ND badly skewed? Now, what about USC? If we try to fix our model to account for these types of anomalies we could ruin it.

Back to what started this for me. Ranking JMU in the top 10 and having uga at #14 is just wrong. I can imagine uga being a 20+ point favorite against them.

 

[FredJacket]

Two thoughts about this discussion:

I wonder where Colley had Colorado "ranked" (I know his model is a "rating" system and does not directly rank teams) after their 3-0 start? The AP voters thought TCU would be a Top 25 team this year given what they did last year, so CU beating them looked damn good. The bias there was the erroneous TCU ranking. I also think there was (and still is) a lot of interest in what CU was doing vis a vis the portal and how that might change college football forever. But that was a unique situation early this season.

When I was in high school (and considered myself the smartest math guy in the world) I wrote a ranking model for Georgia high school football teams. Tweaks can help when things don't seem right, but tweaking itself is building in a new bias to "correct" for the previous bias. Some teams get better during the season while others seem to get worse. Injuries happen and extreme weather affects some teams more than others. If UCF has a road game in Miami and another in Denver, those trips are not the same. A lot of college football games have special situations. A great example is the Louisville loss to Pitt just after the great win over ND. Did UL suddenly get worse? Or, were they never that good? Or, is our view of ND badly skewed? Now, what about USC? If we try to fix our model to account for these types of anomalies we could ruin it.

Back to what started this for me. Ranking JMU in the top 10 and having uga at #14 is just wrong. I can imagine uga being a 20+ point favorite against them.

I'm not arguing with anything you're saying. Reasonable thoughts. I'm not defending Colley. I'm defending the idea a computer model with a decent amount of vigor outperforms voters. There are obvious problems starting a model where all teams are equal. There is also an obvious problem ranking TCU as high as they were in the preseason based on 2022 results. (something measurable probably caused that to be wrong/bias). Your point regarding the portal (imo) exacerbates problems with polls. Pollsters are not going to account [accurately] for the increased turnover the portal has/will generate. I think you can watch teams ranked early (pre-season) and watch the biases be very slow to react when results don't match the ranking. Humans naturally don't 'believe' they are wrong... and pollsters will usually hesitate to move a team down (or up) after an unexpected (but proper) result. ...and this propagates throughout the year.

Let's look at FPI (ESPN). This top 10 is good and I think puts UGA in a reasonable spot. They started year at #3... stayed there for 4 iterations (incl preseason). Have now dropped to #8 over the last 4 weeks. [I do not know how FPI is calculated. ]
OSU
Okla
PSU
Mich
Texas
Oregon
Alabama
UGA
FSU
Wash

Of note: #54 JMU; #55 Ga Tech

 

[CEB]

Not sure we differ. Of course, any 'system' voters or computers have some bias. The feature(s) with computers is the calculations are repeatable using same data across those calculations and any bias (over time) that skews results can be identified and mitigated with tweaks.

The flaw with the AP (specifically) is the voters are asked to 'rank' teams. I believe they make no effort to share the same criteria among fellow voters on how to do that... I believe some voters don't pay attention enough to make justifiable decisions. Again... that is all fine with me. My issue is how much the AP poll is front and center in the 'conversation'... it really isn't that good at what it claims to be... compared to other systems.

The fact that Colorado was ranked in the top 15 by double-digit voters early in the season is evidence they are MOSTLY looking at Win-Loss (regardless of how it happens and not much care as to who the opponents were). The bias you describe is pretty egregious in the AP polling (I know you were not claiming otherwise). My point is there are plenty of pretty robust computer ranking systems that are supremely better at figuring out where teams align top-to-bottom. I would like THOSE to be what is referenced when discussing/comparing teams/resumes.

The biggest flaw in the AP is that the entire notion of the press being the most informed resource is very likely untrue at this point. It probably made sense years ago to let the people who “did this for a living” opine on the relative strength of teams that most of the country didn’t ever get to watch and would never see take the field against one another. In the last decade or two, we have reached a point where any Joe Schmo can sit squarely on his rump in Anytown, USA and see more college football than any sportswriter of 30-40 years ago could see in the course of doing their full time job. Yet, the polling model is unchanged.

Same could probably be said for the coaches poll honestly. Granted, I would put coaches well ahead of the press in evaluation of teams but any active coaches involved aren’t terribly concerned with anyone other than the 12 on their schedule.

Not to say that Joe Schmo is or is not more qualified at this point, but the circumstances that made the AP the best resource changed a while ago. I think CFP has acknowledged that with their independent selection process, which is great, so the AP and coaches polls are just debate fodder at this point, fortunately.

 

[Oldgoldandwhite]

Wasn't the playoff supposed to get rid of this crap? I suppose it might once the playoff is expanded. If you actually judge a team based on resume, UGA is no where near the top. The might actually be good, but we don't know because they have a super soft schedule. Kudos to them for taking care of business thus far.

The committee is biased. All they hear the entire season is SEC and BIG.
And 12 will just allow more prejudice in.

 

[CEB]

Two thoughts about this discussion:

I wonder where Colley had Colorado "ranked" (I know his model is a "rating" system and does not directly rank teams) after their 3-0 start? The AP voters thought TCU would be a Top 25 team this year given what they did last year, so CU beating them looked damn good. The bias there was the erroneous TCU ranking. I also think there was (and still is) a lot of interest in what CU was doing vis a vis the portal and how that might change college football forever. But that was a unique situation early this season.

When I was in high school (and considered myself the smartest math guy in the world) I wrote a ranking model for Georgia high school football teams. Tweaks can help when things don't seem right, but tweaking itself is building in a new bias to "correct" for the previous bias. Some teams get better during the season while others seem to get worse. Injuries happen and extreme weather affects some teams more than others. If UCF has a road game in Miami and another in Denver, those trips are not the same. A lot of college football games have special situations. A great example is the Louisville loss to Pitt just after the great win over ND. Did UL suddenly get worse? Or, were they never that good? Or, is our view of ND badly skewed? Now, what about USC? If we try to fix our model to account for these types of anomalies we could ruin it.

Back to what started this for me. Ranking JMU in the top 10 and having uga at #14 is just wrong. I can imagine uga being a 20+ point favorite against them.

Two cherry picks from this;

1. Team improvement and regression are never linear. Both teams are subject to “ups and downs” each week, so it can sometimes result in wild outcomes. I think you saw a Pitt team that was hyper motivated to salvage a season vs a Lville team that just came off of a huge emotional win and “took a breath” this weekend. For that matter, ND may have suffered a similar fate against Lville last week… it’s hard (massive understatement) to be at 100% mentally AND physically each week. The old adage of “never being as good as your best and never being as bad as your worst” is absolutely true.

2. I know a lot of people point at point spreads, but those are just as biased…. Granted, the “bias” comes from a much larger sample size, but the spread moves based on who bets what and everyone’s reasons for betting are different. I think sports fans put WAAAAY too much emphasis on point spreads as a means of relative strength of teams.

So the jury is still out on the best metrics for evaluation. As long as humans are involved in any aspect of it, there will be bias…

 

[roadkill]

OK. A victory over Vanderbilt on opening day is not better than a loss to uga that day. If the model assumes Vandy and uga are equal at the beginning of the season, it has a bias.

Let me play devil’s advocate.

Wouldn’t the computer model correct for that by the time you are 3 games into the season? Sure, they all START at .500 but you have more data later on what those earlier wins and losses meant. Right? What am I missing?

For GT, a victory over Vandy on opening day would be better than a loss to UGA on opening day. If a human assumes Vandy and UGA are not equal at the beginning of the season, they have a bias toward using historical data. That doesn't mean they would be wrong. Colley's is strictly based on results on the field, with no preset assumptions. It is not possible to rank UGA over Vandy before any games are played unless you use assumptions (bias).

Let's try not to confuse bias with accuracy. Colleys by definition has no accuracy before games are played because it has no data.

For the sake of this discussion, let's assume Vandy is the worst team in football and UGA is the best. By the third game, Vandy is 0-3 and UGA is 3-0. At that point, Colley's ranking would have UGA somewhere in the top 40 (SoS would determine their exact position) and Vandy would be somewhere in the bottom 40.

 

[roadkill]

My point is/was that every model has bias built in. How valuable is the first win over a 0.500 team? Colley assumes that every team has the same probability of winning their first game. But, we know that is not correct, even before any games are played.

But, my argument is not with Colley (or any computer model). My point is that every ranking system has its own bias. Maybe it is unintentional, but it's there. In Colley's rating system the strength of the opponent is not factored into the computer analysis. It is my contention that every win or loss should be viewed through the lens of who each team played. Not just the fact of the win or loss. Either way, that is a bias. It doesn't matter which bias you prefer.

Incorrect. Colleys makes no assumptions. It is results-based only.

 

[roadkill]

Two thoughts about this discussion:

I wonder where Colley had Colorado "ranked" (I know his model is a "rating" system and does not directly rank teams) after their 3-0 start? The AP voters thought TCU would be a Top 25 team this year given what they did last year, so CU beating them looked damn good. The bias there was the erroneous TCU ranking. I also think there was (and still is) a lot of interest in what CU was doing vis a vis the portal and how that might change college football forever. But that was a unique situation early this season.

When I was in high school (and considered myself the smartest math guy in the world) I wrote a ranking model for Georgia high school football teams. Tweaks can help when things don't seem right, but tweaking itself is building in a new bias to "correct" for the previous bias. Some teams get better during the season while others seem to get worse. Injuries happen and extreme weather affects some teams more than others. If UCF has a road game in Miami and another in Denver, those trips are not the same. A lot of college football games have special situations. A great example is the Louisville loss to Pitt just after the great win over ND. Did UL suddenly get worse? Or, were they never that good? Or, is our view of ND badly skewed? Now, what about USC? If we try to fix our model to account for these types of anomalies we could ruin it.

Back to what started this for me. Ranking JMU in the top 10 and having uga at #14 is just wrong. I can imagine uga being a 20+ point favorite against them.

Colley had Colorado ranked 10th after three games. There were 39 total 3-0 teams at that point. Colley's model computed Colorado's SoS based on W-L records of the teams they played.

Colley's makes no claims of accuracy 3 games in, it's just data in, data out. It is not intended to be predictive. It's about a team's resume based on results on the field. JMU and UGA are both undefeated now, but JMU has a slightly better resume at this point of the season. This doesn't mean JMU would beat UGA head-to-head, just that JMU has beaten better teams so far.

Where Colley's can be most useful is to compare its results with the CFP poll towards the end of the season, when you have situations where the top teams may not have played each other or even common opponents. It can assess which teams most deserve to be in the top rankings based on their results on the field.

 

[Sean311]

 

[g0lftime]

Colley had Colorado ranked 10th after three games. There were 39 total 3-0 teams at that point. Colley's model computed Colorado's SoS based on W-L records of the teams they played.

Colley's makes no claims of accuracy 3 games in, it's just data in, data out. It is not intended to be predictive. It's about a team's resume based on results on the field. JMU and UGA are both undefeated now, but JMU has a slightly better resume at this point of the season. This doesn't mean JMU would beat UGA head-to-head, just that JMU has beaten better teams so far.

Where Colley's can be most useful is to compare its results with the CFP poll towards the end of the season, when you have situations where the top teams may not have played each other or even common opponents. It can assess which teams most deserve to be in the top rankings based on their results on the field.

That system has no way to account for injuries to key players until a game or two are played without them. There are always factors that cannot be included accurately in any algorithm. I would put more into the Vegas odds since so much money is involved and they aren't perfect either. They just have to be right more times than they are wrong.

 

[ibeattetris]

Ranking JMU in the top 10 and having uga at #14 is just wrong. I can imagine uga being a 20+ point favorite against them.

Colley isn't meant to be predictive, but to explore who has won games against teams with better records.
Here is another poll called CPI that aims to do the same thing as Colley (but IMO shows its work better): http://www.cpiratings.com/top25.html
JMU 9
UGA 13

UGA and JMU are both undefeated

UGA direct opponent wins% 50%
JMU direct opponent wins% 56.3%

UGA opponent opponent win% 61.9%
JMU opponent opponent win% 63.1%

Both UGA's direct SOS and opponent of opponent SOS is currently lower than JMU's, therefore, a poll that insists all teams are equal to start the year would not rank UGA above JMU.

You are correct that the poll has a bias. Claiming "all teams start the year equal" is a bias. IMO, polls like this are good because it helps highlight how pre-conceived notions of teams warp how we see them. UGA is, imo, a better team than JMU. They have not, however, had a better season than JMU up to this point.

 

[GTJackets]

Colley isn't meant to be predictive, but to explore who has won games against teams with better records.
Here is another poll called CPI that aims to do the same thing as Colley (but IMO shows its work better): http://www.cpiratings.com/top25.html
JMU 9
UGA 13

UGA and JMU are both undefeated

UGA direct opponent wins% 50%
JMU direct opponent wins% 56.3%

UGA opponent opponent win% 61.9%
JMU opponent opponent win% 63.1%

Both UGA's direct SOS and opponent of opponent SOS is currently lower than JMU's, therefore, a poll that insists all teams are equal to start the year would not rank UGA above JMU.

You are correct that the poll has a bias. Claiming "all teams start the year equal" is a bias. IMO, polls like this are good because it helps highlight how pre-conceived notions of teams warp how we see them. UGA is, imo, a better team than JMU. They have not, however, had a better season than JMU up to this point.

Interesting thought that I hadn't really ever thought about. Not sure how I feel about it to be honest.

I love the idea of a true ranking system that rewards on-the-field performance which is what this seems to do. Probably wouldn't have guessed JMU's SOS to be better than UGa at this point of the season. But based on that you could argue that JMU deserves to be ranked higher right now. You'd have to ignore the b-2-b NCs which I choose to do anyway. Do I think JMU is the better team? Probably not. Should they be ranked higher based on results so far? Sure seems like it.

And I guess I should stick with that and be done. Because it's eventually going to work itself out anyway. Either UGa will beat the better teams coming up on their schedule and begin to climb these rankings or they won't. So at the end of the season, this method tells you who has performed better rather than who we think is probably better throughout the course of the season. And at the end of the season that should be what's important. It takes subjectivity out of it. So I can ignore all the guessing and supposing that happens along the way.

I like it.

 

[SOWEGA Jacket]

Wasn't the playoff supposed to get rid of this crap? I suppose it might once the playoff is expanded. If you actually judge a team based on resume, UGA is no where near the top. The might actually be good, but we don't know because they have a super soft schedule. Kudos to them for taking care of business thus far.

Yes, the expanded playoffs will fix a lot of this crap from the past 100 years of corruption. Once we get to the 20’s and above in playoff teams then the champ will be legit. Our grandkids will look back on the history of college football and totally dismiss everything before a real playoff was started. Just like all these BS “claimed Natties” from previous decades are dismissed by my generation.

Even if it will still be a rigged system with only 12 teams, because we all know the committee will always side with the schools who give them the proper bennies, at least we won’t have to worry about top teams being left out because they had 1 loss. And then you’ll see how many upsets happen every year which will show how flawed the “polling” system has always been.

 

[ibeattetris]

Interesting thought that I hadn't really ever thought about. Not sure how I feel about it to be honest.

I love the idea of a true ranking system that rewards on-the-field performance which is what this seems to do. Probably wouldn't have guessed JMU's SOS to be better than UGa at this point of the season. But based on that you could argue that JMU deserves to be ranked higher right now. You'd have to ignore the b-2-b NCs which I choose to do anyway. Do I think JMU is the better team? Probably not. Should they be ranked higher based on results so far? Sure seems like it.

And I guess I should stick with that and be done. Because it's eventually going to work itself out anyway. Either UGa will beat the better teams coming up on their schedule and begin to climb these rankings or they won't. So at the end of the season, this method tells you who has performed better rather than who we think is probably better throughout the course of the season. And at the end of the season that should be what's important. It takes subjectivity out of it. So I can ignore all the guessing and supposing that happens along the way.

I like it.

I think it is another good tool to have in the tool shed.

I think there are other systems that can work as well. The ELO system is another where everything is reset at the start of a season and is based on how chess does ratings. This was how Sagarin did it's computer ranking back in the BCS era (there was still a predictor model based on margin of victory, but wasn't used in the polls).
Here is a post that examines the different polls that used to be used in the BCS: https://statsinthewild.com/my-other-blogs/bcs-review/

I still like predictive based polls to show how "dominant" a team has been playing (or how close its losses have been). I think there is a place for both and help derive a more holistic examination of the season.

 

[roadkill]

Colley isn't meant to be predictive, but to explore who has won games against teams with better records.
Here is another poll called CPI that aims to do the same thing as Colley (but IMO shows its work better): http://www.cpiratings.com/top25.html
JMU 9
UGA 13

UGA and JMU are both undefeated

UGA direct opponent wins% 50%
JMU direct opponent wins% 56.3%

UGA opponent opponent win% 61.9%
JMU opponent opponent win% 63.1%

Both UGA's direct SOS and opponent of opponent SOS is currently lower than JMU's, therefore, a poll that insists all teams are equal to start the year would not rank UGA above JMU.

You are correct that the poll has a bias. Claiming "all teams start the year equal" is a bias. IMO, polls like this are good because it helps highlight how pre-conceived notions of teams warp how we see them. UGA is, imo, a better team than JMU. They have not, however, had a better season than JMU up to this point.

First, thanks for sharing the CPI poll. It appears to take the same basic approach as Colley but limits the computation to two orders (opponents opponents), whereas Colley uses a matrix method to include all opponents (I think - but his math is over my head). Not surprising that the results are similar at this stage of the season.

Your last point is significant. These polls don't attempt to rank the best team - they rank who is having the better season. One can be computed, the other is more subjective. I think that point may have been missed by some posters.

 

[ibeattetris]

One can be computed, the other is more subjective.

I think FEI and SP+ do a good job of taking the subjectivity out of the predictive based polls. Taking things like net points per play/net points per drive into account can help "break the tie" in cases where similar schedules and similar records combine. I think it is safe to say that a 12-0 team who's won every game by 20 is better than a 12-0 team who's on every game by 3. I think it was the correct call at the time of the BCS era to remove margin of victory based stats from the computer rankings, but I don't think it'd be fair to say they don't have their uses.