Stats models and rankings

[TechPhi97]

I know I’ve posted a stat that we’re the #73 ranked in a stat here or #62 in another. I think that’s more precise than is helpful to me.

We had games like BC and Bowling Green where we couldn’t stop them from running and the defense lost us the game. We had games like UVA and Wake where the defense looked like it could win single handedly.

Ole Miss and Clemson —where we had no answers but fought hard—were a little more common for us than games like UNC and Miami.

We came out below average on points allowed, but I don’t think we were the 120th best defense in the country. We were probably one of the bottom 5 P5 teams though.

We had a good secondary and pass defense, which helped us beat UNC and Miami. Run defense was atrocious, which cost us wins against BC and Bowling Green.

There were matchups where we played well and matchups where we didn’t.

In the end, I’d look at the trends, including a respectable game against UGA and holding UCF to 17 to be a reasonable view of how our defense ended up. The team at the end of the season beats Bowling Green and BC—UCF is a really good running team, and we took that away from them. We patched the big hole at the end of the season.

I’d lean toward more of the weighted and normalized measures—below average for FBS and lower end of the P5–than 100th or 120th. We had a rough schedule, and played some of the best offenses in football. I’m glad we didn’t play Oregon or LSU, though.

I’d say 120th in total defense overstates how bad our defense was, and it was a lot of empty yards. Ratings like SP+ that put our defense at 64th are probably as optimistic as you can get. Splitting the difference puts the defense at the top of the weakest 25% of defenses or the bottom of the 25% above that.

Qualitatively, it’s an inconsistent defense that costs you some games that you should have won, and has some occasional good games, but is overall a net negative.

Hopefully, next season is better.

I like to think of efficiency like this - if your offense fumbles on the 20 yard line and you hold the opponent to a field goal, that only happens 30% of the time so it’s a positive outcome. Points against goes up, but positive due to starting position.

Another example: imagine two teams that each have up 200 points and play the same opponents. The opponents started at their own 20 on average for one team, and for the other they started on the 50 yard line. The team where opponents started on the 50 did better because the expected points against was higher.

It’s really about performance vs. expectation on a series by series basis.

 

[Root4GT]

I like to think of efficiency like this - if your offense fumbles on the 20 yard line and you hold the opponent to a field goal, that only happens 30% of the time so it’s a positive outcome. Points against goes up, but positive due to starting position.

Another example: imagine two teams that each have up 200 points and play the same opponents. The opponents started at their own 20 on average for one team, and for the other they started on the 50 yard line. The team where opponents started on the 50 did better because the expected points against was higher.

It’s really about performance vs. expectation on a series by series basis.

While the 2nd example looks better in terms of "analytics", both teams gave up the same amount of points. Points are what matters! Sometimes analytics are not as meaningful as actual results.

 

[Augusta_Jacket]

Sometimes analytics are not as meaningful as actual results.

Analytics are rarely as meaningful as actual results, but proper use of analytics helps you obtain better results.

For instance Team A and Team B both lose a game 31-28. Team A gave up 450 yards and 31 points but Team B gave up 215 yards and 31 points. That would lead you to believe team B maybe had issues on offense/special teams leading to either turnovers or incredibly short fields for the opposing offense. When you are trying to determine where to spend the limited amount of practice time you have to improve your results, analytics are extremely helpful.

 

[roadkill]

While the 2nd example looks better in terms of "analytics", both teams gave up the same amount of points. Points are what matters! Sometimes analytics are not as meaningful as actual results.

Points scored and points allowed matter. But sometimes it’s useful to grade various components of a team. Offense or defense for example, or even down to the performance of individual players. How should you do that? In these cases, advanced stats can give you much more insight than simply using points.

To expand on the previous example, let’s say team A is turnover-prone, but team B isn’t. Team A’s offense is good but frequently turns the ball over near their own goal line, but their outstanding defense typically holds the opposing team to only a field goal or turnover on downs. Team B has a good ball-control offense that eats clock and doesn’t turn the ball over much, but their defense often fails to stop their opponent. Both teams end the season with similar records and their defenses give up about the same number of total points. Which one has the better defense? Are they the same?

Edit: I posted before I saw @Augusta_Jacket's response which makes a similar point.

 

[slugboy]

While the 2nd example looks better in terms of "analytics", both teams gave up the same amount of points. Points are what matters! Sometimes analytics are not as meaningful as actual results.

Let’s play with an example. In the real world, the Falcons had below average offensive and defensive efficiency numbers this past season—they’re both #24 out of 32 teams in DVOA.

However, let’s pretend they had a #10 DVOA defense and a #24 offense. We’re being hypothetical here. Ridder throws a bunch of pick 6’s, and turns the ball over a bunch. The other teams get the ball on Atlanta’s side of the field all the time, often in or near the red zone. This shouldn’t be hard to imagine if you saw them play. Somehow, the Falcons are #28 in points given up, but #10 in defensive efficiency. What does that mean?

It means***

1. Keep your defense, maybe look in particular areas to improve
2. Dump your QB
3. Fix your offense and focus your draft and free agency efforts there
4. Be opportunistic in fixing other areas

In a case like that, “points are what matters” doesn’t tell you anything about how to fix the team. Points tell you that your defense is awful, instead of telling you that the offense sabotaged them.

Points don’t give you the information you need

(Edit: lots of concurrent opinions)

***(This is not the case with the real world Falcons —they need to improve across the board).

 

[TechPhi97]

Being #62 or #75 is close to average in FBS football. Anyone saying that is where we were is saying we were not all that bad a defense. So yes people are saying we were just below average on defense when it's clear we were very very bad on defense and nowhere close to being an average defense.

I also find it humorous that several people are plugging using SOS as a reason we were not so bad on defense when the 2 best teams we played were SEC schools yet many of the same fonts constantly say the SEC is way overrated and their SOS is BS because they were so overrated. Oh the irony!

Either SOS is not real useful or the SEC was actually a very strong conference. I contend SOS is only marginally useful and the SEC was strong at the top 4-5 teams. Meh at best below the top 4-5 teams.

One could argue that we are close to average in college football, because that includes teams like UL Monroe, Kent State or UConn. If you look at just the P5 + ND (which is 69 teams), our DFEI was 54th out of 69. That's bottom quartile. We're average when comparing us to The Sisters of the Poor (as my dad used to say), but we're bad compared to our peer teams in P5.

 

[TechPhi97]

While the 2nd example looks better in terms of "analytics", both teams gave up the same amount of points. Points are what matters! Sometimes analytics are not as meaningful as actual results.

Points are what matters for wins and losses, no argument there. Measuring the performance of the team ultimately falls to that.

But if you're trying to make decisions about coaching changes or diagnose what is really going wrong, then comparison against some baseline can help you do that. I'm a fan of the FEI stuff because it uses a baseline to make comparisons. For our defense last year, our only real redeeming quality was our ability to produce turnovers. We were 8th overall and 4th in the P5+1 group in turnovers per drive. This makes intuitive sense, considering almost all of our big wins relied on big turnovers.

 

[TampaBuzz]

Analytics are rarely as meaningful as actual results, but proper use of analytics helps you obtain better results.

For instance Team A and Team B both lose a game 31-28. Team A gave up 450 yards and 31 points but Team B gave up 215 yards and 31 points. That would lead you to believe team B maybe had issues on offense/special teams leading to either turnovers or incredibly short fields for the opposing offense. When you are trying to determine where to spend the limited amount of practice time you have to improve your results, analytics are extremely helpful.

In your example, I would not need analytics to reach that conclusion. Turnovers, block punts and long kick returns would be burned in my brain.

 

[slugboy]

In your example, I would not need analytics to reach that conclusion. Turnovers, block punts and long kick returns would be burned in my brain.

Look at the Falcons example. The perception this year is that the Falcons turned around their defense. They had interceptions and sacks, and plays that made the defense look good. They had the excuses build in with an offense that was a mistake machine.
But, they played a pitiful schedule, and at the end of the season their defense analytics are just as bad as the offense. When you think about it, that makes sense.
If you just think of one-off plays, you’ll miss the bigger picture.

For us, our offense went from being a train wreck to being kinda respectable. Some say amazing, but the analytics don’t—respectable looks amazing compared to the previous seasons.
Defense regressed badly. It was kinda respectable in the bowl game, but it was Jekyll and Hyde for the rest of the year. In F+ (a composite rating), overall the team was #58, but defense was #87–it held us back. It’s really bad for P5, though.

 

[Augusta_Jacket]

In your example, I would not need analytics to reach that conclusion. Turnovers, block punts and long kick returns would be burned in my brain.

Correct, but it's a somewhat obvious example. Take garbage time drives and scores, for instance. Factoring them out helps to drive analytical understanding of a teams true ability but they still show up in totals only methodologies. There are hundreds of possible ways to end up at similar game outcomes. Diving into what causes a team to stumble leads to better solutions for fixing the problem.

 

[Root4GT]

Points are what matters for wins and losses, no argument there. Measuring the performance of the team ultimately falls to that.

But if you're trying to make decisions about coaching changes or diagnose what is really going wrong, then comparison against some baseline can help you do that. I'm a fan of the FEI stuff because it uses a baseline to make comparisons. For our defense last year, our only real redeeming quality was our ability to produce turnovers. We were 8th overall and 4th in the P5+1 group in turnovers per drive. This makes intuitive sense, considering almost all of our big wins relied on big turnovers.

True. My issue is with the people who fall on their swords that such analytics are the proper way to rank teams. Many simply take analytics too far as team vs team comparison.

For internal use and scouting purposes they are excellent tools.

 

[slugboy]

True. My issue is with the people who fall on their swords that such analytics are the proper way to rank teams. Many simply take analytics too far as team vs team comparison.

For internal use and scouting purposes they are excellent tools.

If you want to choose who is more worthy between Tulane and Liberty, who didn’t play, some analytics are a lot better than raw “scoring defense”.

 

[Root4GT]

If you want to choose who is more worthy between Tulane and Liberty, who didn’t play, some analytics are a lot better than raw “scoring defense”.

Of course actually watching the teams play and studying their film is better yet.

Analytics comparison between Liberty and Washington State is fairly useless.

 

[slugboy]

Of course actually watching the teams play and studying their film is better yet.

Analytics comparison between Liberty and Washington State is fairly useless.

It seems to put together good NCAA basketball tournaments

 

[Root4GT]

It seems to put together good NCAA basketball tournaments

A 32 game regular season plus Conference tournaments give a much greater sample size.

Still there are tons of critics of the old RPI and current NET. Per analytics the ACC is about the 6 best basketball conference.

Either the ACC really sucks now or there are serious issues with NET analytics.

 

[stinger78]

It seems to put together good NCAA basketball tournaments

You pick 68 teams from among those with the best W/L records and your chances are pretty good to get some great games, especially as the field narrows -i.e. self-selects the best/hottest of the field.

If I read your Tableau quote from last September correctly, the ATS predictors all cluster around 50%. That’s basically a coin toss.

IMPO, using point spread or MOV is a bad predictor game by game due to many confounding factors. Massey, which I believe uses MOV, doesn’t distinguish itself in that study and neither does Sagarin ELO Chess.

 

[CuseJacket]

 

[Techster]