Blog Entry

This is how the NCAA should rank its teams

Posted on: February 23, 2012 12:29 pm
Edited on: February 23, 2012 7:37 pm
A snapshot of the mock meetings last week in Indianapolis. (NCAA)

By Matt Norlander

Let's see possibility. Let's see what the NCAA could ultimately be using, should it choose to cast a wider net in its database. Let's see fairness and true objectivity and less room for error in picking and seeding 68 teams into this behemoth of a bracket that takes over millions of American' lives in March.

I wrote last week how every metric officially referenced on the NCAA's Nitty Gritty sheets, in team sheets and on reports only relates to the RPI. It's a problem. The Selection Committee does a lot of things right. The few things it does wrong, it stands to reason, alter the master seed list or create inconsistencies in final selection.

The NCAA and its Selection Committee don't shun the use of other metrics. They just don't endorse them, either. On the Nitty Gritty master sheet, teams are arranged 1 to 344 in accordance to the RPI. Nine of the 16 columns on the Nitty Gritty are RPI-based or influenced. Why not organize the Nitty Gritty based on a collection of metric systems?

The NCAA has its reasons for this, and ultimately, one day, those reasons will give way to logic and a better collective understanding of how -- although we'll never perfect a rankings system -- we can use microscopes instead of magnifying glasses to examine teams' tendencies, weaknesses, strengths and true scope of accomplishment.

Below, I've got a chart of what such a Nitty Gritty could look like, what the NCAA could use as a base to sort its squads and begin the debate. This is something I'd love to take credit for, but the fact is I didn't have the time to get it done, and so you should be as grateful as I am for emailer Dr. Frederick Russ, who did the long division on this. Russ is an NCAA faculty athletics representative, a professor of marketing and former dean at the University of Cincinnati's Carl H. Lindner College of Business. He compiled what's quickly being acknowledged as the five most mainstream/reliable/respectable college basketball metrics and went with the median of the ratings (the chart says average, but it is the median; there is a difference). The chart below shows the positive or negative difference with the myopic RPI.

Plenty of teams don't vary in median rank of the five and the RPI. With others, it's chasm-like. And that means something significant when you get into the tedious but tremendous differentials in seeding, which can alter where teams go and of course who they play.

All teams listed were ranked 80th or better by, LRMC, BPI, RPI, Massey or Sagarin. These numbers, of course, are due to change in the coming weeks. All results are as of Wednesday, so even today there'd be minor shifts in the master list if you compiled one for yourself by dinnertime.

Russ also mentioned the obvious: the downside to every rankings system, with exception to the BPI, currently in somewhat of a test-drive phase, is you can't implement the impact of regular players missing games. Take Cincinnati’s first two losses (to Presbyterian and Marshall), which came when Jaquon Parker was still nursing a preseason injury.

But at least we can claim this is closer and more objective than the RPI's manipulable formula. Here is the master list, 1 through 90, of how the NCAA should be sorting teams. (Russ actually had 98 teams organized, but Google Docs was being less than agreeable in converting the chart beyond 90 teams. Apologies on that, but you get the point all the same.)

The biggest difference near the top is Wisconsin, perhaps overrated by all the other metrics, but not RPI? Let's debate! The distance on Missouri is disconcerting, though, too. Texas, Belmont, Arizona and Southern Miss all have big disparity as well. The largest gaps are UCLA (62 points lower in the RPI) and Colorado State (65 points higher in the RPI).

If anything else, this chart proves there are far too frequent communication breakdowns with teams across the board, enough so that the RPI goes beyond outlier status and continues to prove what many have known for years: If the RPI was introduced in 2012, it's hard to reason that it would be adopted as conventional by the NCAA or in mainstream discussion.


Since: Apr 12, 2007
Posted on: February 24, 2012 10:37 am

This is how the NCAA should rank its teams

Great article.  Clear.  Concise.  And intelligent.  Interesting how, after all the facts are laid out, their is a clear #1 and OSU is #2.  Regardless of your affiliation, can't argue with facts...or metrics.  (And, btw, what goob 'thinks' everyone should stop using the term 'metrics', in the first place.  And second, genius, the term 'metric' is anything BUT a 'buzzword'.  It has been around and been in use for, ohh, a few decades know.  What are you a DU, fan?)

Personally, i think the ncaa should open up and actually create events like the sportswriters selection committee to the people that actually really matter:  THE FANS!  We need a FANS SELECTION COMMITTEE!!!

Since: Feb 24, 2012
Posted on: February 24, 2012 9:06 am

This is how the NCAA should rank its teams

None of you (even you math majors) will ever come up with a PERFECT system.  Simple reason: we are evaluating PEOPLE and the infinate variables contained therein.  From a mathmatical perspective, would someone please explain to the math challenged, how Georgetown can be ranked #11 without a single value in any system being lower than 12?  I am not picking on them exclusively, its just that that fact jumped out at me for some reason.
Having said all this and acknowledging all its flaws, can't we just praise the system that has a PLAYOFF to determine a champion (flawed or not) rather than the BCS that ruins college football?

Since: Feb 24, 2012
Posted on: February 24, 2012 3:41 am
This comment has been removed.

Post Deleted by Administrator

Since: Jan 19, 2008
Posted on: February 23, 2012 10:39 pm

This is how the NCAA should rank its teams


School me... but please, let's all agree to stop using the buzzword "metrics" for just a few days.

Oh, I'm not pretending to be an expert. I work at a job that requires me to use simple statistics all the time. But I let the statisticians do the heavy lifting. That's what they're there for. ;) Still, if you had 344 values, and they fell in roughly a normal (bell-shaped) distribution, you could take either the median or the mean. They'd be the same value, or nearly so.

Instead of "metric" I guess we could say "value used to measure something." That might get kind of old, though. If you have a different synonym, we can use it. I'm drawing a blank.

I guess Norlander's point is that if we are going to use a computer ranking for any stage of the selection or seeding processes, then we might as well pick one that evens out some of the weirdness. Combining the five best rating systems and keeping the middle value accomplishes that. Same principle as Olympic figure skating, where the scoring system throws out the Russian judge's score most of the time. ;) 

Not that I take this discussion too seriously. The NCAA men's basketball tournament is the best event with the best format in all of team sports in my opinion. Even the seriously flawed RPI hasn't messed it up. It's just fun to argue about it while we're waiting for the conference tournaments to start.

Since: Jan 15, 2010
Posted on: February 23, 2012 10:38 pm

This is how the NCAA should rank its teams

You could make the same argument with any one of these ratings systems.  When compared by itself against an average of the others, the one will always show discrepancies.  There may be a case (or more) against the RPI, but this doesn't help make that case.  Maybe they shoudl just do as the BCS, and use an average of them all.

Since: Nov 5, 2007
Posted on: February 23, 2012 10:35 pm

This is how the NCAA should rank its teams

The word "average" is used incorrectly so much of the time, it shouldn't be trusted when you read it.  The formula you are using (sum/count) is the "arithmetic mean value".  At the article points out, the column headed "Avg" actually lists the "median value" of the five numbers to its left for each team.

Since: Nov 5, 2007
Posted on: February 23, 2012 10:09 pm

This is how the NCAA should rank its teams

The bookmaker's job is to keep the betting pool balanced.  The only reason the 'line' is related to the strength of the teams is because the betting public is not completely (just mostly) stupid.  The bettor looks at the teams, the bookie looks at the bettors.

Since: Apr 26, 2010
Posted on: February 23, 2012 10:01 pm

This is how the NCAA should rank its teams

Arizona played a number of strong teams before the PAC-12 season, like Florida, San Diego State, Gonzaga and Mississippi State. True, they lost mostly close games to all these teams (at Florida in OT), but they did play the tough schedule. They also played a few cupcakes, like Bryant, which has won 2 games all season, so that is a RPI killer. And while they did lose 2 games to Washington, they didn't lose to Oregon by 30.

Since: Nov 5, 2007
Posted on: February 23, 2012 9:54 pm

This is how the NCAA should rank its teams

Finding the median of set of numbers does not exclude any (outlier or not) of the numbers.  The median is the number which is less than half the numers and greater than the other half – without regard to how much less or how much greater.

The median value, unlike the mean (average) value, will not be change if one number in the top half is replaced by very high number  (or one in the lower half by a very low number).  This is the reason median is immune from outlier effects, not because they are ignored, but because they are treated simply as 'high' or 'low' no matter if they are very high or nearer to the median.

Since: Mar 23, 2007
Posted on: February 23, 2012 9:12 pm

This is how the NCAA should rank its teams

Yep, looks like I skimmed the words too briefly and dove straight into the digits (Stats 101 review appreciated). Your point about the median being more meaningful than the average for a set of estimates with a single outlying value makes sense in this situation. I wonder, however, if the same would be true for a significantly large set of estimates, say one with 344 cases. Also, how do you determine which estimate value constitutes an outlier? Is an outlier qualified as such by the number of standard deviations it is from the mean of the non-outlying values? Again, your logic applies to the situation at hand, but it seems reasonable to assume that it would not apply in every situation. School me... but please, let's all agree to stop using the buzzword "metrics" for just a few days.

The views expressed in this blog are solely those of the author and do not reflect the views of CBS Sports or