Reconciling tempo-free preseason ranking methodologies

Ken Pomeroy and Dan Hanner both released preseason rankings for all (or almost all in the case of Hanner) 347 Division-I men's college basketball teams in the last couple of days.

Both are based on tempo-free efficiency data to varying degrees, and both provide endless (and often meaningless) fodder for bloggers like this one and this one and this one. But while Pomeroy's yearly rankings releases have become tradition, Hanner is the new guy on the scene. His rankings appear in the latest College Basketball Prospectus book, and employ a system that is quite a departure from Pomeroy's methodology.

Hanner better explains the methodology behind his rankings in this post, but what I'm interested in is how different (and maybe better) his system really is. Of course, we can't properly evaluate all of that until the end of the season, but we can still take a look at how incorporating the impact of transfers, historical data, freshman usefulness, coaching effects, strength of schedule, the effect of D-I transfers taking a year off and other makers can alter a team's ranking.

I put both sets of rankings into a big ol' spreadsheet, and did some analyzing. It's nice working with data, because it just doesn't lie. First the greatest differentials in the datasets. That is, the teams that the Pomeroy and Hanner rankings disagreed most on.

  Team Hanner Rank KenPom Rank Differential
1 Utah 162 309 147
2 USC 45 188 143
3 Vanderbilt 181 68 113
4 Charleston Southern 252 147 105
5 Southern Illinois 138 236 98
6 UC Davis 222 318 96
7 Hawaii 150 243 93
8 Tulsa 274 181 93
9 Pennsylvania 285 196 89
10 Arizona St. 130 218 88
11 Charlotte 229 144 85
12 Harvard 155 71 84
13 Central Connecticut 291 208 83
14 Louisiana St. 72 153 81
15 Wichita St 127 46 81
16 Wyoming 57 138 81
17 Louisiana Lafayette 192 272 80
18 Auburn 100 179 79
19 Holy Cross 284 206 78
20 San Francisco 275 197 78
21 Lamar 303 226 77
22 Denver 158 84 74
23 NC Wilmington 163 237 74
24 Army 220 293 73
25 South Carolina 85 158 73

Hanner told me via tweet that transfer were the likely culprit behing his system liking Utah and USC so much.

He had less for why his system seriously undervalued Vanderbilt compared to Pomeroy's rankings.

I found that Vanderbilt is a pretty good anecdotal illustration of the differences in the ranking systems.

The fact that the Commodores return no starters and also lose 6-11 reserve Festus Ezeli probably has something to do with their varying treatment. Under Hanner's system, head coach Kevin Stallings' recruits and historical ability to work freshman and less utilized players into the starting rotation immediately is called into question.

After some poking around I found Vandy's 2008-2009 season as the closest recent comparison from the last 10 years that could compare with what Stallings is being forced to do this season: Make a solid group of freshman and a few good reserves into a starting five. That season the Commodores only had three returners back, and a great corps of five highly-touted freshman incoming. Pomeroy ranked them 338th in the nation in experience that year.

They turned out to be fairly average. The Commodores won 19 games that season, were bounced in the first round of the SEC tournament (by Alabama), and didn't participate in any postseason tournaments. 

This time around Stallings has even less to work with, a consideration that Hanner's rankings must have picked up on and adjusted for.

Even so, Pomeroy and Hanner's rankings differ by 10 spots or less for 35.7 percent of all 345 teams covered by both lists, and the average differential for all teams between the two rankings was 25.5, so the above teams are certainly outliers. Further, Pomeroy and Hanner agreed exactly on the rankings of five teams: Akron (63rd), Alabama State (336th), Illinois (54th), Jacksonville State (175th) and Oral Roberts (145th).

As a means of maybe combining the strengths of both methodologies as well as the strengths of human bias, which Pomeroy concedes might have an edge over his numbers anyway. I averaged the rankings from Pomeroy, Hanner and the AP voters to come up with an amalgam ranking. Those numbers are below with differentials included.

Some interesting things to note:

– Hanner's score adjusts Indiana to a place that humans agree with.

– UCLA's transfers might make them even better than AP voters think.

– Texas and New Mexico could vye for the Top 25 rankings currently held by San Diego State and Florida State.

 

Rank Team Hanner Rank KenPom Rank AP Rank Avg. Ranking Differential (excl. AP)
1 Indiana 1 3 1 1.7 2
2 Kentucky 2 1 3 2.0 1
3 Ohio St. 8 2 4 4.7 6
4 Louisville 10 8 2 6.7 2
5 Kansas 4 10 7 7.0 6
6 UCLA 3 6 13 7.3 3
7 Michigan St. 5 4 14 7.7 1
8 Duke 7 9 8 8.0 2
9 Florida 6 11 10 9.0 5
10 Syracuse 11 7 9 9.0 4
11 NC State 12 24 6 14.0 12
12 Arizona 9 23 12 14.7 14
13 Gonzaga 13 14 21 16.0 1
14 Texas 20 13   16.5 7
15 UNLV 14 18 18 16.7 4
16 Memphis 18 16 17 17.0 2
17 North Carolina 26 15 11 17.3 11
18 New Mexico 15 21   18.0 6
19 Missouri 16 29 15 20.0 13
20 Michigan 44 12 5 20.3 32
21 Wisconsin 33 5 23 20.3 28
22 Notre Dame 22 17 22 20.3 5
23 Pittsburgh 19 22   20.5 3
24 Baylor 25 20 19 21.3 5
25 Creighton 24 30 16 23.3 6
26 Kansas St. 29 19   24.0 10
27 Minnesota 23 26   24.5 3
28 Marquette 21 32   26.5 11
29 Tennessee 17 38   27.5 21
30 St. Louis 32 25   28.5 7
31 San Diego St. 41 28 20 29.7 13
32 Georgetown 27 33   30.0 6
33 Alabama 28 35   31.5 7
34 Stanford 31 36   33.5 5
35 Miami FL 34 40   37.0 6
36 Florida St. 38 51 25 38.0 13
37 Cincinnati 52 27   39.5 25
38 VCU 48 31   39.5 17
39 Saint Joseph's 37 42   39.5 5
40 California 40 39   39.5 1
41 Iowa St. 30 62   46.0 32
42 Northern Iowa 46 48   47.0 2
43 Rutgers 43 52   47.5 9
44 Davidson 51 44   47.5 7
45 Oklahoma 53 43   48.0 10
46 St. Mary's 50 47   48.5 3
47 Ohio 42 56   49.0 14
48 Butler 36 64   50.0 28
49 Temple 39 61   50.0 22
50 Drexel 62 41   51.5 21
51 Belmont 74 34   54.0 40
52 Illinois 54 54   54.0 0
53 Purdue 73 37   55.0 36
54 West Virginia 55 58   56.5 3
55 BYU 67 50   58.5 17
56 Iowa 64 53   58.5 11
57 Bucknell 60 59   59.5 1
58 Colorado 65 57   61.0 8
59 Villanova 58 66   62.0 8
60 South Dakota St. 77 49   63.0 28
61 Virginia 61 65   63.0 4
62 Akron 63 63   63.0 0
63 Arkansas 35 94   64.5 59
64 Massachusetts 75 55   65.0 20
65 Mississippi 56 74   65.0 18
66 Oklahoma 47 85   66.0 38
67 Robert Morris 59 73   66.0 14
68 Washington 66 70   68.0 4
69 Virginia Tech 68 79   73.5 11
70 South Florida 71 76   73.5 5
71 Murray St. 49 99   74.0 50
72 Connecticut 107 45   76.0 62
73 Oregon St. 83 69   76.0 14
74 Seton Hall 69 83   76.0 14
75 Penn St. 88 67   77.5 21
76 Richmond 96 60   78.0 36
77 Lehigh 79 80   79.5 1
78 Colorado St. 70 92   81.0 22
79 St. John's 76 87   81.5 11
80 Clemson 90 77   83.5 13
81 Boise St. 78 89   83.5 11
82 Princeton 94 75   84.5 19
83 Wichita St 127 46   86.5 81
84 Washington St. 81 95   88.0 14
85 Illinois St. 92 86   89.0 6
86 Middle Tennessee 97 88   92.5 9
87 Central Florida 82 106   94.0 24
88 Oregon 98 91   94.5 7
89 La Salle 118 72   95.0 46
90 Northwestern 102 90   96.0 12
91 North Dakota St. 91 101   96.0 10
92 Marshall 99 93   96.0 6
93 Wyoming 57 138   97.5 81
94 Georgia 116 81   98.5 35
95 Providence 122 78   100.0 44
96 North Texas 93 107   100.0 14
97 Mercer 119 82   100.5 37
98 UTEP 101 102   101.5 1
99 DePaul 105 100   102.5 5
100 Utah St. 108 98   103.0 10
101 South Carolina Upstate 113 97   105.0 16
102 Dayton 109 104   106.5 5
103 Manhattan 84 131   107.5 47
104 Detroit 87 128   107.5 41
105 East Carolina 95 120   107.5 25
106 Wagner 103 112   107.5 9
107 College of Charleston 80 136   108.0 56
108 Vermont 89 127   108.0 38
109 St. Bonaventure 121 96   108.5 25
110 Long Beach St. 104 113   108.5 9
111 Louisiana St. 72 153   112.5 81
112 Harvard 155 71   113.0 84
113 Texas A&M 112 115   113.5 3
114 Xavier 117 114   115.5 3
115 Fresno St. 86 146   116.0 60
116 USC 45 188   116.5 143
117 Valparaiso 106 130   118.0 24
118 Tulane 126 110   118.0 16
119 Nevada 114 122   118.0 8
120 Green Bay 115 121   118.0 6
121 Evansville 129 111   120.0 18
122 Denver 158 84   121.0 74
123 Iona 133 109   121.0 24
124 South Carolina 85 158   121.5 73
125 Montana 140 103   121.5 37
126 George Mason 139 105   122.0 34
127 Maryland 111 135   123.0 24
128 Georgia Tech 128 119   123.5 9
129 Florida Gulf Coast 123 125   124.0 2
130 Vanderbilt 181 68   124.5 113
131 Cleveland St. 141 108   124.5 33
132 Old Dominion 120 132   126.0 12
133 Delaware 137 116   126.5 21
134 Tennessee St. 110 152   131.0 42
135 Quinnipiac 147 117   132.0 30
136 Northeastern 125 148   136.5 23
137 Columbia 132 142   137.0 10
138 New Mexico St. 152 124   138.0 28
139 Long Island 144 133   138.5 11
140 Auburn 100 179   139.5 79
141 Southern Miss 164 118   141.0 46
142 Indiana St. 135 150   142.5 15
143 Georgia St. 124 164   144.0 40
144 Oral Roberts 145 145   145.0 0
145 Fairfield 157 137   147.0 20
146 Air Force 143 151   147.0 8
147 Missouri St. 172 123   147.5 49
148 Niagara 156 139   147.5 17
149 Loyola MD 169 129   149.0 40
150 St. Francis NY 136 163   149.5 27
151 San Diego 160 140   150.0 20
152 Santa Clara 146 154   150.0 8
153 Idaho 142 159   150.5 17
154 Buffalo 178 126   152.0 52
155 Boston College 171 134   152.5 37
156 Texas Tech 131 176   153.5 45
157 George Washington 148 172   160.0 24
158 Louisiana Tech 159 161   160.0 2
159 Texas Arlington 153 168   160.5 15
160 South Alabama 134 191   162.5 57
161 Youngstown St. 166 162   164.0 4
162 UAB 191 141   166.0 50
163 James Madison 149 184   166.5 35
164 Western Kentucky 154 180   167.0 26
165 Kent St. 161 174   167.5 13
166 Savannah St. 197 143   170.0 54
167 UC Irvine 182 160   171.0 22
168 Portland 165 182   173.5 17
169 Arizona St. 130 218   174.0 88
170 Stephen F. Austin 170 178   174.0 8
171 Arkansas St. 183 167   175.0 16
172 Jacksonville St. 175 175   175.0 0
173 Sacred Heart 176 177   176.5 1
174 Loyola Marymount 200 155   177.5 45
175 Northern Colorado 187 170   178.5 17
176 Bowling Green 194 166   180.0 28
177 Toledo 177 183   180.0 6
178 Oakland 216 149   182.5 67
179 Duquesne 208 157   182.5 51
180 Weber St. 179 187   183.0 8
181 Nebraska 151 216   183.5 65
182 Wake Forest 167 201   184.0 34
183 Marist 199 169   184.0 30
184 Houston 173 195   184.0 22
185 Drake 168 204   186.0 36
186 Pacific 180 192   186.0 12
187 Charlotte 229 144   186.5 85
188 Morehead St. 188 185   186.5 3
189 Southern Illinois 138 236   187.0 98
190 North Florida 204 171   187.5 33
191 TCU 189 186   187.5 3
192 Elon 185 194   189.5 9
193 NC Asheville 210 173   191.5 37
194 UC Santa Barbara 228 156   192.0 72
195 Stony Brook 219 165   192.0 54
196 Stetson 186 199   192.5 13
197 Cornell 196 189   192.5 7
198 Southern Methodist 174 215   194.5 41
199 Hawaii 150 243   196.5 93
200 North Dakota 195 203   199.0 8
201 Charleston Southern 252 147   199.5 105
202 NC Wilmington 163 237   200.0 74
203 Milwaukee 201 200   200.5 1
204 Cal Poly 190 217   203.5 27
205 Mississippi St. 226 193   209.5 33
206 Wright St. 209 211   210.0 2
207 Florida Atlantic 193 232   212.5 39
208 East Tennessee St. 240 190   215.0 50
209 Southeast Missouri St. 211 219   215.0 8
210 Texas Southern 184 250   217.0 66
211 Cal St. Fullerton 205 230   217.5 25
212 Coastal Carolina 230 207   218.5 23
213 Arkansas Little Rock 215 222   218.5 7
214 American 207 231   219.0 24
215 Bradley 206 234   220.0 28
216 Delaware St. 236 205   220.5 31
217 Fordham 223 220   221.5 3
218 Texas San Antonio 231 214   222.5 17
219 Siena 241 212   226.5 29
220 NC Greensboro 232 221   226.5 11
221 Tulsa 274 181   227.5 93
222 Albany 213 244   228.5 31
223 Gardner Webb 217 240   228.5 23
224 Lipscomb 233 224   228.5 9
225 Western Michigan 218 242   230.0 24
226 IPFW 225 235   230.0 10
227 Yale 251 210   230.5 41
228 Western Illinois 198 264   231.0 66
229 Loyola Chicago 203 259   231.0 56
230 Louisiana Lafayette 192 272   232.0 80
231 William & Mary 237 228   232.5 9
232 Wofford 269 198   233.5 71
233 Hofstra 202 265   233.5 63
234 Western Carolina 259 209   234.0 50
235 Seattle 214 254   234.0 40
236 Utah 162 309   235.5 147
237 San Francisco 275 197   236.0 78
238 Ball St. 235 239   237.0 4
239 Miami OH 234 241   237.5 7
240 Maine 255 223   239.0 32
241 Pennsylvania 285 196   240.5 89
242 Southern Utah 221 263   242.0 42
243 Boston University 273 213   243.0 60
244 Rider 262 225   243.5 37
245 Canisius 212 276   244.0 64
246 Texas St. 227 261   244.0 34
247 Holy Cross 284 206   245.0 78
248 Mount St. Mary's 243 251   247.0 8
249 IUPUI 267 229   248.0 38
250 Furman 260 238   249.0 22
251 Central Connecticut 291 208   249.5 83
252 Rhode Island 254 245   249.5 9
253 Campbell 250 249   249.5 1
254 Dartmouth 268 233   250.5 35
255 Cal St. Bakersfield 242 260   251.0 18
256 Rice 278 227   252.5 51
257 San Jose St. 238 268   253.0 30
258 Jacksonville 261 246   253.5 15
259 Liberty 239 269   254.0 30
260 Cal St. Northridge 258 253   255.5 5
261 Tennessee Tech 264 248   256.0 16
262 Army 220 293   256.5 73
263 Georgia Southern 263 252   257.5 11
264 Florida International 224 295   259.5 71
265 VMI 253 266   259.5 13
266 Utah Valley 245 277   261.0 32
267 Austin Peay 266 256   261.0 10
268 Lamar 303 226   264.5 77
269 Chattanooga 247 282   264.5 35
270 Sam Houston St. 246 286   266.0 40
271 Northwestern St. 277 255   266.0 22
272 McNeese St. 288 247   267.5 41
273 Eastern Kentucky 244 292   268.0 48
274 Missouri Kansas City 257 281   269.0 24
275 Eastern Michigan 248 291   269.5 43
276 UC Davis 222 318   270.0 96
277 Monmouth 282 258   270.0 24
278 Morgan St. 265 275   270.0 10
279 St. Peter's 286 257   271.5 29
280 Bethune Cookman 271 274   272.5 3
281 Southeastern Louisiana 249 303   276.0 54
282 North Carolina Central 298 262   280.0 36
283 Sacramento St. 289 271   280.0 18
284 Illinois Chicago 272 289   280.5 17
285 Eastern Washington 283 278   280.5 5
286 Hartford 296 267   281.5 29
287 Brown 279 284   281.5 5
288 New Hampshire 281 283   282.0 2
289 Montana St. 256 311   283.5 55
290 Lafayette 300 270   285.0 30
291 Troy 287 285   286.0 2
292 North Carolina A&T 270 310   290.0 40
293 Appalachian St. 308 273   290.5 35
294 Jackson St. 276 305   290.5 29
295 The Citadel 292 297   294.5 5
296 Samford 302 288   295.0 14
297 Portland St. 313 279   296.0 34
298 South Dakota 280 314   297.0 34
299 Pepperdine 297 299   298.0 2
300 Bryant 307 290   298.5 17
301 Presbyterian 314 287   300.5 27
302 Tennessee Martin 295 308   301.5 13
303 Radford 309 296   302.5 13
304 Howard 304 301   302.5 3
305 Norfolk St. 327 280   303.5 47
306 High Point 319 294   306.5 25
307 MD Baltimore County 290 324   307.0 34
308 UC Riverside 293 323   308.0 30
309 Hampton 312 304   308.0 8
310 Northern Illinois 321 298   309.5 23
311 Winthrop 318 302   310.0 16
312 Navy 301 321   311.0 20
313 Texas A&M CC 294 329   311.5 35
314 Prairie View A&M 311 312   311.5 1
315 SIU Edwardsville 325 300   312.5 25
316 Houston Baptist 310 315   312.5 5
317 Idaho St. 299 328   313.5 29
318 Nicholls St. 315 313   314.0 2
319 Nebraska Omaha 305 327   316.0 22
320 Kennesaw St. 328 306   317.0 22
321 St. Francis PA 330 307   318.5 23
322 Southern 306 332   319.0 26
323 Northern Arizona 316 325   320.5 9
324 NJIT 322 319   320.5 3
325 Central Arkansas 326 316   321.0 10
326 Central Michigan 324 322   323.0 2
327 Texas Pan American 320 330   325.0 10
328 Colgate 333 320   326.5 13
329 Binghamton 317 341   329.0 24
330 Fairleigh Dickinson 332 326   329.0 6
331 Louisiana Monroe 329 331   330.0 2
332 Coppin St. 344 317   330.5 27
333 Florida A&M 323 340   331.5 17
334 Arkansas Pine Bluff 334 333   333.5 1
335 Alabama St. 336 336   336.0 0
336 Maryland Eastern Shore 331 342   336.5 11
337 Mississippi Valley St. 341 334   337.5 7
338 Alabama A&M 340 335   337.5 5
339 Towson 337 338   337.5 1
340 Alcorn St. 335 343   339.0 8
341 Eastern Illinois 343 339   341.0 4
342 South Carolina St. 338 346   342.0 8
343 Chicago St. 339 345   342.0 6
344 Longwood 342 344   343.0 2
345 Grambling 345 347   346.0 2

 

Quantcast