Of the many rating systems out there, Ken Pomeroy's is the widest known and my favorite. Now, the formula behind the ratings is getting a tweak or two. You can read the full explanation at Pomeroy's site, but a couple of main points:
1. Margin of victory is at least semi capped now. While in the past dominating an inferior opponent and beating them by a lot but not a humilating margin would make a big difference as to one's rating, now "whether a team beat [Grambling] by 30 or 60 would make little difference in its rating." I'm not sure whether this is a good thing or not actually. I doubt teams are running up the score against a Grambling just to improve their KenPom rating, but it's very possible for a team to struggle for much of a game against a horrible team like Grambling and still beat them by a large margin, which doesn't really tell the full story.
2. It's still not going to be able to pick out the insanely crazy upsets. No surprise here. If anyone can design a ratings system that can predict when, in Pomeroy's words, a Cal Poly will beat UCLA, that person would get hired by any number of people in record time.
3. The biggest impact seems to be on really good mid majors. This is because "more emphasis is placed on postseason play when they finally get to battle teams of comparable strength." The 2008 Davidson team that made the Elite 8 gets a boost, while the recent Belmont teams take a bit of a hit
In all, the formula isn't radically different from before. It should give a clearer picture of what teams can actually do on offense and defense based on the fact that outlier performances won't be taken into account as much, but it still won't be perfect obviously. Nonetheless, it is a good step forward, and taking new information and making tweaks is (almost) always a good thing.
