The U.S. News ranking formula in... 20XX

The U.S. News formula currently consists of 9 factors.
Click the buttons to learn more about each factor and any subfactors that may go into it.

Test-Optional Schools & Other Pandemic Tweaks

A few minor changes were made for this ranking, which is the latest edition as of May 2022. As part of the selectivity factor, U.S. News used to give a 15% penalty to the standardized test scores of any school where fewer than 75% of applicants reported scores. For 2022, the threshold was lowered to 50%. As a result, U.S. News says only 4% of ranked schools got penalized in this ranking. This was probably changed because starting in the 2020–21 admissions cycle, many schools went test-optional.

The pandemic probably factored into another set of changes U.S. News made. They switched to 2-year averages of the data for class size, faculty salary, and student debt (instead of just getting their data from 1 year). However, this also means individual rankings are less likely to change from year to year. The last tweak was that schools with fewer than 10 reputation survey ratings are no longer removed from the ranking. Why the reputation survey still exists at all is another question entirely.

Student Debt Added to the Formula, While Selectivity Continues to Be Deemphasized

Just two years after adding the social mobility factor, U.S. News added another brand-new factor measuring graduate indebtedness. The factor aims to reward colleges that leave their graduates with less student debt. College affordability had previously been identified as an omission from the rankings by, for example, this research paper.

In a 180° shift from previous policy, test-blind schools are now allowed to be ranked, but now all schools must report a 6-year graduation rate. The pandemic-era shift away from standardized testing may have driven this change; we’ll see a few more coronavirus-caused changes in next year’s methodology.

To make room for the new student debt factor, the weight of alumni giving was reduced from 5% to 3%, and the weight of selectivity was reduced from 10% to 7%. Both remaining selectivity subfactors (test scores and class rank) continue to decline in relevance in today’s college application landscape, but they remain in the formula. Keep in mind that selectivity was once weighted at 15%.

There are a couple of the usual minor tweaks as well. Graduation rate performance is now based on a 2-year average, continuing the trend of smoothing out the data used for the rankings. On the other hand, in faculty resources, salary is now based on “full-time faculty salaries” over 1 year rather than 2-year average of “both salaries and nonsalary compensation.”

U.S. News Dumps High School Counselor Survey but Can’t Quit Class Rank

After 9 years, high school counselor reputation surveys have been discontinued. U.S. News says they did this because they “had greater confidence in the data and the significantly increased response rates from the peer assessment surveys”. Look at the downward trend in response rates on the high school survey (in my 2019 write-up) and you’ll understand why they dumped it. The peer assessment survey now accounts for all 20% of this category’s overall weight.

A change made to the selectivity factor revealed how unreliable class rank data has become for measuring selectivity. If class rank is available for <10% of students at a school, the school gets an estimate instead of a penalty. (It’s unclear how they calculate this estimate.) However, schools still get a penalty if the data is available for 10–20% of students; this may be because some schools don’t report class rank data for any of their students, which puts them in the <10% category. Also, schools get a footnote in the ranking table if class rank is available for <20% of students (was <50%). Before this change, 79% of schools in the top 50 had a footnote about their class rank data; after this change, only 8% of top-50 schools had a footnote. This means most of the top-ranked schools have class rank data for <50% of their students. So why is that subfactor still part of the formula?

The prediction formula for graduation rate performance now includes the percentage of students who are first-generation students and uses a “multiyear average instead of a single cohort” for calculating the percentage of students on Pell grants.

U.S. News also made some changes to where they got their data from. They switched to “open-source data from the Bureau of Economic Analysis” for salary data in the faculty resources category. This data is presumably more reliable than what the colleges were previously feeding to U.S. News. In addition, substitute data from the Department of Education’s Integrated Postsecondary Education Data System was used if schools do not report data in some areas. This suggests some schools were not self-reporting data (or reporting inaccurate data) in these areas.

Negative Media Coverage Drives Major Formula Changes

For the 2019 ranking, U.S. News made the most significant formula changes they’d made in a while — perhaps since they moved away from the all-survey model 30 years before. This is interesting timing to say the least, given that the year before, Politico and The Washington Post both released articles criticizing many aspects of the ranking methodology.

For the first time since 1997 (when graduation rate performance was added), a completely new factor was added to the formula: social mobility. This was almost certainly created in response to the above accusations that the rankings “promote economic inequality”. Click the ? button in the Key to learn more about it and its subfactors.

Some of the existing categories have been given new names on the “what’s new” page (click the “2019” link to read that page). Some of these names (“Student Excellence” for “Selectivity”; “Expert Opinion” for “Reputation”) give off the impression that U.S. News is trying to “sell” the usefulness of these factors to readers. Both of these factors were changed in 2019 to remove or reduce controversial subfactors. For selectivity (oh, sorry, “Student Excellence”), acceptance rate was removed as a subfactor, though U.S. News still publishes a separate ranking of the schools with the lowest acceptance rates.

Only 35.5% of college administrators responded to the reputation survey (down about 5% from the year before). The Washington Post’s 2017 article said that many administrators have admitted they “don’t fill out the forms themselves because they don’t really have a deep understanding of other schools’ programs... [a]nd they doubt that many of those who do complete the survey possess a deep understanding”. “Expert Opinion”? Give me a break.

And then there’s the high school counselor survey, which in its 9th year is looking like a failed experiment, as the weight of the survey was lowered from 7.5% to 5% overall. (The survey of college administrators remained weighted at 15%.) What’s more, unlike in previous years, U.S. News did not publish the response rate for the high school counselor survey. Given that only 7% of counselors responded the year before, there’s a good chance U.S. News didn’t publish this information because they were embarrassed about how low the response rate was.

Yawn...

The only changes made this year were subtle, behind-the-scenes ones. Specifically, the graduation rate performance prediction formula now accounts for what percentage of degrees granted are STEM degrees (because STEM students graduate at lower rates). Additionally, U.S. News verified graduation rate data (part of the retention factor) using the government-run National Center for Educational Statistics instead of the NCAA. Yeah, it's not very exciting.

Class Size Gets More Complicated

Class size still counts for 40% of the faculty resources score (8% of the overall score) but more gradations of “credit” (in descending order: <20; 20–29; 30–39; 40–49; and ≥50 gets no credit). This may reduce the advantage gained if a school tries to game the system like Clemson did in 2009. Clemson, and presumably other schools, tried to improve their class size score by reducing a class with, say, 22 students down to 19, while allowing large classes to get even larger.

Every year, U.S. News publishes the response rates for their reputation surveys, allowing us to see how much these rates have declined over the years. In 2017, 39% of college administrators filled out the peer assessment survey (down from 65% in 1997), and 7% of high school counselors filled out their survey (down from 21% in 2011). This reflects the declining popularity of the rankings among people with ties to higher education.

Test-Optional Schools Get Penalized, in a Decision That’s Sure to Age Well

The percentage weights did not change. However, “2 small changes” were made to reputation: peer assessment ratings are now based on the last 2 years, and high school counselor ratings are now based on the last 3 years. In both cases, an additional year has been added to the average.

And in a move that seems surprising now, if less than 75% of applicants submitted test scores, a 15% penalty is applied to the school’s test scores. This has the potential to punish test-optional schools, though they changed it once the pandemic prompted many top-ranked schools to go test-optional. This is the first ranking where they mentioned that they do this, but they say they did it last year as well.

No Shirt, No Shoes, No Service, No Changes

No changes were made to the methodology.

The Slow Decline of Class Rank

Class rank is deemphasized in the selectivity calculation because “as each year passes, the proportion of high school graduates with class rank on their transcripts is falling,” making the data they have less representative. This coincides with selectivity’s weight being reduced from 15% to 12.5% (with retention gaining 2.5% as a result).

My Friend Made a Tiny Change to the U.S. News Rankings and All I Got Was This Lousy T-Shirt!

In 2013, the methodology only got “one small methodology change” to the reputation factor: the high school counselor reputation ratings are now averaged from the last 2 years, rather than being based on 1 year.

“Strategic Voting” and For-Profit Colleges

The percentage weights did not change. However, U.S. News did make a couple other methodology changes. On the reputation survey, the 2 highest and 2 lowest scores are now removed for each school “[i]n order to reduce the impact of strategic voting by respondents.” This may be a response to the “strategic voting” by Clemson and other schools. The article also discusses how Clemson manipulated their class sizes to boost their ranking; U.S. News made a change to that part of the formula in 2017.

In addition, accredited for-profit colleges that give bachelor’s degrees are now ranked. Shoutout to the University of Phoenix, which as of the 2022 ranking is in the bottom 25%, so it doesn’t get an official ranking, but it ranks last in almost every category in the latest ranking, so it would clearly be last if it did get a number.

High School Counselor Survey Added to Last Ranking Published in Print Magazine

High school counselors are now given their own reputation survey, which counts for one-third of the reputation score (7.5% of the overall score). U.S. News also took this opportunity to lower the factor's weight from 25% to 22.5%, which makes the overall weight of the existing college administrator survey 15%. The high school counselor survey uses the same methodology as the existing college administrator survey (i.e. rate each of these colleges on a scale from 1 to 5). U.S. News claims that high school counselors have been asking to be included for a long time.

Meanwhile, the weight of graduation rate performance was increased (from 5% to 7.5%), because according to the magazine, it’s “well received by many higher education researchers.” (Does that mean their other metrics aren’t well-received by researchers?)

In previous years, the bottom half of colleges in the ranking were listed alphabetically instead of hierarchically. Starting in 2011, the top 75% get numerical ranks, leaving the remaining 25% alphabetized. U.S. News says this expansion of the rankings was by popular demand, but it raises the question of how accurate/distinguishable the rankings of these third-quartile schools really are.

And this isn’t something I would normally include because it isn’t about the ratings per se, but this was the last ranking before U.S. News ceased publishing its print magazine (though they continue to publish a print edition of their college rankings). They started out as a national news-magazine à la Time or Newsweek, but looking at their website today, you can see they’ve become a shell of their former self. All of their top 5 most popular articles (as of writing) are repackaged articles from Reuters and the Associated Press. Their homepage is packed with stock-market listicles. Original news reporting is present on the homepage, but many prominently-featured articles in this category are days old. There is, however, one part of their business that continues to expand: their rankings.

Test Score Tweaks

The percentage weights did not change. The only notable change to the ranking was in selectivity, where the test scores calculation now includes both SAT and ACT scores (instead whichever one was submitted by more students). Outside of the main ranking, U.S. News mentioned their “Up-and-Coming Schools” ranking, created “in response to criticism that [their] academic peer assessment survey is too slow to pick up improvements at colleges.” (This is the second year they published this ranking, though they did not mention it in their 2009 methodology.) They don’t even consider removing the peer assessment survey from the main ranking formula.

No changes were made to the methodology.

Test-Blind Schools Get Unranked, in a Decision That’s Sure to Age Well

The percentage weights did not change. One change that was made is that test-blind schools are now unranked. While discussing this change, U.S. News noted several other reasons that schools were unranked:

“schools with total enrollment of fewer than 200 students; schools where a vast proportion of students are nontraditional; colleges that don't accept first-year students, sometimes called upper-division schools; private universities that are for-profit; and a few specialized schools in arts, business, or engineering.”

These schools may have been excluded because test scores are a key part of the selectivity factor (counted for 7.5% overall at the time). In addition, the percentage of students on Pell grants is now part of the graduation rate performance prediction formula.

Still More of the Same

No changes were made to the methodology (for the 3rd year in a row).

Even More of the Same

No changes were made to the methodology (for the 2nd year in a row).

More of the Same

No changes were made to the methodology, but there was one change to how the methodology was introduced. Their methodology article began with a story about a high school student who was trying to decide between two colleges that she got accepted to: Swarthmore and Carnegie Mellon. She chose Swarthmore because “It was ranked No. 2 in liberal arts. … That meant a lot.” It’s hard to see U.S. News publishing an anecdote like that today. Instead, their website now has messages such as “Finding the right college for you is just as, if not more, important than finding the best college.”

Goodbye, Yield Rate

In the selectivity category, yield rate was finally dropped from the formula, after years of outside pressure to do so from other media sources. For the other selectivity subfactors, the weights of test scores and class rank were increased, and the weight of acceptance rate was decreased.

3 Years, No Changes

No changes were made to the methodology (for the 3rd year in a row).

2 Years, No Changes

No changes were made to the methodology (for the 2nd year in a row).

No Changes, Except for a Clarification

No changes were made to the methodology, though a clarification was made about how class size is calculated/weighted (¾ of the class size score is based on classes with fewer than 20 students; the remaining ¼ is based on the number of classes with 50 or more students).

Tweak to Financial Resources; Controversy Over Yield Rate

U.S. News changed how they weighted some categories to reflect when there were large disparities between schools. For example, the spending figures for financial resources might now look like $200k, 100k, 50k; where they would have previously been 1, 2, 3. This might make the rankings more accurate according to the criteria U.S. News sets out, but it could also disproportionately benefit the institutions with larger endowments.

There are two things that aren’t formula changes that I want to mention. First, Caltech was #1 in this ranking — the only time the #1 college is not Harvard, Yale, Princeton, or Stanford (all of which have held or tied for the top spot multiple times). How long will it be before another college joins the club?

Second, in the middle of the methodology, there’s a paragraph that attempts to address accusations that by including yield rate in their selectivity formula, they were encouraging colleges to admit more students through early decision programs. Yield rate is the percentage of accepted students who choose to enroll, and students accepted through early decision are required to enroll, raising a school’s yield rate. This hurts low-income applicants because they lose the ability to compare the financial aid packages offered by the colleges that accept them. U.S. News doesn’t make any changes as a result of this criticism, but in the next few years, they continue to be attacked for their use of yield rate...

Financial Resources Calculations Get Simplified

The financial resources factor now only includes educational spending, not the “other spending” category present in the 1998 ranking. U.S. News cited “changes in reporting rules for private colleges and universities” as their reason for the change. But it also removes the spending that’s less related to academics. When a school spends a lot of money on, say, building maintenance, that doesn’t really say anything about the school’s academic quality, which is what U.S. News purports to measure. In addition, the reputation survey switched from rating colleges on a 4-point scale to a 5-point scale. The 5-point scale is still used today.

The First 15 Years

The first ranking was published in 1984; it was based entirely on a reputation survey of college presidents, who were asked to select the 5 best institutions in the same category as theirs. (For example, Regional Liberal Arts College presidents select the “best” Regional Liberal Arts Colleges.)

U.S. News released new versions of the ranking in 1986 and 1988; for the latter ranking, they asked college presidents to pick the 10 best colleges, instead of 5.

In 1989, the ranking became an annual feature, and U.S. News completely changed the ranking methodology. Instead of just using the reputation survey, the 1989 ranking used the following 5 factors:

• Selectivity — subfactors: acceptance rate, SAT or ACT scores of first-year students, and class rank (% of freshmen in top 10% of high school class)
• Faculty resources — subfactors: % of faculty with doctorates, student:faculty ratio, and instructional budget per student (incl. faculty salaries)
• Financial resources — subfactors: endowment per student and library budget per student
• Retention — subfactors: % of freshmen who return as sophomores and 4-year graduation rate
• Reputation survey of “college presidents, deans and admissions officers”; these administrators were asked to rank colleges on scale from 1 to 4

In 1990, U.S. News added state and local government funding per student as a subfactor for financial resources. They also switched to a 5-year graduation rate for the retention factor.

In 1991, yield rate was added as a subfactor under selectivity. In addition, the financial resources factor was simplified to consist solely of total spending divided by total enrollment.

In 1993, financial resources was reworked again. It’s now based on education spending per full-time student, plus other spending (e.g. research, maintenance) per student.

In 1994, alumni giving was added as a factor — the first new factor since the methodology was reworked 5 years earlier. In addition, class size was added as a subfactor under faculty resources, in order to reward schools with smaller classes.

In 1997, graduation rate performance (then known as “value added”) was added to the formula. It initially counted for 5% of a school’s overall score, but this number has now increased to 8%. Click the ? button in the Key to learn more about how it works.

1997 was also the first year that U.S. News published the percentages they use to weight the main factors (e.g. selectivity, faculty resources); 1998 was the first year that U.S. News published percentage weights for subfactors (e.g. acceptance rate, class size).