Test-Optional Schools & Other Pandemic Tweaks
A few minor changes were made for this ranking, which is the latest edition as of May 2022. U.S. News used to give a 15% penalty to the standardized test scores of any school where fewer than 75% of applicants reported scores. For 2022, the threshold was lowered to 50%. As a result, U.S. News says only 4% of ranked schools got penalized in this ranking. This was probably changed because starting in the 2020–21 admissions cycle, many schools went test-optional.
The pandemic probably factored into another set of changes U.S. News made. They switched to 2-year averages of the data for class size, faculty salary, and student debt (instead of just getting their data from 1 year). However, this also means individual rankings are less likely to change from year to year. The last tweak was that schools with fewer than 10 reputation survey ratings are no longer removed from the ranking. Why the reputation survey still exists at all is another question entirely.
Student Debt Added to the Formula, While Selectivity Continues to Be Deemphasized
Added a new “Graduate indebtedness” factor, which aims to reward colleges that leave their graduates with less student debt.
Graduation rate performance is now based on a 2-year average, continuing the trend of smoothing out the data used for the rankings.
Test-blind schools are now allowed to be ranked, but now all schools must report a 6-year graduation rate.
Faculty resources: salary based on “full-time faculty salaries” over 1 year rather than 2-year average of “both salaries and nonsalary compensation.”
Selectivity (from 10% to 7%): including test scores (from 7.75% to 5%) and class rank (from 2.25% to 2%). Both remaining data points in this section continue to decline in relevance in today’s college application landscape, but they remain in the formula. Keep in mind that selectivity was once weighted at 15%.
Headline
Grad. rate performance: prediction formula now includes % of students who are first-generation students (helps colleges that are better for first-gen students); uses “multiyear average instead of a single cohort” for calculating % of students on Pell grants.
Reputation (still 20%): No more high school counselor surveys. U.S. News says they did this because they “had greater confidence in the data and the significantly increased response rates from the peer assessment surveys”. Look at the downward trend in response rates on the high school survey and you’ll understand why they dumped it. The peer assessment survey now accounts for all 20% of this category’s overall weight.
Faculty resources: Switched to “open-source data from the Bureau of Economic Analysis” for faculty salaries. This data is presumably more reliable than what the colleges were feeding to U.S. News.
Selectivity: If class rank is available for <10% of students, school gets an estimate instead of a penalty. However, schools still get a penalty if the data is available for 10–20% of students; this may be because some schools don’t report class rank data for any of their students, placing them in the <10% category. Also, schools get a footnote in the ranking table if class rank is available for <20% of students (was <50%). Before this change, 79% of schools in the top 50 had a footnote about their class rank data; after this change, only 8% of top-50 schools had a footnote.
Substitute data from the Department of Education’s Integrated Postsecondary Education Data System was used if schools do not report data in some areas. This suggests some schools were not self-reporting data (or reporting inaccurate data) in these areas.
Headline
For the 2019 ranking, U.S. News made the most significant formula changes they’d made in a while. This is interesting timing to say the least, given that the year before, Politico and The Washington Post both released articles criticizing many aspects of the ranking methodology.
Some categories have been given new names on the “what’s new” page. Some of these names (“Student Excellence” for “Selectivity”; “Expert Opinion” for “Reputation”) give off the impression that U.S. News is trying to “sell” the usefulness of these factors to readers.
Added a new “Social Mobility” factor.
Selectivity (“Student Excellence”): Eliminated acceptance rate from the formula.
Reputation (“Expert Opinion”) the weight of high school counselors’ ratings has been lowered from 7.5% to 5% overall. (The survey of college administrators remains weighted at 15%.) Is this because of low response rates?
- Only 35.5% of college administrators responded to the reputation survey (down about 5% from the year before). Unlike in previous years, U.S. News did not publish the response rate for the high school counselor survey. Given that only 7% of counselors responded the year before, there’s a good chance U.S. News didn’t publish this information because they were embarrassed about how low the response rate was.
Headline
Percentages unchanged.
The “graduation rate performance” prediction formula now accounts for what percentage of degrees granted are STEM degrees (because STEM students graduate at lower rates).
Headline
Every year, U.S. News publishes the response rates for their reputation surveys, allowing us to see how much these rates have declined over the years. In 2017, 39% of college administrators filled out the peer assessment survey (down from 65% in 1997), and 7% of high school counselors filled out their survey (down from 21% in 2011). This reflects the declining popularity of the rankings among people with ties to higher education.
Faculty resources: class size still counts for 40% but more gradations of “credit” (in descending order: <20; 20–29; 30–39; 40–49; and ≥50 gets no credit). This may reduce the advantage gained if a school tries to game the system like Clemson did in 2009. Clemson, and presumably other schools, tried to improve their class size score by reducing a class with, say, 22 students down to 19, while allowing large classes to get even larger.
Headline
Percentages unchanged.
“2 small changes” to Reputation: peer assessment ratings are now based on the last 2 years, and high school counselor ratings are now based on the last 3 years.
If less than 75% of applicants submitted test scores, a 15% penalty is applied to the school’s test scores. This has the potential to punish test-optional schools, though they changed it once the pandemic prompted many top-ranked schools to go test-optional. This is the first ranking where they mentioned that they do this, but they say they did it last year as well.
No Shirt, No Shoes, No Service, No Changes
No changes were made to the methodology.
The Slow Decline of Class Rank Begins
Class rank is deemphasized in the selectivity calculation because “as each year passes, the proportion of high school graduates with class rank on their transcripts is falling,” making the data they have less representative.
My Friend Made a Tiny Change to the U.S. News Rankings and All I Got Was This Lousy T-Shirt!
In 2013, the methodology only got “one small methodology change” to the reputation factor: the high school counselor reputation ratings are now averaged from the last 2 years, rather than being based on 1 year.
Headline
Percentages unchanged.
Accredited for-profit colleges that give bachelor’s degrees are now ranked. Shoutout to the University of Phoenix, which as of the most recent ranking is in the bottom 25% so doesn’t get an official ranking, but it ranks last in almost every category in the latest ranking, so it would clearly be last if it did get a number.
On the reputation survey, the 2 highest and 2 lowest scores are now removed for each school “[i]n order to reduce the impact of strategic voting by respondents.” This may be a response to the “strategic voting” by Clemson and other schools. The article also discusses how Clemson manipulated their class sizes to boost their ranking; U.S. News made a change to that part of the formula in 2017.
Headline
High school counselors are now given their own reputation survey
Reputation: added high school counselor survey (33.3%; original peer assessment survey weighted at 66.7%) —
The high school counselor survey uses the same methodology as the existing college administrator survey (i.e. rate each of these colleges on a scale from 1 to 5).
U.S. News claims that high school counselors have been asking to be included for a long time.
The weight of graduation rate performance was increased (from 5% to 7.5%), because according to the magazine, it’s “well received by many higher education researchers.” (Does that mean their other metrics aren’t well-received by researchers?)
In previous years, the bottom half of colleges in the ranking were listed alphabetically instead of hierarchically. Starting in 2011, the top 75% get numerical ranks, leaving the other 25% alphabetized. U.S. News says this expansion of the rankings was by popular demand, but it raises the question of how accurate/distinguishable the rankings of these schools really are.
Headline
Percentages unchanged.
For selectivity, the test scores calculation now includes both SAT and ACT scores (instead whichever one was submitted by more students). Outside of the main ranking, U.S. News added an “Up-and-Coming Schools” ranking “in response to criticism that [their] academic peer assessment survey is too slow to pick up improvements at colleges.”
No changes were made to the methodology.
Headline
Percentages unchanged.
Test-blind schools are now unranked. U.S. News noted several other reasons that schools were unranked: “a total enrollment of fewer than 200 students; a vast proportion of nontraditional students; no first-year students (these are sometimes called upper-division schools). We did not rank private, for-profit universities; nor did we rank a few specialized schools in arts, business, or engineering.” (Quote from 2009 edition where the same rule was in effect.) This may be because test scores are a key part of the “selectivity” factor (counted for 7.5% overall at the time).
Grad. rate performance: “% of students on Pell grants” is now part of the prediction formula.
Still More of the Same
No changes were made to the methodology (for the 3rd year in a row).
Even More of the Same
No changes were made to the methodology (for the 2nd year in a row).
More of the Same
No changes were made to the methodology, but there was one change to how the methodology was introduced. Their methodology article began with a story about a high school student who was trying to decide between two colleges that she got accepted to: Swarthmore and Carnegie Mellon. She chose Swarthmore because “It was ranked No. 2 in liberal arts. … That meant a lot.” It’s hard to see U.S. News publishing an anecdote like that today. Instead, their website now has messages such as “Finding the right college for you is just as, if not more, important than finding the best college.”
Goodbye, Yield Rate
In the selectivity category, yield rate was finally dropped from the formula, after years of outside pressure to do so from other media sources. For the other selectivity subfactors, the overall weights are now 1.5% accept rate (down from 2.25%), 6% class rank (up from 5.25%), and 7.5% test scores (up from 6%).
3 Years, No Changes
No changes were made to the methodology (for the 3rd year in a row).
2 Years, No Changes
No changes were made to the methodology (for the 2nd year in a row).
No Changes, Except for a Clarification
No changes were made to the methodology, though a clarification was made about how class size is calculated/weighted (¾ of the class size score is based on classes with fewer than 20 students; the remaining ¼ is based on the number of classes with 50 or more students).
Headline
Caltech was #1 in this ranking — the only time the #1 college is not Harvard, Yale, Princeton, or Stanford (all of which have held or tied for the top spot multiple times). How long will it be before another college joins the club?
U.S. News changed how they weighted some categories to reflect when there were large disparities between schools (for example, the numbers for “financial resources” might now look like $200k, 100k, 50k; where they would have previously been 1, 2, 3). This might make the rankings more accurate according to the criteria U.S. News sets out, but it could also disproportionately benefit the institutions with larger endowments.
In the middle of the methodology, there’s a paragraph that attempts to address accusations that by including yield rate in their formula, they were encouraging colleges to admit more students through early decision programs. Yield rate is the percentage of accepted students who choose to enroll, and students accepted through early decision are required to enroll, raising a school’s yield rate. This hurts low-income applicants because they lose the ability to compare the financial aid packages offered by the colleges that accept them. U.S. News doesn’t make any changes as a result of this criticism, but in the next few years, they continue to be attacked for their use of yield rate...
Headline
The financial resources factor now only includes educational spending, not the “other spending” category present in the 1998 ranking. U.S. News cited “changes in reporting rules for private colleges and universities” as their reason for the change. But it also removes the spending that’s less related to academics. When a school spends a lot of money on, say, building maintenance, that doesn’t really say anything about the school’s academic quality, which is what U.S. News purports to measure. In addition, the reputation survey switched from rating colleges on a 4-point scale to a 5-point scale. The 5-point scale is still used today.
The First 15 Years
1998: First year U.S. News publishes percentage weights for subfactors (e.g. acceptance rate, class size)
1997: First year U.S. News publishes the percentages they use to weight the main factors (e.g. selectivity, faculty resources)
1994:
- Added class size as a subfactor under “faculty resources” (schools with smaller classes on average get more points)
- Added alumni giving rate as a new factor
1993: Resources becomes based on education spending per full-time student plus other spending (e.g. research, maintenance) per student
1991:
- Added yield rate to Selectivity
- Simplified Financial Resources to total spending ÷ total enrollment
1990:
- Added state and local government funding per student to Financial Resources
- Switched to 5-year graduation rate for Retention
1989: Ranking switches from biannual to annual and now uses the following 5 criteria:
- Selectivity: acceptance rate, SAT/ACT, and class rank (% of freshmen in top 10% of high school class)
- Faculty resources: % of faculty with doctorates, student:faculty ratio, and instructional budget per student (incl. faculty salaries)
- Financial resources: endowment per student and library budget per student
- Retention: % of freshmen who return as sophomores and 4-year graduation rate
- Reputation survey: of “college presidents, deans and admissions officers”; asked to rank colleges on scale from 1 to 4
1984: First edition of rankings; based entirely on reputation survey
icons to learn more about each part of the U.S. News formula.