How U.S. News Calculated the Best Global Universities Rankings
Find out how we determined the world's top universities overall and by region and country.
School-Level Indicators
Publicly available data sources were used to create the school-level indicators.
Number of Ph.D.s awarded (5 percent): This indicator reflects the total number of doctoral degrees awarded in 2012. The number of doctorates awarded can be considered an alternative indicator of research output and is linked to volume.
Number of Ph.D.s awarded per academic staff member (5 percent): This is the number of Ph.D.s awarded per the number of academic faculty members for the same year. This is a size-independent measure of the education environment at the university.
How the Overall Global Scores and Numerical Rankings Were Calculated
To arrive at a school's rank, the overall global scores were calculated using a combination of the weights and z-scores for each of the 10 indicators used in the rankings. In statistics, a z-score is a standardized score that indicates how many standard deviations a data point is from the mean of that variable. This transformation of the data is essential when combining diverse information into a single ranking because it allows for fair comparisons between the different types of data.
Several of the indicators were highly skewed, so the logs of the original values were used. The indicators that used logs were:
- Publications
- Total citations
- Number of highly cited papers
- Number of Ph.D.s awarded
- Global research reputation
- Regional research reputation
This log manipulation rescaled the data and allowed for a more normalized and uniform spread across each of the indicators. After these six indicators were normalized, the z-scores for each indicator were calculated in order to standardize the different types of data to a common scale.
In order to calculate a school's overall global score, the calculated z-scores for each of the 10 indicators were then weighted using the assigned weights described earlier. U.S. News determined the weights based on our judgment of the relative importance of the ranking factors and in consultation with bibliometric experts.
The overall global score for each school was calculated by summing the school's weighted values for each indicator. The minimum score from the pool of 750 schools was then subtracted from each of the scores in order to make zero the lowest possible score.
The scores were then rescaled by multiplying the ratio between the overall performance of each university and the highest-performing university by 100. This forced the scores to fall on a 0-100 scale, with the highest-performing school earning an overall global score of 100.
The top 500 universities out of the 750 were then numerically ranked in descending order from 1 to 500 based on their weighted, rescaled overall global score. Each school's overall global score was rounded to one decimal place in order to increase variance between scores and to minimize the occurrence of ties.
In addition, the 750 universities received a numerical rank for each of the 10 ranking indicators, such as publications, total citations and global academic reputation, based on their z-score for that indicator. The highest-scoring university for each of the 10 indicators received a rank of 1 and the lowest-scoring university received a rank of 750. Ties were allowed.
As noted earlier, the numerical ranks for each of the 10 indicators are published on usnews.com for each school ranked in the top 500. This means that there are some schools in the top 500 rankings that have ranking indicators with numerical ranks in the 501 to 750 range. The numerical ranks published for each ranking indicator are to be used to determine the relative position of each school in that indicator. The numerical indicator ranks were not used to calculate the overall global score.
Bibliometric Indicators
The bibliometric indicators used in our ranking analysis are based on data from the Web of ScienceTM for the five-year period from 2008 to 2012. The Web of ScienceTM is a Web-based research platform that covers more than 12,000 of the most influential and authoritative scholarly journals worldwide in the sciences, social sciences, and arts and humanities.
The bibliometric indicators used in our ranking analysis are based on data from the Web of ScienceTM for the five-year period from 2008 to 2012. The Web of ScienceTM is a Web-based research platform that covers more than 12,000 of the most influential and authoritative scholarly journals worldwide in the sciences, social sciences, and arts and humanities.
Publications (12.5 percent): This is a measure of the overall research productivity of a university, based on the total number of scholarly papers (reviews, articles and notes) that contain affiliations to a university and are published in high-quality, impactful journals. This indicator is closely linked to the size of the university. It is also influenced by the discipline focus of the university, as some disciplines, particularly medicine, publish more than others.
Normalized citation impact (10 percent): The total number of citations per paper represents the overall impact of the research of the university and is independent of the size or age of the university; the value is normalized to overcome differences in research area, the publication year of the paper and publication type.
NCI is considered one of the core measures of research performance and is used by various research evaluation bodies globally. The subject fields used in the analysis came from Thomson Reuters' InCitesTM product, which helps institutions evaluate research output, performance and trends; understand the scope of an organization’s scholarly contributions; and articulate outcomes to inform research priorities. InCites utilizes the content and citation indicators found in the Web of ScienceTM.
Total citations (10 percent): This indicator measures how influential the university has been on the global research community. It is determined by multiplying the publications ranking factor by the normalized citation impact factor. Total citations have been normalized to overcome differences in research area, publication year of the paper and publication type.
NCI is considered one of the core measures of research performance and is used by various research evaluation bodies globally. The subject fields used in the analysis came from Thomson Reuters' InCitesTM product, which helps institutions evaluate research output, performance and trends; understand the scope of an organization’s scholarly contributions; and articulate outcomes to inform research priorities. InCites utilizes the content and citation indicators found in the Web of ScienceTM.
Total citations (10 percent): This indicator measures how influential the university has been on the global research community. It is determined by multiplying the publications ranking factor by the normalized citation impact factor. Total citations have been normalized to overcome differences in research area, publication year of the paper and publication type.
Number of highly cited papers (12.5 percent): This indicator reflects the number of papers that have been assigned as being in the top 10 percent of the most highly cited papers in the world for their respective fields. Each paper is given a percentile score that represents where it falls, in terms of citation rank, compared with similar papers (same publication year, subject and document type). As the number of highly cited papers is dependent on the size of the university, the indicator can be considered a robust indication of how much excellent research the university produces.
Percentage of highly cited papers (10 percent): This indicator is the percentage of a university's total papers that are in the top 10 percent of the most highly cited papers in the world (per field and publication year). It is a measure of the amount of excellent research produced by the university and is independent of the university's size.
International collaboration (10 percent): This indicator is the proportion of the institution's total papers that contain international co-authors divided by the proportion of internationally co-authored papers for the country that the university is in. It shows how international the research papers are compared with the country in which the institution is based. International collaborative papers are considered an indicator of quality, as only the best research will be able to attract international collaborators.
Percentage of highly cited papers (10 percent): This indicator is the percentage of a university's total papers that are in the top 10 percent of the most highly cited papers in the world (per field and publication year). It is a measure of the amount of excellent research produced by the university and is independent of the university's size.
International collaboration (10 percent): This indicator is the proportion of the institution's total papers that contain international co-authors divided by the proportion of internationally co-authored papers for the country that the university is in. It shows how international the research papers are compared with the country in which the institution is based. International collaborative papers are considered an indicator of quality, as only the best research will be able to attract international collaborators.
School-Level Indicators
Publicly available data sources were used to create the school-level indicators.
Number of Ph.D.s awarded (5 percent): This indicator reflects the total number of doctoral degrees awarded in 2012. The number of doctorates awarded can be considered an alternative indicator of research output and is linked to volume.
Number of Ph.D.s awarded per academic staff member (5 percent): This is the number of Ph.D.s awarded per the number of academic faculty members for the same year. This is a size-independent measure of the education environment at the university.
How the Overall Global Scores and Numerical Rankings Were Calculated
To arrive at a school's rank, the overall global scores were calculated using a combination of the weights and z-scores for each of the 10 indicators used in the rankings. In statistics, a z-score is a standardized score that indicates how many standard deviations a data point is from the mean of that variable. This transformation of the data is essential when combining diverse information into a single ranking because it allows for fair comparisons between the different types of data.
Publicly available data sources were used to create the school-level indicators.
Number of Ph.D.s awarded (5 percent): This indicator reflects the total number of doctoral degrees awarded in 2012. The number of doctorates awarded can be considered an alternative indicator of research output and is linked to volume.
Number of Ph.D.s awarded per academic staff member (5 percent): This is the number of Ph.D.s awarded per the number of academic faculty members for the same year. This is a size-independent measure of the education environment at the university.
How the Overall Global Scores and Numerical Rankings Were Calculated
To arrive at a school's rank, the overall global scores were calculated using a combination of the weights and z-scores for each of the 10 indicators used in the rankings. In statistics, a z-score is a standardized score that indicates how many standard deviations a data point is from the mean of that variable. This transformation of the data is essential when combining diverse information into a single ranking because it allows for fair comparisons between the different types of data.
Several of the indicators were highly skewed, so the logs of the original values were used. The indicators that used logs were:
- Publications
- Total citations
- Number of highly cited papers
- Number of Ph.D.s awarded
- Global research reputation
- Regional research reputation
This log manipulation rescaled the data and allowed for a more normalized and uniform spread across each of the indicators. After these six indicators were normalized, the z-scores for each indicator were calculated in order to standardize the different types of data to a common scale.
In order to calculate a school's overall global score, the calculated z-scores for each of the 10 indicators were then weighted using the assigned weights described earlier. U.S. News determined the weights based on our judgment of the relative importance of the ranking factors and in consultation with bibliometric experts.
The overall global score for each school was calculated by summing the school's weighted values for each indicator. The minimum score from the pool of 750 schools was then subtracted from each of the scores in order to make zero the lowest possible score.
The scores were then rescaled by multiplying the ratio between the overall performance of each university and the highest-performing university by 100. This forced the scores to fall on a 0-100 scale, with the highest-performing school earning an overall global score of 100.
The top 500 universities out of the 750 were then numerically ranked in descending order from 1 to 500 based on their weighted, rescaled overall global score. Each school's overall global score was rounded to one decimal place in order to increase variance between scores and to minimize the occurrence of ties.
In addition, the 750 universities received a numerical rank for each of the 10 ranking indicators, such as publications, total citations and global academic reputation, based on their z-score for that indicator. The highest-scoring university for each of the 10 indicators received a rank of 1 and the lowest-scoring university received a rank of 750. Ties were allowed.
As noted earlier, the numerical ranks for each of the 10 indicators are published on usnews.com for each school ranked in the top 500. This means that there are some schools in the top 500 rankings that have ranking indicators with numerical ranks in the 501 to 750 range. The numerical ranks published for each ranking indicator are to be used to determine the relative position of each school in that indicator. The numerical indicator ranks were not used to calculate the overall global score.
Data Collection and Missing Data
The data and metrics used in the ranking were provided by Thomson Reuters InCitesTM research analytics solutions. The bibliometric data were based upon the Web of ScienceTM.
Publications are limited to those published between 2008 and 2012. However, the citations to those papers come from all publications up to the most recent data available. For the 2015 edition of the U.S. News Best Global Universities, published in 2014, this cutoff was around April 2014. It is necessary to use a slightly older window of publication to allow for citations to accumulate and provide statistically relevant results.
The subject fields used in the analysis came from Thomson Reuters' InCitesTM schema and did not include arts and humanities journals, and therefore they are excluded for the citation-based indicators; but articles from arts and humanities journals were included in the papers count used in the publications indicator. Arts and humanities journals accumulate few citations and citation analysis is less robust; therefore, the deliberate exclusion of arts and humanities improves the robustness of the results.
Publications are limited to those published between 2008 and 2012. However, the citations to those papers come from all publications up to the most recent data available. For the 2015 edition of the U.S. News Best Global Universities, published in 2014, this cutoff was around April 2014. It is necessary to use a slightly older window of publication to allow for citations to accumulate and provide statistically relevant results.
The subject fields used in the analysis came from Thomson Reuters' InCitesTM schema and did not include arts and humanities journals, and therefore they are excluded for the citation-based indicators; but articles from arts and humanities journals were included in the papers count used in the publications indicator. Arts and humanities journals accumulate few citations and citation analysis is less robust; therefore, the deliberate exclusion of arts and humanities improves the robustness of the results.
When data were not available, such as Ph.D.s awarded, a z-score of zero was used so as to neither reward nor penalize the university (i.e., it is treated as an average of all the other universities).
When the value is zero it is not possible to calculate the log value; therefore, a substitute is used. The substitute is one-tenth of the minimum value of all other institutions. There were no missing data in the bibliometric or reputation indicators.
University Rankings by Region
After the overall top 500 rankings were calculated, U.S. News then produced additional rankings. The U.S. News Best Global Universities rankings by region show the top institutions in four regions with a large number of globally ranked schools. Those regions are Asia, Australia/New Zealand, Europe and Latin America. To determine which countries are in which region, we used the United Nations definition of geographical regions.
When the value is zero it is not possible to calculate the log value; therefore, a substitute is used. The substitute is one-tenth of the minimum value of all other institutions. There were no missing data in the bibliometric or reputation indicators.
University Rankings by Region
After the overall top 500 rankings were calculated, U.S. News then produced additional rankings. The U.S. News Best Global Universities rankings by region show the top institutions in four regions with a large number of globally ranked schools. Those regions are Asia, Australia/New Zealand, Europe and Latin America. To determine which countries are in which region, we used the United Nations definition of geographical regions.
The methodology for the region rankings is based entirely on how a school ranked in the overall Best Global Universities rankings covering the top 500 schools worldwide. Universities are numerically ranked in their region based on their position in the overall Best Global Universities rankings.
For example, in Europe, the highest-ranked university in the overall top 500 rankings is the United Kingdom's University of Oxford, at No. 5 globally, which also makes the school No. 1 in Europe. The second highest-ranked university in Europe is the U.K.'s University of Cambridge, which is ranked No. 6 globally, making it No. 2 in Europe.
University Rankings by Country
The U.S. News Best Global Universities rankings by country show the top institutions in 11 countries with a large number of globally ranked schools. Those countries are Canada, China, France, Germany, Italy, Japan, the Netherlands, South Korea, Spain, Sweden and the United Kingdom.
University Rankings by Country
The U.S. News Best Global Universities rankings by country show the top institutions in 11 countries with a large number of globally ranked schools. Those countries are Canada, China, France, Germany, Italy, Japan, the Netherlands, South Korea, Spain, Sweden and the United Kingdom.
The methodology for the country rankings is based entirely on how a school ranked in the overall Best Global Universities rankings covering the top 500 schools worldwide. Universities are numerically ranked in their country based on their position in the overall Best Global Universities rankings.
For example, in Canada, the highest-ranked university in the overall top 500 rankings is the University of Toronto, at No. 14 globally. That means it is also ranked No. 1 in the Best Global Universities in Canada rankings. The second highest-ranked university in Canada in the overall rankings is the University of British Columbia, ranked at No. 30 globally, which means it's ranked No. 2 in Canada.
No comments:
Post a Comment