[12] The ratings are described by the Department for Education as:[13] Institutions that do not enter the TEF or that do not meet the minimum quality threshold will not receive an award.
[14] Institutions with insufficient data for a full assessment but which meet the quality standards can receive an unrated provisional award.
The ratings are thus a measure of whether a university exceeds, meets or falls short of expectations based on the profile of students admitted and subjects taught.
[5][6][20] After the publication of the results, the acting director of the Russell Group said that "TEF does not measure absolute quality and we have raised concerns that the current approach to flags and benchmarking could have a significant unintended impact",[21] while the vice-chancellor of the University of Southampton, which was rated bronze, said "There is no logic in our result at all", and that he had "deep concerns about its subjective assessment, its lack of transparency, and with different benchmarks for each institution removing any sense of equity and equality of assessment".
Wonkhe further noted that "it seems perverse that an institution – in Bristol’s case – which was ‘notably’ below benchmark should receive a higher outcome than Liverpool for which the statement is softer" and that "for institutions with a similar data pattern to Bristol’s, such as Southampton (with two negative flags in the same categories, but which wasn’t upgraded to Silver) there could be some well-deserved anger.
He also noted that "whilst universities have been impressive at widening participation they have been less assiduous in combatting the impact of disadvantage after students enroll", and said that the TEF was with doing as it had "raised the profile of teaching" and "focused attention on things which need to be done better".
[26] At a conference held in late June, Imperial College London's Vice Provost, Simone Buitendijk, stated that the TEF was a 'godsend' for higher education.
The Guardian reported in May 2017 that a number of "world-renowned" universities were at risk of receiving a bronze rating; particularly London institutions, which normally have lower student satisfaction scores.
The Times Higher Education also reported in early June 2017 that the members of the Russell Group (two in London) were in danger of being rated bronze, while post-1992 universities were expected to do well.
However, universities also submitted additional written information to the TEF to clarify their institutional context: the head of King's College London said that he hoped this would raise the institution from a bronze to a silver rating, while SOAS noted that the financial cost of living in London meant that the student retention rate in the city was lower than the national average.
[9][30] Similarly, a "Mock TEF" carried out by the data analytics team at Times Higher Education in 2016 showed that while Russell Group institutions did well on absolute results, once results were adjusted for student intake only Cambridge, Durham, Birmingham, Exeter and Newcastle (in order of their ranking) were definitely rated as gold, although this did not include any adjustments that may be made for the qualitative submissions from institutions.