What Gets Measured in Education
What Gets Measured in Education
The world over, the performance of colleges is under fire. It’s about time that happened, but there should also be serious concerns about the new report cards that are being fashioned for tertiary educational institutions.
The Obama Administration in the U.S., for instance, plans to create a new performance-based rating system with teeth. In future, it says, resources will flow only where tangible student-focused outcomes justify their deployment. Those outcomes will be, most likely, improved retention and graduation rates; fewer wasted credits; lower student debt-burdens; easier access to financial support; greater efficiency estimated by linking progress to degrees and demonstrations of competency, not to credit hours or seat times; more students hired within a reasonable period after graduation; higher salary levels for them; and so on.
Are these useful measures? Of course. Will tracking them prove helpful to college managements? Of course. Will knowing them be relevant to students and families? Of course.
But these are not measures of educational performance; these measure only the efficiency of the educational process. Think, for a moment, of a college as if it were a factory, a pipeline that takes in raw materials and puts them through a structured series of steps that leads to the creation of “finished products,” namely well-educated students. The measurements under discussion are yardsticks of the pipeline’s asset utilization and process efficiency levels. If we improve them, the “factory” will run better.
However, if colleges use only these metrics to evaluate their performance, they will continue to repeat past errors. For, they will be measuring virtually everything except the one thing that matters most: Student learning.
Nonsense, will be the predictable rebuttal; colleges already measure learning. What do you think grades indicate? What do you think degrees stand for? What do you think the Latin on a diploma signals?
Even if you believe that colleges grade, certify, and award degrees accurately, there are grave limitations to the way they do it. Their measurements primarily reward discipline-based knowledge — not the capabilities in critical thinking, analytic reasoning, communication skills, and interpersonal effectiveness that employers most care about and that are essential for students to succeed as adults in the real world.
Research shows that there are links between traditional academic performance and economic status. For instance, students with the advantages that prepare them to test well at one level tend to test well at other levels too. So the fact that students are performing well according to standard measures may have little, if anything, to do with a college’s learning-related performance. It may just have a great admissions office and a powerful brand, taking in talented kids through the front door and not messing them up.
Meanwhile, two great ironies are unfolding. One, while the accuracy of traditional grading stagnates, the ability to carry out true learning-related assessments has advanced with lightning speed. Improvements in the U.S. Collegiate Learning Assessment; the skills-and-employability assessment instruments pioneered by organizations such as Aspiring Minds in India; the algorithms used to track learning in online video games; the analytics that underlie the learning experiences offered by massive open online courses; and US Education Testing Services’ new proficiency profiles and skills instruments are all changing what assessments can lead to. (That’s a topic I will revisit in my next post.)
Two, this is also a time when corporations and executives can help create the outcomes they desire as long as they don’t focus only on helping colleges to boost process efficiency or re-shape curriculums. The corporate world knows a lot about how to evaluate the kinds of learning that matter to it. It’s time business shared that expertise with colleges, and joined them in efforts to build novel tools that will help measure students’ real learning performances.
No comments:
Post a Comment