It’s Time to Measure What Matters

Recent trends indicate a significant decline in the number of high school graduates choosing to attend college. In 2020, only 63% of high school graduates enrolled in college in the fall, a drop from 70% in 2016. This decline is part of a broader skepticism about the value of higher education, with less than 53% of Americans believing in its worth as of last year, down 11 percentage points since 2017.

The perception of the value of a college education has become increasingly mixed. Only 32% of Americans currently say college is worth the cost, and fewer than half of adults have a ‘great deal’ or ‘quite a lot’ of confidence in higher education. This skepticism is even reflected among the youth, with only 45% of 14- to 18-year-olds believing that education beyond high school is necessary. Moreover, nearly half of the teenagers planning on college think that education should take two years or less.

There are two fundamental elements in determining the value: quality and price. Price, we understand. But how much are we spending to get a college education?

In determining quality, the real stakeholders who are paying for a quality college education – parents, students, and employers – have a different take than colleges, universities, and the academic establishment in general.

Historically, the job of assuring quality in higher education has been the responsibility of accrediting agencies. Accreditation does focus heavily on process and compliance. In particular, accreditation standards often focus on the structure and governance of an institution, its resources, processes, and the existence of continuous improvement mechanisms. These organizations measure quality largely based on inputs: the enrollment numbers, faculty qualifications, library volumes, research facilities, and institutional endowments. In addition, they typically look at things like governance and administration, student support services, faculty qualifications, curriculum, and financial stability. This input-based approach suggests that the more resources an institution has and the more compliant they are with rules and regulation, the better the quality of education.

Colleges and universities spend countless hours ensuring that they can demonstrate to the accrediting bodies that they are meeting these input requirements. When they do, they are accredited with the big reward being their ability to partake of federal resources such as Pell grants, federal student loans, etc.

On the other hand, parents investing their hard-earned money in their children’s future could care less about whether or not the faculty was properly represented on the college’s landscaping committee; students wanting to learn give little thought to whether or not the college had reached a certain threshold of doctorly prepared faculty; and employers don’t count the number of volumes in a college’s library before deciding whether or not to hire one of their graduates.

Instead, they see quality as the knowledge, skills, and experiences gained by the students during the course of study. This includes critical thinking skills, organization, self-discipline, and the ability to take on complex tasks.

Having a faculty senate and involving faculty in decision-making is important in satisfying accreditation standards for governance and transparency, but it doesn’t tell us much about the quality of teaching or student learning. Similarly, having resources and facilities doesn’t necessarily mean they are being used effectively to enhance learning.

Today, more and more parents and students are turning away from college and employers are looking beyond a mere credential in making hiring decisions. As such, critics are challenging the traditional model of “quality assurance” highlighting its neglect of the primary purpose of higher education: to equip students with necessary skills and knowledge for personal and professional success. This criticism prompts a pivotal question: how should we define and measure the quality of higher education?

The answer lies in shifting our focus from inputs to outcomes. Quality in higher education shouldn’t be judged by the amount of money spent, the prestige of the faculty, or how many books the library holds. Instead, it should be determined by the outcomes it produces—knowledge and skills acquired by students, their success in the workforce, their contribution to their communities, and their pursuit of lifelong learning.

Several outcome-based metrics can be defined to measure these results:

1.     Employment Rates and Earnings: The relevance of education to the labor market can be gauged by measuring graduates’ ability to secure employment and their earning potential.

2.     Performance on Standardized and Licensure Exams: Objective measures of students’ knowledge and skills in their field of study.

3.     Major Field Tests (MFTs): These comprehensive assessments measure critical knowledge and understanding obtained in a major field of study, indicating students’ ability to apply concepts, theories, and methodologies.

4.     Student Satisfaction: While subjective, this measure can provide insight into the quality of teaching, support services, and overall educational experience.

5.     Long-term Success and Career Progression: Tracking graduates’ career trajectories can provide long-term insights into the value of the education provided by an institution.

6.     Lifelong Learning: Measuring graduates’ participation in further education and professional development can indicate the institution’s success in instilling a commitment to continuous learning.

7.     Civic Engagement: Measuring graduates’ participation in community service, political activities, and other forms of civic involvement can reflect the institution’s success in producing responsible citizens.

We purposely exclude the use of graduation rates as an outcome-based metric. Graduation rates are often used by academic institutions as a key performance indicator of their success. However, there are several reasons why this measure can be more self-serving than determinative.

Firstly, graduation rates can be manipulated in various ways. For instance, an institution might lower its academic standards to ensure that more students graduate, thereby artificially inflating the graduation rate. This practice, while boosting the institution’s statistics, does not necessarily reflect the quality of education provided or the readiness of the graduates for the workforce or further studies.

Secondly, graduation rates do not consider the varying lengths of time it takes students to complete their degrees. Some students may take longer due to part-time study, work commitments, or other personal circumstances. These students may be just as successful in their studies, but their longer completion times can negatively impact the institution’s graduation rate.

Thirdly, graduation rates do not consider the quality of the education received or the skills gained. A high graduation rate might simply mean that the institution is good at getting students to complete their degrees, but it doesn’t necessarily mean that those students have acquired the skills and knowledge they need to succeed in their careers or further studies.

Lastly, graduation rates do not reflect the diversity of student experiences. They do not account for the varying challenges and barriers faced by different student populations, such as first-generation students, low-income students, or students with disabilities. As such, they can mask disparities in educational outcomes among different student groups.

Thus, while graduation rates can provide some insight into an institution’s performance, they are not a reliable measure of quality. Other factors, such as student satisfaction, employment outcomes, and the development of essential soft skills, should also be considered to provide a more comprehensive and accurate picture of an institution’s effectiveness.

Outcome-based measures of quality present their own set of challenges, as not all learning outcomes are easily quantifiable or measurable. However, they offer a more direct and meaningful assessment of what higher education institutions achieve. In that regard, our colleges and universities should not be shy in marketing such achievements.

By reshaping our understanding of quality in higher education, state legislatures and governing boards need to hold institutions accountable for delivering their promise: to provide an education that enables students to succeed in a rapidly changing world. This transition from inputs to outcomes won’t be easy, but it is essential to the evolution and relevance of higher education in the 21st century.

It’s time to measure what matters – by Robin Capehart (substack.com)