Karen W. Kershenstein - June 2002

Accreditors have certainly been talking about assessing success with respect to student achievement for a long time, almost 20 years now. They might use terms like "educational effectiveness" or "educational outcomes" or "student outcomes" instead of "success with respect to student achievement," but one thing is clear: they have been grappling with this issue for many years. ASPA’s Spring 2002 conference provided an opportunity to look at the issue from the perspective of three different kinds of accreditors - regional, national, and specialized - and to see the differences and commonalities among their approaches.

Certain patterns emerged in the panelists’ presentations, patterns that reinforced some of my own research into accreditors’ approaches to the assessment of student achievement. There were clear differences too, dictated more, in my opinion, by the types of institutions or programs they accredited than anything else.

The National Accrediting Agencies: The national accreditors’ approach to the assessment of success with respect to student achievement is quantitative. Their approach is dictated in large part by the Higher Education Act of 1965, as amended, which requires accreditors to have standards that effectively address the quality of the institution in terms of its "success with respect to student achievement in relation to the institution’s mission, including, as appropriate (emphasis added), consideration of course completion, State licensing examination, and job placement rates." The U.S. Department of Education has interpreted the words as appropriate to apply to agencies that accredit institutions that offer prebaccalaureate vocational education. Since this describes the vast majority of institutions accredited by the nationals, it’s obvious why the nationals have taken the approach they have: the Department requires it.

To their credit, at least some of the nationals were using quantitative standards long before the Department required them to do so. Now, however, all of them have a quantitative standard (or benchmark) for completion rates, job placement rates, and pass rates on licensing exams. Some of them have absolute standards, i.e., a specific rate that all institutions must meet, while others have relative standards, e.g., an institution’s rate must be within one standard deviation of the mean for similar institutions. Some require institutions to meet the standards for each program they offer. The Accrediting Council for Continuing Education and Training (ACCET), for example, sets benchmarks of 67 percent for completion and 70 percent for placement for each program an institution offers. An institution can lose ACCET approval of a program it offers if that program fails to achieve either benchmark; if circumstances warrant, ACCET will remove the institution’s accreditation for failure to reach benchmark in a program within a certain period of time.

Another aspect of the nationals’ approach is their verification requirements for any rates institutions must report to them. Some agencies require institutions to provide 100 percent verification of all outcomes data (some even require this verification annually). Some agencies require their evaluation teams to examine a sample of an institution’s outcomes data while on site to verify that what the institution reports is accurate. Whatever method they use, all of the nationals take verification of the outcomes reported by their institutions very seriously.

To summarize, the nationals use a two-fold approach to the assessment of success with respect to student achievement. First, they use completion, licensing exam, and job placement rates for student outcomes. They may use other measures, but at least they use those three. Second, they require institutions to develop an institution-wide system to assess student outcomes, as measured by those rates, and they look to see if an institution’s assessment program is appropriate, in terms of relevance to the institution’s mission and effectiveness in assessing the three outcomes measures. They verify that the data reported by the institution are accurate, and they take follow-up action, as appropriate, if the institution fails to document that its outcomes meet the standards.

The Regional Accrediting Agencies:The regional accrediting agencies face certain challenges in assessing success with respect to student achievement that neither the nationals nor the specialized accreditors do. Generally, their institutions are larger, more complex, and more diverse than those accredited by the nationals, and they typically offer not just one or two programs but a host of different programs.

Given this complexity and diversity, the regionals place considerable emphasis on institutions developing effective institution-wide systems that assess and document student achievement. One of the criticisms of this approach is that while it is all right to require an institution to assess and document the achievement of its students, it does not represent a standard for student achievement. There is no level of expectation, no baseline, set by the accreditor regarding what is an acceptable level of student achievement and what is not.

While there may appear to be some validity to this criticism, one needs to look further at what the regionals require of their institutions and not jump to that conclusion. North Central, for example, requires its institutions to provide a "pattern of evidence" that they are achieving their educational purposes. While North Central does not dictate the specific pattern of evidence an institution must use, its expectations for student achievement are quite clear; it states that an appropriate assessment program will document (emphasis provided by North Central) the following:

• Proficiency in skills and competencies essential for all college-educated adults,
• Completion of an identifiable and coherent undergraduate level general education component, and
• Mastery of the level of knowledge appropriate to the degree attained.

Another common feature of the regionals’ approach to the assessment of success with respect to student achievement is its focus on general education. This is evident in the North Central statement above, but it is also evident in the following Middle States’ statement:

The Specialized Accrediting Agencies: There was a time when most specialized accreditors’ curriculum standards looked pretty much like a list of required subjects and courses and the required hours of instruction in each. While vestiges of this approach can still be found in a few specialized accreditors’ standards, what is far more likely today is a set of competencies students are expected to achieve before they graduate. The Commission on Dental Accreditation, for example, says in its standards, "At a minimum, graduates must be competent in providing oral health care within the scope of general dentistry..."and then lists the specific competencies graduates of dental schools must possess. The preamble to the standards for occupational therapy begins with the statement, "A contemporary entry-level occupational therapist must..."and the standards then define very specific knowledge, skills, and abilities graduates must possess. The introduction to the educational standards for interior design states, "The quality of student performance demonstrates learning outcomes and is the major factor that determines...whether the educational standards are met." The standards that follow are expressed in terms of required student competencies, e.g., "Students understand and apply the knowledge, skills, processes, and theories of interior design."

The standards for optometric education provide yet another example. There is an overarching requirement that the purpose of the professional optometric degree program must be to produce graduates with the levels of knowledge, skills, and ethical values required for entry-level practice of optometry. There is also a specific set of outcomes that all programs must achieve, each expressed in the competency-based language of "Graduates must be able to..."

While adequately defining competencies for entry into a field or profession is a challenge for the specialized accreditation community, key to this competency-based approach is the involvement of both educators and practitioners in the development of the competency-based standards. This ensures not just that the standards are current, but also that there is an informed discussion between and among educators and practitioners as to what a student must know and be able to do in order to function at an acceptable level upon entry into the profession.

There is a second component of the specialized accreditors’ approach to the assessment of success with respect to student achievement. It is the requirement that each program must have in place a system for reviewing its success with respect to student achievement and to document, using that system, that it is in fact successful with respect to student achievement.

All in all, the specialized accreditors’ approach is to define, to the extent possible, the competencies they expect graduates of accredited programs to have achieved and to require programs to demonstrate that their students have achieved those competencies in order to be accredited. This means that each program must have in place an ongoing assessment program that examines how well it is achieving its own objectives and is producing graduates who have acquired the requisite competencies. A program that either doesn’t produce graduates who have achieved those competencies or doesn’t have in place an effective assessment program that demonstrates its graduates have achieved the requisite competencies simply won’t be accredited.

Conclusions: What emerges from this examination of the practices of national, regional, and specialized accreditors with respect to the assessment of success with respect to student achievement - and what we heard from the panelists at the ASPA conference - is somewhat different approaches to the issue by each of the sectors yet a common thread. The differences, it seems to me, are necessary because each sector is dealing with a very different sector of postsecondary education. The nationals are dealing with a sector of postsecondary education for which the U.S. Department of Education has dictated a quantitative approach to assessing student outcomes, namely the use of completion rates, job placement rates, and pass rates on licensing exams. So the nationals’ approach is grounded in the use of these rates. The regionals are dealing with a different sector of postsecondary education: the majority of the degree-granting institutions in this country. That is a very diverse and complex sector that offers many different types of degree programs, which makes the establishment of a single outcomes measure or a few measures not just difficult but, more importantly, of questionable use. So they have chosen to focus on one particular area to define some rather specific student outcomes: general education, which is common to all undergraduate degrees. The specialized accreditors, as their name implies, focus primarily on programs that prepare students for a particular field or profession. Whatever that field or profession is, they have examined in depth what competencies are essential for entry into it, and they base their assessment of student outcomes on those competencies. These are all very different approaches, it seems to me, but all quite appropriate to the sector they are examining.

The commonalities, it seems to me, are equally important. What are they? They are the requirement that the entities they accredit must have in place an effective assessment program that focuses on student outcomes, the requirement that the entities they accredit must provide credible (verifiable) evidence for any claims they make with respect to student achievement, and the policy of refusing to accredit an entity if that entity (a) does not have appropriate student outcomes in light of its own objectives and commonly accepted norms and (b) does not have in place a comprehensive assessment program that can and does document that the entity is successful with respect to student achievement.

What seems clear to me is that the national, regional, and specialized accreditors, who have been dealing with the issue of assessing success with respect to student achievement for several years now, have all developed some very practical systems that work reasonably well for their particular sector of accreditation. Could those systems be more effective? Certainly! But the systems that are in place now are working. More importantly, they are changing the culture of accreditation, and as accreditors gain more experience in using those systems, the systems themselves will become more effective. As one of the panelists at the ASPA Spring 2002 conference said, "We learned a lot by making student outcomes the centerpiece of our accreditation process, and we got better as we gained experience. But for us, there’s no turning back. The focus on student outcomes is where we belong, and we aren’t turning back!"

Comments or questions about the issues discussed in this article may be sent to Karen W. Kershenstein, KWK Enterprises, 11842 Clara Way, Fairfax Station, Virginia 22039, e-mail: karen@kershenstein.com, fax: 703-250-5065.