Toward a Culture of Continuous Assessment in Online Distance Learning

Toward a Culture of Continuous Assessment in Online Distance Learning
President Susan Aldridge of University of Maryland University College (UMUC)delivered this keynote address, titled “Toward a Culture of Continuous Assessment in Online Distance Learning” and delivered in the China Central Radio and TV University 30th Anniversary Celebration, Beijing, China.

The old Chinese proverb tells us that if we are planning for a lifetime, we must
educate our people. Certainly excellent advice in today’s highly interconnected
global economy…where we are far more likely to work with our heads than with
our hands. Even factory workers must operate sophisticated equipment that
requires a great deal of critical thinking and technical know-how. That said,
education is quickly becoming the engine of opportunity…and intellectual capital,
the common currency of trade.

It is an economics of abundance, rather than scarcity, because unlike other
resources, knowledge continues to reproduce and grow, especially when it is
shared and applied. And given our common need to create and disseminate
knowledge in sustaining our competitive edge…particularly during times of global
financial duress…we have become increasingly dependent on technology as the
medium of both innovation and exchange.

To be sure, it offers unlimited opportunities for scholarship, at a time when the
worldwide demand for advanced education far exceeds our current capacity for
delivering it.

Not surprisingly then, colleges and universities in every corner of the globe…
including such publicly funded mega-institutions as China Central Radio and TV University and my own University of Maryland University College are
incorporating distance learning in all of its many formats to teach beyond the faceto-
face classroom.

And as we provide broader access to quality education, universities such as ours
are fueling a global realignment of the attitudes and principles, norms and practices
that have traditionally driven the academic enterprise….giving rise worldwide to an
entirely new culture of learning.

This new culture values higher education as essential to personal and professional
enrichment…global citizenship and social inclusion. Therefore, we have come to
view it as a necessary investment, rather than a discretionary expenditure…a
lifelong pursuit, rather than a diploma-driven activity.

So by taking full advantage of the many available distance delivery technologies,
educators everywhere may now transcend language barriers, geographic
boundaries, uneven abilities, and cultural divides…while also overriding high fuel
costs, disparate economic resources, and national security issues.

Consequently, we are paving the way for collaborative communication and
problem-solving that is multi-generational, multi-cultural, and multidisciplinary…
among individuals, communities, institutions, and even nations.

Yet while we have succeeded in using technology to automate the learning process,
we are still discovering the many unique learning opportunities it affords. This is
especially true of online distance education.

Certainly, as it evolves, online — or e-learning — provides far more than a
platform from which to teach. It offers an extraordinary tool through which to
learn…enabling universities to move beyond course management and delivery, to
engage students and faculty in active, collaborative, and authentic knowledge
creation.

For all of its many opportunities, however, e-learning presents more than its share
of challenges. In fact, nothing sparks a heated debate among educators quite like
distance learning…particularly among the non-believers.

Some say it is too expensive for the average institution to sustain…some contend
that only the most motivated and disciplined students will actually succeed at it.
Still others see it as a money-making venture that sacrifices the purpose of
knowledge for the limited utility of profit.

And even among the true believers, the term distance learning evokes nearly as
many definitions as there are possible applications. Moreover, the demands of a
knowledge-rich, technology-enhanced learning environment require colleges and
universities to make significant and ongoing capital investments….in addition to
the traditional research, labor, and facility expenses associated with delivering
higher education. All of which becomes increasingly difficult to justify in these
tough economic times.

Regardless, however, of where we come out in the debate, the bottom line remains
the same. For distance learning to fulfill its promise as an open and cost-effective
gateway to lifelong learning, it must enhance…and even improve upon…the
traditional face-to-face learning experience.

Thus, given both rapid changes in technology and scarce resources for developing
its potential, we will need to balance the desire to expand and upgrade existing
delivery systems with the need to ensure both quality academic products and
positive student outcomes.

Collaboration is one way to generate first-rate academic products and overall costefficiencies.
Indeed, a growing number of university consortiums have now joined
forces to recycle course content….unbundle and share student support services
within common domains…promote the use of open source software…design highly
scalable delivery systems…and develop mutually beneficial partnerships and
demonstration projects.

But more importantly, in making wise investments, while serving the public
interest, online educators must be prepared to measure the overall impact of
distance delivery…not only on learning…but also on institutional performance.
With this in mind, my own university — UMUC — uses a highly effective, datadriven,
360-degree quality assurance process…which establishes coherent
standards and incorporates reliable benchmarks.

UMUC is by no means a conventional university, having opened its doors in 1947
as a truly visionary experiment in adult education…founded in the belief that
advanced learning should be available to anyone, anywhere, at anytime.

Yet today, as the largest public university in the United States, it is one of
academia’s greatest success stories…serving more than 90,000 students in 28
countries around the world. And as an open university, UMUC has always been at
the forefront of distance education…bringing quality, affordable academic opportunities to the people, at times and in locations that are both convenient and
accessible.

That said, e-learning has become vital to our success as a global university.
Having been inaugurated in 1994…with little more than 100 enrollments in only a
handful of courses…our online campus is now one of the largest and fastest
growing in the world. With some 189,000 enrollments in close to 700 distinct
courses…in more than 100 undergraduate and graduate programs offered fully
online.

It has also garnered numerous awards, including the 2006 Sloan Consortium
Excellence in Faculty Development for Online Teaching award…and the
prestigious 2004 Sloan-C Award for Excellence in Institution-Wide Asynchronous
Learning Network Programming.

UMUC’s Graduate School of Management and Technology is also the only such
academic entity in the United States to offer a master’s degree in Distance
Education…provided in partnership with the Carl von Ossietzky University of
Oldenburg, Germany. And as one of the university’s signature programs, it
recently received EFMD CEL accreditation…the highest international standard for
technology-enhanced learning programs in the field of management education.

So in the interest of sustaining its hard-earned reputation for world-class online
learning, UMUC has created a culture of continuous assessment…with which to
measure both student learning outcomes and institutional efficiencies. As such, we
have built this process around certain guiding principles we believe are essential to
effective teaching and learning in any setting.

At UMUC, we always put our students first…by providing both the programs and
the services they need to meet their academic goals. In doing so, we foster a
learning community in which highly competent professionals are collaboratively
engaged in developing, teaching, and refining the instructional program. Likewise,
students have ample opportunities to demonstrate acquired knowledge…and
receive prompt feedback around their efforts.

UMUC also organizes its academic programs around sound and substantive
curricula…with consistent learning objectives and clearly articulated learning
outcomes. And to ensure that they are met, the university assesses every aspect of
the student’s learning experience…from acquired knowledge to applied skills…selfconcept
to worldview…learning styles to learning attitudes.

With these principles in mind, UMUC has developed its 360-degree quality
assurance process to measure its progress in five core areas: faculty teaching
effectiveness…curriculum and program development…service provision and
operational efficiency…institutional integrity…and, of course the most important
area of all, student learning outcomes.

Although the formal university accreditation process in my country has
traditionally dealt with curriculum, faculty qualifications, library holdings, and
physical facilities…distance learning has refocused the spotlight on student
learning outcomes. In response, UMUC has developed one of the most aggressive
and broadly-based student learning outcomes assessment initiatives in the United
States.

It begins with the university’s Institutional Plan for the Assessment of Student
Learning Outcomes…which creates a framework for all learning evaluation
activities by identifying key units of responsibility…designating core learning
areas…and creating timelines for implementation.

The plan also outlines an evaluation process…using a variety of proven assessment
tools and “value-added” reviews to measure student learning in a variety of Core
Learning Areas…including written communication, technology fluency,
information literacy, scientific and quantitative literacy, and critical thinking.
Needless to say, this process not only yields a tremendous amount of
information…it also suggests new ways of using it to support investments in
academic program development.

In addition to mining data, UMUC has created a variety of online courses and
tutorials that support learning outcomes, by familiarizing students with…among
other things…the university’s online learning management system…its institutional
culture…and its research protocols.

We also monitor student retention to graduation, as well as individual student
progress, by establishing a variety of information systems that alert academic
advisors to any number of potential barriers to academic success. Barriers such as
lagging performance in mathematics or writing…poor study habits…and financial
or other personal challenges. Once these issues are identified, advisors work with
students to develop remediation plans, which include appropriate student support
services, as needed.

Technology has made it possible for UMUC to recruit an exceptional faculty. As
a global university dedicated to meeting the lifelong learning needs of working
adults, UMUC hires its online instructors from among the growing number of
international scholar-practitioners…who as renowned academics and recognized
experts in their fields…have taught and worked all over the world.

But having knowledge and experience is one thing…teaching effectively is quite
another…especially from a distance, to students with diverse learning styles,
computer skills, and academic backgrounds. So to reinforce the quality of their
instruction, while also measuring its impact, we are harnessing the power of digital
technology to provide our faculty members with professional development and
peer mentoring that is both continuous and self-paced.

Even more important, it enables distance instructors to experience online education
first-hand…an important first step in understanding what it actually means to be a
distance learner.

UMUC s award-winning Center for Teaching and Learning…or CTL…offers an
interactive e-learning environment, with an abundance of faculty resources in a
variety of formats…from written articles to self-paced tutorials and podcasts.

Before teaching online, UMUC faculty members must first complete an intensive
and experiential five-week preparation course, conducted year-round and entirely
online. They may also earn teaching credentials through a series of structured
distance training workshops in six subject categories…each of which is critical to
successful teaching and learning in any environment.

In addition, the CTL provides a unique Faculty Media Lab, which brings faculty
members, course designers, and distance education coordinators together online to
create innovative e-learning enhancements. Moreover, this lab publishes “best
practice” case studies, for distance educators at other academic institutions to use
in their own instructional and resource development efforts.

The CTL facilities a vigorous peer mentoring program, as well…which is designed
to support the university’s continuing focus on collaborative teaching and learning.
This program links new and emerging faculty members…in even the most remote
locations…with their more experienced peers to share proven teaching strategies;
explore diverse teaching philosophies; and develop new teaching methodologies.

Veteran distance educators also observe less-seasoned colleagues by visiting their
online classrooms at regular intervals. These visits are structured to evaluate the learning environment across a number of variables. For example, we might check
to see if the instructor adheres to stated learning objectives…or uses student
learning assessments properly.

And in keeping with its commitment to quality teaching, UMUC’s academic
leadership team has embedded a variety of ongoing quality assessment tools into
its faculty evaluation process. For instance, deans and department heads maintain
a constant watch on course grading patterns to determine how closely a given
instructor follows university-wide grading norms.

Likewise, when evaluating faculty performance, the university makes use of
student course evaluations to measure the frequency with which student learning
outcomes and expectations have been met. We also track faculty participation in
professional development…both internally and externally…as a basis for promotion
and pay.

Of course, even good teaching cannot make up for inferior academic programs.
Which is why UMUC has made curriculum development one its highest priorities.
That said, our academic leadership team regularly scans the marketplace for
workforce-critical learning opportunities… and polls our students around their
professional aspirations and lifelong learning needs.

Consequently, UMUC now offers 130 cutting-edge degree and certificate
programs. And to guarantee consistency across all sections of a given course, we
have adopted an approach known as mass-customization. This strategy allows
large universities such as ours to adapt their coursework to individual learning
profiles…without incurring additional costs. Here is how it works.

For every course, there is one common curriculum, along with standard learning
objectives, shared assessment tools, and linkable course units. We then encourage
instructors to customize their teaching strategies in accordance with the unique
learning needs of their students.

And to measure curriculum effectiveness, UMUC periodically examines what we
refer to as key outcomes courses…by collecting and analyzing student and faculty
performance data at various time-points throughout the semester…around such
variables as student scores on so-called “high stakes” final exams that count for as
much as 30 percent of the total grade.

UMUC also periodically reviews its courses and programs…weeding out those
with lagging enrollments…and revising others that need updating. And although overhauls like this are time-consuming, they sharpen the university’s competitive
edge…by focusing precious resources where they are needed most…on the
academic enterprise itself.

Certainly, in putting its students first, UMUC places student support on par with
academic excellence.

While most universities do an admirable job of serving their students during the
day, nearly half of all American college students are busy, working adults, for
whom the school day begins once the sun sets. What’s more, because these
students are balancing the responsibilities of life with the demands of lifelong
learning, they need a great deal of help along the way.

So with that expectation in mind, UMUC has created an exceptional complement
of student support services…and seen to it that every service provided on campus is
also furnished online…with help desks operating around the clock.

And today, UMUC students now have virtually instantaneous online access to
everything from tutoring, mentoring, and writing assistance…to honor societies and
professional organizations, career counseling, class registration, and financial
assistance.

We are especially proud of our Information and Library Services division, given
the number of students UMUC enrolls in remote locations…where good libraries
and books stores are few and far between. Yet as isolated as they may be, these
learners are only a mouse click away from an abundance of information
resources…along with automated forms to request postal delivery for hardcopy
materials…whenever and wherever needed.

This digital library system maintains a vast webliography of online library
resources, including more than 100 databases…many of which furnish full-text
versions of articles from among some 74,000 professional journals.

Highly trained reference librarians are also available 24 hours a day, seven days a
week, to assist students by e-mail; online chat rooms; or telephone…while also
helping faculty members obtain copyright permissions and digitize selected books
and articles for classroom reading. And so that students know how to effectively
use both the library and the technology, UMUC provides a mandatory course on
library skills and information literacy.

Of course, high-quality support services make it much easier for us to recruit and
retain students…which is why UMUC consistently evaluates both their efficacy and
their efficiency. For example, the university regularly assesses its library and
information services by tracking the number of text requests received against the
time it takes to fully process them. It also monitors help desk response rates and
customer satisfaction.

In rounding out its 360-degree quality assurance process, UMUC participates in
external evaluations, including academic accreditation by a regional accrediting
body. This process has always been an important means of self-regulation and
peer review for colleges and universities in the United States…used to strengthen
and sustain both the quality and the integrity of their academic offerings.

As such, it serves as a seal of approval for the institution…thereby ensuring that its
students receive an appropriate educational experience, based upon a common set
of standards…whether on campus or online. Therefore, it must be renewed every
ten years, with five-year periodic reviews in between, thus ensuring that the
university remains attentive to its mission and goals, while at the same time
accepting responsibility for its own continuous improvement.

In preparing for this re-accreditation process, UMUC completes an exhaustive
Self-Study report…after an intense self-examination process, during which time a
number of internal subcommittees carefully evaluate every aspect of the
university’s day-to-day activities. From academic programs, student support
services, and distance education…to faculty qualifications, university governance,
and resource allocation.

Certainly, it is safe to say that distance learning is more than just a passing fad…but
rather a permanent…although rapidly evolving…dimension of today’s global higher
education market. Yet in maximizing its benefits and meeting its challenges, we
must carefully nurture its potential, by building effective partnerships and
sustaining true pipelines for innovation with other like-minded, public and private
institutions and organizations.

Perhaps the greatest of these challenges lies in our ability to cultivate an
international system of postsecondary education that actually balances excellence
with unfettered access. That will mean reaching beyond our own university walls
to collectively research, evaluate, and exchange promising practices with our
colleagues…in the interest of allocating our institutional resources more efficiently.

We will also need to discourage the mass production and export of e-learning
programs and products, as a profit-making venture…in addition to adopting
transnational quality standards that regulate what we teach, how we teach it, and
the means by which we assess what our students have learned.

And in the spirit of globalism, we should reach out to students and faculty alike
from around the world…giving them an opportunity to learn what they need
without ever leaving home. That said, we must look for ways to make our distance
education programs more affordable and accessible.

Needless to say, mutual cooperation and shared responsibility on this scale is by no
means an easy undertaking…given that most universities have always been
reluctant to share their turf with outsiders. Furthermore, we have seen how custom,
law, language, cost, and demand from one country to the next make it harder to
negotiate…much less enforce…quality standards of any kind.

Foreign currency controls and telecommunications oversight also create
complications…at times restricting distance education providers from investing in
or implementing more highly stable and accessible e-learning technologies.
Even more problematic, the criteria for professional qualifications vary greatly
throughout the world…with distance learners often facing transfer credit
obstacles…and countries sometimes refusing to recognize foreign educational
credentials.

The first step comes with developing and encouraging multinational distance
education collaboratives among universities and other learning
organizations…which clearly articulate a mutual purpose, along with consistent
standards of performance. We must also invoke our entrepreneurial spirit to
overcome such specific challenges as, for example, transferring academic credits
between universities and across national borders.

To be sure, a shift of this magnitude demands transformational leadership of the
highest order…with university presidents and provosts, willing and ready to
champion the cause, facilitate the coalitions, and jumpstart the technologies…even
in the face of economic challenge.

So we will need to remain diligent in our efforts to verify both the authenticity of
our theories and the effectiveness of our systems, by developing the appropriate
metrics and collecting the necessary data to analyze the actual impact of lifelong
learning from a distance on both our students and our institutions.

It is an extraordinary undertaking indeed…but one that promises an equally
extraordinary return on investment…especially in a world where the only constant
is change and the race for knowledge, more critical than ever.