AQIP Category One
HELPING STUDENTS LEARN

Context for Analysis

1C1 Common Student Learning Objectives

The University of Saint Mary (USM) is informed by a liberal arts tradition which holds that a critical intelligence, depth and breadth of learning, and liberating aspects of the humanities and sciences are more important than ever in the diverse, interdependent, global society of the 21st century. Thus, Saint Mary's concept of applied liberal arts embraces the rich history and meaning of the liberal arts while intentionally connecting learning to the real world in concrete and practical ways. In this way, the liberal arts combine with active, experiential learning and career preparation so that students may use talents, knowledge, competencies, and energies to make a positive contribution in the world.

The Applied Liberal Arts curriculum identifies Areas of Investigation (AI) which state USM's Learning Goals for the 21 st century. Through discipline-specific requirements, students investigate:

•  the human imagination, expression in literature and the arts, and other artifacts of cultures
•  inductive and deductive reasoning to model the natural, social, and technical world especially through but not limited to mathematics, the natural sciences, the behavioral sciences, information systems and technology
•  the values, histories, and interactions of social and political systems across global cultures, with emphasis on American democracy
•  spirituality, faith, and the wholeness of the human person, understanding interconnections of mind, heart, and hand
•  ethical and moral dimensions of decisions and actions

The University Learning Outcomes (ULOs) that students are expected to achieve are integrated in general education, program requirements, and assessments. As a result, students achieve competencies in their:

• I. ability to use effective written and oral communication in a variety of settings, to a range of individuals and within groups and teams, using various modes
• II. ability to formulate questions that reveal an intelligent curiosity about texts or concepts and to analyze questions with imagination and creativity
• III. ability to apply both qualitative and quantitative analysis in a variety of disciplines and situations
• IV. ability to use, evaluate, and interpret complex information, solve complex problems, and understand complex systems
• V. intellectual and emotional flexibility and adaptability to understand and shape lives within a culture of change
• VI. ability to evaluate information from disparate sources, to transform information into meaningful knowledge
• VII. ability to connect theory and practice
•  intellectual honesty, responsibility and accountability in learning and life

+Adapted to USM from “Learning Goals for the 21st Century”, AACU, Greater Expectations Initiative

Common Learning Experience Objectives are integrated within interdisciplinary Learning Communities (LC) and Idea Seminars (IS). In these courses, students receive general education credit for satisfying one or two Areas of Investigation through an interdisciplinary and discipline-specific emphasis. Freshman Learning Community objectives state that students will:

•  Gain factual knowledge in both disciplines
•  Learn fundamental principles, generalizations, and theories in both disciplines
•  Demonstrate integration of knowledge and principles in aspects of both disciplines
•  Learn to apply knowledge/principles and improve thinking, problem solving, evaluating, and decision-making
•  Develop creative capacities
•  Develop critical and reflective thinking skills
•  Develop personal responsibility for learning
•  Develop skill in expressing ideas clearly and completely in writing
•  Develop skill in oral communication

Idea Seminar objectives state that students will increase their:

•  general understanding of global interdependence
•  understanding of change as a component of global interdependence
•  understanding of interconnectedness as a component of global interdependence
•  understanding of diversity as a component of global interdependence
•  writing ability

Co-curricular goals are connected with the core curriculum through Service Learning. These outcomes state that students will:

1. develop an enhanced sense of civic responsibility, making students aware of their responsibility to address injustices and inequities
2. recognize the dignity of each person's capacity to learn, to relate, and better our diverse world
3. use reflection to connect theory with experiences and thought with action

Graduate program goals and outcomes are aligned with the USM mission and ULOs. Each program tailors outcomes to meet advanced level standards in the discipline and as defined by psychology, education, and business professional associations and accreditation agencies.

1C2 Student Learning Expectations Aligned with Mission

USM ensures alignment of student learning with its mission, vision, and philosophy through the curriculum review and governance process as well as through course syllabi. Through Program Review cycles, faculty engage in ongoing improvement deliberations that focus on student outcomes, best practice, mission and values, and environmental scans. All new academic programs must complete a proposal with business plan that explains how the program meets mission, goals, and outcomes. Recommendations for general education and program changes are reviewed by Academic Affairs Committee, Faculty Senate, Academic Affairs Committee of the Board of Trustees, with final approval by Board of Trustees (BOT).

The Areas of Investigation define the required general education courses and the University Learning Outcomes are aligned within these discipline areas. USM's Learning Framework Matrix in Table 1-1 shows this alignment. The minimum requirements for course syllabi require faculty to link the ULOs to program outcomes, course objectives, assignments, and assessment tools.

Table 1-1: USM Learning Framework Matrix

* Complete ULO statements in IC1

1C3 Key instructional programs, delivery methods, and technology use

The instructional programs at USM are at the associate, bachelor and master levels. See Figure O.2 in Organizational Overview for majors in program areas.

USM's instructional delivery approaches fall into three categories to serve the diverse needs of students and model best practice in teaching and learning. First, USM tailors the undergraduate, day program to traditional-age students, integrating active, applied, and service learning components. Second, evening undergraduate, degree-completion, and graduate programs target adult learners and incorporate adult learning research-based strategies. Third is online learning. Two programs (MAT & RN-BSN) are exclusively online, while other undergraduate and graduate coursework blends online and on-ground delivery.

Beginning in 2001 with the comprehensive curricular revitalization initiative (CCI), faculty committed to collaborative, active, experiential, and interdisciplinary approaches to curriculum and instruction. USM schedules at least 3 days each semester for in-service and workshops to assist faculty in learning new strategies. Department chairs and the VPAA also allocate time and resources for deliberating with internal and external colleagues and receiving training that applies to their discipline and to on-line delivery.

Faculty development emphasizes technology integration within instruction. From presentations on the media demands of millennials to workshops on USM's interactive learning management system, faculty use technology to engage students, to provide up to date resources, and to facilitate cooperative learning among peers. USM features upgraded “smart” classrooms and wireless networks and issues freshmen a free laptop, allowing faculty to integrate this resource within class activities. Additionally, program faculty establish technology literacy requirements for their majors, aligned with the demands of their profession.

1C4 Preparing students to live in a diverse world through diverse learning opportunities

Preparing students to live in and contribute to a diverse world is central to USM education. The mission and values that pervade the programs stress “preparing them for value-centered lives and careers that contribute to the well being of our global society” and “believing in the dignity of each person's capacity to learn, to relate, and to better our diverse world. We value community, respect, justice, and excellence.” These commitments translate into specific areas of investigation and measurable learning outcomes, such as:

•  investigating “the human imagination, expression in literature and the arts, and other artifacts of cultures” (AI-A)
•  investigating “the values, histories, and interactions of social and political systems across global cultures, with emphasis on American democracy” (AI-C)
•  demonstrating “the ability to use, evaluate and interpret complex information, solve complex problems, and understand complex systems” (ULO-IV)
•  demonstrating “the intellectual and emotional flexibility and adaptability to understand and shape lives within a culture of change” (ULO-V)

General education requires that students engage in cultural studies and can evaluate complex issues from multiple perspectives. All students take at least one Cultural Studies course offered in most disciplines, thereby students are expected to examine issues within multiple contexts. Additionally, students complete 2-4 Idea Seminars. The Idea Seminars are a unique feature of the curriculum and include three integrated themes: diversity , change, and interconnectedness. Faculty evaluate students' achievements on these themes. The VPAA analyzes achievement across course and over time (see 1R1) and confers with Academic Affairs and Academic Leadership Committees to act on findings.

The Masters programs in education, psychology, and business integrate outcomes throughout courses to address cultural diversity. Furthermore, each program requires at least one course on multicultural, global issues. USM's Global Studies Institute extends the application of diversity concepts through lecture series, campus-wide activities, and internships that highlight issues of diversity, human rights, and global perspectives.

To honor their commitment to interactive, collaborative learning, faculty use multiple instructional strategies. They share strategies for actively engaging all students. Through Faculty Institutes twice a year, four In-service Days per year, and Faculty Senate meetings each month, faculty enrich their repertoire of approaches to engage multiple learning styles. This occurs through presentations, research-based strategy discussions, simulations, peer sharing, and critique. Moreover, the academic assessment system leads faculty to assess student achievement of learning outcomes through multiple avenues. Based on the requirements for syllabi development, faculty design assignments and assessment tools that represent a variety of ways to showcase learning so that students can represent what they have learned in diverse ways.

There are many opportunities for real-life applications of the curriculum through Service Learning. Departments offer courses that have a Service Learning component and Student Life and the Global Studies Institute sponsor frequent local, regional, and international trips devoted to service learning aims. Not only do participants engage in “bettering their diverse world” but they critically reflect upon the experience and how it ties to beliefs, values, and core curriculum outcomes. These experiences capture multiple learning styles and make vivid the applied feature of USM's applied liberal arts curriculum.

1C5 Creating and maintaining a reflective, inquiring, and ethical climate

From policy to practice, students, faculty, and staff participate in a reflective, inquiring, and ethical climate. Official handbooks state: “University of Saint Mary is committed to conducting its business activities with integrity, fairness and in accordance with high ethical standards and every employee is obligated to uphold these standards in performance of work and every activity related to work.” Also in handbooks and in all course syllabi, academic honesty statements outline the responsibilities to promote a mission-driven, ethical climate:

Academic honesty is expected of all members of the University of Saint Mary community. It is an essential component of higher education and is necessary for true academic growth. Christian tradition and professional excellence demand that truth, honesty and academic integrity be valued in all of our interactions. Consistent with the University mission and values, academic honesty is essential in achieving one's God given potential. Value-centered lives are meaningless if honesty is not one of those values. The bonds of community require that none of us knowingly discredit the value of a Saint Mary education. Respect for others requires that we acknowledge the sources of our information. Justice requires that we possess the skills and learning that we profess to have. Excellence requires that we do the best we can in everything we do, without resorting to dishonesty or other immoral behavior.

The ULOs, core curriculum, and teaching-learning relationships support and extend this commitment to create a thriving community of inquiry, discourse, and reflection. Professors model and lead students to demonstrate “intellectual honesty, responsibility and accountability in learning and life” (ULO-VIII) and the “ability to formulate questions that reveal an intelligent curiosity about texts or concepts and to analyze questions with imagination and creativity” (ULO-II). Fundamentally, the core curriculum is designed:

• to stimulate intellectual discourse and differing viewpoints
• to explore connections across disciplines and with real-world, global issues
• to critically reflect upon the meaning of knowledge in learning and living.

For example, Learning Communities are team-taught by faculty from different disciplines and they model through respectful debate how different perspectives on concepts and content enrich understanding and problem solving. Activities within courses are interactive and push students to use scholarly investigations to take a stand and argue for differing points of view, thus deepening their analysis and critical reflection. Teaching-learning relationships extend beyond the classroom, and inquiry and discourse occur informally and in regularly scheduled events such as the annual Lincoln Douglas Debate, GSI panel presentations, and theatre productions.

Processes

1P1 Determining common and program-specific student learning outcomes

Faculty, administrators, and the Board of Trustees are involved in setting the university and program learning outcomes. Faculty begin and drive this process. At the point when recommendations have been discussed and debated among faculty at the department level and at-large through Faculty Senate, recommendations follow the governance process. The taskforce seeks approval through a vote at Faculty Senate, which is then forwarded by the VPAA to the President. If supported, the recommendation is forwarded to Academic Affairs Committee of the Board, and then to BOT for final approval.

The Comprehensive Curricular Initiative resulted in the current educational program and built a model for ongoing improvement (OGI) to ensure currency and excellence. The year-long process led to an operational system within University policy and practice to ensure ongoing quality and informed decision making. The steps of the CCI process outlined below describe the steps and the involvement of key participants.

• Compiled “What If” brainstorming ideas from each department that provided rich images of a revitalized curriculum and what it meant for student learning
• Created a “jump start” summer work group to organize the “What If” brainstorming ideas, to research curricular and assessment trends, to examine market demands, and to present a summary to faculty peers at Fall Institute ( Report Nov 03: Why Curricular Change at USM?)
• Established a “G6” task force of six elected and appointed faculty led by the VPAA to develop concrete recommendations to faculty, administration, and BOT
• Created and adopted new University Learning Goals and Outcomes adapted from AACU “Greater Expectations”
• Designed OnGoing Improvement (OGI) concept, graphic, and template for use by all university departments—initiatives and annual goals linked to mission, research, collaborations, actions, evaluation, and improvement
• Developed descriptive rubrics for assessing levels of competency on ULOs
• Offered a two-day Faculty Institute led by a consultant from Alverno College on outcomes assessment—how to link course and program outcomes
• Appointed Associate Dean of Assessment and expanded Assessment Committee to include student services staff, Academic Resource Center director, and Information Services Director
• Established a Learning Framework Matrix aligning university and program requirements with learning outcomes and course assessments
• Developed minimum essentials and evaluated all syllabi for consistency in linkages among outcomes, assignments, and assessments
• Designed university outcomes database for reporting of student achievement in courses
• Established Faculty and Program evaluation criteria that expect the integration and assessment of learning outcomes, active learning strategies, professional trends, and market demands

1P2 Designing New Programs that Balance Student and Market Needs

USM takes a strategic and data-based approach when designing new programs that balance the diverse needs of students and the emerging needs of the educational and global markets. Environmental scans that occur during strategic planning cycles also direct faculty and staff as they develop new program proposals and growth plans for existing programs. They examine internal and external factors in defining strengths, weaknesses, opportunities, and threats. The diverse needs of students and the contemporary demands for jobs define the mandate for service that is considered within the mission of USM and its sponsors, the Sisters of Charity of Leavenworth. The confluence of strengths and opportunities yield proposals that are further analyzed using a business model that evaluates risk within the context of financial stewardship.

During 2006-2008, the Academic Leadership Council (ALC), comprised of department chairs and program directors, planned a process for academic program analysis that resulted in a growth plan template which uses Activity Based Costing (ABC) and includes positions of relevance, market demand, and analysis of program changes in light of University mission and student needs. Currently, Biology, Chemistry, and Art have implemented growth plans that meet the criteria for balancing financial stewardship, student needs, curriculum integrity, and market demands. Criteria for these plans are highlighted in Table 1-2.

Table 1-2: Criteria for Program Growth Plans

1P3 Student Preparation and Academic Readiness

Systematic procedures are in place to determine the preparation of students so they can be successful in USM academic programs. Faculty identify the basic foundational skills required for entry level coursework and use assessments to determine skill levels. The assessments are tailored for freshman and transfer students. During pre-enrollment and depending on American College Testing (ACT) scores, freshman take the COMPASS Test. Achievement ranges align with course levels in math, science, and business and students are enrolled in appropriate courses. Writing skills are monitored in Composition I, which all freshman are required to take and pass with a C or better. If a grade lower than C is achieved, USM requires students to retake the course and they are strongly encouraged to work with Academic Resource Center (ARC) tutors and software programs. Transfer students are required to take Transfer Core which has built-in assessments of foundational skills used to determine learning needs and potential for academic success.

Each program provides an introductory foundations course with an overview of the field and the performance expectations of the major. Content, concepts, and skills for the major reflect professional standards of the learned societies. Through the foundations course, student achievement yields information that faculty, advisors, and students use to make decisions about potential for success based upon interests and abilities.

1P4 Communicating Expectations for Student Preparation and Learning Objectives

Expectations for student preparation and learning objectives are communicated in a variety of ways including written materials, advising/mentoring for prospective and current students, and in required courses. Written materials describe proficiency levels that are prerequisites for admissions, enrollment, and honors; program of study requirements; and university/program learning outcomes. The statements are included in the Student Handbook, University Catalogs, and Specific Program handouts which are made available in print, on the USM website, and on eSpire—USM's learning management system. Faculty, advisors, and admissions/registrar staff inform students how to access this information and explain it in classes and advising sessions.

The advising and mentoring system for prospective and current students is an important feature for communicating expectations and key components are listed in Table 1-3.

Table 1-3: Advising and Mentoring System

The Academic Resource Center actively provides support to student and faculty with the primary goal to aid students of all abilities with becoming effective learners. The ARC does this by helping students to improve in the critical academic areas of reading, writing, and learning skills. In addition, the ARC provides tutors to students for a wide range of academic fields, including the team taught, interdisciplinary Learning Communities. The ARC also hosts seminars on “student survival skills” which include time management, stress management, note-taking skills, test preparation skills, math seminars, reading seminars, and study skills seminars. These small seminars are designed to develop the skills and abilities to succeed in university life.

1P5 Helping Students to Select Relevant Programs of Study

To help students select programs, academic advisors provide guidance through their regularly scheduled meetings. Further, the ARC as well as Student Life career center and counseling services work directly with students to clarify their goals and expectations, to reflect on their abilities, and to analyze the match between interests and abilities. Each of these resources probes students—through reflection, requirements, surveys, and computer career inventories—to think realistically about program requirements and career options in light of their goals and abilities.

While these resources have been effective for many students, USM has identified a need to build a more systematic process for ensuring student success. This is central to USM's strategic plan, particularly the initiatives for growing enrollment and improving retention. Consequently, a new AQIP Action Project was initiated in June 2008. The Student Success Action Project originated through a convergence of issues:

• decreasing accessibility to students because of changing student profiles, especially the significant increase in transfer students and other students who live and work off campus and do not avail themselves of current advising opportunities
• campus satisfaction surveys that indicated inadequate communication of information leading students to feel overlooked and uninformed
• an increase in the number of students on academic probation requiring creative ways to use existing resources to address their academic, advising, and mentoring needs with more immediacy.

The goal for this project is to design a system for advising and mentoring that will enhance student learning and retention. This system will:

1. ensure that all new students (including transfers), especially the undeclared, are immediately linked with a team of well-trained faculty and student services advisors for optimum academic and career advising
2. support all students in investigating academic and career options while allowing early effective contact with programs of choice
3. assure strong mentoring relationships for all students throughout their course of study

1P6 Documenting Effective Teaching and Learning

The Comprehensive Curricular Initiative was instrumental in defining best practice for effective teaching and learning. It places a strong emphasis on active and collaborative teaching and learning strategies that engage multiple learning styles. Faculty institutes, departmental in-service days, and professional workshops are some of the ways in which USM supports the development of pedagogical skills. Faculty use various procedures to document that this emphasis occurs, including a nnual reports (Table 1-4), course and faculty evaluations, program reviews, rank and tenure process, and post tenure review. (See 1P11-13 for documenting student achievement of outcomes.)

Table 1-4: Annual Report & Program Review Highlights

Course and Faculty Evaluations In order to promote open communication between students and faculty and to maintain high levels of instruction and student learning, faculty in all courses give students the opportunity for written course evaluations at mid-term. Faculty discuss with students the mid-term evaluations in light of course goals in order to provide on-going assessment of the teaching/learning process. Final course evaluations, given at the end of course, are confidential, kept in the office of the VPAA or the VP for Extended Sites, and not distributed to faculty until after final grades are posted. When released, faculty analyze results and make teaching improvements or curriculum changes when warranted. VPs monitor results and discuss problem areas or trends with department chairs for their follow-up with faculty.

Rank and Tenure Process Rank and Tenure Committee makes recommendations for promotion and/or tenure based on teaching, scholarship, and service criteria, with teaching and advising effectiveness heavily weighted. Major areas of review are knowledge of subject matter, pedagogical techniques, advising, and out of class activities with students. Candidates provide evidence for each in their teaching portfolio. Guidelines and criteria are outlined in the Faculty Evaluation section of the Faculty Handbook.

Post-Tenure Review Using Ernest Boyer's “scholarship of teaching” framework, USM has adopted policy and procedures to monitor ongoing teaching effectiveness and scholarship. Every five years after tenure, faculty formally analyze their work and provide evidence for teaching as discovery, integration, application, and in classroom time. Faculty document their growth, inquiry, and innovations and present their accomplishments for peer evaluation through the Rank & Tenure Committee. The VPAA, President, and BOT present public commendations for successful reviews and faculty are invited to present their accomplishments at University sponsored colloquia. (See detailed criteria Faculty Evaluation section of the Faculty Handbook.)


1P7 Effective Course Delivery System

In developing an effective and efficient course delivery system, USM addresses the pedagogical issues for the three delivery categories: traditional-day; adult-evening, and online. The Academic Affairs Committee, ALC, and VPAA have specific roles and responsibilities as outlined in the Faculty Handbook to review and monitor curriculum integrity. Therefore, the checks and balances that are built into the USM program review and governance systems ensure a balance between student and institutional needs. Faculty development, course scheduling, and technological resources are key to tailoring the delivery systems. The features of each system are outlined in Table 1-5.

Table 1-5: Course delivery system features

USM expects the three delivery systems to operate within existing systems and structures for curriculum, instruction, and assessment. Therefore, the quality assurance processes in place apply to each delivery approach, from course syllabi and faculty development to schedules, pedagogy, assessment, resources, and overall governance. Necessary modifications or additions—suggested by any stakeholder, e.g. students, faculty, staff, partners—are reviewed through appropriate governance or administrative lines to assure comparability of quality. Administrators tested the integrity of this systems approach to effective course delivery during the expansion of Distance Education, an AQIP Action Project, which led to a change in affiliation status. While the project encountered challenges in marketing efforts for various delivery formats, the Administrative Council (AC) confirmed that the system for designing and delivering courses and programs had the checks and balances in place to ensure integrity, quality, collaboration, and ongoing improvement.

1P8 Monitoring curriculum currency and effectiveness

USM systematically monitors curriculum effectiveness and currency through its reliance on the OGI—OnGoing Improvement Model used throughout the University. Specific to curriculum, OGI is integrated within the policy and procedures for annual reports, program reviews, and new program development. As highlighted in 1P1 and explained in Category 8, OGI criteria are embedded in annual report guidelines and require faculty and programs to examine curriculum goals, analyze evidence toward meeting the goals, communicate findings, and make informed decisions for improvement based upon the evidence.

Faculty and chairs conducting 5-year academic Program Reviews use the OGI model to examine quantitative and qualitative data in light of professional standards, workforce and societal trends, mission integration, student achievement, and stakeholder satisfaction. After departmental analysis and evaluation, faculty establish improvement goals. Then the chair invites an outside reviewer to evaluate the report and schedule an on-site visit. This external content expert uses the standards of learned societies to judge the quality of the review process, the curriculum, student achievement, and improvement goals. Reviewer evaluations and feedback are used by department faculty and forwarded to the Academic Affairs Committee and the VPAA, who then reports findings and recommendations to the President and BOT. The OGI model enables faculty to make informed program decisions and ensures built in accountability to stakeholders.

Faculty participation at learned society meetings also ensures that curriculum and instruction reflect best practice, professional standards, and current trends in higher education. Professional development awards support faculty to attend national and regional meetings on a regular basis. Faculty must complete a report that specifically explains the value of this experience to USM, “especially in terms of the curricular and pedagogical initiative i.e. how you will integrate what you learned into teaching, learning, research and how you will share with colleagues.”

USM's process for changing or discontinuing programs uses the same model as for new program and program growth proposals as described in 1P2. Department faculty develop data-based proposals that use internal and external trends to support recommendations and submit them through the appropriate channels outlined in the governance system.

1P9 Faculty and Student Needs for Learning Support

USM identifies learning support needs during activities for prospective students (see 1P3) and new faculty. Also, at various points throughout the semester USM identifies student and faculty needs as listed in Table 1-6.

Table 1-6: Learning Support Needs Identification

1P10 Aligning Co-curricular and Curricular Goals and Outcomes

Alignment between service learning co-curricular and curricular goals occurs because most service learning experiences are integrated in courses and students are required to reflect on connections with ULOs. Further, linkages are made across USM due to the broad representation of the USM community in establishing ULOs during CCI. A stated expectation during adoption was that staff as well as faculty have a role in assisting students to achieve the ULOs. The 2003 University-wide Fall Institute involved all faculty and professional staff in exploring creative ways, within their areas, to interact with students to model and advance the learning outcomes. The Vice President for Student Life joined the university assessment committee to participate in aligning curricular and co-curricular learning experiences and to ensure an integrated, comprehensive system for assessing ULOs. This led to improved coordination between academic assessment and student life development, including plans for a co-curricular transcript.

 1P11 Determining the processes for student assessment

The CCI process as described in 1C1 affirms that faculty are the central drivers of developing, using, and modifying assessment processes. This was formalized as stated in the Faculty Handbook:

Faculty participate in the assessment activities of the University which has adopted an ongoing improvement model that dovetails with the Academic Quality Improvement Program (AQIP) of the Higher Learning Commission. These activities include syllabi development which requires alignment among university outcomes, course objectives, and student assignments (see syllabi requirements for more information), assessment of student learning in relation to course, program and university outcomes, self-assessment of faculty performance in relation to analysis of student performance, course evaluations, and ongoing improvement initiatives, program evaluation through annual individual and departmental reports and regular program reviews, and institutional self-study improvement efforts based upon the ongoing and systematic use of data based decision making. Institutional assessment activities are coordinated by the Vice President for Academic Affairs.

The assessment system is designed to empower students, staff, and faculty to make informed decisions about how they teach, learn, and achieve their goals. The feedback loop of the teaching-learning-assessment process is shown in Figure 1-1 which provides information not only about teaching and learning, but about the assessment system itself. (See Assessment System Design )

Figure 1-1: Teaching-Learning-Assessment System

1. Course design links the key components in the following manner:

• University Learning Outcomes (ULOs) link to USM Mission
• Learning Framework Matrix links the ULOs to general education requirements
• Course Syllabi link ULOs to Course Objectives to Required Assignments
• ULO Rubrics describe criteria and levels of student achievement with a rating scale
• Student achievement scores—ULO ratings—are reported in Outcomes Database
• ULO database reports show achievement patterns
• ULO findings are used in advising, curriculum, and instruction

2. Data-based findings interpreted by faculty and administrators lead to professional conversations about the needs and outcomes for students, faculty, and curriculum.

3. Recommendations presented in OGI reports drive the goals for the next year's initiatives for faculty and programs.

4. Findings lead to plans for curriculum improvements and faculty development by those responsible for making change and improving programs.

1P12 Student Preparation for Further Education and Employment

USM system to determine student preparation for further education and employment is multi-pronged and includes Program Reviews, career center alumni surveys, practica evaluations, and standardized test results. The 5-year Program Reviews require that graduates are surveyed regarding effectiveness of USM preparation. Program chairs summarize results and initiate program improvements. The career center director surveys alumni requesting information on the effectiveness of the overall USM program in preparing them for their workforce participation. These two methods are consistently applied across all programs and for all graduates.

Several programs require practica in the field of their major. The practica supervisor completes an evaluation which includes items that solicit feedback on the student preparation to perform effectively in the field. Program chairs summarize these results on an annual basis, and where applicable, program faculty address these findings in their annual ongoing improvement reports.

Program chairs use standardized test results when available. Students in licensure programs (e.g. nursing, teaching, counseling) must pass a standardized test for program completion and/or licensure and chairs analyze sub-scores in terms of curriculum alignment, making revision when warranted. Students applying to graduate school take standardized tests (e.g. MCAT, Graduate Record Examination [GRE]) and when scores are released to the University, program faculty analyze how the results may impact curriculum. Acceptance rate to graduate school is also evidence used to evaluate student preparation. As a part of the OGI cycle, this information is reviewed by faculty and chairs in light of program implications.

1P13 Student Performance Measures

The VPAA regularly analyzes with the ALC measures of student performance including grade point averages (GPAs), common learning objectives in Learning Communities and Idea Seminars, and achievement on Learning Outcomes—program and university. GPAs indicate progress on achieving proficiency in Areas of Investigation and in major program requirements, and are monitored by advisors and the VPAA on a semester basis. The registrar calculates high achievement (Dean's List) and low achievement (Probationary List) rates. The VPAA recognizes Dean's list students and refers Probationary list students to the ARC director for follow-up contracts, support, and mentoring. At the program level, program chairs review GPAs of graduating seniors and based upon results, improvement goals are established in the annual OGI review and annual report.

LCs for freshman and ISs for sophomores and juniors target foundational skills and concepts for the general education curriculum. LCs measure 9 objectives related to learning skills and ISs measure 5 concepts related to the global interconnectedness theme. The VPAA collects and tabulates semester data and conducts a trend analysis on a bi-annual basis.

Program Learning Outcomes at Bachelors and Masters levels are evaluated by faculty in major program requirements. As faculty advise majors, progress in meeting outcomes is assessed. Capstone experiences in each major/level provide a formal review of program outcome achievement. Each program designs its collection and monitoring procedures, and chairs summarize evidence in Annual Reports and provide trend analysis of the results in the 5-year Program Review.

Faculty assess ULOs across the curriculum as required in their syllabi. In courses that satisfy general requirements, faculty collect data on 3 designated outcomes by rating achievement using a standard rubric. Faculty submit these ratings along with course grades to the Registrar each semester. During 2007-08, the ULO database was piloted, with a mid-year review by faculty during Spring Faculty Institute. The Assessment Committee led the pilot and currently is examining issues of validity and reliability. The committee anticipates refining the rubric rating process so that results will provide reliable information to guide students, advisors, and programs, and provide internal and external stakeholders with a valid picture of ULO achievement.

Results

1R1 Common Student Learning Objective Results

USM tracks results of common learning objectives and key concepts in required LC and IS courses as shown by data from Fall 2005—Spring 2008 in Figure 1-2. Faculty evaluation of IS concepts indicates satisfactory (2.0) achievement and above. Student self-assessment of LC objectives indicates above average progress over time. Cohort ratings from fall-spring (e.g. Fa 05-Sp 06) remain consistently high average.

Figure 1-2: Historical trends of Idea Seminar and Learning Community Results

LC: Please rate the progress you have made in this course: 1=low 2=low average 3=average 4=high average 5=high

The Assessment Committee analyzed the results of the ULO pilot data and found that after faculty intervention between semesters the variation among ratings remains unsatisfactory. Data were analyzed by course level (100, 200, etc) and program area (e.g. fine arts, behavioral sciences, etc) and the standard deviations revealed an unacceptable variance to claim reliable measures. Revisions are underway as noted in 1P13. However, data were also analyzed based upon faculty involvement in the pilot to determine knowledge and commitment to the overall goal of assessing ULO achievement in general education requirements. These data in Figure 1-3 indicate faculty support of the goal and identify a need for greater oversight of adjuncts.

Figure 1-3 ULO pilot data report

* 75% not reporting are adjuncts
** 92% not reporting are adjuncts

1R2 Achievement of Knowledge and Skills

USM monitors GPA reports and standardized tests to verify that students have acquired the knowledge and skills required for degrees and professional credentials.

Figure 1-4 represents GPA data and shows the percent of students with honors (3.0 and above) and on probation (less than 2.0) by semester and by delivery approach. Overall there has been an increase in honors students and a decrease in probationary students from Fall 2004 to Spring 2008. Results indicate that within this time period more than 85% of all students have satisfactorily acquired knowledge and skills each semester, with the rate increasing to 93% in Spring 2008 (% GPAs above 2.0). Leavenworth represents the traditional-day program, OPC represents evening-adult program, and online includes the undergraduate distance program, initiated in Fall 2006 with 5 students. Variation among delivery approach for probationary rates seems to reflect the national trends of highly motivated adult students compared with traditional age students.

Figure 1-4 GPA distribution by semester and delivery approach (site)

Standardized tests for licensure in nursing and education provide data on student achievement of requisite content knowledge and skills. The NCLEX-RN Licensure Exam Pass Rate for the first USM nursing graduating class (2008) was 81%, above national norms. Education majors are required to take content area tests which provide external validation of their level of achievement. Results shown in Table 1-7 indicate the successful pass rate for majors in English, elementary education content, math, biology, chemistry, and history and government.

Table 1-7: Pass Rate for Content-Area Tests

Graduate programs have systems for tracking student knowledge and skills. Each program has a tailored approach for monitoring student achievement on program learning outcomes. For example, the director of the graduate education program assessment system analyzes and publicizes on the USM website the results of student achievement on outcomes that are assessed in each course.

1R3 Processes for Helping Students Learn

Course evaluations and alumni surveys assess USM's success in helping students learn. Students complete course evaluations each semester, and faculty and supervisors use them to analyze effectiveness in teaching. Faculty design their own evaluation forms and tailor them to the goals and outcomes targeted within the course. Alumni surveys are distributed by the Student Life Office annually and by program chairs at least every 5 years during program reviews. Results are tracked by delivery approach and show a high satisfaction rate.

Qualitative indicators of USM's success in helping students learn include progress on University systems and processes. Documents (e.g. Learning Framework Matrix, syllabi), dialogue, and data (e.g. rubric ratings) reveal that course assignments and assessment instruments align with university learning outcomes and provide flexibility to accomplish common goals. Ongoing improvement reports (e.g. annual reports, program reviews) attest to the use of assessment to inform and improve teaching and learning. Faculty members analyze data on student outcomes to target efforts for improving learning. An emerging pattern indicates students are achieving outcomes through active engagement and critical reflection.

1R4 Results compared with state and national standards

The pass rates of licensure programs indicate how USM results compare with other higher education institutions. Education majors must take the Principles of Learning and Teaching (PLT) exam which compares achievements with other teacher education programs and is reported nationally in the Title II report, as shown in Table 1-8.

Table 1-8 USDOE Title II Report Results

The 2008 nursing majors 81% pass rate on the National Council of State Boards Licensing Exam (NCLEX-RN) Licensure Exam is comparable with 2007 data on other Kansas Bachelor of Science in Nursing (BSN) programs and higher than the national rate (78%). 2008 was the program's first graduating class and national comparison rates are not yet available. The Assessment Technology Institute (ATI) test is a predictor of NCLEX-RN performance and provides subtest scores and national percentile ranks in nursing content areas. First year results as shown in Table 1-9 indicate overall satisfactory program preparation. Scores less than the 50 th percentile are used to target program improvements.

Table 1-9: BSN Graduates ATI Scores May 2008

External recognition of USM's assessment system by the Council for Higher Education Accreditation (2007 CHEA Award) and National Council for the Accreditation of Teacher Education (2003 NCATE Spotlights on School of Education ) affirms that the process for helping students learn is progressing well. Award criteria highlighted evidence of content standards, learning outcomes, success with regard to outcomes, informing the public about outcomes, and using outcomes for improvement.

Improvement

1I1 Improving current processes and systems for helping students learn

Using the OGI model, USM analyzes the quantitative and qualitative results at least annually to determine what improvements are necessary. Improvements are implemented and performance is tracked to assess the results of the intervention. Faculty do this each semester after interpreting their course evaluation results. Annual reports are central to the ensuring USM processes help student learn. The presentation of findings and the recommendations for improvement come from those closest to and most informed about specific processes and operations. As AC summarizes reports annually, they are in a position to ensure overall institutional coordination.

1I2 Setting Improvement Targets and Communicating Results

Program chairs with faculty set targets by analyzing performance results in relation to program and student goals. Monitoring results allows patterns to be examined and to pursue follow-up data collection if warranted. As an example, the results on the ATI nursing test will be used by the nursing faculty to review and revise the Leadership, Pharmacology and Medical Surgical components of the program. Also, the pilot of the ULO assessment system raised issues of reliability and validity, interventions were made for the spring administration, and current results lead to recommendations for specific procedures for course level ratings and possible realignment of the Learning Framework Matrix. Additionally, the Honors-Probation semester results have signaled the need to provide tailored supports for different categories of students. This led to the Student Success Action Project which was initiated in June 2008.