University of Saint Mary
Planned project kickoff date: 2006-01-23
Target completion date: 2006-10-31
Actual completion date:
A. Give this Action Project a short title in 10 words or fewer:
Identify, evaluate, and prioritize reports and processes for informed, data-based decision-making
B. Describe this Action Project's goal in 100 words or fewer:
Successful completion of this project will allow us to move toward a closed loop system for ongoing improvement university-wide that is accessible and integrated. This system will allow us to access and use better and more relevant data/information and data collection processes for institutional decision-making. This project will be Phase 1 of a multi-phased approach -- we will identify, evaluate and prioritize the processes and reports that we need and/or are currently using for institutional decision-making including what data we need/have, where is it collected/stored, who collects it and how is it collected, what are our priorities for future collection and storage of needed data/information.
C. Identify the single AQIP Category which the Action Project will most affect or impact:
Primary Category: Planning Continuous Improvement
D. Describe briefly your institution's reasons for taking on this Action Project now -- why the project and its goals are high among your current priorities:
The need to create more systemic and systematic processes for data based decision-making has been voiced at many levels, including recent feedback from employees asked to help identify Action Projects for AQIP. The need for this project has been stressed as a result of two initiatives: 1) a 3-year academic assessment initiative, driven by faculty and staff, highlighted the need for systematic integration of academic data with a university-wide comprehensive system for data collection, analysis, and decision-making; and 2) our self assessment and feedback through Kansas Award for Excellence reinforced our need for formalized processes that ensure institutional effectiveness through data-based decision making. The Summer 2005 conversion to an integrated information system allows us to create and manage a clearinghouse of data collection activities and processes for reports that are easily accessible and that ensure relevant data are used for informed decisions.
E. List the organizational areas - -institutional departments, programs, divisions, or units -- most affected by or involved in this Action Project:
All stakeholders will benefit from a comprehensive system that enables sound rationales for institutional decisions as evidenced by supporting data, including formal student and stakeholder input. Our primary stakeholders, SCLs and Alumnae, will be able to track how strategic initiatives are decided based on trend analysis and to monitor the impact of their financial support in light of mission driven priorities. Additionally, student learning will be enhanced by the identification, analysis, and integration of trend data from both academic and student life. Students will benefit because they will be able to monitor their progress on learning outcomes throughout their programs and their advisors will become more effective in recommending learning experiences that address their development needs.
F. Name and describe briefly the key organizational process(es) that you expect this Action Project to change or improve:
This project will result in enhanced employee input in decisions leading to improvements and change. Because we intend an integrated, systemic process for decision-making, all employees will have a (routine) level of involvement. Employees will have access to relevant information for analyzing progress and determining their priorities. Their efficiency and effectiveness will be increased because redundant paperwork will be reduced and scheduled and timely report information for assessing progress and outcomes will be provided. This will increase buy-in and morale and mitigate anxiety toward institutional change. Clearer expectations and rationale for priorities will be possible, thus unifying and mobilizing staff efforts. Employee satisfaction will also be evaluated and shared.
G. Explain the rationale for the length of time planned for this Action Project (from kickoff to target completion):
A committee of 8 will interview all university departments identifying the processes/reports we currently use by March 1, 2006. By May 1, 2006, committee members will evaluate these processes, making recommendations. During the summer, work will continue and by October 31, 2006, a prioritzed list of processes and reports that we currently use and that we need to refine or develop will be identified.
H. Describe how you plan to monitor how successfully your efforts on this Action Project are progressing:
A designated committee will have this Action Project as a primary task. Work will be divided among members and bi-monthly meetings will be used for status reports and problem-solving. The committee chair will monitor progress toward meeting the established timelines and will report to the President and AQIP team on progress.
I. Describe the overall "outcome" measures or indicators that will tell you whether this Action Project has been a success or failure in achieving its goals:
A list identifying reports/processes that provide data-based information relevant to all AQIP criteria. Status of the reports will be identified in terms of whether they exist, provide satisfactory information for decision making, need revision, or need development. The list will prioritize the development work needed on defining and constructing the essential data-based reports and processes most relevant for informed, institutional decision making.
J. Other information (e.g., publicity, sponsor or champion, etc.):
Last Action Project Update: 2006-09-13
A. Describe the past year's accomplishments and the current status of this Action Project.
The university Assessment Committee has lead responsibility for Action Project #3 and meets as a whole at least once a month, often twice, with sub-group tasks and meetings occurring more frequently. Step 1, to identify reports/processes USM currently uses, was completed by March 1. To accomplish this the committee a) reviewed the KAE Self Assessment Report and the AQIP Application and created a status report, b) developed an interview guide, c) interviewed/surveyed representative members of the community with responsibilities related to the AQIP criteria to find out what reports/processes exist (if any), if results are summarized & reported, and how results are used for ongoing improvement.. This allowed us to determine what information USM currently collects that relates to organizational effectiveness as defined by each AQIP criterion and helped to determine if what we collect is relevant for decision making and what holes we have.
To evaluate existing reports and processes (step2), completed by May 1, the committee categorized and ranked items identified in step 1 according to AQIP criteria, benefit to multiple divisions, and type of tool (e.g. report, process, product). We committed to some guiding principles: 1) we cannot collect everything; 2) we must decide what information is a priority; 3) we must plan for how the data will be housed in a centralized clearinghouse, and 4) there is potential for creating some “bridges across towers” so that the USM community can know what different divisions are doing.
Currently work is underway on prioritizing tools and reports (step 3), with an anticipated completion date of October 31. Guidelines for prioritization include describing tools and reports: 1) for ongoing assessment of the whole institution with special emphasis on current action projects and university initiatives; 2) that measure how effective we are; 3) that effect more than one area; 4) that provide useful information for decision making; and 5) that allow us to monitor our effectiveness in addressing AQIP criteria. Preliminary findings indicate that USM has a) tools that provide relevant information to address characteristics of multiple AQIP criteria, b) lots of processes where no data are captured, c) reports that capture data but are often used within “towers,” and d) an incomplete system for regularly tracking or disseminating data analysis and trends.
B. Describe how the institution involved people in work on this Action Project.
Many people at USM see a need for and the benefits of informed decision making that rely upon accurate and shared information. Through our recent technology conversion USM now has the capability to systematize and share meaningful information that can be used to make decision making more transparent and data-based. We, as a community, are excited by this potential because it will move us closer to realizing our mission and values.
As an Assessment Committee, we were conceptually challenged by how to discover what tools existed that linked with the AQIP criteria. This served as a motivator, although at times we were frustrated with the breadth of the task. We analyzed what we currently knew about USM data gathering practices. Then we decided to interview those closest to and those responsible for decisions related to each criterion. We formed interview questions that we thought likely to surface the ways in which people gather and use information. Individual committee member input was shared, which then guided the development of an uniform approach to interviewing that allowed for tailored questions too. Collectively we interviewed about 75% of the faculty and staff at the university. The interviewees were engaged in describing what is useful for their work to be effective and what they wish they collected to better guide their decisions and those of the university. Through the interview process they had the opportunity to share their successes, their insights, and their needs. This proved to be an affirming, engaging, and motivating process for them as well as the interviewers.
The committee presented their progress on the action project at meetings and in writing. The members represent a variety of divisions and roles. Members shared preliminary findings within the context of their professional responsibilities, for example at department meetings, during strategic planning sessions, in annual evaluations. Four members of the committee are on the strategic planning committee and two members are on administrative council. These connections facilitated regular communication about progress, insights, and needs.
C. Describe your planned next steps for this Action Project.
Currently, as we complete step 3, we are defining each priority tool, identifying who collects the information, where it is stored, how it is used, how it can/should be used, and what reports can be automated by USM’s information management system. We are also analyzing the priority tools in relation to overlap, quality of information generated, type of information needed, and AQIP criteria. We will report the findings from this analysis in terms of recommendations for developing or improving tools/reports that can systemize and streamline our decision making. These recommendations will frame our objectives for the next Action Project which will define Phase 2 of this project.
D. Describe any "effective practice(s)" that resulted from your work on this Action Project.
Two “effective practices” that emerged from this project are: 1. analyzing the tools we use to collect information and 2. interviewing faculty and staff about tools and reports.
1. When analyzing the existing tools, we studied and discussed if and how they produced information that could help us answer any of the questions listed for each AQIP criterion (context, process, results, improvement). While this was a laborious and intense process it yielded great benefits, two in particular: a) a working knowledge and value of the AQIP criteria, and b) determining the effectiveness and transference of information that we collect.
2. When interviewing faculty and staff about what tools, reports, and processes they use that may relate to one or more of the AQIP criteria, we were able to share and celebrate accomplishments, seek their insight on improvement, model the process of ongoing improvement, and acknowledge their role and contributions to institutional effectiveness.
E. What challenges, if any, are you still facing in regards to this Action Project?
The specific outcomes for this project do not pose major challenges at this point. However, one is emerging as we transition into our next phase, which is identifying the tools we need and want that are not currently used. The questions for us are:
• how do we identify and develop new and needed tools that focus us on our priorities, integrate relevant information, and reflect a systems approach to data collection?;
• what are others doing or what other tools exist so that we don’t have to reinvent the wheel?
• how do we get transparency without publicizing our secrets?
F. If you would like to discuss the possibility of AQIP providing you help to stimulate progress on this action project, explain your need(s) here and tell us who to contact and when?