Assessment


Every degree-granting program and many of the administrative and student support units at Texas A&M University participates in the assessment process.

The purpose of assessment is not to gather data, to report the completion of required operations, or even to place a value judgment on performance. At its root, assessment is "an intentional, systematic  collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and development, or in the case of administrative and student support units, to improve operations, efficiency, and satisfaction" (taken from Palomba and Banta, 1999).

Texas A&M University utilizes WEAVEonline as the assessment reporting mechanism through which departments and programs document their outcomes, measures, findings, and action plans.

What does "good assessment" look like?


The assessment cycle is the process by which departments and programs gather and evaluate data (i.e., student-produced work, survey results, process evaluations, etc.) aligned with their outcomes to inform the continuous improvement of their curriculum, pedagogy, and operations. WEAVEonline breaks the steps of the cycle down into 7 pieces or steps. Examples of units at Texas A&M University who have excelled at documenting their assessment processes are available for academic programs and administrative units.
assess_process_graphic.jpg

The mission statement serves as the foundation of a quality assessment plan because it defines the purpose and intent of the program or unit. A good mission statement is specific to the program or unit while still aligning with the division’s and university’s missions. It should define the unit’s most critical roles and responsibilities clearly and concisely.
The "Outcomes/Objectives" highlight the key functions of a unit or the defined learning outcomes of a program. An assessment plan should identify 3-7 objectives, or key functions, all of which should align with the mission of the unit or program. Resources for writing strong learning outcomes or objectives can be found here.

Keep in mind that the university has established student learning outcomes for each degree level, which programs should consider as they write their learning outcomes.
A "Measure" is the WEAVEonline term for the tools utilized to evaluate the extent to which an outcome is achieved. There are two different types of measures: Direct and Indirect.

A "Direct Measure" is a tool utilized to assess quality. This could include the quality of student performance or of a service provided, or the volume of activity and production. Examples include rubrics or evaluations of performance or capacity. In academic degree programs, there should be a greater emphasis placed on direct measures.

An "Indirect Measure" typically relies on the perception of others in regard to the quality of a service or its popularity. These tools include surveys, interviews, focus groups, or participation statistics. These measures are best utilized in conjunction with a Direct Measure to triangulate data or as measures for programmatic objectives.
"Targets" indicate a level which will define success towards achieving a particular Outcome or Objective. Targets should be specific and measurable, aligned with the Measure being utilized, and represent a reasonable level of success. Examples include specific volume goals, a percentage of positive responses on a survey, or average turn-around time related to a received request. 

When possible, utilize "Targets" based on benchmark data or previous results to define the reasonable level of success. "Stretch Targets," or setting the Target higher than previously achieved, provide a good opportunity to work towards improvement.

Again, assessment is about continuous improvement based on data; there are no consequences at the university level for not achieving these targets.
"Findings" are the calculated results based on the data collected. "Findings" should provide solid evidence and enough detail to show that Targets were met, partially met, or not met.

There is not an expectation that all "Measures" be used every year, or even that every "Outcome/Objective" be measured every year. Some departments have developed rotations where they focus on one or two items in a given year. This is an acceptable approach and can help keep things manageable. If a Measure is not being used, be sure to mark “Not Reported this Cycle” when updating WEAVEonline. 
Each program is required to establish at least one new "Action Plan" near the conclusion of each assessment cycle. The "Action Plan(s)" should be informed by data collected through the assessment process. The big question is, now that data have been collected and findings entered, what are you going to do with them?

The Action Plan is a simple explanation of the data-informed response to the Findings. It may be simplest, and most clearly explained, by the following format: Based on our findings that ______, we will (explanation of tweak, adjustment, change, etc.).
Action Plans should:
  • Be informed by and clearly connected to data/findings collected through the assessment process
  • Action Plans should not:
    • Address changes to outcomes, measures, or other assessment processes or data collection methods
    • Be a continuation of existing practices (i.e., Targets met, continue existing practices)
      • Exception: If an action plan in a previous cycle was to implement some new practice(s) on a small scale or as a pilot, the new action plan, as long as it is based on findings, could be to make the practice larger in scale or a permanent fixture in the program.
For example:
Based on the finding that only 80% of survey respondents were satisfied with the timeliness of the dissemination of the End of Year Report, the timeline has been adjusted such that the deadline for final edits will be moved up 10 days to allow for faster completion.
Need help writing an Action Plan? Contact us.
The final element of the WEAVEonline assessment report are the Analysis Questions. These questions are embedded in WEAVEonline under the “Achievement Summary/Analysis” section and ask the following:
  • Based on the analysis of your findings, what changes are you currently making to improve your program?
Provide an update for completed or ongoing action plans from the previous year(s). Highlight your improvements.

The first question is a narrative explanation of the Action Plan and an opportunity to provide a bit more detail as to why the action is being taken. What in the analysis of the findings led to this decision? Why is this the best course of action to take? What other options were considered? Why do you think this action will lead to improvement? All of these questions can help to further explain the change being made.

The second question asks for an update of Action Plans established in previous assessment cycles. In other words, what happened when a particular action plan was implemented? This is an opportunity to explain how the action led to improvement, or even to explain why an action may not have had the desired effect.