The obligation placed on an educational institution by public officials, employers, and taxpayers for school officials to ensure that money invested in education has led to measurable learning. Accountability is often viewed as an important factor in education reform. An assessment system connected to accountability can help identify needs so that resources are equitably distributed.
Official recognition that an institution meets required standards. UTEP is accredited by the Southern Association of Colleges and Schools – Commission on Colleges (SACS-COC).
Alternatives to traditional, standardized, norm- or criterion-referenced testing. An alternative assessment might require students to answer an open-ended question, work out a solution to a problem, demonstrate a skill, or in some way produce work rather than select an answer from choices on a sheet of paper. Portfolios and observation of students are alternative forms of assessment.
Anonymity is guaranteed when neither the researcher nor the readers of the findings can identify a given response with a given respondent.
“Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards, and using the resulting information to document, explain and improve performance.” (Angelo, 1995)
“Assessment is any effort to gather, analyze, and interpret evidence which describes institutional, departmental, divisional, or agency effectiveness.” (Upcraft and Schuh, 2001)
“Assessment is the process of providing credible evidence of resources, implementation actions, and outcomes undertaken for the purpose of improving the effectiveness of instruction, programs, and services.” (Banta & Palomba, 2015)
A document that outlines the learning and program outcomes for the upcoming academic school year.
Attrition is the loss of students through means other than graduation.
An internal or external standard used to compare assessment findings. The measurement of individual or group performance against an established standard.
Biased sample means that a sample is selected in such a way that some members of the population are more likely than others to be picked for sample membership.
Bloom’s Taxonomy of Cognitive Objectives
Bloom’s taxonomy is a classification system used to define and distinguish different levels of human cognition—i.e., thinking, learning, and understanding. The taxonomy provides a useful structure in which to categorize assessment questions. There are six levels arranged in order of increasing complexity (1=low, 6=high):
- Remembering: Recalling information without necessarily understanding it. Includes behaviors such as describing, listing, identifying, and labeling.
- Understanding: Comprehending learned material and includes behaviors such as explaining, discussing, and interpreting.
- Applying: The ability to put ideas and concepts to work in solving problems. It includes behaviors such as demonstrating, showing, and making use of information.
- Analysing: Breaking down information into its component parts to see interrelationships and ideas. Related behaviors include differentiating, comparing, and categorizing.
- Evaluating: The ability to put parts together to form something original. It involves using creativity to compose or design something new.
- Creating: Judging the value of evidence based on definite criteria. Behaviors related to evaluation include convincing, criticizing, prioritizing, and recommending.
The assessment of outcomes structured into learning experiences occurring at the end of a program. The experiences involve demonstration of a comprehensive range of program outcomes through some type of product or performance.
An upper division class designed to help students integrate their knowledge. For assessment purposes, student work should be evaluated by faculty members responsible for the program, not just the instructor of the course. Capstone experiences and standardized exams are sometimes part of a capstone course.
Closing the Loop
Closing the loop in the cycle of assessment means taking the assessment data that has been collected and analyzed and using it to improve the programs and services that will continue to enhance student learning and program effectiveness.
Activities, programs, and learning experiences that supplement and complement the curricular or main syllabi activities.
A group of individuals closely resembling the control group in many variables but not receiving the factor under study and thereby serving as a comparison group when results are evaluated.
Responses are identifiable, but the data is kept contained by a specific set of reviewers.
A research method that is used to pull out themes from existing qualitative data.
The group in an experiment or study that does not receive treatment by the researchers and is then used as a benchmark to measure how the other tested subjects do.
Performance descriptors that are indicators of success. They indicate how well students will meet expectations of what they should be able to think, know or do. They are descriptive benchmarks against which performance is judged.
Attempts to uncover the strengths and weaknesses of an individual in terms of what they know or do not know, understand or don’t understand, or can or cannot do, when measured against a benchmark or standard. It is the comparison of outcomes to some pre-established criterion. Performance is compared to an expected level of mastery in a content area rather than to other students’ scores.
Culture of Assessment
Culture of assessment is an environment in which continuous improvement occurs through assessment that is expected and valued.
Culture of Evidence
Culture of evidence is an environment in which the use of research and assessment results to make data driven decisions is promoted.
An established score used to determine the minimum-level performance needed to pass a test or performance.
The process of systematically applying statistical and/or logical techniques to describe/illustrate, condense/recap, and evaluate data.
The process of gathering and measuring information on variables of interest, in an established systematic fashion that enables one to answer stated research questions, test hypotheses, and evaluate outcomes.
The excessive acquisition and reluctance to delete outdated material that is no longer valuable to the department or unit.
A dependent variable is something that depends on other factors.
Descriptive statistics describe or summarize a set of data. Measures of central tendency (mean, medium, and mode) and measure of dispersion (spread or variation around the central tendency) are two examples.
Direct measures of student learning that require students to display their knowledge and skills as they respond to the instrument itself. Examples include objective tests, written assignments, presentations, and classroom assignments.
Ethics is concerned with determining what actions or behaviors are “right” or “ought to be done/not done”. They are the moral principles that govern a person’s behavior.
“Evaluation refers to the process of determining the merit, worth, or value of something, or the product of that process.” (Scriven, 1999)
A collective group of participants assembled to provide their perspectives or feedback towards a specific subject or product. Effective focus groups are carefully planned group discussions conducted by trained moderators.
Observations which allow one to determine the degree to which students know or are able to do a given learning task, and which also identifies the part of the task that the student does not know or is unable to do.
Statements that define what we intend to achieve. They are general statements about what we need to accomplish in order to meet our mission or to serve our purpose. The purpose for designing a set of goals is to give a brief and broad picture of what we expect our students to know and do, and how our programs or services will help them accomplish that. Goals are further defined by being broken down into measurable objectives.
An independent variable stands alone and is not changed by other variables we are trying to measure.
Measurements about student knowledge, skills, attitudes, learning experiences, and opinions that involve perceptions of learning rather than actual demonstrations of outcome achievement. In indirect assessment, learning is inferred instead of being supported by direct evidence and students reflect on their learning rather than demonstrate it. Examples include focus groups, alumni surveys, employer surveys, and exit interviews.
Informed consent is typically a statement that participants sign before they participate in a study. The statement includes language that will guarantee participants certain rights.
Group of professionals from UTEP’s Student Affairs Division dedicated to creating a culture that uses assessment and evaluation practices to incorporate data in strategic planning, decision-making processes, and in the development, execution, and refinement of programs and services.
Key Performance Indicator (KPI)
Key Performance Indicators are metrics that track the performance of tasks critical to attaining important objectives in a time-sensitive manner. They help gauge the effectiveness of various functions and processes important to achieving organizational goals. For example, one key performance indicator of fall orientation could be fall course registration, or tracking the number of students who register successfully for at least one fall term course.
A collection of experiences, ideas, or reflections kept regularly for unit/departmental use. A reflective process often found to consolidate and enhance learning.
Learning communities are a variety of approaches that link or cluster classes, during a given term, often around an interdisciplinary theme, and enroll a common cohort of students.
Learning Outcome Statements
Statements that describes what a participant should obtain or should be capable of doing after the completion of a learning experience. More specifically, learning outcome statements are the knowledge, skills, attitudes, and habits of mind that students take with them from a learning experience. The ABCD Structure of a Learning Outcome includes
- Audience: To who does the outcome pertain?
- Behavior: What do we expect the audience to know or be able to do? The more specific the verb, the better the outcome.
- Conditions: Under what circumstances will the learning occur?
- Degree: At what level does the behavior need to be performed or to what criteria must the student perform?”
For example: “As a result of participating in the Edge presentation at New Student Orientation, students will be able to identify their top three core values.”
The Likert scale is an item-type used on objective measures that allow respondents to indicate their level of agreement with a statement by marking their response on a five-point scale, usually ranging from “strongly agree” to “strongly disagree”.
Involves the collection of data at different points in time. Data is collected on the same individuals over a period of time. It investigates development, learning, or other types of change in individuals over a specific time frame.
Simple or arithmetic average of a range of values or quantities, computed by dividing the total of all values by the number of values. It can be affected by extremely low or high scores.
The middle number in a given sequence of numbers, taken as the average of the two middle numbers when the sequence has an even number of numbers. For example, 12 is the median of 1, 9, 12, 28, 39.
The knowledge of one's own thinking processes and strategies, and the ability to consciously reflect and act on the knowledge of cognition to modify those processes and strategies.
Method of Assessment
Practices, tools, and instruments used for collecting data to determine if students are demonstrating desired learning outcomes.
Practices, tools, and instruments used for collecting data to determine if students are demonstrating desired learning outcomes.
A formal summary of the aims and values of a company, organization, or individual. A holistic vision of the values and philosophy of a department, program, unit or institution. The central purpose that provides focus, direction and destination for our work. It describes the purpose of our organization, who we serve, and our hopes. It is the “big picture of what we currently do” and why we exist.
That one value of a range of values that has the highest frequency as determined statistically.
Assessment that gathers information about a broad spectrum of abilities and skills.
A distribution of scores obtained from a norm group. The norm is the midpoint (or median) of scores or performance of the students in that group. Fifty percent will score above and fifty percent below the norm. A norm-referenced test is a standardized test that compares a student's test performance with that of a sample of similar students who have taken the same test.
Objectives or Outcomes
Brief statements that describe what our research population should learn by the completion of a specified time. The intended effect of a services or intervention. It is much more specific than a goal. They are statements that describe measurable expectations of what students should be able to think, know or do after completion of a given task or program.
A ranking scale ranging from a low of 1 to a high of 99 with 50 as the median score. A percentile rank indicates the percentage of a reference or norm group obtaining scores equal to or less than the test-taker's score. A percentile score indicates the test-taker's standing relative to the norm group standard.
Direct, systematic observation and rating of an individual’s performance, often an ongoing observation over a period of time, and typically involving the creation of products. The assessment should be a real-world performance with relevance to the learning process. Assessment of the performance is done using a rubric or analytic scoring guide to aid in objectivity. Performance-based assessment is a test of the ability to apply knowledge in a real-life setting.
A measurable value that validates how effectively a department or unit is achieving their learning objectives.
Population is a group of entities from which a sample is drawn or about which a conclusion is stated.
An assessment that is distributed to research subjects after the completion of a program to measure its effectiveness and to see if learning outcomes were met.
An assessment that is distributed to research subjects at the beginning of a program to gauge their existing knowledge and preparedness for the upcoming educational experience.
A complex assignment involving more than one type of activity and production.
Relating to, measuring, or measured by the quality of something rather than its quantity. Qualitative methods of assessment rely on descriptions rather than numbers. They typically involve asking participants broad, general questions, collecting the detailed views of the participants in the form of words or images, and analyzing the information for descriptions and themes. Examples include ethnographic field studies, logs, journals, participant observations, and open ended questions on interviews and surveys.
Relating to, measuring, or measured by the quantity of something rather than its quality. Quantitative methods of assessment rely on numerical scores or ratings. They use structured, predetermined response options that can be summarized into meaningful numbers and analyzed statistically. It is the assignment of numbers to objects, events or observations according to some rule. Examples include tracking, experiments, surveys, testing instruments, checklists and rubrics.
The breakdown of an aggregate of percentile rankings into four categories: the zero to 25th percentile, 26th to 50th percentile, etc.
Random sample is a sampling method in which each element has an equal chance of selection independent of any other event in the selection process.
“A truth-seeking activity which contributes to knowledge, aimed at describing or explaining the world.” (Coryn, 2006)
The measure of consistency for an assessment instrument. The instrument should yield similar results over time with similar populations in similar circumstances.
Response bias is a type of cognitive bias that can affect the results of a statistical survey if respondents answer questions in the way they think the questioner wants them to answer rather than according to their true beliefs.
The rate of completion or return; the number of participants that completed the study divided by the number in the sampled population.
The evidence or data produced from the completion of assessments.
A rubric is a scoring tool developed to measure performance, achievement, or mastery on a given activity. A set of criteria specifying the characteristics of a learning outcome and the levels of achievement in each characteristic.
A sample is a subgroup of a population selected to participate in an activity, program or service. The assessment results from the sample are used to generalize to the larger population from which the sample was drawn. Examples of sampling methods include simple random, stratified sampling, and cluster sampling.
Describes the possible difference in findings and results if one were able to obtain valid responses from the entire population. The more people in the sample, the lower the sample error will be.
Saturation is referred to in qualitative research when a researcher determines that more data will not provide any new information on the topic under study.
Rules for assigning a score or the dimensions of proficiency in performance used to describe a student's response to a task. May include rating scales, checklists, answer keys, and other scoring tools.
Selection bias is bias that occurs when the method of selecting study participants means that they come from a particular (skewed) social or economic group, and not from others.
A process in which a student engages in a systematic review of a performance, usually for the purpose of improving future performance. May involve comparison with a standard (established criterion), critiquing one's own work, or may be a simple description of the performance. Reflection, self-evaluation, and metacognition, are related terms.
A person, group or organization that has interest or concern in an organization or institution.
Standard is the broadest of a family of terms referring to statements of expectations for student learning, including content standards, performance standards, and benchmarks. Agreed upon value used to measure the quality of student performance, instructional methods, curriculum, etc.
A measure used to quantify the amount of variation or dispersion in a set of data values. A measure of how spread out numbers are. It is equal to the square root of the variance, which is the average of the squared differences from the Mean.
Statistical significance is a general term referring to the unlikeliness that relationships observed in a sample could be attributed to sampling error alone. The statistical significance of a relationship that is observed in a set of sample data is always expressed in terms of probabilities. For example, significant at the .05 level simply means that the probability of a relationship as strong as the observed one being attributable to sample error alone is no more than 5 in 100.
Stop out occurs when a student does not enroll in a program of study in consecutive terms.
Strategic planning refers to the process in which organizations engage in reviewing their mission statement and goals, and then designing and adopting action steps to achieve their goals.
A test in which the impression or opinion of the assessor determines the score or evaluation of performance. A test in which the answers cannot be known or prescribed in advance.
Evaluation of student learning that occurs at the end of an instructional program or course; compares student learning against some standard or benchmark.
An assessment that gathers one’s opinions and recommendations on certain aspects of a program. Surveys can be paper or electronic and are used to collect data from many people quickly and easily. Most often surveys are questionnaires or structured interviews, requiring a specific set of questions, appropriate instructions and a carefully constructed data collection instrument.
The collection of data via multiple methods in order to determine if the results show a consistent outcome. The process of corroborating evidence from different individuals, types of data, or methods of data collections in descriptions and themes in qualitative research.
Measures the desired performance and appropriate inferences can be drawn from the assessment results.
A road map, indicating both what an institution wants to become and guiding transformational initiatives by setting a defined direction for the institution’s growth. A Vision Statement clarifies what the institution should look like and how it should behave as it fulfills its mission.
Campus Labs Student Affairs Assessment Credential Online course material