Glossary
Accountability
The obligation placed on an educational institution by public officials, employers, and taxpayers for school officials to ensure that money invested in education has led to measurable learning. Accountability is often viewed as an important factor in education reform. An assessment system connected to accountability can help identify needs so that resources are equitably distributed.
Accreditation
Official recognition that an institution meets required standards. UTEP is accredited by the Southern Association of Colleges and Schools – Commission on Colleges (SACS-COC).
Alternative Assessment
Alternatives to traditional, standardized, norm- or criterion-referenced testing. An alternative assessment might require students to answer an open-ended question, work out a solution to a problem, demonstrate a skill, or in some way produce work rather than select an answer from choices on a sheet of paper. Portfolios and observation of students are alternative forms of assessment.
Anonymity
Anonymity is guaranteed when neither the researcher nor the readers of the findings can identify a given response with a given respondent.
Assessment
“Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards, and using the resulting information to document, explain and improve performance.” (Angelo, 1995)
“Assessment is any effort to gather, analyze, and interpret evidence which describes institutional, departmental, divisional, or agency effectiveness.” (Upcraft and Schuh, 2001)
“Assessment is the process of providing credible evidence of resources, implementation actions, and outcomes undertaken for the purpose of improving the effectiveness of instruction, programs, and services.” (Banta & Palomba, 2015)
Assessment Plan
A document that outlines the learning and program outcomes for the upcoming academic school year.
Attrition
Attrition is the loss of students through means other than graduation.
Benchmark
An internal or external standard used to compare assessment findings. The measurement of individual or group performance against an established standard.
Biased sample
Biased sample means that a sample is selected in such a way that some members of the population are more likely than others to be picked for sample membership.
Bloom’s Taxonomy of Cognitive Objectives
Bloom’s taxonomy is a classification system used to define and distinguish different levels of human cognition—i.e., thinking, learning, and understanding. The taxonomy provides a useful structure in which to categorize assessment questions. There are six levels arranged in order of increasing complexity (1=low, 6=high):
- Remembering: Recalling information without necessarily understanding it. Includes behaviors such as describing, listing, identifying, and labeling.
- Understanding: Comprehending learned material and includes behaviors such as explaining, discussing, and interpreting.
- Applying: The ability to put ideas and concepts to work in solving problems. It includes behaviors such as demonstrating, showing, and making use of information.
- Analysing: Breaking down information into its component parts to see interrelationships and ideas. Related behaviors include differentiating, comparing, and categorizing.
- Evaluating: The ability to put parts together to form something original. It involves using creativity to compose or design something new.
- Creating: Judging the value of evidence based on definite criteria. Behaviors related to evaluation include convincing, criticizing, prioritizing, and recommending.
Capstone Assessment
The assessment of outcomes structured into learning experiences occurring at the end of a program. The experiences involve demonstration of a comprehensive range of program outcomes through some type of product or performance.
Capstone Course
An upper division class designed to help students integrate their knowledge. For assessment purposes, student work should be evaluated by faculty members responsible for the program, not just the instructor of the course. Capstone experiences and standardized exams are sometimes part of a capstone course.
Closing the Loop
Closing the loop in the cycle of assessment means taking the assessment data that has been collected and analyzed and using it to improve the programs and services that will continue to enhance student learning and program effectiveness.
Co-curricular
Activities, programs, and learning experiences that supplement and complement the curricular or main syllabi activities.
Comparison group
A group of individuals closely resembling the control group in many variables but not receiving the factor under study and thereby serving as a comparison group when results are evaluated.
Confidential
Responses are identifiable, but the data is kept contained by a specific set of reviewers.
Content Analysis
A research method that is used to pull out themes from existing qualitative data.
Control Group
The group in an experiment or study that does not receive treatment by the researchers and is then used as a benchmark to measure how the other tested subjects do.
Criteria
Performance descriptors that are indicators of success. They indicate how well students will meet expectations of what they should be able to think, know or do. They are descriptive benchmarks against which performance is judged.
Criterion-Referenced Assessment
Attempts to uncover the strengths and weaknesses of an individual in terms of what they know or do not know, understand or don’t understand, or can or cannot do, when measured against a benchmark or standard. It is the comparison of outcomes to some pre-established criterion. Performance is compared to an expected level of mastery in a content area rather than to other students’ scores.
Culture of Assessment
Culture of assessment is an environment in which continuous improvement occurs through assessment that is expected and valued.
Culture of Evidence
Culture of evidence is an environment in which the use of research and assessment results to make data driven decisions is promoted.
Cut Score
An established score used to determine the minimum-level performance needed to pass a test or performance.
Data Analysis
The process of systematically applying statistical and/or logical techniques to describe/illustrate, condense/recap, and evaluate data.
Data Collection
The process of gathering and measuring information on variables of interest, in an established systematic fashion that enables one to answer stated research questions, test hypotheses, and evaluate outcomes.
Data Hoarding
The excessive acquisition and reluctance to delete outdated material that is no longer valuable to the department or unit.
Dependent Variable
A dependent variable is something that depends on other factors.
Descriptive Statistics
Descriptive statistics describe or summarize a set of data. Measures of central tendency (mean, medium, and mode) and measure of dispersion (spread or variation around the central tendency) are two examples.
Direct Assessment
Direct measures of student learning that require students to display their knowledge and skills as they respond to the instrument itself. Examples include objective tests, written assignments, presentations, and classroom assignments.
Ethics
Ethics is concerned with determining what actions or behaviors are “right” or “ought to be done/not done”. They are the moral principles that govern a person’s behavior.
Evaluation
“Evaluation refers to the process of determining the merit, worth, or value of something, or the product of that process.” (Scriven, 1999)
Focus Group
A collective group of participants assembled to provide their perspectives or feedback towards a specific subject or product. Effective focus groups are carefully planned group discussions conducted by trained moderators.
Formative Assessment
Observations which allow one to determine the degree to which students know or are able to do a given learning task, and which also identifies the part of the task that the student does not know or is unable to do.
Goals
Statements that define what we intend to achieve. They are general statements about what we need to accomplish in order to meet our mission or to serve our purpose. The purpose for designing a set of goals is to give a brief and broad picture of what we expect our students to know and do, and how our programs or services will help them accomplish that. Goals are further defined by being broken down into measurable objectives.
Independent Variable
An independent variable stands alone and is not changed by other variables we are trying to measure.
Indirect Assessment
Measurements about student knowledge, skills, attitudes, learning experiences, and opinions that involve perceptions of learning rather than actual demonstrations of outcome achievement. In indirect assessment, learning is inferred instead of being supported by direct evidence and students reflect on their learning rather than demonstrate it. Examples include focus groups, alumni surveys, employer surveys, and exit interviews.
Informed Consent
Informed consent is typically a statement that participants sign before they participate in a study. The statement includes language that will guarantee participants certain rights.
Impact Team
Group of professionals from UTEP’s Student Affairs Division dedicated to creating a culture that uses assessment and evaluation practices to incorporate data in strategic planning, decision-making processes, and in the development, execution, and refinement of programs and services.
Key Performance Indicator (KPI)
Key Performance Indicators are metrics that track the performance of tasks critical to attaining important objectives in a time-sensitive manner. They help gauge the effectiveness of various functions and processes important to achieving organizational goals. For example, one key performance indicator of fall orientation could be fall course registration, or tracking the number of students who register successfully for at least one fall term course.
Journaling
A collection of experiences, ideas, or reflections kept regularly for unit/departmental use. A reflective process often found to consolidate and enhance learning.
Learning Communities
Learning communities are a variety of approaches that link or cluster classes, during a given term, often around an interdisciplinary theme, and enroll a common cohort of students.
Learning Outcome Statements
Statements that describes what a participant should obtain or should be capable of doing after the completion of a learning experience. More specifically, learning outcome statements are the knowledge, skills, attitudes, and habits of mind that students take with them from a learning experience. The ABCD Structure of a Learning Outcome includes
- Audience: To who does the outcome pertain?
- Behavior: What do we expect the audience to know or be able to do? The more specific the verb, the better the outcome.
- Conditions: Under what circumstances will the learning occur?
- Degree: At what level does the behavior need to be performed or to what criteria must the student perform?”
For example: “As a result of participating in the Edge presentation at New Student Orientation, students will be able to identify their top three core values.”
Likert Scale
The Likert scale is an item-type used on objective measures that allow respondents to indicate their level of agreement with a statement by marking their response on a five-point scale, usually ranging from “strongly agree” to “strongly disagree”.
Longitudinal Study
Involves the collection of data at different points in time. Data is collected on the same individuals over a period of time. It investigates development, learning, or other types of change in individuals over a specific time frame.
Mean
Simple or arithmetic average of a range of values or quantities, computed by dividing the total of all values by the number of values. It can be affected by extremely low or high scores.
Median
The middle number in a given sequence of numbers, taken as the average of the two middle numbers when the sequence has an even number of numbers. For example, 12 is the median of 1, 9, 12, 28, 39.
Metacognition
The knowledge of one's own thinking processes and strategies, and the ability to consciously reflect and act on the knowledge of cognition to modify those processes and strategies.
Method of Assessment
Practices, tools, and instruments used for collecting data to determine if students are demonstrating desired learning outcomes.
Metrics
Practices, tools, and instruments used for collecting data to determine if students are demonstrating desired learning outcomes.
Mission Statement
A formal summary of the aims and values of a company, organization, or individual. A holistic vision of the values and philosophy of a department, program, unit or institution. The central purpose that provides focus, direction and destination for our work. It describes the purpose of our organization, who we serve, and our hopes. It is the “big picture of what we currently do” and why we exist.
Mode
That one value of a range of values that has the highest frequency as determined statistically.
Multidimensional Assessment
Assessment that gathers information about a broad spectrum of abilities and skills.
Norm
A distribution of scores obtained from a norm group. The norm is the midpoint (or median) of scores or performance of the students in that group. Fifty percent will score above and fifty percent below the norm. A norm-referenced test is a standardized test that compares a student's test performance with that of a sample of similar students who have taken the same test.
Objectives or Outcomes
Brief statements that describe what our research population should learn by the completion of a specified time. The intended effect of a services or intervention. It is much more specific than a goal. They are statements that describe measurable expectations of what students should be able to think, know or do after completion of a given task or program.
Percentile
A ranking scale ranging from a low of 1 to a high of 99 with 50 as the median score. A percentile rank indicates the percentage of a reference or norm group obtaining scores equal to or less than the test-taker's score. A percentile score indicates the test-taker's standing relative to the norm group standard.
Performance-Based Assessment
Direct, systematic observation and rating of an individual’s performance, often an ongoing observation over a period of time, and typically involving the creation of products. The assessment should be a real-world performance with relevance to the learning process. Assessment of the performance is done using a rubric or analytic scoring guide to aid in objectivity. Performance-based assessment is a test of the ability to apply knowledge in a real-life setting.
Performance Indicator
A measurable value that validates how effectively a department or unit is achieving their learning objectives.
Population
Population is a group of entities from which a sample is drawn or about which a conclusion is stated.
Post-Test
An assessment that is distributed to research subjects after the completion of a program to measure its effectiveness and to see if learning outcomes were met.
Pre-Test
An assessment that is distributed to research subjects at the beginning of a program to gauge their existing knowledge and preparedness for the upcoming educational experience.
Project
A complex assignment involving more than one type of activity and production.
Qualitative Data
Relating to, measuring, or measured by the quality of something rather than its quantity. Qualitative methods of assessment rely on descriptions rather than numbers. They typically involve asking participants broad, general questions, collecting the detailed views of the participants in the form of words or images, and analyzing the information for descriptions and themes. Examples include ethnographic field studies, logs, journals, participant observations, and open ended questions on interviews and surveys.
Quantitative Data
Relating to, measuring, or measured by the quantity of something rather than its quality. Quantitative methods of assessment rely on numerical scores or ratings. They use structured, predetermined response options that can be summarized into meaningful numbers and analyzed statistically. It is the assignment of numbers to objects, events or observations according to some rule. Examples include tracking, experiments, surveys, testing instruments, checklists and rubrics.
Quartile
The breakdown of an aggregate of percentile rankings into four categories: the zero to 25th percentile, 26th to 50th percentile, etc.
Random Sample
Random sample is a sampling method in which each element has an equal chance of selection independent of any other event in the selection process.
Research
“A truth-seeking activity which contributes to knowledge, aimed at describing or explaining the world.” (Coryn, 2006)
Reliability
The measure of consistency for an assessment instrument. The instrument should yield similar results over time with similar populations in similar circumstances.
Response Bias
Response bias is a type of cognitive bias that can affect the results of a statistical survey if respondents answer questions in the way they think the questioner wants them to answer rather than according to their true beliefs.
Response Rate
The rate of completion or return; the number of participants that completed the study divided by the number in the sampled population.
Results
The evidence or data produced from the completion of assessments.
Rubric
A rubric is a scoring tool developed to measure performance, achievement, or mastery on a given activity. A set of criteria specifying the characteristics of a learning outcome and the levels of achievement in each characteristic.
Sample
A sample is a subgroup of a population selected to participate in an activity, program or service. The assessment results from the sample are used to generalize to the larger population from which the sample was drawn. Examples of sampling methods include simple random, stratified sampling, and cluster sampling.
Sampling Error
Describes the possible difference in findings and results if one were able to obtain valid responses from the entire population. The more people in the sample, the lower the sample error will be.
Saturation
Saturation is referred to in qualitative research when a researcher determines that more data will not provide any new information on the topic under study.
Scoring Criteria
Rules for assigning a score or the dimensions of proficiency in performance used to describe a student's response to a task. May include rating scales, checklists, answer keys, and other scoring tools.
Selection Bias
Selection bias is bias that occurs when the method of selecting study participants means that they come from a particular (skewed) social or economic group, and not from others.
Self-Assessment
A process in which a student engages in a systematic review of a performance, usually for the purpose of improving future performance. May involve comparison with a standard (established criterion), critiquing one's own work, or may be a simple description of the performance. Reflection, self-evaluation, and metacognition, are related terms.
Stakeholders
A person, group or organization that has interest or concern in an organization or institution.
Standard
Standard is the broadest of a family of terms referring to statements of expectations for student learning, including content standards, performance standards, and benchmarks. Agreed upon value used to measure the quality of student performance, instructional methods, curriculum, etc.
Standard Deviation
A measure used to quantify the amount of variation or dispersion in a set of data values. A measure of how spread out numbers are. It is equal to the square root of the variance, which is the average of the squared differences from the Mean.
Statistical Significance
Statistical significance is a general term referring to the unlikeliness that relationships observed in a sample could be attributed to sampling error alone. The statistical significance of a relationship that is observed in a set of sample data is always expressed in terms of probabilities. For example, significant at the .05 level simply means that the probability of a relationship as strong as the observed one being attributable to sample error alone is no more than 5 in 100.
Stop Out
Stop out occurs when a student does not enroll in a program of study in consecutive terms.
Strategic Planning
Strategic planning refers to the process in which organizations engage in reviewing their mission statement and goals, and then designing and adopting action steps to achieve their goals.
Subjective Test
A test in which the impression or opinion of the assessor determines the score or evaluation of performance. A test in which the answers cannot be known or prescribed in advance.
Summative Assessment
Evaluation of student learning that occurs at the end of an instructional program or course; compares student learning against some standard or benchmark.
Survey
An assessment that gathers one’s opinions and recommendations on certain aspects of a program. Surveys can be paper or electronic and are used to collect data from many people quickly and easily. Most often surveys are questionnaires or structured interviews, requiring a specific set of questions, appropriate instructions and a carefully constructed data collection instrument.
Triangulation
The collection of data via multiple methods in order to determine if the results show a consistent outcome. The process of corroborating evidence from different individuals, types of data, or methods of data collections in descriptions and themes in qualitative research.
Validity
Measures the desired performance and appropriate inferences can be drawn from the assessment results.
Vision Statement
A road map, indicating both what an institution wants to become and guiding transformational initiatives by setting a defined direction for the institution’s growth. A Vision Statement clarifies what the institution should look like and how it should behave as it fulfills its mission.
References:
Campus Labs Student Affairs Assessment Credential Online course material
University of Wisconsin Stevens Point Division of Student Affairs Glossary of Terms online
Portland State University Enrollment Management and Student Affairs Glossary of Terms online
University of Nebraska Kearney Division of Academic and Student Affairs Glossary of Terms online
University of Wisconsin Milwaukee Division of Student Affairs Glossary of Terms online