Competency N

Evaluate programs and services using measurable criteria.

Section 1: Introduction

A program can be carefully planned, well funded, and thoughtfully designed and still fail to accomplish what it was meant to do. But program assessment is sometimes treated as an afterthought, rather than being integrated into program design from the beginning, particularly when time, funding, or organizational capacity is limited. McClure (2018) argues that planning and evaluation should be treated as two sides of the same coin: a plan sets out what an organization intends to accomplish, and evaluation is what determines whether it actually did. He defines evaluation as the process of determining “the success, impact, results, costs, outcomes, or other factors related to a library activity, program, service, or resource use” (p. 179). Linking intentions and evidence is at the heart of meaningful program assessment.

This competency focuses specifically on assessing services and programs, not collections or content, and this distinction is important. Evaluating a collection tells us what an organization has; evaluating a service tells us whether those resources are actually reaching and helping the people the organization intends to serve. McClure distinguishes between several kinds of assessment criteria, including efficiency, how well a program uses its resources, and effectiveness, whether a program actually accomplishes the objectives it was designed to achieve. Crucially, measuring effectiveness is only possible when a program is built around clearly defined goals from the start. He also notes that evaluation can be summative, occurring at the end of a program or service, or done on an ongoing basis as a form of monitoring.

Clarke (2022) offers a useful way of thinking about where assessment fits into a larger process, through what she calls the design thinking cycle: investigating a problem, planning possible solutions, developing and testing a prototype, and evaluating the result. Critically, Clarke frames the evaluative phase not as an endpoint, but as something that “feeds back into investigation for continued improvement,” used to “expose and identify new problems” (“The Design Thinking Process” section) and restart the cycle. This cycle shapes how I think about the relationship between planning and assessment: they are not two separate, one-time stages, but part of a feedback loop through which a program continually cycles as conditions and user needs change.

McDonald (2022) grounds this in more concrete technique. She describes the “Five Whys,” a method of asking why a problem occurred five times in succession, each answer leading to the next question, as a way of tracing a complaint or problem back to its underlying cause. She also points to heuristic evaluation, in which a small set of evaluators judge an interface or service against a recognized set of usability principles. Both approaches highlight that meaningful assessment does not necessarily require extensive time or resources. What matters is having a deliberate method for asking what is happening, why it is happening, and whether the service is accomplishing what it is supposed to accomplish. McDonald’s broader argument—that a willingness to evaluate and reevaluate a service’s content, scope, method, and extent is a hallmark of an organization that takes its users’ needs seriously—captures why assessment is crucial, regardless of the specific evaluation techniques used.

These writings, along with my experiences evaluating websites, software tools, and search and discovery interfaces throughout my coursework, have shifted my understanding of assessment from a discrete task to an ongoing way of thinking. Throughout my career, I will treat assessment as a state of mind that shapes the way I design or maintain any program or service. Good planning establishes intentions; good assessment tests those intentions against the outcomes; and the resulting data can help us change our approach so that we can continually improve how we serve our users.

References

Section 2: Evidence

My understanding of assessment comes from coursework designing evaluation criteria before a program launches, evaluating an existing service, applying a structured methodology to assess a tool, and directly assessing an existing information system against real user needs.

My first piece of evidence is a strategic planning document from INFO 204, in which I wrote a goal, objective, target population, and measurable outcomes for improving researcher satisfaction with a collection’s online discovery interface. I built concrete, measurable outcome indicators, including a target that 75% of researchers would report increased satisfaction with the interface by a specific date, along with a detailed timeline of output measurements, from forming a UX working group to launching an initial user survey to implementing and re-testing design changes. This artifact demonstrates my understanding that effective assessment must be designed before a program launches, not added afterward: without pre-established, measurable criteria and a plan for gathering both baseline and follow-up data, there would be no way to determine whether the completed program actually improved anything.

My second piece of evidence is a website evaluation and redesign report from INFO 202, in which I assessed the information architecture of a nonprofit movie theater’s website and proposed specific improvements. (The report is written in “we” throughout, following the assignment’s convention of framing the deliverable as if produced by a design team, though I completed the work individually.) My evaluation identified concrete usability problems: redundant navigation categories that provided little additional information over each other, confusingly similar labels for pages aimed at entirely different audiences (prospective members versus prospective business partners), a lengthy showtimes page with no filtering mechanism despite listing dozens of events, and an active blog that was not linked from the homepage at all despite containing content users would likely want. My recommendations addressed each of these problems directly, including clearer, action-oriented labeling, reorganized navigation groupings, and specific features like a calendar filter and an integrated site search tool. The report closes with a discussion of methods for evaluating whether these changes actually improve the site going forward, including usability testing to identify where users get stuck, A/B testing to compare specific design variations, and user surveys to gather direct feedback, with a recommendation that researchers explain to participants how their feedback will actually be used. This artifact demonstrates my understanding of evaluation as an ongoing practice: identifying current problems is only the first step, and any proposed improvement should come with a plan for verifying, through direct evidence, whether it has actually bettered user experience.

My third piece of evidence is a structured software evaluation from INFO 281, in which I assessed Oxygen XML Editor against a set of explicit criteria before recommending it for institutional adoption. Instead of reviewing the tool impressionistically, I evaluated it against named categories, including installation ease and cost, testability through a free trial, learnability, documentation quality, technical support access, and the strength of its user community, recording a clear yes/no judgment for each criterion along with supporting notes. This process revealed real trade-offs a less structured review might have missed: the software’s advanced features, like line-by-line validation debugging and the ability to apply refactoring across multiple documents at once, came with a genuine learning curve compared to a simpler tool I had used previously, and its enterprise licensing costs were significant despite generous trial and discounted academic access options. This artifact demonstrates my understanding that assessment is only useful when it is criteria-based and transparent. By using a consistent rubric, we can produce recommendations that can be defended, reproduced, or applied to future evaluations of similar tools.

My fourth piece of evidence is a group evaluation of a peer team’s pen database, created for INFO 202, in which our group assessed another team’s information retrieval system built on the Caspio platform. My specific written contribution was the evaluation of the database’s search page, and I contributed more broadly to the group’s discussion of the project’s field design and controlled vocabulary choices. In my search page evaluation, I identified a mismatch between how the database’s fields were structured and how a less experienced user would actually search: a user looking for a non-smudging pen, a use case the original team’s own statement of purpose mentioned, would have no way to find one through a free-text ink field, since manufacturers rarely use the term “non-smudging” on packaging. I recommended that fields like pen type, ink type, and tip type be converted from free-text entry to controlled drop-down lists, which would reduce inconsistent data entry and also expose users to the correct technical vocabulary needed to search effectively, functioning as a built-in vocabulary aid rather than assuming users already knew the right terms. I also evaluated the system’s price range field, noting that its top category, “$10+,” failed to discriminate between a $12 pen and an $850 pen, a real problem for the specialized collector audience the database was designed to serve. This artifact demonstrates my ability to assess an existing system against real user needs, identifying where a service’s design creates barriers between users and the information they’re trying to find, and translating that assessment into specific, actionable recommendations.

Section 3: Conclusion

Assessment enables information professionals to determine whether services are meeting their intended goals and to use evidence to make those services more effective. These four pieces of evidence demonstrate different forms of assessment: designing measurable outcome indicators before a program launches, evaluating an existing public-facing service to identify concrete problems and propose testable improvements, applying a structured rubric to assess a tool, and directly assessing an information system against real user needs and search behavior. In my future career, I intend to build assessment into services from the outset, setting measurable goals before a program launches and building in a real plan to evaluate results. To stay current in this area, I plan to consult the American Library Association’s Library Assessment Repository, the Society of American Archivists’ Standardized Statistical Measures and Metrics for Public Services in Archival Repositories and Special Collections Libraries,as well as the the Association of College and Research Libraries’s Project Outcome toolkit, all of which are resources specifically focused on developing and applying assessment methods in library and information settings.