Frequently Asked Questions
Academic Departments and Programs
- What is being required and when?
- Should we include intangibles?
- How much detail needs to be included in our statements of student learning goals and objectives?
- Are we supposed to set goals for majors, courses, or what?
- What level of detail goes to Middle States?
- What happened to Course Evaluations?
- What is the difference between "Direct" and "Indirect" assessment?
- Can graded coursework be used in assessment?
- Are we supposed to address students' improvement or absolute levels of abilities?
- How rigorous does this assessment work need to be?
- Don't we do this already?
Each academic department articulated its departmental goals and objectives for student learning in 2011-12, and began assessing them in 2012-13. Now each department should be fully engaged in an annual cycle of assessment, as outlined in our Academic Assessment Plan. The Provost's office provides feedback, guidance, and assistance with these efforts.
As a Liberal Arts institution, Swarthmore places great value on many qualities that are intangible. Departments' goals should reflect what is important to them, and so we have taken the view that we ought to include these. Certainly there are things that can't be measured, although a variety of evidence might give us confidence that we're on the right track. But omitting them would be incomplete. For some interesting reading on "Essential Learning Outcomes" of a Liberal Education, see AAC&U's VALUE Initiative.
It is logical to start with broad goals, and then to refine each into more detailed objectives. The more detail you are able to articulate, the better positioned you will be when you set out to assess them. However, it is fine to leave a goal statement at somewhat of a broad level until your department is ready to engage with it more fully. Once that happens, we would expect the goal to be more clearly defined.
We should articulate goals for majors, minors, and courses (which would include non-majors). Many departments have are still focusing on goals for majors, which is fine.
We report on assessment to Middle States in narrative form, describing our processes and activities, providing examples, and referencing supplemental materials as appropriate. They do not require departmental submissions directly to them, but would expect to see evidence of our work in published materials, on our website, and in internal reporting that might be reviewed by an external team of evaluators for our decennial self-studies and interim reports.
Course evaluations should continue to be administered. Please see the Provost's clarification about our current policy. However, course evaluations constitute indirect evidence of student learning — they tell us about student perceptions of what has been learned, which are important, but they don't show in a direct or objective way what has actually been learned. Therefore departments should also be assessing student learning in direct ways.
Direct assessment requires looking at actual student work, such as projects, exams, and presentations (graded to reflect individual learning goals), for evidence of learning. Indirect assessment uses other information that suggests that learning is taking place, such as self-reported learning gains in a course evaluation, or graduation rates. Middle States provides a useful sheet of examples that helps to illustrate, called "Evidence of Student Learning." [pdf] Indirect assessment is valuable and important, but it cannot by a substitute for Direct assessment.
Traditionally, a grade reflects a combination of achievements, such as a range of materials covered on an exam, or multiple competencies reflected in a paper. In this way, they are not useful for assessment. But if a grade can be formulated (compartmentalized, deconstructed, unpacked...) in a way that reflects individual learning goals, it could potentionally yield very useful feedback, when summed across all students. In this configuration grades would be entirely appropriate for assessment. See under Resources-Tools: Embedded assignments, and Rubrics.
It depends on your goals. Your department may have a goal that all students will improve in a particular competency, and not be concerned about the absolute level of achievement. Another goal may suggest that there is a threshold level of achievement that all students must meet.
The purpose of assessment is to provide for ourselves compelling information that will help us understand what students are learning, and where changes might be needed and effective. We do not need scientifically developed procedures, double-blind studies, multiple independent raters, etc., or to produce publishable, generalizable research. We do need good information that helps us in making curricular decisions. Do what you reasonably can to maximize objectivity and rigor, but assessment should support your primary work — not deter you from it.
Swarthmore has a long history of self-reflection and continuous improvement. Our Honors Program and capstone experiences in every major are evidence of our commitment to meaningful assessment. However, in other areas we are not always as systematic as we might be, nor do we document the careful consideration that is behind curricular changes. The framework of assessment provides a way of linking pedagogy and results that is useful for both our own illumination and for others to better understand what we do.