Frequently Asked Questions
Academic Departments and Programs
- What is being required and when?
- Should we include intangibles?
- How much detail needs to be included in our statements of student learning goals and objectives?
- Are we supposed to set goals for majors, courses, or what?
- What level of detail goes to Middle States?
- What happened to Course Evaluations?
- What is the difference between "Direct" and "Indirect" assessment?
- Can graded coursework be used in assessment?
- Are we supposed to address students' improvement or absolute levels of abilities?
- How rigorous does this assessment work need to be?
- Don't we do this already?
For now, academic departments are only being asked to include departmental goals and objectives along with the reports submitted after the 2011-12 end-of-year meetings. In 2012-13 departments will be asked to focus on one or two goals or objectives to assess. The Provost's Office will provide guidance and assistance.
As a Liberal Arts institution, Swarthmore places great value on many qualities that are intangible. Departments' goals should reflect what is important to them, and so ought to include these. Certainly there are things that can't be measured, although a variety of evidence might give us confidence that we're on the right track. But omitting them would be incomplete. For some interesting reading on "Essential Learning Outcomes" of a Liberal Education, see AAC&U's VALUE Initiative.
It is logical to start with broad goals, and then to refine each into more detailed objectives. The more detail you are able to articulate, the better positioned you will be when at a later point you set out to assess them. However, it is fine to leave a goal statement at somewhat of a broad level until your department is ready to engage with it more fully.
Ideally we will articulate goals for majors, minors, and courses (which would include non-majors). But if it makes sense for your department to start with a subset, that's fine.
We report on assessment to Middle States in narrative form, describing our processes and activities, providing examples, and referencing supplemental materials as appropriate. They do not require departmental submissions directly to them, but would expect to see evidence of our work in published materials, on our website, and in internal reporting that might be reviewed by an external team of evaluators for our decennial self-studies.
Course evaluations should continue to be administered. However, these constitute only indirect evidence of student learning — they tell us about student perceptions of what has been learned, which are important, but they don't show in a direct or objective way what has actually been learned. Therefore we should also be assessing student learning in direct ways.
Direct assessment requires looking at actual student work, such as projects, exams, and presentations (graded to reflect learning goals), for evidence of learning. Indirect assessment uses other information that suggests that learning is taking place, such as self-reported learning gains in a course evaluation, or graduation rates. Middle States provides a useful sheet of examples that helps to illustrate, called "Evidence of Student Learning." [pdf]
Traditionally, a grade reflects a combination of achievements, such as a range of materials covered on an exam, or multiple competencies reflected in a paper. In this way, they are not useful for assessment. But if a grade can be formulated (compartmentalized, deconstructed, unpacked...) in a way that reflects individual learning goals, it could potentionally yield very useful feedback, when summed across all students. In this configuration grades would be entirely appropriate for assessment. See under Resources-Tools: Embedded assignments, and Rubrics.
It depends on your goals. Your department may have a goal that all students will improve in a particular competency, and not be concerned about the absolute level of achievement. Another goal may suggest that there is a threshold level of achievement that all students must meet.
The purpose of assessment is to provide for ourselves compelling information that will help us understand what students are learning, and where changes might be effective. We do not need scientifically developed procedures, double-blind studies, multiple independent raters, etc., or to produce publishable, generalizable research. We do need good information that helps us in making curricular decisions. Do what you reasonably can to maximize objectivity, but assessment should support your primary work — not deter you from it.
Swarthmore has a long history of self-reflection and continuous improvement. Our Honors Program and capstone experiences in every major are evidence of our commitment to meaningful assessment. However, in other areas we are not always as systematic as we might be, nor do we document the careful consideration that is behind curricular changes. The framework of assessment provides a way of linking pedagogy and results that is useful for both our own illumination and for others to better understand what we do.