Speech Communication
Department
Assessment Report
2002-2003
Assessment Schedule:
|
Year |
BA/BS |
BST |
MA/MS |
GEN ED |
|
2000-2001 |
LO 1 |
LO 1 |
LO 1 |
|
|
2001-2002 |
LO 2 |
LO 2 |
LO 2 |
LO 1 |
|
2002-2003 |
LO 3 |
LO 3 |
LO 3 |
|
|
2003-2004 |
LO 4 |
LO 4 |
LO 4 |
|
|
2004-2005 |
LO 5 |
LO 1 |
LO 1 |
|
Additional Assessments:
Starting in 2001-2002 and every four years following, the Department will conduct a survey of alumni from the previous five years.
Starting in 2003-2004 and every four years following, the Department will conduct a survey of employers of speech communication graduates from the previous two years.
BA/BS Report:
1. What learning
outcomes were assessed this year and why?
Learning Outcome 3:
Utilize knowledge of communication theories and principles to describe, analyze
and evaluate communication messages. This was on the departmental assessment
schedule for 2002-03.This learning outcome is linked to departmental goal #1: To
offer quality undergraduate and graduate programs that require students to
master the basic concepts and practices of the field, to begin to explore the
many areas of study within the field, and to master a specialized area of
study.
2. How
were they (was it) assessed?
All speech communication students in
the BA/BS program must complete SPEE 485, Senior Seminar. As part of this
course, students complete an electronic portfolio documenting student
performance and reflecting on that performance. One of the elements of the
portfolio, #9, asks students to include an artifact from a 200-400 level
course demonstrating critical analysis of an interpersonal, group,
organizational, or public communication event. The Speech Communication
Department Long Range Planning and Assessment Committee reviewed the artifacts
in #9 for all students in SPEE 485 in spring 2003.
a. What methods were used?
The artifacts were assessed using
the assessment rubric below.
b. Who was assessed?
All students in SPEE 485 spring 2003
(22). Of the 22 students in the class, three had incomplete or unfinished
portfolios at the time of assessment, so they were not part of the assessed
population, leaving 19 students in the assessed population.
c. When was it assessed?
The assessment took place
3. What was learned?
Results:
|
|
Student Performance |
Fail to meet outcome |
Minimally Meet Outcome |
Meet Outcome |
Exceed Outcome |
Percent Demonstrating Mastery |
|
1 |
Student assignments/activities demonstrate that student is able to use communication theories and principles to describe a communication message. |
1 |
1 |
|
|
95% |
|
2 |
Student assignments/activities demonstrate that student is able to use communication theories and principles to describe and analyze a communication message. |
|
2 |
1 |
|
95% |
|
3 |
Student assignments/activities demonstrate that student is able to use communication theories and principles to describe, analyze and evaluate a communication message. |
4 |
5 |
4 |
1 |
53% |
Student performance is
developmental; students meeting the third level demonstrate acceptable
performance on the first two levels.
Level of Mastery: 100% of assessed population meets or exceeds level 1; 80% of assessed population meets or exceeds level 2; 50% of assessed population meets or exceeds level 3.
Level of mastery was not met for
level 1; LOM was met for level 2; LOM was met for level 3.
Discussion:
After assessing the artifacts, the
LRPA committee discussed the results. We felt there were at least three reasons
why student performance did not meet mastery:
1. Students selected inappropriate
artifacts to demonstrate the skills being sought.
2. Students did an acceptable job
describing, analyzing and evaluating artifacts, but were not as strong at using
communication theory in the description, analysis or evaluation of the
artifacts.
3. Insufficient numbers of students
possess the skills needed to successfully use communication theory to describe,
analyze and evaluate a communication message.
The committee felt that the first
possibility did apply to some of the students, as the assignments selected did
not meet the criteria. The committee also noted that many students were not as
strong at using communication theory as the department would wish, and that the
assignments did not demonstrate sufficient knowledge and utilization of
appropriate communication theory. Finally, the committee also felt that the
assignments may have represented student skill level, and that some students
are not able to demonstrate this ability.
4. What will the department do as a result of that
information?
This information will be shared with
the department for discussion at the first fall departmental meeting. The committee
did make two recommendations: (a) Remind faculty to list on their course
syllabi assignments that would be appropriate for meeting #9, and remind
students to retain copies of those assignments; (b) Reinforce the use of
communication theory in the description, analysis and evaluation of
communication messages. The department expects more students to demonstrate
that ability as more students are required to take SPEE 385, Communication
Theory and Research.
5. How did the department make use of the feedback from last
year’s assessment report?
Last year’s assessment feedback
report was largely positive, but did ask the department to be more specific on
the use of levels of mastery, which we attempted to do this year.
________________________________________________________________________
BST Report:
1. What learning outcomes were assessed this
year, and why?
LO 3-- Students demonstrate sufficient knowledge and skills to
assess student learning in the area of public communication. This was on the
departmental assessment schedule for 2002-03.This learning outcome is linked to
departmental goal #1: To offer quality
undergraduate and graduate programs that require students to master the basic
concepts and practices of the field, to begin to explore the many areas of
study within the field, and to master a specialized area of study. This
learning outcome also includes the
following embedded outcomes: NCATE 2000 Candidate Performance Standard 1; MNBOT
Communication Arts and Literature Standards C: 3; ISTE Category 4; NCA K-12:
Standards 4.15
2. How
were they (was it) assessed?
a. What methods were used? All students in SPEE 4/504 Methods
of Teaching Speech complete an electronic portfolio. One part of that portfolio
is a teaching unit, and another part includes the creation of learning
activities. As part of both the unit and the learning activities, students in
SPEE 4/504 must develop assessment methods for evaluating student performance.
These assessment methods include the use of rubrics, checklists, more
“objective” quiz and exam questions, and more “subjective” essay and short
answer questions. The teaching unit and the learning activities were assessed
using the rubric below:
|
Level |
Competency |
Number meeting competency |
|
1 |
Student unit plans and learning activities contain
minimally acceptable assessment and evaluation tools as evidenced by poorly
constructed rubrics and/or poorly created objective and essay questions and
other assessment and evaluation tools. |
17 |
|
2 |
Student unit plans and learning activities contain acceptable
assessment and evaluation tools as evidenced by well constructed rubrics and
successfully created objective and essay questions and other assessment and
evaluation tools. |
17 |
Level
of Mastery: 100% of assessed population meets or exceeds level 1; 80% of
assessed population meets level 2.
b. Who was assessed? All students in SPEE 4/504 Methods of Teaching Speech, fall 2002 (N=17)
c.
When was it assessed? The assignments were
completed in fall 2002. The assessment was completed
3. What was learned?
All students assessed met level 2 of the rubric. Assessment
shows that students exceeded the level of mastery set for this outcome. The
result is not surprising, as the course uses mastery learning and requires
students to redo activities until they meet acceptable levels. As the course is
designed to prepare future teachers, neither the course instructor nor the
department believe it is acceptable to allow students NOT to meet or exceed
mastery levels. Students unable to meet the requirements are provided
additional help. If students are still unable to meet the requirements, they
are counseled to consider another career option.
4. What will the department do as a result of
this information?
No specific changes are planned for the outcome or the
course. However, there will be a new instructor teaching the course fall 2003.
No major changes to the course nor the assessment are
planned. Any changes made to the course would have to be made in conjunction
with departmental consideration, and also would have to be made in conjunction
with NCATE accreditation requirements.
5. How did the department make use of the
feedback from last year’s assessment?
Feedback from last year noted that the outcome was met, so
no major changes were needed.
MA/MS Report:
LO 3-- Upon completion of
the graduate program in Speech Communication, all graduate students will be
able to demonstrate mastery of a research
methodology. This
was on the departmental assessment schedule for 2002-03. This learning outcome
is linked to departmental goal #1: To
offer quality undergraduate and graduate programs that require students to
master the basic concepts and practices of the field, to begin to explore the
many areas of study within the field, and to master a specialized area of
study.
a. Method
of assessment: Spring
2003 Qualifying Examinations: Question 3. This question was assessed using the
rubric below. The assessment rubric was based on the general rubric following
the specific rubric.
b. Assessment Population: All
graduate students who took qualifying exams in Spring
2003 (N=9)
c.
Qualifying Exam Question 3 Rubric:
|
Outcome |
Does not meet
Outcome |
Meets Outcome |
Exceeds Outcome |
N |
Percent Meeting
or Exceeding Outcome |
|
Describe a
research methodology |
1 |
6 |
2 |
9 |
90 |
|
Compare and
Contrast at least two research methodologies |
0 |
7 |
2 |
9 |
100 |
|
Apply the
methodology to a communication issue |
2 |
6 |
1 |
9 |
80 |
Standard of Mastery: 100% of population meets or exceeds all outcomes.
Results: In one category (compare and contrast) 100% met the
level; in the other two categories, the 100% threshold was not met. Overall,
the level of mastery was not met, but no serious problems seem to be present.
The specific rubric used to assess responses also provided more detail, as the
area that needed most work was in application of a methodology to a
communication issue. The assessment committee also noted a general improvement
in the quality of responses from previous assessments.
The results of the qualifying exams was shared with the department. The qualifying exam committee made two recommendations based on the results: 1) Students in the required 601 Research Methods course should be provided added instruction in and opportunities to implement research methodologies; 2) Students need to better demonstrate mastery of style guidelines, either MLA or APA, so all graduate faculty are asked to emphasize style requirements and assess student mastery of style requirements..
Feedback from last year’s report, and the assessment data
gathered showed no need to make any changes in assessment methods or curricular
or pedagogical improvements.
________________________________________________________________________