2005-2006 Assessment Report

Page address: http://www.mnsu.edu/cmst/assmnts/2005_2006assessreport.html

 

This year the departmental assessment plan calls for an alumni survey.  The results of that survey are included as Appendix A of this document.

 

 

Check here if your assessment report covers all undergraduate degree programs: NO

 

Check here if your assessment report covers all graduate degree programs: NO

 

1. What learning outcomes were assessed this year and why?

·        Learning Outcome 1: know communication theories, principles, concepts, and the development of the field of communication.

·        This outcome is designated on the departmental 5 year assessment schedule for 2005-06.

·        Linked to department goals #1, 2, 3, 6, 7

·        Linked to College goals # 1, 2, 4

·        Linked to University goals # 1 and 2

 

2. How were they (was it) assessed?  Tools employed: 

 

a. What methods were used?

 

All speech communication students in the BA/BS program must complete SPEE 485, Senior Seminar. As part of this course, students complete an electronic portfolio documenting their performance over the course of their study and writing personal reflections about their performance.  

                        The artifacts and reflections demonstrating competency for this outcome include:

1.      Assignment from a lower division (100-200) course that demonstrates comprehension of a basic theory, principle or concept in communication.

2.      Assignment from a lower division (100-200) course that demonstrates knowledge of the development of the field of communication.

3.      Assignment from an upper-division (300-400) course that demonstrates advanced knowledge of a communication theory, principle or concept

Since this competency was last assessed in 2001, the department developed a new required course in communication theory.  Part of the purpose of this course is to improve student use and analysis of theories and concepts in communication.

 

Additionally, the reflection statements about each competency address student disposition and attitudes about the competency.  All students are able to sense a connection between the theory and the practice and its application to their world view and communication style.

 

Artifacts and self-reflection statements associated with the student learning outcomes listed above were assessed using the assessment rubric below.   

 

b. Who was assessed?

All students in SPEE 385 for fall 2005 and spring 2006 (n=28) and all students in 485, Senior Seminar, fall 2005 an spring 2006 were assessed; (fall, 12; spring 21)

 

c. When was it assessed?

At the end of fall 2005 and spring semester 2006

 

3. What was learned? (Rubric applied to tools) (N=28) in 385; (N=28) in 485

 

	Outcome`	SP 385 competency 3	Sp 485 competency 1	Sp 485 competency 2	Sp 485 competency 3
4	Student activities and/or assignments demonstrate knowledge of basic theories/concepts/skills and their development in the communication discipline, as well as the ability to use that knowledge. This knowledge is demonstrated through activities and/or assignments that show students are able to define key theories and concepts, are able to utilize basic skills, and are able to analyze and evaluate these theories and concepts, including showing how these concepts have developed over time.	7	28	28	28
3	Student activities and/or assignments demonstrate knowledge of basic theories/concepts/skills, as well as the ability to use that knowledge. This knowledge is demonstrated through activities and/or assignments that show students are able to define key theories and concepts, are able to utilize basic skills, and are able to analyze and evaluate these theories and concepts.	10
2	Student activities and/or assignments demonstrate knowledge of basic theories/concepts/skills, as well as the ability to use that knowledge. This knowledge is demonstrated through activities and/or assignments that show students are able to define key theories and concepts, and are able to utilize basic skills.	7
1	Student activities and/or assignments demonstrate knowledge of basic theories/concepts/skills, or the ability to use that knowledge. This knowledge is demonstrated through activities and/or assignments that show students are able to define key theories and concepts, or are able to utilize basic skills.	4

 

Student performance is developmental; students meeting the third and second levels obviously demonstrate acceptable performance on the first level.

 

Level of Mastery: 100% of assessed population meets or exceeds level 1; 100% of assessed population meets or exceeds level 2; 100% of assessed population meets level 3.

 

Level of mastery was met for level 1; LOM was met for level 2; LOM was met for level 3.

 

4. What will the department do as a result of that information?

The Department is satisfied with the overall results of the assessment and will continue to develop assignments which allow students to demonstrate knowledge of the development and history of the field of communication.   Students are understanding and applying theory in a variety of contexts.  Upon completion of 385 students are, in general, less proficient in communication research than we would like.  This situation has been dealt with by the department by replacing Speech 385 with Speech 290 and other courses. The benefit of a Communication research course is that students will receive a deeper understanding of the research process (and the opportunity to create a real research project, rather than the mini-projects necessitated by the combination of their and research in sp 385.

.

 

5. How did the department make use of the feedback from last year’s assessment report?

 

This outcome has not been revisited since 2001.  The department has learned from these results that students were not comfortable with moving away from the text in order to analyze or critique theory and concept. Students had trouble seeing the connection between theory and application. The department developed a required course in communication theory. Part of the purpose of this course is to improve student use and analysis of theories and concepts in communication.  Additionally, the course has now been split into 2 courses one emphasizing theory and the other emphasizing research.  The change to 2 course sequence will allow more time for students to assimilate and apply theory. 

 

 

Check here if your assessment report covers all undergraduate degree programs: NO

 

Check here if your assessment report covers all graduate degree programs: NO

 

1. What learning outcomes were assessed this year and why?

Learning Outcome 2: demonstrate sufficient pedagogical and professional skills to succeed as a teacher of public communication

Linked to College Goals # 1, 2, 4

Linked to University Goals # 1, 2

Dept goals #1, 2, 3, 6, 7

This learning outcome also includes the following embedded outcomes:

Embedded Outcomes: NCATE 2000 Candidate Performance Standard 1

MNBOT Communication Arts and Literature Standards A: 3-15; B: 3-5; C: 3-7; ISTE Category 1; NCA K-12: Standard 3

2. How were they (was it) assessed?

a. What methods were used? Tools employed:  Electronic Teaching Portfolios, Philosophy of Speech Education, and Micro-teaching Reflection Papers in speech 404

All students in SPEE 404 Methods of Teaching Speech complete an electronic teaching portfolio. The portfolio includes the creation of lesson plans for a micro-teaching session and a reflection paper which demonstrate student’s ability to assess their own performance as a teacher.  Each student also crafts a philosophy of speech education.  These three “critical” assignments serve to demonstrate students’ ability to apply knowledge of speech communication to their own plans to teach public speaking.  These opportunities provide perception checks for attitudes/dispositions about career teaching for each student through self-reflection.

 

(The rubrics for the 3 critical assignments use a 4 point rubric rather than a 3 point rubric so they correspond with COE assessment data.)

 

b. Who was assessed?

All students in SPEE 404 for fall 2005 were assessed. (n=21)

 

c. When was it assessed?

Fall semester 2005

 

3. What was learned? (Rubric applied to tools)

 

	Digital Teaching Portfolio	Met Outcome
4	The portfolio clearly addresses course content. Student includes all required elements. Contents of the portfolio demonstrate exemplary understanding of course concepts and skills. Portfolio has no spelling errors or grammatically incorrect sentences.  Sentences are well constructed. Appropriate punctuation is used in all written material.	21
3	Most components of the portfolio are well chosen and adequately demonstrate an understanding of course concepts. Student includes most required elements. Contents of the portfolio demonstrate satisfactory understanding of course concepts and skills. Portfolio contains none or very few spelling errors.  Material is in most cases grammatically correct. Written material is punctuated correctly.	21
2	The portfolio indicates a general understanding of course concepts. Student is missing required elements. Contents of the portfolio demonstrate only a general understanding of course concepts and skills. Portfolio contains a few spelling errors or grammatically incorrect statements. Written material lacks some punctuation.	21
1	The portfolio is superficial in its approach and does not appropriately address the student’s learning. Contents of the portfolio demonstrate unacceptable levels of understanding of course concepts and skills. Portfolio contains many spelling errors or grammatically incorrect sentences. Written material lacks punctuation or is not correctly punctuated.	21

 

 

	Micro Teaching Reflection Paper	Met Outcome
4	The analysis paper contained very useful information including what modifications the student would use when teaching this lesson again, how pupils might react to the lesson, and personal reflections/reactions to the lesson. There are no grammar or style errors.	21
3	The analysis paper contained very useful information including what modifications the student would use when teaching this lesson again, how pupils might react to the lesson, and personal reflections/reactions to the lesson. There are a few grammar or style errors.	21
2	The analysis paper contained some useful information and more than a few grammatical errors.  Some requirements are not discussed. There are more than a few grammar or style errors	21
1	The analysis paper contained little if any useful information and numerous grammatical errors.  Many requirements not discussed. There are many grammar or style errors.	21

 

	Philosophy of Speech Education	Met Outcome
4	Student addresses the minimum five (questions) components of the philosophy and rationale statement and offers additional considerations. Student response shows thorough consideration of all components, and the paper contains no grammatical errors.	21
3	Student addresses the minimum five (questions) components of the philosophy and rationale statement. Student response shows thorough consideration of the components, and the paper contains several grammatical errors.	21
2	Student addresses the minimum five (questions) components of the philosophy and rationale statement. Student response shows limited consideration of the components, and the paper contains many grammatical errors.	21
1	1 Student fails to address all five (questions) components of the philosophy and rationale statement. The paper contains numerous grammatical errors.	21

 

Student performance is developmental; students meeting the fourth and third levels obviously demonstrate acceptable performance on the first and second levels.

 

Level of Mastery: 100% of assessed population meets or exceeds all levels.

 

Assessment indicates that based on the three critical assignments associated with this outcome, students exceeded the level of mastery set for this outcome. The result is not surprising, as the course uses mastery learning and requires students to redo activities and assignments until they meet mastery levels. As the course is designed to prepare future teachers, neither the course instructor nor the department believe it is acceptable to allow students NOT to meet or exceed mastery levels. Students unable to meet the requirements are provided additional help. If students are still unable to meet the requirements, they are counseled to consider another career option.  Additionally, by crafting the philosophy of speech education, the students are addressing dispositions and attitudes about education.  This philosophy allows students to see the connection between their own future students, their parents, and their colleagues in the field.

 

4.  What will the department do as a result of this information?

No specific changes are planned for the outcome, the assignments, or the course. Any changes made to the course would have to be made in conjunction with departmental consideration, and also would have to be made in conjunction with NCATE, BOT, and COE accreditation requirements.

 

5.  How did the department make use of the feedback from last year’s assessment?

 

Feedback from last year noted that the LO #1 was met; therefore, no major changes were needed for this year.

 

 

 

Check here if your assessment report covers all undergraduate degree programs: NO

 

Check here if your assessment report covers all graduate degree programs: YES

 

1. What learning outcomes were assessed this year and why?

Learning Outcome 1: Upon completion of the graduate program in Speech Communication, all graduate students demonstrate professional competence in oral communication.

To offer quality undergraduate and graduate programs that require students to master the basic concepts and practices of the field, to begin to explore the many areas of study within the field, and to master a specialized area of study. To promote the performance of oral communication in both creative and scholarly venues. To encourage students, graduate students, and faculty to take part in scholarship, research, and creative activity that will contribute to the personal and professional development of all involved.

·        Linked to College Goals # 1, 2, 4

·        Linked to University Goals # 1, 2

·        Linked to department goals #1, 2, 3, 6, 7

 

2. How were they (was it) assessed?  Tools employed: 

 

a. What methods were used?

Student oral presentations in Departmental Colloquia were assessed by graduate faculty using the rubric below. Presentations were assessed Spring 2006.

 

b. Who was assessed?

All graduate students who will graduate this spring (N=11)

 

c. When was it assessed?

Spring 2006

 

3. What was learned? (Rubric applied to tools)

 

		Met Outcome
2	Student oral defense of thesis or alternate plan paper and/or oral presentation in public setting demonstrates that the student is able to successfully synthesize information in a written form of communication for presentation in an oral format. Student is able to highlight the thesis, the key supporting evidence, the key theoretical elements, and the conclusion and present these in a manner that helps the audience follow and understand the content and the organizational structure. The student uses appropriate verbal and nonverbal skills in the presentation, and is able to successfully defend statements made in the oral presentation.	11
1	Student oral defense of thesis or alternate plan paper and/or oral presentation in public setting demonstrates that the student is able to adequately synthesize information in a written form of communication for presentation in an oral format. Student is able to highlight the thesis, the key supporting evidence, the key theoretical elements, and the conclusion and present these in a manner that does not hinder the audience from following and understanding the content and the organizational structure. The student uses appropriate verbal and nonverbal skills in the presentation, and attempts to defend statements made in the oral presentation.	11

 

 

Student performance is developmental; students meeting the third and second levels obviously demonstrate acceptable performance on the first level.

 

Level of Mastery: 100% of the assessed population meets or exceed both levels 1and 2

 

4. What will the department do as a result of that information?

 

Individual student evaluations, from which the above results were derived, indicated that our graduate students demonstrate satisfactory and adequate oral communication skills which lead to successful oral communication.  As a department, we will continue to provide opportunities for public presentations and feedback and constructive criticism for our students and ourselves.  Achievement of competency in the  affective domain is an implied domain within the graduate program.  Students are involved in the program because they “like” or “value” the skills and knowledge they receive.  While there are very few tools in place to measure their attitudes, it seems inherent and self evident.  An examination of the alumni survey also indicates students are satisfied with the program; again, not a specific tool, but an inherent positive result of the degree.  As indicted in the alumni survey of graduate students, the most important reason for pursuing the Speech Communication degree was a strong interest in the subject matter.

 

 

5. How did the department make use of the feedback from last year’s assessment report?

 

Regarding last year’s plan assessed (Learning Outcome 1: Upon completion of the graduate program in Speech Communication, all graduate students will be able to demonstrate professional competence in written communication.), the Department continues to strengthen mastery of written communication.  There were many formal discussion about the department’s qualification exam (quals).  Discussion revolved around how to assess student writing, how to use the data and skills generated by the quals process, and how to increase written competency by all graduate students. 

Appendix A

Alumni Survey Results

2005-06

Speech Communication Department

 

The Speech Communication Department conducted a regularly scheduled survey of alumni from the last five years. (Every four years the department will conduct a survey of alumni from the last five years.) The survey was modeled on previous surveys conducted for the 1989 Program Review, and for the 2001-2002 regular assessment plan directive. This survey was designed to assess alumni satisfaction with the program, gauge program strengths and weaknesses, and determine how students were using their speech communication degrees. Relevant date from the 2001-2002 survey is included and shown in red.

 

Results:

 

199 postcards were mailed asking alumni from the last five years to respond to an online survey. There was no follow-up. The return rate was 10 percent with 19 surveys returned.

 

Background Data:

 

Degree Received: (N=19)

 

·        3 respondents received a BA degree

·        10 respondents received a BS degree

·        3 respondents received a BST (Teaching) degree

·        1 respondents received an MA as a second degree from this program)

·        3 respondents received a MS degree

·        0 respondents received a MFA degree

 

Reasons for seeking degree/enrollment status:

 

·        15 respondents reported they received the degree in order to get a job

·        6 respondents reported they received the degree in order to enhance their education

·        2 respondents reported "other" as a response, with no preponderance of specificity

·         

·        89% of the respondents were enrolled full-time as they sought their degree

·        The average student took 3.66 (previous 4.01) years to achieve an undergraduate degree

·        The average student took 2.25 (previous 2.95) years to achieve a graduate degree

 

Gender

·        11 female

·         8 male

 

Age

17-20     0

21-23     5

24-26     10

27+      4

 

Satisfaction Measures:

 

Overall Curriculum

 

On a scale of 1-5 (1=very dissatisfied; 5=very satisfied)

The response was a mean of 4.21 (previous was based on a reverse 1-4 scale 1.53)

 

Internship Opportunities

 

On a scale of 1-5 (1=very satisfied; 4=very dissatisfied)

The response was a mean of 3.31 (previous was based on a reverse 1-4 scale 2.0)

 

Independent Study Opportunities

 

On a scale of 1-5 (1=very satisfied; 4=very dissatisfied)

The response was a mean of 3.94 (previous was based on a reverse 1-4 scale 1.64)

 

Pedagogy

 

On a scale of 1-4 (4=very useful; 1=not at all useful)

 

·     Discussions: Mean of 3.63 (3.48)

·     Group Projects: Mean of 3.21 (3.28)

·     Presentations: Mean of 3.68 (3.55)

·     Individual Projects: Mean 3.63 (new category for 2005-06)

·     Lectures: Mean of 3.15 (3.23)

·     In-class Activities: Mean of 3.21 (3.25)

 

 

Overall Quality Measures

 

Interest in graduate degree:

·        3 yes

·        12 no

·        3 undecided

 

Employment

 

·        Sought employment that made direct use of your degree in Speech Communication: 19 yes (41 Yes 17 No)

·        Found employment that made direct use of your degree in Speech Communication: 17 yes, 2 no (36 Yes 6 No)

·    Currently employed: 100% (58 Yes 2 No)

·    Employed Full time with overtime 4 (no previous data)

§     Employed Full-time: 13 (49 Yes 9 No)

§     Employed part time 2 (no previous data)

 

Type of Employment:

 

·        0 - Public/Government

·        8 - Private

·        1 - Self-employed

·        7 - Education

·        3 - Other

 

In present position, to what degree do you find your Speech Communication degree useful:

 

·        Very Useful: 12

·        Somewhat useful: 5

·        Slightly Useful: 42

·        Not at all: 0

·        Not applicable: 0

 

Would you prefer a job that makes more use of your education

·        8 yes 11 no (30 Yes 25 No)

 

Annual Income:

 

·     0-9999:

·     10-19999: 1

·     20-29999: 4

·     30-39999: 7

·     40-49999: 1

·     50-59999: 2

·     60-69999: 0

·     70-79999: 3

·     Prefer not to disclose: 1

Undergraduate Alumni Comments

 

·        The statistics generated the following generalizations.

 

·        89% (56%) of our alumni obtained a BS Major in speech communication.

 

·        The most important reasons for pursuing a speech Communication degree was to get a job.

 

·        Most of our alumni, 89% (66%), were enrolled as fulltime students when they attended college.

 

·        The satisfaction measures of all four areas pointed to general contentment with the degree, the department and its members, and the program.

 

·        100% of the respondents sought and 89% found (69% and 61%) employment making use of their degree in speech communication.

 

·        63% (66%) found the speech communication degree very useful in their present position.

 

·        26% (27%) responded the speech communication degree was somewhat useful in their present position.

 

·        The results seem to positively support the work of the Speech Communication department over the past five years.

 

 

Graduate Comments

 

The Speech Communication Department conducted a survey of current graduate students. The survey was modeled on a previous survey conducted for the 1989 Program Review, and was designed to assess student satisfaction with the program, gauge program strengths and weaknesses, and determine how students perceived the program.

 

In spite of the small return on this survey, the following generalizations may point to strengths and weaknesses. The statistics generated the following generalizations.

 

21% (44%) of our students are working toward an MA degree in speech communication.

 

The most important reason for pursuing the Speech Communication degree was a strong interest in the subject matter. The second most important reason to get the MA in Speech was as the stepping stone to a Ph.D. program. The students who answered the survey were classified as full-time students.

 

The satisfaction measures pointed to general contentment with the degree, the department and its members, and the program in general. Students indicted they were at least somewhat satisfied with the all measures of satisfaction. However, 3 students were not satisfied with the quality of instruction and four students were not satisfied with the quality of textbooks and reading assignments. These two areas, as core to the teaching and learning process, should be examined and studied to determine how improvement can be made.

 

The opportunities for career and academic advising and opportunities internships appeared to be problematic. Even though only two or three respondents were not satisfied with academic or career advising, it may be important to determine how the department can increase the career and academic advising process. If students wanted more opportunities and could not get them or if they truly did not care or were not interested in pursing internships or independent study was not determined. The department seems to be providing for the needs of our students in the areas of mentoring both research and teaching and in the area of technology use and instruction. Faculty members are easily accessible and communication between students and staff are highly rated.

 

Satisfaction with classroom activities appeared very strong. All experiences in the classroom setting were reported as somewhat useful, useful, or very useful. Two non-classroom settings, the colloquia, and TA training received three and two responses as no opinion or not applicable. The colloquia are still rather new. The two and three students responding with no opinion or not applicable could be part time students who do not have access to colloquia held during the school day or they are not TA’s and therefore would not have any opinion about TA training.

 

The five verbatim comments also indicated satisfaction with the program, the faculty, and the courses offered. The weaknesses and suggestions echoed the department’s conclusions about having providing more information early-on in the program about the graduate program and its organization. In response, the department initiated a required meeting of all graduate students early this year.

 

Discussion on Strength and Weaknesses Within the Department

 

Results from the  2005-2006 Alumni Survey indicate departmental strengths and weaknesses. The results provide direction for the Department in enhancing our strengths and correcting our weaknesses.

 

Overall scores in the Satisfaction Measures indicate the significant majority of responses fall within the “very satisfied” to “satisfied” range. The Evaluation of Classroom Activities scores indicate students are fairly pleased with the broad diversity of teaching and learning styles employed in Speech Communication courses. The lowest scores were with lectures and group projects. The low numbers with group projects are not surprising. Students often do not see the benefit of group projects until later on in life. The Department does not envision abandoning group projects in its courses. The Department may, however, decide that when group projects are assigned more emphasis should be placed on explaining the benefits of the project to the students.  In addition, with the recent emphasis on active learning generated by the newest pedagogy research, many students may be finding active learning, rather than lectures, to be a better learning tool.  As these non-traditional learners become the norm, lectures will most likely be replaced to some degree with active learning activities  While lectures are not bad pedagogy, the emphasis is shifting.  These students may be reflecting a change in teaching and learning style from heavy emphasis on the lecture which needs to be addressed by faculty.

The Verbatim Comments highlight the greatest strength of the Department—The faculty. Faculty are the most frequently identified strength of the Department. The Weaknesses section of the Verbatim Comments identifies many undergraduate concerns. Many of the concerns have already been addressed by the Department (e.g., scheduling conflicts with English and Theatre), other concerns are addressed above. One concern the Department should address is the repetition of theories across different courses. The size of the Department does not permit the luxury of courses addressing similar theories from different courses/perspectives. The Department’s curriculum committee should review the course offerings and make recommendations to the Department on courses that may be trimmed from the curriculum in order to avoid repetition. 

Specific issues within the Satisfaction Measures deserve further attention:

 

STRENGTHS

The Department is proud of the very high scores in “overall quality of instruction,” “overall quality of academic advising” and “communication between faculty and students.” All three areas indicate the Department is having a positive impact on undergraduate student education.

 

AREAS FOR IMPROVEMENT

Second, the Department needs to address its overall quality of career advising and internship opportunities. The comments related to this area clearly indicate a need for more focused attention to the professional careers our students hope to attain.