MINNESOTA STATE UNIVERSITY

Department: Speech Communication

2004-2005 Assessment Report

Program: Undergraduate  Degree Awarded: BA/BS

 

 

Check here if your assessment report covers all undergraduate degree programs: NO

 

Check here if your assessment report covers all graduate degree programs: NO

 

1. What learning outcomes were assessed this year and why?

·        Learning Outcome 5: Demonstrate critical thinking, communication proficiency, and knowledge of methodology by conducting a scholarly project utilizing a critical, humanistic or social scientific perspective.

·        This outcome is on the departmental 5 year assessment schedule for 2004-05.

·        This learning outcome is linked to departmental goal #2: To offer general education courses that provide students with the knowledge and skills to be critical thinkers and effective communicators in both professional careers and advanced study.

·        Linked to College goals # 1, 2, 4

·        Linked to University goals # 1 and 2

 

2. How were they (was it) assessed?  Tools employed:  485 final project paper and final project speech); and special assessment grant survey by Drs. Cronn-Mills and Johnson-Curiskis

a. What methods were used?

All speech communication students in the BA/BS program must complete SPEE 485, Senior Seminar. As part of this course, students complete an electronic portfolio documenting both their performance over the course of their study and writing personal reflections about their performance. Students also complete an exit interview which is analyzed for department curricular decision making.  The Speech Communication Department Long Range Planning and Assessment Committee reviewed the artifacts for students in SPEE 485 in spring 2005.  The student learning outcomes/competencies included:

demonstrate comprehension of a basic theory, principle or concept in communication.

demonstrate knowledge of the development of field of communication.

demonstrate advanced knowledge of a communication theory, principle or concept.

demonstrate individual public speaking ability.

demonstrate group communication ability.

demonstrate effective listening skills.

demonstrate critical and/or reflective thinking and responding.

demonstrate critical analysis of an interpersonal, group, organizational, or public communication event.

pre and post test scores in the PRCA

pre and post test scores on the WTC (Willingness to Communicate) survey

demonstrate the ability to complete a research project

demonstrate the ability to present research orally

 

Artifacts and self-reflection statements associated with each student learning outcome listed above were assessed using the assessment rubric below.  In addition, Dan Cronn-Mills and Nan Johnson-Curiskis were awarded an assessment grant to analyze the results of a survey focusing on the senior project (paper and speech).  (Assessment grant information revealed………………………..)

 

b. Who was assessed?

All students in SPEE 485 for fall 2004 and spring 2005 (n=35; fall, 12 spring 23)

 

c. When was it assessed?

At the end of spring semester 2005

 

3. What was learned? (Rubric applied to tools)

 

Outcome

Met

Outcome

3

Student plans, researches and carries out a scholarly project. The student demonstrates the ability to work with at least one methodology as it relates to at least one communication phenomenon, and to use that methodology to critically evaluate that phenomenon. The student uses and correctly cites appropriate academic sources. The project must have both a written and oral component. The written component demonstrates proficiency in written communication evidenced by few if any grammatical errors, correct citation style, and well-constructed paragraphs that develop a central thesis. The oral component demonstrates proficiency in oral communication through the use of appropriate verbal and nonverbal skills, and the ability to effectively synthesize written communication for an oral format.

12

2

Student plans, researches and carries out a scholarly project. The student demonstrates the ability to work with at least one methodology as it relates to at least one communication phenomenon, and to use that methodology to critically evaluate that phenomenon. The  student uses and correctly cites appropriate academic sources. The project must have both a written and oral component. The written component demonstrates acceptable proficiency in written communication evidenced by a writing style in which the form and content allows the reader to identify central claims and supporting evidence. The oral component demonstrates acceptable proficiency in oral communication through the use of appropriate verbal and nonverbal skills, and the ability to synthesize written communication for an oral format.

31

1

Student plans, researches and carries out a scholarly project. The student demonstrates the ability to work with at least one methodology as it relates to at least one communication phenomenon. The student uses appropriate academic sources. The project must have both a written and oral component. The written component demonstrates minimal proficiency in written communication evidenced by a writing style in which the form and content does not prevent the reader from identifying central claims and supporting evidence. The oral component demonstrates minimal proficiency in oral communication evidenced by audience being able to identify a central focus and follow a progression of that focus to a conclusion.

35

 

Student performance is developmental; students meeting the third and second levels obviously demonstrate acceptable performance on the first level.

 

Level of Mastery: 100% of assessed population meets or exceeds level 1; 75% of assessed population meets or exceeds level 2; 50% of assessed population meets level 3.

 

Level of mastery was met for level 1; LOM was met for level 2; LOM was met for level 3.

 

4. What will the department do as a result of that information?

The Department is satisfied with the overall results of the assessment and will continue to emphasize assignments which allow students to demonstrate scholarly writing and speaking; however, this section of the outcome (students demonstrate the ability to use methodology by conducting a scholarly projects utilizing a critical, humanistic or social scientific perspective) appears to need work or revision.  Within the context of current courses, it appears the building blocks can be laid for scholarly research with appropriate methodology in Speech 385, which is now a required course for speech majors.  It will take at least 2 more years for all majors to “cycle” through this required course to allow this competency to be fully realized in Speech 485..   Regardless, the Department will continue to emphasize and monitor content designed to satisfy the learning outcome.  These issues will be fodder for the departmental curriculum retreat being scheduled for the summer of 2005.

 

Student perception of the success of the final project (speech and paper) indicate students are satisfied with the capstone writing and speaking experiences.  The appendix contains commentary and support for this is based on the the summary of a special assessment grant prepared by Drs. Cronn-Mills and Johnson-Curiskis for 2004-2005.

 

5. How did the department make use of the feedback from last year’s assessment report?

Feedback from last year regarding Learning Outcome#4, which was in the assessment plan’s regular rotation, noted that the outcome was met, so no major changes were needed.  The Department is satisfied with the results of last year’s assessment and will continue to emphasize content designed to satisfy the learning outcome.  However, as noted in last year’s report, while analyzing data for LO #4 (demonstrate an awareness of how our communication influences others and how the communication of others influences us) we uncovered another discrepancy regarding Learning Outcomes 1, 2, and 3.  While the assessment levels of mastery were met in previous years, we noticed some students had difficulty finding artifacts emphasizing speech communication theory.  Most students were able to document theory through application or reflection but were lacking advanced theoretical knowledge of speech communication theories.  This fall, faculty were again urged to redesign course work to include demonstration of advanced theoretical knowledge while still supplying students with the opportunities to apply and reflect on their use of theory.  These three Learning Outcome (LO 1, 2, 3) are each scheduled for formal reassessment in future assessment plan years..

 


 

MINNESOTA STATE UNIVERSITY

Department: Speech Communication

2004-2005 Assessment Report

Program: Undergraduate  Degree Awarded: BST

 

 

Check here if your assessment report covers all undergraduate degree programs: NO

 

Check here if your assessment report covers all graduate degree programs: NO

 

1. What learning outcomes were assessed this year and why?

Learning Outcome 1: Students demonstrate content knowledge sufficient to teach public communication.  This was on the departmental assessment schedule for 2004-05.

Linked to departmental goal #1: To offer quality undergraduate and graduate programs that require students to master the basic concepts and practices of the field, to begin to explore the many areas of study within the field, and to master a specialized area of study.

Linked to College Goals # 1, 2, 4

Linked to University Goals # 1, 2

This learning outcome also includes the following embedded outcomes:

Embedded Outcomes: NCATE 2000 Candidate Performance Standard 1

MNBOT Communication Arts and Literature Standards A: 3-15; B: 3-5; C: 3-7; ISTE Category 1; NCA K-12: Standard 3

2. How were they (was it) assessed?

a. What methods were used? Tools employed:  Electronic Teaching Methods Portfolios, Clinical Experience Papers, and Micro-teaching Reflection Papers in speech 404

All students in SPEE 404 Methods of Teaching Speech complete an electronic portfolio. The portfolio includes the creation of lesson plans for a micro-teaching session and a reflection paper which demonstrate students ability to assess their own performance as a teacher. The students participate in a clinical experience observing and/or working with a practicing speech teacher and also reflect on their experience.  These three “critical” assignments in particular serve to demonstrate students’ ability to apply knowledge of speech communication to their own plans to teach public speaking.  There opportuities as well as provide perception checks for attitudes/dispositions about career (teaching) for each student through self-reflection. 

(The rubrics for the 3 critical assignments use a 4 point rubric rather than a 3 point rubric so they correspond with COE assessment data.)

 

b. Who was assessed?

All students in SPEE 404 for fall 2004 were assessed. (n=15)

 

c. When was it assessed?

Fall semester 2004

 

3. What was learned? (Rubric applied to tools)

 

 

Clinical Experience Reflection Paper

Met

Outcome

4

The student self reflection paper offers a critical analysis and useful reflection on the clinical experience.  The student makes a clear connection between what was observed and how  the experience can be used in the classroom. The paper contains no grammatical errors.

14

3

The student self reflection paper offers a description and explanation of the clinical experience.  The student makes a connection between what was observed and how  the experience can be used in the classroom. The paper contains a few grammatical errors

14

2

The student self reflection paper provides very little explanation or description of the experience.  The student makes only a limited connection between what was observed and how  the experience can be used in the classroom.  The paper contains more than a few grammatical errors.

14

1

The student self reflection paper makes no connection between what was observed and how  the experience can be used in the classroom.  The paper contains numerous grammatical errors.

15

 

 

Micro Teaching Reflection Paper

Met

Outcome

4

The analysis paper contained very useful information including what modifications the student would use when teaching this lesson again, how pupils might react to the lesson, and personal reflections/reactions to the lesson. There are no grammar or style errors.

14

3

The analysis paper contained very useful information including what modifications the student would use when teaching this lesson again, how pupils might react to the lesson, and personal reflections/reactions to the lesson. There are a few grammar or style errors.

15

2

The analysis paper contained some useful information and more than a few grammatical errors.  Some requirements are not discussed. There are more than a few grammar or style errors

15

1

The analysis paper contained little if any useful information and numerous grammatical errors.  Many requirements not discussed. There are many grammar or style errors.

15

 

 

Digital Methods Portfolio

Met

Outcome

4

The portfolio clearly addresses course content. Student includes all required elements. Contents of the portfolio demonstrate exemplary understanding of course concepts and skills. Portfolio has no spelling errors or grammatically incorrect sentences.  Sentences are well constructed. Appropriate punctuation is used in all written material.

15

3

Most components of the portfolio are well chosen and adequately demonstrate an understanding of course concepts. Student includes most required elements. Contents of the portfolio demonstrate satisfactory understanding of course concepts and skills. Portfolio contains none or very few spelling errors.  Material is in most cases grammatically correct. Written material is punctuated correctly.

15

2

The portfolio indicates a general understanding of course concepts. Student is missing required elements. Contents of the portfolio demonstrate only a general understanding of course concepts and skills. Portfolio contains a few spelling errors or grammatically incorrect statements. Written material lacks some punctuation.

15

1

The portfolio is superficial in its approach and does not appropriately address the student’s learning. Contents of the portfolio demonstrate unacceptable levels of understanding of course concepts and skills. Portfolio contains many spelling errors or grammatically incorrect sentences. Written material lacks punctuation or is not correctly punctuated.

15

 

Student performance is developmental; students meeting the fourth and third levels obviously demonstrate acceptable performance on the first and second levels.

 

Level of Mastery: 100% of assessed population meets or exceeds level 1; 80% of assessed population meets or exceeds level 2; 75% of assessed population meets level 3; 25% of assessment population meets level 4.

 

Assessment indicates that based on the three critical assignments associated with this outcome, students exceeded the level of mastery set for this outcome. The result is not surprising, as the course uses mastery learning and requires students to redo activities and assignments until they meet mastery levels. As the course is designed to prepare future teachers, neither the course instructor nor the department believe it is acceptable to allow students NOT to meet or exceed mastery levels. Students unable to meet the requirements are provided additional help. If students are still unable to meet the requirements, they are counseled to consider another career option.

 

4.  What will the department do as a result of this information?

No specific changes are planned for the outcome, the assignments, or the course. Any changes made to the course would have to be made in conjunction with departmental consideration, and also would have to be made in conjunction with NCATE, BOT, and COE accreditation requirements.

 

5.  How did the department make use of the feedback from last year’s assessment?

Feedback from last year noted that the LO #4 was met; therefore, no major changes were needed for this year.

 


 

MINNESOTA STATE UNIVERSITY

Department: Speech Communication

2004-2005 Assessment Report

Program: Graduate  Degree Awarded: MFA/MA/MS

 

 

Check here if your assessment report covers all undergraduate degree programs: NO

 

Check here if your assessment report covers all graduate degree programs: YES

 

1. What learning outcomes were assessed this year and why?

Learning Outcome 1: Upon completion of the graduate program in Speech Communication, all graduate students will be able to demonstrate professional competence in written communication. This was on the departmental assessment schedule for 2004-05.

This learning outcome is linked to departmental goals #1, 2, 3, and 5: To offer quality undergraduate and graduate programs that require students to master the basic concepts and practices of the field, to begin to explore the many areas of study within the field, and to master a specialized area of study. To promote the performance of communication in both creative and scholarly venues. To encourage students, graduate students, and faculty to take part in scholarship, research, and creative activity that will contribute to the personal and professional development of all involved.

Linked to College Goals # 1, 2, 4

Linked to University Goals # 1, 2

 

2. How were they (was it) assessed?  Tools employed:  Theses, APP papers; qualifying exams

The results of qualifying exams for graduate students is related to the learning outcome. The exams were reviewed to determine if the learning outcome was achieved. The exams were assessed using the assessment rubric below.

 

a. What methods were used?

The artifacts were assessed using the assessment rubric below.

 

b. Who was assessed?

Graduate students qualifying exam, 2004-2005; completed APP papers, and Theses.  n = 12

 

c. When was it assessed?

School year 2004- 2005

 

3. What was learned? (Rubric applied to tools)

 

 

Outcome

Outcome Met

3

Student writing in the thesis or alternate plan paper, or in the qualifying or comprehensive exams, demonstrates that the student is able to develop a central thesis, support that thesis with appropriate and credible evidence, and defend that thesis against other interpretations.  Students critically analyze evidence for and against the stated thesis and the other interpretations.  Information is presented in a written style with few if any grammatical or mechanical errors, and in a style that allows the audience to easily follow the organizational pattern of the communication.

4

2

Student writing in the thesis or alternate plan paper, or in the qualifying or comprehensive exams, demonstrates that the student is able to develop a central thesis, support that thesis with appropriate and credible evidence, and defend that thesis against other interpretations.  Students analyze evidence for and against the stated thesis, and present all of this information in a written style with more than a few grammatical or mechanical errors, and in a style that hinders the audience from following the organizational pattern of the communication.

11

1

Student writing in the thesis or alternate plan paper, or in the qualifying or comprehensive exams, demonstrates that the student is able to develop a central thesis and support that thesis with appropriate and credible evidence.  The information is presented in a written style that does prevents the audience from understanding the content and following the organizational pattern of the communication.

12

 

Student performance is developmental; students meeting the third and second levels obviously demonstrate acceptable performance on the first level.

 

Level of Mastery: 100% of the assessed population meets or exceeds level 1; 90% of the assessed population meets or exceeds level 2; 75% of the assessed population meets or exceeds level 3.(population determined by?)

 

4. What will the department do as a result of that information?

Twelve graduate students took preliminary exams in the Spring of 2005.  Four of these students wrote exams of high enough quality to warrant a ranking of three, the top ranking established.  Seven students received a score of two, a score demonstrating competence but not necessarily excellence.  One student received a score of one, a failing score.  Overall these numbers, while positive, suggested to the committee a need to continue helping graduate students better integrate theoretical observations into their writing and improve their use of APA and MLA.  Although it is not reflected in the above numbers, the Qualifying Exam Committee has also expressed a need to revise the department’s qualifying exams with changes expected to be implemented in the Spring of 2006.

 

The Department will strive to strengthen mastery of writing skills by specifically focusing on how to analyze an audience and writing techniques appropriate to several venues as relevant to student areas of emphasis.  Furthermore, the department will continue its strong focus on writing by serving as advisors for theses, independent study projects, APP papers, and other areas of academic and professional writing.

 

5. How did the department make use of the feedback from last year’s assessment report?

 

Regarding last year’s plan assessed (Learning Outcome 4: Demonstrate mastery of communication theories within their area of emphasis), the Department continues to strengthen mastery of communication theories by specifically focusing on how to defend these theories as relevant to each student’s area of emphasis. In addition, the hiring of a new faculty member with specific training in communication theory and quantitative research methods will continue to strengthen this area.


Appendix A

ASSESSMENT GRANT REPORT 2004-2005

 

Assessing the Speech Communication Senior Capstone Project

Department of Speech Communication

 

Project Coordinators

Dr. Nanette Johnson-Curiskis

Dr. Daniel Cronn-Mills

 

 

Purpose:

The purpose of the assessment grant was to support faculty work in continued assessment of undergraduate work. The grant was awarded for assessing a senior capstone project which directly measures student learning at the department level.

 

Title of assessment project:

Assessing the Speech Communication Senior Capstone Project

 

The following specific student learning outcomes were addressed by the project:

1.      The assessment determined if the Speech Communication Department curriculum is meeting the following departmental goal established in its Undergraduate Assessment Plan:  (to assess Learning Outcome #5 for BA/BS “Demonstrate critical thinking, communication proficiency, and knowledge of methodology by conducting a scholarly project utilizing a critical, humanistic or social scientific perspective” (see Appendix A).

2.      The Senior Capstone Project assessment addressed the following issues:

The assessment determined if the current Senior Capstone Project demonstrates appropriate and effective standards for the department and its students.

Specifically, the assessment raised the following questions:

·        Do the standards established for the Senior Capstone Project clearly reflect department goals and assumptions?

·        Do the standards established for the Senior Capstone Project clearly reflect disciplinary goals and assumptions?

·        Is the Senior Capstone Project an appropriate means of assessing the capstone experience?

·        What are the advantages of the Senior Capstone Project?

·        What are the disadvantages of the Senior Capstone Project?

 

The project assessment is related department assessment plan:

The Speech Communication Department is committed to a 5-year assessment schedule. A significant component of the assessment plan is the students’ demonstration of the outcomes defined in each Learning Outcome. This project was designed to assess the principles and practices of the Senior Capstone Project in order to assess its overall effectiveness and to directly measure student learning at the program level.

 

Assessment of the success of the project

 

The following questions were addressed as part of assessment process in order to determine the efficacy of the Senior Capstone Project:

 

1.      Do the standards established for the Senior Capstone Project clearly reflect department goals and assumptions?

Results of the assessment indicate the department has relatively clear expectations embedded within the department’s assessment plan. However, the associational link between the assessment plan and the curriculum is not being fully realized. SPEE 385: Communication Theory and Research and SPEE 485: Senior Seminar are the two primary courses established by the department for enabling students to effectively carry out at least one methodology as it relates to at least one communication phenomenon, and to use that methodology to critically evaluate that phenomenon. Exit interviews conducted with students in SPEE 485 in fall and spring semesters indicate a desire to engage in more original research. However, the current curricular approach does not provide sufficient grounding in research methods for students to effectively carry out an entire program of original research. The department still has students who started the program before SPEE 385 became a required course for all majors. Once all students must take SPEE 385, more stringent expectations may be implemented. The assessment plan was devised with this long-term goal in mind.

 

2.      Do the standards established for the Senior Capstone Project clearly reflect disciplinary goals and assumptions?

The assessment standards established for the project are clearly in line with disciplinary goals and assumptions. However, the department will need to review if the disciplinary standards under which the assessment operates are more appropriate at the graduate level than the graduate level. Disciplinary goals and assumptions at the undergraduate level are more nebulous than at the graduate level. Part of the department’s future agenda is to more carefully establish disciplinary goals and assumptions for an undergraduate senior capstone project.

 

3.      Is the Senior Capstone Project an appropriate means of assessing the capstone experience?

Exit interviews conducted with students in SPEE 485 in fall and spring semesters indicate the senior capstone project has proven to be a worthwhile endeavor. However, students also want and expect clearer standards for what shall be considered appropriate within the framework of a senior capstone project. The faculty involved in teaching SPEE 485 have given relatively broad latitude for what a student may write for their senior capstone project. The latitude was necessary due to the fairly recent incorporation of SPEE 385 into the curriculum. The department still has students who started the program before SPEE 385 became a required course for all majors. Once all students must take SPEE 385, more stringent expectations may be implemented. The assessment plan was devised with this long-term goal in mind.

 

4.      What are the advantages of the Senior Capstone Project?

Select comments from the student interviews best highlight the advantages of capstone project:

·        I think it is a good wrap up of our program. It helps us to make sure we have everything that we need to graduate.

·        It gives seniors an opportunity to really dig into a topic that may have captured their attention over their time here at MSU. Also, it is nice for some of us done with the major to be able to give one last speech in front of our friends.

·        use of all knowledge for good closure

·        Because everyone who graduates with a speech communication degree must demonstrate that they can effectively speak in front of an audience and write at a high level

·        I believe it forces students to organize themselves so they are able to prepare a heavy project. It forces students to recall things they have learned and apply them to thier current lives.

·        It makes you prove that you're capable of doing a project like this.

·        Requiring the senior project gives closure to a degree. You know you are finished and have something to show for what you have accomplished over the past four or more years.

·        I think it helps Speech Communication majors to utilize what we have learned and also have an indepth knowledge of our skills as a whole. The project also makes us aware of what we need to continue to work on when communicating whether it be written communication or oral.

 

5.      What are the disadvantages of the Senior Capstone Project?

Select comments from the student interviews best highlight the disadvantages of capstone project:

·        In a sense, it is sort of a repeat of what we have been doing all through school.

·        I wish our final presentation was more about our experience with the Speech curriculum instead of doing more research on other Speech topics.

·        I think it limits those who are capable of more, it does not give some people the chance to connect their future field with Speech Communication.

 

Plan for program improvement as a result of the assessment project, and
This assessment project highlights the need for better streaming of students through the curriculum in order to prepare the students for the senior capstone project. The assessment standards shall also be reviewed to determine if the original plan under which they were devised is appropriate, or if the broader latitude currently being employed is more appropriate and beneficial for the students in our programs.

Plan for assessing the effectiveness of program changes made.

Assessment rubrics will be reviewed and revised as appropriate in order to meet department program expectations. The assessment learning outcome rotation will be evaluated to ascertain if the senior capstone project should be moved up within the current cycle to provide for more timely review.

 


Appendix A

Assessment Plan for 2004-2005

 

Learning outcome #5 for BA/BS “Demonstrate critical thinking, communication proficiency, and knowledge of methodology by conducting a scholarly project utilizing a critical, humanistic or social scientific perspective.”

 

Rubric from assessment plan

 

3

Student plans, researches and carries out a scholarly project. The project requires that the student demonstrate the ability to work with at least one methodology as it relates to at least one communication phenomenon, and to use that methodology to critically evaluate that phenomenon. The project also requires that the student use and correctly cite appropriate academic sources. The project must have both a written and oral component. The written component demonstrates proficiency in written communication evidenced by few if any grammatical errors, correct citation style, and well-constructed paragraphs that develop a central thesis. The oral component demonstrates proficiency in oral communication through the use of appropriate verbal and nonverbal skills, and the ability to effectively synthesize written communication for an oral format.

2

Student plans, researches and carries out a scholarly project. The project requires that the student demonstrate the ability to work with at least one methodology as it relates to at least one communication phenomenon, and to use that methodology to critically evaluate that phenomenon. The project also requires that the student use and correctly cite appropriate academic sources. The project must have both a written and oral component. The written component demonstrates sufficiency in written communication evidenced by a writing style in which the form and content allows the reader to identify central claims and supporting evidence. The oral component demonstrates sufficiency in oral communication through the use of appropriate verbal and nonverbal skills, and the ability to synthesize written communication for an oral format.

1

Student plans, researches and carries out a scholarly project. The project requires that the student demonstrate the ability to work with at least one methodology as it relates to at least one communication phenomenon. The project also requires that the student use appropriate academic sources. The project must have both a written and oral component. The written component demonstrates minimal sufficiency in written communication evidenced by a writing style in which the form and content does not prevent the reader from identifying central claims and supporting evidence. The oral component demonstrates minimal sufficiency in oral communication evidenced by audience being able to identify a central focus and follow a progression of that focus to a conclusion.