This paper is a slightly shortened version of a presentation of Julia Pape-Neumann at the IFA-congress in Montreal 2003, published in the proceedings of the IFA-congress in Montreal 2003 (responsable for the shortening: Peter Schneider).

Abstract.

On the initiative of the German self-help organisation the PEVOS program was developed to evaluate different stuttering therapies over a time period of two years after therapy. The concept was designed by a group of therapists and scientists and was tested since 2001. In the test-phase data were collected from ten therapists with 100 clients. Fluency data were obtained by telephone calls. Functional outcomes and changes in attitudes and emotions were measured with questionnaires. Results of the test-phase¹s evaluation including the first two assessments will be presented, organisational problems and possible solutions will be discussed.

1. Introduction

In Germany there exists a contrast between the high number of stutter therapists offering different stuttering therapies and the low number of published studies, evaluating those therapies. Those few evaluation studies which were published used predominantly different measurement-instruments so that the results could hardly be compared to each other. So in times when German consumers and health insurances have increasing interest in formal evaluation of treatment efficacy there is a lack of objective and comparable data. For this reason the German stuttering association (Bundesvereinigung Stotterer-Selbsthilfe e.V.) invited therapists and scientists to develop an uniform program for the evaluation of different stuttering therapies conducted in the field in Germany. The Program for the evaluation of stuttering therapies PEVOS was constructed during several meetings of the working pool since 1998 (Langefeld et al., 2001; Oertle et al., 2001).

The aims of the program were as follows:

1. The usefulness of measurement-instruments integrated in this program should be evaluated.
2. Short- and long-term outcomes of stuttering therapies of different therapists with varying therapy programs should be evaluated with the same measurement instruments.
3. Stuttering therapies should be improved by feedback provided to the therapists about the results of their therapies.
4. The situation of people who stutter in Germany should be improved by offering exact and objective information about the long-term-outcomes of individual therapies.

2. Method

2.1 Evaluation procedure

In the beginning of the test-phase therapists registered those clients who were willing to take part in PEVOS-program, and those who refused to take part were registered anonymously. Data was collected immediately before (pre-treatment condition) and 4 to 6 weeks following therapy (post-treatment condition) and follow-ups will be carried out one and two years after the end of therapy. In intensive therapy programs the end of therapy was defined as the end of the intensive phase. Data were collected and processed independently of the therapists by employees of the German stuttering association: questionnaires were sent directly to the clients who were asked to return them directly to the association and speech samples were obtained by telephone interviews without appointments if possible. Speech samples of young children less than seven years old were recorded by the therapists.

At the end of the treatment the therapists indicated which therapy modules they had used in the particular therapy and they were also asked to specify the time over which these modules were administered. For this a list of modules (modules like "informing about stuttering", "relaxation", "identification", "prolongation" and "pullout") was provided. Clients were categorized in two age groups: the children's group consisted of clients younger than 17 and the adult group consisted of clients 17 years old or older. Because of different evaluation conditions for children of different age the children's group was further divided into subgroups of pre-school children (younger than 7 years), elementary school children (7-10 years old), pupils (11-14 years old) and youths (15-16 years old).

Therapists and their clients will be individually informed about their results after the first follow-up. Therapists are allowed to publish the results of their therapies on a non-selective base. The self-help association guarantees that results will be published only in such a form that privacy of both therapists and clients will be protected.

2.2 Questionnaires

Several questionnaires which had already been translated into German and partly modified and had been used in a few German evaluation studies before were combined in PEVOS. The following questionnaires were integrated in the program to evaluate the change of functional therapy outcomes, emotions and attitudes belonging to stuttering or communication in adults:

S24 (Erickson,1967; Andrews & Cutler, 1984; translated by Jehle, Kühn & Renner, 1989), avoidance PSI-V (Woolf, 1967; translated and worked over by Frischmuth, 1978), stuttering frequency in social situations (Brutten, 1973; modified by Fiedler & Standop, 1986), and a revised version of scales for assessing preferred functional outcomes of stuttering treatment (Cooper & Cooper, 1998). The latter is a translation of questionnaires, which were developed by the Special Interest Division No. 4 of the American Speech-Language-Hearing-Association.

Oertle (1999) designed a questionnaire for children out of several items from different questionnaires to reflect attitudes, emotions and behaviour related to stuttering and speaking. The questionnaire is identical for parents of the client and the child himself. Depending on the age of the child only the parents, parents and child or the child only have to answer the questions.

2.3 Speech samples

Speech samples were obtained from telephone interviews and consisted of 300 to 500 syllables. Following Yairi disfluencies were categorized as "stuttering-like" and "other disfluencies" (Yairi, 1997). "Stuttering-like" disfluencies in PEVOS are defined as repetitions of sounds and syllables, prolongations and blocks. "Other disfluencies" are defined as interjections, revisions, unfinished words, word repetitions and phrase repetitions. The frequency of syllables with stuttering-like and other disfluencies was counted and the overall-percentage calculated. Speech rate was calculated on the basis of spoken syllables which included stuttered syllables. Every spoken syllable was only counted once irrespective of how often it was repeated. Breaks between interviewer's and clients' utterances were deleted from the speech sample whereas breaks within the clients' utterances were not. Speech naturalness was rated with the 9-point scale of speech naturalness (Martin, Haroldson & Triden, 1984) with lower scores indicating more natural speech.

2.4 Subjects

Nine therapists registered 100 clients for the participation in PEVOS during the test-phase. Five therapists offered ambulant therapies in their private clinics in single sessions or single sessions combined with group therapy and four therapists offered intensive group therapies, one of them in a stationary setting. 19 of the registered clients had therapies in private clinics and 81 took part in intensive group therapies. 51 clients were older than 16 years (mean age=28.7 years, SD=12.4 years) and hence received questionnaires for adults. 49 clients were younger than 16 years (mean age=13.5 years, SD=3.4 years) and received questionnaires for pupils and/ or parents. 70 additional clients refused to take part and were registered anonymously.

3. Results

97 clients returned their questionnaires to the German stuttering association and 97 speech samples were obtained before therapy. At the second assessment 4-6 weeks after the end of therapy only 82 clients returned their questionnaires and 82 speech samples were obtained whereas 89 clients had finished therapy. 11 clients had not finished therapy yet.

3.1 Children

Speech data

Speech data immediately before and 4 to 6 weeks following therapy were based on 42 children. There was a highly significant decrease in the mean percentage of stuttered syllables between pre-treatment condition (M=11.93%, SD=11.06) and post-treatment conditions (M=5.00%, SD=5.85, effect size 0.63 - effects are strong if the effect sizes are 08 or bigger).

The mean speech rate from pre-treatment condition was 127.43 syllables/min. (SD=55.79) and increased very significantly to 147.98 syllables/min. (SD=53.85) in the post-treatment condition (effect size 0.37).

The overall mean speech naturalness rating for all children improved very significantly from 5.02 (SD=2.10) in the pre-treatment condition to 3.76 (SD=1.99) in the post-treatment condition (effect size 0.60).

Questionnaires

The evaluation of the children's attitudes, emotions and behaviour related to stuttering and speaking was based on questionnaires from 34 children and 20 questionnaires from parents at the pre- and post-treatment condition and showed a highly significant improvement. (The sum of added means of all items for children's questionnaires was 79.62 (SD=14.18) and 72.87 (SD=18.35) for parents' questionnaires. The effect size concerning the difference from pre- to post-treatment condition was 1.70 for the children's questionnaires and 0.95 for the parents' questionnaires).

3.2 Adults

Speech data

Speech data immediately before and 4 to 6 weeks following therapy were based on 43 adults. There was a highly significant decrease in the mean percentage of stuttered syllables between pre-treatment condition (M=13.69%, SD=11.56) and post-treatment conditions (M=4.81%, SD=6.19, effect size 0.77).

The mean speech rate from pre-treatment condition was 135.95 syllables/min. (SD=65.25) and increased very significantly to 158.98 syllables/min. (SD=56.53) in the post-treatment condition (effect size 0.35).

The overall mean speech naturalness rating for all subjects was 5.77 (SD=2.56) in the pre-treatment condition and improved very significantly to 4.65 (SD=2.32) in the post-treatment condition (effect size 0.44).

Questionnaires

The evaluation of avoidance was based on questionnaires from 35 clients at the pre- and post-treatment condition. There was an highly significant improvement between the mean score between pre-treatment condition (M=10.92, SD=3.86) and post-treatment condition (M=3.81, SD=3.55, effect size 1.84).

Attitudes and behaviour related to stuttering and speaking were evaluated by 39 S-24 questionnaires. The observed improvement between the pre-treatment score (M=18.15, SD=3.99) and the post-treatment score (M=8.93, SD=5.69) was highly significant (effect size 2.26).

Assessment of stuttering frequency in social situations was based on questionnaires from 41 clients at the pre- and post-treatment condition. The mean score improved very significantly between pre-treatment condition (M=1.98, SD=0.46) and post-treatment condition (M=2.95, SD=0.59, effect size 2.15).

Preferred functional outcomes of stuttering treatment were assessed by 40 FOAI questionnaires and showed a highly significant improvement between the pre-treatment score (M=3.28, SD=0.87) and the post-treatment score (M=5.41, SD=1.03, effect size 2.46).

4. Discussion

One of the characteristics of the present study is that data were collected from different therapists working with different therapy approaches, in different settings and with possibly different client populations. In spite of this heterogenity of source the present results allow a general conclusion: the clients who participated in different stuttering therapy approaches could improve the fluency of their speech as well as their attitudes, emotions and behaviour related to stuttering and communication from pre- to post-treatment condition although a final evaluation of therapy effects must be postponed until long-term catamnestic data are available. In any case the present data are encouraging in that they indicate an overall improvement in fluency and attitudes.

One important finding of this study is that improvements in attitudes, emotions or behaviour related to stuttering or communication or improvements in functional outcomes of therapy are more pronounced than improvements in speech fluency data. As reported before correlations of differences between pre- and post-treatment in stuttering frequency and questionnaires' scores were quite low. While the improvements at the post-treatment condition in all questionnaires showed strong effect sizes (between 1.84 and 2.46 for adults, 0.95 for parents and 1.70 for children) the effect sizes for percentage of syllables stuttered were comparably smaller (0.77 for adults and 0.63 for children).

There are two possible reasons for therapy effect size to be smaller for syllables stuttered. Therapy can either improve speech fluency of all clients to a smaller degree than communication attitudes or it can improve speech fluency of only part of the clients whereas it influences the attitudes of all clients in a comparable way. These alternative interpretations can be identified when the standard deviations of the pre- and post-treatment measures are taken into consideration. If therapy improves the speech fluency of only some clients but leaves that of others unaffected or even making it worse then therapy would increase the individual differences between clients. If on the other hand pre- and post-treatment measures show comparable variances this can be taken as an indication that effect size differences result from stronger or weaker therapy effects. As all standard deviation values (except for Erickson S 24) are comparable for pre- and post-treatment measures it can be assumed that weaker effect sizes in speech fluency compared to effect sizes of attitudes result from weaker therapy effects on speech fluency.

The test-phase of PEVOS was carried out to eliminate mistakes and optimise the program's concept. Several problems occurred while the first clients and therapists went through the program.

For some therapists it turned out to be difficult to register their clients in time so that the pre-treatment evaluation could take place before the beginning of therapy. In some cases the course of registration could be speeded up by improved communication between the stuttering association and the therapists whereas in other cases the therapy began before a client could fill in the questionnaires or give the telephone interview. Experiences showed that single clients should be registered at least 7-10 days and therapy groups at least 14 days before beginning treatment to enable the stuttering association to perform the pre-treatment evaluation in time.

In cases where clients did not return their questionnaires to the German stuttering association at all, the questionnaires were sent out a second time. If more than 10% of one questionnaire's items were not answered, the questionnaire was returned to the client and he was requested to fill it in completely.

It was difficult to reach clients on the phone so appointments had to be arranged and the changed interview condition was registered. Some Clients who initially became convinced of the program's advantages by their therapist indirectly refused to take part in the evaluation by keeping the questionnaires, leaving most of the questionnaires' items unanswered or being not cooperative during the telephone interviews. Experiences showed that therapists must leave the decision about participation in PEVOS to the clients themselves as pressure might have negative consequences on the evaluation.

Motivation to take part in post-treatment evaluation sometimes decreased when clients were not satisfied with the results of therapy. This became clear during the telephone interviews with clients who cooperated well at the pre-treatment condition and were less cooperative at the post-treatment condition. Some of these clients expressed their frustration about the therapy in the beginning of the telephone call. In these cases the employees of the stuttering association had to stress that the program is independent of participating therapists. Additionally objective information about therapy results were explained to the clients to improve their motivation.

To gain the necessary size of speech samples the standard set of questions for the telephone interview sometimes had to be extended by more detailed questions about the same topic when clients' answers were monosyllabic. Additionally the clients were instructed to answer as detailed as possible.

Speech samples of children younger than 7 years were recorded by the therapist because it was assumed that young children would not talk to strangers on the phone. The quality of the therapists' recording usually was worse than the quality of the telephone interviews so that the therapists will be loaned better recording equipment from the stuttering association in future evaluations.

Another problem of evaluation was the returning of the therapists' final report in which the therapists were asked to indicate for every client which therapy modules they had used and to specify the time over which these modules were administered. Some of the therapists forgot to send them back and thus had to be requested repeatedly to return them to the association. Additionally some therapists did not fill in one report per client but filled in only one report for a complete course. In these cases information on the therapy of individual clients was not available. In the beginning of the test-phase the standard form had to be supplemented by a second form which was specified for the needs of intensive courses because the standard form was too detailed.

The described problems showed that it is necessary to be in good contact with therapists as well as with the clients. Additionally it is indispensable to control the return of all forms and questionnaires that were sent to therapists and clients. In addition the program is very time-consuming and therefore expensive. PEVOS has been financed by the German stuttering association during the test-phase and its follow-ups. Several unsuccessful attempts have been made to secure funding for the main implementation of PEVOS. Considering these failed attempts it is necessary to think the concept over: it must be considered that therapists and/or professional associations of therapists contribute to the costs. At the same time it must be considered how the program could be less time-consuming.

Note: Besides the authors the working pool consists of Reiner Nonnenberg, Patricia Sandrieser, Peter Schneider, Nicole Spindler and Hartmut Zueckner.

Andrews, G. and Cutler, J. (1974). Stuttering therapy, The relation between changes in symptom level and attitudes. J. Speech Hear. Disord., 39, 312-319.

Bortz, J., Doehring, N. (1995). Forschungsmethoden und Evaluation fuer Sozialwissenschaftler. 2. Auflage. Berlin: Springer-Verlag.

Brutten, E.J (1973). Behaviour assessment and the strategy of therapy. In: Lebrun, Y and Hoops, R. (Eds.): Neurolinguistic approaches to stuttering. Mouton: The Hague, 82-89.

Cooper, E.B. and Cooper, C.S. (1998). The impact of identifying preferred treatment outcomes on conceptualizing, assessing, and treating chronic stuttering. unpublished work..

Erickson, R.L. (1969). Assessing communication attitudes among stutterers. J.Speech Hear.Res. 12:711-724.

Fiedler, P. and Standop., R. (1986). Stottern. AEtiologie, Diagnose, Behandlung, Weinheim:Psychologie-Verl.-Union.

Frischmuth, M. (1995). Untersuchung des Einflusses einer verhaltenstherapeutischen Gruppentherapie nach Van Riper auf verschiedene Verhaltensbereiche des Stotterns ­ unter Beruecksichtigung der Situationsabhaengigkeit von Veraenderungen. Unpublished draft, Universitae Hamburg, 1978; nach J.A. Renner: Erfolg in der Stottertherapie, Edition Marhold, Berlin.

Jehle, P., Kuehn, T. and Renner, J.A. (1989). Einstellungen Stotternder und Nicht-Stotternder zur Kommunikation: Einige Ergebnisse aus der Anwendung der Skala "S 24" von Erickson und Andrews/Cutler. Die Sprachheilarbeit 34:121-128.

Langefeld, S., Bosshardt, H.-G., Natke, U., Oertle, H.M., & Sandrieser, P. (2001) The German program for the evaluation of stuttering therapies (PEVOS). In H. G. Bosshardt, J. S. Yaruss & H. F. M. Peters (Eds.), Fluency Disorders: Theory, Research, Treatment and Self-help. Proceedings of the Third World Congress of Fluency Disorders in Nyborg, Denmark. Nijmegen: Nijmegen University Press, 359-360.

Martin, R.R., Haroldson, S.K., Triden, K.S. (1984). Stuttering and speech naturalness. Journal of speech and hearing disorders, 49, 23-58.

Oertle, H..M. (1999). Fragebogen für Schueler und Schuelerinnen (FF-SS)/ Fragebogen fuer Eltern (FF-E), unpublished questionnaire.

Oertle, H.M., Natke, U., Bosshardt, H.G., Sandrieser, P. (2001). Wie wirkt Stottertherapie? PEVOS, das Programm zur Evaluation von Stottertherapien der Bundesvereinigung Stotterer-Selbsthilfe e.V.. Sprache Stimme Gehoer 25, 39-41.

Woolf, G. (1967). The assessment of stuttering as struggle, avoidance and expectancy. Brit. J. Disord. Commun., 2, 158-171.

Yairi, E. (1997). Disfluency characteristics of childhood stuttering. In: R. F. Curlee and G. M. Siegel, Nature and Treatment of Stuttering. (pp. 49-78) Needham Heights: Allyn & Bacon.

You can post Questions/comments about the above paper to the authors before October 22, 2003.