Journal of Deaf Studies and Deaf Education Advance Access originally published online on September 28, 2005
The Journal of Deaf Studies and Deaf Education 2006 11(1):3-20; doi:10.1093/deafed/enj007
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Empirical Articles |
Look Who's Being Left Behind: Educational Interpreters and Access to Education for Deaf and Hard-of-Hearing Students
University of Colorado at Boulder
Boys Town National Research Hospital
University of Haifa
Correspondence should be sent to Brenda Schick, Speech, Language, and Hearing Sciences, 2501 Kittredge Loop Road, University of Colorado, Boulder, CO 80309-0409 (e-mail: Brenda.Schick{at}colorado.edu).
Received May 2, 2005; revised August 28, 2005; accepted August 30, 2005
For many deaf and hard-of-hearing students, access to the general education curriculum is provided, in part, by using the services of an educational interpreter. Even with a highly qualified interpreter, full access to the content and social life in a hearing classroom can be challenging, and there are many aspects of the educational placement that can affect success. The skills and knowledge of the educational interpreter are one critical aspect. This study reports results from a study of approximately 2,100 educational interpreters from across the United States. All the interpreters were evaluated using the Educational Interpreters Performance Assessment (EIPA), an evaluation instrument used to assess and certify classroom interpreters (see Schick, Williams, & Bolster, 1999). The results show that approximately 60% of the interpreters evaluated had inadequate skills to provide full access. In addition, educational interpreters who had completed an Interpreter Training Program had EIPA scores only .5 of an EIPA level above those who had not, on average. Demographic data and its relationship with EIPA ratings are explored. In general, the study suggests that many deaf and hard-of-hearing students receive interpreting services that will seriously hinder reasonable access to the classroom curriculum and social interaction.
 |
Introduction |
|---|
 TOP Introduction MethodResultsDiscussionSummaryAppendixReferences |
|---|
In many countries, educational services for students with hearing loss have changed markedly in the past few decades. For example in the United States, since the first federal law that mandated access to local community schools, many deaf and hard-of-hearing (deaf/hoh) students moved to those schools from center-based and residential educational programs (Jones, Clark, & Stolz, 1997
; Moores, 1992). Along with changes in the location of education, students with a hearing loss have been educated within the same classrooms as their hearing peers in increasing numbers. These changes have occurred in other countries, such as the United Kingdom (Powers, 2002), Australia (Power & Hyde, 2002), and Spain (Fernandez-Viader & Fuentes, 2004). For many of these students, both deaf and hard of hearing, provision of an educational interpreter is required to support classroom communication. Theoretically, the educational interpreter is one aspect of providing access to all teacher and peer communication, which allows the deaf/hoh student to learn in the same manner as his or her hearing peers. The deaf/hoh student has access to the classroom content, and hopefully also to the classroom social life, but many have raised questions about learning through an interpreter and how the educational experience may differ given that it is mediated and is not direct (Kurz, 2004; Marschark, Sapare, Convertino, Seewagen, & Maltzen, 2004; Ramsey, 1997; Schick, 2004).
Providing full access to a hearing classroom is very complex, and it involves more than just the skills of the educational interpreter (La Bue, 1998; Marschark, Sapare, Convertino, & Seewagen, 2005; Ramsey, 1997; Roy, 2000; Schick, 2004; Winston, 2004). Classrooms are complex social environments and they involve discourse styles that are unique to K-12 education. Accurate representation of all classroom communication is extremely challenging for many reasons. For example, classroom communication typically is distributed among multiple speakers, and an understanding of the content requires a student to integrate what many individuals say, not just the teacher. It is very difficult to represent this type of discourse, which often involves shifts in register as well as speaker. Interpreting results in a time delay for the deaf/hoh student, which can affect turn taking. In addition, teachers' and students' communication contains a great deal of information about their beliefs, expectations, and understanding that is not often contained in their language or vocabulary but rather in aspects of speaking, such as tone of voice and prosody (see Schick, 2004). This type of information seems to be particularly difficult for interpreters to convey. Deaf/hoh students have the additional challenge of coordinating visual attention to the interpreter and visual materials. In reality, we should acknowledge that even in the best of situations, deaf/hoh students who accesses the curriculum via an educational interpreter have a different educational experiences than their hearing peers.
Clearly, the quality of access to classroom content is highly dependent on the skills of the educational interpreter. These skills involve much more than the ability to interpret, although that is probably one major predictor of access. Educational interpreters need to have a complex constellation of interpreting performance skills, language skills, as well as knowledge of education and child development across an age span (Antia & Kreimeyer, 2001; Hayes, 1992; Schick, 2001; Seal, 1998; Witter-Merithew & Johnson, 2005). They need to be able to help implement an educational program for a diverse range of students, including those who depend fully on sign communication to those who have intelligible speech and can manage some communication situations independently. About two decades ago, a national commission was formed and funded by the U.S. Congress to review the state of education of the deaf. This commission stated that access to a classroom was a "mockery" if an interpreter was not qualified (Commission of Education of the Deaf, 1988, p. 103).
Even in optimal learning circumstances, we know very little about what students can learn through an interpreted education. There is evidence that deaf/hoh students may have more difficulty learning information through an interpreter in comparison to what their hearing peers learn, even from the same lecture (Marschark et al., 2004, 2005). For example, Marschark et al. (2004) investigated a group of postsecondary students who watched either an interpreted or transliterated1 version of a college lecture, as produced by experienced and Registry of Interpreters for the Deaf (RID) certified interpreters.2 Although the results showed no differences in learning of the material in the interpreted versus transliterated versions, the deaf/hoh students learned less than their hearing counterparts. For the hearing college students, the test scores ranged from 85% to 90%, but for the deaf/hoh students, the scores were lower, 60–75%. Marschark et al. conducted a thorough analysis of what background variables might be related to comprehension but found that none of their variables accounted for the differences, including: reading levels, degree or age of onset of hearing loss, parental hearing status, use of assistive listening devices, or the age at which sign language was learned. They also reported that the deaf/hoh students were less able to predict their own level of comprehension after watching a lecture than their hearing peers. It seems that it can be more difficult to learn and to know what you know when you learn through an interpreted lecture. Of course, the participants were college students at a university with respectable entrance requirements, and clearly the students had language and cognitive skills well beyond those of a student in elementary or middle school. The fact that learning through an interpreter might be more difficult highlights the need to have highly qualified professionals, qualified to work in a K-12 setting.
There is little information about how well children learn through an interpreter. There is some research that shows that children can learn in both direct and interpreted situations. Kurz (2004) investigated how much information deaf children learned when a lesson was interpreted compared with when a lesson was presented directly in American Sign Language (ASL). She investigated a group of 19 deaf children, with approximately half enrolled in direct-communication educational settings and the other half attending interpreted-educational settings. She compared how well they learned new information from lessons presented by two certified secondary science teachers, one a deaf educator, with an MA degree in secondary science education, who was a native signer, and the other a hearing science teacher who taught in a public school and has had educational interpreters work in some of his classes. The hearing teacher's lessons were interpreted into ASL by an interpreter who had RID certification and RID-Legal certification and who was a native signer with more than 25 years of experience. Kurz used a within-subjects design in which each student saw three different lessons with direct communication and three other lessons in the interpreted condition. She found that all students were able to learn in the interpreted condition. For two of the six lessons, the students learned more information in the direct instruction condition; comparisons in the other lessons did not reach statistical significance. Seven of the students scored comparably in both conditions, with scores that differed by less than 10%, and 2 of the 7 students scored higher in the interpreted condition. Interestingly, and possibly relevant, Kurz also found that the signing in the direct communication condition took almost twice the time as the spoken English version, even though the deaf and hearing educators worked together to balance the content and vocabulary in the lessons. This study shows that when the interpreter is highly qualified, even elementary-age students can learn in an interpreted setting. It is also important to note that Kurz used the traditional teacher lecture, which is not reflective of current educational practice that incorporates distributed discussion, student participation, and social interaction. It may be possible that an interpreted education provides reasonable access to classroom content when the teacher lectures, which Winston (2004) believes is the form of classroom discourse that is the most accessible to interpreting, but lecture may also be a relatively poor form of teaching, particularly with young children.
However, looking at learning over a longer time line indicates that in reality, an interpreted education may not support learning as well as it should. Research shows that the use of an interpreter is associated with differences in writing skills for deaf/hoh students. A study of the written language in deaf/hoh children in Grades 3–12 found that those students who used an interpreter also scored significantly poorer in measures of grammar, vocabulary, and story construction (Antia, Reed, & Kreimeyer, 2005). For the ability to produce grammatically correct sentences, the proportion of the variance explained by the use of the interpreter (.215) was greater than the variance accounted for by the student's hearing loss (.141) and grade (.096). Only gender and grade accounted for a larger proportion of the variance. Of course Antia, Reed, and Kreimeyer acknowledge that students who use an interpreter might be more likely to sign, which changes the educational experience for the student in more ways than just the addition of an interpreter. Yet they found that the degree of hearing loss explained only a portion of the data. They questioned whether the quality of the interpreted instruction might be a factor in causing this relationship. Further research might help us understand the impact of an interpreted education on skills that develop over long periods of time, such as writing and storytelling ability.
Another limiting aspect of research on learning through interpreting is the problem that most studies have included only deaf participants. We know that many hard-of-hearing students also utilize interpreters even when they have intelligible speech and can communicate well in some situations. For them, situations that involve complex topics and vocabulary, distributed discussions, noise, multiple speakers, to name a few, may be much more challenging and require an interpreter. We know nothing about how a student with usable hearing learns through an interpreter, but it might be different than what we see with deaf students.
It is clear that access to a general education classroom is different for deaf/hoh children, and perhaps reduced, even when a highly qualified interpreter is utilized. However, we know that many students receive interpreting services from individuals who are not highly qualified or in many situations even minimally qualified. In a study of the performance skills of interpreters working in public schools from 1991 to 1994, Schick, Williams, and Bolster (1999) found that even when school districts and interpreters volunteered to be evaluated, the majority (56%) could not meet minimum standards suggested by the state of Colorado at that time (3.5 on a 5.0 scale). It is important to note that in the United States, Colorado was virtually alone in the early 1990s in recognizing a need to establish minimum performance standards for interpreters who work in K-12 settings. Federal standards in the United States did not and still do not provide much guidance or criteria in terms of minimum requirements for interpreters who work with children. In fact, recent federal laws in the United States have established professional requirements for many individuals who work in the public schools; and the No Child Left Behind Act of 2001 specifies that only "highly qualified" staff work with children. But despite defining what constitutes highly qualified for many professionals and para-professionals, No Child Left Behind does not mention educational interpreters or attempt to define what highly qualified might mean for interpreting performance standards.
Fortunately, the number of states requiring minimum standards for educational interpreters has increased dramatically since the Schick et al. (1999) study of Colorado interpreters. Currently throughout the United States, approximately 25 states3 require some form of minimum performance standards. Typically, states have ensured performance skills by using one or more of a set of nationally recognized evaluation tools, which include RID certification or the test designed by the National Association of the Deaf (NAD),4 both of which were designed for adult community interpreting. Twenty-one states require some specified level of performance on the Educational Interpreter Performance Assessment (EIPA; Schick et al., 1999), a tool designed specifically for the educational setting.5 Most states have adopted a level of 3.5 as the minimum standard.
This article investigates the performance skills of educational interpreters who work in K-12 settings, both in elementary and secondary settings. It reports national data from a large sample of educational interpreters using the EIPA. Three primary questions were addressed by this research.
- Given the positive changes in public laws, requirements, and awareness, how probable is it that a student who is deaf/hoh will receive services from an interpreter who is highly qualified?
- What are the demographic characteristics of these interpreters in terms of completion of Interpreter Training Program (ITP) or a BA degree, years of experience in a general interpreting setting versus a K-12 setting, and how do demographic variables relate to performance?
- How well do interpreters perform in domains particularly relevant to education, such as the use of prosody, discourse mapping, finger spelling, managing distributed discourse, and representing key concepts?
|
Method |
|---|
|
TOPIntroductionMethodResultsDiscussionSummaryAppendixReferences |
|---|
Participants
All individuals who requested and completed an EIPA evaluation from 2002 through 2004 were included in this analysis, resulting in a sample of 2,091 interpreters, from more than 35 states and Canadian provinces; 25 states have at least 10 evaluations in the sample. Demographic data were available for a subset of these interpreters (n = 1,505) and are summarized in Table 1. Most of the interpreters were Caucasian (79%), with approximately 15% of the interpreters reporting membership in a minority ethnic group. A large proportion of the group reported that they had completed an ITP (46%), and a smaller proportion had completed a BA degree (26%). In general, the interpreters represent a broad range of experience, reporting on average 7.9 years of general interpreting experience (SD = 7.0) and 6.5 years in an educational setting (SD = 5.6). They were a relatively young group, with a mean age of 37 years (SD = 10.4). As a comparison, in 1992, the average age of a K-12 teacher in Colorado was 41 years, with 11.1 years of experience (Colorado Department of Education website
).
|
Measures
The EIPA is a tool that is designed to evaluate the interpreting skills of educational interpreters in a classroom setting (Schick & Williams, 1992
, 2004). The EIPA is not limited to any one sign language or system, which is essential given the diversity of sign languages used by deaf/hoh students in the public schools. The tool can be used to evaluate interpreters who use predominantly ASL, typically viewed as the sign language of the adult deaf community, predominantly Pidgin Sign English (PSE)6 the type of English signing found among the adult deaf community, or Manually Coded English7 (MCE; see Bornstein, 1990). There are also different versions of the EIPA for interpreters who work in an elementary school versus a secondary setting. Videotaped stimulus materials are used to collect two samples of the interpreter's work. The overall procedure is as follows:- Interpreter chooses grade level and language.
Collect two video samples of interpretation using EIPA videotapes.
- Video samples are evaluated using the EIPA rating form by three evaluators.
- Interpreters receive an overall score and specific feedback.
One sample is of the interpreter's Voice-to-Sign skills, translating or transliterating spoken English in the classroom environment into sign communication. The other sample is of the interpreter's Sign-to-Voice skills, translating or transliterating what a deaf child signs into spoken English. A specially trained evaluation team, using a standardized EIPA rating form, evaluates both samples. See Schick and Williams (2004) or www.classroominterpreting.org for a more thorough description of the tool and procedures.
The EIPA rates 37 different skill areas using a Likert scale of 0 (no observable skills) to 5 (advanced skills). See the Appendix for a summary of what each level represents in terms of skills. The score for each skill is the average of the three evaluators ratings. There are four main domains on the rating form that are evaluated:
- Voice-to-Sign Grammar: syntax, spatial grammar, and nonmanual aspects of prosody.
- Sign-to-Voice: production of a spoken English version of a student's signed communication.
- Voice-to-Sign Vocabulary: the range and depth of vocabulary, finger spelling, and numbers.
- Voice-to-Sign Overall: aspects of interpreting that are discourse based, such as discourse mapping and cohesion.
The EIPA design has several unique aspects that arguably contribute to its validity. First, it uses videotape segments of authentic classrooms to elicit an interpreting performance. Because of this, teachers are using their typical register and discourse style, which means that their language, vocabulary choice, interaction, and presentation style is a realistic representation of what interpreters are likely to encounter in typical classrooms. Because language and discourse directed at adults differ from that directed towards children and youth, it is more ecologically valid to use stimulus materials of actual classroom pedagogy. For example, EIPA videotapes of both the elementary and secondary classrooms include sections where there is distributed discussion and the interpreter must represent the other students' communication as well as the teacher's. A second aspect that contributes to validity is that the EIPA videotapes involve interviews of real deaf/hoh children and youth. Children sign differently than adults. In particular, elementary-school-age children continue to make grammatical errors and they are just beginning to understand how to monitor and repair conversations. Further, many deaf/hoh students have mild to severe language delays, which can make them more difficult to understand. A third contribution to EIPA validity is its use of current research and linguistic analysis to target the 37 particular skills that are assessed. For example, recent linguistic inquiry has shown that sign languages frequently use spatial mapping to represent discourse concepts, such as comparing and contrasting concepts (Winston, 1995), or to signal new topics or convey information about time lines. Because conceptual development is a major goal of education, this linguistic feature has educational relevance. Another group of items on the EIPA evaluates how well interpreters use prosody in sign language to communicate affect, grammar, and speaker intention. These aspects of communication are essential to the development of language (Fernald, 1989; Fernald & McRoberts, 1996), as well as a Theory of Mind, or social cognition (Schick, 2004). Similarly, research on the development of social cognition indicates that knowing the personality characteristics of classmates is key to social–emotional and personality development (Hartup, 1996; Newcomb & Bagwell, 1995). Therefore, an EIPA item assesses how well the interpreter represents communication when a hearing classmate is talking. The EIPA is a tool that was constructed for the K-12 environment, and many aspects of its design focus on aspects unique to that environment.
Procedures and Analysis
All participants requested an evaluation from the EIPA Diagnostic Center,8 and all evaluations were conducted at a site in the participant's regional area, proctored and managed by trained test administrators. Most of the participants were either working as educational interpreters or hoping to be employed as one. Prior to the evaluation, each participant selected the grade level (elementary or secondary) and the language (MCE, PSE, or ASL), with the help of preview materials. Each interpreter was provided with lesson plans for each classroom to be interpreted (five classrooms for the elementary version and two classrooms for the secondary version). Voice-to-sign sample and sign-to-voice samples of the interpreter's performance were sent to the EIPA Diagnostic Center where the performance was evaluated. Each sample was rated by a team of three trained evaluators. Two evaluators were hearing and held RID certification and one was deaf or hard of hearing, fluent in PSE and ASL. All raters were trained until they reached satisfactory performance levels. All scores, for each evaluator and for each rated item, were entered into a computer database for later analysis.
Reliability and Validity
In order to determine interrater reliability, a group of 20 videotaped evaluations were evaluated by two independent rating teams, who were blind to the fact that the individual had been previously rated. Estimates of interteam correlations and internal consistency are shown in Table 2. Interteam rating correlations, ranging from .86 to .94 for the different domains evaluated in the EIPA, revealed a high degree of consistency in ratings across teams. These results demonstrate that the EIPA is a highly reliable instrument when two independent teams rate the performance.
|
In addition, measures of internal consistency for the entire group of raters were calculated, as shown by Cronbach alpha estimates in Table 2. Internal consistency is an assessment of how reliable a tool is across all items comprising a domain. A high coefficient indicates that the individual items are performing reliably, that is, each item contributes in a consistent way to the overall EIPA domain score. A Cronbach alpha coefficient (essentially the average of all split half correlations) above .70 is considered acceptable, and a value of .90 is considered to be very good. Results showed very high coefficients of internal consistency, ranging from .93 to .98. Overall, results show that the EIPA assessment is highly reliable, both in terms of agreement among teams of raters and in internal consistency of skills comprising each domain.
Correlations between measured domains were calculated for the average scores as well as the total EIPA score, as shown in Table 3. Interdomain correlations suggest that they tap different aspects of performance, rather than each domain reflecting a common overall rating. This is especially true of the Sign-to-Voice domain, which had only moderate correlations with Grammar (.57) and Vocabulary (.56). This indicates that the Sign-to-Voice domain contributes unique variance to the total EIPA scores. Higher correlations were found for the Overall Picture domain with the other three domains, as well as between Vocabulary and Grammar.
|
Additional evidence on validity comes from examining the performance of previously certified interpreters. EIPA total scores are shown in Figure 1 for all interpreters who reported some form of RID certification (n = 42). For the group, the average EIPA score was 4.2 (SD = .06). As can be seen in the figure, there was somewhat high variability in the Sign-to-Voice domain, which indicates that RID-certified interpreters may not reliably interpret what a deaf/hoh student signs. Only one interpreter scored the maximum rating of 5 in all four domains, and very few consistently received a 5 in other domains. The results indicate that a RID-certified interpreter could be expected to obtain a score above 4.0 but would not be expected to easily achieve high ratings in all skills and domains. This pattern adds to the validity argument in that, as expected, RID-certified interpreters score in the Advanced range on the EIPA but do not reach the ceiling in most instances.
|
|
Results |
|---|
|
TOPIntroductionMethodResultsDiscussionSummaryAppendixReferences |
|---|
A multivariate analysis of variance (MANOVA) was conducted with the average score in each domain and the score in each domain (Grammar, Sign-to-Voice, Vocabulary, and Overall) as the dependent variables. Independent variables included the grade level, language, age as a categorical variable, how long the individual had been interpreting, as well as whether the individual had completed an ITP or a bachelor's degree (BA).
EIPA by Language and Grade Level
Comparisons of EIPA scores according to language (ASL, PSE, MCE) and grade (Elementary and Secondary) for each skill domain (Grammar, Sign-to-Voice, Vocabulary, and Overall) are shown in Figure 2. Results from the MANOVA showed a significant main effect for language for the Grammar domain, F(2, 1088) = 3.873, p = .02, effect size = .007, and the Overall domain, F(2, 1088) = 3.000, p = .050, effect size = .005. Post hoc analysis confirmed that MCE interpreters received lower skill ratings than the PSE interpreters who were lower than the ASL interpreters in every domain except vocabulary, where ASL and PSE interpreters were equivalent but both were significantly higher than MCE interpreters.
|
There was also a significant main effect for grade in two domains, Sign-to-Voice, F(1, 1088) = 43.746, p = .000, effect size = .039, and Overall, F(1, 1088) = 6.423, p = .011, effect size = .006. Secondary level interpreters had significantly better skills in each of these two domains than interpreters who took the Elementary version. Finally, there was a significant Grade x Language interaction in all domains except Vocabulary, Grammar: F(2, 1088) = 3.237, p = .04, effect size = .006; Sign-to-Voice: F(2, 1088) = 6.55, p = .001, effect size = .012; and Overall: F(2, 1088) = .946, p = .007, effect size = .009. Post hoc analysis confirmed that MCE interpreters in the Elementary levels had skill levels that were significantly lower in comprehending and interpreting children than interpreters in other grades and languages (p < .05).
It should be noted that the effect sizes for the significant comparisons were quite low, ranging from .001 to .039, indicating that although these comparisons are significantly different (due to the large sample size), the variables of grade and language account for only a small portion of the variance in the data.
EIPA Scores and Demographic Descriptors
For 62% of the interpreters, there were both EIPA evaluations and demographic data available (n = 1505). EIPA total scores were compared for those interpreters who reported completion of an ITP or a BA degree, as shown in Figure 3. Results from an MANOVA showed no significant differences for interpreters with a BA degree for any domain (p = .387–.640). There was a significant effect for completing an ITP in two domains, Grammar, F(1, 1088) = 2.895, p = .053, effect size = .003, and Vocabulary, F(1, 1088) = 5.083, p = .024, effect size = .005. ITP graduates score higher on the EIPA in only those domains. However, the effect size for these comparisons was not large, which means that only a small proportion of the variance is predicted by these variables. There was no significant BA x ITP program interaction (p = .272–.970). A small proportion of individuals reported a BA degree in interpreting (n = 32), but their average EIPA score was no different than those who had completed an ITP only (M = 3.4, SD = .81) and the variability was fairly high.
|
The data also were analyzed, using a MANOVA, to determine whether years of general interpreting experience was related to the total EIPA score. Results approached significance for Sign-to-Voice skills, F(5, 1088) = 2.129, p = .060, meaning that interpreters who worked longer also were better able to interpret children. There were no significant differences in EIPA scores for the other three domains (p = .170–.935). Age was a slightly stronger predictor, and the results were significant for each domain, Grammar: F(5, 1088) = 4.466, p = .000, effect size = .020; Sign-to-Voice: F(5, 1088) = 3.324, p = .006, effect size = .015; Vocabulary: F(1, 1088) = .000, effect size = .022; and Overall: F(1, 1088) = .000, p = .021. However, like other measures, the effect sizes were small. Post hoc testing for age group revealed that the 20-year-old interpreter group scored significantly higher than the 40- and 50-year-olds (p < .05) in every domain, but not higher than the 30- or 50-year olds. Clearly, there is not a simple "younger is better" relationship here.
The results of the MANOVA showed that the background variables predicted different amounts of variance for the different domains (R2: Grammar = .279; Sign-to-Voice = .388; Vocabulary = .230; Overall = .323. This means that at best, these background variables predict between a quarter to a third of the overall variance, leaving approximately 60% of the variance unexplained.
Proportion of Educational Interpreters Who Meet Minimum Standards
Most of the states that have adopted the EIPA to establish minimum performance standards have set a minimal level at 3.5 (13 of 17 states); two states require an overall EIPA rating of 4.0, and only one state requires 3.0 as minimum qualification. Figure 4 shows the proportion of the interpreters evaluated within each skill level. Only 17% of the interpreters would be able to meet minimum standards if set at 4.0, and 38% would meet minimum standards if set at 3.5.
|
Changes in EIPA Scores on Subsequent Testing
There were 205 interpreters who elected to take the EIPA a second time typically because they did not meet state minimum standards the first time. The average time between tests was nearly 1 year (M = 11.4 months, SD = 6.8) with waits ranging from 1 month to nearly 3 years, which represents the limits of the database. As shown in Figure 5, many interpreters showed gains from Time 1 to Time 2 (64%), but a small proportion demonstrated better skills at Time 1 than at Time 2 (27%). As a whole, subsequent testing revealed moderate gains of .3 EIPA level with a large variance (SD = .65). A Pearson correlation showed that the length of time between testing was not significantly correlated with improvement in EIPA scores (r = –.08, p = .26), meaning that interpreters who waited a longer period of time did not necessarily receive a better EIPA score. There was a significant negative correlation between the interpreter's EIPA score at Time 1 with the subsequent Time 2 score (r = –.48, p < .0001), meaning that interpreters who scored lower on the Time 1 testing were more likely to get a higher score on subsequent testing than those interpreters already scoring in the upper range. Such a result is expected when performers strive to meet a preset cut-score criterion, demanding low performers in Time 1 to demonstrate larger gains in order to meet the standard.
|
EIPA Scores for Skills With Particular Educational Relevance
Although the results show that, in general, educational interpreters are not highly qualified, it is important to specifically look at skills that may have direct educational relevance. For this analysis, several individual items on the EIPA were averaged to create composite domains in key areas: use of prosody, discourse mapping, finger spelling, indicating who is speaking, and representing key vocabulary. Results, as shown in Table 4, reveal particularly low skills in some domains that could be considered essential to learning. There were relatively low ratings for prosody, for example, which should be considered particularly important for younger language learners. Discourse mapping, or using space to represent cohesion, discourse structure, and relationships, was particularly problematic for all interpreters with average scores falling in the Advanced Beginner range on the EIPA. Similar results were found for representing key vocabulary and indicating who was speaking. Clearly, educational interpreters have difficulty with those aspects of the classroom content that are essential for development and academic learning.
|
|
Discussion |
|---|
|
TOPIntroductionMethodResultsDiscussionSummaryAppendixReferences |
|---|
Skill Levels of Educational Interpreters
Despite the important role that educational interpreters have in the education of deaf/hoh children, it is clear that many of them do not have the interpreting skills necessary to work effectively in classrooms. The majority of the interpreters evaluated in this study scored below an EIPA score of 3.5, considered a minimal proficiency level in many states (we do not really know what minimum level of proficiency would ensure access). Only 38% of the interpreters were able to meet that standard, even though the vast majority of the individuals who were tested were already working in classroom settings. The average EIPA score in this group of interpreters was 3.2 (SD = .8), with an average of 7.9 years of interpreting experience. The educational result is that the majority of students who use interpreters do not have access to the same classroom content as their hearing peers. The quality of the interpretation will most likely put these students at greater academic risk. That is, the accommodation that is intended to provide access to a free and appropriate education cannot ensure access to basic classroom content.
We also found that interpreters' skills vary by grade level and language. Interpreters who took the elementary version of the test scored significantly lower than those taking the secondary version for two domains, Sign-to-Voice and Overall, although the effect sizes were small indicating that these differences are not that great, there is a great deal of within-group variability. The most straightforward interpretation of this result is that elementary interpreters are less skilled than secondary interpreters. This means that the interpreters with the least adequate skills may be assigned to work with younger children, with the mistaken assumption that interpreting at that level is easier or that younger children have "less language" so that it is acceptable if the interpreter has less language.
It is possible that the differences between the elementary and secondary interpreters are, in part, the result of the task demands. There are significant differences between elementary and secondary classrooms in terms of teacher discourse styles, the tendency to teach interactively, and the use of child-directed patterns of prosody that are used in teacher-directed discourse with younger students (Cazden, 2001). There is a greater proportion of distributed discussion and co-constructed meaning. Teachers in the elementary classroom spend more instructional time providing feedback and monitoring of students' behaviors. Teachers in secondary classrooms tend to use more adult-like discourse forms with less-exaggerated prosodic structures. Middle school and secondary teachers tend to use more lecture-oriented teaching styles, which Winston (2004) indicated might be easier to interpret. It may be harder to interpret the content of an elementary classroom because of these classroom discourse issues. Of course, without further data, we cannot address this issue.
Participants also differed significantly in EIPA scores according to the language used, with significant differences in Grammar and Overall but not in Sign-to-Voice or Vocabulary. The ASL interpreters scored the highest on most measures, followed by PSE interpreters, with MCE interpreters scoring the lowest. In fact, MCE interpreters scored significantly lower than the ASL and PSE interpreters in their ability to understand elementary-age deaf children, scoring in the advanced beginner range as a group. Interpreters at this level would have difficulty understanding substantial portions of the student's communication (which may cause the student to simplify language and content in order to be understood). However, it is important to note that although there were significant differences for both grade level and language, the effect sizes were rather small. This indicates that there are differences among the groups, but the differences may not be functionally significant. Basically, there are both good and poor interpreters at both grade levels and using all languages, although MCE interpreters as a group have the weakest skills. It is notable that for both ASL and PSE interpreters, approximately half completed an ITP (45% and 47%) but only 20% of the MCE interpreters had done so. It is important to keep in mind that only 10% of the interpreters in the current sample requested an MCE evaluation; most interpreters took the PSE version.
It is a reasonable question to ask whether the results for MCE interpreters reflect a bias in the EIPA towards ASL and PSE. This does not seem likely for several reasons. The differences in vocabulary scores among the ASL, PSE, and MCE interpreters were not that great (M = 3.9, 3.8, and 3.5, respectively). The MCE interpreters have vocabulary skills nearly equal to the other interpreters. However, there are considerable differences in the scores in other domains, such as in the ability to understand and interpret the student signer (ASL = 3.2, PSE = 2.8, MCE = 2.1), the ability to use prosody correctly (ASL = 3.2, PSE = 2.8, MCE = 2.3), and the ability to use discourse mapping (ASL = 2.6, PSE = 2.0, MCE = 1.5). Our interpretation is that many MCE interpreters have learned sign language as a vocabulary translation exercise in which an English word is replaced with its sign equivalent. MCE systems, by design, attempt to faithfully represent the actual English words and order. However, these interpreters often lack prosody in their signing, which makes their signing look like a string of unrelated vocabulary items. In addition, the lack of prosody impacts their ability to represent the intention of the speaker, which results in lower scores on other EIPA items, such as "representing key concepts." The EIPA requires these elements and weights them consistent with their importance to development and education. It is highly likely that the MCE interpreters would score higher if the evaluation tool looked at vocabulary use only. However, even the manuals designed to teach MCE, such as the Signing Exact English (SEE) II manual (Gustason & Zawolkow, 1993), advocate the use of prosody and space in signing.
From the demographic data, we know that this sample is fairly representative of educational interpreters. Their ages and gender are similar to those reported by Jones (2004). However the current data are ethnically more diverse (approximately 15% ethnic minorities) than the samples reported by Jones (approximately 96% "White"). They also have more years of experience than the Jones samples: the current study reports a mean of 7.9 years of experience (SD = 7) in general interpreting whereas Jones reports 2–5 years. Jones sampled from predominately the Midwest, and the current sample reflects greater geographic diversity, which might better represent the population of educational interpreters in the United States. Given the large sample size (n = 2091) and geographical diversity, it is likely this sample reflects the national population.
It is equally important to consider the other half of these data, those educational interpreters who demonstrate skills above the EIPA standard used by many states, a 3.5 standard, who comprise a full 38% of the sample. There are likely many examples of deaf/hoh students who are receiving excellent services from interpreters who are well qualified to provide them. Interpreters in the Advanced range of the EIPA (4.0–5.0) score similar to individuals who have RID certification. The data indicate that interpreters who hold RID certification score high scores (M = 4.2) when evaluated using the EIPA. Concurrent validity with the RID is very good, indicating that the EIPA is not an easier test.
Interpreter Training, Education, and Skill Levels
Many of the interpreters in this sample had completed an ITP (46%), and 26% had completed a BA degree. Interpreters who had attended an ITP scored significantly higher in two domains, Grammar and Vocabulary, than those who did not, but only .4 of an EIPA level, with a very small effect size, which means that although these differences are significant, these are no strong differences between the groups. The ITP graduates were no different than nongraduates in their Sign-to-Voice skills, which means that they do not reliably understand children's signing. In fact, the average score of ITP graduates was below the common 3.5 standard, meaning that the typical ITP graduate would not meet states' minimal standards. As a group, the interpreters had 7 years of experience, which indicates that students who have just completed an ITP might score even lower. Similarly, those participants who had a BA degree scored higher than those who did not (by .3), even though they did not score above 3.5, yet the differences were not significant. The only group of interpreters who had an average score above 3.5 were those who had completed an ITP and also had a BA degree, although as a group, they were not significantly different than just having a BA or completing an ITP.
It is somewhat disheartening that formal interpreter training in the United States does not ensure that an educational interpreter will enter the classroom meeting minimal standards. This means that K-12 schools will have a very difficult time hiring an educational interpreter who is ready to work even if they hire an ITP graduate. There are many possible reasons for this, and researchers have speculated that 2-year programs cannot meet our training needs (see Davis, 2005; Jones, 2004). Many students enter an ITP with limited skills in sign communication. Most ITPs in the United States are 2-year associate degree programs in community colleges, not 4-year bachelor degrees. These programs attempt to teach language skills and interpreting skills in a very short period of time. However, neither the national RID nor the Conference of Interpreter Trainers (CIT), the U.S. organization of interpreter trainers, has recommended a 4-year ITP. It is also notable that the curriculum recommended by the CIT (2005) does not include any competencies related to the K-12 setting, rather it is designed exclusively for adult community interpreting. There are few programs in the United States that test exit competencies in interpreting, which makes program evaluation difficult. In addition, ITPs rarely provide specialized training in the types of discourse, and language is common in the schools and with children who are still developing language skills (recall that Sign-to-Voice skills did not differ between ITP graduates and nongraduates). In addition, Davis (2005) observed that most ITPs focus mostly on ASL and not the range of transliteration needed to work as an educational interpreter. Clearly our model of interpreter training is inadequate and does not address the needs of a large number of consumers—families, children, and schools—who need an educational interpreter who is qualified to provide access.
On the other hand, we believe that it is incorrect to conclude that ITPs necessarily provide inadequate training. Many ITPs are squeezing as many skills as possible into a 2-year program. Rather we would suggest that we need to ask a different question: What training is required to ensure that schools are able to hire graduates who are ready to work? Clearly our current training model cannot ensure this. These data also indicate that a generic BA degree added to a 2-year ITP program will probably not result in significant improvement over the current situation. It is hard to see how the qualifications of interpreters working in public schools will improve until we design a real 4-year program that allows skill development and focus on the developmental aspects of K-12 interpreting. It is likely that this is an impossible task within a 2-year program and that, as a field, interpreting training needs to adopt a model that focuses on exit competencies and not just an accumulation of sufficient credits. In addition, training programs should ensure that interpreters are prepared work within an educational and developmental model.
Improving EIPA Scores
Even though many of the interpreters who work with children do not have performance skills that are necessary to provide children access, it is not clear that they are receiving training that significantly improves their skills. For participants who took the EIPA a second time, the duration of time between the first and second test did not predict higher scores, even though the second testing was as much as 3 years later. That is, individuals who waited longer did not score higher on the second test. We do not have data that details the types of in-service training these participants received prior to the second test. It is possible that training helped specific interpreters, but as a group, more time did not help.
There are other data that help us understand what it takes to improve EIPA scores. One ITP has collected pretraining and posttraining EIPA scores from participants in an intensive educational ITP, developed by Johnson and Witter-Merithew (2004). They designed an ITP program for working educational interpreters, delivered using a mixed distance education and intense summer onsite sessions, which included 16 credit hours of skill development and 14 hr of content knowledge over a 3-year time period. Participants were assessed prior to entering the program and when they completed the program using the EIPA.9 For 31 individuals in Cohort 2 (2000–2003), the average gain was .8 of an EIPA level, nearly one level higher. In comparison, the interpreters in the current study made a gain of .3 of an EIPA level. Clearly, making significant and meaningful improvements in interpreting skill requires a curriculum with scope and sequence, as well as time and intensive training.
Accessing an Education
The results of this study are in sharp contrast with the role that educational interpreters have in the education of a deaf/hoh student. Many working interpreters have performance skills that will result in a fragmented message that is likely to have distortions, omissions, and simplifications of the teacher's message. An investigation of those skills with particular relevance to language and cognitive development in an educational setting found particularly low EIPA scores. The participants' ability to use finger spelling was not very different than their overall skills, which is positive given the strong relationship between finger spelling skills and reading vocabulary in deaf/hoh students (Sedey, 1995). However, other skills that are relevant to development and education were not as strong. All the participants scored low on the use of discourse mapping and prosody, both essential to convey meaning in ASL, PSE, and MCE. Participants also had difficulty indicating who is speaking, which is essential to understand distributed discussion and to know the personalities and characteristics of peers (Schick, 2004). Also, relatively low ratings were found for the ability to represent key vocabulary, important for recognizing key concepts and teacher emphasis. These data indicate that the educational interpreters had considerable difficulty representing those aspects of the message that have direct relevance to classroom instruction.
Quite simply, educational interpreters with inadequate interpreting skills can and do render the classroom content incoherent (La Bue, 1998; Langer & Schick, 2004; Napier, 2002, 2004). Interpreters with weak performance skills do not simply modify the teacher's message so that it is simpler. There are many omissions of concepts and concepts that are not understandable in the interpreted version. These random errors, distortions, and deletions must have a large, detrimental affect on a young learner, especially one who may already be behind his hearing peers. The classroom content, as it is delivered to the student, is unlikely to be the same as what the hearing peers are receiving.
The fact that it is children and youth who are receiving these inaccurate interpretations is also troubling. Hearing, deaf, and hard-of-hearing children are still learning how to use language in the elementary years, and school provides an essential form of linguistic input for all children. Even typically developing hearing children increase their language skills as a result of the teacher's input. Research has shown that the complexity of the teacher's language in preschool is significantly correlated with the children's comprehension of complex syntax at the end of the school year (Huttenlocher, Levine, & Vevea, 1998; Huttenlocher, Vasilyeva, Cymerman, & Levine, 2002). Other research shows similar effects for hearing students in first through third grade (Dorval & Eckerman, 1984). That is, even young hearing children are learning to comprehend and use grammar and discourse from the teacher's language model. In addition, language in the elementary school classroom is quite complex in terms of grammar, vocabulary, and discourse structure. A student with delayed language may not be able to learn very well from a language model that can only provide ungrammatical and simplified input.10
The language errors, omissions, and distortions in an unqualified interpreter's signing may be very difficult for a deaf/hoh student to recognize and discuss with either the interpreter or the teacher. Young hearing children do not have the cognitive skills to realize when a message is incomprehensible and they should ask for clarification. For example, Markman (1977, 1979) found that hearing children who are in third and sixth grade do not spontaneously recognize their own comprehension failure when they are presented with material that has obvious inconsistencies and contradictions. Markman concludes that the cognitive skills required to process and identify the source of the misunderstanding are complex and that even (hearing) preadolescents cannot manage this on their own (1979). This cognitive processing requires a student to monitor their comprehension, which requires metacognitive skills. They must maintain conflicting propositions in working memory in order to compare them and make inferences based on the information. Markman argues that preadolescent children may have the component cognitive skills, but coordinating them spontaneously is much more difficult (1979). She also concludes that comprehension monitoring has clear relevance for a child's ability to learn in school.
Of course, Markman studied hearing students who had age-appropriate language and perspective-taking skills, who were communicating directly with peers and teachers (1977, 1979). We know that many deaf/hoh students may have language delays compared with the hearing peers in their classroom. Research also shows that deaf children also show delays in their Theory of Mind skills (Courtin, 2000; Peterson & Siegal, 1999; Schick, de Villiers, de Villiers, & Hoffmeister, submitted manuscript), which will directly impact how they are able to consider information from various sources (Schick, 2004). In reality, the deaf/hoh student may be forced to monitor comprehension even more closely than his hearing peers, because the interpreted version is fragmented and often incorrect, which only increases the cognitive workload for a student who is already at educational risk. There is evidence that deaf students are less able to predict their comprehension than hearing peers (Marschark et al., 2004) and that adolescent students may overestimate their comprehension more than college-level students (Kurz & Langer, 2004).
Relevant to this discussion is the current model of interpreting that many interpreters advocate in which the responsibility for comprehension monitoring and negotiating clarification are the consumer's responsibility. The interpreter provides a faithful rendition, and if there are problems, the teacher and student should handle it. In many classrooms, interpreters convey classroom information but are not responsible for the student's comprehension, even when the interpreter knowingly misses and distorts content. In reality, this model is more appropriate for an autonomous and independent adult than it is in an educational system that has a legal obligation to ensure learning. In fact, most educational interpreters do not have the training and expertise to support the kinds of comprehension probing that these students may need. Clearly, we need to develop a model of interpreting for children and youth that is educationally and developmentally appropriate and is able to accommodate the range of children we see in education today, including children who are hard of hearing or have cochlear implants.
|
Summary |
|---|
|
TOPIntroductionMethodResultsDiscussionSummaryAppendixReferences |
|---|
Clearly, many deaf/hoh students are being left behind. It is likely that many students access their education through an interpreter who cannot provide full access. There are many reasons why we are faced with this current state of affairs. In the past, school districts and parents were poorly informed about interpreter qualifications, but that is rapidly changing. Clearly our current situation is, in part, due to our model of interpreter training in which the duration of the program dictates the skills rather than the skills we require dictating the duration of the program, not an attitude problem on the part of the schools. In addition, most ITPs provide limited exposure or training related to children. However, it is important to also note that whereas 25 states have minimum requirements for educational interpreters, we know of no state that requires teachers of the deaf to be fluent in sign language. It is easy to see the problems associated with an interpreted education, but it is also important to evaluate these problems in the context of the educational choices available to families. Although U.S. education law ensures educational access and choices for all deaf/hoh students, in practical reality, many families are faced with multiple compromises and no single ideal placement for their child.
In conclusion, there are many successful educational placements that involve an interpreted education. It is simplistic to see these challenges as an indictment for a form of educational access. The students interviewed by Kurz and Langer (2004) were well aware of the advantages and disadvantages inherent in their educational placement. The question is how to serve these students, not whether we will serve them. Families and the educational team retain that decision, as well they should. However, these results are a sober reality check of what deaf/hoh students confront on a daily basis. We provide a student at educational risk limited access to classroom content. We increase their cognitive workload because the content is interpreted and not direct communication. We assume a model where the student monitors understanding. It is difficult to predict that any student could make adequate yearly progress in these conditions. There are excellent interpreters who work in K-12 settings, and there is evidence that students can learn through an interpreter. We need to learn more about a model of interpreting for children so that no child is left behind.
|
Appendix |
|---|
|
TOPIntroductionMethodResultsDiscussionSummaryAppendixReferences |
|---|
Profile of Skills at Each Rating Level of the EIPA
Level 1: Beginner.
Demonstrates very limited sign vocabulary with frequent errors in production. At times, production may be incomprehensible. Grammatical structure tends to be nonexistent. Individual is only able to communicate very simple ideas and demonstrates great difficulty comprehending signed communication. Sign production lacks prosody and use of space for the vast majority of the interpreted message. An individual at this level is not recommended for classroom interpreting.
Level 2: Advanced beginner.
Demonstrates only basic sign vocabulary, and these limitations interfere with communication. Lack of fluency and sign production errors are typical and often interfere with communication. The interpreter often hesitates in signing, as if searching for vocabulary. Frequent errors in grammar are apparent, although basic signed sentences appear intact. More complex grammatical structures are typically difficult. Individual is able to read signs at the word level and simple sentence level, but complete or complex sentences often require repetitions and repairs. Some use of prosody and space, but use is inconsistent and often incorrect. An individual at this level is not recommended for classroom interpreting.
Level 3: Intermediate.
Demonstrates knowledge of basic vocabulary, but may lack vocabulary for more technical, complex, or academic topics. Individual is able to sign in a fairly fluent manner using some consistent prosody, but pacing is still slow with infrequent pauses for vocabulary or complex structures. Sign production may show some errors but generally will not interfere with communication. Grammatical production may still be incorrect, especially for complex structures, but is in general, intact for routine and simple language. Comprehends signed messages but may need repetition and assistance. Voiced translation often lacks depth and subtleties of the original message. An individual at this level would be able to communicate very basic classroom content but may incorrectly interpret complex information resulting in a message that is not always clear. An interpreter at this level needs continued supervision and should be required to participate in continuing education in interpreting.
Level 4: Advanced intermediate.
Demonstrates broad use of vocabulary with sign production generally correct. Demonstrates good strategies for conveying information when a specific sign is not in their vocabulary. Grammatical constructions are generally clear and consistent, but complex information may still pose occasional problems. Prosody is good, with appropriate facial expression most of the time. May still have difficulty with the use of facial expression in complex sentences and adverbial nonmanual markers. Fluency may deteriorate when rate or complexity of communication increases. Uses space consistently most of the time, but complex constructions or extended use of discourse cohesion may still pose problems. Comprehension of most signed messages at a normal rate is good, but translation may lack some complexity of the original message. An individual at this level would be able to convey much of the classroom content but may have difficulty with complex topics or rapid turn taking.
Level 5: Advanced.
Demonstrates broad and fluent use of vocabulary, with a broad range of strategies for communicating new words and concepts. Sign production errors are minimal and never interfere with comprehension. Prosody is correct for grammatical, nonmanual markers, and affective purposes. Complex grammatical constructions are typically not a problem. Comprehension of signed messages is very good, communicating all details of the original message. An individual at this level is capable of clearly and accurately conveying the majority of interactions within the classroom.
|
Acknowledgments |
|---|
Brenda Schick and Kevin Williams developed the original EIPA when they both worked at Boys Town National Research Hospital, where the tool is currently managed by Frances Beaurivage. This research was supported by the U.S. Department of Education, Office of Special Education Programs, Programs of National Significance award to Brenda Schick, Project Director (H325 N010013).
|
Notes |
|---|
1 Interpreting generally refers to the cross-rendering of two languages, such as English and ASL. Transliteration refers to a form of signing that represents the spoken language directly, such as using English grammatical structures and vocabulary. However, English transliteration borrows heavily from ASL, especially in those elements that are not lexical, such as prosody, nonmanual adverbial and clausal markers, and the use of spatial mapping for discourse and cohesion.
2 The RID is a professional organization in the United States that certifies interpreters who work with adults. See www.rid.org.
3 There is no full compilation of state standards for educational interpreters, and it is difficult to summarize requirements succinctly. Using state-specific data provided by L. Johnson (personal communication, April 30, 2005) and supplemented with other sources, we know the following (although this is probably not completely correct). Twenty-five states require some type of national test for educational interpreters (RID, EIPA, or NAD). Of these, 12 allow only the EIPA, 2 allow only RID, 2 allow RID or NAD, and 9 require either the EIPA, RID, or NAD. Eight states require some national assessment or a state-managed quality assurance program.
4 The NAD no longer administers their evaluation tool, but is developing, with RID, a new set of certification tools for community interpreters.
5 In many states, such as Colorado, requirements include standards on content knowledge, continuing education, and/or university degrees in addition to performance standards.
6 PSE, as it is intended in the EIPA, is a form of nativized English, used by members of the Deaf community. We do not refer to the type of English signing that hearing people produce because they are not yet fluent signers. We consider PSE to be rule governed, complex, and capable of representing a hybridization of English and ASL. PSE has also been described as a contact language (Lucas & Valli, 1989). See Davis (2005) and Kuntze (1990) for interesting discussions about nativized English signing.
7 MCE is the form of English signing that was developed specifically to teach deaf students English in a more accessible form. It follows the syntax and semantics of spoken English, although it borrows aspects of ASL, such as prosody, adverbial and clausal nonmanual morphology, and some spatial mapping. Interpreters do not have to follow a specific MCE system perfectly and, as with ASL and PSE interpreters, how well the message is conveyed is important, not just the ability to represent classroom communication using MCE signs.
8 Contact information for the EIPA Diagnostic Center: Boys Town National Research Hospital, 555 North 30th Street, Omaha, NE 68131, 402 452-5033 or e-mail: eipa{at}boystown.org.
9 The first assessment did not use official EIPA videotapes, but the ratings were conducted by the EIPA Diagnostic Center. The second assessment followed all current EIPA procedures.
10 Some school districts have adopted a rule that the student must have language skills within 1.5–2 standard deviations of his hearing peers in order to be placed in a full-time interpreted-education setting. Note that this is the same criterion used to qualify students as having special needs in the domains of language development, speech, and many other areas.
|
References |
|---|
|
TOPIntroductionMethodResultsDiscussionSummary |
|---|