Journal of Deaf Studies and Deaf Education Advance Access originally published online on April 28, 2006
The Journal of Deaf Studies and Deaf Education 2006 11(3):337-359; doi:10.1093/deafed/enj040
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
National Survey of Accommodations and Alternate Assessments for Students Who Are Deaf or Hard of Hearing in the United States
Online Research Lab
Walden University
Correspondence should be sent to Stephanie W. Cawthon, 15501 Waxler Court, Austin, TX 78754 (e-mail: scawthon{at}waldenu.edu).
Received January 9, 2006; revised March 28, 2006; accepted April 3, 2006
This paper reports the results of the National Survey of Accommodations and Alternate Assessments for Students who are Deaf or Hard of Hearing in the United States (National Survey). This study focused on the use of accommodations and alternate assessments in statewide assessments used with students who are deaf or hard of hearing. A total of 258 participants responded to the survey, including 32 representing schools for the deaf, 168 from districtwide/school programs, and 58 from mainstreamed settings. These schools and programs served a total of nearly 12,000 students who are deaf or hard of hearing nationwide. The most prevalent accommodations used in 2003–2004 statewide standardized assessments in mathematics and reading were extended time, an interpreter for directions, and a separate room for test administration. Read aloud and signed question-response accommodations were often prevalent, used more often for mathematics than in reading assessments. Participants from mainstreamed settings reported a more frequent use of accommodations than those in schools for the deaf or districtwide/school programs. In contrast, schools for the deaf were most likely to have students participate in alternate assessments. The top three alternate assessment formats used across all settings were out-of-level testing, work samples, and portfolios. Using the National Survey results as a starting point, future research will need to investigate the validity of accommodations used with students who are deaf or hard of hearing. In the context of the No Child Left Behind Act of 2001 accountability policies, the accommodations and alternate assessment formats used with students who are deaf or hard of hearing may result in restrictions in how scores are integrated into state accountability frameworks.
 |
Introduction |
|---|
TOP
Introduction
NCLB ContextStudents who are deaf or hard of hearing are a diverse group served in a wide range of educational settings (Gallaudet Research Institute [GRI], 2005
). Current educational policies, including the No Child Left Behind (NCLB) Act of 2001 (2002) (Public Law 107-110) and Individuals with Disabilities Education Act of 1997 (2001) (Public Law 108–4,46), emphasize assessment for all students, including students who are deaf or hard of hearing. Limited research has been published to date on assessment practices for this specific population of students with disabilities, particularly within statewide accountability frameworks. Cawthon (2004, in press-b) showed that students at schools for the deaf are participating in state assessments. In states that reported student participation and performance at schools for the deaf in 2004, 75% of schools reported student assessment participation rates of at least 95% (Cawthon, 2004). Case studies of accommodations with students who are deaf–blind showed that there is variability in the accommodations allowed and inconsistency in how they are implemented in classroom assessments (Horvath, Kampfer-Bohach, & Kearns, 2005). Available summaries for students who are deaf or hard of hearing do not, however, include information on the prevalence of accommodations use in statewide standardized assessments. Furthermore, current literature does not indicate assessment information for students who attend districtwide/school programs for students who are deaf or hard of hearing or who are enrolled in mainstreamed settings.1 The purpose of this paper is to report results from the National Survey of Accommodations and Alternate Assessments for Deaf or Hard of Hearing Students in the United States (National Survey) conducted in the spring of 2005 and reports 2003–2004 school-level data on accommodations and alternate assessment use for students who are deaf or hard of hearing. This study builds upon the current literature by providing data about assessment practices for (a) students who are deaf or hard of hearing, specifically, and (b) students who are enrolled in a range of educational settings. Below, we briefly review current NCLB federal policy on assessment and the purpose of accommodations and alternate assessments. The discussion then turns to factors that affect which accommodations and alternate assessments are used. Validity is a critical issue in the field. For this reason, the literature review includes discussion of the validity of the three prevalent accommodations: extended time, read aloud, and out-of-level testing. We then describe available research on the effects of accommodations on assessments for students who are deaf or hard of hearing. The article continues with a description of the National Survey methods, results, and implications for future research on assessment practices for students who are deaf or hard of hearing.
|
NCLB Context |
|---|
The goal of assessment is to identify areas where students are making academic progress as well as areas where schools, districts, and states may need to provide additional assistance to ensure student achievement. The NCLB Act of 2001 (2002)
is designed to guide high-stakes testing and to provide a uniform data reporting process for all schools. Large-scale, state-administered assessments are high-stakes tests because they are used as part of a broader decision-making process, such as graduation, or as a factor in school evaluations (Elliott, Braden, & White, 2001). Under NCLB, student scores on state assessments are used to develop school report cards of student progress. Decisions regarding school sanctions, such as required tutoring for students or, at its most severe, school reconstitution, are made based in part on student test scores. "Who" is tested and "how" they are tested are thus critical components of the NCLB framework.
The NCLB guidelines build upon the Individuals with Disabilities Education Act of 1997 (2004) regulations that require students with disabilities to be included in state assessments. Both policies emphasize that the majority of students with disabilities, including those who are deaf or hard of hearing, must be accommodated in standardized assessments. Although some individuals advocate for exemption from NCLB (Jarrell, 2005), these policies are intended to integrate students with disabilities into the same accountability framework as general education students. Successful integration of students with disabilities thus relies upon the capacity of states to provide appropriate assessment opportunities for all students.
|
Testing Accommodations and Alternate Assessments |
|---|
Testing accommodations are meant to increase the number of students with disabilities who are able to meaningfully participate in standardized assessments (Phillips, 1994
). "Accommodations" can refer to a range of changes to test administration and test content; they are designed to remove factors that penalize students because of their disability, resulting in assessment scores that do not represent their content knowledge. For example, students with a visual impairment may receive a Braille version of the assessment thus removing student inability to see the test as a barrier. The goal for accommodations is to make certain that the test measures content knowledge (target skill) and not the ability to take the test (access skill) (Elliott & Braden, 2000; Phillips, 1994; Shriner & DeStefano, 2003). The Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999) describes all test changes under the general umbrella of accommodations. However, all accommodations are neither considered equal nor to have similar properties (Koretz & Barton, 2003). Accommodations, in this paper and in other relevant literature, refer only to changes in test administration that also do not change the target skill of the assessment.
In contrast, modified tests, including alternate assessments, change both the target skill and the access skill of the assessment. Deleting an item and changing an essay response into a multiple-choice question are both examples of modifications that effectively change what is being measured by the assessment (Elliott, McKevitt, & Kettler, 2002). Alternate assessments are, in essence, comprehensive modifications to the standardized test format. For the purpose of this discussion, changes such as out-of-level testing, shortened assessments with items removed, or other changes to the assessment will be considered an alternate assessment. Descriptions of accommodations and alternate assessments used in this study are provided in Table 1.
|
|
Factors Affecting Accommodations Use and Participation in Assessments |
|---|
There is a growing body of literature to document the relationship between the use of accommodations and participation of students with disabilities in assessments. Also known as the Nation's Report Card, the National Assessment of Educational Progress (NAEP) provides nationwide, longitudinal data on the effects of accommodations on participation rates for students with disabilities. The results, so far, are mixed. In 1996, NAEP's use of accommodations increased participation in the 4th and 8th grades but not in 12th grade (Reese, Miller, Mazzeo, & Dossey, 1997
). Koretz and Barton (2003) point out that these findings did not continue in the 2000 administration of NAEP. As states move to comply with NCLB guidelines, it is assumed that an increasing number of students with disabilities will participate in standardized assessments with accommodations. This trend has been true for students who are deaf or hard of hearing and attend schools for the deaf (Cawthon, 2004, in press-b), where most schools reporting data had a near 100% participation rate. It is not yet clear, however, whether recent testing policies for students with disabilities have increased participation rates for students who are deaf or hard of hearing in district programs or mainstreamed settings.
Literature regarding assessment practices, including specific factors that contribute to accommodations use, comes from a variety of sources. Testing policies on a state- and federal-level articulate how accommodations are to be used within the context of standardized testing for accountability purposes. The National Center on Education Outcomes (NCEO) publishes regular reports on state policies regarding assessments and accommodations for students with disabilities (the most recent reports include Clapper, Morse, Lazarus, Thompson, & Thurlow, 2005; Minnema, Thurlow, Anderson, & Stone, 2005; Wiley, Thurlow, & Klein, 2005). Although students, whose primary disability is that they are deaf or hard of hearing, may come from homes where English is the family language, some of these students come from homes where English is not the family language. There are few policies (13 states) that explicitly outline accommodations for students who may fit in multiple-language use groups.
All states have policies for how to include students with disabilities in standardized assessments (Clapper et al., 2005). Some accommodations are "allowed without restriction," meaning that a test with the accommodation is viewed to be as valid as an unaccommodated test. Other accommodations carry restrictions in how the score is then aggregated with the overall assessment results. Large print is designated as "allowed" in most states (n = 47), as are individual administration (n = 46), sign interpreting for directions (n = 45), and read aloud directions (n = 35). Policies for accommodations that change presentation of the test questions were less likely to be allowed. Only three states allowed "questions" to be read aloud without restrictions, with the majority designating this accommodation as "allowed in certain circumstances" (n = 31) or "with implications for scoring" (n = 13). States were similarly cautious in policies to allow questions to be sign interpreted.
In addition to policies for the above accommodations, Clapper et al. (2005) also collected data on response accommodations that change how the student gives answers to the test items. Most state policies allowed a range of response accommodations without restrictions, including a scribe (n = 32), computer (n = 37), or writing in the test booklet instead of the bubble sheet (n = 35). Fewer states allow the student to use gestures such as pointing to the correct answer (n = 21) or responding with sign language to an interpreter (n = 20), without restrictions and implications for scoring. Depending on the prevalence of the use of accommodations for students who are deaf or hard of hearing, these policy guidelines have an impact on how student scores are integrated into the overall assessment framework.
Alternate assessment policies have also risen to the forefront of discussion surrounding inclusion of students with disabilities in state accountability frameworks (Wiley et al., 2005). Alternate assessments are meant to allow students with the most significant disabilities to participate in statewide assessments. NCLB provides guidelines on the use of alternate assessments and how scores are to be reported for school and state reports of Adequate Yearly Progress (NCLB Act of 2001, 2002). Under the original NCLB guidelines, no more than 1% of a school's test scores could be from students who participated in alternate assessments. More recent discussions have led to revisions to this cap, with the potential for up to 2%–3% of scores of students in alternate assessments that can be counted towards school proficiency rates (Federal Register, December 15, 2005). The remaining scores are counted for participation, but not towards proficiency rates. The impact of alternate assessment policies on inclusion of student scores therefore depends, in part, on the percentage of students taking alternate assessments at a school site.
|
Prevalence of Accommodations Use For Students With Disabilities |
|---|
At present, there is more information on policy than on the prevalence of specific use of accommodations in assessment (Koretz & Barton, 2003
). Much of the current literature focuses on assessment within a single state. For example, Johnson, Kimball, Brown, and Anderson (2001) studied the impact of accommodations in the state of Washington. The authors sampled from all districts as part of the review of the 1998 Washington Assessment for Student Learning. Read aloud was the most prevalent accommodation across all categories in this study. Also, elementary-grade students received more accommodations than those in junior high school. The authors suggest that differences between elementary and junior high school accommodations use may be due, in part, to the fact that elementary-grade students tend to have one teacher, whereas middle school students are served by multiple instructors. The more teachers involved, perhaps the less flexible teachers can be in providing test accommodations.
Kentucky was one of the states that urged full inclusion of students with disabilities in assessment before NCLB, largely through the use of accommodations (Thurlow et al., 2000). In the Kentucky analysis, nearly half the students with disabilities had someone read the test to them as an accommodation (read aloud). Smaller proportions of students were given the test in a paraphrased format (31%) or with a scribe (20%). Even with this full-inclusion policy, accommodations used on the state tests were unevenly applied to other standardized or classroom tests. Extended use of accommodations peaked in 1996 but declined once again in 1997. The relationship between policy and practice is therefore important but is only one factor in the prevalence of the use of accommodations for students with disabilities.
|
Validity of Tests Taken With Accommodations |
|---|
Accommodations are used to increase the proportion of students with disabilities who participate in standardized assessments. The validity of accommodated assessments therefore remains a central question in the research literature. Validity refers to the extent to which a test measures what it is meant to measure and gives a meaningful representation of student knowledge (Messick, 1995
). A valid accommodation will improve student access to the test content without altering the skill that is meant to be measured (Phillips, 1994). Accommodation validity is a complex construct that includes considerations of how the students' disability interacts with the test format, the accommodation, and the construct being measured (McDonnell, McLaughlin, & Morison, 1997). Although validity is described here as a single concept, validity itself is a result of multiple factors (Elliott et al., 2002; Fuchs & Fuchs, 2001).
Although still limited, an empirical base for determining the validity of accommodations use for students with disabilities is emerging. For example, Elliott, Kratochwill, and McKevitt (2001) looked at the impact of accommodations use on student achievement scores. The same students were tested both with and without accommodations, allowing for an intergroup analysis of test results. Accommodations showed a medium-to-large effect size on scores for students with disabilities. Phillips (1994) asserted that one can determine whether accommodations are effective when students with disabilities benefit more from the use of these accommodations than those without disabilities who also receive the accommodations (differential boost). However, students "without" disabilities in the Elliott, Kratochwill, et al. study also showed an increase in performance when they participated in the assessments with accommodations. Although this differential boost was only half as much for those without disabilities when compared to the boost for peers with disabilities, this finding is still important when considering the validity of the accommodations. If the purpose of accommodations is to offset access issues to assessments due to a disability, there should not be a significant increase in scores for students without disabilities. Some accommodations currently used in large-scale assessments may therefore act as modifications because they change both the access and the target skills measured by the assessment.
Valid use of accommodations is a difficult balancing act between removing barriers to test participation and not overestimating student proficiency on the area assessed (Koretz & Barton, 2003; Tindal & Fuchs, 2000). Accommodations should allow students with disabilities to perform their best. However, are they meant to allow them to perform to the point where they are taking a different test? The remainder of this discussion on accommodations validity focuses on three specific accommodations and modifications: extended time, read aloud, and out-of-level testing. Extended time is one of the most common accommodations and has been one of the most extensively researched in the empirical literature. Read aloud is one of the most controversial accommodations, particularly when used for assessments of reading skills. The discussion regarding read aloud is centered not only on psychometrics but also on the purpose of reading assessments and on fair inclusion of students with disabilities. Out-of-level testing is also controversial and is usually considered a modification or an alternate assessment. A student's individualized education plan (IEP) team is the decision-making vehicle for accommodations students receive both in instruction and assessment. Out-of-level testing is used when the IEP team believes that a student will be more accurately assessed at a different (usually lower) grade level than for same-aged peers.
Extended Time
Extended time is one of the most commonly used accommodations (Elliott, Braden, et al., 2001; Thurlow & Bolt, 2001). The assumption behind granting extended time is that timed tests discriminate against students with disabilities because parsing the meaning of test problems is often a challenge for students with reading disabilities or other processing deficits. By increasing the time available to complete each problem, extended time allows students to demonstrate their knowledge of the concepts, not the speed with which they complete the tasks. Depending on the test, extended time can be "time and a half," "double time," or "unlimited time." Because extended time can also lead to fatigue, this accommodation may be given in conjunction with frequent breaks or multiple testing periods.
Retrospective analysis of student performance with extended time shows that this accommodation may improve scores for students with disabilities (Huynh, Meyer, & Gallant, 2004). Yet, the empirical evidence for the effectiveness of extended time is mixed (Abedi, Courtney, & Leon, 2001; Abedi, Lord, & Hofstetter, 1998). Fuchs, Fuchs, Eaton, Hamlett, and Karns (2000) and Fuchs, Fuchs, Eaton, Hamlett, Binkley, et al. (2000) describe the findings of an empirical study on the use of extended time for students with learning disabilities. On average, students with and without learning disabilities showed no increased score differences with the extended time condition. In a further analysis, the authors focused specifically on students who saw significant (greater than 1 SD) increases on the accommodated test. A total of 23% of students with learning disabilities showed a significant increase on their reading scores, while 10% showed a significant increase on their mathematics tests. Fuchs, Fuchs, Eaton, Hamlett, and Karns and Fuchs, Fuchs, Eaton, Hamlett, Binkley, et al. found smaller proportions of students who showed significant gains on other accommodations such as large print, calculators, and read aloud. These findings are important in that they demonstrate how some students can truly benefit from extended time accommodations. Although group averages did not show significant differences, individual performances showed significant improvements. This finding supports the need to focus on individual student characteristics when assigning accommodations, not diagnostic or disability group membership.
Read Aloud
The read aloud accommodation is controversial (Elliott et al., 2002), particularly for use with a reading test. "Read aloud" generally refers to an accommodation where the test questions are read to the student. This accommodation leads to the following questions about validity. Is listening to test items (or viewing sign language) the same as reading them for the purpose of these assessments? If the test is meant to measure the ability of the student to decode the printed word, comprehend its content, and make inferences about its meaning, read aloud accommodation risks changing the intent of the test. However, if the intent focuses only on the last steps, to comprehend content and make inferences, does the mode by which information is received (through the printed word or through read aloud/signed administration) affect the purpose of the test? The validity of the accommodation depends, in part, on whether the purpose of the assessment is to measure the ability to decode information or to demonstrate reading comprehension. These questions have serious implications for students with English language delay or deficits, including those who are deaf or hard of hearing. If decoding text is a significant component of a student's disability, a read aloud accommodation may remove decoding as an access skill.
Tests of the validity of read aloud accommodation yield inconclusive results. For example, McKevitt and Elliott (2003) studied the effects of read aloud accommodations on reading test scores. Eighth-grade students in special education and general education were tested using the TerraNova assessments with teacher-recommended accommodations, both with and without an additional read aloud accommodation. Read aloud did result in an average increase in test scores for both groups of students. This effect is in contrast to the accommodations that the teachers recommended (aside from read aloud) that did not result in a significant increase in test scores. These findings imply that although read aloud is beneficial for students with disabilities, it may not provide the differential boost associated with valid accommodations.
In contrast, Huynh et al. (2004) described the effects of an oral administration of the mathematics portion of the South Carolina Exit Exam (high school graduation exam). This retrospective study included a regression analysis of accommodated versus standardized test administration, as well as student disability characteristics. The archival analysis in Huynh et al. determined that students with disabilities participated in both forms of testing, standard and orally administered tests. Students with disabilities benefited from the oral administration, but only to a small extent. The authors interpreted the small effect size as a reasonable way to "level the playing field" for students with disabilities in standardized assessment. Similar findings from Johnson et al. (2001) and Karkee, Lewis, Barton, and Huang (2003) support this claim.
The impact of read aloud as an accommodation may vary by the type of test items on the assessment. In a comparison of multiple-choice and reading passages, Koretz and Hamilton (1999, 2001) found that, overall, students without disabilities did better on items that were administered with the read aloud accommodation than the unaccommodated condition. In contrast, students with disabilities scored relatively lower on read aloud versions of the multiple-choice items. The effect of read aloud on student scores for student-constructed items (such as responses to short-answer questions) showed a different pattern. Students with disabilities who received the read aloud accommodations did relatively better on the reading passages with short-answer questions. Although the question content was not controlled for, this finding does indicate that the read aloud passages are easier for students overall, particularly for student-constructed items.
Out-of-Level Testing
Out-of-level testing refers to the practice of giving a student an assessment designed for a grade above or below the student's grade (or age) level. Although the test is still a standardized assessment, out-of-level testing is considered to be a modified or alternate assessment practice. Only 12 states permitted some form of out-of-level testing in the 2003–2004 school year, the targeted time frame for this study (National Center on Education Outcomes [NCEO], 2005). Depending on the state policy and data reporting practice, passing scores may be folded into those at the student's grade level or reported as "not proficient" (Minnema, Thurlow, Bielinksi, & Scott, 2000).
If a student is learning below grade level educational content, giving a student an out-of-level test may more appropriately measure what the student knows or is learning in classroom instruction. At the very least, it may lower potential frustration students may have completing an assessment that is too far above or below their ability level (Minnema et al., 2000). Yet, there are few empirical data about the impact of alternate assessments on student scores on achievement assessments (Thurlow & Minnema, 2001). One concern for state assessment practices is how to score out-of-level assessments in relation to the overall data reporting in an accountability framework (Minnema et al., 2000). If used as an alternate assessment, out-of-level assessments would likely fall under the policies outlined in NCLB for those who cannot meaningfully participate in standardized assessments, even with an accommodation. Proposed revisions to NCLB policy designates that up to 3% of alternate assessment scores can be counted towards school proficiency benchmarks (Federal Register, December 15, 2005). However, some scholars note that the increase will not benefit many schools because the percentage of students who participate in alternate assessments is greater than 3% of the population (Popham, 2005). Schools for the deaf or schools with programs for the deaf or hard of hearing may be among those that have more than 3% of students participating in alternate assessments. An increased focus on alternate assessments, including out-of-level policies, will continue to be important in our understanding of assessment practices for students with disabilities.
|
Students Who Are Deaf or Hard of Hearing |
|---|
There is very little information on the impact of testing accommodations with students who are deaf or hard of hearing. Overall, studies of testing accommodations for students with disabilities do not have sufficient numbers of students who are deaf or hard of hearing to report specific findings for this population. Most of the available research for students who are deaf or hard of hearing looks at the effects of different communication modes as an accommodation in intelligence testing. For example, Sullivan and Schulte (1992)
examined differences in test results when tests were administered either by the test administrator or an interpreter. Signed tests were signed in the child's mode of communication (e.g., American Sign Language or Signed Exact English) by the test administrator, whereas the interpreted test was interpreted through a separate staff person. Sullivan and Schulte found that there was little effect of administration on Wechsler Intelligence Scale for Children-Revised (WISC-R) intelligence test scores.
However, Braden (1992) demonstrated that there may be biases in testing due to test administration or format. For students who are deaf or hard of hearing, particularly those who use signed communication, verbal administration was likely to lead to lower scores than signed administration. Furthermore, performance assessments and nonverbal tests yielded higher mean IQ scores than verbal tests. In tests of student achievement, alternate assessments may skew student test scores higher than standardized assessments. These findings are relevant to this discussion because there is significant variability in the assessments and accommodations used by states under NCLB. Issues of interpreter quality are also essential in understanding the impact of an interpreted test on score validity. Variability in test format and administration raise questions about how to meaningfully interpret test scores for students who are deaf or hard of hearing.
|
Study Motivation and Objectives |
|---|
Appropriate accommodations that do not invalidate the assessment are important for the successful inclusion of students with disabilities in large-scale assessment (Elliott et al., 2002
). Some accommodations may change what is being measured (target skills) in addition to alleviating access issues for students with disabilities. For students who are deaf or hard of hearing, we do not know the extent to which specific accommodations improve test scores on statewide standardized assessments. Before the field can empirically test the impact of accommodations on scores for students who are deaf or hard of hearing, we must first understand what accommodations are used in statewide standardized assessments. The extent to which educators use alternate assessments (including out-of-level testing or special norms) for students who are deaf or hard of hearing is also unknown. Because NCLB regulations include specific guidelines for the use of alternate assessments, it is important to know how students who are deaf or hard of hearing may be affected by current and revised testing policies. The purpose of the National Survey was to gather national data on assessment practices for students who are deaf or hard of hearing. This study specifically looked at accommodations used for statewide standardized assessments in mathematics and reading. Participants also provided information on alternate assessment formats used by students who did not participate in state standardized assessments. The guiding research questions for this study are as follows.
- What are the most prevalent accommodations used with students who are deaf or hard of hearing on state standardized assessments? To what extent are extended time and read aloud used as accommodations for deaf or hard-of-hearing students?
- What differences are there, if any, in the accommodations used by schools for the deaf, districtwide/school programs for students who are deaf or hard of hearing, and mainstreamed settings?
- What are the most prevalent forms of alternate assessments used with students who are deaf or hard of hearing? To what extent is out-of-level testing used as an alternate assessment for students who are deaf or hard of hearing?
The National Survey served as a starting point to continue investigation of accommodation validity with students who are deaf or hard of hearing. By identifying key issues in assessment practice, subsequent work can focus more specifically on the impact of accommodations and alternate assessment on evaluations of student achievement.
|
Methods and Results |
|---|
Participants
The goal for the National Survey was to survey teachers and administrators who work with students who are deaf or hard of hearing in a range of settings. The challenge, however, has been that we often do not know where mainstreamed deaf and hard-of-hearing students are educated (Mitchell, 2004
). Recruitment strategies for this study were therefore conducted on multiple fronts. Primary recruitment was from the contact list for the Annual Survey of Schools and Programs conducted by the GRI (2005) as well as a national random sample of school districts. The national random sample was conducted because there are a number of schools and programs as well as many mainstreamed settings that are not included on the GRI contact list. Students represented by the GRI survey results represent only about half the estimated number of students who are deaf or hard of hearing nationwide (Mitchell, 2004). The national random sample participants were sent, where available, an e-mail invitation to the study. Those without working e-mail addresses were sent an invitation postcard by regular post. Recruitment was supplemented with awareness-raising efforts through contacts at the researcher's host institution, groups that serve individuals who are deaf or hard of hearing, and invitations to state and regional program administrators. A detailed description of recruitment strategies and response rates can be found in Cawthon (in press-a). Data from a total of 258 schools and programs were used in the current analysis. Participants identified which educational setting category best described their institution or agency. Schools for the deaf (n = 32) were either public or private schools that served students with either residential or day programs in separate facilities. Districtwide/school programs were those settings where students were served in a district program housed in a public school (n = 168). The last category, mainstreamed, describes settings where students are served by a public school but are not a part of a larger district-sponsored program (n = 58).
The approach for calculating response rates varies by educational setting. The highest response rate was from schools for the deaf. Of the estimated 90 schools for the deaf on the GRI list, 37% (n = 32) responded to the current study. The response rate for districtwide/school programs from the GRI list was much lower than that of schools for the deaf. Of the estimated 790 district and regional programs for students who are deaf or hard of hearing, a total of 21% (n = 168) responded to the survey. It is challenging to determine the response rate of mainstreamed settings. Our national random sample efforts targeted 987 districts across the country; it is unknown how many of these districts serve mainstreamed students who are deaf or hard of hearing. A total of 58 schools with mainstreamed students participated in the survey; 24 of these were matches from the original national random sample.
Online Survey
The National Survey was made available for participants from January through May 2005 on the study Web site, http://www.dhh-assess-survey.org. The survey instrument was administered via an online survey developed using http://www.surveymonkey.com. The survey consisted of four parts: demographics, accommodations, alternate assessment, and participant perspectives. Findings reported here are from the first three sections of the survey; results from the participant perspectives section will be addressed in a separate report.
School demographics.
Each participant represented a single school or program site. Due to the varied characteristics and sizes of schools and programs, participants served students in various capacities, including teachers of the deaf, program administrators, and district assessment coordinator. Although the respondent's own identity remained anonymous, the survey included items such as the state, school/program name, and district name to ensure that there were no duplicate respondents for the same student population. For those few (n = 4) schools with duplicate records, those with complete responses to survey items were included over incomplete surveys. In the demographic section, participants described school setting characteristics and the students it served. School- and program-related items included grades served, communication mode(s) used in instruction, sign language services, and itinerant services. Student-related items in the demographics section included number of students with different hearing levels, cochlear implants, additional disabilities, and IEPs.
Respondents were aggregated first by state and then by census region to look at the overall geographic representation of the study participants. Table 2 shows the distribution of participants by educational setting and census region.
|
Because the unit of analysis for this study was the school or program that served students who are deaf or hard of hearing, each participant reported data for a group of students. As shown in Table 2, nearly 12,000 students are represented by the 258 schools and regional programs included in this analysis. This number is likely an underestimate of student enrollment as four schools and programs did not provide student totals for this study. Of the 11,975 students represented in this study, 4,734 (40%) were served by schools for the deaf, 7,089 (59%) by districtwide/school programs for deaf or hard-of-hearing students, and 152 (1%) in mainstreamed settings. Compared to national averages, this distribution overrepresented students who attend schools for the deaf and who live in the South census region, underrepresenting those who attend mainstreamed settings or who live in the Northeast and Midwest census regions (GRI, 2005
; Mitchell, 2004).
The average number of students varied by educational setting F(2, 251) = 41.9, p < .01. On average, schools for the deaf served more students (M = 148, SD = 120) than either of the other two settings. Likewise, districtwide/school programs served more students (M = 42, SD = 72) than mainstreamed programs (M = 2.8, SD = 1.8). However, a closer look at the distribution of enrollment by educational setting reveals important information about the program size. Student enrollment data were used to create a program-size variable: small (7 students or fewer), medium (8–23 students), and large (24 or more students). A chi-square analysis, 
2(2, N = 257) = 121, p < .01, revealed a significant difference in distribution of educational setting by program size. With the exception of two schools, all the schools for the deaf were large programs. Mainstreamed settings were predominantly small programs, with only one serving more than seven students who are deaf or hard of hearing. Unlike the schools for the deaf and mainstreamed settings, which showed clear patterns in one end of the spectrum or the other, the size of programs at the districtwide/school program level ranged across all categories. Of the 167 districtwide/school programs that reported student enrollment, there were 39 small, 78 medium, and 59 large programs in this study sample. Districtwide/school programs may draw upon a larger area than a single school, often serving students from throughout the district. The size of the program will therefore vary depending on the size of the district or attendance area and the density of students with hearing loss in the local population.
Language and communication are essential components of services for students who are deaf or hard of hearing. The survey collected information about the primary communication mode used in instruction as well as information about itinerant and sign language interpreter services. Participants indicated the most prevalent mode of communication used in instruction; this does not account for the likely variability in communication used with specific students in each school or program. Data for the most common forms reported are shown in Table 3: oral (speech) only, total communication (oral and sign together by instructor), oral (speech) plus interpreter, and other (including sign language only). Only a few participants indicated that they used only sign language. Total communication was the most frequently cited communication mode across all respondents (37%). For example, a within-setting analysis showed that schools for the deaf were more likely to use total communication than speech, with or without an interpreter, and other communication modes, 2(6, N = 29) = 49.3, p < .01.
|
Participants in districtwide/school programs or mainstreamed settings provided information about the presence of sign language interpreters and itinerant teacher services as part of their program (data for these variables were not collected from schools for the deaf). These are program-level data; the presence of an interpreter neither demonstrates how many students may have access to the interpreter nor does it assess the quality of these services. Most of the districtwide/school programs in this sample had both interpreter services (81% of programs) and itinerant services (73% of programs) as part of their service to students who are deaf or hard of hearing. A lower proportion of mainstreamed settings (64%) had interpreter services than did districtwide/school programs,
2(1, N = 226) = 7.67, p < .01. However, mainstreamed settings were more likely to have itinerant services (88% of settings), 2(1, N = 220) = 4.81, p < .05. An analysis of services by program size indicates that students in small programs (districtwide/school and mainstreamed combined) are less likely to receive sign language interpreter services (67%) than those in medium (85%) or large programs (79%), 2(2, N = 228) = 5.92, p = .05. There were no differences by program size for itinerant services, 2(2, N = 222) = 3.77, p = .15.
Student demographics.
The National Survey also collected information about the students served by each participating school or program. The first item was the total number of students at varying levels of hearing loss (unaided) as well as the use of cochlear implants. A summary of available information about hearing loss characteristics is shown in Table 4. Unfortunately, only 26% (n = 67) of the school-level participants provided information about specific levels of hearing loss for their student populations (a total of 6,009 or 50% of students in the sample). These figures therefore may not be representative of the survey sample as a whole.
|
The National Survey also asked for information about additional disabilities. An estimated 45% of the national deaf or hard-of-hearing student population has one or more additional disabilities (Mitchell, 2004
). In the National Survey, an average of 36% of students in each school or program had an additional disability. However, this percentage varied by educational setting. A higher percentage of students in schools for the deaf (48%) had an additional disability than in districtwide/school programs (33%), though not in mainstreamed settings (41%), F(2, 205) = 3.03, p = .05. Each respondent listed additional disabilities of students in their programs. Overall, the most prevalent additional disability across all school settings was learning disabilities (number of programs n = 161), followed by cognitive disabilities (n = 141), and then visual deficits (n = 107). Of those who indicated that one or more student had a visual deficit, 42% served at least one student who was deaf–blind. The demographics portion of the National Survey ended with a question about the number of students who had an IEP. Because the survey was about the use of accommodations in standardized assessments, as well as participation in alternate assessments, it was presumed that the majority of students at participant schools would have an IEP. This assumption was confirmed by the response to this question. The average percentage of students who were deaf or hard of hearing with an IEP was 99% at schools for the deaf, 94% at districtwide/school programs, and 95% in mainstreamed settings. These percentages were not statistically different from each other, F(2, 221) = 0.37, p = .69.
Accommodations.
The second section of the survey focused on the use of accommodations in reading and mathematics. Participants were first asked to identify the number of students, by grade range, who participated in state standardized assessments in the 2003–2004 school year. Participants were then asked to identify what accommodations students at their school or program used for mathematics and reading. These items were first derived from the research literature on frequently used accommodations on standardized assessments (Thurlow & Bolt, 2001). They were then piloted with teachers and administrators who work with students with disabilities, including those who are deaf or hard of hearing and have additional disabilities.2 Participants also had the opportunity to provide other examples of accommodations used with their students. A sample accommodations grid is shown in Figure 1.
|
A total of 177 school representatives reported accommodations information for 3,209 students who participated in state standardized assessments. An additional three participants indicated that they did not have students who participated in standardized assessments. A total of 1,190 students were in 1st–5th grades, 993 in 6th–8th grades, 888 in 9th–11th grades, and 138 in 12th grade. The 12th-grade students represented those who took an exit exam required for high school graduation. These data represent only those students who were required to participate in standardized assessments for state accountability, not all students at all grades. Although NCLB legislation requires states to move towards annual assessments, few states had made this requirement a part of their accountability plans by 2003–2004. For most states, only one grade per grade range (1st–5th, 6th–8th, 9th–11th, and 12th) was assessed with state tests. For example, Wisconsin used assessments in three grades: 4th, 8th, and 10th grade. Therefore, National Survey participants from Wisconsin would only report student assessment practices for these three grades.
Figure 2 shows results for overall accommodations use for math and reading. There were similar patterns in the use of accommodations for both subjects across all participants. Prevalence rates were calculated by dividing the number of participants who selected the accommodation at least once across all grade ranges by those who had students use accommodations in standardized assessments. It should be noted that these are program-level, and not student-level, data; student-level analyses may demonstrate different frequencies of accommodations use with individual students. The most prevalent accommodation was extended time (76% and 80% for reading and mathematics, respectively), followed by the use of an interpreter for directions, and a separate room for test administration. It is logical that these three cluster together: having test directions interpreted may both take additional time and be distracting to those who are not receiving this accommodation.
|
There were, however, a few notable differences between mathematics and reading. The first is with the use of signed question-response as part of test administration. The signed question-response accommodation is when the test items are interpreted into sign language by the test administrator or an interpreter. The student then gives a response in sign language. Overall, 57% of participants indicated that signed question-response was used for mathematics, 41% used it in reading. This difference between academic subjects may be due to the nature of the test items or because of concerns about validity of a signed reading assessment.
Educational setting.
There were enough respondents for seven accommodations to do further analysis of differences between educational settings. It was necessary to first convert the accommodations frequency items into a measure that was weighted for the grades that the school or district served. Without this weighting, the frequencies would represent only the total number of grade ranges indicated by the respondent, resulting in figures skewed towards those settings that served 1st–12th grades over those serving a single grade range (e.g., 6th–8th). Analyses were conducted on a ratio variable: the number of grade ranges in which the accommodation was used (1st–5th, 6th–8th, 9th–11th, and 12th) divided by the number of grade ranges served by the school (same ranges). Ratios were on a scale of 0–1, with 0 = no accommodation use and 1 = accommodation used across all grade ranges served. For example, a school or program using extended time in the early elementary grades (1st–5th) but serving all four grade ranges would have a ratio of 0.25. In contrast, a school or program using extended time in 1st–5th grades that only served students in these grades would have a ratio of 1.0. Because many states assessed only one grade per range, ratios were a reasonable estimate of overall use of accommodations at each participating school or program.
A Kruskal–Wallis F test was conducted on accommodations that were used by at least 50 participants to see if there were any differences in their use in schools for the deaf, districtwide/school programs, and mainstreamed settings. Table 5 shows that some of the accommodations in reading and mathematics were used at significantly different rates in the three educational settings. For all accommodations with differences in ratios by educational setting, mainstreamed settings were more likely to use accommodations than districtwide/school programs or schools for the deaf.
|
Analysis included a look at potential relationships between accommodations use and percentage of students with additional disabilities, levels of hearing loss in the student population, and program size. There were no significant correlations between accommodations use and the percentage of students with additional disabilities at each school or program (results not shown). Correlations between use of accommodations and percentage of students by hearing loss are shown in Appendixes A and B. Schools or programs with a higher proportion of students with "mild" hearing loss are less likely to use accommodations such as extended time (reading r = –.50, mathematics r = –.50), separate room (reading r = –.35, mathematics r = –.39), interpreter for directions (reading r = –.65, mathematics r = –.59), or signed questions and response (reading r = –.48, mathematics r = –.60; all Pearson correlations significant at .05). Schools or programs with a higher proportion of students with severe-to-profound hearing loss are more likely to use these accommodations. Due to the low response rate on information regarding levels of student hearing loss, these figures should be viewed with caution. Furthermore, it should be noted that these relationships are more complex than what can be demonstrated with a single variable. As shown in the demographic analysis, schools for the deaf are more likely to have students with severe-to-profound hearing loss. However, when controlled for educational setting, there were no significant correlations of accommodations use with levels of hearing loss (results not shown).
Analysis of the relationship between program size and prevalence of accommodations yielded several significant results. Small schools or programs, those with seven or fewer students who are deaf or hard of hearing, were more likely than medium or large programs to use separate rooms (mathematics M = 0.79), individual administration (reading M = 0.74, mathematics M = 0.78), interpreter for directions (reading M = 0.82; mathematics M = 0.79), and read aloud accommodations (mathematics M = 0.89) (all p < .05 Kruskall–Wallis test). With only a few exceptions, mainstreamed settings had a smaller enrollment of students who were deaf or hard of hearing than districtwide/school programs or schools for the deaf. Yet, as with levels of hearing loss, significant differences in accommodations use by program size disappear when controlled for by educational setting (results not shown).
Alternate assessments.
The third section of the survey addressed the use of alternate assessments for state accountability testing. The first question asked participants the name of the statewide alternate assessment students used in the 2003–2004 testing year. Participants then completed a grid similar in format to the one used for the accommodations questions. A sample alternate assessment grid is shown in Figure 3. Unlike the accommodations grid, participants were not asked for subject-specific alternate assessments. Alternate assessment formats and strategies were based on the NCEO (Thompson & Thurlow, 2003) list of common formats used in state testing. Alternate assessment formats were also piloted with professionals who work in special education.
|
Fewer students who are deaf or hard of hearing participated in alternate assessments than in standardized assessments with accommodations. A total of 71 schools or programs had students who participated in alternate assessments; these participants indicated that nearly 900 students who were deaf or hard of hearing participated in alternate assessments. The majority of survey responses (n = 50 programs, 70%) came from districtwide/school programs, a smaller proportion from schools for the deaf (n = 16, 22%), and only a few (n = 5, 7%) from mainstreamed settings (totals do not add to 100% due to rounding). However, there was a significant difference in the percentage of participants who reported alternate assessment use within each setting. Of the programs represented by survey respondents, schools for the deaf were most likely to have students participate in alternate assessments (76%), followed by districtwide/school programs (40%), and then by mainstreamed settings (12%),
2(2, N = 71) = 25.4, p < .01. Even though these alternate assessment data mainly represent responses from districtwide/school programs, schools for the deaf, as a whole, are the most likely to have students participate in alternate assessments. The overall pattern of alternate assessment formats is shown in Figure 4. Participants were able to choose more than one format in their response. These data are at the program or school level, not for individual students. Unlike the accommodations use data, no one single assessment format was used most often across all settings. The top three formats, out-of-level, work samples, and portfolios, each represented no more than 50% of the respondents who used alternate assessments with deaf or hard-of-hearing students. In the midrange, curriculum-based assessments were selected by 37% of participants. Less frequently used formats include structured and unstructured observations (24% and 17%, respectively) and checklists (24%).
|
As with the accommodations analysis, a use ratio was computed for each alternate assessment format. This ratio is the number of grade ranges (1st–5th, 6th–8th, 9th–11th, and 12th) for which the participant selected the alternate assessment format divided by the number of grade ranges served by that setting. The ratio score extends from 0 to 1, with a lower score indicating a lower rate of using each specific alternate assessment format for students who are deaf or hard of hearing. There was little difference in the ratio scores for each of these alternate assessment formats. The average ratio scores across study participants ranged from M = 0.60 to 0.69, a tight range. In other words, school and programs that used each format did so for just over half their assessed grade ranges. Because of the small number of settings that used alternate assessments, it was not possible to compare results between schools for the deaf, districtwide/school programs, and mainstreamed settings.
The final analyses in this study looked at the relationship between the use of alternate assessments and student characteristics at the school or program. We first looked at differences in the levels of hearing loss in schools with students who participated in an alternate assessment. These analyses are limited to the 60 participants who provided information both on student characteristics and alternate assessment. Schools that did have students participate in the alternate assessment (n = 32) served groups with an average of M = 58% (SD = 31%) severe or profound hearing loss. In contrast, schools that confirmed their students did not participate in an alternate assessment (n = 28) served an average of M = 40% (SD = 27%) of students with a severe or profound hearing loss. There is no significant difference, however, in the percentage of students who have additional disabilities for those who have students participating in alternate assessments (M = 0.41, n = 69) and those who do not (M = 0.33, n = 106; t(173) = .187, p = .06