Journal of Deaf Studies and Deaf Education Advance Access originally published online on April 20, 2005
The Journal of Deaf Studies and Deaf Education 2005 10(3):232-243; doi:10.1093/deafed/eni025
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Empirical Articles |
Evaluating Deaf Students' Readiness to Meet the English Language and Literacy Demands of Postsecondary Educational Programs
National Technical Institute for the Deaf
Correspondence should be sent to Joseph H. Bochner, National Technical Institute for the Deaf, Rochester Institute of Technology, 52 Lomb Memorial Drive, Rochester, NY 14623–5604 (e-mail: jhbncp{at}rit.edu).
Received July 27, 2004; revised November 11, 2004; accepted November 15, 2004
The purpose of this study was to investigate alternative methods for evaluating deaf students' readiness to meet the English language and literacy demands of postsecondary educational programs. In the first part of the study, scores obtained by a large sample of deaf students on the ACT Assessment (ACT Composite score and scores on the ACT English and Reading tests) were compared to their scores on various measures of English language and literacy skills. In the second part of the study, the performance of a smaller sample of deaf students on the ESL Reading and ESL Grammar/Usage components of COMPASS/ESL was compared to their performance on a set of concurrent measures of English skills. The results of this investigation demonstrate that neither the ACT Assessment nor COMPASS/ESL are appropriate for the full range of deaf students seeking admission to postsecondary educational programs. However, the ACT Assessment is appropriate for deaf students seeking admission to transferable (BS and AAS) degree programs, and the ESL Reading and Grammar/Usage tests appear to be appropriate for deaf students seeking admission to nontransferable (AOS) degree programs. Taken together, the combination of the ACT Assessment and COMPASS/ESL appear able to provide a valid, reliable, and coherent approach to admissions screening assessment for the full range of deaf students seeking admission to postsecondary programs.
 |
Introduction |
|---|
 TOP Introduction Statement of the ProblemMethodResultsDiscussionReferences |
|---|
Language acquisition is strongly influenced by the integrity of the linguistic environment to which the learner is exposed during infancy and early childhood. In fact, the influence of linguistic intake in combination with the learner's age is so powerful that the interaction between these variables largely determines the course and extent of language acquisition (Bochner & Albertini, 1988
). The effects of this interaction are readily apparent in prelingually deaf children. Numerous studies of deaf students' attainments in the acquisition of English language and literacy skills have been conducted over the past century consistently showing that most prelingually deaf children enter adolescence and young adulthood without having achieved proficiency in English. The results of many of these studies are summarized in papers and monographs describing English language and literacy acquisition and variation in the deaf population (Albertini & Schley, 2003; Berent, 1996; Bochner & Albertini, 1988; Paul, 1998).
The acquisition of English language and literacy skills probably represents the most formidable challenge confronting educators of deaf students in America (Karchmer & Mitchell, 2003; Scouten, 1984), having implications for both assessment and instruction. In order to address the instructional needs of students having limited English proficiency, appropriate assessment tools are necessary. For example, postsecondary educational programs serving deaf students need valid and reliable assessments in order to make admissions and placement decisions. However, conventional admissions and placement tests are not appropriate for applicants having limited English proficiency, including the majority of deaf applicants. Accordingly, assessments designed to measure the English skills of foreign students, such as the Test of English as a Foreign Language (TOEFL) and the Michigan Test of English Language Proficiency, can be considered as alternatives to conventional admissions and placement tests for deaf students. The TOEFL and Michigan Test are designed to measure general proficiency in the English language and assist in determining the degree to which international students can pursue academic study in a college or university where English is the language of instruction (Educational Testing Service, 1997; English Language Institute, 1977). Scores on these tests are typically combined with other information about applicants (e.g., performance in previous educational programs and scores on other tests) to make admissions and placement decisions.
The use and interpretation of scores on English language proficiency tests are based on the premise that without adequate command of English, students cannot meet the demands of a regular or even a modified program of study in an American college or university. Adequate English language and literacy skills are commonly considered an essential component of academic readiness for international or other postsecondary students having limited English proficiency.
Assessments such as TOEFL and the Michigan Test are routinely administered to foreign students seeking admission to American colleges and universities, and the results of these tests have proven useful in planning students' academic programs (i.e., planning the type and amount of credit-bearing course work for individual students). The TOEFL and Michigan Test are two of the most widely used and highly regarded measures of English language proficiency in the world. A description of each test, details pertaining to its validity and reliability, and information concerning the use and interpretation of scores are provided in the TOEFL Test and Score Manual (Educational Testing Service, 1997) and the Michigan Test of English Language Proficiency Manual (English Language Institute, 1977).
Although the TOEFL and Michigan Test measure a common construct having practical significance for the education of deaf students (i.e., English language proficiency), neither of these tests was designed specifically for use with deaf individuals, and the tests are not equally appropriate for deaf students. Specifically, one of the three sections of TOEFL is a listening comprehension test, and this section is not appropriate for deaf individuals for obvious reasons. The Michigan Test, on the other hand, does not include a listening comprehension section and is fully accessible, making it preferable to TOEFL for use with deaf students.
The construct of English language proficiency is highly pertinent in the education of deaf students largely because adequate English language and literacy skills are a necessary (but not sufficient) prerequisite for meeting the demands of postsecondary educational programs in American colleges and universities. Also, there are important similarities between deaf students and adult learners of English as a second language (ESL) with respect to the manifestation of distinct grammatical patterns in usage (Langston & Maxwell, 1988; Swisher, 1989; also see Greenberg & Withers, 1965), and it has frequently been pointed out that teaching methods, materials and approaches used with ESL students have proven useful with deaf individuals enrolled in secondary and postsecondary educational programs.
The first adaptation of ESL methodology explicitly designed for deaf students was implemented by Goldberg and Boardman (1974) at Gallaudet University in the early 1970s. Oral-aural (audio-lingual) methods involving drill and practice (mimicry and memorization) exercises commonly used in ESL instruction were adapted to teach grammatical structures to deaf students through print, especially constructions which deaf students find particularly troublesome (e.g., inflectional morphemes such as –ed and –ing). As such, adaptations of ESL methodology closely resemble traditional structural approaches that had been used for teaching English to deaf students over the course of many decades (e.g., Wing, 1887; Fitzgerald, 1926).
In the area of assessment, the Michigan Test of English Language Proficiency was first administered to deaf students in a series of studies conducted by Bochner (1976, 1977) at the National Technical Institute for the Deaf (NTID) at Rochester Institute of Technology (RIT) in the mid-1970s. Developed by the English Language Institute at the University of Michigan, the Michigan Test is a 100-item assessment of English grammar (40 items), vocabulary (40 items), and reading comprehension (20 items). The test is used throughout the world to assess the readiness of students with limited English proficiency to pursue academic study in universities where English is the language of instruction. The Michigan Test has been routinely administered to students entering NTID since the late 1980s, and it has been especially useful for assessing the English language proficiency of deaf students seeking to enroll in baccalaureate programs.
A new generation of tests based on advances in psychometrics and computer technology has emerged in recent years. One example of this new generation of tests is a measure of academic skills known as COMPASS/ESL (American College Testing, 2000a). COMPASS/ESL is a computerized adaptive testing system designed to assess English and mathematics skills for course placement, advising, retention and outcomes-oriented purposes at the secondary and postsecondary levels. Test items are presented in a multiple-choice format, and the examinee selects responses by clicking the computer's mouse. In adaptive testing, the difficulty of items presented to each examinee varies depending on their responses to prior items. Respondents who perform well on a given task are subsequently presented with more challenging tasks, while those who perform poorly are presented with less challenging tasks.
As stated in the manual, COMPASS/ESL has been designed to increase the likelihood that students entering postsecondary programs will achieve educational success and retention (American College Testing, 2000a). The ESL portions of COMPASS/ESL have been developed specifically for secondary and postsecondary students of English as a second language. As the use of the Michigan Test has shown, ESL tests can provide a practical alternative for assessing the English skills of deaf students. In this regard, the ESL Reading and Grammar/Usage portions of COMPASS/ESL may provide an alternative means of assessing deaf students' readiness to meet the English language and literacy demands of postsecondary educational programs.
|
Statement of the Problem |
|---|
|
TOPIntroductionStatement of the ProblemMethodResultsDiscussionReferences |
|---|
Since 1997, the ACT Assessment, a widely used college entrance examination, has been used to evaluate the academic readiness of applicants for admission to NTID. The ACT Assessment is a curriculum-based test measuring students' educational development in English, mathematics, reading, and science reasoning. According to the ACT Assessment User Handbook (American College Testing 2003
), students' performance on the test has a direct and obvious relationship to their academic development. Colleges and universities commonly use this test, along with other information, for making admissions decisions about applicants, and the test is known to be an effective measure for deaf students seeking admission to baccalaureate programs. Although ACT scores provide useful information concerning deaf students' readiness for admission to academic programs, they tend to be highly problematical for evaluating candidates' readiness for technical programs. In this paper, academic programs refer specifically to baccalaureate of science (BS) and associate of applied science (AAS) degree programs, whereas technical programs refer specifically to associate of occupational studies (AOS) degree programs. Both BS and AAS degrees include courses that are transferable from one college to another. In contrast, AOS degrees do not include transferable courses because these degrees are considered career-focused credentials. The use of ACT scores to evaluate applicants' readiness for technical (nontransferable associate degree) programs is problematical because ACT Composite scores below 16 are not sufficiently sensitive to differences in the abilities of individuals with limited English proficiency. Specifically, ACT Composite scores below 16 do not provide useful and accurate information about applicants placed in remedial/developmental English courses, and most students entering technical programs at NTID achieve ACT Composite scores falling in the range from 12 to 16. Since the English language proficiency of these applicants is of particular concern, the ESL Reading and Grammar/Usage portions of COMPASS/ESL were administered to samples of students entering NTID in the fall of 2001 and 2002, primarily individuals with ACT Composite scores of 16 or below.
The purpose of this study was to discover a valid, reliable, and coherent approach to assessing deaf students' readiness to meet the English language and literacy demands of postsecondary degree programs. The study is comprised of two parts. The first part describes the advantages and disadvantages of using a conventional test of general academic skills, the ACT Assessment, for evaluating the college readiness of deaf students. In particular, scores on the English and Reading components of the ACT Assessment were investigated for their efficacy in determining students' placement into a series of postsecondary English courses offering instruction ranging from developmental/remedial levels (i.e., levels of the NTID English curriculum) through a sequence of English courses required for graduation with a baccalaureate degree. The second part of the study describes the efficacy of the ESL Reading and Grammar/Usage portions of COMPASS/ESL as potential admissions screening tools for deaf students with limited English proficiency seeking entry to technical (i.e., nontransferable AOS degree) programs. As in the first part of this study, the efficacy of these tests in determining students' placement into levels of the NTID English curriculum was investigated.
Unfortunately, the ACT Assessment is of limited value for making distinctions among deaf students seeking admission to technical (AOS) degree programs, primarily because these individuals tend to have limited English proficiency. The ACT Assessment has been designed for native speakers of English and is based on major areas of instruction in American high schools and colleges. In particular, the ACT English Test measures a student's understanding of the conventions of standard written English (mechanics and grammar) and of rhetorical skills (strategy, organization, and style). The ACT Reading Test measures a student's reading comprehension, with emphasis on referring and reasoning skills (American College Testing, 2003). These components of the ACT Assessment are especially problematical for students having limited English proficiency.
While the ACT Assessment has been developed for native speakers of English seeking admission to postsecondary educational programs in America, the ESL portions of COMPASS/ESL have been developed specifically for students of English as a second language. As described in the COMPASS/ESL Reference Manual (American College Testing, 2000a), the ESL Grammar/Usage Test assesses students' knowledge of aspects of English grammar and usage including verbs, subjects and objects, agreement, modifiers, function words, word order, relationships between and among clauses, discourse and mechanical conventions (spelling and punctuation). The ESL Reading Test assesses the degree to which examinees can read at a literal level, recognizing main ideas and significant details (referring), as well as at an inferential level (reasoning). The passages included in the ESL Reading Test vary along two main continua: (a) context, including vocabulary, and (b) grammatical structure. Materials at the lower end of these continua are limited to areas of nearly universal knowledge, basic vocabulary, and relatively simple syntax without idiomatic or metaphorical usage. Reading passages at the higher levels of these continua, however, incorporate academic and unfamiliar contexts and include complex syntax and idiomatic and metaphorical language.
The problem, then, is to find a means of identifying applicants possessing sufficient English language and literacy abilities to be successful in transferable (BS and AAS) and nontransferable (AOS) degree programs, differentiating them from one another and from applicants not possessing sufficient English skills to be successful in either category of postsecondary program. The solution to this problem entails developing a valid, reliable, and coherent approach for estimating deaf students' ability to meet the English language and literacy demands of postsecondary programs extending from the baccalaureate level to the associate degree level and below.
|
Method |
|---|
|
TOPIntroductionStatement of the ProblemMethodResultsDiscussionReferences |
|---|
This investigation was conducted in two parts. In the first part of the investigation, ACT Assessment data were collected from students who had completed the test and been admitted to NTID over the period from 2000 to 2003. The tests were administered at approved sites using standard administration procedures prescribed by the American College Testing Corporation (ACT). Table 1 provides a summary of pertinent demographic characteristics of the subject sample used in the first part of this study, specifically the mean and standard deviation of their age, age at onset of hearing loss, and hearing loss for pure tones at 0.5, 1, and 2 kHz. The sample was comprised of 520 males and 385 females, with 29% of the sample being members of a minority group. In the second part of the investigation, the ESL Reading and Grammar/Usage portions of COMPASS/ESL were administered to a sample of students who had enrolled in programs of study at NTID in 2001 and 2002. The data were collected at NTID in the fall of 2001 and the summer of 2002. Table 2 provides a summary of pertinent demographic characteristics of the subject sample used in the second part of the study, specifically the mean and standard deviation of their age, age at onset of hearing loss, and hearing loss for pure tones at 0.5, 1, and 2 kHz. The sample was comprised of 82 males and 71 females, with 33% being members of a minority group.
|
|
The subjects participating in each part of the study spanned a wide range of English abilities. Scores on the ACT Assessment (Composite score and English and Reading component scores) and the ESL Reading and Grammar/Usage tests were compared to one another and to scores on other measures of English language and literacy skills collected upon entry to NTID, namely the California Reading Test, NTID Writing Test, and Michigan Test of English Language Proficiency. Scores on these measures are used for placement into levels of the NTID English curriculum (i.e., a developmental/remedial English program) and for determining the readiness of students to enroll in conventional college-level English courses. In each part of this investigation, data were not available from all students on every measure. Consequently, the sample size is not uniform across all variables because of missing data.
The efficacy of the ACT Assessment and the ESL Reading and Grammar/Usage tests was investigated for students in various programs of study, and the validity of these tests was explicitly addressed. Concurrent validity was estimated by comparing scores on these tests to scores obtained on measures administered upon entry to NTID and to students' placement into levels of the NTID reading and writing curricula. In addition, the predictive validity of the ACT Assessment was investigated by comparing examinees' ACT Composite scores to their acceptance into programs of study at NTID. Unfortunately, sufficient data were not available to permit an investigation of the predictive validity of the ESL Reading and Grammar/Usage tests, in part because the subject sample was not large enough. The intent of the analyses was to discern the degree to which the tests could effectively identify applicants possessing sufficient English language and literacy abilities to be successful in transferable (BS and AAS) and nontransferable (AOS) degree programs and to differentiate them from one another and from applicants not possessing sufficient English skills to be successful in either category of postsecondary program.
|
Results |
|---|
|
TOPIntroductionStatement of the ProblemMethodResultsDiscussionReferences |
|---|
Part I: ACT Assessment
Table 3 summarizes the performance of students entering NTID from 2000–2003 on the ACT Assessment (Composite and component scores), as well as their performance on three independent measures of English language and literacy ability (California Reading Test, NTID Writing Test, and Michigan Test). The range of standard scores on the ACT Assessment and its component tests extends from 1 to 36; grade equivalent scores on the California Reading Test range from 5.0 to 13.0; the range of possible scores on the NTID Writing Test extends from of 0 to 100; and the range of standard (equated) scores on the Michigan Test extends from approximately 35 to 100. Correlations among these measures are presented in Table 4.
|
|
Table 5 provides a breakdown of ACT Composite and component scores by the program of study in which students are enrolled. Students admitted to the college but not accepted into a degree program have been classified into a category called "Preparatory." The Preparatory category represents students who are not well prepared for college and, consequently, are placed in Career Exploration, Diploma, or Affiliated programs at NTID. The AOS category represents students enrolled in associate of occupational studies degree programs; the AAS category represents students enrolled in associate of applied science programs; and the Baccalaureate category represents students preparing to enroll in baccalaureate degree programs, as well as students already enrolled in baccalaureate degree programs. For purposes of comparison, average ACT Assessment scores obtained nationally by students seeking admission to two- and four-year colleges and universities are also displayed in Table 5.
|
A one-way ANOVA was conducted on the ACT Composite scores obtained by NTID students with their program classification as the factor. The intent of this analysis was to provide an indication of the predictive validity of the ACT Assessment for use with deaf students enrolled in postsecondary educational programs. The results of the analysis indicated that the difference in ACT Composite scores among the student groups was significant (F = 184.29, df = 3,875, p < .0001). The results of Scheffe' tests revealed significant differences among students enrolled in Preparatory programs, AOS degree programs, AAS degree programs, and Baccalaureate and related programs. That is, all possible combinations of contrasts were significantly different from one another.
Table 6 provides a breakdown of scores on the English component of the ACT Assessment by student placement into levels of the NTID writing curriculum, and Table 7 presents a breakdown of scores on the Reading component of the ACT Assessment by student placement into levels of the NTID reading curriculum. Levels A–D represent a sequence of courses leading to enrollment in the RIT College of Liberal Arts (CLA). Scores on the NTID Writing Test are used for placing students into levels of the writing curriculum, and scores on the California Reading Test are used to simulate placement into levels of the reading curriculum. A linear combination of scores on the Reading and Vocabulary sections of the California Achievement Tests is actually used for placement into levels of the NTID reading curriculum. Use of California Reading scores alone provides a good approximation of actual course placement, with the added advantage of providing information concerning examinees' academic achievement expressed in terms of grade-equivalent scores.
|
|
A one-way ANOVA was conducted on ACT English test scores and on ACT Reading test scores, with students' placement level in the curriculum as the factor. The intent of these and subsequent analyses of the ACT English and Reading tests was to provide an indication of the validity of these tests as measures of the English skills of deaf students enrolled in postsecondary programs. The results of each analysis indicated significant differences among groups of students for the ACT English test (F = 88.67, df = 4,624, p < .0001) and for the ACT Reading test (F = 79.29, df = 4,777, p < .0001). The results of Scheffe' tests indicated that the ACT English test can distinguish among all of the placement levels (groupings) of students except levels A and B of the NTID writing curriculum. Similarly, the results of Scheffe' tests indicated that ACT Reading test scores can distinguish among all of the groupings of students except for levels A and B and levels A and C of the reading curriculum.
Table 8 displays means and standard deviations on the English and Reading components of the ACT Assessment for students meeting the criteria for entry into the writing curriculum offered by the RIT College of Liberal Arts and for students not meeting these standards. The criteria involve obtaining scores exceeding at least two of the three thresholds specified below (the NTID Reading Test represents the combination of scores on the California Reading and vocabulary tests):
- NTID Writing Test >65
- NTID Reading Test>130
- Michigan Test >75
- NTID Reading Test>130
|
The results of one-way ANOVAs indicated that the difference in ACT English scores between these groups of students was significant (F = 167.4, df = 1,876, p < .0001) and the difference in ACT Reading scores between these groups of students was also significant (F = 145.3, df = 1,799, p < .0001).
Table 9 shows a breakdown of means and standard deviations on the ACT English and Reading tests according to placement into each of the three levels of the writing curriculum offered by the RIT College of Liberal Arts—a sequence of three courses extending from Written Communication I to Written Communication II to Writing and Literature I. These data apply at the time of enrollment to students entering NTID during the summers of 2000, 2001, and 2002.
|
Part II: ESL Reading and Grammar/Usage Tests
Descriptive statistics for the ESL Reading test, ESL Grammar/Usage test, and a set of concurrent measures are displayed in Table 10. The range of standard scores on the ESL Reading and Grammar/Usage tests extends from 25–99 (American College Testing, 2000b
). Score ranges for the other tests have been indicated previously. Correlations among the ESL Reading, ESL Grammar/Usage, and the concurrent measures are shown in Table 11. The concurrent measures include the California Reading, NTID Writing, and Michigan tests, as well as ACT English, Reading, and Composite scores. In addition to the correlation statistics presented in Table 11, a multiple correlation/regression analysis was conducted. In this analysis, scores on the ESL Reading and Grammar/Usage tests served as independent variables with scores on the Michigan Test serving as the dependent variable. The multiple correlation/regression analysis resulted in a multiple R of 0.79 (p < .0001), indicating that scores on the ESL Reading and Grammar/Usage tests together accounted for 62% of the variance in Michigan Test scores.
|
|
Table 12 provides a breakdown of performance on the ESL Reading test for four groupings of examinees roughly corresponding to placement at levels A, B, C, and D of the NTID reading curriculum (the last grouping combines students placed at the D-level of the curriculum with students having more advanced reading skills), and Table 13 provides a breakdown of performance on the ESL Grammar/Usage test for four groupings of subjects roughly corresponding to placement at levels A, B, C, and D of the NTID writing curriculum (the last grouping combines students placed at the D-level of the curriculum with students having more advanced writing skills).
|
|
The results of a one-way ANOVA conducted on the ESL Reading scores (with placement in the NTID reading curriculum as the factor) were significant (F = 14.26, df = 3,149, p < .0001). The results of Scheffe' tests indicated that ESL Reading scores differentiated students placed at Level A of the reading curriculum from students in every other group. The ESL Reading test also distinguished students placed at the B-level of the reading curriculum from those placed at the D-level and above (i.e., the D+ group). Only two contrasts were not significant, the contrast between the B-level and the C-level groups and the contrast between the C-level and D+ groups. ESL Reading scores correlated with actual placement into the four levels of the NTID reading curriculum at r = 0.55 (p < .0001).
The results of a one-way ANOVA conducted on the ESL Grammar/Usage scores (with placement in the writing curriculum as the factor) also were significant (F = 18.17, df = 3,121, p < .0001). Scheffe' tests indicated that all contrasts except one were significant, the exception being the contrast between students categorized in the C-level group and those placed at the D-level and above (i.e., the D+ group). ESL Grammar/Usage scores correlated with actual placement into the four levels of the NTID writing curriculum at r = 0.49 (p < .0001).
|
Discussion |
|---|
|
TOPIntroductionStatement of the ProblemMethodResultsDiscussionReferences |
|---|
The results of this investigation demonstrate that neither the ACT Assessment nor COMPASS/ESL are appropriate for the full range of deaf students seeking admission to postsecondary educational programs. However, the ACT Assessment is appropriate for deaf students seeking admission to transferable (BS and AAS) degree programs, and the ESL Reading and ESL Grammar/Usage tests are appropriate for deaf students seeking admission to nontransferable (AOS) degree programs. Data indicate that the ACT Assessment (especially the ACT English and Reading tests) can effectively identify deaf students possessing sufficient English language and literacy abilities to be successful in baccalaureate and associate of applied science programs and that the ESL Reading and Grammar/Usage tests have the potential to identify deaf students possessing sufficient English language and literacy abilities to be successful in associate of occupational studies degree programs. In general, the ACT Assessment appears to be a good indicator of college readiness for deaf students who are relatively proficient in English. For deaf students having limited English proficiency, however, the ESL Reading and Grammar/Usage tests appear to be good measures of their ability to meet the English language and literacy demands of nontransferable (AOS) degree programs. These findings suggest that the ESL Reading and Grammar/Usage tests may be used to enhance the quality of admissions decisions for applicants to nontransferable (AOS) degree programs, in particular applicants attaining ACT Composite scores below 16. As shown in Table 5, the mean ACT Composite score for students enrolled in AOS degree programs is 15.5. This score is very low—only about three points above "chance" performance. Accordingly, it appears that the results of the ESL Reading and Grammar/Usage tests would lead to better admissions decisions for applicants obtaining ACT Composite scores of 15 and below.
Data presented in Part I of the investigation provide insight into the validity of ACT scores obtained by deaf students at the postsecondary level. Mean ACT English, Reading, and Composite scores are low (see Table 3), with standard deviations indicating a significant number of examinees obtaining scores near chance. Correlation data presented in Table 4 indicate a pattern of moderate correlations involving ACT English and Reading scores, and data presented in Table 5 demonstrate that ACT Composite scores are related to the type of program in which deaf postsecondary students enroll. This pattern of results, combined with data presented in Tables 6–9
, casts doubt on the validity of ACT English, Reading, and Composite scores obtained by a subset of the population of deaf students seeking admission to postsecondary educational programs as described below.
Data presented in Tables 6–9 help define the subset of deaf students for whom the ACT Assessment appears inappropriate and invalid. These data point to a dichotomous situation in which the test appears to be valid for students having relatively good English language and literacy skills, but not for students having relatively weak English skills. Data indicating that the ACT English and Reading tests are able to effectively determine students' placement into the middle and upper levels of the NTID English curriculum support the validity of these tests and suggest that the ACT Assessment is appropriate for use with deaf students having relatively good English language and literacy skills. Conversely, data indicating that ACT English and Reading scores are not able to effectively discriminate among students placed at the lower levels of the NTID English curriculum indicate that the ACT Assessment is not appropriate for use with deaf students having relatively weak English language and literacy skills.
In examining the manner in which the ACT English and Reading tests are associated with the placement of students into levels of the NTID writing and reading curricula (Tables 6 and 7), it is clear that these tests are not sensitive measures for students exhibiting weaker English skills. ACT English, Reading, and Composite scores tend to cluster near chance performance for these students, and the tests generally are unable to distinguish among examinees (i.e., the tests are not able to differentiate among students placed at the lower ends of the writing and reading curricula). These tests, however, exhibit greater sensitivity for students having relatively good proficiency in English. In particular, they are able to provide useful information concerning students placed at the middle and upper ranges of the writing and reading curricula, especially those qualifying for admission to the RIT College of Liberal Arts writing course sequence (Tables 8 and 9).
Data presented in Tables 10 and 11 indicate that the ESL Reading and Grammar/Usage tests are more sensitive to differences among individuals having limited English proficiency. The mean score obtained by subjects on each of these tests is located in the upper-middle range of the test score distribution for these measures (range of possible scores extends from 25–99). In contrast, subjects' mean ACT English, Reading, and Composite scores are very low, with standard deviations indicating many examinees obtaining scores near (and below) the chance level of performance. Moderate correlations involving the ESL Reading and Grammar/Usage tests and concurrent measures of English skills (i.e., Michigan Test, NTID Writing Test, and California Reading Test) also were found, supporting their validity for use with a subset of examinees (i.e., students having relatively weak English skills). As suggested below, the magnitude of these correlations appears to have been limited (truncated) by the inclusion of subjects having relatively good English skills (i.e., students placed at the D+ level). In addition, the results of the multiple regression analysis support the construct validity of the ESL Reading and Grammar/Usage tests and suggest that these measures are able to provide insight into examinees' English language proficiency.
In examining the manner in which the ESL Reading and Grammar/Usage tests are associated with the placement of students into levels of the NTID reading and writing curricula (Tables 12 and 13), it is clear that, in comparison to the ACT English and Reading tests, these measures are far more sensitive for examinees having weaker English language and literacy skills. Unlike the ACT Assessment, the ESL Reading and Grammar/Usage tests have been designed for second-language students at the secondary and community college levels. Scores of students with limited English proficiency on these tests are considerably above chance performance, and the tests generally are able to differentiate among students placed at the lower ends of the reading and writing curricula. These tests, however, are not nearly as effective in distinguishing among students placed at the upper ends of the reading and writing curricula. Similar to the ACT Assessment, the ESL Reading and Grammar/Usage tests appear most useful in providing information concerning the English language and literacy skills of a particular subset of the population. Further research with the ESL Reading and Grammar/Usage tests will be needed to determine optimal cut-points for admissions decisions and for investigating their predictive validity and potential use for course placement and outcomes assessment purposes.
At this point in the discussion, it may be helpful to digress a little and offer a cautionary comment concerning reasonable expectations for the improvement of English language and literacy skills in young adults having limited English proficiency. In brief, postsecondary students having limited English proficiency are very likely to make noticeable improvements in English language and literacy skills if given appropriate instruction. However, it is important to recognize that very few postsecondary students lacking the English skills to enroll in a transferable (BS or AAS) degree program will be able to improve their skills sufficiently to qualify for enrollment in such a program within one or two years. The effect of the learner's age on linguistic attainments is such that improvements of this magnitude are rare in postsecondary students (see Bochner & Albertini, 1988).
In addition to its use in college admissions, the ACT Assessment may also assist in course placement. For example, at NTID, ACT Reading test scores are used for initial placement in literature, humanities and social sciences courses. The ACT Reading test provides scores in two distinct areas of reading content, Arts/Literature Reading and Social Studies/Science Reading. The Arts/Literature Reading test is comprised of passages containing reading content in the arts, humanities, and literature. Similarly, the Social Studies/Science Reading test is comprised of passages containing reading content in the social and natural sciences. We have found these content-area reading scores useful for the initial placement of students in the literature, humanities, and social sciences curricula at NTID. One reason that these tests have proven useful for initial course placement is the fact that, unlike the reading and writing curricula, these particular components of the NTID curriculum do not distinguish between the A and B levels. That is, the literature, humanities, and social sciences curricula are divided into three levels each, essentially combining the two lowest levels (A and B) into one.
On occasion, students arrive on campus without having taken the ACT Assessment or feeling that their scores do not accurately reflect their true ability. In these situations, we refer students for content-area reading tests we have devised using the conventional reading assessment portion of COMPASS/ESL. We have used these content-area reading tests to complement the Arts/Literature Reading and Social Studies/Science Reading tests and have found them to be an effective alternative for initial placement in literature, humanities, and social sciences courses.
In summary, data collected in this investigation demonstrate that, if attention is paid to examinees' English skills, both the ACT Assessment and the ESL Reading and Grammar/Usage portions of COMPASS/ESL have the potential to be useful admissions screening tools for postsecondary deaf students, and further research with these tests is warranted. If used in tandem, it appears that the tests can provide a valid, reliable, and coherent approach to admissions screening because the tests are appropriate for different subsets of the student population. As such, these tests have the potential to provide an effective means of identifying applicants who possess sufficient English language and literacy skills to meet the demands of postsecondary educational programs, distinguishing applicants ready for transferable (BS and AAS) and nontransferable (AOS) degree programs from one another and from applicants who are not ready for either category of postsecondary program.
|
References |
|---|
|
TOPIntroductionStatement of the ProblemMethodResultsDiscussionReferences |
|---|
-
American College Testing. (2000a). COMPASS/ESL Reference Manual. Iowa City, IA: ACT Corp.
American College Testing. (2000b). ACT ESL Placement Test Research and Data Report. Iowa City, IA: ACT Corp.
American College Testing. (2003). ACT Assessment User Handbook. Iowa City, IA: ACT Corp.
Albertini, J. A., & Schley, S. (2003). Writing: Characteristics, instruction and assessment. In M. Marschark and P. E. Spencer (Eds.), Oxford handbook of deaf studies, language and education. New York: Oxford University Press, 123–135.
Berent, G. P. (1996). The acquisition of syntax by deaf learners. In W. Ritchie and T. Bhatia (Eds.), Handbook of second language acquisition. San Diego, CA: Academic Press, 469–506.
Bochner, J. (1976). An alternate means of assessing the English skills of deaf students. Teaching English to the Deaf, 3(2), 14–18.
Bochner, J. (1977). Assessing the language proficiency of young deaf adults. Teaching English to the Deaf, 4(2), 26–29.
Bochner, J., & Albertini, J. (1988). Language varieties in the deaf population and their acquisition by children and adults. In M. Strong (Ed.), Language learning and deafness. New York: Cambridge University Press, 3–48.
Educational Testing Service. (1997). TOEFL test & score manual. Princeton, NJ: Educational Testing Service.
English Language Institute. (1977). Michigan Test of English Language Proficiency. Ann Arbor, MI: English Language Institute, University of Michigan.
Fitzgerald, E. (1926). Straight language for the deaf. Staunton, VA: McClure.
Goldberg, J. P., & Boardman, M. B. (1974). English language instruction for the hearing-impaired: An adaptation of ESL methodology. TESOL Quarterly, 8, 263–270.
Greenberg, B. L., & Withers, S. (1965). Better English usage: A guide for the deaf. Indianapolis, IN: Bobbs-Merrill.
Karchmer, M. A., & Mitchell, R. E. (2003). In M. Marschark and P. E. Spencer (Eds.), Oxford handbook of deaf studies, language and education. New York: Oxford University Press, 21–37.
Langston, C.A., & Maxwell, M. M. (1988). Holistic judgment of texts by deaf and ESL students. Sign Language Studies, 60, 295–312.
Paul, P. V. (1998). Literacy and deafness: The development of reading, writing and literate thought. Boston: Allyn and Bacon.
Scouten, E. L. (1984). Turning points in the education of deaf people. Danville, IL: Interstate Publishers.
Swisher, M. V. (1989). The language-learning situation of deaf students. TESOL Quarterly, 23(2), 239–257.
Wing, G. (1887). The theory and practice of grammatical methods. American Annals of the Deaf, 32(2), 84–89.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||