Journal of Deaf Studies and Deaf Education Advance Access originally published online on November 17, 2006
The Journal of Deaf Studies and Deaf Education 2007 12(2):158-171; doi:10.1093/deafed/enl028
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Validity and Reliability of the Classroom Participation Questionnaire With Deaf and Hard of Hearing Students in Public Schools
University of Arizona
Rochester Institute of Technology
Correspondence should be sent to Shirin D. Antia, Department of Special Education, Rehabilitation and School Psychology, College of Education, University of Arizona, Tucson, AZ 85721 (e-mail: santia{at}u.arizona.edu).
Received January 20, 2006; revised September 11, 2006; accepted October 24, 2006
The Classroom Participation Questionnaire (CPQ) was administered to 136 deaf or hard of hearing (D/HH) students attending general education classrooms in Grades 4–10. The CPQ is a student-rated measure that yields scores for Understanding Teachers, Understanding Students, Positive Affect, and Negative Affect. Validity and reliability of a long (28-item) and a short (16-item) form are reported. We provide evidence of (a) internal structure validity through an examination of the relationships between the subscales and an analysis of interitem reliability within each scale, (b) reliability over time by examining the scores of students over a 3-year period, and (c) external structure validity through an examination of the relationships of the CPQ with measures of teacher-rated academic competence and Stanford achievement scores. The results suggest that both the long and short form of the CPQ can be used to assess participation of D/HH students in general education classrooms.
 |
Introduction |
|---|
 TOP Introduction MethodsResultsDiscussionAppendix AReferences |
|---|
Today, the majority of students who are deaf or hard of hearing (D/HH) and receiving special education services are educated in public schools; of these, a large number attend general education classrooms for some or all of their academic subjects. Data from the Annual Survey of D/HH Children and Youth (Karchmer & Mitchell, 2003
) show that during the 2000–2001 school year, 75% of all D/HH students were reported to attend public schools and 44% spent some proportion of the school day in general education classrooms. These numbers are likely to be underestimates of the true numbers of D/HH students in general education classrooms because school districts that serve only a few D/HH students may not respond to the annual survey. A major concern expressed by researchers regarding D/HH students in general education classrooms is their ability to actively and fully participate in classroom instruction and discussion because of their communication difficulties (Garrison, Long, & Stinson, 1994; Saur, Layne, Hurley, & Opton, 1986). For all students, the ability to communicate with teachers and peers can be a major component of academic success as teacher–student communication and student–student communication are the primary means of learning in classrooms. Students who have difficulty communicating may choose not to participate in classroom activities; nonparticipation can adversely affect their learning and eventual academic success (Long, Stinson, & Braeges, 1991). Conversely, students who think they understand the communication of teachers and classroom peers are likely to be academically engaged because they have a sense of control over the learning outcome; they are also likely to believe that they have a good chance of succeeding academically.
Several researchers have documented the specific communication difficulties of D/HH students in classrooms with hearing students. Saur et al. (1986) observed D/HH students in college classrooms over three academic quarters. They reported that a major barrier to classroom participation was the rate of classroom presentation and discussion. Rapid rates resulted in a time lag in sign language interpreting; consequently, D/HH students were not able to respond to instructor questions in a timely manner or responded inappropriately (Saur et al., 1986). Similar results were obtained by Stinson, Liu, Saur, and Long (1996) who interviewed 50 D/HH college students at the National Technical Institute for the Deaf. They divided students into two groups by their preferred mode of communication (oral only or oral and/or sign). They reported that, regardless of the preferred mode, all students perceived classroom communication as a challenge. For the oral students, difficulties with teachers included their lack of clarity of speech; students who depended in part or wholly on sign language interpreters expressed frustration keeping up with the flow of classroom communication. Both groups also indicated that communication with hearing students was difficult, especially during class discussions, because of rapid turn taking, lack of topic coherence, and frequency of interruptions.
Classrooms in which D/HH students find it difficult to participate can lead to "an island of deafness ... where hearing-impaired students appear passive and unresponsive" (Saur et al., 1986, p. 327). Such passivity can negatively impact academic achievement (Braeges, Stinson, & Long, 1993; Long et al., 1991). Long et al. (1991) examined the relationship between communication ease, academic engagement, and academic achievement of D/HH high school students at a school for the deaf. These researchers found significant positive correlations between student self-rated communication ease, student-rated academic engagement, and Stanford achievement scores in language, mathematics, and science. Furthermore, student-rated communication ease was a significant predictor of academic achievement. In a related study, Braeges et al. (1993) found that, for high school D/HH students, teacher-rated communication ease and teacher-rated academic engagement were significantly related. Teacher-rated academic engagement was also a significant predictor of academic achievement (as measured by the Stanford Achievement Test).
The previous studies were conducted with college-age students in mainstream college classes or high school students at schools for the deaf. Some limited and indirect data are available about participation difficulties of school-age D/HH students in general education classrooms. Clearly, classroom participation for students who use sign language is dependent on an interpreter. Sign language interpreters in public schools may not be highly skilled, especially in rural school districts (Antia & Kreimeyer, 2001). Schick, Williams, and Bolster (1999) report that 56% of interpreters in Colorado had less than the minimal skills needed for interpreting in classrooms. A more recent study that included a national sample of 2,091 educational interpreters (Schick, Williams, & Kupermintz, 2006) showed that only 38% met the minimum standards set by the state. Thus, many interpreters may not be able to adequately interpret teacher talk. Moreover, any sign language interpreter will have difficulty interpreting all the communication in the classroom and may have difficulty deciding what classroom communication to interpret (Schick et al., 1999). Consequently, interpreters may omit peer responses to teacher questions, peer questions, and other communication that would contribute to the student's understanding of classroom content (Shaw & Jamieson, 1997). Schick et al. (2006) found that areas of particular difficulty for interpreters included discourse mapping, accurately representing key vocabulary, and accurately indicating the speaker. These aspects of interpreting are crucial for students not only to understand teacher lectures but also to be able to understand the content of, and participate appropriately in, classroom discussion. Another area of interpreting that might affect the student's ability to contribute to classroom discussion is sign-to-voice interpreting, as the student's comments that are inaccurately interpreted may be perceived by peers and teachers as irrelevant or off the point. Schick et al. (2006) found that Registry of Interpreters for the Deaf certified interpreters were quite variable in their sign-to-voice interpreting scores on the Educational Interpreters Performance Assessment and that interpreters who took the elementary version of the test scored significantly less well in sign-to-voice interpreting than those who took the secondary version of the examination. Thus, it is possible that some students in general education classrooms may have interpreters who have difficulty interpreting their comments.
As is true for college students, school-age students who use oral communication may find it easier to join classroom discussions than students who use sign communication (Stinson et al., 1996). However, unlike their hearing peers, they may have to visually locate the speaker during a classroom discussion and, consequently, miss much of what the speaker says. Average classroom noise levels may make it impossible for D/HH students to understand the communication of peers, and they may also have difficulty comprehending other students when several are talking at the same time (Preisler, Tvingstedt, & Ahlstrom, 2005). The degree of hearing loss, classroom acoustics, and the use of group and individual amplification all may affect these students' access to classroom communication and classroom participation.
Because many D/HH students spend a proportion of their day in general education classrooms, it is important to obtain some measure of their classroom participation. One method is to obtain the student's self-perceptions of participation. Such information can assist teachers of D/HH students to work appropriately with general education teachers and hearing students, as well as help the D/HH students develop strategies to enhance their own participation.
Measuring Classroom Communication
To measure classroom communication, Long and colleagues at the National Technical Institute for the Deaf developed a rating scale named the Perceived Communication Ease Questionnaire (Long et al., 1991) and a subsequent version, the Classroom Communication Ease Scale (Garrison et al., 1994). These authors conceptualize communication ease as having both a cognitive and an affective component. The cognitive component includes the student's self-perception of the amount and quality of information received and expressed in the classroom. The affective component includes individual subjective responses to the communicative situation. These subjective responses may be positive feelings, such as feeling good, relaxed, or comfortable, or negative feelings, such as being frustrated, nervous, or upset. The questionnaire statements are written so students can respond to them regardless of the mode of communication. The questionnaire used by Long et al. (1991) consisted of 28 statements rated on a four-point scale and yielded ratings for four subscales. The two cognitive subscales are Understanding Teachers (UT) and Understanding Students (US); the two affective subscales are Positive Affect (PA) and Negative Affect (NA). This version of the scale was validated with students in Grades 7–10 at a school for the deaf (Long et al., 1991). An adapted, revised, but unpublished, version of the scale was used with deaf college students (Stinson et al., 1996).
Given the large numbers of D/HH students in public elementary and middle schools, it is important to know whether the scale can be used with students in elementary and middle school as well as with high school students. Consequently, we modified the administration of the most recent scale so that it could also be used with younger students. The modified scale is renamed the Classroom Participation Questionnaire (CPQ). The purpose of this article is to present reliability and validity data on the CPQ when given to school-age students in public schools and to provide a short 16-item version that would be of considerable practical use to teachers. We envision that the CPQ can be used for two purposes: one is to obtain information about an individual student's participation in the general education classroom and the second would be to measure group performance on a pre- and post-evaluation.
Determining Validity of Educational Measurements
Anastasi and Urbina (1997) suggest that "almost any information gathered in the process of developing or using a test is relevant to its validity" (p. 138). For purposes of analysis, we use the categories of relevant validity information suggested in Nitko (2001) that pertain to the CPQ. Nitko, like Anastasi and Urbina, treats interitem reliability as a separate aspect of the test, although it is a precursor of validity. Because validity information (as well as reliability) must be interpreted as being relevant to the sample of students from which the data are obtained (Sechrest, 2005), we do not generalize any of this information to any other population of students and certainly not to the CPQ as an instrument. The following is a brief summary of the relevant categories of validity evidence for the CPQ derived from Nitko.
- Content representativeness and relevance is the evaluation of whether the tasks are representative of the domain being assessed. In the case of the CPQ, an examination of the items can show whether communication difficulties in the classroom are being addressed.
- Internal structure evidence is the relationship among the parts of the assessment. In the case of the CPQ, an examination of the relationships between the different scales provides this evidence. Internal structure evidence is also provided by evidence of interitem reliability.
- Reliability over time provides information about the stability of the measured construct. We are able to present data about the relationships of CPQ scores obtained for the same students over a 3-year period.
- External structure evidence is the evaluation of whether the assessment procedure predicts current or future performance on external criteria. We expect the students' perception of their classroom participation to be influenced by their degree of hearing loss and their use of interpreters. Because the CPQ is meant to measure classroom participation, which in turn is expected to influence academic achievement, the relationship of the CPQ scores to external measures of academic achievement provides additional evidence of external structure validity. Consequently, we examine the relationships between classroom participation as measured by the CPQ, degree of hearing loss, interpreter use, and students' academic outcomes as measured by the Academic Competence Subscale of the Social Skills Rating System (SSRS; Gresham & Elliott, 1990) and the reading, language, and mathematics subtests from the Stanford Achievement Test (9th edition).
- Practicality evidence is the evaluation of the cost and efficiency of obtaining the assessment information. We are able to provide some data on how many students were unable to take the CPQ.
|
Methods |
|---|
|
TOPIntroductionMethodsResultsDiscussionAppendix AReferences |
|---|
The data for this article were collected as part of a 5-year longitudinal study of academic and social progress of D/HH students in public schools. Students were eligible to participate if they (a) had an identified bilateral or unilateral hearing loss, (b) did not have additional severe cognitive disabilities, (c) received direct or consultative services from teachers of D/HH or had an Individual Education Plan, (d) attended general education classrooms in public schools for 2 or more hours each day, and (e) were in Grades 2–8 at the beginning of the study. Students were recruited from Arizona and Colorado through state agencies and school districts. Requests to allow students to participate in the study were sent to parents of all eligible students. Initially, permission for participation was obtained for 187 students. Data were obtained annually for all participating students on a battery of teacher and student questionnaires that assessed academic performance and social behavior. One of the instruments used was the CPQ. The CPQ and academic data presented here were collected from the students in Year 3 (2003–2004) of the 5-year study. Students responded to the CPQ in the fall semester while the academic data for this report were obtained in the spring of the same academic year. To examine reliability over time, we additionally used the CPQ data obtained with the same students in Years 1 and 2 of the study.
Participants
Data on the CPQ were obtained on 136 students active in the study during Year 3. The students not participating that year were missing for various reasons, primarily because they had moved out of the school districts and we were unable to locate them. Data on student characteristics shown in Table 1 were obtained from each student's teacher of D/HH who completed a demographic form on the student annually. Unfortunately, we were not always able to obtain complete data on each student despite repeated contacts with teachers. Data that were unavailable to (or from) teachers are recorded as missing. Students who received only monitoring or consultation services from the teacher of D/HH were most likely to have missing data.
|
Table 1 shows that the sample of students is quite evenly distributed across grades except for the youngest and oldest students. Almost equal numbers of males and females are included. Hearing loss is presented as pure tone average (PTA) in the better ear. A majority of these students had mild and moderate hearing loss as would be expected in public school classrooms (Karchmer & Mitchell, 2003
), and a substantial percentage of students had a unilateral hearing loss. Only about 20% of students used sign language in school; most of these students used a sign language interpreter in the classroom.
Instruments
Classroom Participation Questionnaire.
The CPQ consists of 28 statements that a student rates on a four-point scale (1, almost never; 2, seldom; 3, often; 4, almost always). The questionnaire yields four subscale scores: Understanding Teachers (UT) (eight statements), Understanding Students (US) (five statements), Positive Affect (PA) (six statements), and Negative Affect (NA) (nine statements). To respond to the teachers who suggested that the CPQ was too long for annual administration, we developed a 16-item questionnaire with four items in each subscale. The 28-items with the selected 16-items for the short version are provided in Appendix A. As mentioned earlier, UT and US were envisioned as cognitive components, whereas PA and NA were envisioned as affective components.
To develop the 16-item scale as part of this study, we requested M. Stinson, one of the authors of the original Perceived Communication Ease Questionnaire and the Classroom Communication Ease Scale, to select four items for each subscale. The intent was to best represent the constructs envisioned by the original authors. In this paper we present the validity evidence for the 28-item (long) and 16-item (short) forms.
The modifications made to the CPQ to adapt its use for younger students were primarily in presentation and administration. The test was formatted to be in reasonably large print, and care was taken that items were not crowded together on a page. Students took the test individually (i.e., not in a group situation as with the original test). They could take the test in more than one sitting as necessary. They could read the items independently or have the items read or signed to them by their itinerant or resource teacher of D/HH. These teachers could give explanations of words that were not understood by the student but were instructed to cease administering the questionnaire if the student needed explanation of more than two or three words.
Academic Competence Scale of the SSRS.
The Academic Competence Scale is one of three scales of the SSRS (Gresham & Elliott, 1990). The other two scales yield information on the student's social skills and problem behaviors. The Academic Competence Scale is a nine-item scale completed by teachers. Teachers rate students on a five-point scale, placing each student in the lowest 10%, the next lowest 20%, middle 40%, next highest 20%, or highest 10% on reading and mathematics compared to classmates and grade expectations. They also rate the students on motivation, intellectual functioning, classroom behavior, and parental encouragement. The Academic Competence Scale yields a standard score with a mean of 100 and an SD of 15. The SSRS was normed on a national sample of 2,109 students in Grades 3–12. Seventeen percent of the students in that sample were identified as having a disability. Coefficient alpha reliability for the Academic Competence Scale was .95 and test–retest reliability over a 4-week period was .93 in the standardization sample.
Stanford Achievement Test—9th edition.
The Stanford Achievement Test (Harcourt Brace Educational Measurement, 1997) is a standardized, norm-referenced assessment including reading, mathematics, and language that was administered to Arizona students in Grades 2–9 each spring as required for the state of Arizona accountability purposes. Colorado students did not take the Stanford and are therefore not included in analyses that included this test. The Stanford reading subtest measures reading vocabulary and comprehension; the mathematics subtest measures problem solving and mathematics procedures; and the language subtest includes prewriting, composing, and editing. The Stanford Achievement Test was normed on approximately 250,000 students from 1,000 school districts across the United States. Students with disabilities or limited English proficiency who received instruction within a regular education classroom composed 5.5% of the norming sample. The Gallaudet Research Institute conducts a norming study and publishes normative data on each version of the Stanford with D/HH students. However, because the sample for this study included primarily students in general education classrooms, we have used the norms for the general population, not the D/HH population. The norms for D/HH students do not provide a good comparison for our sample because they only provide percentile ranks and include large numbers of children who have taken out-of-level tests. Therefore, these norms do not necessarily provide comparisons to children at the same age or grade level. These norms also do not provide comparisons for the children with minimal hearing loss who are in our sample. The scores reported on the Stanford are normal curve equivalents; these are standard scores with a mean of 50 and an SD of 21.06.
Procedure
Each student's itinerant or resource teacher of D/HH administered the CPQ in the fall semester during the regular time to meet with the student. Teachers gave the CPQ using the modifications described. A space was provided for teacher notes, so that teachers could inform us if the student's responses seemed invalid for any reason.
All students completed the ratings in reference to their general education classrooms. Middle and high school students were asked to think of their language arts or social studies classes. If none of these were appropriate, they were asked to choose another class including both lecture and class discussion. Students who used interpreters were instructed to answer the questions about communication with other students and with teachers taking into account communication through the interpreter.
The SSRS Academic Competence Scale was completed by each student's general education teacher in the spring semester. For elementary students the classroom teacher completed the scale, whereas for secondary students the scale was completed by the language arts teacher with input from other teachers as needed. Data were obtained on 130 of the 136 students for both the SSRS Academic Competence Scale and the CPQ.
The Stanford achievement scores on reading, mathematics, and language were obtained from student records. These tests were administered once a year (during the spring) only for students in Arizona (the students in Colorado did not take the Stanford Achievement Test). There were no makeup days; therefore, scores were not available for students who were absent on the testing days. Data were available for 69 students for the reading subtest, 66 for mathematics, and 70 for language.
|
Results |
|---|
|
TOPIntroductionMethodsResultsDiscussionAppendix AReferences |
|---|
The following section provides the data for each category of validity: content representativeness, internal structure, external structure, reliability over time, and practicality.
Content Representativeness
The best evidence of content representativeness is the list of the items in Appendix A. The potential user of the CPQ should examine this set of items to determine whether the items are appropriate for measurement of the construct of interest. A further examination of the content is included later when we discuss the determination of total scores.
Internal Structure Evidence: Results and Interpretation
We present first the relationships between the four subscales of the CPQ (UT, US, PA, and NA). These relationships help determine whether the components of the CPQ work together so that each component contributes positively toward assessing the relevant construct. We then present data for interitem consistency. For the NA scale, the scoring is reversed so that for all scales high scores are desirable.
Subscale correlations.
Table 2 presents the correlations between pairs of subscales. Because the correlation obtained between two subscales depends on the reliability of each subscale, it is easier to examine the correlations if they are corrected for attenuation (Nunnally & Bernstein, 1994). The correction for attenuation is intended to estimate the correlation that would be obtained if each subscale were perfectly reliable, and thus, the corrected correlations are an attempt to directly infer the degree to which the subscales measure the same construct. The correction for attenuation used to adjust the correlations in Table 3 is applied by dividing an obtained correlation by the square root of the product of the reliabilities of the two variables correlated (see Nunnally & Bernstein, 1994, for the formula and the explanation of the approach). The correlation between two perfectly reliable measures would best describe the relationship between the two constructs being compared. Table 3 presents the corrected-for-attenuation correlations between pairs of subscales for the 28-item and the 16-item forms.
|
|
The data in Tables 2 and 3 address the issue of how the tasks help define the construct of classroom participation. Because of some differences in the reliability coefficients described above, the data in Table 3 are the better indicators of task intercorrelations of the parts of the construct. As shown in Table 3, the correlations among the UT, US, and PA subscales are substantially higher than the correlations involving NA. The highest relationship occurs between US and PA, although US was intended to measure a cognitive component and PA was intended to measure an affective component of communication. Thus, for our data, the cognitive and affective aspects of the CPQ are not well differentiated.
The type of reasoning used in the previous paragraph involves an examination of what are termed convergent and discriminant aspects of validity. The examination of convergence asks whether the task correlates highly with those traits that it should resemble; the examination of discriminant validity asks whether the task correlates lower with traits that it should not resemble. The NA subscale was originally intended to have more convergent validity with PA and more discriminant validity with respect to UT and US. In this respect, NA is not correlated high enough with PA but does have the desired lower correlations with UT and US. UT and US correlate well enough with each other, but the US with PA correlation is problematic because discriminant validity was expected for that relationship by the original authors (Garrison et al., 1994).
A possible resolution to the lack of convergent/discriminant validity of the subscales is to avoid a cognitive versus affective interpretation of the data. Thus, it is not appropriate to report averaged scores for cognitive (UT, US) and affective (PA, NA) communication or to report a total score. However, each subscale can be considered a separate construct. Given the high correlations between UT, US, and PA, an additional solution is to have a summative score for the three convergent subscales (UT, US, and PA) and use the NA subscale as an additional measure. Therefore, the following tables will include reliability and validity data on the composite, UT/US/PA, as well as the individual subscales.
Interitem consistency.
Interitem consistency (Anastasi & Urbina, 1997) measures the degree to which the items in a set measure the same construct, and we used coefficient alpha (Cronbach, 2004) for this assessment. Interitem consistency reliability was examined separately for the UT/US/PA composite and each subscale of the 28-item and 16-item forms. Table 4 presents these data obtained from the total sample of 136 students. Reliability for each subscale of the 28-item scale is higher than the reliability for the corresponding subscale of the 16-item scale, as is expected due to the increased number of statements. The reliability coefficients in Table 4 for the 28-item test range from .81 to .89 and on the 16-item test from .78 to .82. The reliabilities for the subscales for both the 28-item and 16-item scales of the CPQ from this sample are higher than those reported by Long et al. (1991) who reported interitem consistency reliabilities of .77 for UT, .73 for US, .63 for PA, and .80 for NA.
|
Means and SDs for the subscales for the 28-item and 16-item forms are similar. Mean subscale scores are calculated by summing individual item ratings and dividing by the number of items in the subscale; so the number of statements does not change the magnitude of the scores. Note that for the NA scale the scoring is reversed to make scoring comparable for all scales.
Reliability Over Time
For measuring reliability over time (stability), we used the test–retest method over 1- and 2-year intervals for those students who had data available for those years. These correlations are presented in Table 5. The stability estimates presented in Table 5 are, as expected, considerably lower than the intercorrelations between subscales reported in Tables 2 and 3, probably because of the changing situations occurring as the students moved into different classrooms with different classmates and teachers each year. We were unable to give the CPQ to students more than once a year to get a better measure of test–retest reliability under more consistent conditions.
|
External Structure Evidence
Relationships of the CPQ to other meaningful variables provide external structure evidence of validity. This category of evidence allows the reader to examine whether the tasks exhibit meaningful relationships with variables that might be used to better understand the construct measured by the CPQ. To examine the external structure evidence, we correlated each of the four subscales and the composite of UT/US/PA with teacher ratings of the SSRS Academic Competence Scale and with the Stanford test scores. We also correlated the CPQ scores with two demographic variables that we expected to be related to the student's perceptions of classroom participation: better ear PTA and whether the student used an interpreter.
Table 6 shows the correlations of the CPQ subscales and the UT/US/PA composite with the SSRS Academic Competence Scale and the Stanford reading, mathematics, and language scores; better ear PTA; and interpreter use. Student ratings of classroom participation and teacher ratings of academic competence and Stanford scores are positively related (p < .01), with the exception of the Stanford mathematics on the 16-item form. A less severe hearing loss, as measured by better ear PTA and not having an interpreter is associated with UT and US but not with PA ratings. Degree of hearing loss and not having an interpreter is associated with NA ratings only on the 16-item form. The composite UT/US/PA is significantly related to all the external academic measures.
|
To determine whether the CPQ can explain variance in academic achievement beyond explanations based on demographic variables such as degree of hearing loss and use of interpreters, we analyzed the data using sequential multiple regression. Two analyses were conducted: In the first analysis, interpreter use was entered as an independent variable in Step 1, better ear PTA and interpreter use were entered as independent variables in Step 2, and finally the CPQ composite score (UT/US/PA) and NA were entered additionally in Step 3. In the second analysis, PTA was entered as an independent variable in Step 1; the remaining analysis was the same. The results for the 28-item and 16-item scales are shown separately in Table 7. The first two columns show the results when interpreter use is entered first; the last two columns show the results when PTA is entered first. In both pairs of columns, CPQ scores are entered as a third step. These data show that PTA alone does not appear to predict much variance in academic outcomes for this sample of students or to add to the prediction based on interpreter use. Interpreter use and PTA together significantly predict academic outcomes, while the prediction is improved further when the CPQ scores are added. The complete model (i.e., better ear PTA, interpreter use, and 28-item CPQ scores) explains 13% of variance for Academic Competence scores, 32% of the variance for reading, 21% of the variance for mathematics, and 30% of the variance for language scores. The CPQ scores explained the most additional variance (18%) for language scores. The 16-item scale explains less variance than the 28-item scale for each outcome except Academic Competence scores.
|
Practicality
The CPQ is fairly easy for most students to complete. Only two students between Grades 4 and 10 were reported not to be able to take the test because of their inability to comprehend the items when either read or signed to them. Teacher comments that the 28-item scale was too long provided the rationale for developing the 16-item scale.
|
Discussion |
|---|
|
TOPIntroductionMethodsResultsDiscussionAppendix AReferences |
|---|
The purpose of this article is to provide evidence for using the CPQ to assess the participation of D/HH students in general education classrooms. The CPQ has previously been used only with high school and college students; the data presented here provide evidence that the instrument can be used with students in elementary and middle school classrooms as well. Additionally, the CPQ had previously been used with high school students at a center school; our data indicate that elementary and middle school students can reliably rate their own participation in general education classrooms. Our sample also provides evidence that the CPQ can be used with hard of hearing students, as our sample, unlike previous samples, includes a high percentage of students with unilateral, mild, and moderate hearing losses. Because classroom participation is a concern for D/HH students in these classrooms and can impact student learning, teachers can use the results of the CPQ in addition to other assessments to assist in decision making about placement, interpreter use, and other issues. Our findings suggest that both the long (28-item) and short (16-item) forms can be useful to teachers or researchers, although they may be used for different purposes. We recommend using the long form when individual decisions are to be made (e.g., changes in classroom placement for a student). However, teachers might be able to use the short form if time is a factor, for example, when teachers have to read the scale to the students. The short form would also be of interest to researchers who are interested in examining primarily group data. Some possible uses for researchers might be to measure change after specific interventions to increase communication participation, to examine student's perceptions of differences in communication in different educational settings, or to compare student's self-perception with teacher's perceptions of student communication.
Our data indicate that items on the CPQ do not measure separately the cognitive and affective components of communication in classrooms as envisioned by the authors of the original scale. The results show a high correlation between UT, US, and PA, whereas NA does not correlate highly with any of the other subscales, perhaps because responding to the NA negatively criterial statements is different than responding to positively criterial statements in the other three subscales. We therefore recommend using a composite score (UT/US/PA) and a separate score for NA. One of the reasons for the high correlations between US and PA may be because both subscales include several items about group discussion.
In this study, NA does not correlate significantly with any of the Stanford scores in the 28-item scale but it does correlate significantly with reading and mathematics in the 16-item scale. The correlations reported here are lower than those reported by Long et al. (1991) who reported significant correlations of .42 for mathematics and .29 for language. Differences in the characteristics and school experiences of the two sets of participants may have contributed to the different results. The students in the Long et al. study were enrolled in a school for the deaf, in contrast to this study where the students were in public schools. Despite our results, we do not at this time recommend deleting the NA scale. Instead, we recommend that teachers use it diagnostically and consider interventions for students who complete ratings of often or almost always on the NA statements.
The CPQ scores are significantly, though modestly, correlated with academic achievement. The correlation between communication participation and student achievement lends support to the idea that students who participate and who feel positively about their participation are more likely to do well academically, although, of course, a cause-and-effect relationship cannot be established. It is interesting that each of the CPQ subscale scores is positively and significantly correlated to teacher-rated academic competence. Braeges et al. (1993) also reported significant positive correlations between teacher-rated academic engagement and student-rated communication ease for students in Grades 7–10 attending a school for the deaf. Their data are not exactly comparable with ours as they used a total score of all four subscales and reported correlations of .48 for mathematics and .32 for language; our data (on the 28-item scale) show a correlation of .29 for mathematics and .43 for language with the composite score of UT/US/PA. Of course, the samples in the two studies are quite different; their sample consisted of deaf high school students attending a center school; our sample can be characterized as mostly hard of hearing students in general education classes. Also, our sample of students are likely to have a much wider range of academic achievement than those in their study. Our data, therefore, extend their findings and show a relationship between teacher-rated academic competence and student-rated classroom communication for younger students in general education classrooms. It should be remembered that the teachers who rated academic competence were the students' general education teachers, whereas the teachers of D/HH administered the CPQ to the student. The general education teachers were not aware of the students' CPQ ratings. Thus, the general education teachers' ratings were independent of the students' ratings.
We had expected that degree of hearing loss and interpreter use would be significantly correlated with CPQ scores. Our data show that they are moderately (though significantly) correlated with UT and US but not PA or NA (on the 28-item scale). Our data also show that CPQ scores are reasonably good predictors of academic achievement when added to demographic predictors such as degree of hearing loss and interpreter use. Degree of hearing loss alone added very little to the prediction of academic achievement. Interpreter use predicted achievement in reading, mathematics, and language, as well as teacher-rated academic competence. Whereas hearing loss and interpreter use together accounted for between 6% and 16% of variance on academic outcomes, adding the CPQ scores allowed us to account for between 13% and 32% of variance on academic outcomes. Although these predictions are significant, they are still modest, and clearly several other factors, not measured here, affect outcomes. However, our data make clear that an assessment of students' perceptions of classroom participation is valuable and that assumptions about the ability or ease of student participation in the classroom should not be based solely on student demographic variables.
An examination of Table 6 shows that US consistently has the highest correlations with all the academic outcome measures. The relationship between US and academic achievement points to the importance of understanding peers as part of classroom learning. As noted by Stinson et al. (1996), college-age D/HH students indicated that communication with peers was a challenge. Clearly, the same challenge might exist for school-age D/HH students.
Although the interitem reliabilities for both the 28- and 16-item scales are respectable, they are probably not high enough to make important decisions about individuals, but important decisions should not be made based on a single test. The coefficients are high enough for most group decisions. Reynolds, Livingston, and Willson (2005) suggest that judgments about reliability should be made in the context of decisions being made based on test results, and reliabilities of .80 are acceptable in many testing situations. The interitem correlations for the 16-item form provide evidence that the items chosen are more reliable than a random sample of items would be.
Although the correlations between years are significant, they are quite modest. We expected lower correlations across years because students encounter different teachers and must adjust to a new environment each year. The higher correlations across years for the composite (UT/US/PA) compared to the individual subscales are expected because of the increased number of items when three subscales are combined. However, the expectation that additional items result in higher correlations was not evident in the comparison of the stability of the 16-item subscales or composite compared with the 28-item subscales. Again, the similarity of the stability estimates for these two forms is evidence that the items chosen for the short form are more stable than a random sample of items would be.
Teachers can use the individual scales of the CPQ diagnostically to alert them to the student's problems in classroom communication that may result in academic difficulties. The results of the CPQ might help them to determine when and how to intervene. For example, if students rate themselves low in understanding their peers, the teacher of D/HH could observe the classroom to identify specific group discussion practices that might be modified. The teacher of D/HH might also provide information to classroom teachers and hearing classmates to increase their sensitivity about classroom communication difficulties that the D/HH student might have. The teacher could use specific CPQ responses to engage with the student in developing self-advocacy strategies such as requesting clarification during group discussions or requesting a quiet space for group projects.
The modifications made to the questionnaire administration allowed students who were not able to read fluently to complete the self-rating. Teachers did not report that students had difficulty understanding the questionnaire items. However, we did not obtain data on how many students read the items independently. Future research on the use of the CPQ should examine carefully the effects of test-taking modifications.
In conclusion, the CPQ may be a useful tool to examine participation in general education classrooms for elementary and middle school students as well as high school students. The instrument could be used diagnostically by teachers to intervene in difficult classroom communication situations where necessary. However, we do not advocate using the CPQ alone to make decisions about educational placement.
|
Appendix A |
|---|
|
TOPIntroductionMethodsResultsDiscussionAppendix AReferences |
|---|
CPQ Items Arranged by Subscale
Understanding Teachers
- My teacher understands me
- *I understand my teacher
- I have enough time to answer the teachers' questions
- I understand the homework assignments my teacher gives me
- I understand when my teacher tells me what to study for a test
- *I understand my teacher when she gives me homework assignments
- *I understand my teacher when she answers other students' questions
- *I understand my teacher when she tells me what to study for a test
- *I understand my teacher
Understanding Students
- The other students in class understand me
- *I understand the other students in class
- *I join in class discussions
- *I understand other students during group discussions
- *I understand other students when they answer my teacher's questions
- *I understand the other students in class
Positive Affect
- *I feel good about how I communicate in class
- I feel relaxed when I talk to other students
- *I feel relaxed when I talk to my teacher
- I feel relaxed in group discussions
- *I feel happy in group discussions in class
- *I feel good in group discussions in class
- I feel relaxed when I talk to other students
Negative Affect
- I feel lonely because I cannot understand other students
- *I feel frustrated because it is difficult for me to communicate with other students
- *I get upset because other students cannot understand me
- *I get upset because my teacher cannot understand me
- I feel nervous when I talk to other students
- I feel nervous when I talk to my teacher
- I feel nervous in group discussions in class
- I feel frustrated in group discussions in class
- *I feel unhappy in group discussions in class
- *Items included in the 16-item short scale.
- *I feel frustrated because it is difficult for me to communicate with other students
|
Acknowledgments |
|---|
This research was supported by the U.S. Department of Education, Office of Special Education Programs, Field Initiated Research (H324C010142).
|
References |
|---|
|
TOPIntroductionMethodsResultsDiscussionAppendix AReferences |
|---|
-
Anastasi A and Urbina S. (1997) Psychological testing 7th ed. (Prentice-Hall, Upper Saddle River, NJ).
Antia SD and Kreimeyer KH. (2001) The role of interpreters in inclusive classrooms. American Annals of the Deaf 146:355–365.[ISI][Medline]
Braeges J, Stinson MS, Long G. (1993) Teachers' and deaf students’ perceptions of communication ease and engagement. Rehabilitation Psychology 38:235–247.
Cronbach LJ. (2004) My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement 64:391–418.[Abstract]
Garrison W, Long G, Stinson MS. (1994) The classroom communication ease scale. American Annals of the Deaf 139:132–140.[ISI][Medline]
Gresham FM and Elliott SN. (1990) Social skills rating system(American Guidance Service, Circle Pines, MN).
Harcourt Brace Educational Measurement. (1997) Stanford achievement test series, ninth edition: Spring multilevel norms book(Author, San Antonio, TX).
Karchmer M and Mitchell RE. (2003) Demographic and achievement characteristics of deaf and hard-of-hearing students. In Marschark M and Spencer PE (Eds.). Oxford handbook of deaf studies, language and education(Oxford University Press, New York) pp. 21–37.
Long G, Stinson MS, Braeges J. (1991) Students' perception of communication ease and engagement: How they relate to academic success. American Annals of the Deaf 136:414–421.[ISI][Medline]
Nitko AJ. (2001) Educational assessment of students 3rd ed. (Merrill Prentice Hall, Upper Saddle River, NJ).
Nunnally JC and Bernstein IH. (1994) Psychometric theory 3rd ed. (McGraw-Hill, New York).
Preisler G, Tvingstedt A, Ahlstrom M. (2005) Interviews with deaf children about their experiences using cochlear implants. American Annals of the Deaf 150:260–267.[CrossRef][ISI][Medline]
Reynolds CR, Livingston RB, Willson V. (2005) Measurement and assessment in education(Pearson, Boston).
Saur RE, Layne CA, Hurley EA, Opton K. (1986) Dimensions of mainstreaming. American Annals of the Deaf 131:325–330.[ISI][Medline]
Schick B, Williams K, Bolster L. (1999) Skill levels of educational interpreters working in public schools. Journal of Deaf Studies and Deaf Education 4:144–155.
Schick B, Williams K, Kupermintz H. (2006) Look whose being left behind: Educational interpreters and access to education for deaf and hard of hearing students. Journal of Deaf Studies and Deaf Education 11:3–20.
Sechrest L. (2005) Validity of measures is no simple matter. Health Services Research 40:1584–1604.[CrossRef][ISI][Medline]
Shaw J and Jamieson J. (1997) Patterns of classroom discourse in an integrated, interpreted elementary school setting. American Annals of the Deaf 142:40–47.[ISI][Medline]
Stinson MS, Liu Y, Saur RE, Long G. (1996) Deaf college students' perceptions of communication in mainstream classes. Journal of Deaf Studies and Deaf Education 1:40–51.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||