Journal of Deaf Studies and Deaf Education Advance Access originally published online on November 27, 2007
The Journal of Deaf Studies and Deaf Education 2008 13(2):241-256; doi:10.1093/deafed/enm058
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Hyperlink Format, Categorization Abilities and Memory Span as Contributors to Deaf Users Hypertext Access
University of Manchester
University of Verona
University of Padova
The authors wish to express their gratitude to the deaf and hearing students from the Istituto Magarotto di Padova who contributed to this study by spending their time with us. We also want to thank Dr Ladislao Salmerón for his help in the development of the experimental tools. No conflicts of interest were reported. Correspondence should be sent to Inmaculada Farjardo, Manchester Business School, Booth Street West, Manchester M15 6PB, UK (e-mail: ifajardo{at}manchester.ac.uk).
Received January 17, 2007; revised October 5, 2007; accepted October 15, 2007
Sixty deaf and hearing students were asked to search for goods in a Hypertext Supermarket with either graphical or textual links of high typicality, frequency, and familiarity. Additionally, they performed a picture and word categorization task and two working memory span tasks (spatial and verbal). Results showed that deaf students were faster in graphical than in verbal hypertext when the number of visited pages per search trial was blocked. Regardless of stimuli format, accuracy differences between groups did not appear, although deaf students were slower than hearing students in both Web search and categorization tasks (graphical or verbal). No relation between the two tasks was found. Correlation analyses showed that deaf students with higher spatial span were faster in graphical Web search, but no correlations emerged between verbal span and verbal Web search. A hypothesis of different strategies used by the two groups for searching information in hypertext is formulated. It is suggested that deaf users use a visual-matching strategy more than a semantic approach to make navigation decisions.
 |
Factors Contributing to Deaf Users Hypertext Access |
|---|
 TOP Factors Contributing to Deaf... MethodResultsDiscussionReferences |
|---|
A general concern among teachers and educators is that deaf students often do not seem to benefit from traditional educational settings and tools. Deaf students' challenges in scholastic and academic learning settings seem to be related to, among other general skills, their difficulties in reading and accessing information from books (Harris & Moreno, 2004
; Kelly, 1996; Marschark & Harris, 1996; Musselman, 2000). In turn, these difficulties are mainly attributed to deaf individuals' limited knowledge of the phonology, vocabulary, and syntax of printed language (Harris & Moreno, 2004; Kelly, 1996; Marschark & Harris, 1996; Musselman, 2000; Wauters, van Bon, Tellings, & van Leeuwe, 2006). To overcome these difficulties, technology and media have become an integral part of deaf students educational environment (Loeterman, Paul, & Donahue, 2002). Because they lean simultaneously on verbal and visual information, these tools are considered potentially useful and powerful for deaf students. Among these resources, the Internet is an important and widely used medium for accessing knowledge and developing literacy skills. Hypertexts or Web sites contained on the Internet are documents composed of a set of graphical, textual, or audible information pages (also called "nodes") connected by links that allow users access to information in different and flexible ways. This multimodal nature of hypertext documents represents a potential benefit for deaf persons, who may rely on visual information to support their understanding of textual information.
Accessing information through the Web is far from a simple task. It requires effective searching strategies, the coordination of language, visual, and planning processes in memory, and the use of linguistic and pictorial information to direct or redirect the Web search (Salmerón, Cañas, & Fajardo, 2005). In order to easily access information through the Web, users must recognize Web organization and hypertext structures and the searching strategies that best cope with each structure and searching goal. They must have knowledge of the language used in the hypertext, which might be technical or specific, and have reading skills that allow them to decode the written text of the hypertext. Finally, they must have some general knowledge of the Web site conventions (e.g., color of visited links) and some knowledge of the specific topic of interest, which leads them in selecting the right links and nodes to reach the target information in the Web. These abilities and knowledge could interact with the hypertext design such as the hyperlink format or the hypertext topology (numbers of nodes, connectivity degree, etc.) in determining the Web users performance.
Accordingly, Web search outcomes of deaf users could be mediated by two components: one represented by the hypertext design and the other by deaf students' knowledge of the language used in the hypertext and of search strategies appropriate to the tool/mean (the hypertext). On the one hand, some hypertexts on the Internet present elaborate and abundant visual stimuli, multiple links and levels of information, and a highly specific and technical vocabulary. These factors make information search difficult for all nonexpert Web users and for deaf persons in particular. On the other hand, deaf students' difficulties in using verbal hypertext might have the same roots of their text comprehension difficulties: being the result of a lack of both low-level (word processing) and high-level (inference making, reading strategies) reading processes (Kelly, 1996; Loeterman et al., 2002).
In the following, based on previous literature, we provide a more comprehensive illustration on the impact of these factors on deaf students' accessibility to hypertext information.
Previous Findings on Factors Contributing to Deaf Users Hypertext Access
As in the case of hearing users (Vicente, Hayes, & Williges, 1987), reading comprehension ability has been shaped as a critical factor contributing to the hypertext search efficiency of deaf users. In one of the studies of an experimental series about deafness and Web interaction, Fajardo (2005) asked deaf and hearing users to search for a set of headlines in a digital newspaper composed of textual links exclusively ("culture," "weather forecast," "sports," etc.). In addition, users completed both a reading comprehension test and, immediately after the searches, a relatedness judgment task for evaluating the knowledge about hypertext's content that was acquired during the interaction. Hearing users outperformed deaf users in terms of targets found and search time in hypertext. For deaf users, percentage of targets found and knowledge acquisition were predicted by reading comprehension. The higher the deaf users reading comprehension level, the more targets they found and the better the knowledge acquisition. In the same line of results, Namatame and Kitajima (2005) found that deaf participants performed less efficiently than hearing participants in text-based Web information search. Therefore, because textual links seem to be one of the hypertext elements generating accessibility problems to deaf users, Fajardo, Cañas, Salmerón, and Abascal (2006) performed a new experiment in order to explore the possibility of improving deaf students' performance by using graphical links instead of textual links. Such a hypothesis about a graphical superiority effect for deaf students was derived from two arguments:
- According to hypertext information search models, the semantic similarity judgment is an important process during information search in hypertext Comprehension-based Linked model of Deliberate Search-CoLiDeS of Kitajima, Blackmon, & Polson [2000]).
- In agreement with a central assumption in cognitive psychology, pictures are superior to words in semantic tasks (e.g., Nelson, Reed, & Walling, 1976; Paivio, 1977, 1991).
found that the effect of interface format was not significant for targets located in shallow nodes of the hypertext structure (higher levels of the hypertext contents hierarchy) and, more unforeseen, both deaf users and hearing users in the verbal interface outperformed those in the graphical interface when the targets were located in deep layers of the hypertext (lower levels of the hypertext contents hierarchy), that is, when it is hypothesized that semantic processing becomes more difficult (Norman, 1991).
This result contrasted not only with the classical effect in cognitive psychology research but also with other findings in the field of menu or hypertext information search where the predicted graphical interface superiority was found (Blankenberger & Hahn, 1991; Dillon & Song, 1997). In particular, the results of Fajardo et al. (2006) seem to be inconsistent with the finding of Namatame, Nishizaki, and Kitajima (2007), who found that hard-of-hearing participants answered more correctly than hearing participants in a task consisting of pairing directory names, typically used in representative Web sites, with pictograms. This surprising finding led us to inquire into the stimulus characteristics or the cognitive factors underlying verbal or pictorial superiority. It is possible that the word superiority in our previous experiments was due to uncontrolled factors such as the visual distinctiveness, the familiarity, or the concreteness of the concepts used. For instance, abstract concepts (e.g., culture) would be processed faster by the verbal code than by the visual one (Paivio, 1986). The picture superiority effect has been contrasted with normalized sets of pictures in cognitive psychology research (Lotto, Job, & Rumiati, 1999; Snodgrass & McCullough, 1986), whereas, in our previous studies, we just controlled the semantic distance of icons and their color (black and white), but each picture came from a different source and their familiarity and frequency of use was not controlled.
Thus, this study was conducted in order to accomplish two aims: (a) clarifying previous findings by contrasting information search performance in graphical versus verbal hypertext interfaces with a normalized set of pictures and words (in terms of familiarity, typicality, and frequency). In particular, we predicted that the graphical interface (hypertext with normalized graphical links) would facilitate the performance of deaf users compared to a verbal interface (hypertext with verbal links) and reduce the disadvantages regarding hearing users, and (b) exploring in greater detail cognitive factors contributing to deaf students hypertextual search performance.
With respect to this second objective, three considerations motivated the study. First, if, as suggested by hypertext information search models, semantic decision is a core process during searching (Kitajima et al., 2000), then the postulated difference between deaf and hearing people in access strategies or organization of semantic memory (e.g., Marschark, De Beni, Polazzo, & Cornoldi, 1993) would influence the performance in this tasks. Specifically, categorical decisions, a particular kind of semantic decisions, would be involved in hierarchical hypertexts, as the hypertext system utilized in Fajardo et al. (2006). In this kind of hypertext, links are hierarchically organized in categories and subcategories. Therefore, users have to follow a route from subordinate links located in the first layer of nodes of a specific hypertext (e.g., sport), to superordinate links located in deeper layers of nodes (e.g., football), that is, they have to use their categorical representation of a set of concepts in order to make pathway decisions. Consequently, the finding that deaf people underperform their hearing peers in tasks that imply the use of taxonomical or categorical information (e.g., Marschark, Convertino, McEvoy, & Masteller, 2004; Marschark & Everhart, 1999) should be relevant for information search in hierarchical hypertext.
In addition to the relevance of the exploration of categorical abilities and hypertext accessibility relationship, to our knowledge, deaf people's categorization abilities with pictorial versus textual material have not been tested before. A classical task for picture–word semantic contrasting is the single-stimulus classification task, in which participants are asked to classify stimuli in one of two categories (Job, Rumiati, & Lotto, 1992; Snodgrass & McCullough, 1986). We were interested in finding out which pattern of results appeared in deaf individuals in this task, and if there were any differences between deaf and hearing individuals, which could potentially explain differences in hypertext performance. Essentially, our second hypothesis stated that the higher the student's scores in a categorization test, the better was supposed to be his or her hypertext performance (in terms of target found and response time).
A second consideration was related to another dimension along which deaf and hearing persons seem to differ: working memory span. Deaf people usually show lower verbal span than hearing people (e.g., Chincotta & Chincotta, 1996; Flaherty, 2000; Logan, Maybery, & Fletcher, 1996), and on some occasions, the opposite pattern is found regarding spatial span (Wilson, Bettger, Niculae, & Klima, 1997). These resources have been shown to be involved in complex and demanding tasks such as hypertextual information search (e.g., Larson & Czerwinski, 1998; Lee & Tedder, 2004). Thus, differences in verbal and spatial span could be a potentially explicative factor of differences in the hypertext tasks between deaf and hearing users. Specifically, we expected to find that the higher the verbal span of participants, the better the performance in verbal hypertext (in terms of targets found and response time), and the higher the visual span, the better the graphical hypertext performance (in terms of target found and response time).
Third, some final considerations pertain to the role of vocabulary knowledge in hypertext search. Another disadvantage in Web search could arise for deaf people from their relatively poor verbal vocabulary, which may limit and seriously constrain their performance (Fabbretti, Volterra, & Pontecorvo, 1998; Goldin-Meadow & Mayberry, 2001; Kelly, 1996). If students have to make pathway decisions following linguistic directions, such as happens when the hypertext nodes are verbal, assessing deaf students verbal vocabulary knowledge becomes essential. Therefore, the measure of this factor was also introduced in the study in order to test the relation between deaf students' verbal vocabulary level and their hypertext search performance.
|
Method |
|---|
|
TOPFactors Contributing to Deaf...MethodResultsDiscussionReferences |
|---|
Participants
Sixty-one high school students participated, with a mean age of 17.9 years (SD = 3.1). Thirty participants (15 males and 15 females; age range, 13–25 years; M = 16.8; SD = 2.6) were deaf students from the Istituto Magarotto of Padova, a vocational school for the deaf, and the remaining 31 were hearing students of a mainstream vocational high school in Padova, Italy (15 males and 16 females; age range 14–25 years; M = 16.9; SD = 2.3), matched to the deaf students for their grade level. Strict criteria were followed for the selection of the deaf students: IQ equal or above 85, hearing loss greater than 70 dB, knowledge and use of sign language or signed Italian, no related behavioral difficulties, and no diagnosis of attention deficits or other learning disabilities. Students who did not meet these criteria were free to participate to the hypertext task if they wished to, but they were not included in the study. Seventy students were originally contacted. Forty of them met the inclusion criteria; however, only 30 voluntary completed all the tests and tasks of the study.
The 30 deaf students were prelingually deaf, 22 of which were native Lingua dei Segni Italiana (LIS) signers and 8 used the signed Italian in school. Signed Italian is an Italian-based sign system, the purpose of which is to transpose spoken Italian visually for the deaf and people hard of hearing. It follows Italian grammar and syntax but also adopts much of the vocabulary of Italian Sign Language. This system is derived from an artificial mixture between two natural languages (Italian and Italian Sign Language), and it is a bimodal system of communication, in which both oral and visual modes are used. In contrast, Italian Sign Language is a visuospatial language naturally developed by deaf users and different from Italian. That is, it has its own grammar, syntax, and vocabulary.
Deaf students IQ as measured by Raven's Progressive Matrices (Raven, Court, & Raven, 1991) was normal. The Istituto Magarotto is a typical residential high school at one of the oldest Italian schools for the deaf. Within the school, signed Italian is used by educators and teachers. As in many other schools for the deaf, the background experience of the students tends to be quite heterogeneous. Students come from different regions, and their educational and linguistic experiences vary considerably; some of them had attended orally oriented schools for the deaf before, in which bimodal communication (signed Italian) and oral communication are used, whereas others had attended schools, in which LIS in addition to bimodal communication is used.
The two groups were familiar with the Internet, and all students were Web users. The Internet was widely used within the school for the deaf both for didactical activities and social purposes. The school also has a Web page where students present their initiatives, projects, and works online.
Design
The study followed a 2 x 2 quasi-experimental design, with hypertext format (graphical versus textual) and users (deaf vs. hearing) as independent variables. Additionally, students performed a picture and word categorization task (a measure of semantic memory organization), a verbal and spatial memory span, and a vocabulary test (Peabody Picture Vocabulary Test; Dunn, Stella, Pizzioli, & Tressoldi, 2000). The latter was only used for the deaf group.
Material and Tasks
Hypertext system and search task.
A hypertext supermarket was implemented using an HTML editor in two versions: graphical and verbal. In the graphical hypertext, nodes and links were represented by pictures of the goods (vegetables, fruits, domestic objects, etc.). In the verbal hypertext, nodes and links were represented by words solely (see Figure 1).
|
Half of the participants in each hearing status group were randomly administered the verbal hypertext and half were administered the graphical hypertext. The hypertext was composed of 62 nodes organized in four layers (this content organization can be seen in Appendix A). Each node contained three section links and two additional links on the top of the page, one for going back one step and another for going back to the main menu. The labels (verbal or graphical) that served as links in the main menu and second layer of nodes were obtained from Modler (1976)
. The third-layer links (verbal and graphical) were selected from the Dell'Acqua, Lotto, and Job (2000) database with the criteria of having high frequency, familiarity, and typicality (M = 2.16, 6.16, and 5.99, respectively, see Appendix B). Frequency refers to the frequency of occurrences of the printed pictures' names, familiarity refers to how familiar the object depicted in each picture was, and finally typicality refers to how typical each picture was within the corresponding category. The fourth layer of nodes of the hypertext system contains nonlinkable information about the supermarket products.
The main task of the users was to find 18 goods in the hypertext supermarket. Instructions were given both in written and oral forms to hearing participants. In the case of deaf participants, signed Italian was used in addition to the written text. Each target was presented individually in the format corresponding to the user experimental condition (picture or word) preceded by a textbox with the message "Find this good" (see Figure 2). As soon as users clicked on the bottom "Continue" of the textbox, the target appeared in the top right corner of the browser (where it remained visible during the trial) and the main menu of the hypertext was loaded. Users had 1 min to find each target. Once participants found that the target or the search time expired, a new textbox gave feedback about their performance and led them to the next search trial. The same order of target presentation was used in the graphical and verbal interface. Participants completed a training session supervised by the researcher before the experimental session. The hypertext task was administered and registered by means of a program written in Visual Basic (Salmerón et al., 2005).
|
The dependent variables of hypertext search were percentage of targets found, response time (total time to find the target from the main menu), and S or number of total nodes visited, which is considered a component of user disorientation (Smith, 1996
). In the search task, all targets were reachable by visiting three nodes, so it was assumed that the higher S was, the less direct the users path to find the target was, that is, the more disoriented they became.
Categorization test.
Replicating the experimental paradigm of Snodgrass and McCullough (1986), the participants were asked to categorize items (exemplars) as belonging to one of two categories. Thirty-two pairs of categorically related items (fruits, vegetables, and animals) were selected from the Dell'Acqua et al. (2000) corpus. The names of pictures varied from two to four syllables in length. Their mean frequency, familiarity, and typicality were 1.8 (range = 0.69 ± 2.94), 5.8 (range = 4.86 ± 6.60), and 5.4 (range = 3.40 ± 6.80), respectively (see Appendix C). Participants were tested in a 2 x 2 within-subject design with stimuli format (picture vs. word) and visual similarity (high similarity: fruit/vegetable vs. low similarity: fruit/animal) as factors. Therefore, each participant performed four blocks of eight trials of categorizations: picture-low similarity, picture-high similarity, word-low similarity, and word-high similarity. The order of presentation of the blocks was counterbalanced across participants, and the order of stimulus presentation within blocks was randomized for each condition. The items were presented twice in each condition. After a fixation point (+) presented for 500 ms, the target appeared in the center of the visual display for 1.5 s, followed by 1 s lighted blank field. Participants pressed the "d" key (red color), if the stimulus pertained to category "fruit," and "l" (green color), if it pertained to the categories "vegetable" or "animal". In order to familiarize participants with the stimuli, they were shown a list of the 10 different pictures and words they would encounter in each of the categories. Ten different practice items were selected from the categories of animals and fruits. The practice trials were presented at the beginning of each experimental block. Error rate and response times were recorded.
Span tests.
The "digit span test" (Wechsler, 1981) in direct and inverse order was administered to all the participants bimodally (oral and sign language). The total span score is the sum of items correctly repeated in the direct and inverse order.
As a "spatial span test," a computerized version of the Corsi Block-Tapping Task (Berch, Krikorian, & Huha, 1998; Kessels, van Zandvoort, Postma, Kappelle, & de Haan, 2000) was used. In the Corsi Block-Tapping Task, an increasing sequence of blocks (from three to nine blocks) arranged in a board was displayed on a computer monitor. After the presentation of each sequence, the individuals had to reproduce the sequence in the correct order. The total span score is the number of sequences correctly reproduced.
The trials of spatial and verbal span tasks were blocked by format (verbal or spatial), balanced, and preceded by practice trials.
Peabody picture vocabulary test.
This test of receptive vocabulary (Italian validation of Dunn et al. [2000]) measures vocabulary knowledge for Italian. It is composed of a set of 350 items, represented by pictures. The participants have to indicate which of four pictures corresponds to the word pronounced by the experimenter. Because we were interested in measuring deaf students' spoken language skills, the test was administered orally solely, without using signs. Bimodal communication (signed Italian) was only used for giving instructions. Participants' answers were, however, nonverbal, they were simply asked to point to the picture corresponding to the word pronounced.
General Procedure
Participants were tested individually in 2-day sessions in the presence of a hearing experimenter competent in both spoken Italian and LIS. For deaf participants, the instructions were administrated in their preferred modality (sign language or bimodal, that is, oral plus signed Italian). Students were informed of the general aim of the study: to investigate the effects of Web structures on information search tasks. They were assured that any kind of psychodiagnostic assessment of their intellectual level and other aspects of their personality was outside of the goals of the present research. After the general instructions, the experiment administered the tasks according to a counterbalanced order.
|
Results |
|---|
|
TOPFactors Contributing to Deaf...MethodResultsDiscussionReferences |
|---|
Hypertext Performance
In this section, we report the results for the information search task, making reference to percentage of targets found, response times, and disorientation (S, number of total nodes visited by the participant). Table 1 shows means and SDs of each parameter for each experimental condition.
|
Analyses of variance (ANOVAs) were conducted with interface format and hearing status as factors for each dependent variable. Results show that performance of deaf and hearing user was significantly different with respect to response time, F(1, 57) = 30.02, p < .001, MSE = 5.67, and total nodes visited, S: F(1, 57) = 12.69, p < .01, MSE = 0.2, but not in terms of percentage of targets found, F(1, 57) = 3.56, p < .06, MSE= 43.4. That is, most of deaf and hearing users found almost 100% of targets, although deaf students made more attempts and spent more time searching compared to hearing students. The effect of interface format was significant for disorientation, F(1, 57) = 4.69, p < .03, MSE = 0.23; students visited more pages in graphical (M = 4.9; SD = 0.5) than in textual (M = 4.6; SD = 0.4) interfaces. However, there were no effects of interface format, neither for percentage of target found, F(1, 57) = 0.12, p < .73, MSE = 43.35, nor for response time, F(1, 57) = 2.48, p < .12, MSE = 5.67. Finally, no interactions between interface format and hearing status emerged (targets found: F(1, 57) = 0.01, p < .93, MSE = 43.35; response time: F(1, 57) = 2.91, p < .09, MSE = 5.67; S: F(1, 57) = 0.34, p < .6, MSE = 0.23).
The differences between the two groups of users in response times might be potentially related to both longer latencies in decision making (i.e., longer times spent at each node) and number of nodes visited (e.g., students disorientation in performing the task). Therefore, in order to discriminate the effects of these two factors, we conducted a multiple analysis of variance (MANCOVA) with response times as dependent variable, interface format and hearing status as factors, and disorientation (S index) as a covariate. Our results showed that the advantage in response time of hearing over deaf users persists even when we control for the number of total nodes visited or disorientation, F(1, 56) = 14.28, p < .001, MSE = 4.03. Moreover, covarying disorientation, an effect of interface format appeared, F(1, 56) = 9.91, p < .01, MSE = 4.03: users in the graphical interfaces, M = 9.8 s (2.4) were faster that those in the verbal interface, M = 10.7 s (3.4).
The interaction between the type of user and interface format was not significant, F(1, 56) = 2.69, p < .1, MSE = 4.03; however, the analysis of simple effects showed that the advantage in response time of graphical interface was significant for deaf users, F(1, 56) = 1.66, p < .001, MSE = 4.03, and not for hearing users, F(1, 56) = 1.27, p < .3, MSE = 4.03 (see Figure 3). These results suggest that (a) disorientation does not entirely explain longer response times in the deaf students. That is, deaf students probably spend more time than hearing students processing information on each page; (b) consistent with our first prediction, deaf students take advantage of searching in a graphical hypertext when the target graphical links are represented by highly frequent, familiar, and typical pictures. However, even if the effect of interface format on users' response times seems to support the hypothesis of advantages of searching in a graphical hypertext, our results suggest more caution: students are faster in getting to the target when nodes are represented by pictures but they become more disorientated using pictures as well.
|
With the aim of evaluating the origin of the difference between users, we analyzed their differences in categorization tasks and working memory spans and correlated these scores with the hypertext performance.
Categorization, Working Memory Span, Vocabulary Knowledge, and Hypertext Retrieval Categorization Task
Mean correct reaction times (RTs) and error rates for each type of participant (hearing and deaf) in each experimental condition are shown in Table 2.
|
We performed two different ANOVAs with hearing status, similarity, and stimuli format as factors for each dependent variable: error rates and RT. Regarding error rates, results showed that deaf and hearing users were equally accurate in performing the categorization task: hearing status did not have a significant effect on the error rate, F(1, 59) = 0.03, p < .9, MSE = 6.9. On the other hand, the effects of stimuli format, F(1, 59) = 136.35, p < .001, MSE = 1.4, and similarity, F(1, 59) = 210.58, p < .001, MSE = 2.3, were significant, as well as the effect of the interaction stimuli format x similarity, F(1, 59) = 206.33, p < .001, MSE = 1.1. Picture categorization performance was affected more significantly by similarity than word categorization performance. That is, error rate was greater for pictures than for words when the categories had high similarity (fruit/vegetable condition), F(1, 59) = 220.6, p < .001, MSE = 1.9. However, the picture superiority effect did not appear in any condition. Finally, the interaction stimuli format x hearing status was also significant for error rate, F(1, 59) = 4.08, p < .045, MSE = 1.4. The difference between picture and word categorization (in favor of words) was greater for hearing than deaf users, F(1, 59) = 4.1, p < .048, MSE = 1.4.
Regarding RTs, the effect of type of user was significant, F(1, 59) = 8.04, p < .001, MSE = 284,366. Hearing participants were faster than deaf participants in the categorization task independent of the experimental condition (words or pictures). The effect of stimuli format was not significant, F(1, 59) = 2.80, p < .09, MSE = 49142.93. The effect of similarity was also significant for RTs, F(1, 59) = 25.91, p < .001, MSE = 34338.7, as low similarity trials were faster than high similarity trials. Additionally, there was a significant interaction between format and similarity, F(1, 59) = 7.39, p < .01, MSE = 22611.25. As Figure 4 shows, there was a picture superiority effect for the low similarity condition, F(1, 59) = 1.5, p < .001, MSE = 28873.8, that disappeared in the high similarity condition, F(1, 59) = 0.02, p < .89, MSE = 42880.41.
|
To explore whether the differences in the categorization skills among students might explain the effects we found in the hypertext search task, we performed a MANCOVA analysis with response times of the four categorization conditions as covariates, search time as the dependent variable, and hearing status and interface format as fixed factors. Contrary to our predictions, the results showed no differences with the ANOVA previously performed; that is, the difference between deaf and hearing students remains significant, F(1, 53) = 21.09, p < .001, MSE = 5.66, which suggests that the longer hypertext search time of deaf students is not related to their slower categorization of items.
The two hearing status groups obtained similar spatial span scores, F(1, 59) = 0.23, p < .63, MSE = 1.6 (hearing students: M = 5.32, SD = 1.39; deaf students: M = 5.17, SD = 1.12), but hearing users showed a greater verbal span (M = 15.98, SD = 4.53) than deaf users (M = 10.4, SD = 3.08), F(1, 59) = 31.33, p < .001, MSE = 15.09 (see distribution of frequencies in Table 3). As can be seen in Table 4, Pearson correlations between verbal and spatial spans and hypertext performance showed that for deaf students in graphical hypertext, spatial span correlated positively with percentage of target found and negatively with search response time. That is, the higher the deaf students spatial span, the more targets were found and the faster they completed the search task.
|
|
Surprisingly, Pearson correlations did not reveal any significant association between deaf students' receptive vocabulary (Peabody Picture Vocabulary Test [PPVT] scores) and hypertext search measures. In particular, the correlations between PPVT scores and percentage of targets found, response time, and number of different nodes visited were, respectively, r(15) = 0.15, p = .45; r(15) = –0.07, p = .71; and r(15) = 0.02, p = .91.
|
Discussion |
|---|
|
TOPFactors Contributing to Deaf...MethodResultsDiscussionReferences |
|---|
Failures in finding relevant information in the Web might be attributed to two general factors: (a) subject factors, that is, limitations associated with human information processing (e.g., memory limitations) and b) system factors, that is, the Web environment itself (Head, Archer, & Yuan, 2000
). The ascription of deaf students' problems of accessibility and information search via the Web to subject factors seems reasonably sound. Actually, deaf students' verbal memory span, semantic memory representations, linguistic, and reading ability are some of the possible explicative factors highlighted from past studies (Fajardo, 2005; Fajardo et al., 2006). However, considering these factors alone is misleading, and a complete explanation of deaf students' accessibility problems to Web information should take into account the interaction between subject factors and hypertext environment format. This was the primary goal of this study.
Our first hypothesis stated that in an information search task, graphical interface (hypertext with graphical links) would facilitate the performance of deaf users compared to a verbal interface (hypertext with verbal links) and reduce their disadvantages compared to hearing users. Partially supporting our prediction, highly frequent, familiar, and typical pictures led to faster hypertext information search in the case of deaf students, reducing their disadvantages compared to hearing students. In addition, regardless of interface format, accuracy differences between deaf and hearing users did not appear, contrary to what was found in Fajardo et al. (2006) with a not-normalized set of pictures and words. However, hearing users were faster and became less disorientated (they visited fewer nodes) than deaf users in both modalities of hypertext search task. Finally, all users in general became more disorientated in graphical than in verbal hypertext.
Regarding our main prediction, there are thus two primary results. On the one hand, comparable accuracy between the two groups of users seems to support our hypothesis of facilitative effect of highly frequent, familiar, and typical stimuli for deaf users. However, this comparability could be due not only to the use of frequent, familiar, and typical stimuli but also to the fact that the supermarket hypertext used in this study was simpler in terms of number of nodes and structure than the newspaper hypertext used by Fajardo et al. (2006). Consequently, this effect must be tested in more complex hypertext systems before its reliable generalization.
On the other hand, the higher disorientation in hypertext information search might reveal a lack of navigation and searching strategies on the part of deaf students, which affected their performance independently of the hypertext format (verbal or graphical). According to Head et al. (2000), proficient Web navigators extend to hypertexts a four-phase information search process useful for searching information from texts. They (a) formulate their goal or identify their target or information needs, (b) actively conduct the search on the text and/or hypertext, (c) examine the retrieved information to determine its relevance, and (d) review the search results and refine the objective. Our deaf students apparent disorientation led us to suspect a kind of trial and error strategy used in solving the task more than the planned strategy suggested by Head et al. Actually, the findings of Namatame and Kitajima (2005) could support such a hypothesis about searching strategies differences as they found that deaf participants used less semantic scan patterns (measure extracted from eye movement logs) of Web page content comparing to hearing participants. However, in addition to this isolated evidence, further research is needed to conclude that participants searched randomly or simply followed an "assess-all" decision strategy (Brumby & Howes, in press), which be would less efficient in terms of number of nodes visited relative to a more semantic strategy.
Our second hypothesis stated that categorization task scores could explain the difference between deaf and hearing users in hypertext performance. However, whereas deaf and hearing students did differ in the speed with which they classified picture and words (deaf individuals were slower than hearing individuals, overall), the analysis of covariance shows that such differences in categorization cannot explain the search disparities among students. In the case of deaf people, converging with the data on disorientation, the lack of relation between categorization task and hypertext information search might suggest that these students are using a strategy based on "assess-all items" or "visual search"and "match" instead of semantic categorization processing of the information. This hypothesis would agree with the conclusions of Marschark et al. (2004), suggesting that deaf students may evidence lesser spontaneous use of categorical information in problem-solving tasks, such as the hypertext search task, and could have important implications for education and teaching of the Internet and computer literacy. Nevertheless, the no-relationship between the categorization task and the hypertext search task also appears for hearing students. This result suggests that perhaps the categorization test used in this experiment was not sufficiently sensitive or valid to evaluate the kind of categorical knowledge involved in hypertext search.
On the other hand, the results from the single-stimulus classification task are relevant by themselves because, to our knowledge, categorization abilities of deaf and hearing users with pictures and words had not been contrasted directly before. Similarly to what emerged for the hypertext performance, the results show that word and picture categorization differs between groups of users in efficiency but not in accuracy. Furthermore, the categorization task data validate the RT data of Snodgrass and McCullough (1986), that is, there is a picture superiority effect that disappears when similarity between categories increases. Similarity does not affect word categorization; consequently its effect would be visual and not semantic.
Regarding our third prediction and contrary to previous findings (Larson & Czerwinski, 1998), users verbal span did not correlate significantly with hypertext performance. However, deaf users' spatial span and graphical hypertext's accuracy and efficiency did correlate with each other, that is, the higher spatial span, the faster deaf users were and the more targets they found searching with graphical hyperlinks. Once more, the spatial span correlation could be seen as supporting the hypothesis of strategic differences between the hearing status groups; specifically, these data suggest that deaf students could be using a search strategy based more on visuospatial information than on verbal or semantic one. The absence of equivalent correlations between verbal span and response times in verbal hypertext for hearing users could be attributed to the lack of variability frequencies of users in low-score ranges of the span test (see distribution of frequencies in Table 3). Approximately 95% of them obtained a high verbal span, sufficient to accomplish the verbal demands of the task.
Another interesting result regarding span tests is that, contrary to previous findings of Wilson et al. (1997), hearing and deaf signers showed no difference in spatial span measured by means of the Corsi Block-Tapping Task. As these authors suggest, spatial memory advantage of deaf individual would be due to the early use of spatial locations and relationships in sign language for coding information. However, our deaf participants were more heterogeneous in terms of age and early exposure to sign language compared to participants in the study of Wilson et al. This could explain the absence of spatial memory advantage for the deaf signer participants. Therefore, this variable is worthy of further consideration in future research.
Our final hypothesis predicted a correlation between deaf students' spoken language vocabulary and verbal hypertext performance, which, however, did not emerge. This finding might be alternatively explained in two ways. On the one hand, we assessed oral receptive vocabulary in deaf signers with a relatively poor exposure to oral language. The vocabulary measure we obtained in this way might reflect the deaf students' lipreading skills more than their absolute vocabulary knowledge, better assessed by Italian Sign Language. That is, we did not find correlations between the students' oral vocabulary and verbal hypertext performance because we did not assess, in fact, verbal vocabulary but "visual" receptive skills. An alternative explanation of the correlation lack is that longer search times in hypertext are related to more complex linguistic processes (reading comprehension or fluency), as Fajardo (2005) found, rather than to simple vocabulary knowledge.
In general, our results converge in indicating that deaf users' accessibility to hypertext information is given by a complex relationship between the subject and the medium factors, where some variables modulate and even mask the expression of others. This means that simple responses to deaf students' needs, such as simplifying the information format or translating it in a graphical form, might only partially and superficially facilitate their information search, leaving most of the problem unsolved and unexplored. At the same time, there is need of further investigation of the factors affecting the students performance: some subject factors (i.e., procedural and strategic knowledge, information processing efficiency) left out of the focus of this and previous studies might nevertheless be potentially relevant for the task.
As traditional teaching and educational techniques have been largely insufficient, there is currently a general request for interventions improving deaf students' literacy and learning experience. The modern technological era has delivered to teachers and educators new and powerful instruments, such as hypertexts and hypermedia, whose use needs to be more thoroughly investigated and tested. These instruments, multilinguistic and multimedia in essence, should be particularly suitable to deaf students' learning needs. They enlarge the students' communicating and learning to environments beyond the classes, schools, and institutes, thus being crucial vehicles of knowledge construction. However, despite their potentialities, these tools did not initially result more effective than traditional textbooks.
In exploring new paths for educative interventions, we need to study in greater depth how these instruments shape deaf students' learning processes and benefit them. Our results do not provide many specifics with regard to this objective but do nonetheless generate relevant questions regarding the relationships that characterize the processing of complex structures of knowledge involved in Web interaction. They clearly show that Web environmental and subject factors are associated in a very articulated and complex way in explaining deaf students' difficulties in Web information search. Therefore, only integrated interventions, centered on the subjects characteristics (increasing the learners' linguistic and strategic knowledge) and on the learning material (restructuring the text/hypertext), would probably produce significant improvements and benefits.
Appendix A Hierarchical structure of contents in the Supermarket Hypertext distributed along 3 layers of nodes. Each concept in the first, second and third layer was represented by clickable words or graphics which conducted to the next level of the hierarchy.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Appendix B Normative data of the pictures and words obtained from Dell'Acqua, Lotto and Job's database (2000) and used as links in the experimental hypertext.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Note. CAT = category (FRU = fruit, FLO = flowers, REC = recipients, INS = instruments, VEG = vegetables), FRQ = frequency of use, FAM = familiarity, TYP = typicality, and AoA = age of acquisition.
Appendix C Normative data of the items selected from the Dell'Aqua et al. (2000) set for the Categorization Task.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Note. CAT = category (FRU = fruit, VEG = vegetables, MAM = mammals), FRQ = frequency of use, FAM = familiarity, and TYP = typicality.
|
References |
|---|
|
TOPFactors Contributing to Deaf...MethodResultsDiscussionReferences |
|---|
-
Berch D, Krikorian R, Huha E. The Corsi Block-Tapping task: Methodological and theoretical considerations. Brain and Cognition (1998) 38:317–338.[CrossRef][Web of Science][Medline]
Blankenberger S, Hahn K. Effects of icon-design on human-computer interaction. International Journal of Man-Machine Studies (1991) 35:363–377.[CrossRef]
Brumby DP, Howes A. Strategies for guiding interactive search: An empirical investigation into the consequences of label relevance for assessment and selection. Human-Computer Interaction (in press).
Chincotta M, Chincotta D. Digit span, articulatory suppression and the deaf: A study of the Honk Kong Chinese. American Annals of the Deaf (1996) 141:252–257.[Web of Science][Medline]
Dell'Acqua R, Lotto L, Job R. Naming times and standardized norms for the Italian PD/DPSS set of 266 pictures: Direct comparisons with American, English, French, and Spanish published databases. Behavior, Research Methods, Instruments, and Computers (2000) 32:588–615.[Web of Science][Medline]
Dillon A, Song M. An empirical comparison of the usability for novice and expert searchers of a textual and a graphic interface to an art-resource database. Journal of Digital Information (1997) 1(1). Retrieved December 1, 2006, from http://dlist.sir.arizona.edu/1299/.
Dunn L, Stella G, Pizzioli C, Tressoldi P. Peabody: Test di vocabolario recettivo: manuale (2000) Torino, Italy: Omega.
Fabbretti D, Volterra V, Pontecorvo C. Written language abilities in Deaf Italians. Journal of Deaf Studies and Deaf Education (1998) 3:231–244.
Fajardo I. Cognitive accesibility to hypertext systems. The role of verbal and visuospatial abilities of deaf and hearing users in information retrieval (2005) Unpublished Ph.D. Thesis, University of Granada, Granada, Spain.
Fajardo I, Cañas JJ, Salmerón L, Abascal J. Improving deaf users' accessibility in hypertext information retrieval: Are graphical interfaces useful for them? Behaviour and Information Technology (2006) 25:455–467.[CrossRef]
Flaherty M. Memory in the deaf: A cross cultural study in English and Japanese. American Annals of the Deaf (2000) 145:237–244.[Web of Science][Medline]
Goldin-Meadow S, Mayberry R. How do profoundly deaf children learn to read? Learning Disabilities Research and Practice (2001) 16:221–228.
Harris M, Moreno C. Deaf children's use of phonological coding: Evidence from reading, spelling, and working memory. Journal of Deaf Studies Deaf Education (2004) 9:253–268.[CrossRef]
Head M, Archer N, Yuan Y. World Wide Web navigation aid. International Journal of Human-Computer Studies (2000) 53:301–330.[CrossRef]
Job R, Rumiati R, Lotto L. The picture superiority effect in categorization: Visual or semantic? Journal of Experimental Psychology: Learning, Memory and Cognition (1992) 18:1019–1028.[CrossRef][Web of Science][Medline]
Kelly L. The interaction of syntactic competence and vocabulary during reading by deaf students. Journal of Deaf Studies and Deaf Education (1996) 1:75–90.
Kessels R, van Zandvoort M, Postma A, Kappelle L, de Haan E. The Corsi Block-Tapping task: Standardization and normative data. Applied Neuropsychology (2000) 7:252–258.[CrossRef][Medline]
Kitajima M, Blackmon MH, Polson PGA. Comprehension-based model of Web navigation and its application to Web usability analysis. In: People and computers XIV—Usability or else!—McDonald S, Waern Y, Cockton G, eds. (2000) London: Springer. 357–373.
Larson K, Czerwinski M. Web page design: Implication of memory, structure, and scent for information retrieval. In: Proceedings of CHI'98 Human Factors in Computing Systems Structure—Karat C-M, Lund A, Coutaz J, Karat J, eds. (1998) New York: ACM Press. 25–32.
Lee MJ, Tedder MC. Introducing expanding hypertext based on working memory capacity and the feeling of disorientation: Tailored communication through effective hypertext design. Journal of Educational Computing Research (2004) 30:171–195.[CrossRef]
Loeterman M, Paul PV, Donahue S. Reading and deaf children. Reading Online (2002) 5(6). Retrieved March 5, 2007, from http://www.readingonline.org/articles/loeterman/index.html.
Logan K, Maybery M, Fletcher J. The short-term memory of profoundly deaf people for words, signs and abstract spatial stimuli. Applied Cognitive Psychology (1996) 10:105–119.[CrossRef][Web of Science]
Lotto L, Job R, Rumiati R. Visual effects in picture and word categorization. Memory and Cognition (1999) 27:674–684.[Web of Science][Medline]
Marschark M, Convertino C, McEvoy C, Masteller A. Organization and use of the mental lexicon by deaf and hearing individuals. American Annals of the Deaf (2004) 149:51–61.[CrossRef][Web of Science][Medline]
Marschark M, De Beni R, Polazzo MG, Cornoldi C. Deaf and hearing-impaired adolescents' memory for concrete and abstract prose: Effects of relational and distinctive information. American Annals of the Deaf (1993) 138:31–39.[Web of Science][Medline]
Marschark M, Everhart V. Problem-solving by deaf and hearing students: Twenty questions. Deafness and Education International (1999) 13(1):87–102.
Marschark M, Harris M. Success and failure in learning to read: The special case of deaf children. In: Children's reading comprehension disabilities: Processes and intervention—Oakhill J, Cornoldi C, eds. (1996) Mahwah, NJ: Erlbaum. 279–300.
Modler R. Handbook of pictorical symbols. 3250 examples from international sources (1976) New York: Dover Publication.
Musselman C. How do children who can't hear learn to read an alphabetic script? A review of the literature on reading and deafness. Journal of Deaf Studies and Deaf Education (2000) 5:9–31.
Namatame M, Kitajima M. Differences in Web interaction styles of hard-of-hearing and hearing persons. Paper presented at the HCI International Conference, Las Vegas, Nevada. (2005, July).
Namatame M, Nishizaki Y, Kitajima M. Improving usability of Web pages for hard-of-hearing persons: An investigation of the utility of pictograms. (2007, July) Poster session presented at the HCI International Conference, Beijing, China.
Nelson DL, Reed US, Walling JR. Picture superiority effect. Journal of Experimental Psychology: Human Learning and Memory (1976) 2:523–528.[CrossRef][Web of Science]
Norman K. The psychology of menu selection: Designing cognitive control of the human/computer interface (1991) Norwood, NJ: Ablex.
Paivio A. Images, propositions and knowledge. In: Images, perception and knowledge—Nicholas JM, ed. (1977) Dordrecht, The Netherlands: Reidel. 47–71.
Paivio A. Mental representations (1986) New York: Oxford University Press.
Paivio A. Dual-coding theory: Retrospect and current status. Canadian Journal of Psychology (1991) 45:255–287.[Web of Science]
Raven JC, Court JH, Raven J. Manual for the Raven's progressive matrices and vocabulary scales (1991) London: Lewis.
Salmerón L, Cañas JJ, Fajardo I. Are expert users always better searchers? Interaction of expertise and semantic grouping in hypertext search tasks. Behaviour and Information Technology (2005) 24:471–475.[CrossRef]
Smith P. Towards a practical measure of hypertext usability. Interacting with Computer (1996) 8:365–381.[CrossRef]
Snodgrass JG, McCullough B. The role of visual similarity in picture categorization. Journal of Experimental Psychology: Learning, Memory and Cognition (1986) 12:147–154.[CrossRef][Web of Science][Medline]
Vicente KJ, Hayes BC, Williges RC. Individual differences in computer-based information retrieval. In: Ergonomics and human factors—Mark LS, Warm JS, Huston RL, eds. (1987) New York: Springer. 225–229.
Wauters LN, van Bon WH, Tellings AE, van Leeuwe JF. In search of factors in deaf and hearing children's reading comprehension. American Annals of the Deaf (2006) 151:371–380.[CrossRef][Web of Science][Medline]
Wechsler D. Manual for the Wechsler adult intelligence scale revised (1981) New York: Psychological Corporation.
Wilson M, Bettger JG, Niculae I, Klima ES. Working memory for sign language: A window into the architecture of the working memory system. Journal of Deaf Studies and Deaf Education (1997) 2:121–130.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||