Abstract: A new approach to teaching web source evaluation is necessary for an internet that is increasingly littered with sources of questionable merit and motivation. Initially pioneered by K–12 educational specialists, the journalistic model avoids the cognitive duality of the checklist and a reliance on opaque terms and concepts. Instead, it recommends students apply the six journalistic questions of what, who, where, when, why, and how when evaluating freely available web sources. This approach outlines an evaluative procedure that is open-ended, discursive, and analytic in nature as opposed to formulaic and binaristic. It also requires students to consider both the context of the information need and a source’s potential use as central to its evaluation.
Keywords: web source evaluation, information literacy pedagogy, library instruction, higher education, secondary education, primary education, lifelong learning
Source: Communications in Information Literacy. Volume 14, Issue 2, Article 6. 12-2020. (https://doi.org/10.15760/comminfolit.2020.14.2.6) This open access Research Article is distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0).
_______________________________________________________________________________________________
The Journalistic Approach: Evaluating Web Sources in an Age of Mass Disinformation
Victoria Elmwood, Loyola University New Orleans
As digital media continues to be the primary means of distributing and accessing information in the twenty-first century, the urgent need to cultivate information literacy awareness and habits of consumption and use persists. This imperative to educate users at multiple levels subtends both the academy’s standards for what constitutes knowledge and, equally important, the urgent need for civic responsibility in a functioning democracy. The power of social media’s influence, the proliferation online of cleverly disguised paid content as well as mis- and disinformation, and the ubiquity of data mining and algorithms as determinants of how information and goods are distributed—all of these developments suggest an urgent need for effective, accessible methods of information literacy instruction. Though instructors have been teaching digital information literacy since the onset of widespread household internet use, the ways in which they teach it have become more sophisticated (Ostenson, 2014), and the need for pedagogical methods that respond to an evolving infosphere persists. This article responds to that need by proposing a versatile, simple instructional method that can be used on a broad range of audiences, from grade school children to lifelong learners.
Much research on information literacy covers users’ perceptions of their own abilities, real-life user practices, and pedagogical approaches used by classroom instructors. We know, for example, that students consistently overestimate their own ability to discriminate between high-quality information and low-quality information (Hinchliffe et al., 2018; Library Journal, 2017; Mason et al., 2014; and Metzger, 2007). Hinchliffe et al. (2018) isolated two specific beliefs of first-year students that suggest an urgent need for instruction on free web source evaluation—namely that Google alone is a sufficient research tool and that sources on the open web are appropriate for college-level work. Khalid Mahmood’s (2016) exhaustive statistical analysis of 53 scholarly studies demonstrated novice overconfidence, a pervasive pattern that may contribute to dissuading students from seeking librarian input in the research process. Even secondary-level instructors have demonstrated overconfidence in their ability to consistently identify high-quality, freely available information (Andreassen and Bråten, 2013).
If older research were not disheartening enough, even the more recent research gives evidence that students need guidance to help them develop an approach to evaluation that does not consist of simply identifying or locating surface-level tokens. To wit, instructors should be helping students to develop a more sophisticated awareness of key indices of reliability (Mason et al., 2014). Most recently, the Stanford History Education Group (SHEG) found that 52% of high school students tended not to take the extra step of tracking down the initial context of a misleadingly labeled video of voter fraud (Breakstone et al., 2019). Additionally, two-thirds of the students did not recognize the term “sponsored content” as a euphemism for “paid advertising,” and nearly all students (96%) did not recognize the conflict of interest inherent in a climate change website being sponsored by major stakeholders in the fossil fuel industry.
On the bright side, however, approaches to teaching users how to find quality information fit for their purposes have also been developing over time, though perhaps not quickly enough. One significant challenge in devising effective instruction methodologies has been the need to balance scarcity of instruction time and of users’ time, on one hand, with the need for complex, thoroughgoing analysis of sources on the other. This balancing act has served as a key impetus for the ongoing development of how best to teach web source evaluation techniques, and the pedagogical methodology for which this article advocates responds to those major needs, among others. Though this method cannot solve the problem of balancing the competing demands of time and thoroughness, the approach outlined below makes suggestions for ways to mitigate this challenge.
A Brief History of Evaluation
The earliest models for teaching web source evaluation can largely be traced to Jim Kapoun’s (1998) approach, which provided a checklist-like set of items or traits that would indicate the quality of information on a website. Though the amount of information on the internet over twenty years ago pales in comparison to the amount it contains today, the amount that was then popularly accessible is still significant. A more important difference between now and then was the amount of technological knowledge and skills necessary to maintain an online presence, a barrier that made it easier to root out less legitimate sources. However, the quality of a source is not necessarily indicated by the quality of the design and maintenance of the site on which it is hosted. For instance, early checklists prompted users to take notice of details such as domain name, the presence of dead links, a clearly identifiable webmaster with contact information, and a balance between image and text content (see Judd et al., 2006). Thus, the earliest approaches to web evaluation privileged simple surface traits that tended to address the quality of the site itself rather than its content. While these traits are effective at helping judge the structural quality, currency, and maintenance of a site, they have less to do with showing users how to evaluate the quality and provenance of a source (Meola, 2004).
Reacting to the widespread adoption of checklist-oriented approaches, Mark Meola (2004) famously articulated the critique that evaluation should involve deeper engagement with the source’s content rather than its surface features. Meola urged, among other things, that users seek corroboration of information in one source by seeking out that same information in other sources. Though Meola’s critique was met with widespread approval, the tendency toward scrutinizing less relevant traits of a web source can be found in pedagogical approaches to evaluation as late as 2014. Educational specialists such as Ostenson (2014) have continued to advocate the application of a checklist for sources destined to be used in only minor capacities for a given project. Taking a somewhat different tack, Miriam Metzger (2007) recommended gauging student motivation and using approaches of varying intensity with students according to the individual’s level of motivation. Thus, recommending abbreviated evaluation processes for sources in some cases has been a pragmatic, time-saving strategy suggested by some researchers.
Nevertheless, Kapoun’s checklist method provided the foundation for more meaningful evaluative practices in that later models still mentioned some traits considered in those early days: the author’s identity and affiliations, the purpose motivating publication, and any kind of advertising featured on the site. However, as the internet became more democratic with the development of content management systems such as WordPress, social media, and highly accessible domain-hosting options, instructors sought to devise more qualitative perspectives on evaluation, ushering in the acronym paradigm of website evaluation.
Evolved Criteria for a Developing Information Landscape (early 2000s)
The now-pervasive CRAAP test was first devised by Sarah Blakeslee in cooperation with a team of librarian instructors at California State University, Chico (Chico State) in 2004, and it named specific, detailed criteria bundled as a mnemonic of five traits that are key indicators of informational integrity and usability. By now, these qualities should be a familiar litany to any information literacy instructor: currency, reliability, authority, accuracy, and purpose. Though the CRAAP test outlines a well-defined set of relevant attributes, it is still a checklist, a format that can hamper deep qualitative evaluation due to the yes/no nature of possible responses (Ostenson, 2014). Asking students to simply identify the absence or presence of a quality shuts down deeper inquiry and tends to limit the user’s cognitive tasks to the lower order thinking skills. Notably, the current page that Meriam Library at Chico State maintains for the CRAAP test acknowledges the model’s limitations. It cautions users against solely relying on a checklist-based approach to website evaluation, even when the items on that checklist are themselves sophisticated criteria. Though the test’s own creators acknowledge its most evident shortcomings, the CRAAP test is still a significant improvement. The ease with which a succinct yet sophisticated checklist can be applied constitutes an obvious benefit, as many checklists had become too byzantine and lengthy to be of practical use. Additionally, its introduction to students of complex concepts such as authority and accuracy is also a noteworthy improvement, at least when students successfully grasp the meaning of these concepts.
At the same time, the nature of the traits named in the CRAAP test is potentially as limiting as the earlier checklist approach. While the meanings of currency and purpose may be clear, concepts such as reliability and accuracy are not immediately transparent, especially to novices. For instance, isn’t reliability the end assessment that the user is trying to establish more generally rather than a single facet? And accuracy can mean very different things depending on discipline or field. Furthermore, authority can be a thorny, context-specific quality that may be difficult to establish objectively without considering a source’s proposed use in a given project. All these potential gray areas suggested a need for further development in the ongoing evolution of web evaluation pedagogy. In particular, newer methods would need to become more qualitative in nature and the destined (or possible) use of a source would need to be integrated into the evaluation process.
Nimble, Versatile, and User-Friendly (late 2000s–onward)
The current concern for emphasizing process over outcome has its roots in earlier critiques. Judd et al. (2006), for instance, prioritized both context and process as objectives for the evaluation process. Initially skeptical about students’ ability to complete such complex cognitive tasks, Metzger (2007) finally recommended approaches with different levels of intensity and complexity for more motivated users and less motivated users. Candace Dahl (2009) recommended Meola’s paradigm as a response to the widespread accessibility and consequent use of non-peer reviewed sources. To this she also added emphasis on the need for students to figure into their evaluation process the context of a source’s use, allowing for the possibility that a non-academic source might be acceptable for use in some academic contexts.
The 2010s have seen instructors and researchers continuing these calls for a more holistic approach, with more recent critics linking process-oriented, qualitative evaluation practices to the development of critical thinking skills (Cmor & Li, 2012; Fahey et al., 2011; Seeber, 2015). Jennifer Fielding (2019) recently outlined an approach called “lateral reading,” a term first coined in Wineburg and McGrew’s (2017) work through SHEG that referred to using parallel sources to investigate the background of the source in question. Kevin Seeber (2015) stressed the importance of arming students with the ability to think independently, asserting that “[i]nstruction must be focused on fostering critical thinking skills, rather than how to perform tasks, and assessment must be qualitative in nature” (p. 19). This attitude reflects the values of the Association of College and Research Libraries (ACRL, 2015) Framework for teaching information literacy. Both Seeber (2015) and Ostenson (2014), in particular, have emphasized the importance of a source’s format (or genre) as essential for teaching students to think meaningfully about the processes behind that source’s publication online.
At the same time, researchers have still maintained the need for a simple, easy-to-apply method (Dahl, 2009; Metzger, 2007). Some have anticipated and responded to these two seemingly irreconcilable demands by devising acronymic evaluation models that combine a step-by-step procedural approach (reminiscent of a checklist) while also fostering more thoroughgoing, complex analysis of a source. It is to one such model that this article now turns.
The Journalistic Approach
The roots of what is herein referred to as the journalistic (or investigative) approach can be found in work by Zhang et al. (2011) and Benjes-Small et al. (2013). Its earliest incarnation, however, is traceable to educational technologist Kathy Schrock’s (2001) model using the classic questions of the professional journalist (what, who, when, where, why, and how). Schrock’s journalistic questions approach, which began development even earlier than Chico State’s CRAAP test, suggested a different model for teaching (and practicing) web evaluation. (It is difficult to date Schrock’s schema as the copyright date given on the handout currently available online gives a fairly broad date range for the resource: 2001–2018 [Schrock, n.d.].) This approach took five of the six journalistic questions (only the how question is left out) as the major cues that users should rely on to guide their analysis of a site. This model is significant for its focus on the analysis of a source rather than the more simplistic good/bad evaluative judgement reached by applying a mnemonic. Schrock’s version of the investigative model included several related questions under each of the five headings, which may be difficult for students to keep track of in the absence of a handout. However, this model is noteworthy for the kinds of questions it had students asking, questions that are not yes/no oriented but rather more open ended.
Working in the field of educational psychology, Zhang et al. (2011) also instrumentalized the journalistic questions. They only included the questions of who, why, and when, but they also gestured toward the how question by urging students to consider how they are going to use the source. Benjes-Small et al. (2013) used cognitive development theory as inspiration for their approach, which (like Schrock’s approach) had students deploying five out of the six journalistic questions. Recognizing that early college students often ascribe to a dualistic right/wrong view of information, they led students to identify traits of prescribed “good” and “bad” sites before asking the students to assess a third site with a more analytical lens.
Somewhat different from Benjes-Small et al.’s (2013) piece, the remainder of this article will present a more detailed set of possible issues behind each journalistic question, mapping the skills used in considering each question to both the ACRL Framework and Bloom’s revised taxonomy (Anderson & Krathwohl, 2001). By mapping the journalistic approach to existing sets of widely recognized educational objectives used in higher education and in the K–12 environment, respectively, this article seeks to facilitate the journalistic approach’s widespread adoption. This article also adds to the work of Schrock (2001) and Benjes-Small et al. in considering two additional facets related to the how question, as well as the aforementioned extended discussion of possible considerations behind each of the journalistic questions.
The Model
In addition to invoking many of the ACRL’s information literacy frames, the investigative approach also engages the 2001 revision of Bloom’s taxonomy, particularly the cognitive tasks of applying, analyzing, and evaluating (the higher order thinking skills, or HOTS). In asking students to apply a model where they investigate different facets of a source (Analyze) and determine its appropriateness for a project (Evaluate), this approach also requires students to exercise both lower and higher order thinking skills. Its focus on qualitative analysis and avoidance of simple yes/no questions asks for a greater level of engagement but also provides a mildly structured procedure and mnemonic that can be used at all educational levels. Below, the questions and issues involved in each facet are broken down, with the ACRL frames and Bloom’s skills most likely to be invoked for that facet.
It should also be said that one enduring difficulty of teaching web source evaluation is the preponderance of different types or genres of sources. In addition to the traditional newspaper article, book review, or blog, students will also come across white papers, podcasts, videos, slide shows, and data sets, among other formats. Many may be encountering these in an academic context for the first time, so instructors should seek out a variety of source types to use when introducing how to go about answering each of the journalistic questions. Using a data set to model how to ascertain who is behind a source, for example, can not only teach students about considering professional credentials but it can also be used to complicate their notions of what constitutes a source’s content, not to mention also possibly complicating their existing notions about authorship. Moreover, that same data set may be useful in getting students to discuss the importance of currency relative to the model’s When question.
Using a small number of actual web sources to have students practice applying the journalistic questions can emphasize the practical nature of web source evaluation while also exposing them to the variety of informational genres or formats in use. Furthermore, to give students more time to process new types of sources, instructors may consider implementing flipped classroom techniques as part of their instruction. Distributing an explanation of the journalistic questions as homework before instruction takes place can allow extra time for students to explore the basic concepts behind the evaluative model before they come to class. Instruction time, then, could be spent having students apply the basic principles behind each facet to a set of sources that the instructor chooses to help introduce students to a diverse set of source types. Instructors can be even more effective in addressing these major obstacles related to source types and time available by using the lesson plan’s format as well as its content.
What?
For the most part, the order in which the user applies each of the journalistic questions is not important. However, beginning the analysis by asking what kind of source something is constitutes a more effective approach because the source’s type or genre informs the remainder of the assessment to a great extent. Knowing whether an item is an encyclopedia entry, a letter to the editor, a narrated slide show, or a podcast, for instance, often tells a user what the item can reasonably be expected to supply for its readership as well as who might be included in that readership. Considering the different capabilities and limitations of the source type asks students to also think about the purpose of its creation and appropriate uses of it in their own work. More fundamentally, students must develop a greater awareness of the many different types of sources. Therefore, a key challenge facing instructors is that students will struggle to identify and describe unfamiliar source types.
The ACRL frame of “Information Creation as a Process” intersects most directly with this facet, as understanding a taxonomy of source types presupposes a basic awareness of the different types in use. “Research as Inquiry” can also be invoked as students realize that background sources are as crucial to the research process as the sources toward which those background items often point them, regardless of what ends up in the final bibliography. Finally, considering the relative appropriateness of source types asks students to engage in “Searching as Strategic Exploration.”
While only the lower level thinking skills of Bloom’s taxonomy (Understand and Apply) are likely to be activated by considering a source’s type, this task is the first step of a larger analytical procedure that invokes the most basic of the higher order thinking skills (HOTS), Analyze. Indeed, the investigative questions process itself constitutes a vigorous foray into two of the three HOTS (Analyze and Evaluate). The skill at the top of the taxonomic pyramid—Create—correlates with the objective of a research project, namely gathering and advancing existing ideas and information on a topic.
Who?
Though previous evaluative methodologies emphasize authority, asking students to flesh out the identity of the speaker is a more productive approach. Authority implies a neat positive or negative evaluation of a source’s author but tends to encourage a reductive focus on the absence or presence of specific academic or professional credentials and work experience. The aim here is not to discourage students from ascertaining these important details about an author. Rather, it is to encourage asking the question of who as constituting a more thorough assessment of that author’s identity and their relationship to their subject matter. Students are certainly still meant to consider credentials and experience, but they are given more latitude to consider how other factors relate to the content and purpose of a source. For instance, an author featured on a website with content about the medical ethics of cloning who has a clearly identifiable theological affiliation may seem to present a limited viewpoint. However, students are more likely to see greater nuance if they are pushed to also discover that the same author holds an M.D. or chairs an organization dedicated to medical ethics. To add another wrinkle, sources with multiple authors or interviewees—for example, Ken Burns’ 8-part series on the Vietnam War—will require students to make an extra effort to find information about more than one person or organization. The greater potential for multiple voices to be speaking in freely available digital resources can make this seemingly straightforward task a more demanding one. By asking learners to get to know authors more fully (as well any organizations that fund their work), we get them to keep a specific speaker in mind as they read the source instead of simply granting or not granting authority to that speaker.
The most obvious ACRL frame invoked here is “Authority Is Constructed and Contextual”; however, students will also develop a greater understanding of “Scholarship as Conversation” when they scrutinize the background of an author, namely that author’s place within communities that foster professional discourse. Bloom’s HOTS exercised here include Analyze and Evaluate; students analyze the different facets of an author or editor’s identity to evaluate their relationship to the subject matter they are covering.
When?
Ascertaining the currency of a source seems to be one of the easier facets for readers to determine; however, the extent to which online sources can be revised or added to can complicate matters. Equally possible, students may run into sources that bear no date, forcing them to find ways to estimate the relative age of a source, as is often the case with undated PDFs. Specific to the online environment is the presence of reader comments, which help to establish an upper boundary for publication date while also giving the student an awareness of a community of readers discussing a piece rather than viewing it as hermetically isolated from other texts.
Contrastingly, the comments section can also serve to confuse novices about the original date of the source that is being commented on. Commentary, however, invokes the frame of “Scholarship as Conversation,” emphasizing that sources are generally not created in a vacuum. Instructors may also want to prompt students to consider the larger historical context of the world in which the source intervenes as well as the publication date’s proximity to major historical events. For instance, a blog entry that appeared in the days after 9/11 should be evaluated not just in terms of its age but also for its publication in chronological relation to key world historical events. Thus, where relevant, instructors may also use this question as an occasion for introducing a key distinction between primary and secondary sources.
“Information Has Value” is also invoked here. As a result of considering the When question, students further develop their awareness of the value of more current information over less current information or of the value of information created in proximity to a certain event. In considering reader comments, students may engage in analysis of conversation around a piece of scholarship, invoking the Bloom’s higher order thinking skill of Analyze.
Where?
The Where question serves to underscore the related issues of audience and purpose while also foregrounding the social context of the publication process. By identifying an organization’s mission, its board of directors, and any editors involved in the publication process, students catch on quickly that there are multiple interests undergirding the publication and maintenance of freely available web sources. The related concern of audience can help students manage frustration when reading a source written for readers with significant levels of specialization. If a sophomore in college undertakes the task of reading an article in an open access journal that they found on PubMed anticipating that they will not understand parts of it, they may be more likely to have reasonable expectations about how useful that source may be for them.
Increased awareness about the nature of a specific publication venue can also help a user prioritize which sources to include or exclude from a research project, depending on that project’s focus. For instance, a student’s exploratory project on the nature of immigration law in the U.S. after 2009 might not need to focus too intensely on technical treatments of the subject for which the main audience is highly specialized readers. Instead, the student might make an informed choice to integrate sources more appropriate to the goals of their project while still ascertaining that those sources come from sufficiently trustworthy publications and authors. Opting to use information from the website of an organization dedicated to helping immigrants navigate the legal system rather than information from a legal journal article could be a good choice in this instance.
Frames that come most closely into play with the Where question include “Searching as Strategic Exploration” and “Information Has Value.” Students explore the organizations facilitating a source’s publication and must determine the value of a source relative to their project’s needs. Considering this question may even help push students to further clarify the scope and nature of their projects. Furthermore, thinking about where a source comes from can create an awareness in students of the bibliographic information necessary to document their sources appropriately. The lower order Bloom’s skill of Understand is likely to be activated here, but students will also use the Analyze skill as they consider both the audience appropriateness and the composition and motivations of the organizations that make certain sources freely available online.
Why?
The purposes for distributing a source can be varied, and often the place of publication (Where) and type of source (What) provide significant context clues to this. Thus, it is best to leave the Why question until later in this analytical process. In analyzing the Why of a source, learners should consider the piece’s authors as well as its publishers. Some students may seek more existential reasons for a source’s composition and availability, but they should also be encouraged to consider more material reasons. For instance, a taxpayer-funded study completed by a government agency or grantee might be published on the agency’s website as a stipulation associated with that funding. An online book or resource review might have the dual purpose of evaluating the item and of promoting its sale. Somewhat differently, online conference proceedings and open access scholarly journal articles might present or summarize new ideas in a field while also facilitating professional discourse about them. Digital trade publications, by contrast, will usually have a more practical orientation. For instance, an article on the best ways to set up a public multimedia show might not contain new ideas about the mechanics of light or sound, but it will contain hands-on advice about setup or innovative pieces of hardware or other technology.
This facet asks students to consider the “Information Has Value” frame most keenly. In asking questions about the motivation for writing and distributing a piece, students are essentially asking about either its use value or its monetary value (or both). Furthermore, students may also see the frame of “Scholarship as Conversation” at work as they consider why authors invest so much effort into publishing a source for dissemination. Though the highest Bloom skill involved here may only be Understand, the variety of motivations to be understood can be nonetheless complex.
How? #1
The first How question considers how the source is supported, asking students to examine the research and viewpoints the author integrated into the source. Students could get practice using this question by looking at a white paper or newspaper article and determining how its author uses outside sources. Statistical evidence, for instance, might be used to establish the existence or acuteness of a trend while an eyewitness source might be quoted to represent a certain perspective on current events. Similarly, expert testimony might be used to provide an authoritative assessment or analysis of an issue. Sometimes the type of source being used can even dictate how it should be used; encyclopedia articles, when cited at all, are generally useful for background information only. Finally, students may learn to detect bias in cases where an author is unconvincingly downplaying the strength of a source that presents an opposing argument.
By determining how a source’s author is engaging with the outside materials incorporated into that source, students can get a picture of how they themselves might engage with the material they are researching. The ACRL frame of “Scholarship as Conversation” intersects with the How question most obviously, but “Searching as Strategic Exploration” can also be highlighted here. By examining the ways that professional authors deploy their outside sources, students can become more aware of the multi-voiced and often divergent nature of a well-researched product. In identifying the ways that authors handle the work of others (related to the “Information Has Value” frame), students can also begin thinking about how they might integrate research into their own writing.
How? #2
The second How question asks how a source could be used in the student’s project. A noteworthy addition to the approach to teaching web source evaluation, this question puts stress on the “use” portion of the “find, evaluate, use” mantra that informs teaching objectives in much of library instruction. Though a source may not make it into the final bibliography, it will be helpful for students to acknowledge that these sources are still crucial for enabling them to reach other sources. This helps students see research as a process rather than a product, creating an attitude toward research that can help them engage in more complex projects down the road.
Equally significant, learners can benefit from considering possible uses for a source: serving as an example/illustration, data/evidence, background/summary; identifying points of controversy; defining a key trait or concept; proposing an argument or counterargument; or introducing a theory, to name a few. Thinking of sources as structurally essential to a project will encourage students to use them not simply to fulfill a requirement but rather as tools whose purpose is to advance the students’ own thinking on their topics. Users who are coached to think more instrumentally about their sources can also be prompted to make better selections.
Frames most germane to this second How question include those of “Information Has Value” and “Scholarship as Conversation.” Students think about sources as having specific kinds of use value and, by using them, engage in the conversation created by scholarship. This last facet also addresses the highest cognitive skill included in the Bloom’s 2001 schema, Create, as it asks students themselves to plan on integrating sources as they have observed authors doing. In choosing which sources to use, students must also consider a source’s appropriateness for use in their project—an activity that invokes the Bloom’s HOTS of Evaluate.
There is a sense in which not all of these questions are created equal. That is to say that some questions—Who and When, for example—may be easier for beginning-level users to answer. Even where it is difficult to determine the identity of an author or an exact publication date, these questions are less likely to require existing background knowledge or close scrutiny of the source’s content. The What and How questions, though, ask for an existing awareness of different source types and for a more sustained look at actual content, respectively. Awareness of many different source types and of how authors often deploy outside sources are likely to constitute knowledge that students have not yet mastered and will require more long-term practice than can be achieved in a one-shot instruction session.
In order to create space for students to begin developing this knowledge, instructors may want to have them look at a slideshow or handout explaining the journalistic approach beforehand, so they can come to class ready to have more involved discussions about applying the different questions to a given source. Perhaps a better alternative, though, might be for the instructor to choose three or so web sources of different types that present common challenges for novice users. In this flipped assignment, students could be asked to write brief reflections in which they evaluate each of the sources without the benefit of a clear evaluative model. Then, during class time, the same three or so sources could be used to illustrate how to find the information that will help students answer each of the key questions. Such a lesson plan would give students the chance to practice the evaluation process while simultaneously introducing them to new source types. Having already encountered the sources in their preparation before the instruction session, students would be more likely to have a productive discussion highlighting the ways that authors have used outside sources in their work. This pedagogical approach can help mitigate the problematic extent to which the journalistic approach requires that students have some kind of developed awareness about source types as well as how those sources can be used.
Conclusion
This discussion articulates a further refinement to web source evaluation pedagogy that has taken shape throughout the twenty-first century, one whose approach is less strictly evaluative and is instead discursive and analytical as a result of its focus on description of and context-specific uses. A key benefit of this new approach lies in its open-endedness—namely its avoidance of both yes/no questions and good/bad assessments. Equally important, its incorporation of very simple criteria and questions that are easy to understand and remember makes it an accessible method for users throughout a broad range of educational levels.
The benefit of a highly analytical evaluative process is that it allows for a more thorough consideration of the source but therein lie some new drawbacks. One of this method’s primary drawbacks is its exhaustive, potentially time-consuming nature. Though simple in theory, this model of web source evaluation takes both time and persistence to apply fully both of which are often in short supply for the average user. The simplicity of the questions and relative ease with which they might be invoked will hopefully offset the cognitive load constituted by an inquiry of extended duration. Moreover, as users accrue more practice with this analytical process, they are more likely to be able to deploy it quickly and nimbly enough, leaving out some criteria when others present sufficiently compelling evidence for students to confidently render a verdict. A second limitation of the journalistic approach is that it asks for an existing familiarity with different source types and their potential uses. A plan for instruction that deliberately uses source types less likely to be familiar to students can help build that awareness while also introducing an evaluative methodology and process. Further steps necessary to spread this method might include further suggestions about how to practically implement this methodology in actual classroom practice. Bringing the investigative process out of the world of theory and into the world of praxis, as those such as Benjes-Small et al. (2013) have done, will help to identify possible areas in the journalistic approach for further development and improvement.
References
Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s Taxonomy of educational objectives (Complete edition). Longman.
Andreassen, R., & Bråten, I. (2013). Teachers’ source evaluation self-efficacy predicts their use of relevant source features when evaluating the trustworthiness of web sources on special education. British Journal of Educational Technology, 44(5), 821–836. https://doi.org/10.1111/j.1467-8535.2012.01366.x
Association of College and Research Libraries. (2015). Framework for information literacy for higher education. http://www.ala.org/acrl/standards/ilframework
Benjes-Small, C., Archer, A., Tucker, K., Vassady, L., & Whicker, J. R. (2013). Teaching web evaluation: A cognitive development approach. Communications in Information Literacy, 7(1), 39–49. https://doi.org/10.15760/comminfolit.2013.7.1.133
Blakeslee, S. (2004). The CRAAP test. LOEX Quarterly 31 (3), 6–7. https://commons.emich.edu/loexquarterly/vol31/iss3/4
Breakstone, J., Smith, M., Wineburg, S., Rapaport, A., Carle, J., Garland, M., & Saavedra, A. (2019). Students’ civic online reasoning: A national portrait. Stanford History Education Group & Gibson Consulting. https://purl.stanford.edu/gf151tb4868
Cmor, D., & Li, X. (2012). Beyond Boolean, towards thinking: Discovery systems and information literacy. Library Management, 33(8/9), 450–457. https://doi.org/10.1108/01435121211279812
Dahl, C. (2009). Undergraduate research in the public domain: The evaluation of non-academic sources online. Reference Services Review, 37(2), 155–163. https://doi.org/10.1108/00907320910957198
Fawley, N., & Krysak, N. (2012). Information literacy opportunities within the discovery tool environment. College & Undergraduate Libraries, 19(2-4), 207-214. https://doi.org/10.1080/10691316.2012.693439
Fielding, J. A. (2019, December). Rethinking CRAAP: Getting students thinking like fact-checkers in evaluating web sources. College & Research Library News, 80(11), 620–622. https://doi.org/10.5860/crln.80.11.620
Hinchliffe, L. J., Rand, A., & Collier, J. (2018). Predictable information literacy misconceptions of first-year college students. Communications in Information Literacy, 12(1), 4–18. https://doi.org/10.15760/comminfolit.2018.12.1.2
Judd, V. C., Farrow, L. I., & Tims, B. J. (2006). Evaluating public web site information: A process and an instrument.” Reference Services Review, 34(1), 12–32. https://doi.org/10.1108/00907320510631571
Kapoun, J. (1998, July/August). Teaching undergrads web evaluation: A guide for library instruction. College & Research Library News, 59(7), 522–523. https://doi.org/10.5860/crln.59.7.522
Library Journal. (2017). First-year experience survey: Information literacy in higher education. https://s3.amazonaws.com/WebVault/research/LJ_FirstYearExperienceSurvey_Mar2017.pdf
Mahmood, K. (2016). Do people overestimate their information literacy skills? A systematic review of empirical evidence on the Dunning-Druger effect. Communications in Information Literacy, 10(2), 199–213. https://doi.org/10.15760/comminfolit.2016.10.2.24
Mason, L., Junyent, A. A., & Tornatora, M. C. (2014). Epistemic evaluation and comprehension of web-source information on controversial science-related topics: Effects of a short-term instructional intervention. Computers & Education, 76, 143–157. https://doi.org/10.1016/j.compedu.2014.03.016
Meola, M. (2004). Chucking the checklist: A contextual approach to teaching undergraduates web-site evaluation. portal: Libraries and the Academy, 4(3), 331–344. https://doi.org/10.1353/pla.2004.0055
Meriam Library. California State University, Chico. (n.d.) The CRAAP test. https://library.csuchico.edu/help/source-or-information-good.
Metzger, M. J. (2007). Making sense of credibility on the web: Models for evaluating online information and recommendations for future research. Journal of the American Society for Information Science and Technology, 58(13), 2078–2091. https://doi.org/10.1002/asi.20672
Ostenson, J. (2014). Reconsidering the checklist in teaching internet source evaluation. portal: Libraries and the Academy, 14(1), 33–50. http://doi.org/10.1353/pla.2013.0045
Schrock, K. (n.d.) The 5w’s of web site evaluation [handout]. http://www.schrockguide.net/uploads/3/9/2/2/392267/5ws.pdf
Schrock, K. (2001, July). E-valuating the web: Six questions to help you decide which Cable in the Classroom sites are suited to your needs. Cable in the Classroom Magazine. https://www.schrockguide.net/uploads/3/9/2/2/392267/07_01_cic.pdf
Seeber, K. P. (2015). Teaching “format as a process” in an era of web-scale discovery. Reference Services Review, 43(1), 19–30. https://doi.org/10.1108/RSR-07-2014-0023
Wineburg, S., & McGrew, S. (2017, September). Lateral reading: Reading less and learning more when evaluating digital information (Stanford History Education Group Working Paper No. 2017.A1). https://doi.org/10.2139/ssrn.3048994
Zhang, S., Duke, N. K., & Jiménez, L. M. (2011). The WWWDOT approach to improving students’ critical evaluation of websites. Reading Teacher, 65(2), 150–58. https://doi.org/10.1002/TRTR.01016
_____________________________________________________________________________________________