Chapter 9 Research
As a college student, it is important that you can find reliable sources for your class work and assignments. It is also essential that you know how to correctly use and handle the sources when you make them part of your own work.
Outside of school, most of us also feel that it is important to be informed about current issues and ideas. Knowing what is going on in the world is, arguably, a citizen’s responsibility. Plus, it feels good to join in a conversation and know the facts, or to be sure we are sharing a meme, social media post, or news article that’s accurate and trustworthy.
Snopes is a widely respected, non-partisan site dedicated to investigating rumors, memes, social media statements, and news stories, and then issuing decisions about whether the materials are correct or false.
Go to Snopes’ “What’s New” page—a page that updates daily and includes the latest rumours.
Scroll through the list until you find an interesting hot topic. Click and read, then write a quick paragraph that summarizes what you found. What did you learn? Were you surprised?
If this captured your interest, you may want to explore Snopes a little more. It’s a fun place to poke around and a great place to fact-check information. The next time you are forwarded a bit of information and you want to know if it’s true before you become outraged and send it to all your friends, do a search for it on Snopes.
As humans living in the digital age, people should know how to navigate the Web successfully, find the best materials, and evaluate and use them with confidence. Alas: in an age where a quick Google search nets millions of “hits” in half a second, evaluating the sources of the chosen material can be trickier than it sounds. There’s a lot of great material on the Web, but there’s a lot of garbage, too. Being able to tell which is an important life skill in the digital age.
We also need to understand who “owns” information—whether hard copy (printed) or digital—and how and when to give credit to the owner; this keeps us safe from accidentally committing plagiarism. This will be discussed in the next section, Chapter 9.2 Plagiarism.
Finding Quality Texts
In the world of academia, our gold standard for texts requires them to be created by people with substantial education, advanced degrees, and life expertise, making them experts in their fields. If I’m reading a cookbook, I want it to be by someone who really knows their way around a kitchen—not someone who’s a mediocre cook but decided it would be fun to collect the family favourites into a self-published book.
You’re a college student. Without a doubt, the best place for you to find quality information is the college library—and you can do this by walking into the library or searching it via the Web. Many college libraries have adopted a new set of guidelines for helping students find good materials. It’s called CRAP. Yes, really! CRAP stands for currency, reliability, authority, and purpose/point of view.
Note: some libraries use CRAAP instead of CRAP, adding a second “A” for “accuracy.” The simple CRAP method, below, incorporates “accuracy” into the “reliable” category. Besides, using CRAP is more fun.
|P: Purpose and Point of View
Sounds like our gold standard, yes? Keeping the metaphor going, your college library is a gold mine for students.
Your College Library
- Gives you access to a world of source materials that have already been reviewed and approved by the librarian staff.
- Allows you free database access that would be extremely expensive if purchased as a non-student. (For example, accessing an EBSCO online database—one of the best college research standards— can cost $40-50 per article! Fortunately, the same article would be free through your college library.)
- Provides current, constantly updated sources.
- Allows you to obtain materials that your library may not have. How? Through a wonderful service called inter-library loan, where your library will actually contact other libraries—all over the country—to find the materials you need and get them to you.
- Provides study spaces, tutoring, research assistance, and other helps.
- Gives you access to librarians—the library’s greatest resource. Where a library database can give you thousands of results in response to a search, the librarian can help you figure out where to start looking, or what search terms to use. They can answer any and all of your questions relating to research.
Wander into your college library (or search the library’s online help) to get started—and if you have problems, ask a librarian! They love to work with students.
Can you also find good material on the World Wide Web?
Of course. But doing so can be tricky. Think for a moment. If you’ve found a website or resource you feel might be useful, how do you convince yourself that it follows the CRAP approach?
Keep in mind that your college work is different from your day-to-day activities. For instance, you may surf the Web for a number of different reasons, perhaps for pleasure, perhaps out of boredom, or maybe chasing links. You can search the Web when you need simple information like what to do for doggy diarrhea or what stores carry light fixtures. We can search and read as we like; nothing is at stake, so to speak.
But in your college work, there’s plenty at stake. Part of doing solid work at the college level has to do with finding strong source materials and using them correctly and effectively.
Find Good Materials on the Web
Finding good materials on the internet takes a bit of detective work. You can use your CRAP detective skills, and rate it using the following system.
Use the CRAP test on your sources and rate them using the following scale:
- “CRAP+” means the source passed the test and had a positive result.
- “CRAP−” means the source did not pass the test and had a negative result.
- “CRAP+ /−” means there wasn’t enough information to know whether the source was credible, and it could be positive or negative.
It also helps to know a little something about how to navigate and use Web materials. Let’s explore!
Domain Name Endings
The domain name ending refers to the letters that follow the period at the end of a Web address (also called a url, or uniform resource locator). For example, .com, .edu, and .org are all domain name endings.
Example of a Web address: www.cnn.com
Example of a domain name ending: .com
Different domain name endings refer to different kinds of websites and can be related to the quality of the site’s material. Therefore, you have to examine them to decide whether they’re reliable. Here are some examples:
.com: a commercial or personal site. These are generally considered to be some of the least reliable sources because anyone can create one and they’re typically used for either private blogs, web pages, and other personal uses or for commercial purposes and sales. So it is often difficult to know if it is reliable. (CRAP+ /−)
.org: these used to belong solely to non-profit sites, such as The American Cancer Society. But these days, anyone can purchase and use a .org site for any purpose. Thus the content on a .org site may vary widely in terms of its authority. (CRAP +/−)
.edu: American educational sites, usually maintained by colleges and universities but sometimes by high schools as well. These sites are considered to be very reliable. (CRAP+)
.gov and .mil: government and military sites, maintained by the governments and the military. These sites are considered to be very reliable. (CRAP+)
Let’s look at a couple of examples and see how they hold up to the CRAP method:
The noted food journalist Michael Pollan uses a .com site, michaelpollan.com, to host many of his writings. He’s a respected writer and resource, and his .com site is a wonderful resource for anyone writing about food.
- C: Is it current? Yes, it is. He is continually adding new materials and updating the site.
- R: Is it reliable? Yes. Pollan uses sources and/or provides source lists for his writings. His work is objective and fact-based.
- A: Is it authoritative? Yes. Pollan provides an extensive biography and a list of his publications and honours. His work is widely respected throughout the publishing and journalism communities, and his books have been published by some of our best-known publishing companies.
- P: What is its purpose? Pollan is a journalist who tries to share science-based information about food and the food industry. He seeks to inform, and he does this with the intention of wanting to make people’s lives better. He is addressing a vast audience: the American people.
Let’s try another one. The .org site, cellphonesafety.org may appear, at first glance, to be a reliable site. But not all .org sites are reliable. Let’s look closer:
- C: Is it current? Although the date at the bottom looks recent, an exploration of the site will show a reliance on articles that are outdated or lead to broken links. This is a problem, because a topic that changes as quickly as one like technology must rely on current and constantly updated materials. No—we cannot say this is current.
- R: Is it reliable? The site does not provide a list of authors. Clicking “About” at the bottom of the page leads to a set of names that do not click through to the actual organizations—a bad sign. Further, it says it was “created by the National Consumer Advocacy Commission.” But a Google search reveals that this organization doesn’t exist! So we’ll give “reliable” a big NO.
- A: Is it authoritative? Again, this is a no. The materials used are not current, and many lead to broken links. The comments tend to use biased language and seem more focused on persuasion (or even manipulation) toward a certain view than on presenting facts.
- P: What is its purpose? The purpose seems to be to convince readers that there are no dangers or hazards associated with cell phone use. We certainly know there are several safety concerns such as road safety, and so again, this fails the test.
In many cases, it’s easy to find an author’s name on an online site. Evaluate the author fully—don’t just assume they know what they’re doing. For example,
- Do they have the right academic credentials or professional experience to back up their authority? For instance, someone who’s spent their life as a short-order cook wouldn’t be considered an authority on astronomy, nor would a PhD-level astronomer be considered an expert on the art of donut making.
- Have they published work in the field?
- Does a quick review of the topic or field suggest that they’re a known expert in that area?
Sometimes pages will list an author’s *, **, or *** on the site, allowing you to find out more about their education, work, and publication history. You can always do a Web search to find out more about them.
*A “bio” (biographical sketch) is a short piece of information about the author and their life, often highlighting unique or interesting events—especially those relevant to the piece they’ve just written.
**A résumé is a short listing of a person’s education, qualifications, and relevant job skills. Résumés are typically used when applying for a job. They are usually 1-2 pages long.
***A curriculum vitae (CV) is like a résumé on steroids. Where a résumé tends to be limited in scope, a CV is a comprehensive listing of one person’s lifetime educational accomplishments and honours, professional memberships, employment, and accomplishments (including publications, lectureships, conference participations, and so forth). These may be dozens of pages in length!
Sometimes an online site will look good but won’t show an author’s name. Does that mean you shouldn’t trust the site? Not necessarily.
Many sites employ a staff of writers or freelance writers to create content on the site but don’t list the author’s name. For example, the National Institute of Health’s information page about headaches lists no authors. Scroll to the page bottom, and you’ll see the page was “prepared by: Office of Communications and Public Liaison.” However, the NIH is a highly respected national institution, and their site is full of information that absolutely meets our CRAP criteria. They list no authors, but they point to the information’s origin, and we can be confident that they are relying on strong writers for their material. Is this a useful site? Absolutely.
* sites may post articles that don’t credit an author. Many of these sites have their own department of **, writers, and *** who create their content; these writers are often not credited individually.
*A periodical is anything that is printed on a regular schedule (i.e., periodically). Periodicals include newspapers, magazines, journals, zines, and more.
**A journalist is a skilled writer who has completed a college bachelor’s degree in journalism. Journalistic writing follows a specific style that is fact-based and objective.
***A freelance writer is a professional writer who is hired and assigned to write specific stories or articles. Freelancers may not be experts in subjects they’re assigned to write about, but they are skilled researchers, enabling them to write about varied topics.
You’ll want to check to see if the material you’re looking at has a date. As a general rule, the more current the date on the material, the better—especially if you’re discussing something that undergoes near-constant change, like politics, science, or technology.
But sometimes, information can be dated and still be useful. For instance, if I was writing a paper about organic gardening, I might be interested in some of Dr. Rudolph Steiner’s original lectures on biodynamic farming. These can be found on the Rudolph Steiner Archive and eLibrary. They date back to the 1920s, but their content is still considered useful and informative by many farmers (CRAP+). This example shows how important it is to consider date when evaluating a source.
Sometimes, you won’t find any date on the material. Again, you’ll need to evaluate this in terms of the strength of the rest of the page. Scrolling to the bottom of a web page will often reveal a “last update” date at the bottom, and this can help your decision process. If you can’t locate any dates on the material and the website hasn’t been updated in years, you should probably find a better source.
Other Points to Check
Consider the visual layout and appeal of the page:
- Does it look modern (CRAP+) or dated (CRAP−) (i.e. as if someone hasn’t updated it in years)?
- Are there lots of advertisements or direct attempts to sell products? (CRAP−)
- Are there pop-ups that interfere with navigating or reading the page? (CRAP−)
And take a look at the page content:
- Are the articles or content well written and carefully proofread? Do they “sound” authoritative and feel reliable? (CRAP+)
- Do articles include links to other materials or links to credible and/or reliable source materials? (CRAP+) Has content been carefully edited, or can you detect lots of errors? (CRAP−)
- Is the language smart and objective? (CRAP+) Or does it include biased language, slang, or frankly rude or negative words? (CRAP−) For example, let’s imagine you were researching a question of why people buy beverages in single-use plastic bottles. An objective, fact-based statement: Studies show that many people buy beverages in single-use plastic bottles because the bottles are convenient, easy to carry, and available just about anywhere. A biased, non-factual statement: Let’s face it—most people who buy single-use plastic bottles are just too lazy to carry reusable bottles. Or they just don’t care if they single-highhandedly destroy the environment.
Video: The CRAP Test for Evaluating Websites (length 5:45)
Look in the Right Places
Start your search in the right place. When looking for a specific piece of evidence, don’t just automatically type a word into Google: instead, ask yourself, “What’s the best place I might find this?” While thinking about the subject, consider the persons or organizations that are considered experts on the topic, and try beginning a search with those names. For instance, if you wanted to answer a question about spaceflight, you might think of starting with NASA.
If you begin in the right place, you’re more likely to find useful information right away, and it’s more likely to be credible. Likewise, try and find the best human sources as well. With a little research on your topic, you can identify the big names in the field.
Don’t always start by turning to the internet and Google. Yes, this may be the easiest way to go, but is it always the best? No—not usually. It is better to start with sites that you know will carry reliable information. Choose Google Scholar over a typical Google search. Google Scholar is a web search engine that focuses on scholarly literature and includes peer-reviewed online academic journals, books, conference papers, dissertations, technical reports, and more. You are far more likely to find a good credible source here than in a general Google search.
Visit your college library, or search it electronically. Read textbooks or periodicals. Seek out human experts. Put your hands on your topic, if you can, by diving into it in a personal way. Try making an observation, conducting a survey, or interviewing a subject. In a recent research writing class, a student writing about Starbucks’s business practices actually drove to company headquarters in Seattle and interviewed a top executive. Another student—this one investigating the Ebola virus—met with two local microbiologists, while a third student researching the geology of Crater Lake went on a weekend outing to experience the national park “up close and personal”.
Practice these strategies when you evaluate sources and websites, and you should be able to find strong materials that will boost your college work.
Whenever you identify a good printed source—book, journal, etc.—go to the end of it and read the bibliography. Voila: a brand new list of potential source materials!
Part 1: Consider what you’ve just learned about currency, reliability, accuracy, and purpose or point of view to help you evaluate the academic merit of a source.
Keep these qualities in mind as you explore one (or more) of these sites.
- Feline Reactions to Bearded Men (found at www.improbable.com)
- Aluminum Foil Deflector Beanie (found at zapatopi.net)
- Dihydrogen Monoxide Research Division (found at www.dhmo.org)
Part 2: Answer and consider the following.
- What was your first impression when you first glanced at the site? Why did you have this impression?
- Explore the site a little—clicking links, reading content, looking for authors and dates and so forth. Did your opinion change? Why? What did you discover? Would you rely on the site for your college work? Does it meet the CRAP test?
Hopefully you found that none of those sites passed the CRAP test. But likely you had fun reading through them at least. As you evaluate websites in the future, remember these examples and be sure to explore the sites carefully as you decide whether or not they are reliable.
“Hard Copy” Periodicals
Follow the same guidelines given above for finding strong Web materials. Look at the author, date, and the material itself. Consider the publication itself: a mainstream, respected newspaper or magazine—like The New York Times, Washington Post, or The Atlantic—more or less automatically meets our CRAP test, while smaller or local publications may require a detailed evaluation.
Wikipedia and Other Open Encyclopedias
A is a website that functions as a public, open encyclopedia or collection of information. The best known of these is Wikipedia. Wiki sources typically can be created and edited by anyone. The best part of wikis is their openness, along with the fact that people collaborate to create them. The worst part is that wikis can be created and edited by anyone—including people who have no business doing so or, even worse, who intentionally enter false or defamatory information.
As a rule, Wikipedia and other ”wiki” sources are not considered to be acceptable sources for college work.
The beauty of Wikipedia is its egalitarianism: it’s billed as a public encyclopedia for everyone. The problem with this is that anyone can create a Wikipedia entry, and likewise, anyone can edit the entries. Unfortunately, “anyone” is usually not an authority in the field. Remember, we’re looking for sources that meet the CRAP criteria and that are written by people with degrees, education, and/or expertise in the field. Wikipedia doesn’t follow this model, and so we don’t rely on it as a reliable source.
But now that I’ve told you not to use Wikipedia, be aware that sometimes a teacher may ask you to use Wikipedia for a specific purpose. In that case, they’ll explain why they’re asking you to use it and how you should proceed.
Also, consider this: studies have shown that the information in Wikipedia is, in most cases, as accurate as that from standard encyclopedias (Taraborelli; Terdimann). Yet despite these findings, two problems remain:
- As discussed above, Wiki entries can be made and edited by anyone. (CRAP−)
- There are multiple instances of Wikipedia entries being changed as a “joke” or to defame or damage a source’s credibility. (CRAP−)
These problems once again point out why we don’t rely on Wikipedia for academic work.
Wikipedia, however, has two great features that you can use right away:
- Most entries have long lists of source materials at the bottom of each topic page. Many of these listed materials will be useful when you’re doing research, or at least will aim you in a sound direction.
- Wikipedia can be a great place to start learning more about a topic. Just remember that it is a springboard—not a reliable source itself.
In addition to not using Wikipedia as an academic source, you should also avoid DotDash (formerly About.com), Yahoo! Answers, eHow, and other similar public information sites. If you have questions about using these sources, discuss this with your instructor.
You will find and evaluate a few reference sources for a topic of your choice.
- Pick a topic to research. Here are a few ideas: misogyny, racism, anxiety, bipolar disorder, the benefits of exercise on the brain, the Keto diet, climate change. Choose one of these or one of your own ideas.
- Find a number of sources where you found information about your topic (about 3-5 sources). Record the sources.
- Evaluate the sources using the CRAP test. Did any of your sources pass the CRAP test? If not, keep looking until you find one or two sources that do. Record the sources that did and didn’t pass the CRAP test and the reasons why.
This chapter is a remix of the following chapters:
- “Learning About Plagiarism and Guidelines for Using Information,” “Why is Information Literacy Important?, and “Finding Quality Texts” in The Word on College Reading and Writing by Carol Burnell, Jaime Wood, Monique Babin, Susan Pesznecker, and Nicole Rosevear. Adapted by Mary Shier. CC BY-NC.
- “The CRAP Test for Evaluating Websites” by Stephanie Stephens. Standard YouTube licence.
Biographical sketch (bio) is a short writing about the author and their life, often highlighting unique or interesting events as well as notable accomplishments in their work.
a brief (often 1-2 pages) account of a person’s education, work experience, and qualifications, typically sent with a job application. A longer version of the résumé is the curriculum vitae, often called the CV.
A lifelong account of a person’s education, awards and honors, conferences and presentations, publications, professional memberships, employment, and so forth. A CV may be dozens of pages long; it doesn’t attempt to be brief—as would a résumé; rather, it is an encyclopedic collect of a person’s accomplishments.
a publication that is printed on a regular schedule (i.e., periodically). Periodicals include newspapers, magazines, journals, and more. Periodical titles should always be italicized in your academic writing.
a skilled writer who has completed a college bachelor’s degree in journalism. Journalistic writing follows a specific style that is fact-based and objective. Journalists subscribe to codes of ethics and behavior in terms of creating factual, verified, and unbiased writing.
a professional writer who is hired and assigned to write specific stories or articles. Freelancers may not be experts in subjects they’re assigned to write about, but they are skilled researchers, enabling them to write about varied topics.
a website that functions as a public, open encyclopedia or collection of information.