PART TWO

HOW TO ASSESS THE INFORMATION YOU HAVE FOUND

EVALUATE YOUR SOURCES OF INFORMATION

Even when you get information from a reliable source such as a government document or a respected publication, ask yourself how that information was obtained and how it can be verified. Use this chapter as a checklist to evaluate Internet sources.

BE SKEPTICAL

Errors can occur. The most prominent experts can make mistakes. Typos and editing errors happen—especially with computerized corrections—even in prestigious publications. If a particular statistic seems off-base, see whether you can corroborate it in another source.

Appearances can be deceiving. It’s easy to create a professional-looking Website. Even grade schoolers can now do so if they have the proper software and a little training, so don’t be fooled by a sharp image. However, don’t be put off by resources using plain text (just straight typing). Many government, educational, and nonprofit sites post valuable reports but do not have the budget to hire technicians who could design the site for visual interest and hyperlinks.

Anyone can claim to be an authority. In particular, don’t rely on information you get from discussion groups; always find another source for verification.

Human beings have biases. Even a person who is trying hard to be fair may not understand another perspective, or may be motivated by a particular political or personal agenda.

The popularity of a particular view doesn’t make it correct. The speed with which data is transmitted on the Internet makes its material particularly vulnerable. Notice how quickly jokes and rumors spread, and also be aware that the number of "hits" (visitors) to a site may be the result of a number of marketing ploys, promising something the site doesn’t deliver.

Be very suspicious if you find these characteristics on a Website:

• No author or organization listed on the page
• No date of last revision
• Typographical errors

EVALUATE THE QUALITY OF THE MATERIAL

The last three digits designate the type of institution at the Internet address.

Just as there is a difference between scholarly journals and magazines for the general public, there is a difference between a document on a commercial Website and an article posted by an educational, governmental, or nonprofit site. Often the address is a clue to the purpose of the document.

There is obviously a hierarchy of quality to articles you can find electronically. Some very good information is published (in the spirit of the Internet) outside the conventional review and editing process. However, some individuals can sound very authoritative when they really don’t understand the complexities of the subject.

Distinguish between primary and secondary sources. Primary sources are produced by someone involved with the subject you are studying. For example, for a research project on the film director Martin Scorsese, primary sources include his films and publications as well as interviews with him. Secondary sources are produced by people who have studied the subject. For a study of Scorsese, secondary sources would include film reviews, biographies, and histories—as well as interviews with actors and film technicians he has worked with.

Identify the intended audience. Often the first few lines will tell you whom that website is addressing. For example, the primary audience may be children, people committed to a particular agenda, the general public, or scholars. Once you’ve identified the intended audience of the website, you will be able to sift out the information that is not appropriate for a college report.

Identify the purpose. A commercial website (one whose address ends in .com) is often selling a product; a non-profit (.org) site may have a political, religious, or social agenda. An article at an educational (.edu) site may have been written by a student in a class rather than by a professor.

Check the author and affiliated organization. You may have come across an author before in your research, making it easier to evaluate the level of expertise. But what if you can’t tell whether the author has published on this topic before or is affiliated with a professional organization or institution? Try doing a search just for the author’s name—either with a search engine or with one of the directories listed in the appendix of this book.

When you encounter a Web page and there is no date, author, or institution given, look at the Internet address. Try reaching the main page by deleting all the extensions (letters and numbers after slashes or periods) after what is the main address (which would end, for example, in .edu or .org or .gov). The main page should tell you the affiliation and date of original posting or latest revision; the links should identify the author, department, or agency. Find out the reputation of a sponsoring organization or affiliation by plugging its name into a search engine.

Look for signs of high quality. Footnotes and lists of references demonstrate careful research. Links to other high quality sites (and links by others to this particular site) show that the site is "in the loop" of reputable information providers.

Try to find out whether the document was published under professional review. Professional publications (with editors, fact checkers, and review panels) stand behind the quality of their articles. And now, those articles are often published on the Web. But an article can look just as professional (even more attractive) and not be authoritative. Look for references to an affiliated organization or to previous publication in a respected journal. Very often, this information appears at the end of the document. Sometimes you will only see a logo (symbol with some initials). If so, go to a search engine and see what you can find for those initials.

Note the date of latest revision. Your topic will determine how recent your information must be. Most sites should be updated at least monthly, but some information is obsolete within a few hours. On the other hand, material for historical topics should not be updated except to make corrections; a scholarly paper on Zora Neale Hurston will probably be an important resource, even though it was posted six years ago.

Examine the coverage of the topic. In-depth analysis requires length. What subtopics are covered? Does the coverage appear to be balanced with opposite points of view presented? Does the article refer to evidence to support the various points made? Are there citations and/or links to other sources?

Is the article well written? Many scholars are not stylists, to be sure, but the material should be well organized, with no more than an occasional typo or error in punctuation. Look at the overall tone of the presentation; it should be objective, with the implied intention to inform.

VERIFY THE INFORMATION

Use statistical sources and focused search engines to get the facts you will need for your report.

Find an independent verification for most information. Use the journalists’ guidelines and find an independent source to verify your major information. Obviously, you don’t need two sources for population figures; the US Census Bureau’s statistics (although contested) are used in the United States as a matter of law. But you should always try to find another source for any material presented as fact in discussion groups and on individuals’ homepages.

Sometimes it’s as simple as checking another Website, but at other times you may have to interpret information by examining the larger picture. For example, the rate of divorce in the United States is currently on the decline. What does this mean? At first glance, you might think that the number of stable marriages must therefore be rising. However, that is not the case. Instead, there are fewer Americans marrying in the first place, and when they do marry, it is at a later age. You can get these statistics from the Census Bureau at <www.fedstats.gov>.

Be particularly careful about getting all your data from a single report, no matter how reliable it may appear to be. Not only might the author have selected only those statistics that support a particular point of view, but there may be errors in the document itself. Letters and numbers may get transposed, and spellcheckers won’t catch an inaccurate number or correctly spelled wrong word. Corrections are often published well after the original dateline. It is worth checking the next few issues of a journal or newspaper publishing a substantial report, just to see whether there were any corrections.

For example, in researching "urban sprawl" you may have come across an article in the New York Times (July 14, 1999) saying that for every 10% increase in road size there has been a 5% increase in travel time. How can you verify that statistic?

Consult federal agencies. The United States Bureau of Transportation posts all sorts of studies of highways, road use, travel time, and driving behavior. You can find the results fastest by going to www.fedstats.gov or for a fee to www.usgovsearch.com, or you can use a search engine, entering the query highway "travel time" increase. Any of these searches will take you to the Bureau of Transportation studies at < http://www.bts.gov>.

Consult a site that allows you to ask questions. A number of sites allow you to ask a question and then link you to sites that may have the answer. See the appendix of this book for the addresses.

Ask a person. Who would be most likely to know? Regarding the statistic on highway travel time, it would be the reporter who wrote the article or the authors of the studies cited. It’s a long-shot, but you might try e-mailing the author in care of the New York Times. Since one of the studies mentioned was completed in England, you might also try using a search engine for the phrase highway and "travel time," specifying a site in the United Kingdom.

Use reference resources. At the Librarians’ Index to the Web <http://www.lii.org/InternetIndex/> as well as on the the homepages of most search engines are statistical sources, databases, encyclopedias, and atlases with the results of surveys, studies, charts of demographics, and so forth. See also the appendix of this book for other key resources.

OUTLINE YOUR INFORMATION

Review what you have discovered; study your notes and then identify the important points.

List the categories for the information you have accumulated.

Go through your notes

• list the categories of information each source has covered

or

• list the categories you have found and which sources cover them

or

• imagine a paper based on the notes you have and then list the topics your notes can support in such a paper

Arrange the categories into a simple outline (or a topic "tree" with sub-topic "branches"). At this point, you should be listing only the categories of information you have found with topics and subtopics, not the places where you found the information, or categories you wanted to find.

Identify where you need more information. You should immediately see where the holes are—either where evidence is needed to support a point or where further explanation is needed.

Identify information that doesn’t fit. You won’t be able to find a place for everything you have discovered.

For example, suppose that you got a lot of information on corporate spying, and these were the categories you identified:

profiles of corporate spies (examples)

foreign capture of US trade secrets (examples)

FBI investigations

description of US law

definition of legal corporate spying

surveys of companies—trends

means of spying—insider, outsider, electronic, telephone

descriptions of security precautions

These topics could be organized into a good problem-solution paper, perhaps with this outline:

1. Definition of Intellectual Property (summary of the law)
2. Corporate Spying—Legal

   A. "Competitive intelligence" (definition)

   B. Examples

3. Corporate spying— Illegal

You could then mark on your outline where your examples and notes from your research would fit as well as where you need to get more information.

Reminder: Don’t discard information that doesn’t work for you at this stage; you can’t predict what the next research session may find. You may discover some new material that helps you see what you have from a different perspective.

CORRECT GAPS AND OVERLAPS IN INFORMATION

Look at your outline for any places where you need facts, for any gaps in the reasoning, or for any confusing or contradictory overlaps. These spots will tell you where you need to do more research.

It often happens that searches produce uneven results—lots of information in one area and nothing in another aspect of your topic. Your interests will dictate what you do at this point. Here are some possible next steps:

Eliminate some subtopics. Let’s say that you have lots of stories about what corporate spies did. You might decide to expand your discussion of the methods of corporate spying and then to refer only briefly in your conclusion to the preventive methods. On the other hand, you might prefer to discuss the mechanics of prevention and analyze the process of control. Depending on what you already have, you could then determine whether further research would be necessary.

Get more facts. You may have plenty of opinions and commentary but need some dates or statistics. See the appendix of this book for some addresses for encyclopedias, statistical sources, or experts to query.

Follow up on the names of experts. You might want to get more information on the professional expertise of some of the authorities you cite, to give added weight to their remarks.

Get a third opinion. If two of your sources contradict one another, you’ll need to find someone who can resolve the discrepancy. When two sources overlap, you can choose the better phrasing for quotations, paraphrase them both, or find out whose credentials are better respected.

Modify your topic. There are two reasons for changing your topic: your interests were engaged by one area of the research, or you didn’t find what you expected.

For example, you might have begun your search on corporate spying and then gotten interested in the subtopic of the privacy of e-mail and voice-mail. Developing the topic of privacy in the workworld would probably take you to sites discussing the use of employer-provided equipment, reports of lawsuits against employers who inspected employees’ private messages, how corporate files are maintained, and so forth. With privacy as your main topic, you would need to write a different outline from the one in the previous chapter, but you could probably include some of the information you have already gathered.

On the other hand, you could have been really excited about a topic but none of your searches turned up enough data. If you’ve followed the different avenues to information described in Part 1 with poor results, don’t continue the search. Instead, browse again, as discussed in Chapter 1 "Find Your Focus." Get tips on time management in "Acknowledge the Time Limits of Your Project" in Part 3 before you go too much further in your research. Then use your modified topic and begin your search again through the various types of resources described in Part 1.

At this point, the amount and quality of the information you have found will determine whether you can start writing the report or whether you need more research sessions.