Evaluating Sources: What Is Your Question?

See introduction for this series here.

Before you begin searching for sources, it’s important to consider the nature of your question and its importance. These two dimensions will help direct your search while also providing a better foundation for evaluating sources.

What Is Your Question?

It seems so simple, but it isn’t always. Here are some points to consider:

1. Does your question have a fact as its answer? (e.g., “What is the population of Kansas?”)
2. Is your question a matter of opinion? (e.g., “What restaurant in town serves the best steak?”)
3. Is your question one on which even experts might disagree? (e.g., “What is the best course of treatment for [rare disease]?”)

If your question is seeking a straightforward fact, the answer is often easy to come by via reliable sources, though it might help to check a few. The most important things to establish are that the source is reliable and that it is timely. (For example, population counts obviously change over time, so you want to make sure the source you’re looking at is appropriate for the time period in which you’re interested. Census number from 1900 are going to do little to help you with today’s values.) However, you still want to consider its importance.

If your question falls into #2, you will likely want to ask yourself next about its importance before searching for sources. Same with point #3.

(NOTE: Most of the time this process is fairly intuitive and automatic, particularly with minor questions. That being said, it is still valuable to break the process down because seeing the pieces can help identify the points at which it could go awry.)

Is this question important?

Why should we consider importance before seeking information? Because the importance of the question, combined with the nature of the question, will determine the kind of source you seek out. Obviously, importance is relative, but generally the more important the question is, the more vital it is to find the best possible source for the information. Importance can be based on a variety of factors — time, money, potential implications of a mistake — so each person may estimate it differently, even for similar situations. Let’s break down the examples above.

What is the population of Kansas?

If this question is simply to provide a basic estimate for a minor point in a paper, a value from a reliable source (e.g., latest Census) is probably sufficient. But if the question has broader implications, for example if it will be used to allocate resources to actual Kansans, you should dig deeper. How old is the latest Census? Is there reason to think that there has been a significant population shift? Is there other data that may be more accurate? In this instance, it is ideal to seek multiple sources of information and see how they compare to one another.

What restaurant in town serves the best steak?

If you’re just trying to figure out a good option for a date night, information sources that pull together consensus ratings (e.g., Yelp) or asking a friend should be sufficient, because the “cost” of ending up at the wrong restaurant is small.

But if you’re asking this question because you have a big potential client who is a steak aficionado whom you need to impress to earn her business, then you may want to also consult reviews by local critics or ask a trusted friend who really knows the city’s culinary scene. Because the “cost” of choosing wrong is higher, you want to pull in additional expertise. You may also simply spend more time reading through the reviews on sites like Yelp for additional nuances and info.

What treatment is best for [rare disease]?

If the disease is the kind that is more annoying than dangerous — particularly if trying different treatments in sequence is a possibility — then while you will obviously still want to consult with a medical professional, you may not be driven to cross the country to find the best-possible expert.

However, if the disease is terminal or extraordinarily painful, or if pursuing one treatment means forgoing another, you will likely want to seek out multiple experts and examine where their opinions converge. This process will be much more complex, and how to weigh these types of opinions will be explored as the series continues.

Breaking It Down

Seeking a fact? Focus on reliable sources (e.g., official/government records, textbooks, etc.). If particularly important, compare info across these sources.

Seeking an opinion? Crowd-sourcing may be enough, particularly if it is a unimportant question. If important, be sure to consult trusted experts as well.

Seeking an answer to a disputed question? Seek out experts and be sure to evaluate expertise (to be discussed later). This is a common situation and the one in which most people find themselves failing to appropriately consider their sources, so it will continue to be a focus.

Learning Activity

Have students consider different questions they’ve had recently where they’ve consulted sources and have them assign each to one of the three categories above. For each question, what source(s) did they use to find an answer?

Have students generate ideas of the third type of questions that exist in society. What sources do they consult to learn more about these?