Chapter 25 Critical Reading of Scientific Papers

New scientific knowledge is communicated in many ways: at conferences, through seminar talks, over the web. The primary method, however, for years to come is the journal. A journal is a publication in which the articles are written by researchers in a particular discipline, discussing their latest research. Typically, journal articles are not solicited by the journal; instead authors submit manuscripts to the journal editor. By the time an issue of a journal was ready (typically, a new journal issue appears weekly to quarterly), the manuscript (called a paper once it is published) was read and commented upon by several expert referees (reviewers). In theory, the reviewing process should guarantee the best possible science. Of course, papers are published with flaws, some intentional, some not. The intentional flaws include not exhaustively exploring every interpretation of an experimental result and not designing an experiment to test every set of conditions. Often, the result of the experiments that were done were so interesting that journal editors and reviewers give a pass on the flaws just so the results will reach a wider audience.

Reading published papers is actually an indispensable part of research work. The purposes of reading are unexceptional for learning the new concepts and/or developments and reviewing the works in a specific field that might be relevant to the research. This is a task requiring experience, skill, and vocabulary of a specialized field. It may take several hours to read a paper for several times before one may understand major results and flaws of the paper. However, first of all, do not panic. Even an elusive subject can be cracked if you read carefully.

Skimming

Skim the headings, authors, abstract, figures, and the like quickly to get an overview of the paper. Although it may take several minutes, for an experienced researcher, it might be adequate enough to determine whether it is worthwhile to read the article.

Vocabulary

Go through the paper word by word, underlining or highlighting new words and phrases. Use a professional dictionary frequently for finding their correct definitions. A relevant textbook may give more complete explanations. The ordinary shelf dictionary may not give you precise definitions or may not reflect how scientists use a word. For instance, “discharge” has a common definition of releasing from service or duty, but its definition in chemistry is much more restricted to releasing electrical energy from a battery. Another well known example, “base” means the fundamental part of something in

common English, but in chemistry its meaning is a compound that in solution has a p H greater than 7.

In most cases, the knowledge and logics you learn from the textbooks are helpful to make connections between these new words and phrases, but the risk of misunderstanding or a superficial understanding is obvious. Therefore, you’d better get an understanding from the context in which it is used. Frequently words applied to describe the procedures of an experiment can be understood from the context, and may be very specific to the paper. Mark such phrases as belonging to one of the major concepts of the paper. For instance, a paper about bond orders and charge distributions might refer to “reactivity”, which you would need to understand in context and in some depth.

Make Notes in the Paper

It is a good habit to make notes in the margins of copies of papers. Use whatever style you prefer. Besides the new words and phrases, if you have questions or criticisms, write them down so you do not forget them. Flag the important points of the paper as well as the suspicious data or key data. This kind of reading habit helps you save time when you have to go over the paper again, specially when you have to re-read it after several months.

Comprehension, Section by Section

Read each section of the paper scrupulously in order to cover the whole picture.

In Introduction, notice how the context is set. What background knowledge is this a part of? The author summarized and commented on previous researches. It is often necessary to distinguish between the actual current study and previous research. Recognize the hypothesis of the paper and the ways this will be tested.

In Experimental {Methods), try to get a clear image of what was done at each step. A good habit is to make an outline and/or sketch of the procedures and instruments. Write down your questions no matter how simple or technical, but some may point to more fundamental considerations that you will use for reflection and criticism below.

In Results, scrutinize the figures and tables, as they are the heart of most papers and may imply whether it is worthwhile to read the rest of the article. If you can coarsely draw conclusions from a figure or table, it means you “understand” the paper. For example, in a table, “the conversions of reactants increase with the increase of a reaction time, but some samples are exception. ” So, you decide to read the explanation of the exception.

In Discussion, the author would like to draw conclusions from the data. Usually, this section consists of a lot of interpretation and is very important. It is where the author reflects on the work and its meaning in connection with other findings and to the field in general.

Short Summary

When you have read the paper for the first time, try to summarize the paper in one or two sentences.

Research papers are often written to communicate an answer to a specific scientific question. Once you can succinctly describe a paper, you have probably noticed what questions the authors of the paper are trying to raise and answer. After you have paid attention to the main idea, you can try to outline the paper to gain insight into more specific details. In cases when it is easy to summarize the paper in one or two sentences, go over the paper again and try to polish your outline in depth by summarizing the three or four most important sub-points of the main idea.

If Possible, Compare the Paper to Other Works

The scientific merit of the paper can be evaluated by comparing the paper to other relevant works in the area. Check if the authors’ proposals and ideas are just a repeat of previous thoughts of other researchers or of the authors themselves? Several forms of scientific contributions exist, including offering novel original new ideas, practicing ideas of others, or demonstrating how a concept or principle works. It is possible that a paper may actually combine several early thoughts or concepts into a novel framework. Therefore, it is important to know and acknowledge clearly in the paper who else are working in the same field and are cited in the reference section accordingly.

Reflection and Criticism

Never suppose that the authors are completely correct. Instead, be suspicious and read a research paper in a critical process. Raise a few questions frequently while analyzing various kinds of research papers.

Introduction

1. What is the overall purpose of the research?

2. How does the research fit into the context of its field? Is it, for example, attempting to settle a controversy? Does it show the validity of a new technique or open up a new field of inquiry?

3. Do you agree with the author’s rationale for studying the question in this way? Methods

1. Were the measurements appropriate for the questions the researcher was approaching?

2. Can the quantities measured exactly reflect something that the authors desire to express, for instance, “peak area” to reflect the concentration of a species, or“saturated swelling degree” to evaluate the crosslinking degree of hydrogel. Were the measurements in this research clearly related to the variables in which the researchers were interested?

Results

1. What are the main findings?

2. Is enough of the data presented so that you can judge for yourself how the experiments turned out?

3. Did you see patterns or trends in the data that the author did not mention? Were there problems that were not addressed?

Discussion

1. Do you agree with the conclusions drawn from the data?

2. Are these conclusions over-generalized or appropriately careful?

3. Are there other factors that could have influenced, or accounted for, the results?

Reading Creatively

Criticism is easy because it is always easier to tear something down than to build it up. Reading creatively involves harder, more positive thinking. What are the valuable merits of this paper? Do the ideas of the authors have other applications or extensions that they have not been aware of? Can they be improved, move forward, or be generalized further? Suppose you would like to start or continue research from this paper, what would be the immediate work you would do?