How to read scientific studies — a practical guide

In today’s information explosion, almost nobody has time to read everything that interests them. Often they do not even read what would genuinely be useful. That is one reason why conspiracy sites, blogs and similar media are on the rise. They are “easy” to read and they work with facts selectively or not at all. They deliberately play to various clichés and prejudices, and not infrequently they confirm readers’ subjective opinions, which need not be backed by objective facts at all. It is harder still with complex topics, among which is the correct understanding and interpretation of scientific studies and research.

There are thousands even of the serious ones. Working through them is practically impossible. On top of that they are written in specialist language, and to a lay reader several studies on the same topic can appear contradictory. We know it well ourselves: one study praises something, another criticises it. But we are rarely able, in the tangle of specialist jargon, to identify a different methodology, different starting points or different research procedures and to evaluate correctly what may look contradictory at first sight. The truth no longer shines from afar with a clear light. It is often rather in the shadow of half-truths and disinformation. So the key today is to know not only what to read, but also how to read it and how to understand a text in the context of your own life. With the results of scientific work that goes double.

Key messages

The fact that a study is intensively promoted is no guarantee of the highest-quality research.
Even the best or most attractive theory is still only a theory until research confirms it.
Let us not automatically look for correlation where there may in fact be none at all.
The relevance of research without a control group is very low and often none at all.
The standard in serious scientific experiments is so-called double blinding.
If two things are compared, it is important to know the whole context so that the comparison is relevant.
The size and composition of the sample studied directly influences the relevance of the results.
We all have a different starting point; what works for one person immediately may take another a long time.
If the wording of research results is not unambiguous, we recommend always paying closer attention.
The most relevant research is the research whose results various scientists manage to repeat.
Not all research brings ground-breaking discoveries; some only confirms what was assumed.
The fact that something does not fit us 100 % does not yet mean the research is not relevant.
It is always good to apply critical thinking, even with verified sources; nobody is infallible.

At Powerlogy we read studies so that we can live better. Less theory, more practice. A short ingredient list, no added sugar, a clear origin for the ingredients. When we understand research methods, we can also make better decisions in the kitchen and in the shop.

So let us look at how to proceed in order to pick out what matters in the tangle of scientific studies and to adapt it to our individual needs.

1.The influence of those who commission research

Thousands of different research projects are running around the world. They are done by people who feed their families and therefore need to be paid for their work by those who commission the research. It is natural, then, that those commissioners often pick out and promote precisely the studies that best suit their needs. A widely promoted study therefore need not always be the one with the best scientific methodology.

2. Theory is not the same as research

People like theories, studies and research. They help them find their way in a complicated world. But even the best theory is still only a theory. It must always be verified by research based on a study. That often takes several years. Researchers sometimes identify so closely with the theory or hypothesis they are studying that they do not want to admit it when the research shows it was wrong. Especially if it is a theory on which they built their career or their scientific standing.

A tip for your own mini-experiment: try keeping one steady habit for 14 days. Every morning have the same coffee with Powerlogy MCT oil. Leave everything else unchanged. Watch your energy until 11:00 and make notes on your concentration.

3. The influence of automatic correlation

In everyday life we operate on the basis of connections. When a siren wails, we expect an ambulance, the police or the fire brigade to be rushing somewhere. A text message beeps and we instinctively look at our phone, even when it is currently on silent. All of those are correlations. We read the latest study on healthy eating and we may, for example, assume that if a menu contains a lot of vegetables at first sight, such a way of eating is bound to be good. More vegetables is certainly fine, but it says nothing about how balanced the diet is or whether we will feel better eating that way. Our brain has a tendency to help itself with shortcuts. Where it “spots” a correlation, it often automatically assumes causality. But that relationship is not automatic at all. So if a quick and simple explanation turns up, we tend to believe it before we even check the facts or truly understand the often complicated relationships between real causes and effects.

An example from an ordinary day: I see the word “fit” and I believe it is healthy. That is not enough. You have to look at the nutrition table and the ingredients: sugar, fats, protein, fibre, and only then decide. That is how we approach products at Powerlogy too. Facts first, emotion after.

4.The importance of a control group

Every serious piece of research works with control groups. It is the basic rule that lets research be compared and lets us find out whether the processes studied support or refute the hypothesis put forward. The control group takes part in the experiment without doing anything. Without a control group no study has much evidential value. Without it a theory remains only an unverified hypothesis.

5.Double blinding

All serious experiments are also so-called double blind. Put simply, that means that neither the scientist organising the experiment nor the subject being tested knows all the parameters of the testing. This is done to eliminate any side influence, whether deliberate or subconscious. This is the scientific standard, and if research reaches us that does not work with such a methodology, its relevance immediately drops.

Blind tasting works in the kitchen too. Put two chocolates side by side. Cover the wrappers, taste them, and only then check the ingredients. Taste without prejudice, facts without advertising.

6.Relevant comparison

But even a control group does not automatically guarantee that a study is good, true and applicable in practice. For example, with research on whether it is healthier to have only fruit juice or toast with jam for breakfast, we should be on our guard straight away. What value does such research have if we do not learn what the people studied ate for the rest of the day? How would the results be shaken up if, say, evening drinking of alcohol were taken into account?

7. The influence of sample size and composition on the results

This is a fairly well-known fact, but let us keep it in mind all the same. The size of the sample studied directly influences the results. So does its composition. For research to have evidential value, the sample must be sufficiently representative. On top of that, everyone is a unique individual who may tolerate or perceive anything sometimes very differently from a statistically relevant sample.

8.The influence of age, sex and ethnicity

Let us think about this when we read, for example, a study on the harmfulness of milk. It may have been prepared by a team from Asia, where practically the whole population is lactose intolerant. Sleeping pills also work differently for people in their twenties and for 80-year-old seniors. On top of that, uniqueness depends on many factors, not only external or obvious ones. So we should look at research from around the world, applied to samples from different populations, from this angle too.

9. The influence of placebo and of the starting situation

The placebo effect works, but other factors also have to be met, and they are more or less responsible for whether the placebo “takes”. A placebo can therefore be anything — from an evening prayer to lose weight to the 10-kilometre run the next morning. We are all in a different starting position. That is why the same things work differently on different people. That is nothing unusual. On the contrary, it belongs to the set of things that make up our uniqueness as a whole.

10.The influence of wording on how research results are stated

Some articles often use the phrase that something is “significantly associated” with something. Good sleep is significantly associated with a healthy metabolism, for example. If such a statement has no supporting arguments and above all no scientific data, this wording does not have much evidential value. If it contains no scientific, statistical or verifiable arguments, it is not science.

Relative numbers mislead. Look at both the per 100 g figure and the portion, and look for clear words. Not ‘significantly associated’, but ‘contains X g of sugar’. With food, the facts on the label are what count.

11.Relative versus absolute

Let us stay with “word games” for a moment longer. It is also worth paying attention to wordings that involve the notion of relativity. It is easy to operate with, but such statements need not have any relevance. Getting into a lift is also a relative risk, and yet it is almost absolutely safe.

12.Nutrients and foods

Let us be careful when we read or hear somewhere about nutrients and their specific consequences for health. Because we do not eat pure palmitic acid; we do eat dairy products and meat, which contain it in their fats. It is important to know which components a food contains and in what amounts. Only then do we know whether the information is relevant. We also avoid excluding some foods purely on the basis of rough information about the suitability or unsuitability of some of their components, which they may contain only in trace amounts.

13.Replication of research is the sticking point

Researchers often cannot repeat experiments with the same results. If we come across research that has repeatedly, and ideally by different scientific teams, been repeated with analogous results, let us pay closer attention and know that we are reading something very relevant.

14.People want excitement in science too

Not only the classic tabloids but often also more serious media and portals prefer interesting and clickbait stories over ones that are important but boring. If some research confirms old, familiar things, that is often a media bore. Such research will not produce headlines of the “scientists were wrong for years” type. People want tension. But that does not mean that if a study has no sensational results, it is not good. Perhaps the opposite is true, or perhaps it simply does not have enough media support or research sponsors who need its results published.

15.Intellectual blindness

In a flood of information it can easily happen that we cannot see the wood for the trees. We are often so sure of the credibility of a scientific study that we do not even notice that the topic is going in a different direction from the one we originally expected. Not everything follows the trajectory we have mapped out in our heads. At the same time we will never reach a hundred percent match. We will always differ in something.

16.We know a lot, and it is still not enough

Our generation knows a very great deal. And each of us knows somewhere in our subconscious what suits us best. But educating yourself is still a fantastic thing. A person should always remain critical and always take in new facts in context, and through the lens of their own individuality.

So what is key when reading scientific research?

The central theme of all our recommendations is that we should read the texts of scientific studies attentively, but always with perspective. That we should always look at them through the lens of critical thinking too. That means evaluating context and the significance of the individual factors. Ultimately nobody can guarantee that even the best recipe will not cause us, say, an allergic reaction. Every person is a strong individual, an original with no copy. Every person also has different health predispositions, just as each of us has a different metabolism or a different intensity and length of sleep. Working with information may seem easy at first sight, but implementing it is considerably harder. Just as it is easy to be yourself and at the same time very hard not to be swept along by the majority. Much depends on strength of personality, on how and where we get information and how we sort it.

We all want a good and long life. Leaving aside individual or genetic predispositions, it is always worth finding enough energy, applying critical thinking, being able to recognise relevant information, evaluating it in context and adapting it to our own individuality.
It is the road to discovering and activating your own potential and to setting up your long-term optimal performance.

How to turn this into practice? Start reading labels and choose products with a short ingredient list. Test one change for two weeks, watch your energy and sleep. If you are looking for support, reach for Powerlogy foods with no added sugar and a clear origin. Simply, a little better every day.

Sources:

https://www.marksdailyapple.com/15-reasons-not-to-trust-that-latest-nutritional-study/

https://blog.mindvalley.com/human-canvas/

http://www.foodpolitics.com/2016/03/more-industry-funded-studies-with-industry-favorable-results-the-score-14012/

http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0040006

http://www.ncbi.nlm.nih.gov/pubmed/26611124

http://www.ncbi.nlm.nih.gov/pubmed/17473138

http://ajcn.nutrition.org/content/early/2012/11/ajcn.112.047142.abstract

http://www.ncbi.nlm.nih.gov/pubmed/25612907

http://www.srl.cam.ac.uk/epic/epicffq/ffq.pdf

http://www.ncbi.nlm.nih.gov/pubmed/14681273

https://proteinpower.com/drmike/2009/02/more-statin-madness/