Should you trust running shoe research done or funded by Nike?

Of course, you should!

So many people have whined for years that the running shoe brands do not do research to back up the claims that they make for their shoes, yet when they do do it they whine that they don’t trust the research as it was done or funded by a running shoe company. They can’t win. What brought this up for me was this Nike funded project that was getting so many negative comments in social media that they do not believe the results as it was funded by Nike. I got into a number of spats in social media over this issue, with most just not getting it (or not wanting to get it).

The study in question was funded by Nike and was a test of the shoe, the Vaporfly that was developed as part of the Breaking2 project. It was conducted at the University of Colorado. The study found that the shoes lowered the energetic cost of running by 4% on average. Nothing in the methods and analysis jumps out at me as being problematic and I see no reason to not accept the results as finding what they found. Not so, for many in social media who took exception to this declaration:

This study was supported by a contract from Nike, Inc. with the University of Colorado, Boulder.
Conflict of interest
Wouter Hoogkamer, Shalaya Kipp, and Jesse H. Frank have no conflicts of interest relevant to the content of this article. Emily M. Farina and Geng Luo are employees of Nike, Inc. Rodger Kram is a paid consultant to Nike, Inc.

Surprisingly, so many were dismissive of the results of the study as (and I paraphrasing here) the study just gave Nike the results that they wanted, so the results should not be trusted. Nothing was mentioned about any fatal problems with the methods, protocol or analysis of the data that should have led to a blanket dismissing of the study.

To claim that the study should be dismissed due to the funding source is the same thing as alleging that the researchers faked the data, in which case, that is defamatory and you may want to find a good lawyer. How else could they have given Nike the results what they wanted if they did not fake the data? What did the researchers do? They measured the oxygen uptake in a group of athletes wearing a standard shoe and the experimental shoe. They then recorded what the data showed and did a statistical test to show that the experimental shoe was better. The only way they could deliberately deliver the result that Nike would have liked is that if they falsified that data that they collected. That is a very serious allegation to make. It strikes at the heart of research integrity (and runs a risk of being defamatory). I am sure the institutional review board (ethics committee) of the University would like to hear if the data was faked, as if that is the case, those doing the research would no longer be employed by that University.

At some point, you have to trust the integrity of researchers. University’s have research integrity units for this purpose. Yes, research fraud happens and is extremely rare. I have done industry-funded research. Yes, the results more often than not did turn out to be favourable to the company providing the funding (but not always), but please do not accuse me of giving them the results they wanted. That is like accusing me of deliberately faking the data. The other way the industry-funded research could be affected by the funding source is during the actual generation or collection of the data is when there is no blinding of those doing the outcome measures and the actual parameter under investigation is subject to a more qualitative type of assessment, so the researcher could be tempted to bias that in a particular direction to get a desired result for the industry funder. That is easy to deal with by blinding those who do the outcome assessments as to the group allocation. For example, if a manual muscle test was an outcome measure, they may be tempted to encourage the participant to push harder or less hard to get the desired result. This could easily be avoided by blinding the person doing the muscle strength testing as to which group the participant was in. The nature of the variables collected in the study in question here could not be biased in such a way.

I did challenge a number of clowns in social media to explain how the researchers were able to “give Nike the results that they wanted“. It could only happen for one of two reasons: Either (1) the results are what they are and that is what the researchers found; or (2) they faked the data. I kept challenging the clowns to explain which one of the two was it. Never could get a satisfactory response from any of them.

Additionally, I am also was somewhat surprised looking at the responses to the Ryan et al study from 2010 that showed that allocating running shoes based on the ‘pronation paradigm’ was not supported and the more recent study by Malisoux et al in 2016 that gave support to the ‘pronation paradigm’. Not once did I hear or see a single comment that the Ryan et al study’s results should be rejected as it was funded by Nike. Yet I have seen numerous comments that the Malisoux et al study should be rejected as it too was funded by a running shoe company (Decathlon). Hypocrisy, much? It is not difficult to see why. One study supported the particular preconceived agenda of certain people and one study did not support that. See the problem with that? (There may or may not be other valid reasons to reject the studies, but the funding source is not one of them).

Yes, industry-funded research is not without its problems. Good industry partners will do nothing to bias the integrity of the researchers or have any influence over the publication of the results. However, it is widely known that studies with negative results (eg drug trials) may not ever get published, therefore biasing the collective body of literature on a topic. There are strategies in place, such a clinical trial registries to deal with that.

A significant amount of research in Universities is supported by industry and that has to be supported and encouraged. Nike have ‘put their money where their mouths are‘ and funded the studies mentioned above and others; as have a number of other running shoe companies. More should be encouraged, not discouraged by blindly disparaging the integrity of the researchers that do the hard yards just because the results of a study do not fit your ‘worldview’.

The way to assess industry-funded research is to focus on the methods and results and ask the hard questions of that. Look at things like subject selection, allocation, blinding of assessors etc (ie the sort of things I do with research I comment on in this blog). Does that stack up to scrutiny regardless of the funding source. What I do see as a big problem in this sort of research is the spin on the results in the introduction and discussion sections if things are not quite as the funder would like. It is almost as they make excuses for the results not turning out how the funder would have liked. That is why you need to see past the spin and look as what and how the study was done.

As always, I go where the evidence takes me until convinced otherwise …. and there is no reason to reject the above Nike funded research just because it was funded by Nike. Hats off to Nike for funding it and hats of to the research team for doing it and publishing it.

Last updated by .

2 Responses to Should you trust running shoe research done or funded by Nike?

  1. Matthew December 18, 2017 at 1:31 am #

    I too find it amusing that any industry funded research is dismissed by some. As if government funded research is “pure.” But government research suffers from the same motivational biases. Grants tend to be allocated to those doing work aimed at confirming the peer groups existing views or at least based on the assumption those views are true. It is the motivation for the old joke that science progresses one funeral at a time rings so true.

  2. Matt July 20, 2018 at 4:13 pm #

    If people are skeptical they could rerun the protocol on their own which I have done back when Adidas released boost claiming 1% economy improvement and more recently Nike’s Vaporfly. The results were pretty darn close to what the researchers found. (Yes n=1 but the data is relevant to myself)

    There are many universities with an exercise physio lab and I found most are willing to offer running tests apart from the usual VO2max protocol.

Leave a Reply