It's worth noting that their false discovery rate was not study-wide, but rather group-wide. As they state at the very end of the paper, "FDR control was performed separately by group (e.g., severity level) to allow for group differences in the proportions of truly null hypotheses". So, if you try to look at all significant P-values, after adjustment, you'll find that for that purpose, the P-value significance is going to be inflated.
It's also worth noting that there is no clear causal relationship. There is a strong and well-established association between depression and inflammatory biomarkers, but we have very little idea of whether depression increases the risk of inflammatory illness or vice-versa.
The research I'm familiar with (and teach) suggests you see associations prospectively in both directions, but no clear evidence of causality (although I've grown skeptical of causality as a concept).
This PNAS paper is nice to see, although as some are pointing out, there's a bit of p-hacking probably going on.
My sense is that certain research topics are kind of ground zero in this misled, outdated mind-body war. So you see people trying to demonstrate something along the lines of "see here, there's a biological basis to this, so patients aren't just inventing this," as if the psychosomatic conceptualization of the problem was ever just "inventing it," and as if people with psychogenic psychosomatic problems can't mimic the same things in their subjective reports (what happens when you do have this cytokine screen, and you still have people who look normal on that?)
As you're alluding to here, the problem is that even if you do find inflammation markers it's difficult to tell if this is due to stress in a broad sense, and how these markers relate to perceptions of stress across a broad range of individuals.
Just to take an example: how do these markers look in psychiatric patients who don't report CFS symptoms? I don't mean to suggest that these individuals have psychiatric problems, but it's unclear to me from this study what is going on with these inflammatory markers.
Let's say these cytokines do cause increased pain and fatigue experience. Do CFS patients have a stronger relationship between those inflammatory markers and experienced pain? If so, what does that mean?
The paper itself is more appropriate in tone than the NPR article, for what it's worth.
I am at the half of this long thread, comments are incredibly good and interesting --even for HN-- but yours seems to me outstanding. Thanks for taking the time to write it.
The answer to that question just muddies the waters further. Anti-inflammatory drugs have been shown to have some anti-depressant effect, while anti-depressant drugs have been shown to have some anti-inflammatory effect. In both cases, the effect size is fairly small.
It would be nice to find a simple cure for a complex and diffuse set of symptoms, but we haven't had any luck so far. In all likelihood there are multiple interacting factors that affect the progression of depressive disorders, with no singular cause.
This is not my field, but you can measure inflammation a couple of ways. One is by quantifying cytokines present and another one is to count blood cells.
If you follow their methods, specifically [63], they are modelling the proportion of null hypotheses in their tests. Performing FDR by group is a reasonable approach, and does not necessarily cause P-value inflation. This is different from a Bonferroni correction, in which total numbers of tests done is the only factor. These FDR models of the proportion of null hypotheses rely on the distribution of p-values, not their raw count (assuming a sufficient count of P-values is present in which to model). Dividing the FDR correction as they did provides more information about group-specific influences, and should not be considered p-value hacking.
Nevertheless, thank you for calling this out. There is way too much p-hacking in science, especially biology.
[63] Benjamini Y, Krieger AM, Yekutieli D (2006) Adaptive linear step-up procedures that control the false discovery rate. Biometrika 93:491–507.
Right. My point is actually that they're doing an FDR within each group, not across the whole study. So, the adjusted P values are accurate within each group, but to look globally, you either need to then Bonferroni or re-perform the FDR globally. I'm not saying that my point is in disagreement with yours, by the way, I'm just trying to clarify that my point is independent of whether they had done in-group FDR or in-group Bonferroni.
...p. 4 esp: 1) Analysis of ME/CFS Cases vs. Healthy Controls (case versus control means comparisons): We employed pseudo-likelihood ratio testing (8)2 to compare pMFI means between a) each disease severity group versus control and b) the equally-weighted average across all three severity groups versus control.
It's worth noting that their false discovery rate was not study-wide, but rather group-wide. As they state at the very end of the paper, "FDR control was performed separately by group (e.g., severity level) to allow for group differences in the proportions of truly null hypotheses". So, if you try to look at all significant P-values, after adjustment, you'll find that for that purpose, the P-value significance is going to be inflated.