The specific counterintuitive result is mentioned toward the end of the article, and I'm having some trouble understanding it:
> when analyzing average trends in groups of children, slower reaction times to the “Go” signal were linked to increased activity in many brain regions, including the default mode network
> However, when an individual had a slower reaction time to the “Go” signal, activity decreased in the default mode network — the opposite of the group-level pattern.
One way to think of it — I didn't read the article in depth so this is just an example — is in terms of overall individual differences in speed and activity level. Then, you could have slower persons having increased activity relative to faster persons, but it still be true that when a slower person had an even slower signal reaction, their activity went down, and when a faster person had a slower signal reaction, their activity went down as well.
It's a classic psychological phenomenon, where individual differences are obscuring time course patterns and vice versa.
Of course, this sidesteps the question of why (in the hypothetical example) the overall individual differences exist. Assuming those general individual differences are reliable and "real", you still have to explain why they are there, and if they predict significant outcomes, why they do, and so forth.
The message of the paper is good, although I think the press release (not surprisingly) overstates the significance of the paper. I think these kinds of issues have received a lot more attention in the literature in the last decade or so in neuroscience. It also sort of sidesteps a lot of the more thorny questions about truly person-specific patterns and how to determine when they're meaningful.
"This approach was also able to identify subgroups of children with different levels of cognitive control and performance monitoring, or the ability to modify one’s strategy after making an error."
This should surprise no one. You took a large population and found subpopulations within it. If you want to look at a population average, then use the population data. If you want to look at kids with specific attention needs (guessing ADHD since medical related) then design a study to select for children fitting that criteria, including subtypes.
This seems like the type of thing that should have had a study about study design done long ago that they could have followed to help them structure their own population selection.
Vaguely reminds me of the USAF (?) story of building planes using average measurements of pilots and finding none of the actual pilots fit those specs.
In a sense, but it is a bit more devious. It basically invalidates all past fMRI studies. Not that anyone should have taken those seriously, but it looks like another nail in the coffin. fMRI analysis is (was?) basically: squeeze each brain scan into a standard box, then average the BOLD responses (that's roughly oxygen usage between 3s and 9s after activity). This abstract says that --at least in some cases-- those averages are wrong. Not just hiding information through aggregation, but flat-out lying.
Just from reading the link, I do see an objection: they studied repetitions, which are known to be different from the initial response, so this may not be the fMRI's eulogy.
Seems like a case of Simpson’s Paradox https://en.wikipedia.org/wiki/Simpson%27s_paradox
Not to be confused with Flanderization.
https://tvtropes.org/pmwiki/pmwiki.php/Main/Flanderization
The specific counterintuitive result is mentioned toward the end of the article, and I'm having some trouble understanding it:
> when analyzing average trends in groups of children, slower reaction times to the “Go” signal were linked to increased activity in many brain regions, including the default mode network
> However, when an individual had a slower reaction time to the “Go” signal, activity decreased in the default mode network — the opposite of the group-level pattern.
One way to think of it — I didn't read the article in depth so this is just an example — is in terms of overall individual differences in speed and activity level. Then, you could have slower persons having increased activity relative to faster persons, but it still be true that when a slower person had an even slower signal reaction, their activity went down, and when a faster person had a slower signal reaction, their activity went down as well.
It's a classic psychological phenomenon, where individual differences are obscuring time course patterns and vice versa.
Of course, this sidesteps the question of why (in the hypothetical example) the overall individual differences exist. Assuming those general individual differences are reliable and "real", you still have to explain why they are there, and if they predict significant outcomes, why they do, and so forth.
The message of the paper is good, although I think the press release (not surprisingly) overstates the significance of the paper. I think these kinds of issues have received a lot more attention in the literature in the last decade or so in neuroscience. It also sort of sidesteps a lot of the more thorny questions about truly person-specific patterns and how to determine when they're meaningful.
"This approach was also able to identify subgroups of children with different levels of cognitive control and performance monitoring, or the ability to modify one’s strategy after making an error."
This should surprise no one. You took a large population and found subpopulations within it. If you want to look at a population average, then use the population data. If you want to look at kids with specific attention needs (guessing ADHD since medical related) then design a study to select for children fitting that criteria, including subtypes.
This seems like the type of thing that should have had a study about study design done long ago that they could have followed to help them structure their own population selection.
Vaguely reminds me of the USAF (?) story of building planes using average measurements of pilots and finding none of the actual pilots fit those specs.
Is this common sense and by definition of what an average is?
In a sense, but it is a bit more devious. It basically invalidates all past fMRI studies. Not that anyone should have taken those seriously, but it looks like another nail in the coffin. fMRI analysis is (was?) basically: squeeze each brain scan into a standard box, then average the BOLD responses (that's roughly oxygen usage between 3s and 9s after activity). This abstract says that --at least in some cases-- those averages are wrong. Not just hiding information through aggregation, but flat-out lying.
Just from reading the link, I do see an objection: they studied repetitions, which are known to be different from the initial response, so this may not be the fMRI's eulogy.