Thursday 26 September 2013

A word of caution about testing for U-shaped functions

It is not uncommon in many domains of neuroscience, in particular in fMRI,  to design an experiment to test whether neural activity has a U-shaped response to a parametrically varied stimulus. The analysis to test for this would seem rather trivial. Most commonly researchers simply including a quadratic, or U-shaped term, as a regressor in their design matrix, X, at the first level. They then test whether the beta value for this regressor is significant at the second level. Unsurprisingly, if the data does have a U-shaped function then this method will produce a significant result. So far so obvious. However, there are plenty of other functions that will also produce a significant result with this test even if the data do not have a U-shaped function. Perhaps the most obvious is if the data, Y, is an exponential function of the stimulus range tested. It is easy to show that in simulations that such data will also return a significant result at the second level, even though the data is not a U-shaped function of the stimuli tested. Unfortunately, it is common for studies to claim to have shown a U-shaped relationship because they have shown that the data is correlated with a quadratic regressor. In other words, simply because a U-shaped regressor is significant does not mean the data is actually a U-shaped function of the stimulus. One way to test whether data is truly U-shaped rather than exponential is to test whether the tertiary term, X.^3, is also significant. If the data is an exponential function of the stimulus then it should be significant if it is truly U-shaped then this term should not be significant. If you are interested you can read how we used this logic in this paper with Dr Joel Winston. Winston JS, O'Doherty J, Kilner JM, Perrett DI, Dolan RJ. (2007) Brain systems for assessing facial attractiveness. Neuropsychologia. 45(1):195-206. 

No comments:

Post a Comment