Natural speech used to map semantic system of cerebral cortex [2016’s top 100 journal articles]

This is part 9 of a series reviewing selected papers from the top 100 most-discussed journal articles of 2016.

Past neuroimaging studies have identified a group of regions of the cerebral cortex of the human brain that appear to represent information about the meaning of language, known collectively as the semantic system. However, the previous studies used very few stimulus conditions in their testing, and no study had comprehensively mapped the semantic system.

To address this, a recent study¹ used a data-driven approach to model brain responses elicited by naturally spoken narrative stories containing many different semantic domains. The paper reporting the study is article #93 of the top 100 most-discussed journal articles of 2016.

In the study, seven research subjects listened to more than two hours of stories from a radio program while whole-brain blood-oxygen-level-dependent (BOLD) responses were recorded by functional magnetic resonance imaging (fMRI). Voxel-wise modelling was then used to estimate the semantic selectivity of each voxel across the cortex. A voxel represents a value on a regular grid in three-dimensional space, similar to the way in which a pixel represents a value on a regular grid in two-dimensional space. The word ‘voxel‘ is an amalgam of the initial letters of ‘volume’ and ‘element’, with the ‘x’ added to aid pronunciation. Voxel-wise modelling is a highly effective approach for modelling responses to complex natural stimuli.

A word embedding space was used to identify the semantic features of each word in the stories:

The embedding space was constructed by computing the normalized co-occurrence between each word and a set of 985 common English words (such as ‘above’, ‘worry’ and ‘mother’) across a large corpus of English text. Words related to the same semantic domain tend to occur in similar contexts, and so have similar co-occurrence values. For example, the words ‘month’ and ‘week’ are very similar (the correlation between the two is 0.74), while the words ‘month’ and ‘tall’ are not (correlation is −0.22).

The study analysis suggests that the fMRI data contain about four statistically significant semantic dimensions that are shared across the research subjects:

The first dimension is that which captured the most semantic variance across the voxel-wise models of all seven subjects. One end of this dimension favours categories related to humans and social interaction, including ‘social’, ’emotional’, ‘violent’ and ‘communal’. The other end favours categories related to perceptual descriptions, quantitative descriptions and setting, including ‘tactile’, ‘locational’, ‘numeric’ and ‘visual’. This is consistent with previous suggestions that humans comprise a particularly salient and strongly represented semantic domain. Subsequent dimensions of the semantic space captured less variance than the first and were also more difficult to interpret. The second dimension seems to distinguish between perceptual categories, including ‘visual’ and ‘tactile’, and non-perceptual categories, including ‘mental’, ‘professional’ and ‘temporal’. The third and fourth dimensions are less clear.

With apparent consistency in the patterns of semantic selectivity across individuals, the study researchers sought to create a single atlas describing the distribution of semantically selective functional areas in the human cerebral cortex. To do this, they developed a new Bayesian algorithm called ‘PrAGMATiC’ that produces a model of areas tiling the cortex.

The mapping revealed a number of interesting findings. One is that the distribution of semantically selective areas was found to be relatively symmetrical across the two cerebral hemispheres. This is inconsistent with studies supporting the idea that semantic representation is lateralized to the left hemisphere, suggesting that right hemisphere areas may respond more strongly to narrative stimuli than to the words and short phrases used in most studies. However, more research is needed before conclusions can be drawn.

A further finding is that the organization of semantically selective brain areas appears to be highly consistent across individuals, suggesting that innate brain structure constrains the organization of high-level semantic representations. However, this may also be because of the common life experiences of the subjects, so there is a need for future studies to include subjects from more diverse backgrounds.

Another interesting indication that emerged is that the semantic areas representing the meaning of language appear to represent the same semantic domains during conscious thought, suggesting that the contents of thought or internal speech might be decoded using the same voxel-wise models.

The researchers conclude their paper by emphasizing how their study demonstrates that data-driven approaches are a powerful and efficient way of mapping functional representations of the human brain:

Although our experiment used a simple design in which subjects only listened to stories, the data were rich enough to produce a comprehensive atlas of semantically selective areas.

Reference:

Also published on Medium.