Integrating the data from large numbers of neurons responding to an ensemble of stimuli in behaving animals is one of the key challenges facing computational neuroscience. We introduce neural encoding manifolds, a construct in which each point is a neuron and nearby neurons respond similarly in time to similar stimuli. The advantages of this unsupervised machine learning approach will be demonstrated in two very different neural systems.

First, in the mouse, naturalistic stimuli drove both the retina and visual cortex. Encoding manifolds were developed for each, and trajectories across the manifold reveal how stimulus selectivity is organized differently in these two populations. Surprisingly, convolutional neural networks are even more topologically extreme.

Second, when applied to the nematode C. elegans, our encoding manifold organizes neurons into neighborhoods that relate to specific functional roles. These inform whether the available anatomical connectomes are sufficient to explain behavior, and suggest direct combinations of neurons that could comprise behavioral modules.