If temporal straightening is a fundamental goal of visual processing, we might expect that each stage in the visual hierarchy further straightens its inputs. To test this hypothesis, we compared the straightness of natural videos with the straightness of neural population activity elicited by these videos in the primary and secondary visual cortex (area V1 and V2, respectively). We presented random sequences of static frames taken from 16 short videos and used multi-electrode arrays to record population activity in the visual cortex of awake, fixating macaque monkeys (four V1 populations, ranging in size from 52-87 units; three V2 populations, 84-114 units). We obtained temporal trajectories of population activity by arranging neural responses in the videos' natural order.

We found that neural straightening of natural videos increases along the visual hierarchy. Both in V1 and V2, neural response trajectories were straighter than their pixel-domain inputs. This effect was more prominent in V2 than in V1. To test whether straightening is specific to natural videos, we also presented artificial videos that fade from an initial to a final frame. These movies, which follow straight paths in the pixel-domain, elicited the opposite effect in their neural response trajectories, which were significantly curved. Together, these results suggest that temporal straightening may be an objective that shapes the function of multiple stages of the primate visual system.