Motion Clouds: Model-based stimulus synthesis of natural-like random textures for the study of motion perception



Figure 1: (A) Top: a natural movie with the main motion component consisting of a horizontal, rightward full-field translation. Such a movie would be produced by an eye movement with constant mean velocity to the left (negative $x$ direction), plus some residual, centered jitter noise in the motion-compensated natural scene. We represent the movie as a cube, whose $(x,y,t=0)$ face corresponds to the first frame, the $(x,y=0,t)$ face shows the rightward translation motion as diagonal stripes. As a result of the horizontal motion direction, the $(x=54,y,t)$ face is a reflected image of the $(x,y,t=0)$ face, contracted or dilated depending on the amplitude of motion. The bottom panel shows the corresponding Fourier energy spectrum, as well as its projections onto three orthogonal planes. For any given point in frequency space, the energy value with respect to the maximum is coded by 6 discrete color iso-surfaces (i.e.: 90\%, 75\%, 50\%, 25\%, 11\% and 6\% of peak. The amplitude of the Fourier energy spectrum has been normalized to 1 in all panels and the same conventions used here apply to all following figures. (B) to (C): The image is progressively morphed (A through B to C) into a Random Phase Texture by perturbing independently the phase of each Fourier component. (upper row): Form is gradually lost in this process, whereas (lower row): most motion energy information is preserved, as it is concentrated around the same speed plane in all three cases (the spectral envelopes are nearly identical).


Figure 2: From an impulse to a Motion Cloud. (A) The movie corresponding to a typical edge", i.e., a moving Gabor patch that corresponds to a localized grating. The Gabor patch being relatively small, for clarity, we zoomed 8 times into the non-zeros values of the image. (B): By densely mixing multiple copies of the kernel shown in (A) at random positions, we obtain a Random Phase Texture (RPT), see Supplemental Movie 1. (C): We show here the envelope of the Fourier transform of kernel $K$: inversely, $K$ is the impulse response in image space of the filter defined by this envelope. Due to the linearity of the Fourier transform, apart from a multiplicative constant that vanishes by normalizing the energy of the RPT to $1$, the spectral envelope of the RPT in (B) is the same as the one of the kernel K shown in (A): $\mathcal{E}_{\bar{\beta}}=\mathcal{F}(K)$. Note that, the spectral energy envelope of a classical" grating would result in a pair of Dirac delta functions centered on the peak of the patches in (C) (the orange hot-spots"). Motion Clouds are defined as the subset of such RPTs whose main motion component is a full-field translations and thus characterized by spectral envelopes concentrated on a plane.}


Figure 3: Equivalent MC representations of some classical stimuli. (A, top): a narrow-orientation-bandwidth Motion Cloud produced only with vertically oriented kernels and a horizontal mean motion to the right. (Bottom): The spectral envelopes concentrated on a pair of patches centered on a constant speed surface. Note that this speed plane" is thin (as seen by the projection onto the ($f_x$,$f_t$) face), yet it has a finite thickness, resulting in small, local, jittering motion components. (B) a Motion Cloud illustrating the aperture problem. (Top): The stimulus, having oblique preferred orientation ($\theta=\frac{\pi}{4}$ and narrow bandwidth $B_{\theta}=\pi/36$) is moving horizontally and rightwards. However, the perceived speed direction in such a case is biased towards the oblique downwards, i.e., orthogonal to the orientation, consistently with the fact that the best speed plane is ambiguous to detect. (C): a low-coherence random-dot kinematogram-like Motion Cloud: its orientation and speed bandwidths, $B_{\theta}$ and $B_{V}$ respectively, are large, yielding a low-coherence stimulus in which no edges can be identified.


Figure 4: Broadband vs. narrowband stimuli. From (A) through (B) to (C) the frequency bandwidth $B_{f}$ increases, while all other parameters (such as $f_{0}$) are kept constant. The Motion Cloud with the broadest bandwidth is thought to best represent natural stimuli, since, as those, it contains many frequency components. (A) $B_{f}=0.05$, (B) $B_{f}=0.15$ and (C) $B_{f}=0.4$.


Figure 5: Competing Motion Clouds. (A) A narrow-orientation-bandwidth Motion Cloud with explicit noise. A red noise envelope was added to the global envelop of a Motion Cloud with a bandwidth in the orientation domain. (B): Two Motion Clouds with same motion but different preferred orientation were added together, yielding a plaid-like Motion Cloud texture. (C): Two Motion Clouds with opposite velocity directions were added, yielding a texture similar to a counter-phase grating. Note that the crossed shape in the $f_x-f_t$ plane is a signature of the opposite velocity directions, while two gratings with the same spatial frequency and in opposite directions would generate a flickering stimulus with energy concentrated on the $f_t$ plane.}


All material (c) L. Perrinet. Please check the copyright notice.

This work was supported by European Union project Number FP7-269921, "BrainScales".
BrainScaleS logoFET logoFP7 logoEU logo

TagMotion TagBayes TagBrainScales TagYear12 TagPublicationsArticles TagMotionClouds

welcome: please sign in