Coding static natural images using spiking event times : do neurons cooperate?

For a reprint, see and

This publication followed reports (see e. g. (Perrinet,02)) on Sparse Spike Coding:


Figure 1: Progressive reconstruction of the spiking image in the primary visual cortex. To illustrate that the visual information is contained in the spike code, we show the theoretical reconstruction of the Tiger image using the algorithm presented in the paper. The different edges are extracted using a sparse coding scheme that grabs most salient information first. This reconstruction would correspond to the reconstrucion of the image in an afferent area using the spiking information only. This particular reconstruction on the 256x256 image used a Steerable pyramid with 8 different orientations as the linear transform. The theoretical compression rate compares to JPEG at slow bpp Fig. 2 and is more efficient than the retina model (compare with Lena).


Figure 2 Regularity of edge coefficients distribution in natural images. We plotted for 200 natural images the mean and variance (depicted by the dotted lines representing the mean $\pm$ one standard deviation) of the decrease of coefficients values in the probabilistic Matching Pursuit scheme described above as a function of the rank. Assuming that these values represent the edge content of the images, it shows that the probability distribution of edge coefficients is regular in natural images and may be used in an efficient compression scheme. It permits to evaluate the coefficients quickly as a function of the rank, and since the transmission error being proportional to the variance to transform efficiently an analog image in a wave of spikes. It should be noted that this decrease is much more rapid than the one observed in the model retina the coefficients are below .15 after 1\% of relative rank.


Figure 3: Is the spike representation over-complete in the retina? (Left) We compared the progressive transmission of information for different degrees of over-completeness in the retina by plotting the average MSE of the residual as a function of the information to code the spike list (in logarithmic scale, propagation up to $12.5\%$ of the relative rank for clarity). The set of neurons used rotation symmetric Mexican hat filters, with scales from layer to layer growing as $\rho=\{ 2,\sqrt{2 },\sqrt[4]{2 },\sqrt[8]{2 } \}$ (and denoted on the legend respectively as 1, 2, 4 and 8). As a comparison we plotted the method used in~\citep{van-Rullen01a} (line 'Wav'). As a function of rank, the MSE decreases more rapidly for increasing degrees of over-completeness. (Right) But if we plot the trade-off of MSE with CPU usage as a function of the over-completeness, we find that for the same amount of information the adaptive dyadic strategy is optimal. One should note that the results of the method described in the text is better than the wavelet method of [van-Rullen, 01] since it is adaptative.


To understand possible strategies of temporal spike coding in the central nervous system, we study functional neuromimetic models of visual processing for static images. We will first present the retinal model which was introduced by Van Rullen and Thorpe [1] and which represents the multi- scale contrast values of the image using an orthonormal wavelet transform. These analog values activate a set of spiking neurons which each fire once to produce an asynchronous wave of spikes. According to this model, the image may be progressively reconstructed from this spike wave thanks to regularities in the statistics of the coefficients determined with natural images. Here, we study mathematically how the quality of information transmission carried by this temporal representation varies over time. In particular, we study how these regularities can be used to optimize information transmission by using a form of temporal cooperation of neurons to code analog values. The original model used wavelet transforms that are close to orthogonal. However, the selectivity of realistic neurons overlap, and we propose an extension of the previous model by adding a spatial cooperation between filters. This model extends the previous scheme for arbitrary ---and possibly non-orthogonal--- representations of features in the images. In particular, we compared the perfor- mance of increasingly over-complete representations in the retina. Results show that this algorithm provides an efficient spike coding strategy for low-level visual processing which may adapt to the complexity of the visual input.


All material (c) L. Perrinet. Please check the copyright notice.

TagPublicationsArticles TagSparse TagImageProcessing

welcome: please sign in