# Understanding multifractality: reconstructing images from edges

## Abstract

It has been recently proven  that natural images exhibit scaling properties analogue to those of turbulent flows. These properties allow regarding each image as a *multifractal* object , for which its most singular manifold conveys the most of the non-redundant structure. In the present work, we go further in this analysis, proposing a simple propagator that reconstructs the whole image from this set. This fact could have deep implications for biology, technology and statistical mechanics.

Short title: Reconstructing Lena

Preprint number: FTUAM 98-23

PACS numbers: 42.30.Wb, 47.53.+n, 47.54.+r, 87.10.+e

Our main motivation was the understanding of the coding strategies  developed by biological neural systems, which has been a matter of great interest, specially in the case of the visual pathway . This problem has often been analyzed using the Information Theory background, thus introducing a statistical treatment of the visual signal  . It is precisely a statistical analysis based on a very different context (the Fully Developed Turbulence) which leads to the most interesting results. This scope provides what seems a powerful new language; new for image analyists but well known for physicist in the field of turbulence. The structure of the paper is as follows: the theoretical basis about the turbulent-like statistics of natural images is presented. This short review introduces the concept of Most Singular Manifold as a statistical essential in images. Then, a guess about how could this set be used to retrieve the whole image is issued and a simple interpretation of it is given. This guess is self-consistently used to check that multifractality is a robust, intrinsic property of natural images; and a possible interpretation of it, perhaps valid beyond this subject, is proposed.

The work was developed on B/W images, although generalizations to coloured images could be done in the same way. We consider each image to be defined by the scalar field of its luminosities, $`I(\stackrel{}{x})`$, although we will make use of the contrast $`c(\stackrel{}{x})I(\stackrel{}{x})I(\stackrel{}{x})_\stackrel{}{x}`$. The main result of  was to prove that that it is possible design intensive variables, related to the derivatives of $`c(\stackrel{}{x})`$, which exhibit scaling laws known as Self Similarity (SS) and Extended Self Similarity (ESS). These properties are the same as those of turbulent flows, which allowed for using models developed in such a context. It is also possible reinterpret that result in terms of a multifractal measure . Very recently  has been established that for natural images the scalar density $`|c|^2(\stackrel{}{x})d\stackrel{}{x}`$ defines a multifractal measure, i.e.: The image may be decomposed as a collection of fractal sets $`F_h`$ (each having fractal dimension $`D(h)`$), formed by the points for which the measure scales locally as $`r^h`$. This characterization is observed indirectly by the scaling of $`p`$-moments related to the measure, which in turn shows the SS properties (as it is comprehensively described in ).

Our first step was to split an image into its different fractal sets. This can easily be done by means of the wavelet analysis (see ) , as a multiresolution decompositon (see ). We followed the detailed technique presented in . One also needs to know the characteristic values of the multifractal (namely, the exponent associated to the Most Singular Manifold (MSM) ($`h_{\mathrm{}}=0.5`$) and the fractal dimension of this set ($`D_{\mathrm{}}=1.`$; see )). The analysis revealed that the MSM, $`F_{\mathrm{}}`$, can be identified with what could naively be called “edges” of the objects present in the scene (see Figures 1 and 2). More interestingly, the multiplicative process model  used to describe multifractal structures  suggests that there is a hierarchical organization of the different fractal manifolds in the image, as it is shown in . The open question is if one of them (namely the MSM) contains enough information to deterministically reconstruct the whole set. This is precisely the aim of this letter.

We consider the gradient $`c(\stackrel{}{x})`$ as our basic field. We would like to reconstruct the field $`c(\stackrel{}{x})`$ for every $`\stackrel{}{x}`$ of the scene given the gradient on $`F_{\mathrm{}}`$. Under the assumptions of translational invariance and linearity, one obtains:

$$c(\stackrel{}{x})=_F_{\mathrm{}}𝑑l(\stackrel{}{y})\stackrel{}{g}(\stackrel{}{x}\stackrel{}{y})c(\stackrel{}{y})$$
(1)   
where $`\stackrel{}{g}`$ is the linear kernel of the desired propagator, and $`_F_{\mathrm{}}𝑑l(\stackrel{}{y})`$ means line integration along the MSM. This equation can be rewritten in a very useful form defining the field $`\stackrel{}{v}_0`$ as

$$\stackrel{}{v}_0(\stackrel{}{x})=c(\stackrel{}{x})\delta _F_{\mathrm{}}(\stackrel{}{x})$$
(2)   
where $`\delta _A(\stackrel{}{x})`$ stands for the proper Hausdorff measure especialized to the set A. In this way, eq. (1) is elegantly expressed in the Fourier space as

$$\widehat{c}(\stackrel{}{f})=\widehat{\stackrel{}{g}}(\stackrel{}{f})\widehat{\stackrel{}{v}}_0(\stackrel{}{f})$$
(3)   
which is an integral equation equivalent to eq. (1), but now the boundary conditions are contained in the vector field $`\stackrel{}{v}_0`$, which depends on the particular image to be reconstructed. The crucial point in all that follows is to determine $`\widehat{\stackrel{}{g}}`$. It is natural to require it to be isotropic, as the particularities of the image are already contained in $`\stackrel{}{v}_0`$, and we think that $`\widehat{\stackrel{}{g}}`$ is an universal propagator. This would imply $`\widehat{\stackrel{}{g}}(\stackrel{}{f})\stackrel{}{f}`$. To end with, we recall a well established property of natural images, namely the scaling of their power spectrum $`S(\stackrel{}{f})`$ (see ), which is:

$$S(\stackrel{}{f})|\widehat{c}(\stackrel{}{f})|^2\frac{1}{f^{2\eta }},$$
(4)   
where $`\eta `$ is a non-universal, small exponent which depends on the particular image ensemble considered (see for instance ). The simplest possible $`\widehat{\stackrel{}{g}}`$ is then given by:

$$\widehat{\stackrel{}{g}}(\stackrel{}{f})=\frac{i\stackrel{}{f}}{f^2}$$
(5)   
This is quite reasonable, because then $`|\widehat{c}(\stackrel{}{f})|=\frac{1}{f}A(\stackrel{}{f})`$, where $`A(\stackrel{}{f})=|\widehat{\stackrel{}{v}}_0(\stackrel{}{f})\stackrel{}{f}|/f`$ has a weak dependence on $`f`$ and varies from one image to another, which could explain the small exponent $`\eta `$ in eq. (4). In Figure 3 the field $`c(\stackrel{}{x})`$ obtained taking as $`F_{\mathrm{}}`$ the set given in Figure 2 is represented. The performance is really good, although quality is lowered by the (unknown) filtering this image has. This is reasonable because filtering damages the natural propagation of light, thus the reconstruction will work up on non-processed natural images. For this we repeated the process with other, non-filtered images (see Figure 5 for one example). The general performance is good, although if one border is lost (at the time of edge detection, see ) so is all the structure associated to it, which seems quite reasonable.

Some remarks should be made about the guess for the kernel $`\widehat{\stackrel{}{g}}(\stackrel{}{f})`$ given by eq. (5). First, it is not so surprisingly simple. It is reasonable thinking about a tridimensional, stationary field of light intensity $`I(x,y,z)`$ propagating out the secundary sources of light that the objects represent. This scalar field sould hence verify $`\mathrm{\Delta }I(\frac{^2}{x^2}+\frac{^2}{y^2}+\frac{^2}{z^2})I=\mathrm{\hspace{0.17em}0}`$ outside those objects. Under apropriate conditions, the projection of this field on a distant screen could verify that $`\frac{^2I}{z^2}0`$ at every point in the screen except those representing the MSM projection. This would imply that the two-dimensional laplacian of the restriction of $`I`$ to the screen should vanish except at the MSM. Then, the image could be reconstructed by Poissonian diffusion from the MSM, whose propagator is precisely eq. (5) for Neumann boundary conditions.

Second, using the kernel of eq. (5) and no matter the image considered, eq. (3) allows reconstructing the correct $`c(\stackrel{}{x})`$ provided that the set $`F_{\mathrm{}}`$ is large enough: Taking $`F_{\mathrm{}}`$ as the whole image, eq. (3) turns out to be a trivial identity. It just seems that natural images allow taking a rather sparse set $`F_{\mathrm{}}`$: the MSM.

This last remark could be used to enhance the performance of the reconstruction, by making a more precise determination of the MSM. Eq. (3), given the kernel eq. (5), implies that

$$div\left(c\delta _{F_{\mathrm{}}^c}\right)=0$$
(6)   
Had the complementary of the MSM ($`F_{\mathrm{}}^c`$) been an open set, eq. (6) would have been a simple Poisson equation on the contrast, the MSM being the source set. Then, operating the Laplacian on an image would make it vanish at every point except at those of the most singular manifold. This operation would allow a fast, direct way to isolate the MSM and the image would be reduced to a monofractal, one might say. In Figure 4 such an operations is performed over Lena’s image, leading to a set rather different than the MSM previously detected, Figure 2. Moreover, the Laplacian-ed images surprisingly still behave as multifractals, with the same characteristic parameters. We noticed this repeating the calculations done in  and in  over a set of images got from H. van Hateren (see ), and then observing their scaling properties (see Figure 6). To explain this, one should notice that in fact the MSM is a dense set, so its complementary set cannot be open and eq. (6) does not allow for a direct splitting. It is also worth noticing that decorrelation of images (an important strategy in visual coding, see ) also produces multifractality of the same kind (see Figure 6 again).

Both transformations (laplacian and decorrelation) intend to be modifications of the power spectrum, that is, the energy distribution by frequency of the image (see ). This is equivalent to multiply the kernel $`\widehat{g}(\stackrel{}{f})`$ by $`f^2`$ or $`f`$ respectively, and use this new kernel to propagate from the MSM, according to eq. (3). We could even think in a whole family of kernels $`\widehat{g}_\alpha `$ which moduli would scale as $`f^\alpha `$, producing different (in principle) multifractals. Those kernels with moduli decaying faster than $`f^1`$ generate more saturated images, as the spatial light propagation would decay slower than with the reconstructor, eq. (5); those ones decaying slower in the frequency domain generate images in which light estinguishes rather close to the source, $`F_{\mathrm{}}`$. But disregarding the strenght of the propagation, the multifractal hierarchy remains the same (as partially shown in Figure 6). It seems that the MSM alone explains all the multifractal structure, being this essentially a geometrical property: in some sense, the singularity exponent $`h`$ of each point has only to do with its geometrical situation with respect to $`F_{\mathrm{}}`$ alone.

Then, the conclusions of our work are:

1. Natural images are rather well described by a simple Poissonian diffusion of the light out of the Most Singular Manifold (MSM), being this a dense but sparse set in the image.
2. In , a conjecture on the relevance from the informational point of view of the MSM is issued. Our result shows that this could be true: the MSM would allow reconstructing the whole image. Given that maximization of the information transfer seems to be a very general learning issue for the sensorial pathways in living beings, this suggests that neural structures performing a codification based on detecting the MSM should be observed (what was first proposed in ). In fact, eq. (5) has much to do with decorrelation and edge detection, two known features performed by the visual neural circuits in mammals.
3. This coding process seems to be a very efficient one in image processing, which in turn could be useful in image engeneering. ( for instance, image compression, light sources modification, etc. )
4. Multifractality, at least for natural images, is explained by the only presence of the MSM. In this sense, the exponent associated to a point measures something like the local density of edges (segments of the MSM). This issue could easily be translated to other contexts ( for instance, to fully developed turbulence ).

Acknowledgements

We are grateful to Néstor Parga for his support, and to Ángel Nevado for fruitful discussions. We want also to acknowledge J.H. van Hateren for his useful comments and criticism of the manuscript. We also thank Patrick Meessen for his hintfull corrections. Antonio Turiel is supported by a FPI grant from the Comunidad Autónoma de Madrid, SPAIN.