max planck, composite image

New breakthrough in image super-resolution

The context

Image super-resolution is the scientific wording for recovering a high resolution image from its low resolution version. In photographer words, it means upscaling. The traditional approach is to create new pixels by interpolating from the pixels of low-res image (as done in Photoshop with the bicubic scaling tool). This produces smooth and blurry high-res images.

The modern vision of super-resolution is to re-construct the details that have been lost when creating the low-res image. It goes without saying that this is impossible: the recovered image will not be the same pixelwise as the high-res image. Yet, this is not the objective. The aim is to synthesize a high-res image which looks like a natural image.

The applications of this technology range from enhancing the image quality of low resolution sensors (like those in smartphones), to saving space and bandwidth for storing and broadcasting multimedia contents. For instance, in the early 200’s, the mp3pro sound format was based on and was compliant with the legacy mp3 format: the decoder was re-constructing the high frequencies of the audio spectrum which were removed by the mp3 encoder.

The fact

A recent scientific paper brings a breakthrough in image super-resolution.

EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis, Mehdi S. M. Sajjadi, Bernhard Schölkopf, Michael Hirsch

It has been presented at the last ICCV conference in October 2017 (International Conference on Computer Vision). See also the website of the Max Planck Institute.

A neural network learned from hundred of thousand images how to predict textures to be injected onto low-resolution images. Again, the goal is not to recover the lost details, but to synthesize textures that look like natural to human eyes. This paper presents this algorithm so-called Enhance-net. The authors acknowledge the fact it produces good high-res images, but completely fails in some cases. This is research work.

Some photo websites ( like DPreview ) recently mentions this excellent work, dreaming of the increase of smartphone image quality.

What does Imatag think about it?

Being strongly biased to image protection, we immediately think of the following threat: Many photographers publish low-res images on the web or social networks. This advertises their Works while protecting them from pillage. What this scientific paper shows is that this practice will soon be unsafe!

The crucial issue for Imatag is whether our invisible watermark technology is robust to EnhanceNet.

So, let us consider this image protected by Imatag whose size is 4248 x 2832. We download its medium size file (1280 x 853 pixels).

©Teddy Furon

We aggressively downscale it again to 320 x 213 pixels. The watermark is still detected.

©Teddy Furon
low res

Let us run a bicubic upscaling to recover a 1280 x 853 pixels image. The result is blurry, but the watermark is still present.

©Teddy Furon

Now, let us run the Enhance-Net algorithm to recover a 1280 x 853 pixels image (from the 320 x 213 pixels low-res). The result looks much better … and the watermark is still present.

©Teddy Furon

(photo : maxplanck, composite image)