Amazing AI tool reconstructs photos like magic

[Rss Bot 1]

If you're retouching photographs and want to cover up small blemishes or remove unwanted details, Photoshop's Content-Aware Fill can be a life-saver. It's perfect for patching up small areas of images, but if you try it on larger areas the results are guaranteed to turn out looking fairly weird.

• The 34 best photo apps

• The 5 best laptops for photo editing

However, a team of researchers from Nvidia is working on a technique that makes it possible to realistically fill huge gaps in photographs, without the results looking like a genetic experiment gone badly wrong.

The technique's called 'image inpainting', and it uses a state-of-the-art deep learning method to edit photos by removing content and filling in the gaps, and reconstruct images that are badly corrupted with holes or missing pixels.

The team used high-end Nvidia Tesla V100 GPUs to train a neural network with over 55,000 randomly generated masks of streaks and holes that were applied to images from the ImageNet, Places2 and CelebA-HQ datasets, so that the neural network would learn to reconstruct the missing pixels. The team then used a different set of nearly 25,000 masks to test the network's reconstruction accuracy.

Column C shows the AI technique filling in the holes from column A; column D is the original image

We're not going to pretend to understand how it all works, but the results speak for themselves. The image inpainting technique is capable of filling in huge gaps – even with really difficult subjects such as human faces or complex landscapes – and doing it in such a way that the edits don't stick out like a sore thumb. Look closely, of course, and you can see the join, but the overall effect is nowhere near as jarring as other content-aware techniques.

The demo video uses fairly low-resolution images, but researchers say that their technique can scale up to handle super-resolution tasks as well. Don't expect to see it in Photoshop any time soon – it currently relies on extremely powerful and expensive deep learning-focused hardware that you won't see outside of a lab. But give it a few years and you should be able to rescue even the most battered of old snaps with a simple one-click fix.

Image inpainting does an incredible job of filling in the gaps here

To find out more, read Nvidia's report on its image inpainting technique, while if you're feeling clever you can read read the original – and highly technical – research paper.

Related articles:

• The 14 best photo editor apps
• Will designers be replaced by robots?
• The 5 best laptops for photo editing

View the full article