Temporally Coherent Video Harmonization Using Adversarial Networks

A over B plus magic equals…..? Compositing is one of the most important editing operations for images and videos. The process of improving the realism of composite results is often called harmonization. Previous approaches for harmonization mainly focus on images. In this work, we take one step further to attack the problem of video harmonization….Read More

Joint Learning of Intrinsic Images and Semantic Segmentation

We’ve previously covered both semantic segmentation and intrinsic image decomposition. Here we see a novel proposal to combine the two tasks under the premise that knowledge of one can assist the other. Models and datasets are coming soon. Semantic segmentation of outdoor scenes is problematic when there are variations in imaging conditions. It is known…Read More

Direction-aware Spatial Context Features for Shadow Detection and Removal

Shadow detection and shadow removal are fundamental and challenging tasks, requiring an understanding of the global image semantics. This paper presents a novel deep neural network design for shadow detection and removal by analyzing the image context in a direction-aware manner. To achieve this, we first formulate the direction-aware attention mechanism in a spatial recurrent…Read More

Learning to Segment Every Thing

Producing accurate pixel-level masks around specific objects within images is of course a common task in VFX. Current solutions can be labor-intensive, and the results from one task cannot be used directly to improve the quality of future work. Existing tools generally do not know the semantic context of the object whose mask is being…Read More

DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks

Whilst the final image quality might not be quite yet there, there is surely more to come from this extremely promising area of research in the future. We present an end-to-end learning approach for motion deblurring, which is based on conditional GAN and content loss. It improves the state-of-the art in terms of peak signal-to-noise…Read More

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

The FlowNet demonstrated that optical flow estimation can be cast as a learning problem. However, the state of the art with regard to the quality of the flow has still been defined by traditional methods. Particularly on small displacements and real-world data, FlowNet cannot compete with variational methods. In this paper, we advance the concept…Read More

Foundry dips their toe

Here’s a recent announcement from Foundry: Deep learning: the new frontier in visual effects production Artificial intelligence (AI) and deep learning technologies are disrupting our world in unprecedented ways. From transport and infrastructure, to marketing and fintech; self-learning machines are increasingly being deployed to challenge the status quo. Deep learning enables systems to learn and…Read More