Fast and Robust Multiple ColorChecker Detection using Deep Convolutional Neural Networks

Of course it’s not all fluid simulations, high-resolution generative models, and real-time mocap. Sometimes you just need a good solution to a simple problem. ColorCheckers are reference standards that professional photographers and filmmakers use to ensure predictable results under every lighting condition. The objective of this work is to propose a new fast and robust…Read More

Neural Importance Sampling

Here’s an early exploration into using neural networks for guiding Monte Carlo integration, from Disney Research. Interestingly, they note how their learned models can be adapted to slightly modified scenes (e.g. changes in camera) and that might make it quite applicable to optimizing renders of animations. We propose to use deep neural networks for generating…Read More

Joint Learning of Intrinsic Images and Semantic Segmentation

We’ve previously covered both semantic segmentation and intrinsic image decomposition. Here we see a novel proposal to combine the two tasks under the premise that knowledge of one can assist the other. Models and datasets are coming soon. Semantic segmentation of outdoor scenes is problematic when there are variations in imaging conditions. It is known…Read More


SIGGRAPH 2018 is just around the corner. Starting on Sunday August 12th in Vancouver, British Columbia and ending Thursday August 16th it’s looking like this is going to be a bumper year for presentations which cover applications of deep learning and demonstrate some truly stunning results. The full schedule is here, yet you can see…Read More

Texture Mixing by Interpolating Deep Statistics via Gaussian Models

Here’s an interesting technique which may provide new ways in which to generate and synthesize textures from examples via interpolation and style transfer. Recently, enthusiastic studies have devoted to texture synthesis using deep neural networks, because these networks excel at handling complex patterns in images. In these models, second-order statistics, such as Gram matrix, are…Read More

GRAINS: Generative Recursive Autoencoders for INdoor Scenes

Rule-based systems for procedural generation of complex environments such as interiors, cities, and landscapes have been around for a while. Here is a novel data-driven approach to not only learn how objects interact with each other in order to be able to produce new arrangements but also perform conversion of 2D floor plans to 3D…Read More

Mesoscopic Facial Geometry Inference Using Deep Neural Networks

We present a learning-based approach for synthesizing facial geometry at medium and fine scales from diffusely-lit facial texture maps. When applied to an image sequence, the synthesized detail is temporally coherent. Unlike current state-of-the-art methods, which assume ”dark is deep”, our model is trained with measured facial detail collected using polarized gradient illumination in a…Read More

Human Motion Modeling using DVGANs

We present a novel generative model for human motion modeling using Generative Adversarial Networks (GANs). We formulate the GAN discriminator using dense validation at each time-scale and perturb the discriminator input to make it translation invariant. Our model is capable of motion generation and completion. We show through our evaluations the resiliency to noise, generalization…Read More


It’s always fun to see machine learning challenges posed as well as the competing solutions, but even more so when they’re relevant to VFX. PoseTrack is a new large-scale benchmark for human pose estimation and tracking in video. We provide a publicly available training and validation set as well as an evaluation server for benchmarking…Read More

Learning to See in the Dark

Imaging in low light is challenging due to low photon count and low SNR. Short-exposure images suffer from noise, while long exposure can induce blur and is often impractical. A variety of denoising, deblurring, and enhancement techniques have been proposed, but their effectiveness is limited in extreme conditions, such as video-rate imaging at night. To…Read More