Image to Animation model

-

 


What's new: Generating a video sequence with desired motions and object using a driving video sequence. This has lot of applications like movie production, photography etc.

Key insight: Creating a synthesized video from a Deep generative model requires some steps. If you remember Style transfer, we decoupled style and content of the image to get the target image. It’s similar here except that we are decoupling motions and appearance.

Motions: They are achieved by a driving video with a similar object we intend to animate.

Appearance: For this, we use a source image, the object which we will be the object in the generated movie.

Image animation consists of generating a video sequence so that an object in a source image is animated according to the motion of a driving video. 

This framework addresses this problem without using any annotation or prior information about the specific object to animate. Once trained on a set of videos depicting objects of the same category (e.g. faces, human bodies), our method can be applied to any object of this class. 

How it works: To achieve this, researcher decouple appearance and motion information using a self-supervised formulation. To support complex motions, they use a representation consisting of a set of learned keypoints along with their local affine transformations. A generator network models occlusions arising during target motions and combines the appearance extracted from the source image and the motion derived from the driving video. This framework scores best on diverse benchmarks and on a variety of object categories. 

Want your image to look like click here!

Comments

Post a Comment

Popular posts from this blog

Your AI pair programmer

-
Image
  Trained on billions of lines of public code, GitHub Copilot puts the knowledge you need at your fingertips, saving you time and helping you stay focused. GitHub Copilot is powered by Codex, the new AI system created by OpenAI. GitHub Copilot understands significantly more context than most code assistants. So, whether it’s in a docstring, comment, function name, or the code itself, GitHub Copilot uses the context you’ve provided and synthesizes code to match. Together with OpenAI, we’re designing GitHub Copilot to get smarter at producing safe and effective code as developers use it. How it Works Extends your editor:  GitHub Copilot is available as an extension for Neovim, JetBrains, and Visual Studio Code. You can use the GitHub Copilot extension on your desktop or in the cloud on  GitHub Codespaces . And it’s fast enough to use as you type. Speaks all the languages you love:  GitHub Copilot works with a broad set of frameworks and languages. The technical preview does especially we
Read more

The Many Faces of Genetic Illness

-
Image
  People with certain genetic disorders share common facial features. Doctors are using computer vision to identify such syndromes in children so they can get early treatment. What’s new:   Face2Gene  is an app from Boston-based  FDNA  that recognizes genetic disorders from images of patients’ faces. Introduced in 2014, it was upgraded recently to identify over 1,000 syndromes (more than three times as many as the previous version) based on fewer examples. In addition, the upgrade can recognize additional conditions as photos of them are added to the company’s database — no retraining required. How it works:  New  work  by Aviram Bar-Haim at FDNA, Tzung-Chien Hsieh at Rheinische Friedrich-Wilhelms-Universität Bonn, and colleagues describes the revised model.  Face2Gene’s underpinning is a convolutional neural network that was pretrained on  500,000 images of 10,000 faces  and fine-tuned on proprietary data to classify 299 conditions such as Down syndrome and Noonan syndrome.  The devel
Read more