eMotion Engines

Technology

The ability to estimate motion is vital for the high quality manipulation of digital video media. The technology in the eMotion Engine allows efficient and fast motion estimation for a wide variety of image material. The eMotion Engine combines two classes of motion estimation algorithms for maximum effect. Firstly, a global motion estimator measures the apparent camera motion in the scene and is able to decide which areas of the picture are undergoing global motion and which are not. Secondly, a pixel resolution motion algorithm is used to directly quantify the motion in the remaining areas. This hybrid strategy minimises the load on the pixel resolution motion algorithm.

The images below show the results of estimating motion in a picture in which the motion is to the left. The right hand image shows the result using the eMotion Engine motion technology while the left hand images shows the result using a standard motion estimation process called Block Matching (BM). The red arrows show the motion estimated at pixels in the image. BM is prone to errors in motion direction and is unable to estimate correctly the motion when the image material has no texture. The eMotion Technology creates far more accurate motion information, and is able to correctly predict what the motion inside a textureless area should be. This yields more useful motion vectors that can be used in a variety of high quality post production applications.

Block Matching is heavily used in the video compression industry (e.g. for MPEG2) as well as some broadcast equipment e.g. Digital Vision, Digital Revival, Thompson and so on. The underlying idea in Block Matching is simply to directly compare patches of picture in two frames until the lowest picture difference for a patch is found. The resulting displacement is then assigned as the motion vector. This explains why, in flat areas of the picture, the presence of noise can generate spurious matches and hence poor quality motion vectors. In addition, in regions where there are uni-directional features such as a vertical edge, BM generates motion that estimates correctly in one direction but not the other. This is a well known problem called the aperture effect.

The eMotion technology is able to combine motion information across a wide area of the picture. This allows it to solve the aperture effect and also be more robust to noise levels in the image. This explains why the eMotion field is smoother in general since it is able to correctly decipher problems in motion estimation by paying attention to the directionality of details in the image.

The reason why the eMotion technology is able to do this is because it exploits a probabilistic view of image sequence processing. It avoids hard decision making but instead combines soft decisions at several pixels to yield a more intuitive result. In addition, it takes a fully volumetric view of motion estimation and directly incorporates spatial and temporal motion smoothness. This means that it can estimate both occlusion and uncovering. This is crucial for handling picture behaviour at the edges of moving regions.

The technologies are based on a hybrid of ideas using gradient based motion estimation combined with refinement strategies based on a Bayesian view of image sequence modelling. This has the advantage of using rough initial motion estimate that is fast to obtain. That initial estimate is then refined using a more carefully considered image model incorporating occlusion, uncovering and motion smoothness in time and space. A key new component, as far as speed is concerned is the use of image projections instead of direct image matching. This means that real-time motion estimation at High Definition resolutions is now possible. This new technology is an improved version of motion algorithms already developed and in use in the industry.