NEW!  Version 3 of iPi Motion Capture is available. iPi Motion Capture 3 delivers new features for improved workflow including support of Kinect 2, improved arms tracking, simplified calibration and others. Read more >>

This documentation corresponds to version 2 of iPi Motion Capture. The latest documentation for version 3 can be found at http://docs.ipisoft.com/

Clean-up

From iPiSoft Wiki
Jump to: navigation, search

Once initial tracking is performed on all (or part) of your video, you can begin cleaning out tracking errors (if any). Post-processing should be applied after clean-up.

Contents

Cleaning up tracking gaps

Tracking errors usually happen in a few specific video frames and propagate to multiple subsequent frames, resulting in tracking gaps. Examples of problematic frames:

  • Occlusion (like one hand not visible in any of the cameras)
  • Indistinctive pose (like hands folded on chest).
  • Very fast motion with motion blur.

To clean up a sequence of incorrect frames (a tracking gap), you should use backward tracking:

  1. Go toward the last frame of tracking gap, to a frame where actor pose is distinctive (no occlusion, no motion blur etc.).
  2. If necessary, use Rotate, Move and IK (Inverse Kinematics) tools to edit character pose to match actor pose on video.
  3. Turn off Trajectory Filtering (set it to zero) so that it does not interfere with your editing.
  4. Click Refit Pose button to get a better fit of character pose.
  5. Click Track Backward button.
  6. Stop backward tracking as soon as it comes close to the nearest good frame.
  7. If necessary, go back to remaining parts of tracking gap and use forward and backward tracking to clean them up.

Cleaning up individual frames

To clean up individual frames you should use a combination of editing tools (Rotate, Move and Inverse Kinematics) and Refit Pose button.

Note: after Refit Pose operation iPi Mocap Studio automatically applies Trajectory Filtering to produce a smooth transition between frames. As the result, pose in current frame is affected by nearby frames. This may look confusing. If you want to see exact result of Refit Pose operation in current frame you should turn off Trajectory Filtering (set it to zero), but do not forget to change it back to suitable value later.

Tracking errors that cannot be cleaned up using iPi Studio

Not all tracking errors can be cleaned up in iPi Mocap Studio using automatic tracking and Refit Pose button.

  • Frames immediately affected by occlusion sometimes cannot be corrected. Recommended workarounds:
    • Manually edit problematic poses (not using Refit Pose button).
    • Record a new video of the motion and try to minimize occlusion.
    • Record a new video of the motion using more cameras.
  • Frames immediately affected by motion blur sometimes cannot be corrected. Recommended workarounds:
    • Manually edit problematic poses (not using Refit Pose button).
    • Edit problematic poses in some external animation editor.
    • Record a new video of the motion using higher framerate.
  • Frames affected by strong shadows on the floor sometimes cannot be corrected. Typical example is push-ups. This is a limitation of current version of markerless mocap technology. iPi Soft is working to improve tracking in future versions of iPi Mocap Studio.
  • Some other poses can be recognized incorrectly by iPi Mocap Studio. This is a limitation of current version of markerless mocap technology. iPi Soft is working to improve tracking in future versions of iPi Mocap Studio.

Tracking refinement

After the primary tracking and cleanup are complete, you can optionally run the Refine pass (see Refine Forward and Refine Backward buttons). It slightly improves accuracy of pose matching, and can automatically correct minor tracking errors. However, it takes a bit more time than the primary tracking, so it is not recommended for quick-and-dirty tests.

Important. Refine should be applied with the same tracking parameters (e.g. feet tracking, head tracking) as the primary tracking in order not to lose previously tracked data.

Important. Refine should be applied before motion controller data. Also, if you plan to manually edit the animation (not related to automatic cleanup with Refit Pose), then also do this after applying Refine.

In contrast to the primary tracking, this pass does no pose prediction, and bases its computations solely on the current pose in a frame. Essentially, running Refine is equal to automatically applying Refit Pose to a range of frames which were previously tracked.

Post-processing: Jitter Removal

Jitter Removal filter is a powerful post-processing filter. It should be applied after cleaning up tracking gaps and errors.

It is recommended that you always apply Jitter Removal filter before exporting animation.

Jitter Removal filter suppresses unwanted noise and at the same time preserves sharp, dynamic motions. By design, this filter should be applied to relatively large segments of animation (no less than 50 frames).

Range of frames affected by Jitter Removal is controlled by current Region of Interest (ROI).

You can configure Jitter Removal options for specific body parts. Default setting for Jitter Removal “aggressiveness” is 1 (one tick of corresponding slider). Oftentimes, you can get better results by applying a slightly more aggressive Jitter Removal for torso and legs. Alternatively, you may want to use less aggressive Jitter Removal settings for sharp motions like martial arts moves.

Jitter Removal filter makes an internal backup of all data produced by tracking and clean up stages. Therefore, you can re-apply Jitter Removal multiple times. Each subsequent run works off original tracking/clean-up results and overrides previous runs.

Post-processing: Trajectory Filtering

Trajectory Filter is a traditional digital signal filter. Its purpose is to filter out minor noise that remains after Jitter Removal filter.

Trajectory Filter is very fast. It is applied on-the-fly to current Region of Interest (ROI).

Default setting for Trajectory Filter is 1. Higher settings result in multiple passes of Trajectory Filter. It is recommended that you leave it at the default setting.

Trajectory Filter can be useful for “gluing” together multiple segments of animation processed with different Jitter Removal options: change the Region of Interest (ROI) to cover all of your motion (e.g. multiple segments processed with different jitter removal setting); change Trajectory Filtering setting to 0 (zero); then change it back to 1 (or other suitable value).

Personal tools