Clean-up

From iPiSoft Wiki

Once initial tracking is performed on all (or part) of your video, you can begin cleaning out tracking errors (if any). Post-processing should be applied after clean-up.

Contents 1 Cleaning up tracking gaps 2 Cleaning up individual frames 3 Tracking errors that cannot be cleaned up using iPi Studio 4 Post-processing: Jitter Removal 5 Post-processing: Trajectory Filtering

Cleaning up tracking gaps

Tracking errors usually happen in a few specific video frames and propagate to multiple subsequent frames, resulting in tracking gaps. Examples of problematic frames:

• Occlusion (like one hand not visible in any of the cameras)
• Indistinctive pose (like hands folded on chest).
• Very fast motion with motion blur.

To clean up a sequence of incorrect frames (a tracking gap), you should use backward tracking:

1. Go toward the last frame of tracking gap, to a frame where actor pose is distinctive (no occlusion, no motion blur etc.).
2. If necessary, use Rotate, Move and Inverse Kinematics tools to edit character pose to match actor pose on video.
3. Turn off Trajectory Filtering (set it to zero) so that it does not interfere with your editing.
4. Click Refit Pose button to get a better fit of character pose.
5. Click Track Backward button.
6. Stop backward tracking as soon as it comes close to the nearest good frame.
7. If necessary, go back to remaining parts of tracking gap and use forward and backward tracking to clean them up.

Cleaning up individual frames

To clean up individual frames you should use a combination of editing tools (Rotate, Move and Inverse Kinematics) and Refit Pose button.

Note: after “Refit Pose” operation iPiStudio automatically applies Trajectory Filtering to produce a smooth transition between frames. As the result, pose in current frame is affected by nearby frames. This may look confusing. If you want to see exact result of “Refit Pose” operation in current frame you should turn off Trajectory Filtering (set it to zero), but do not forget to change it back to suitable value later.

Tracking errors that cannot be cleaned up using iPi Studio

Not all tracking errors can be cleaned up in iPiStudio using automatic tracking and Refit Pose button.

• Frames immediately affected by occlusion sometimes cannot be corrected. Recommended workarounds:
  • Manually edit problematic poses (not using Refit Pose button).
  • Record a new video of the motion and try to minimize occlusion.
  • Record a new video of the motion using more cameras.
• Frames immediately affected by motion blur sometimes cannot be corrected. Recommended workarounds:
  • Manually edit problematic poses (not using Refit Pose button).
  • Edit problematic poses in some external animation editor.
  • Record a new video of the motion using higher framerate.
• Frames affected by strong shadows on the floor sometimes cannot be corrected. Typical example is push-ups. This is a limitation of current version of markerless mocap technology. iPiSoft is working to improve tracking in future versions of iPiStudio.
• Some other poses can be recognized incorrectly by iPiStudio. This is a limitation of current version of markerless mocap technology. iPiSoft is working to improve tracking in future versions of iPiStudio.

Post-processing: Jitter Removal

Jitter Removal filter is a powerful post-processing filter. It should be applied after cleaning up tracking gaps and errors. It is recommended that you always apply Jitter Removal filter before exporting animation.

Jitter Removal filter suppresses unwanted noise and at the same time preserves sharp, dynamic motions. By design, this filter should be applied to relatively large segments of animation (no less than 50 frames).

Range of frames affected by Jitter Removal is controlled by current Region of Interest.

You can configure Jitter Removal options for specific body parts. Default setting for Jitter Removal “aggressiveness” is 1 (one tick of corresponding slider). Oftentimes, you can get better results by applying a slightly more aggressive Jitter Removal for torso and legs. Alternatively, you may want to use less aggressive Jitter Removal settings for sharp motions like martial arts moves.

Jitter Removal filter makes an internal backup of all data produced by tracking and clean up stages. Therefore, you can re-apply Jitter Removal multiple times. Each subsequent run works off original tracking/clean-up results and overrides previous runs.

Post-processing: Trajectory Filtering

Trajectory Filter is a traditional digital signal filter. Its purpose is to filter out minor noise that remains after Jitter Removal filter.

Trajectory Filter is very fast. It is applied on-the-fly to current Region of Interest.

Default setting for Trajectory Filter is 1. Higher settings result in multiple passes of Trajectory Filter. It is recommended that you leave it at the default setting.

Trajectory Filter can be useful for “gluing” together multiple segments of animation processed with different Jitter Removal options: change the Region of Interest to cover all of your motion (e.g. multiple segments processed with different jitter removal setting); change Trajectory Filtering setting to 0 (zero); then change it back to 1 (or other suitable value).