Getting started with multiple actors

Introduction

Starting from version 2.4, iPi Mocap Studio supports multiple person tracking with the following limitations:

• Currently only two actors tracking is available in the public release.
  • Tracking of more than two actors requires more RnD and currently not feasible due to mutual occlusion of actors.
• At least 5 Sony PlayStation Eye cameras are required to track multiple actors.
• It's recommended to use 6 — 8 cameras to track multiple actors.
• For simple actions when actors are rarely close to each other you can use 4 — 5 Sony PlayStation Eye cameras or even two depth sensors (MS Kinect / ASUS Xtion / PrimeSense Carmine).
• For single depth sensor and for 3 web cameras multiple person tracking is not enabled.
• Multiple person tracking is available only in STANDARD edition. In other editions you can try this functionality but in this case all results will be saved in restricted trial format which prohibits export of result animation.

Actors Clothing

For color cameras:

• All standard recommendations.
• Plus it's highly recommended to use different colors for clothing of different actors. It allows to minimize the probability of miss-tracking in case of close actors positions.

For depth sensors:

• Only standard recommendations. No additional ones, because current version of iPi Mocap Studio doesn't use color data from depth sensors during tracking (color data is used only for visualization).

Cameras Configuration

The main idea is to minimize self and mutual occlusions of actors. In general, 5 — 8 cameras should be placed in full-circle with at least two cameras at 2.5+ meters height. If you're going to use less cameras then all actions should be planned in such way which minimize mutual actors occlusion. For example, screenshot below demonstrates how to shoot two actors handshake with the aid of just two depth sensors:

Scene Configuration Example

T-pose

It's convenient but not strongly required when both actors stay in T-pose at one and the same time as far from each other as possible (both actors should be fully visible in all cameras).

T-poses of Actors

Processing

• During project creation you will be asked about actors count and then about each actor parameters:

How many actors?

First actor parameters

Second actor parameters

• At toolbar you will see buttons for switching between actors. Also you can switch by clicking on actor model in the 3D view port. Skin of "inactive" actor is grayed. Content of Actor tab is displayed for active actor.

Active actor on UI

• For each actor:
  1. Roughly align actor's model with actor's images in all cameras using Move, Rotation and IK tools
  2. Adjust actor parameters if needed.
  3. Click Analyze Actor Appearance button.
  4. Correct actor's colors with the aid of Eye dropper if needed.
  5. Click Refit Pose on Tracking tab.

     Note. Unlike all other content of Tracking tab this button touches only active actor and doesn't affect on the second one.

  6. Repeat steps starting from 3 if needed

• Adjust Region-of-Interest, choose settings for tracking on Tracking tab and execute it with the aid of appropriate button.

  Note. All options and buttons on the Tracking tab except Refit Pose button concern all actors all together.

Tracking Tab Controls

• After processing you can use standard procedures for clean-up and post-processing.