Quick Start Guide for Dual Depth Sensor Configuration

BETA STATUS

Prerequisites

• Computer (desktop or laptop):
  • CPU: x86 compatible (Intel Pentium 4 or higher, AMD Athlon or higher), dual- or quad- core is preferable
  • Operating system: Windows 7 (x86 or x64), Windows Vista (x86 or x64), Windows XP SP3 (x86 or x64)
  • Video card: Direct3D 10-capable (Shader Model 4.0) gaming-class graphics card
    • for more info see Cameras and accessories#Video_Card
  • USB: at least two USB 2.0 or USB 3.0 controllers
    • for more info see USB controllers
  • Storage system: HDD or SSD or RAID with write speed at least 55 Mb/sec
• iPi Recorder 1.6.0.0 or later
  • see Quick Start Guide with MS Kinect Sensor#iPi_Recorder for installation instructions
• iPi Studio 1.3.0.0 or later (Free Trial or Basic or Standard edition)
  • see Quick Start Guide with MS Kinect Sensor#iPi_Studio for installation instructions
• Two Microsoft Kinect Sensors with separate power adapters
  • If you bought "XBox 360 + Kinect" bundle then you need to order power adapter
• Optional: active USB 2.0 extension cables
• Minimum required space: 3m by 3m (10 by 10 feet)

Note that before starting of working with two Kinects it's highly recommended to get appropriate results with single Kinect solution: Quick Start Guide with MS Kinect Sensor.

Recording with 2 Kinects

Important! Each Kinect Sensor should be plugged through different USB controllers. If both Kinects are plugged through one and the same USB controller then the second Kinect will not work. How it looks in the Device Manager:

Good		Bad
Both Kinects are plugged correctly		Both Kinects are plugged through one and the same USB controller

Run iPi Recorder and mark both MS Kinect Sensors in the Camera Selector dialog:

Press “Ok” button. As a result you should see something like this:

Make sure that you can see correct colored “depth video” from both sensors and

1. that you can change motor position for each sensor using appropriate slider and “Motor of” combobox
2. that you can switch between RGB and “depth” video using “View RGB/Depth” button
3. that actual frame rate is 30 fps (or 29.9) and frames drop is in range from 0 to 5 for each sensor

Note that you can turn off RGB streaming using appropriate checkbox in the Advanced Options section of Camera Selector dialog. Also you can change correspondence between cameras and motors in case when they are in the wrong order:

Don't forget to set output directory for recorded video:

Calibration

Preparations

There are two possible arrangements of the two sensors:

1. angle between sensors is between 60 and 90 degrees;
2. angle between sensors in near to 180 degrees that is sensors are opposite to each other.

At present time we strongly recommend to use the first configuration:

Top view	Side view

For calibration purposes flat rectangular veneer/plywood/cardboard/pasteboard is used:

• horizontal size should be at least 0.5m (1m - 1.3m is recommended);
• vertical size should be at least 0.7m (1m - 1.5m is recommended);
• hold it vertically in front of yourself on outstretched arms;
• make sure that this calibration plane is good visible in the both sensors (the amount of yellow is low in the area of the veneer/plywood/cardboard/pasteboard).

Good		Bad
good size good depth data		too many yellow (no depth data) points too small too close to human body

Important! Yellow color in the depth map means "depth here is unknown". Thus it is important to minimize amount of yellow points using appropriate materials and clothes.

Recording

After preparations and sensors setup record the calibration video:

• Start recording by pressing the "Start Video Recording" button.
• Initial two seconds of video should contain only background and static objects. Presence of actor or any moving objects in the initial two seconds of recorded video is unacceptable.
• Go to the capture volume with prepared calibration plane (flat rectangular veneer/plywood/cardboard/pasteboard).
• Turn the veneer/plywood/cardboard/pasteboard to the left and to the right thus view angle is changing but the veneer/plywood/cardboard/pasteboard is still visible in both sensors.
• Stop the recording by pressing the "Stop Video Recording" button.

Note that is important to have variety in angles from one hand and good visibility of the calibration plane in each camera from the other hand. It is not so easy to reach good results for 90 degrees angle thus 60-80 degrees angles could be more practical.

Good		Bad
the calibration plane is good visible in both sensors		the calibration plane is not visible in the left sensor

Open resulted video in the iPi Studio and check the following:

• At least first two seconds of video contains only background and static objects (no actor, no calibration plane)
• The floor (ground) is good visible on the depth maps from both sensors
• The calibration plane is good visible on the depth maps most the time
• The calibration plane are moving and has at least 30 degrees variety in the horizontal view angle
• The amount of yellow points is rather small

Processing

• Make sure that ground plane is determined correctly for both sensor after calibration video opening in the iPi Studio (to switch between sensors use buttons "Camera 1" and "Camera 2" in the top toolbar).

• After that it is convenient to turn off visualization of background points (View -> Hide Background).

• Set the beginning of Region of Interest (ROI) to the point where calibration plane is good visible in the both sensors.

• Adjust the end of ROI:
  • the length of ROI should be at leat 5 sec (150 frames)
  • ROI should contain good variety of horizontal view angles for the calibration plane

• Go to the Calibration tab, select "Angle between sensors is 30 - 90 degrees" option and click "Calibrate Based on 3D Plane" button

• Wait for the end of calculations

• Evaluate the result:
  • turn on "Show Depth From All Sensors" option in the "View" menu
  • rotate the scene and check how 3D-points from different sensors fit with each other

• Save the result:
  • go to the Scene tab
  • click "Save scene..." button
  • choose file name for your scene parameters

Next Steps

• Clean-up
• Animation export and motion transfer