NEW!  Version 3 of iPi Motion Capture is available. iPi Motion Capture 3 delivers new features for improved workflow including support of Kinect 2, improved arms tracking, simplified calibration and others. Read more >>

This documentation corresponds to version 2 of iPi Motion Capture. The latest documentation for version 3 can be found at https://docs.ipisoft.com/

Quick Start Guide for Dual Depth Sensor Configuration

From iPiSoft Wiki
(Difference between revisions)
Jump to: navigation, search
m (Video)
m
 
(21 intermediate revisions by 3 users not shown)
Line 1: Line 1:
== Prerequisites ==
+
For the impatient:
 +
* Get two Microsoft Kinect sensors<!--[XBOX] (for XBOX with separate power adapter or for Windows)[XBOX]-->, or ASUS Xtion Live sensors, or PrimeSense Carmine 1.08 sensors
 +
** To choose see [[Depth Sensors Comparison]]
 +
* Get computer with DirectX 10 videocard and at least two USB 2.0 or 3.0 controllers
 +
* [http://www.ipisoft.com/downloads_index.php Download iPi Recorder and iPi Mocap Studio]
 +
* [[IPi_Recorder_Installation|Install iPi Recorder]] and [[IPi_Mocap_Studio_installation|iPi Mocap Studio]] 
 +
* Find a suitable space (9 by 5 feet minimum)
 +
* Actor should be dressed in casual slim clothing, avoid shiny fabrics
 +
* Connect depth sensors to PC
 +
** Note that both sensors can't be connected through one and the same USB 2.0/3.0 controller
 +
* Point them at the actor (the angle between two depth sensors can be in the range 60-90 degrees, or near to 180 degrees)
 +
* For calibration, take flat rectangular cardboard, and use [[iPi Recorder]] to record calibration video
 +
* Please ensure that the calibration board is visible in both sensors. Turn the calibration board to the left and to the right, holding it vertically in front of yourself on outstretched arms.
 +
* Record video of actor's performance. Please ensure that the whole body including arms and legs is visible during the performance in both sensors. Start from a T-pose. Then goes actor’s performance.
 +
* Run [[iPi Mocap Studio]] to process actor performance video
 +
* Import your character into [[iPi Mocap Studio]] (File >Import Target Character) to adapt the animation for your character rig
 +
* [[Animation export and motion transfer|Export your animation in a desired format]]
  
* Computer (desktop or laptop):
+
<noinclude>
** ''CPU:'' x86 compatible (Intel Pentium 4 or higher, AMD Athlon or higher), dual- or quad- core is preferable
+
For more details see [[User Guide for Dual Depth Sensor Configuration]].
** ''Operating system:'' Windows 7 (x86 or x64), Windows Vista (x86 or x64), Windows XP SP3 (x86 or x64)
+
</noinclude>
** ''Video card:'' Direct3D 10-capable (Shader Model 4.0) gaming-class graphics card
+
*** for more info see [[Cameras and accessories#Video_Card]]
+
** ''USB'': at least two USB 2.0 or USB 3.0 '''controllers'''
+
*** for more info see [[USB controllers]]
+
** ''Storage system'': HDD or SSD or RAID with write speed at least 55 Mb/sec
+
* [http://ipisoft.com/downloads/iPiRecorderSetup.exe iPi Recorder] 1.6.0.121 or later
+
** see [[Quick Start Guide with MS Kinect Sensor#iPi_Recorder]] for installation instructions
+
* [http://ipisoft.com/downloads/iPiStudioSetup.exe iPi Studio] 1.3.1.123 or later (''Free Trial'' or ''Basic'' or ''Standard'' edition)
+
** see [[Quick Start Guide with MS Kinect Sensor#iPi_Studio]] for installation instructions
+
* Two [http://www.amazon.com/gp/product/B002BSA298/ref=as_li_tf_tl?ie=UTF8&tag=wwwipisoftcom-20&linkCode=as2&camp=217145&creative=399369&creativeASIN=B002BSA298 Microsoft Kinect Sensors] with separate power adapters
+
** ''If you bought "XBox 360 + Kinect" bundle then you need to [http://www.amazon.com/gp/product/B004IXRXGY/ref=as_li_tf_tl?ie=UTF8&tag=wwwipisoftcom-20&linkCode=as2&camp=217145&creative=399373&creativeASIN=B004IXRXGY order power adapter]''
+
* Optional: [http://www.amazon.com/s/ref=nb_sb_noss?url=search-alias%3Daps&field-keywords=active+USB+2.0+extension+cable&x=14&y=22 active USB 2.0 extension cables]
+
* Optional: [http://www.amazon.com/gp/product/B004TJSLEK/ref=as_li_tf_tl?ie=UTF8&tag=wwwipisoftcom-20&linkCode=as2&camp=217145&creative=399373&creativeASIN=B004TJSLEK Kinect Adjustable Tripod Stands]
+
* Minimum required space: 3m by 3m (10 by 10 feet)
+
 
+
 
+
'''Note''' that before you start working with two Kinects it's highly recommended to get appropriate results with single Kinect solution: [[Quick Start Guide with MS Kinect Sensor]].
+
 
+
 
+
=== Drivers Update ===
+
 
+
If you have used previous version of '''iPi Recorder''' (1.5.0.x) with Microsoft Kinect Sensor, then it is highly recommended to update PrimeSense drivers for Kinect. To do it, please, download and run the following MSI-installation package:
+
* [http://ipisoft.com/downloads/drivers/PSensor_iPi-x86.msi PSensor_iPi-x86.msi] for 32bit Windows
+
* [http://ipisoft.com/downloads/drivers/PSensor_iPi-x64.msi PSensor_iPi-x64.msi] for 64bit Windows
+
 
+
 
+
You’ll be asked about License Agreement for “PrimeSensor Driver (psdrv3) Registration for MS Kinect”:
+
[[Image:PSensor-install-step2.png|center|450px]]
+
 
+
Please read the license agreement and accept it by checking the appropriate checkbox. Then press “Install” button. As soon as you get the Windows Security screen, please select “Install this driver software anyway”:
+
[[Image:KSensor-install-step3.png|center]]
+
+
At this point the driver will complete its installation and should not give any other errors:
+
[[Image:PSensor-install-step4.png|center|450px]]
+
 
+
 
+
Note that there is no need to manually uninstall the previous version of the driver - the installation package will do it by itself. The only thing which is desirable is to unplug Kinect Sensors from computer during installation.
+
 
+
== Recording with 2 Kinects ==
+
 
+
'''Important!''' Each Kinect Sensor must be plugged in through separate USB controllers. If both Kinects are plugged in through one USB controller then the second Kinect will not work. How it looks in the Device Manager:
+
<center><table border=0>
+
  <tr><td width="45%"><center>[[Image:Dual-kinect-device-manager-good.png]]</center></td>
+
      <td width="10%"/>
+
      <td width="45%"><center>[[Image:Dual-kinect-device-manager-bad.png]]</center></td></tr>
+
  <tr><td><center style="color:DarkGreen">'''Good'''</center></td><td/>
+
      <td><center style="color:DarkRed">'''Bad'''</center></td></tr>
+
  <tr><td><center>Both Kinects are plugged correctly</center></td><td/>
+
      <td><center>Both Kinects are plugged through one and the same USB controller</center></td></tr>
+
</table></center>
+
 
+
 
+
Run '''iPi Recorder''' and mark both MS Kinect Sensors in the Camera Selector dialog:
+
[[Image:Dual-kinect-camera-selector.png|center|300px|Click to enlarge]]
+
 
+
 
+
Press “Ok” button. As a result you should see something like this:
+
[[Image:Dual-kinect-ipirecorder.png|center|500px|Click to enlarge]]
+
 
+
 
+
Make sure that you can see correct colored “depth video” from both sensors and
+
# that you can change motor position for each sensor using appropriate slider and “Motor of” combobox
+
# that you can switch between RGB and “depth” video using “View RGB/Depth” button
+
# that actual frame rate is 30 fps (or 29.9) and frames drop is in range from 0 to 5 for each sensor
+
 
+
 
+
'''Note''' that you can turn off RGB streaming using appropriate checkbox in the Advanced Options section of Camera Selector dialog. Also you can change the position of cameras output and motor controls in the interface in case they are positioned in the wrong order:
+
[[Image:Dual-kinect-ipirecorder-advanced-settings.png|center|300px|Click to enlarge]]
+
 
+
 
+
Please don't forget to set output directory for recorded video:
+
[[Image:KSensor-recording-dest-folder.png|center|500px|Click to enlarge]]
+
 
+
== Calibration ==
+
 
+
There are two possible arrangements of the two sensors:
+
# angle between sensors is between 60 and 90 degrees;
+
# angle between sensors in near to 180 degrees that means that sensors are placed opposite to each other.
+
 
+
 
+
=== The first configuration ===
+
 
+
<center><table border=0><tr>
+
<td>[[Image:Dual-kinect-calibration-90-up-view.png|center|400px|Click to enlarge]]</td>
+
<td>[[Image:Dual-kinect-calibration-90-side-view.png|center|390px|Click to enlarge]]</td>
+
</tr><tr>
+
<td><center>'''Top view'''</center></td><td><center>'''Side view'''</center></td>
+
</tr></table></center>
+
 
+
==== Preparations ====
+
 
+
For calibration purposes flat rectangular veneer/plywood/cardboard/pasteboard is used:
+
* horizontal size should be at least 0.5m (1m - 1.3m is recommended);
+
* vertical size should be at least 0.7m (1m - 1.5m is recommended);
+
* hold it vertically in front of yourself on outstretched arms;
+
* make sure that this calibration board is good visible in both sensors (the amount of yellow is low in the area of the veneer/plywood/cardboard/pasteboard).
+
<center><table border=0>
+
  <tr><td width="45%"><center>[[Image:Dual-kinect-calibration-good.png]]</center></td>
+
      <td width="10%"/>
+
      <td width="45%"><center>[[Image:Dual-kinect-calibration-bad.png]]</center></td></tr>
+
  <tr><td><center style="color:DarkGreen">'''Good'''</center></td><td/>
+
      <td><center style="color:DarkRed">'''Bad'''</center></td></tr>
+
  <tr><td><center>good size<br/>good depth data</center></td><td/>
+
      <td><center>too many yellow (no depth data) points<br/>too small<br/>too close to human body</center></td></tr>
+
</table></center>
+
 
+
 
+
'''Important!''' Yellow color in the depth map means "depth here is unknown". Thus it is important to minimize the amount of yellow points using appropriate materials and clothes.
+
 
+
==== Recording ====
+
 
+
After you finished preparations and sensors setup you're ready to record the calibration video:
+
* Start recording by pressing the "Start Video Recording" button.
+
* Initial two seconds of video should contain only background and static objects. Presence of actor or any moving objects in the initial two seconds of recorded video is unacceptable.
+
* Go to the capture area with prepared calibration plane (flat rectangular veneer/plywood/cardboard/pasteboard).
+
* Turn the veneer/plywood/cardboard/pasteboard to the left and to the right thus view angle is changing but the veneer/plywood/cardboard/pasteboard is still visible in both sensors.
+
* Stop the recording by pressing the "Stop Video Recording" button.
+
 
+
<table border=0><tr>
+
<td>[[Image:Dual-kinect-calibration-pos-0.jpg|center|200px]]</td>
+
<td>[[Image:Dual-kinect-calibration-pos-1.jpg|center|200px]]</td>
+
<td>[[Image:Dual-kinect-calibration-pos-2.jpg|center|200px]]</td>
+
<td>[[Image:Dual-kinect-calibration-pos-3.jpg|center|200px]]</td>
+
</tr><tr><td height=10/></tr><tr>
+
<td>[[Image:Dual-kinect-calibration-pos-2.jpg|center|200px]]</td>
+
<td>[[Image:Dual-kinect-calibration-pos-1.jpg|center|200px]]</td>
+
<td>[[Image:Dual-kinect-calibration-pos-0.jpg|center|200px]]</td>
+
<td>[[Image:Dual-kinect-calibration-pos-4.jpg|center|200px]]</td>
+
</tr><tr><td height=10/></tr><tr>
+
<td>[[Image:Dual-kinect-calibration-pos-5.jpg|center|200px]]</td>
+
<td>[[Image:Dual-kinect-calibration-pos-6.jpg|center|200px]]</td>
+
<td>[[Image:Dual-kinect-calibration-pos-4.jpg|center|200px]]</td>
+
<td>[[Image:Dual-kinect-calibration-pos-5.jpg|center|200px]]</td>
+
</tr></table>
+
 
+
 
+
'''Note''' that is important to have variety in angles from one hand and good visibility of the calibration plane in each camera from the other hand. It is not so easy to reach good results for 90 degrees angle thus 60-80 degrees angles could be more practical.
+
 
+
<center><table border=0>
+
  <tr><td width="45%"><center>[[Image:Dual-kinect-calibration-pos-good.jpg|center|400px]]</center></td>
+
      <td width="10%"/>
+
      <td width="45%"><center>[[Image:Dual-kinect-calibration-pos-bad.jpg|center|400px]]</center></td></tr>
+
  <tr><td><center style="color:DarkGreen">'''Good'''</center></td><td/>
+
      <td><center style="color:DarkRed">'''Bad'''</center></td></tr>
+
  <tr><td><center>the calibration plane is good visible in both sensors</center></td><td/>
+
      <td><center>the calibration plane is not visible in the left sensor</center></td></tr>
+
</table></center>
+
 
+
 
+
Please open the captured video in the '''iPi Studio''' and check the following:
+
* At least two initial seconds of video contains only background and static objects (no actor, no calibration plane)
+
* The floor (ground) is good visible on the depth maps from both sensors
+
* The calibration plane is good visible on the depth maps most the time
+
* The calibration plane is moving and has at least 30 degrees variety in the horizontal view angle
+
* The amount of yellow points is relatively small
+
 
+
==== Processing ====
+
 
+
* Please make sure that the ground plane is determined correctly for both sensors after calibration video opening in the '''iPi Studio''' (to switch between sensors use buttons "Camera 1" and "Camera 2" in the top toolbar).
+
 
+
 
+
[[Image:KSensor-process-step2.png|center|500px]]
+
 
+
 
+
* After that it is convenient to turn off visualization of background points (View -> Hide Background).
+
 
+
 
+
[[Image:Dual-kinect-hide-background.png|center]]
+
 
+
 
+
* Set the beginning of Region of Interest (ROI) to the point where calibration plane is good visible in both sensors.
+
 
+
 
+
[[Image:Dual-kinect-calibration-roi-beg.png|center]]
+
 
+
 
+
* Adjust the end of ROI:
+
** the length of ROI should be at leat 5 seconds (150 frames)
+
** ROI should contain good variety of horizontal view angles for the calibration plane
+
 
+
* Go to the '''Calibration''' tab, select "Angle between sensors is 30 - 90 degrees" option and click "Calibrate Based on 3D Plane" button
+
 
+
 
+
[[Image:Dual-kinect-calibration-start.png|center]]
+
 
+
 
+
* Please wait for the end of calibration process
+
 
+
* Evaluate the result:
+
** turn on "Show Depth From All Sensors" option in the "View" menu
+
** rotate the scene and check how 3D-points from different sensors fit with each other
+
 
+
 
+
[[Image:Dual-kinect-calibration-res-eval.jpg|center]]
+
 
+
 
+
* Save the result:
+
** go to the '''Scene''' tab
+
** click "Save scene..." button
+
** choose file name for your scene parameters
+
 
+
 
+
[[Image:Dual-kinect-calibration-save.png|center]]
+
 
+
==== Video tutorial from [http://www.ipisoft.com/forum/viewtopic.php?f=2&t=5509&p=9364#p9364 TrueBones] ====
+
 
+
<html><iframe width="560" height="315" src="http://www.youtube.com/embed/ueM49oAJSig" frameborder="0" allowfullscreen></iframe></html>
+
 
+
 
+
=== The second configuration ===
+
 
+
<center><table border=0><tr>
+
<td>[[Image:Dual-kinect-calibration-180-up-view.png|center|400px|Click to enlarge]]</td>
+
<td>[[Image:Dual-kinect-calibration-180-side-view.png|center|400px|Click to enlarge]]</td>
+
</tr><tr>
+
<td><center>'''Top view'''</center></td><td><center>'''Side view'''</center></td>
+
</tr></table></center>
+
 
+
==== Preparations ====
+
 
+
In this case flat rectangular veneer/plywood/cardboard/pasteboard is also used. But this box is held in one near-straight arm side by side with body. Each Kinect sensor sees different sides of board. Thus thickness of the board should not exceed 3 cm.
+
 
+
==== Recording ====
+
 
+
It is also important to have variety in angles from one hand and good visibility of the calibration plane in each camera from the other hand:
+
 
+
<center><table border=0>
+
  <tr><td width="45%"><center>[[Image:Dual-kinect-calibration-180-pos-good.jpg|center|400px]]</center></td>
+
      <td width="10%"/>
+
      <td width="45%"><center>[[Image:Dual-kinect-calibration-180-pos-bad.jpg|center|400px]]</center></td></tr>
+
  <tr><td><center style="color:DarkGreen">'''Good'''</center></td><td/>
+
      <td><center style="color:DarkRed">'''Bad'''</center></td></tr>
+
  <tr><td><center>the calibration plane is good visible in both sensors</center></td><td/>
+
      <td><center>the calibration plane is not visible in the left sensor</center></td></tr>
+
</table></center>
+
 
+
==== Processing ====
+
 
+
It's mostly similar to processing of the first configuration. The only difference is that before pressing "Calibrate Based on 3D Plane" button you should choose "Angle between sensors is 100 - 180 degrees" radio button.
+
 
+
== Tracking ==
+
 
+
Recording and processing of performance video is generally the same as for single Kinect solution: [[Quick Start Guide with MS Kinect Sensor#Processing Video from Depth Sensor]].
+
 
+
The main difference is that after loading captured video into '''iPi Studio''' you should load your scene parameters:
+
* switch to the '''Scene''' tab
+
* click "Load scene..." button
+
* select file with scene parameters which has been saved during calibration process
+
 
+
That's all. After that you're ready for the processing:
+
* Position timeline slider to the frame where actor is in T-pose
+
* Adjust actor height using appropriate slider on tab “Actor”
+
* Select “Move” tool on toolbar.
+
* Move actor model to left or right to match roughly actor silhouette on video.
+
* etc.
+
 
+
== Next Steps ==
+
 
+
* [[Clean-up]]
+
* [[Animation export and motion transfer]]
+
 
+
 
+
== Materials ==
+
 
+
=== Video ===
+
 
+
<html><iframe width="420" height="315" src="http://www.youtube.com/embed/msRtIZX529Q" frameborder="0" allowfullscreen></iframe></html>
+
 
+
 
+
* From [http://www.ipisoft.com/forum/viewtopic.php?p=9059#p9059 Greenlaw]:
+
 
+
<html><iframe width="560" height="315" src="http://www.youtube.com/embed/XLWygMW12HI" frameborder="0" allowfullscreen></iframe></html>
+
 
+
 
+
* From [http://www.ipisoft.com/forum/viewtopic.php?f=2&t=5438&p=9199#p9199 argus1000]:
+
 
+
<html><iframe width="560" height="315" src="http://www.youtube.com/embed/tPLrV2PH2YM" frameborder="0" allowfullscreen></iframe></html>
+
 
+
 
+
* From [http://www.truebones.com/ TrueBones]:
+
 
+
<html><iframe width="560" height="315" src="http://www.youtube.com/embed/BhVV0f9tOs0" frameborder="0" allowfullscreen></iframe></html>
+
 
+
 
+
<html><iframe title="YouTube video player" width="560" height="315" src="http://www.youtube.com/embed/tdo2WT3mlhQ" frameborder="0" allowfullscreen></iframe></html>
+
 
+
 
+
<html><iframe title="YouTube video player" width="560" height="315" src="http://www.youtube.com/embed/p-7kU9bHGX4" frameborder="0" allowfullscreen></iframe></html>
+
 
+
=== Users' Feedback ===
+
 
+
* [http://www.ipisoft.com/forum/viewtopic.php?p=8938#p8938 Tentacle:]
+
Brilliant work guys.
+
We just tested our setup with two Kinects at 90 degrees.
+
As imagined, most of the occlusion issues are solved.
+
 
+
* [http://www.ipisoft.com/forum/viewtopic.php?p=9008#p9008 Greenlaw:]
+
I'm seeing that dKS tracking is just slightly slower
+
than with single Kinect tracking, but the time difference is negligible.
+
With a single Kinect, I was getting about 0.67 sec per frame;
+
with dKS I'm seeing about 0.7 sec per frame.
+
I think that's very good for my graphics card.
+
(My graphics card is a Nvidia GTX 460,
+
which these days is probably considered a mid-range card.)
+
+
At a glance, the dKS tracking results appears to be more stable
+
than with a single Kinect.
+
 
+
* [http://www.ipisoft.com/forum/viewtopic.php?p=9020#p9020 and one more:]
+
Okay, I have several tracked dKS motions now and
+
I just finished applying iPi Studio's Configurable Jitter Removal
+
to a few clips. I have to say, this mocap looks incredibly solid!
+
The results are far superior to the data
+
I was getting using a single Kinect.
+
 
+
* [http://www.ipisoft.com/forum/viewtopic.php?f=2&t=5509&p=9373#p9373 TrueBones]
+
I have successfully calibrated the 180 degree angle with iPiSoft.
+
It is great, it calibrates at 180 degrees producing
+
a full 360 Hologram of myself in iPiSoft.
+
The tracking is rock solid and the results are out of this world.
+
I am utterly astonished.
+
http://www.youtube.com/watch?v=WTyzFGJNGEo
+

Latest revision as of 19:13, 19 February 2014

For the impatient:

  • Get two Microsoft Kinect sensors, or ASUS Xtion Live sensors, or PrimeSense Carmine 1.08 sensors
  • Get computer with DirectX 10 videocard and at least two USB 2.0 or 3.0 controllers
  • Download iPi Recorder and iPi Mocap Studio
  • Install iPi Recorder and iPi Mocap Studio
  • Find a suitable space (9 by 5 feet minimum)
  • Actor should be dressed in casual slim clothing, avoid shiny fabrics
  • Connect depth sensors to PC
    • Note that both sensors can't be connected through one and the same USB 2.0/3.0 controller
  • Point them at the actor (the angle between two depth sensors can be in the range 60-90 degrees, or near to 180 degrees)
  • For calibration, take flat rectangular cardboard, and use iPi Recorder to record calibration video
  • Please ensure that the calibration board is visible in both sensors. Turn the calibration board to the left and to the right, holding it vertically in front of yourself on outstretched arms.
  • Record video of actor's performance. Please ensure that the whole body including arms and legs is visible during the performance in both sensors. Start from a T-pose. Then goes actor’s performance.
  • Run iPi Mocap Studio to process actor performance video
  • Import your character into iPi Mocap Studio (File >Import Target Character) to adapt the animation for your character rig
  • Export your animation in a desired format


For more details see User Guide for Dual Depth Sensor Configuration.

Personal tools