|
|
| (19 intermediate revisions by 3 users not shown) |
| Line 1: |
Line 1: |
| − | == System Requirements ==
| + | For the impatient: |
| − | * CPU: x86 compatible (Intel Pentium 4 or higher, AMD Athlon or higher), dual- or quad- core is preferable | + | * Get the Microsoft Kinect sensor<!--[XBOX] (for XBOX with separate power adapter or for Windows)[XBOX]-->, or ASUS Xtion Live sensor, or PrimeSense Carmine 1.08 sensor |
| − | * Operating system: Windows 7 (x86 or x64), Windows Vista (x86 or x64), Windows XP SP3 (x86 or x64)
| + | ** To choose see [[Depth Sensors Comparison]] |
| − | * Video card: Direct3D 10-capable (Shader Model 4.0) gaming-class graphics card (GeForce GTS 250 or Radeon HD 4850 or better recommended) | + | * Get computer with DirectX 10 videocard |
| − | * Microsoft Kinect Sensor (“Depth Sensor”) with separate power adapter | + | * [http://www.ipisoft.com/downloads_index.php Download iPi Recorder and iPi Mocap Studio] |
| − | ** ''If you bought "XBox 360 + Kinect" bundle then you need to [http://store.microsoft.com/microsoft/Kinect-Sensor-Power-Supply/product/9A4CFC08 order power adapter]'' | + | * [[IPi_Recorder_Installation|Install iPi Recorder]] and [[IPi_Mocap_Studio_installation|iPi Mocap Studio]] |
| | + | * Find a suitable space (9 by 5 feet minimum) |
| | + | * Actor should be dressed in casual slim clothing, avoid shiny fabrics |
| | + | * Connect depth sensor to a PC, point it at the actor and use [[iPi Recorder]] to record your performance |
| | + | * Please ensure that the whole body including arms and legs is visible during the performance. Start from a T-pose. Then goes actor’s performance |
| | + | * Run [[iPi Mocap Studio]] process actor performance video. Refer to [http://vimeo.com/24073249 iPi Soft Motion Capture Workflow] from Paul Stapelberg for instructions on how to |
| | + | * Import your character into [[iPi Mocap Studio]] (File->Import Target Character) to adapt the animation for your character rig |
| | + | * [[Animation export and motion transfer|Export your animation in a desired format]] |
| | | | |
| − | == Software Installation ==
| + | <noinclude> |
| − | | + | For more details see [[User Guide for Single Depth Sensor Configuration]]. |
| − | === iPi Recorder ===
| + | </noinclude> |
| − | ::'''Note''' that even if you have used iPi Recorder before you will need to perform actions below anyway. Just letting iPi Recorder to be updated automatically will not work, because drivers for MS Kinect Sensor must be installed in addition.
| + | |
| − | | + | |
| − | | + | |
| − | Before installation:
| + | |
| − | * unplug MS Kinect Sensor from computer
| + | |
| − | * deinstall libusb-win32 driver if it was used before (see http://ipisoft.com/forum/viewtopic.php?f=11&t=4932 for details)
| + | |
| − | | + | |
| − | | + | |
| − | iPi Recorder is a tool for recording video from web cameras and depth sensors. Download the setup package of iPi Recorder latest version: http://ipisoft.com/downloads/iPiRecorderSetup.exe. Run it. In the following dialog window make sure that item “PrimeSensor Driver (psdrv3) Registration for MS Kinect Sensor” is selected:
| + | |
| − | [[Image:PSensor-install-step1.png|center]]
| + | |
| − | | + | |
| − | | + | |
| − | ::'''Note''' that if you do not plan to use Sony PlayStation Eye cameras then you can deselect the appropriate item in the list.
| + | |
| − | | + | |
| − | | + | |
| − | Click “Install” button to start the installation process. You’ll be asked about installation of “PrimeSensor Driver (psdrv3) Registration for MS Kinect”:
| + | |
| − | [[Image:PSensor-install-step2.png|center]]
| + | |
| − | | + | |
| − | Please read license agreement and accept it by checking the appropriate checkbox. Then press “Install” button. As soon as you get the Windows Security screen, please select “Install this driver software anyway”:
| + | |
| − | [[Image:KSensor-install-step3.png|center]]
| + | |
| − |
| + | |
| − | At this point the driver will complete its installation and should not give any other errors:
| + | |
| − | [[Image:PSensor-install-step4.png|center]]
| + | |
| − |
| + | |
| − | After that iPi Recorder itself will be installed:
| + | |
| − | [[Image:KSensor-install-step5.png|center]]
| + | |
| − | <br/> | + | |
| − | [[Image:KSensor-install-step6.png|center]]
| + | |
| − |
| + | |
| − | | + | |
| − | After that, connect MS Kinect Sensor to your PC. Under Windows Vista and Windows 7 device initialization should be performed automatically and you’ll receive notification about it in the system tray area.
| + | |
| − | Under Window XP you’ll be asked three times (for “Xbox NUI Motor”, “Xbox NUI Audio” and “Xbox NUI Camera”):
| + | |
| − | * “Can Windows connect to Windows Update to search for software?”
| + | |
| − | * “What do you want the wizard to do?”
| + | |
| − | In all three cases answer “No, not this time” and “Install the software automatically (Recommended)” respectively:
| + | |
| − | <table border=0><tr>
| + | |
| − | <td>[[Image:KSensor-install-step7.png|center|334px|Click to enlarge]]</td>
| + | |
| − | <td>[[Image:KSensor-install-step8.png|center|334px|Click to enlarge]]</td>
| + | |
| − | </tr></table>
| + | |
| − |
| + | |
| − | | + | |
| − | Run iPi Recorder and make sure that there is “MS Kinect Sensor” in “Camera Selector” dialog:
| + | |
| − | [[Image:PSensor-install-step9.png|center]]
| + | |
| − |
| + | |
| − | Select “MS Kinect Sensor (OpenNI)” and press “Ok” button. Make sure that you can see colored “depth video” from sensor, that actual frame rate is 30 fps, that frames drop is in range from 0 to 5 and that you can change motor position using appropriate slider:
| + | |
| − | [[Image:KSensor-install-step10.png|center]]
| + | |
| − |
| + | |
| − | | + | |
| − | ::'''Note''' that values of Tilt and Roll are updated with a time lag. During recording these values cannot be updated assuming that in this case sensor should be static.
| + | |
| − | | + | |
| − | === iPi Studio ===
| + | |
| − | ::'''Note''' that if you have used iPi Studio before and have Internet connection on your computer then you only need to let iPi Studio to automatically update:
| + | |
| − | [[Image:IPiStudio-auto-update.png|center]]
| + | |
| − | | + | |
| − | | + | |
| − | iPi Studio is a core software of iPi Desktop Motion Capture (iPi DMC) solution. It’s used to process pre-recorded video and export resulted animation into most popular formats. You can use 30 days free trial.
| + | |
| − | To install iPi Studio please download and run setup program: http://ipisoft.com/downloads/iPiStudioSetup.exe. Then, follow instructions of the installation wizard.
| + | |
| − | | + | |
| − | As a result of installation, iPi Studio will be executed. You will be prompted to enter your license key or start 30-days free trial period:
| + | |
| − | [[Image:Welcome_to_ipistudio_dlg.png|center]]
| + | |
| − | | + | |
| − | | + | |
| − | For more info about license protection see [[License]]. | + | |
| − | | + | |
| − | == Recording Video from Depth Sensor ==
| + | |
| − | | + | |
| − | Finally, after all these preparations completed, you are ready for performance.
| + | |
| − | | + | |
| − | It is convenient to put MS Kinect Sensor on a chair or a table. The picture below will help you to understand possible capture area and required space:
| + | |
| − | <center><table border=0><tr>
| + | |
| − | <td>[[Image:KSensor-scene-side-view.png|center|385px|Click to enlarge]]</td>
| + | |
| − | <td>[[Image:KSensor-scene-top-view.png|center|394px|Click to enlarge]]</td>
| + | |
| − | </tr><tr>
| + | |
| − | <td><center>'''Side view'''</center></td><td><center>'''Top view'''</center></td>
| + | |
| − | </tr></table></center>
| + | |
| − | | + | |
| − | | + | |
| − | Please start iPi Recorder and choose “MS Kinect Sensor (OpenNI)” in Camera Selector dialog:
| + | |
| − | [[Image:PSensor-install-step9.png|center|250px|Click to enlarge]]
| + | |
| − | | + | |
| − | Using “Motor” slider, adjust direction of sensor so that you can see floor at the bottom and actor is fully visible (with hands up if needed):
| + | |
| − | [[Image:KSensor-recording-adjust-motor.png|center|457px|Click to enlarge]]
| + | |
| − | | + | |
| − | Set output directory for recorded video:
| + | |
| − | [[Image:KSensor-recording-dest-folder.png|center]]
| + | |
| − | | + | |
| − | Press “Start Video Recording” button to begin video recording.
| + | |
| − | | + | |
| − | ----
| + | |
| − | | + | |
| − | ::<font color="#880000" size="+1">'''Important!''' Initial two seconds of video should contain only background and static objects. Presence of actor or any moving objects in the initial two seconds of recorded video is unacceptable. It’s because initial seconds of video are used to calculate static background necessary to separate actor from other objects of 3D scene.</font>
| + | |
| − | | + | |
| − | ----
| + | |
| − | | + | |
| − | As soon as recorder starts and “clear” scene is recorded for a couple of seconds, go to the capture area and stand in a [http://www.trinity3d.com/media/axyz-design/metropoly2/T-Pose_models/Casual/MeCaT0005/image4.jpg T-pose]:
| + | |
| − | [[Image:KSensor-recording-TPose.png|center|451px|Click to enlarge]]
| + | |
| − | | + | |
| − | ::'''Note''' that current version uses only depth information to track motions. So there are no restrictions on clothes colors (just avoid shiny fabrics). Please use slim clothes to reduce noise in resulted animation.
| + | |
| − | | + | |
| − | After that you can act desired motions.
| + | |
| − | | + | |
| − | To stop recording, press “Stop Video Recording” button.
| + | |
| − | | + | |
| − | Using Windows Explorer, go to destination folder and open the recorded file with name <tt>video_''YYYY-MM-DD''_''hh-mm-ss''.avi</tt> in Windows Media Player. You should see depth information as a blue image:
| + | |
| − | [[Image:KSensor-bluevideo.png|center|358px|Click to enlarge]]
| + | |
| − | | + | |
| − | Now you are ready for processing this video using iPi Studio.
| + | |
| − | | + | |
| − | == Processing Video from Depth Sensor ==
| + | |
| − | | + | |
| − | * Run iPi Studio
| + | |
| − | * Press <tt>Ctrl+O</tt> or push “Open” button on toolbar
| + | |
| − | * Choose recorded <tt>avi</tt> file
| + | |
| − | * In the “Select Video Layout” dialog select the following values:
| + | |
| − | ** <tt>Depth video from MS Kinect Camera</tt>
| + | |
| − | ** Numbers of cameras: <tt>1</tt>
| + | |
| − | ** Resolution: <tt>640x480</tt>
| + | |
| − | | + | |
| − | [[Image:KSensor-process-step1.png|center]]
| + | |
| − | | + | |
| − | * After video opened, background will be calculated automatically and position of camera will be detected based on floor plane.
| + | |
| − | * Make sure, that floor is detected correctly:
| + | |
| − | *# Right mouse button down on 3d view port area
| + | |
| − | *# Move mouse, as a result 3d scene will be rotated
| + | |
| − | *# Right mouse button up
| + | |
| − | *# Visually control that 3d points from floor are located on ground plane which is marked by squared grid
| + | |
| − | | + | |
| − | [[Image:KSensor-process-step2.png|center]]
| + | |
| − | | + | |
| − | ::'''Note''' that you always can get back to the camera point of view by pressing <tt>'''1'''</tt> key on keyboard or pushing “Camera 1” button on toolbar
| + | |
| − | | + | |
| − | * Save created project by pressing <tt>Ctrl+S</tt> or pushing button “Save” on toolbar
| + | |
| − | * Position timeline slider to the frame where actor is in T-pose
| + | |
| − | * Adjust actor height using appropriate slider on tab “Actor”
| + | |
| − | * Select “Move” tool on toolbar.
| + | |
| − | * Move actor model to left or right to match roughly actor silhouette on video. Note that actor model can look smaller due to its position along axis of view. Don’t pay attention to it on this step.
| + | |
| − | | + | |
| − | [[Image:KSensor-process-step3.png|center|484px|Click to enlarge]]
| + | |
| − | | + | |
| − | * Switch to “Tracking” tab and push “Refit pose” button.
| + | |
| − | * As a result model should be matched with actor image from video. If it does not happen then delete result using item “Edit” -> “Delete pose” from main menu and repeat above actions.
| + | |
| − | | + | |
| − | [[Image:KSensor-process-step4.png|center|483px|Click to enlarge]]
| + | |
| − | | + | |
| − | * Using the slider right to the button “Show Skin” in toolbar, make sure that morph of model corresponds to the actor image. If no, than adjust arms/torso/legs morph using appropriate sliders from “Actor” tab.
| + | |
| − | | + | |
| − | [[Image:KSensor-process-step4a.png|center]]
| + | |
| − | | + | |
| − | * Set the beginning of Region-of-Interest (ROI) to the current frame with T-pose by pressing <tt>'''I'''</tt> key on keyboard or by double-clicking on the left edge of ROI bar under timeline.
| + | |
| − | * Switch to “Tracking” tab, check “Enable foot tracking” and “Enable shoulders tracking” checkboxes
| + | |
| − | * To start tracking just push “Track Forward” button
| + | |
| − | | + | |
| − | [[Image:KSensor-process-step5.png|center|483px|Click to enlarge]]
| + | |
| − | | + | |
| − | * Wait and watch...
| + | |
| − | | + | |
| − | == Next Steps ==
| + | |
| − | | + | |
| − | * [[Clean-up]]
| + | |
| − | * [[Animation export and motion transfer]]
| + | |
| − | | + | |
| − | == Video Materials ==
| + | |
| − | | + | |
| − | === Demo video ===
| + | |
| − | | + | |
| − | <html><iframe title="YouTube video player" width="480" height="390" src="http://www.youtube.com/embed/XvRP7qXCBCA" frameborder="0" allowfullscreen></iframe></html>
| + | |
| − | | + | |
| − | === [http://vimeo.com/24073249 iPiSoft Motion Capture Workflow] from [http://www.paulstapelberg.com/ Paul Stapelberg] ===
| + | |
| − | | + | |
| − | <html><iframe src="http://player.vimeo.com/video/24073249?title=0&byline=0&portrait=0" width="480" height="270" frameborder="0"></iframe></html>
| + | |
| − | | + | |
| − | === "Car scene" by [http://www.ipisoft.com/forum/viewtopic.php?f=12&p=7759 JeanDellac] ===
| + | |
| − | | + | |
| − | <html><iframe title="YouTube video player" width="640" height="390" src="http://www.youtube.com/embed/L7cGwPrZjh4" frameborder="0" allowfullscreen></iframe></html>
| + | |
| − | | + | |
| − | === Animation by NextGenReport ===
| + | |
| − | | + | |
| − | <html><iframe title="YouTube video player" width="640" height="390" src="http://www.youtube.com/embed/hlimlZAKnA4" frameborder="0" allowfullscreen></iframe></html>
| + | |
| − | | + | |
| − | === "First results" from [http://ipisoft.com/forum/viewtopic.php?f=12&t=4905 Radiac] ===
| + | |
| − | | + | |
| − | <html><iframe title="YouTube video player" width="480" height="390" src="http://www.youtube.com/embed/x_qNoJl-62M" frameborder="0" allowfullscreen></iframe></html>
| + | |
| − | | + | |
| − | <html><iframe title="YouTube video player" width="480" height="390" src="http://www.youtube.com/embed/u7lLL-J0SS0" frameborder="0" allowfullscreen></iframe></html>
| + | |
| − | | + | |
| − | See also:
| + | |
| − | * [http://www.youtube.com/watch?v=-HNvIkFyYUA header]
| + | |
| − | * [http://www.youtube.com/watch?v=S_RJNdvIZNo idle]
| + | |
| − | * [http://www.youtube.com/watch?v=uOzF4R5G0vs from idle to shot with left foot]
| + | |
| − | * [http://www.youtube.com/watch?v=niFeyZPD9PA from idle to shot with right foot]
| + | |
| − | | + | |
| − | == Frequently Asked Questions ==
| + | |
| − | | + | |
| − | '''Q:''' '''Is tracking realtime?'''
| + | |
| − | | + | |
| − | '''A:''' Not yet. You should capture video first and process the video offline after that. Speed of processing depends on your graphics card (this is a [http://en.wikipedia.org/wiki/GPGPU GPGPU] application so [[Cameras and accessories#Video_Card|a fast video card is recommended]]).
| + | |
| − | | + | |
| − | | + | |
| − | '''Q:''' '''Does iPi Mocap System support facial motion capture?'''
| + | |
| − | | + | |
| − | '''A:''' No, not yet.
| + | |
| − | | + | |
| − | | + | |
| − | '''Q:''' '''Can hands/fingers be tracked by iPi Mocap System?'''
| + | |
| − | | + | |
| − | '''A:''' No, not yet. Resolution of consumer-class cameras is not enough for that.
| + | |
| − | | + | |
| − | | + | |
| − | '''Q:''' '''What about head tracking?'''
| + | |
| − | | + | |
| − | '''A:''' Head tracking is not implemented as of yet but it is being developed.
| + | |
| − | | + | |
| − | | + | |
| − | '''Q:''' '''What output formats are supported by iPiStudio?'''
| + | |
| − | | + | |
| − | '''A:''' iPiStudio can export animations in many popular formats. For a full list please see [[Animation export and motion transfer]].
| + | |
| − | | + | |
| − | | + | |
| − | '''Q:''' '''Is FBX format supported?'''
| + | |
| − | | + | |
| − | '''A:''' Yes.
| + | |
| − | | + | |
| − | | + | |
| − | '''Q:''' '''Can I retarget captured animation to my own character?'''
| + | |
| − | | + | |
| − | '''A:''' Yes. iPi Studio has built-in motion transfer. For more info please see [[Animation export and motion transfer]].
| + | |
| − | | + | |
| − | | + | |
| − | '''Q:''' '''Can I use two or three Kinects with iPi Mocap System?'''
| + | |
| − | | + | |
| − | '''A:''' Not yet. Current version supports only one Kinect sensor.
| + | |
| − | | + | |
| − | | + | |
| − | '''Q:''' '''What drivers for Kinect are used in iPi Mocap System?'''
| + | |
| − | | + | |
| − | '''A:''' We use a lightweight solution based on the driver from PrimeSense (psdrv3.sys) to communicate with device via USB. Protocol implementation is our own.
| + | |
| − | | + | |
| − | | + | |
| − | '''Q:''' '''Is this driver compatible with OpenNI and Brekel plugin?'''
| + | |
| − | | + | |
| − | '''A:''' Yes. If you have already installed PrimeSense's OpenNI drivers for Kinect then there is no need to install drivers again thus you can deselect item “PrimeSensor Driver (psdrv3) Registration for MS Kinect Sensor” during iPi Recorder installation.
| + | |
| − | | + | |
| − | | + | |
| − | '''Q:''' '''Can I use OpenKinect drivers instead?'''
| + | |
| − | | + | |
| − | '''A:''' Yes, this is possible for compatibility but the performance will be sub-optimal. Install OpenKinect drivers and deselect item “PrimeSensor Driver (psdrv3) Registration for MS Kinect Sensor” during iPi Recorder installation. But note that OpenKinect driver is not stable and can cause a hang up of a slow computer.
| + | |
| − | | + | |
| − | | + | |
| − | '''Q:''' '''What does "iPi DMC" means?'''
| + | |
| − | | + | |
| − | '''A:''' "iPi DMC" is the abbreviation of "iPi Desktop Motion Capture". iPi Mocap System consists of:
| + | |
| − | * iPiRecorder - a tool for recording video from web cameras and depth sensors;
| + | |
| − | * iPiStudio - software application for processing pre-recorded video and exporting resulting animation.
| + | |
| − | | + | |
| − | | + | |
| − | '''Q:''' '''Is there a version for Mac?'''
| + | |
| − | | + | |
| − | '''A:''' To run this software on a Mac you need to install Windows on your Mac using [http://www.apple.com/macosx/compatibility/ Boot Camp] or similar solution.
| + | |
| − | | + | |
| − | '''Q:''' '''How can I buy this software?'''
| + | |
| − | | + | |
| − | '''A:''' You can buy [http://ipisoft.com/sales.php online].
| + | |
