NEW!  Version 3 of iPi Motion Capture is available. iPi Motion Capture 3 delivers new features for improved workflow including support of Kinect 2, improved arms tracking, simplified calibration and others. Read more >>

This documentation corresponds to version 2 of iPi Motion Capture. The latest documentation for version 3 can be found at https://docs.ipisoft.com/

User Guide for Multiple PlayStation Eye Cameras Configuration

From iPiSoft Wiki
Revision as of 14:58, 13 December 2011 by Pavel (Talk)
Jump to: navigation, search

Contents

System Requirements

iPi Recorder

  • Computer (desktop or laptop):
    • CPU: x86 compatible (Intel Pentium 4 or higher, AMD Athlon or higher), dual- or quad- core is preferable
    • Operating system: Windows 8, 7, XP SP3, Vista (x86 or x64)
    • USB: at least two USB 2.0 or USB 3.0 controllers
      For more info see USB controllers
    • ExpressCard slot (for laptops)
      Optional, but highly recommended. It allows to install external USB controller in case of compatibility issues between cameras and built-in USB controllers, or if all USB ports are in fact connected to a single USB controller
    • Storage system: HDD or SSD or RAID with write speed:
      • For 4 cameras at 60 fps, 640 x 480 resolution: not less than 70,4 MByte/sec
      • For 6 cameras at 60 fps, 640 x 480 resolution: not less than 105,6 MByte/sec
Note If your write speed is lower, you can work with 320 x 240 resolution and/or lower frame rate. Alternatively, you can use compressed mode (that gives 3-5 times lower required write speed, but the CPU performance may become a bottleneck)

iPi Studio

  • Computer (desktop or laptop):
    • CPU: x86 compatible (Intel Pentium 4 or higher, AMD Athlon or higher), dual- or quad- core is preferable
    • Operating system: Windows 8, 7, XP SP3, Vista (x86 or x64)
    • Video card: Direct3D 10-capable (Shader Model 4.0) gaming-class graphics card
GPUz example.gif

Software Installation

iPi Recorder

Before installation:

  • unplug all cameras from computer


Download and run the setup package of the latest version of iPi Recorder. You will be presented with the following dialog.

IPi Recorder 2.0 Setup.png

  1. Select needed components
  2. Read and accept the license agreement by checking appropriate checkbox
  3. Press the Install button to begin installation
Note. Most of the components require administrative privileges because they install device drivers or write to Program Files and other system folders. On Windows Vista/7 you will be presented with UAC prompts when appropriate during installation. If you plan to use iPi Recorder under user which has no administrative rights, you can pre-install other components separately using administrator's account.
Important!
  1. You can plug only one MS Kinect / ASUS Xtion / PrimeSense Carmine sensor to one USB controller. 1 USB controller bandwidth is not enough to record from 2 sensors.
  2. You can plug not more than 2 Sony PS Eye cameras to one USB controller, otherwise you will not be able to capture at 60 fps with 640 x 480 resolution.
For more info see USB controllers.


Upon installation is complete, iPi Recorder will launch automatically. Continue with user's guide to get a knowledge of using the software.

Components

If some of the components is already installed, it has no checkbox and is marked with ALREADY INSTALLED label. You should not install all optional components in advance, without necessity. All of them can be installed separately at later time. Components descriptions below contain corresponding download links.

  • iPi Recorder 2.x.x.x. This is required component and cannot be unchecked.
    iPi Recorder itself.

Important! You can plug not more than 2 cameras to one USB controller, otherwise you will not be able to capture at 60 fps with 640 x 480 resolution. For more info see USB controllers

iPi Studio

Note that if you have used iPi Studio before and have Internet connection on your computer then you only need to let iPi Studio to automatically update:
IPiStudio-auto-update.png


iPi Studio is a core software of iPi Desktop Motion Capture (iPi DMC) solution. It’s used to process pre-recorded video and export resulted animation into most popular formats. You can use 30 days free trial. To install iPi Studio please download and run setup program: http://ipisoft.com/downloads/iPiStudioSetup.exe. Then, follow instructions of the installation wizard.

As a result of installation, iPi Studio will be executed. You will be prompted to enter your license key or start 30-days free trial period:
Welcome to ipistudio dlg.png


For more info about license protection see License.

Recording Video with Multiple PlayStation Eye Cameras

Environment

Space

For a multiple PlayStation Eye configuration, you need a minimum of 13 feet by 13 feet space (4 meters by 4 meters). At smaller space, actor simply won’t fit into view of cameras.

For 640 by 480 camera resolution, capture area can be as big as 20 feet by 20 feet (7 meters by 7 meters). That should be enough for capturing motions like running, dancing etc.

Background

Light-color background (light walls and light floor) is recommended for markerless motion capture. iPi Desktop Motion Capture is designed to work with real-life backgrounds. A multi-camera configuration (3 cameras and up) can handle certain amount of background clutter. Please keep in mind that the system can be confused if your background has large objects of the same color as actor clothes.

Environment and clothing.jpg

Using a green or a blue backdrop may improve results, but you are not required to use a backdrop if you have reasonable office or home environment with light-color walls and bright lighting.

Lighting

For best results, your environment should have multiple light sources for uniform, ambient lighting. Typical office lighting with multiple light sources located on ceiling should be quite suitable for markeless motion capture. In a home environment, you may need to use additional light sources to achieve more uniform lighting.

Please note that the system cannot work in direct sunlight. If you plan a motion capture session outdoors you should choose a cloudy, overcast day.

Actor Clothing

Actor should be dressed in solid-color long-sleeve shirt, solid-color trousers (or jeans) and solid-color shoes. Deep, saturated colors are preferable. Casual clothes like jeans should be OK for use with markerless mocap system. iPi Desktop Motion Capture uses clothing color for separating actor from background and therefore cannot work with totally arbitrary clothing.

Recommended shirt (torso) colors are black, blue or green. Red is not recommended because red can blend with human skin color making it difficult for the system to see hands placed over torso. Black color is useful for reducing self-shadows on torso. If you have bright uniform lighting you can get better results with a primary-color (blue or green) shirt.

Recommended jeans/trousers color is blue.

Recommended shoe color is black.

iPi Desktop Motion Capture has an option of using T-shirt over long-sleeve shirt for actor clothing. However, simple long-sleeve shirt may result in more accurate motion capture.

Special considerations

Don’t stand too close to a wall. Shadows on the wall may confuse the system. If standing close to a wall is unavoidable you should use additional light sources to light the wall behind actor to minimize shadows.

System can have problems tracking push-ups and similar motions because of shadows on the floor. You can improve tracking of such motions by using additional light sources to light the floor.

Recording Process

Please record a video using iPi Recorder application. It supports recording with Sony PlayStation Eye cameras, depth sensors (Kinect) and DirectShow-compatible webcams (USB and FireWire).

iPi Recorder is a stand-alone application and does not require a powerful video card. You may choose to install it on a notebook PC for portability. Since it is free, you can install it on as many computers as you need.

Please run iPi Recorder and complete setup and background recording steps following the instructions:

iPi Recorder Setup

Framerate

It is recommended that you record all videos at maximum available framerate. High framerate helps reduce motion blur and capture fine details of the motion.

Maximum possible framerate for Sony PlayStation Eye camera is 60 frames per second. Sony advertises PlayStation Eye camera as capable of capturing at 120 frames per second but framerates over 60 FPS result in too much noise in PlayStation Eye camera sensor and are not usable for motion capture.

Framerate lower than 30 frames per second is not recommended for motion capture.

4 cameras at 320 by 240 resolution

A dual-core CPU should be fast enough for recording a 4-camera video at 320 by 240 resolution at 60 frames per second.

4 cameras at 640 by 480 resolution at 60 frames per second

A quad-core CPU is recommended for recording at 640 by 480 resolution at 60 frames per second. If you have a dual-core CPU you may need to configure a lower framerate and/or lower compression quality to be able to record video at 640 by 480.

6 cameras at 640 by 480 resolution at 60 frames per second

A quad-core CPU clocked at 2.0 GHz (or better) is recommended for recording at 640 by 480 resolution at 60 frames per second. You will also need to get additional USB controller.

USB controllers

All modern computers (e.g. dual-core and better) based on Intel, AMD and Nvidia chipsets have two high-speed USB (USB 2.0) controllers on board. That should give you enough bandwidth to be able to record with 4 cameras at 640x480 (raw Bayer format) at 60 FPS, or 6 cameras at 640x480 (raw Bayer format) at 40 FPS.

Under certain circumstances you may need to get additional USB controllers.

Scene Set-up

Three Camera Configuration

Recommended configuration for 3-camera setup is a half-circle:

Click to enlarge

Example:

Click to enlarge

Virtual view of the same scene:

Click to enlarge

Four Camera Configuration

You can set up 4 cameras in a half-circle or a full-circle configuration, depending on available space. You can improve accuracy by placing one of the cameras high over the ground (like 3 meters high).

Recommended configuration for 4-camera setup in half circle:

Click to enlarge

Example:

Click to enlarge

Six Camera Configuration

You can set up 6 cameras in a full-circle or a half-circle configuration, depending on available space. You can improve accuracy by placing one or two cameras high over the ground (like 3 meters high).

Recommended configuration for 6-camera full-circle setup:

Click to enlarge

Camera Setup

Install the cameras on tripods and connect cables.

Click to enlarge

Sony PlayStation Eye cameras do not have standard tripod mounting screw, so you will have to use some kind of ad hoc solution. The simplest approach is to fix the cameras to tripods using sticky tape.

When mixing active and passive USB cables, make sure cable connection order is correct (computer->active cable->passive cable->camera).

If you're using the PlayStation Eye camera, make sure you have the lens set to the wide setting.

Click to enlarge

Calibration

Calibration is a process of computing accurate camera positions and orientations from a video of user waving a small glowing object called marker (for color/color+depth cameras) or a slim rectangular board called calibration board (for depth sensors). This step is essential and required for multi-camera system setup.

Important. Once you calibrated the camera system, you should not move your cameras for subsequent video shoots. If you move at least one camera, you need to perform calibration again.

Importance of high frame rate

You should record calibration video at the same resolution as your action video and at the same (or higher) frame rate.

Calibration at a different resolution may lead to reduced accuracy because cameras usually have different minor distortions at different resolutions (caused by internal scaling algorithm).

Calibration at low frame rate may lead to reduced accuracy because of increased synchronization errors.

Glowing Marker

Mini Maglite flashlight is recommended for calibration. This is a very common flashlight in US and many other countries. Removing flashlight reflector converts it into an ideal glowing marker easily detectable by motion capture software.

Maglite.jpg

If you cannot get a Mini Maglite, you can use some other similar flashlight.

Noname flashlight.jpg

Step 1: Running iPi Recorder in calibration mode

Run iPi Recorder and choose one of the darkening modes in "darkening for calibration" list (for Sony PS Eye cameras)

Calibration-darkening.png

or set Exposure to reasonably small value (for DirectShow-compatible web cameras)

Calibration-exposure.png

This is important because it helps to reduce motion blur during calibration.

Video will look dim in calibration mode.

Important! Do not turn off the light in the room during calibration! This will not help the software but will make it harder for you to see what is happening on recorded video when you view it later.

Step 2: Record calibration video

Start video recording.

Move the marker slowly through your entire capture volume (front-top-right-bottom-left-back-top-right-bottom-left). Start from top and move the marker in a descending spiral motion.

Tip. The exact trajectory of the marker is not so important, just try to cover the whole capture volume, or at least its perimeter.

Tip. For successful calibration, several hundred frames are sufficient. Much more frames (like several thousands) will not produce more accurate results, but instead will increase the processing time and possible number of marker detection errors. So generally it is not recommended to record calibration videos of more than 1 minute length.

Tip. You should make the marker visible to as much cameras as possible at all times. Hold the marker in the straight arm away from your body. In a circle configuration, when approaching the boundary of the capture area, position the marker inside the area, and your body outside.

Calibration1.jpg

Calibration2.jpg

Calibration3.jpg

Put the marker to the ground at each corner and at the center of capture volume. At least 4-5 ground points are needed for the correct detection of the groundplane.

Calibration ground1.jpg

Calibration ground2.jpg

Step 3: stop recording and check recorded video

  • There is no significant motion blur (image of marker looks like a round spot rather than an ellipse or a luminescent line)
  • Most of the time marker is visible in at least 3 cameras and not completely obscured by human body

Step 4: Take note of height of your first camera over the ground.

Take note of height of your first camera over the ground. You will need this parameter later. If you cannot measure this height accurately, then at least make a rough estimation.

Step 5: process calibration video in iPi Mocap Studio

Strictly speaking, you can postpone processing calibration video until after you finished recording other videos (e.g. your action videos). However, it is a good idea to process your calibration video as soon as it was recorded because it helps you ensure that you have good calibration. (Later on, incorrectly recorded calibration video may affect your ability to process action videos. )

To process calibration video please do the following:

  • Load your calibration video to iPi Mocap Studio
  • Important. Adjust the Region of Interest to cover the part of video that contains the glowing marker.
  • Set the Diagonal Field of View (FOV) for your cameras on the “Scene” tab. If you use Sony PlayStation Eye or Logitech QuickCam 9000 cameras, leave the FOV value at the default 75 degrees.
  • Go to “Calibration” tab. Check “Auto detect initial camera positions” checkbox. Click “Calibrate” button and wait while the system finishes calibration.
  • Calibration algorithm may occasionally fail to find correct camera positions. If this happens, you should manually adjust initial camera positions to roughly match your configuration. The main thing is the correct order of the cameras around the capture area, and their approximate view directions.
    • Reset camera positions with one of the standard half-circle or full circle configurations that best suits your configuration.
    • For each camera that requires adjusting, switch to this camera using the toolbar button, and correct its position using controls on the “Scene” tab.
    • Uncheck “Auto detect initial camera positions” checkbox.
    • Click “Calibrate” button. This will rerun calibration process, without recomputing marker positions.

Follow video tutorial:


Resulting scene should look like this:

Calibration result1.jpg

Green points designate correctly detected 3D marker positions. Red points designate misdetected marker positions. 10-20% of red points should be considered normal. Calibration is good if you have at least 70% of green points.

Tip. When doing manual camera positioning before the calibration, you may find useful to match model's position with the actor on a video:

  • Select the frame where the actor is standing relatively straight.
  • Show the model by checking the "View > Skin" menu item.
  • Adjust the camera position to fit the model with the actor's image.

Step 6: Define ground plane

You need at least 3 points in 3D space to define ground plane. For each point, click on it in 3D view and press “Mark as ground” button.

Calibration ground3.png

Calibration ground4.png

WARNING: If you do not mark ground points then the ground plane is incorrect and there is no sense in using Foot tracking option and camera heights values.

Step 7: Set scene scale using camera height as reference

Now cameras in your scene are properly oriented relative to other cameras and relative to ground plane. But you still need to find one more parameter: scene scale.

Use camera #1 height over ground to set correct scene scale.

Calibration scale1.png

Note: Height of camera can only be used if ground plane is properly defined. If ground plane is not defined, you can use distance between camera #1 and #2 to set scene scale.

Calibration scale2.png

Step 8: Save calibration result into *.scene.xml file

Calibration xml.png

Recording Actor's Performance

Recommended layout of an action video

  • Enter the actor.
  • Strike a T-pose.
  • Action
T-pose.png Action.png

T-pose

It is preferable to have actor strike a “T-pose” before the actual action. The software will need T-pose for building actor appearance model during tracking. If you make several takes of one actor you do not need to re-record T-pose before each take.

When using the depth sensors, it is recommended to face the palms down, as it corresponds to the default orientation of the model's hand bones. When using color cameras, it is recommended to face the palms forward, as it helps the software in determining the right color for the model's hands.

Takes

Take is a concept originating from cinematography. In a nutshell, take is a single continuous recorded performance.

Usually it is a good idea to record multiple takes of the same motion, because a lot of things can go wrong for purely artistic reasons.

Iterations

A common problem with motion capture is “clipping” in resulting 3D character animation. For example, arms entering the body of animated computer-generated character. Many CG characters have various items and attachments like a bullet-proof vest, a fantasy armor or a helmet. It can be easy for an actor to forget about the shape of the CG model.

For this reason, you may need to schedule more than one motion capture session for the same motions. Recommended approach is:

  • Record the videos
  • Process the videos in iPiStudio
  • Import your target character into iPiStudio and review the resulting animation
  • Give feedback to the actor
  • Schedule another motion capture session if needed

Ian Chisholm's hints on motion capture

Ian Chisholm is a machinima director and actor and the creator of critically acclaimed Clear Skies machinima series. Below are some hints from his motion capture guide based on his experience with motion capture for Clear Skies III.

Three handy hints for acting out mocap:

  1. Don’t weave and bob around like you’re in a normal conversation – it looks terrible when finally onscreen. You need to be fairly (but not completely) static when acting.
  1. If you are recording several lines in one go, make sure you have lead in and lead out between each one, i.e. stand still! Otherwise, the motions blend into each other and it’s hard to pick a start and end point for each take.
  1. Stand a bit like a gorilla – have your arms out from your sides:
    Gorilla.jpg
    Well, obviously not quite that much. But anyway, if you don’t, you’ll find the arms clip slightly into the models and they look daft.

If you have a lot of capture to do, you need to strike a balance between short and long recordings. Aim for 30 seconds to 2 minutes. Too long is a pain to work on later due to the fiddlyness of setting up takes, and too short means you are forever setting up T-poses.

Takes

Because motion capture is not a perfect art, and neither is acting, it’s best to perform multiple takes. I found that three was the best amount for most motion capture. Take less if it’s a basic move, take more if it’s complex and needs to be more accurate. It will make life easier for you in the processing stage if you signal the break between takes – I did this by reaching out one arm and holding up fingers to show which take it was.

Naming conventions

As it’s the same actor looking exactly the same each and every time, and there is no sound, and the capture is in lowres 320*200, you really need to name the files very clearly so that you later know which act, scene, character, and line(s) the capture is for.

My naming convention was based on act, scene, character, page number of the scene, line number, and take number. You end up with something unpleasant to read like A3S1_JR_P2_L41_t3 but it’s essential when you’ve got 1500 actions to record.

Retrieved from "https://wiki.ipisoft.com/index.php?title=User_Guide_for_Multiple_PlayStation_Eye_Cameras_Configuration&oldid=529"
Personal tools