Camera Tracking Pro for 3D, Game, Film and Television (iPhone/iPad)

Attention CG Artists

1. Do you work for the 3D, Game, Film or Television industries using apps like Blender, Autodesk Maya, 3ds Max, Unity3D, LightWave, Cinema 4D, Houdini, Adobe Premiere, or any other 3D/2D application?
2. Do you want to have a professional camera tracking app, so you can use your iPhone to shoot videos and track all rotations and translations you do while shooting the videos and export everyting later to your 3D scene to easily sync live shots and CGI?

Available for iPhone/iPad

How to use Camera Tracking Pro for iOS

1. 1. Detection Mode – Choose between Horizontal, Vertical or Target detection.
   2. Coaching – Toggles the coaching mode on and off.
   3. Help – Shows you this help.
   4. Export – Select this to export your files to Collada (.dae) and Maya ASCII (.ma) formats.
   5. Universal Time SMPTE 60 FPS NON DROPFRAME timecode;
   6. Recording Time Counter – Shows the recording time.
   7. Frames Per Second – Shows the capturing speed in frame per seconds.
   8. Color Temperature – The scene color temperature.
   9. Lighting Conditions – Shows the environment’s lighting conditions. Green is good.
   10. Tracking Status – Again, green is good. Other colors mean that the app lost track of the environment or is about to. Move the device too fast may cause the tracking to perform poorly.
   11. Detecting Status – the options are: HORIZONTAL PLANES (the engine will try to find horizontal surfaces, like floors, tables, carpets, rugs, roads, etc), VERTICAL PLANES (the engine will try to find vertical surfaces like television screens, outdoors, facade of buildings, walls, etc) and TARGET IMAGE (the engine will try to find the target image).
   12. Coaching Status – shows the coaching status.
   13. Target Image – This thumbnail shows when Camera Tracking Pro for iOS is working in Target Image mode. Download and print this target image, and attach it to the surface you want to detect. If the surface is meant to be tracked from far away, print this image big.
   14. Recording Button – Tap to start/stop recording.

Best Results

Camera Tracking Pro for iOS works well if the surfaces you are tracking are still and have no movement.

The app uses the latest cutting edge algorithms, machine learning techniques and 1000 frames per second readings from the iPhone’s accelerometer and gyroscope to track the device movement based on fixed references on the environment, like floors, rugs, walls, tables, scenery elements, outdoors, buildings, roads, etc.

Camera Tracking Pro for iOS likes things that are still and do not move and have textures or elements that don+t change. Televisions, tablets, smartphones, etc., with moving images are not good. A still image on the screen of those devices is good for tracking.

This is necessary because tracking a scene requires the app to grab textures that it will try to identify from frame to frame. If a textures changes, the tracking cannot recognize the next frame and loses the tracking.

An example of bad texture is a white wall lit by a strong sun light. This will be seen by the app as a white blob of light with no texture. Again, no texture to grab on.

Examples of good tracking are: a poster on a wall, an outdoor, a painting, rugs with defined patterns, a sheet of white paper on a wood desk, a book, tiles, etc. The idea is to find a good horizontal or vertical “rectangle” that are not moving and have a good non-changing texture and to avoid regions with excessive or insufficient light.

Start Tracking

Before using the app verify that the items on the next list are satisfied:

1. Is the scene well lit?
2. Are the detected objects still the whole time?
3. If you are using the target image: is the target image attached to a still object and the paper itself is not moving relatively to anything?

If everything is according to the list, proceed to the next section.

Detecting a Surface

Run Camera Tracking Pro for iOS.

You will probably see the following animation on your iPhone:

Every time you see this animation it means Camera Tracking Pro for iOS is trying to locate an element or lost the tracking. It may be a horizontal or a vertical plane.

This animation and the process behind it is called “coaching”. This helps you know what you have to do to make Camera Tracking Pro for iOS identify, locate and grab a surface for tracking.

Move your iPhone slowly from side to side until the app locates a surface. When it does you will see an arrow pointing to it, like in the picture below.

As soon as the arrow appears, you can now tap the Record Button (11) and start moving and rotating your iPhone. Tap the record button again to stop recording.

All trackings are saved internally and can be accessed by tapping the Export Button (3).

Exporting, Sharing and Deleting a file

Tap the Export Button (3) to see the files you have already saved.

You will see something like the next picture:

Slide an entry to the left to see the options Rename, Share and Delete.

Exporting, Sharing or Deleting multiple files

If you want to share or delete multiple files, all you need to do is to select multiple entries and tp on the proper button at the navigation bar.

Changing the Detection Type

To change the detection type, tap on the Detection Button (1).

The options are: Horizontal Planes, to start detecting things like tables, floor, rugs, tiles, roads and everything that lies horizontally, Vertical Planes, to start detecting things like walls, paintings hanging on walls, outdoors, building facades and everything that lies vertically and Target Image, to detect the image we have provided at the beginning of this document.

Tracking Example

In the next video, we show an example of tracking we did and how it corresponds to real data.

Another example:

Importing Camera Tracking into Maya

In the next video, we show how to import the tracking files from Camera Tracking Pro for iOS into Autodesk Maya.

Importing Camera Tracking into Blender

Importing Carmera Tracking into Unity3D

Unity3D has a problem importing Collada Files containing certain types of animations.

To work around this do one of the following:

1. If you have Maya installed on your system just import the Maya file provided by Camera Tracking Pro into Unity 3D. For some reason Unity 3D needs Maya to be able to use its files.
2. If you don’t have Maya installed on your system, visit this page at Autodesk, and install the latest version of FBX2013. Use that converter to convert the Collada file provided by Camera Tracking Pro to a FBX file, that can be imported into Unity 3D. You can also use one of the links below:

Windows

• • FBX 2013.3 Converter for Windows
  • FBX 2013.2 Converter for Windows
  • FBX 2013.3 Converter for Windows 64-bit
  • FBX 2013.2 Converter for Windows 64-bit
    

Mac

• • FBX 2013.3 Converter for Mac
  • FBX 2013.2 Converter for Mac

Importing Camera Tracking into Houdini

When you import a Collada file into Houdini, you will see the following error message:

Press OK and ignore this error. Houdini imports the tracking files correctly.

NOTE: SideFX confirmed that this is a Houdini problem, as their app is not prepared to receive modern versions of Collada file. They inform that they have no plans to update the Collada import module.

The only thing you have to change is this:

1. Double click on collada_scene on the hierarchy and select the node called CameraTrackingPro4iOS_SuperNode.
2. Change its rotation X from -270 degrees to 0 and the camera will be positioned correctly.

Technical Specifications

• Tracking performed using the latest Machine Learning and Augmented Reality techniques;
• Rotation and translation samples are grabbed from the accelerometer, gyroscope and altimeter at 1000 samples per second;
• Trackings are saved at 60 frames per second;
• Generates Quicktime movie with MP4 video with AAC Audio. Video contains UNIVERSAL TIME SMPTE 60 FPS NON DROP FRAME embedded in the movie.
• Files are exported using rotation matrix to Collada Files, no gymbal lock problems;
• Files are exported using Euler Angles to Maya ASCII files.
• Application is extremely easy to operate: just attach the iPhone at the same rig your video camera, run Camera Tracking Pro for iOS and that’s it.

Available for iPhone/iPad