Rendering Dynamic 3D Scenes in an Augmented Reality Using a 2D-marker

By | January 3, 2017

We continue to describe our R&D Hackathon experience.

    This technology has several interesting use cases.

  • Advertisement (to receive more information about goods);
  • Construction (to animate what is behind a wall: pipes, wires, etc.);
  • Video games (more fun).

An Example 3D Application

A customer chooses goods in a shop. He sees a T-shirt he’s interested in and a label near this T-shirt. The customer points his smartphone camera at the label and sees on the smartphone screen some animated 3d-scene in Augmented Reality. The scene could be represented in any way, like a rotating 3D T-shirt with additional info or some animated character who starts to talk about the item.

The team concentrated on implementing a basic proof of concept: to show a 3D figure in AR after directing a smartphone on a certain 2D marker.

To show a 3D figure in AR after directing a smartphone on a certain 2D marker

Tools for App

OpenCV, Java and Android Development Kit were used to make a simple prototype.

    The OpenCV library has several benefits. It is:

  • Multifunctional,
  • Well tested,
  • Has clear APIs,
  • Crossplatform.

Java and Android are widely used and is well adapted for quick prototypes.

How It Works

The application scans a 2D-marker in a small frame, from top-left corner to right-bottom. Then whole marker is built in memory. After this, the program compares the 2D-marker with a database set of 2D-markers and corresponded 3D-scenes. If a match is found, the OpenCV library starts to render a 3D scene.

Implementation Notes

As it often happens, the main time expense was in setting up. Installing and building the project and installing all the libraries. The team prepared a virtual machine (lightweight Ubuntu) prematurely and installed the Android SDK and NDK. Then, the project on Github was created, where necessary libraries and frameworks were included.

During 4 hours that was provided for the Hackathon the team implemented recognition of a simple marker and rendering of a simple 3D-geometric figure on this marker.
To scale, we could further implement and test recognition of a complex markers.

If we did it again…

    The team concentrated mostly on the technological and algorithmic parts of the task. Next time:

  1. A plan for execution is critical to the success of the project including how to break down the project and delegate to team members.
  2. Peripheral tasks, like project creation or addition of libraries should be solved before the Hackathon.

Leave a Reply

Your email address will not be published. Required fields are marked *