Markerless GPU Accelerated Augmented Reality on Android Devices

This thesis describes an Augmented Reality (AR) framework for Android devices. Using the framework, virtual three dimensional objects are drawn in a view of the world captured with a camera, thus augmenting the reality. AR can for example be useful for visualisation or to assist with tasks. To enable a wide range of applications, the AR framework is developed for Android devices and designed to work in a wide variety of scenes.

To create AR, a virtual camera must be created which matches the real camera. To create a matching camera the camera extrinsic parameters, which are the position and orientation, and the camera intrinsic parameters such as focal length, need to match. This thesis describes how to automatically obtain both the camera intrinsic and extrinsic parameters. The extrinsic parameters are estimated by first detecting objects in the camera image and then using the position of these objects in the camera image to estimate the location and orientation of the camera.

The object recognition system is trained with reference images from objects after which it is able to robustly recognise objects under different viewing conditions such as different camera viewpoints and different lighting conditions. This is achieved by first finding matching keypoints between the camera image and the reference images. For the keypoint detection, description and matching several methods are evaluated in this thesis. Based on these matches an object recognition method is built which is able to reliably tell if and where an object is present in the camera image.

To create an immersive AR experience a high framerate is required. Therefore several optimizations are discussed, such as a graphics processing unit (GPU) implementation of the keypoint detection and description algorithm.

The resulting framework is a fully functional framework which incorporates all aspects required such as pose estimation and rendering. It shows that keypoint based object recognition is an effective way to create the required understanding of the environment needed to perform pose estimation. The framerate without GPU acceleration or tracking is just below one frame per second. Furthermore several methods are shown in which the framerate can be increased.