direkt zum Inhalt springen

direkt zum Hauptnavigationsmenü

Sie sind hier

TU Berlin

Page Content

Indoor localization using augmented reality markers

Introduction

Localization outdoors is an old hat, especially with GPS that is included in even the cheapest smart phones nowadays. Indoor localization, however, is far more complex and still hot topic in the research community. Various approaches and technologies are being tested, ranging from optical to radio signal approaches. At TKN those radio based technologies are also evaluated and TKN actively contributes in cutting edge research.

However, to assess a given technology one always needs a reference system that is required to perform much better compared to the tested system in order to draw conclusions. Such a reference system had not been available so far and this was the motivation and initial point for this project.

Marker
Lupe

The basic idea of this student project was to design and implement an indoor localization system and evaluate it in terms of reliability and accuracy. The localization approach revolves around so called augmented reality markers which are characterized by different patterns of black and white squares arranged in a grid.

Such markers are well suited for pattern recognition approaches and can easily be detected using conventional classification methods.

Combined with other aspects of the available domain knowledge (e.g. size of the markers and position with regards to a given reference point) the following scenario visualized for an indoor localization system can be considered:

Lupe

Markers are placed on the ceiling and their absolute positions — with regards to a given reference point — are known. A rolling cart equipped with a camera aiming at the ceiling delivers streams of images from the markers to the pattern recognition software. This information is then processed within a software module and the absolute position of the rolling cart with regards to the reference point is calculated.

Design and Implementation

This project has been realized inside a framework called Robot Operating System (ROS).
It provides means to develop applications meant for robots and robotic interaction. ROS delivers a set of libraries (such as hardware drivers), which provide a basis to develop own applications: each runnable unit is referred to as a node. Nodes provide a specific functionality and can communicate with other nodes using the message-passing system, which is a topic-based implementation publish-subscribe scheme.

The point of departure in our software development can be summarized as follows:
(1)    Recognize a marker using pattern recognition from camera images
(2)    Estimate the relative position of camera with regard to that marker
(3)    Query marker’s absolute position (with regard to a reference point)
(4)    Calculate absolute position of the camera using information from (2) and (3)

The aforementioned rather simple algorithm is then extended to support recognition of multiple markers at a time and aggregation of all positions of step (4) to estimate the absolute position of the camera. Various ROS components are implemented and reused to realize this algorithm.

The functional blocks the system are depicted in the following.

Lupe

The implementation was done in Python and can be run on a usual notebook that can run the ROS eco system.

The functional blocks from the design transferred into the ROS world is shown in the following picture. Orange arrows represent the communication channels and blue arrows show the relation of a node to a particular channel.

Lupe

To realize this project some ROS components were designed and implemented, and finally integrated with other already available modules.

  • usb_cam:
    Is an available “driver for USB cameras that are supported by V4L”. This module makes it possible to capture raw pictures from USB cameras and publish it for other ROS components to use.
  • ar_pose “Augmented Reality Marker Pose Estimation using ARToolkit library”:
    Is an available ROS module which provides the functionality of recognizing specific markers from usb_cam’s image stream and estimating relative position of camera with regard to a seen marker.
  • tkn_localize:
    Is an implemented module which collects the information published by ar_pose, i.e. the relative position of the camera with regard to a seen marker, calculates the absolute position of the camera with regarding to reference point and publishes this information on a fixed interval.
  • tkn_aggregate:
    Is an implemented node which collects set of calculated positions by tkn_localize each time they are published, aggregates them together and estimates the position of the camera.

Aggregation algorithm

From a theoretical point of view it is sufficient to have a single image of a single marker in order to calculate the camera’s relative position. In order to improve the accuracy and reliability of the system it was decided to increase the number of markers that are being seen from one position at a time and aggregate the calculated positions.

Lupe

For identifying outliers Frank Grubbs proposed a statistical method that shall be used to increase the quality of the sample positions. The test statistic G is given by the following formula where Ȳ is the sample mean and the standard deviation is denoted by s:

Lupe

G is t-distributed and defined for the following hypothesis:

  • H0: No outliers
  • H1: One or more outliers

The Grubbs’ test can be applied iteratively until all outliers are removed. Finally, the mean is calculated and a 95% confidence interval is built around the mean. Assuming that the measurement procedure is not systematically biased, it comes to a very strong conclusion: With a probability of 95% the reference position will be covered by the confidence interval. So the confidence interval is an indicator for the accuracy the system currently delivers. If the sample is widely dispersed the confidence interval tends to be greater than it would be if all sample positions are closely together. In addition to that, the algorithm returns the mean as the best estimate for the true reference position

Lupe

Evaluation

The following picture shows the graphical user interface with the absolute marker positions aligned with the visible grid. The camera was seeing a couple of markers and its estimated position is also depicted in the picture (before the aggregation).

Lupe

Two test cases have been designed to evaluate the accuracy of the system: the first one examined the dependency between different angles and the accuracy of results and the second one studied the influence of the number of recognized markers. First the test cases will be described and then the results of the evaluation are presented.

For the evaluation a room has been equipped with 7 markers. The following figure shows the setup of the mounted markers. Using this constellation it is possible to measure the influence of different angles and numbers of markers.

Lupe
Lupe

Each green number is one estimated position with its displacement to the reference position, displayed in meters. Except two values, which are measured with only one marker, the accuracy is always below 20 cm.
The second test case was designed, to reveal the correlation between the number of markers that are spotted and the accuracy. The next figure gives an impression, how the number of markers influences the accuracy. With two recognized markers, the measuring is getting much better. The mean difference is about 11 cm to the reference position. Three markers only improve the result to 10.5 cm. To get a real accurate measurement, at least four markers are needed. Then the mean difference is only about 7 cm.

Lupe

Conclusion

The objective of this project was to create an indoor localization system using augmented reality markers. With the given equipment and the used setup of markers, it is possible to achieve an accuracy of about 7 cm. The setup is inexpensive because only widely-used equipment and open source software components are utilized. The more markers the camera is able to capture, the more precise is the estimated position.
As an outlook many applications can be considered using the developed indoor localization system. Some applications for example are guided tours of museums, malls and airport maps, store navigation or orientation in hospitals.

Zusatzinformationen / Extras

Quick Access:

Schnellnavigation zur Seite über Nummerneingabe

Auxiliary Functions

Participants

Frank Sager
Tristan Wagner
Pouyan Fotouhi Tehrani

Advisors

Jasper Büsch