Augmented Reality: A story of how graphics became real

Games have been the main drive of real-time graphics technology up from the start with 2D graphics. But there was this sense of interaction with the digital world that was missing, that’s when 3D games started being available at consumer level in the 90’s. Interaction and care for graphics kept getting better with different GPU technologies and graphics APIs. The idea that the phone became aware of its surrounding popped up back before the smartphone age; the moment we saw a camera enabled phone and developers working on JAR and SIS solutions that makes the camera interactable. We would like to take you on a journey of how Augmented Reality became the phenomenon it is today.

The history of orientation sensing

Back in those days the camera was the only sensor. There was no luxury of having an Accelerometer or Gyrometer. Developers only used algorithms that basically tried to figure out how the phone was moving based on the motion blur or differentiation at pixel level. This was a pretty important achievement, because those kinds of games were technically the first motion based games that didn’t use a special kind of sensor.

Touch phones started popping up earlier this decade with the iPhone; showing revolutionary technologies in the usability of touch screen (fun note: LG Prada was actually the first phone to have a capacitive touch screen, not iPhone as it is commonly known). As touch screens began to be the new norm, the need to understand the orientation of the phone rose. Since phone keys can be shown in both ways, that set up a need for understanding orientation. Developers began to work on an accelerometer, which provided orientation values over, X, Y and Z axis (landscape and portrait) taking us a step further closer to Augmented Reality.

The introduction of the Gyrometer

The Gyrometer totally changed the game. It gave you true X, Y and Z axis rotation values. The first device to actually use a Gyrometer was the Wiimote. By Nintendo, it was a controller for the renowned console Wii, which made motion controlling available at consumer level with sports, dance and interaction at body level. The technology’s implementation was so remarkable to the point that we still use the same techniques on other console controllers and modern smartphones. Now, we have phones with full capabilities to understand their surroundings, orientation and heading.

Augmented Reality then

Ever since phones started having Gyrometers, using its capabilities in graphics rendering was there, but was frustratingly limited. Until Augmented Reality which is, in general, a concept in rendering technologies, where the device (acting as a camera) tries to “augment” its digital rendering on top of a stable environment (be it a marker, ground or even convex 3D shapes).

Basically the phone understands its surrounding and works on rendering objects based on the given environment. Optimally this achieves the level of augmentation and performance that lets the user feel the magic presented. Now that Augmented Reality technology is here, let’s examine it then and now.

Marker-tracking AR

This video from ten years ago really shows where Augmented Reality started out. That was purely using marker tracking using ARToolkit; an AR open source kit that was made for easier implementation over platforms. The idea is simple: you point the phone to a marker you have that is supported by the downloaded application and you get the Augmented Reality experience.

This is a simple in-house demo for what Augmented Reality technologies have become, supporting virtual buttons. You can truly see the interactions provided and the lighting effects along with better shades and smooth animations.

Marker-less AR

Mobile devices got better at processing powers relatively fast, opening up the opportunity for marker-less AR using compasses, maps (on a large scale) or ground detection (on a small scale). This opened up the market for exciting Augmented Reality games for enthusiastic gamers.Not to mention new Augmented Reality shopping apps to fill in the gap between shopping IRL and online shopping.

Augmented Reality now!

This video from ten years later really shows the endless possibilities of where we could go. Here Google is demonstrating a larger scale Augmented Reality experience. Using compasses and GPS capabilities combined with Gyrometer values. Pedestrians could have a more fun, functional and advanced user experience through this AR Google Maps.

Augmented Reality in the marketplace

Now, companies are using AR not only for games, but also to show of their products and services. For instance, IKEA created the IKEA Place app that uses ground detection to put down furniture and show how it fits in real size. This app helps you match your new furniture with your existing home decor, while trying different combinations and options.

“IKEA strive to make good design accessible to everybody. The new AR app, IKEA Place, represents the latest effort towards this ambition, to perhaps change the way people buy furniture forever.” -IKEA

Using the same concept, Niantic released Ingress and later the more popular Pokémon GO. This Augmented Reality game gained huge success making Pokémon fans fantasize about collecting rare Pokémons. It fired up old dreams using the gameplay that revolves around going places, collecting Pokémons and gathering around in GYM points, all using the same technology for large scale Augmented Reality.

Augmented Reality is new and vast topic. Later on, we are going to explore ideas and technical topics to cover what we are, at Code95, trying to achieve with AR technologies. Keep checking our blog for more!