The changing landscape of augmented reality

[Rss Bot 1]

Augmented reality has been around for longer than we think, but it's still a new field for most of us. Head-mounted displays with visually coupled systems were being used by the military back in the 1960s and 1970s. 

• Top tips for scaling up AR apps

But it wasn't until the 1990s that we started having motion-stabilised displays and fiducial marker tracking, like ARToolKit with its now famous Hiro marker. With the arrival of location-aware phones in 2008, companies like Layar started working on 'outdoor information overlay'. In its December 2009 issue, Esquire featured AR on the cover, catching the public's attention.

It still took a while for AR to really come into the public's imagination. And this happened… with a Pokémon. Well, with lots of Pokémon. The Pokémon GO app exploded on the scene in 2016. Everybody had to catch them all! But most importantly, people from outside the tech industry started talking about augmented reality.

Pokémon GO was instrumental in bringing the world of AR to the masses

So today AR is trending. But why? Maybe it's because we have dreamed about the possibilities, and now we are starting to consider them as real. This tech could actually have useful purposes that will help society develop. AR taps into our natural tendency to want to evolve even further. Plus, it has that cool factor that will help promote it for massive user adoption, for widespread uptake around the globe.

Who are the big players in AR?

With Hollywood and the gaming industry behind it, it's no wonder that augmented reality is considered a hot topic. But there's more. Another reason why AR is becoming popular is the financial backing, support and innovation from big players in the tech industry. This plays an important role. AR is predicted to have a market revenue of $90 billion by 2020, with VR largely behind with a $30 billion forecast.

So let's take a look at some of the contributions of the big players in the field, from early successes to the most recent developments.

Google 

Google introduces its Glass Enterprise Edition 

Google has made so many efforts in the field of AR and VR that it's hard to know where to begin. It released Google Glass in June 2013, and although some pretty interesting developments have been made – for example in wearable socio-affective aid and disability – these smartglasses haven't been a major success. Last year, it announced an enterprise version of Google Glass so we'll have to wait and see. 

The long but steady path into AR continued with Project Tango, back in June 2014. Although discontinued, Project Tango was one of the most advanced AR computing platforms seen. Google Cardboard was way more successful than Tango; its low-budget and accessible feel appealed to users around the world. There have been more than 10 million Cardboards shipped. And although you can technically use it to display AR, it was first conceived (also in June 2014) as a VR device. Thanks to its success and popularity, Google announced in May 2016, that it would focus its efforts on building a new, improved and enhanced platform for VR, called Daydream. 

Finally, its latest effort in AR has been the greatest to date. Enter ARCore. 

The coming of ARCore

ARCore brings AR to Android devices

In August 2017, Google shook the augmented world by publishing ARCore. Android users finally had their own AR platform. With ARCore, people started creating AR experiences on a massive scale. 

So how does ARCore work? First of all, it has stable motion tracking, an advanced inside-out tracking with monocular camera. Google was not the first to make AR available on a monocular camera; Apple had introduced it before with ARKit (more on that later). 

ARCore also has hit-testing to detect some planes (floor, table, wall) in the space around you and give you their location (environmental understanding). Another important feature is real-world light estimation. ARCore can estimate ambient and directional light. This is key because with good lighting AR merges smoothly in the real world. ARCore is currently available on Google-specific devices like the Pixel, but it will soon be open to millions of Android devices. 

The consolidation of WebAR

Another major contribution from Google is the importance and relevance that it has given to WebAR. Efforts existed before (open-source libraries like AR.js) but Google has made the focus shift and the world notice. Before, when people considered AR they thought only about HMD devices and funny-looking glasses, but now, Google has consolidated AR on the web. It has built two apps to expose both ARCore and ARKit with web tech: WebARonARCore and WebARonARKit. So, by exposing AR to a modified version of Chrome, developers only need a single API to reach both markerless tracking platforms. 

Ultimately the goal is to create a WebAR standard, much like the WebVR standard that you can use to write cross-platform AR applications. Now we can reach ARCore and ARKit from a web browser, and Google will probably include that in a Chrome implementation. Exciting times for WebAR.

Microsoft 

HoloLens is horrifyingly expensive but a great AR platform

Microsoft HoloLens was introduced to audiences in March 2016. It was one of the first modern AR head-mounted displays or smartglasses. The HoloLens offers precise high-quality tracking. The advantage of transparent devices like the HoloLens is that you don't have to hold the device in your hand (like you would a phone or tablet). Applications include a catalogue of 'holographic' 3D objects that you can play around with, and the HoloStudio to create them. It uses mostly Unity and Vuforia to build the apps.

Microsoft has also implemented Skype and Cortana, its virtual assistant, into the HoloLens. And of course, there are a lot of gaming applications that you can try. All in all, the HoloLens is a very interesting device – the price is currently a bit restrictive but it's still a great platform to play with. 

Apple

Apple has just revealed ARKit 2, making it possible to create shared AR experiences

Early in 2017 Tim Cook, CEO of Apple, made some comments on AR. However, it was all very non-committal. Apple had previously made many acquisitions in the AR field, but it had never got round to releasing anything. Then in June 2017, ARKit was announced. It was also integrated on the iOS 11 release. ARKit introduced monocular camera support for AR on the iPhone. Markerless tracking and the ability to do hit-testing on the real world were also part of the package. 

From June 2017, until the release of ARCore, Apple was the leader in the AR world. Not because of the tracking accuracy of its tracking technology: HoloLens and Project Tango had better tracking, but they both required specific hardware. Apple was the best because you got AR in your pocket by default… as long as you had an iPhone. 

ARKit basically offers the same features ARCore offers, but for iOS users. One difference is that Google uses a process called COM, concurrent odometry and mapping, to understand and map the world, while Apple's ARKit uses VIO, visual inertial odometry, to do the same thing. However, iOS is only 12 per cent of the world's mobile market. That's why we needed AR on Android for the other 88 per cent.

ARKit was definitely a key step towards massive user adoption. Everybody was rightfully impressed. But, the thing about Apple is that it likes to keep a tight grip on what it does, meaning that it provides closed – rather than open-source – solutions. Apple was pushing for native App Store and less for WebAR. Even so, ARKit is pretty slick and there are some terrific demos online made with Unity. 

Mozilla

The year 2017 was great for AR development, and following Google and Apple's lead, Mozilla had to throw its hat in the ring. A-Frame, its easy to learn web framework for building VR experiences, has been gaining massive popularity since it was released in July 2017. 

In October 2017, Mozilla also announced a new development program for mixed reality that "would expand their work in VR and AR". It drafted a WebXR API proposal. Later it started experimenting with WebAR on iOS with ARKit. The result: a WebXR viewer app available for download on iPhone devices. The app lets you view web pages created using Mozilla's own JavaScript library.

Facebook

Facebook started dabbling in VR a while back when it bought Oculus Rift for $2 billion in March 2014. Now it has officially started its incursion into AR. First, it announced a developer platform for its AR 'World Effects' (similar to those of Snapchat filters). Then, in December 2017, it finally opened the beta version of the platform to all devs. Facebook is allowing them to use precise location, object recognition and depth detection to get creative. Developers all around the world can start building AR experiences for the Camera Effects Platform, which includes 'World Effects' that augment your environment with 3D objects rather than just your selfies.

Magic Leap 

Magic Leap finally announced its first product in December 2017. The Magic Leap One is an AR headset to compete with devices like Microsoft HoloLens. It includes a pair of round-lensed goggles with audio and external camera sensors called Lightwear. It also includes a handheld controller with a touchpad and a "round computer system that clips to your body" called Lightpack. The Creator SDK is available now. 

Who else is out there?

Besides the ones already mentioned, many other companies have developed their own AR services and products. Unity has been very active in perfecting its multiplatform for AR and VR. Actually, over 91 per cent of HoloLens experiences are made with Unity. More than two-thirds of all content created for AR and VR uses its engine. It offers support for Vuforia, and there are Unity plugins for ARCore and ARKit. 

Another company building an editor specifically targeted towards AR/VR experiences is Amazon, with Sumerian. The premise is that it's quick and easy to use without requiring any specialised skills or 3D graphics expertise.

Other companies that have produced AR include Epson with Moverio, Sony with SmartEyeglass, IKEA with its Place app, Snapchat with Lens Studio, Blippar, Meta, Wikitude, Augment… to name just a few. Even NASA has been making use of AR/VR for several projects on space research.

The market and its applications

Now that there is more visibility around AR, an increasing excitement about its possibilities, and that the tech is made available to more people, what's next? Well, one word: creation. Content creation. This means building applications for the many fields where AR can play an important role. Let's take a look at some of the ways society is making use of AR.

Education

AR is already making changes in the way we learn. Apps for education are being deployed in classrooms around the US and some parts of Europe. Imagine children having trouble being attentive at school. Imagine that instead of reading a boring textbook, they see history happening live in AR. Or mixing elements in chemistry and seeing the reactions fume or explode. The classroom is way more appealing this way. The love of learning will come back to our schools. You can learn with your phone, which means there are fewer materials to buy, the retention rate will be higher, lessons will become more interactive and foster intellectual curiosity. Children can benefit greatly from apps like Aurasma, Quiver, or the amazing flashcards and anatomy app from Octagon Studio, which are both recreational and educational.

AR in the workplace

AR in the workplace has also started to be implemented. Industrial applications are being tested on factories to teach workers how to use the machinery, or to extract information like the current state of the equipment, temperature, location, etc. Obviously this is very interesting for industrial customers because it minimises error and cost. It can also help with maintenance procedures. The Microsoft HoloLens has been used by thyssenkrupp engineering for elevator maintenance, and by Volvo for complex assembly. The same device was a finalist for the RIBA Stirling Prize 2017 with applications for architecture and BIM (building information modelling). Assuring quality is the most important thing for these industries, and the expert support, automation and even AI that AR can bring is a big advantage to be invested upon. 

Retail and advertising

AR for retail and advertising is also being developed. AR could reactivate this field. The amount of revenue is interesting enough for advertisers to be willing to learn how to use AR/VR editors and mobile apps. By being able to visualise the product in AR, the user will be more tempted to purchase it, or at least to interact with it. It will boost sales. In the case of e-commerce the experience can be triggered either from a website or from a printed ad on the street. All they need is a QR code to scan and launch the experience. Remember the window shop from Net-a-Porter or the Pepsi Co's unbelievable bus experience, and even Tesco has a AR app. Other companies like ModiFace have made a name for themselves by offering AR tech to beauty brands. Sephora has launched an AR app to try different makeup looks and products. The possibilities are endless. 

Gaming

Gaming is another field where AR has left its mark. Although mostly a VR thing, popular games like Pokémon GO are what actually made the consumer market massively aware of AR. But there are many other apps to choose from, like shooting games, horror games, sports games, even table and card games.

The future of medicine

AR is also present in other areas like medicine, where it could actually improve the way surgeons train or approach a complicated operation. The Gregorio Marañón Hospital in Madrid decided to use the HoloLens to help surgeons perform procedures on patients. With the device they can quickly see data from CT scans, ultrasounds, X-rays, and 3D models, all through an interactive holographic panel system. In April of 2017, a team of surgeons used the HoloLens to operate on a patient with a malignant muscular tumour, using the headset to visualise MRI and radiography information during the surgery. This is simply outstanding and promises some exciting things to come. 

The next step 

As we have seen, AR has a wide scope of application. Now we need to keep creating content and get that killer app. This killer app will help us find out how people will use AR in a meaningful way that transform their lives. We need to figure out how to make AR a tool of choice, easy to use and shareable. Also, companies need to focus on making transparent devices available for massive use. Maybe these devices will end up having the same market penetration as video game consoles. Manufacturers need to improve interaction and control of the 3D being displayed, making the gestures feel more natural and the devices themselves less cumbersome. 

AR will be perfected on mobile phones, localised in space (home, workplace, classroom, coffee shop) with high quality data visualisation and real-time sharing. WebAR will play an important role in this democratisation, creating visibility but especially opportunity for anybody to experience and create AR… and with opportunity comes change, and this change is closer than we think. There's a lot to look forward to. 

This article was originally published in issue 305 of net, the world's best-selling magazine for web designers and developers. Buy issue 305 here or subscribe here.

Related articles:

• How to code an augmented reality marker
• The best VR headsets for 2018
• VR: is it the future of design?

View the full article