Alibaba and Tencent bringing their CNY 2017 red envelope (红包) war into the AR world is a key indicator that we’re facing what could be the next user & consumer experience revolution in China. Plus during the summer both Apple and Google released their own AR development platforms – ARKit and ARCore – enabling developers to quickly and easily create AR-based app experiences. And with iOS11 just released, expect to see augmented reality experiences exploding onto Chinese smartphones over the next 12-24 months.
Although most analysts predict that the enterprise AR market will lead the consumer market in the West, it would not be a surprise if the AR revolution begins with the consumer in China, starting from marketing and social use cases.
The bottom line is that AR experiences are getting easier to create and audiences are enthusiastically engaging with them.
Left: Tencent’s QQ red envelope game. Right: The original Alipay version.
Why should you care? Well – projections for the size of the AR market by 2020 hover around $90 billion, while the expected number of AR-enabled phones in 2018 is 900 million. Within both estimates, China accounts for the lion’s share. Of the 100 million people Gartner expects to be shopping in augmented reality by 2020, a majority will be Chinese. AR is also expected to play a key role in the current Alibaba-led move to merge online and offline commerce (see their acquisitions in traditional retail), resulting in highly integrated purchasing experiences.
Are you potentially at risk of missing out AR opportunities to engage with your Chinese consumers now? If you’re a consumer-facing brand with a large audience, then the answer is probably yes. The initial novelty of AR will offer excellent virality to the first wave of campaigns and experiences, but as with most new interaction innovation, the novelty effect will die out quickly.
So where does the overall WeChat AR experience currently stand in this revolution?
WeChat as of now already offers a couple of AR features built into the app: real-time translation, street view, book cover analysis.
But China’s most ubiquitous app makes up for its limitations with its 900M+ user reach, of which 80% access it on Android phones. Since WeChat essentially facilitates an AR access point in almost every smartphone owner’s pocket, brands can reach a large portion of their audience through AR campaigns on the app. This already makes AR far more viable than VR for mass consumption potential.
It’s true – there are still significant limitations to WeChat AR, but there are also workarounds. And as tech in this area improves over the remainder of 2017 and into 2018, friction will decrease and power available to creatives will increase. In WeChat’s crowded marketing space, a whole new arena is being born.
In this article, you’ll learn about the two main types of AR and the current state of evolution for each technology. Then, we’ll look specifically at what’s possible in terms of AR on WeChat. Finally, we’ll venture into the future, and look at what the world of augmented reality in China might look like between 2018-19.
For comprehensive technical information on how to get up to speed on WeChat’s AR capabilities, download our free white paper by clicking here.
Think of current day AR, and Pokémon Go is probably the first thing to spring to mind. Downloaded over 500 million times to date, the app was one of the most viral topics in 2016. But there’s the catch, Pokémon Go needs to be downloaded before you can ‘catch ‘em all!’. Which brings us to our first distinction:
Pokémon Go, or Snapchat’s ‘Lenses’, are good examples of native AR. Native to a downloadable mobile application, these kinds of experiences are powerful in combining fantasy with reality with recognition of complex environments encompassing objects, surfaces, textures, and lighting. On the downside, having to download the app creates friction for the user. The advent of ARKit and ARCore allows for easier creation of native AR experiences – giving more control to creative talents and decreasing their reliance on ultra-specialized developers.
In a nutshell: More suitable for frequent-use experiences that require full modern AR capabilities and provide enough user value to be worth downloading a >50Mb app. Think games like Pokémon Go, tablet-based advertising experiences, or IKEA’s ‘preview’ catalog app, “IKEA Places”.
AR shopping: The ‘IKEA Places’ app virtually places furniture in your home
On the other hand, Web AR experiences are accessed by simply opening the Internet browser on your phone (or WeChat! more on that later). These experiences are limited in capabilities compared to native AR. First, they need a marker- a physical symbol in the surroundings for the phone to recognize and trigger the information overlay. Secondly, they are not capable of depth perception and have limited object recognition capacities. But these limitations are being resolved fast. And with no need for an initial app download, Web AR offers an exciting prospect for almost frictionless and highly shareable AR experiences – with just a single touch or swipe. Apple’s release of iOS11 served as a real game changer in WebAR with Safari finally allowing in-browser (WebRTC) use of the camera feed. In short, this update brings AR to every recent Apple smartphone in China.
In a nutshell: perfect for simple and lower-frequency usage, such as O2O experiences designed for one-time marketing campaigns, events or visits etc.
If you’re on desktop, scan this QR with your Android phone to see Web AR in action
Web AR can be a great way to engage with consumers in a new dimension with low barriers to entry. Here are a couple of key use cases where it can be used in the marketing sphere:
A print ad for a new car model that allows the user to view the car in 3D, customize the ‘AR car’, and then book a test drive or make a purchase.
A portal door that allows a glimpse of a virtual world when opened.
Marketing Use Cases: Print advertising, mall advertising, event interaction, live 3D action with products, real estate, etc.
With an AR marker around the user’s wrist, they can virtually ‘try on’ a new watch through their phone.
Marketing Use Cases: Accessories, clothing, furniture, etc.
Left: Piloting a virtual flying car with a hand sensor. Right: Social mini basketball game
Marketing Use Cases: Brand awareness & social engagement campaigns.
Draw any picture on a piece of paper perfectly, just by following a projection of the picture made by your phone
Marketing Use Cases: Product usage guide, assembling assistance etc.
Above are just a few examples. Yet, common in all of them, there’s still a number of limitations to bear in mind when it comes to today’s web AR capabilities
In addition to the above limitations, Web AR abilities through WeChat’s browser are still very much in the experimental stages. Here’s what we know now:
> On Android, WeChat’s native web rendering engine, called ‘X5’, powers its web browsing. When we tested on this browser, the performance of Web AR was inferior to that on Google Chrome, especially in terms of tracking. However, as Web AR frameworks mature, expect the gap to close within the next few months.
> On iOS, access to Web AR directly through WeChat was still not available at the time of publishing. The implementation of WebRTC in iOS 11 allows, among other updates, the direct use of the phone’s camera inside a web page. But Apple is yet to enable WebRTC for its WebView components, which is used by third parties such as Tencent for its WeChat browser.
The good news is that there is a workaround. If Web AR faces any issues in WeChat, whether on Android or iOS, it can be detected and we can prompt the users to open the Web AR experience in their default mobile browser.
Directing the user to open the experience in the default mobile browser takes 2 additional taps.
Based on iOS10 reaching 50% penetration within a month of its release in 2016, expect that 90% of compatible devices will be running Apple’s latest OS – that enables WebAR in Safari – by the end of the year.
Apple’s decision not to open WebRTC to WebView is probably a move to avoid potential cannibalization of their native ARKit. But no doubt behemoth third parties such as Tencent will pressure Apple to allow their apps’ browsers access to camera and audio feeds. We believe there’s a moderate possibility of this limitation being removed within the next year- meaning Web AR will be accessible directly through the WeChat browser on iOS.
On Android, the X5 browser’s performance will continue to be optimized with the growth in number of experiences released and developers improving the software libraries available. The focus in the near future will be on enhancing performance levels to deliver better refresh rates.
We’re currently witnessing an explosion of close collaborations of B/A/T with major brands, extending them preferential access to exclusive or unreleased capabilities. We believe this trend is likely to continue with more opportunities for multinationals and major Chinese companies to leverage ties with Tencent to access WeChat’s app-specific AR capabilities.
Top: “Say it with Starbucks (用星说)” gift cards directly accessible from the WeChat wallet page. Screenshot source Chozan. Bottom: L’Oreal collaboration with Baidu’s AR lab Baidusee
Although Tencent is allegedly abandoning its own QAR platform project, we believe it is probably a strategic move to avoid having to build and maintain their own native AR framework up to the benchmarks set by Google and Apple. Instead, the priority may have shifted to integrating ARKit and ARCore SDKs as a base for AR projects. The strong ambition Tencent has shown by venturing into AR indicates that a third party-enabled AR platform is still on the table. Especially since it would allow mutualizing recognition capacity across a wide range of use cases, in a privacy/data sharing trade-off with brands and businesses using the platform.
Presentation of Tencent’s QAR platform capabilities and integrations at the SIGGRAPH conference, December 2016
And there is a good chance Mini Programs will play a key role in WeChat AR capabilities as they allow the harnessing of device processing power over that of browsers, making for smoother AR experiences.
It can be difficult to judge when the time is right to jump on the Web AR bandwagon. Now? In three months? In six months? Here’s our simple guide for figuring out how soon you should make a move:
It is possible that your creative or technological requirements may demand too much of current technology. Nonetheless, major improvements are on the horizon in the short-, mid- and long-term future – so stay updated! With AR becoming available to mass audiences, the WeChat marketing landscape is likely to change dramatically. First-mover advantage is up for grabs for those who can act fast at the right time.
Gregoire is a partner and the head of digital production at 31Ten, pushing the boundaries of what’s possible within a web browser. As a creative technologist, he specializes in complex and innovative frontend interactions such as 3D, AR, accelerometer inputs etc. Contact him at firstname.lastname@example.org if you want to explore what user interactions and consumer activations can be executed on a smartphone in China.
Charlie is a junior digital marketing manager at 31Ten, helping companies enhance their marketing strategies and planning production of valuable content. Contact him at email@example.com should you want to discuss the right content marketing formula to elevate your business, or just enjoy a pleasant conversation on digital-related topics.
Ruairidh is a copywriter at WE Red Bridge Communications. He creatively concepts and writes for a number of major brands, helping them tell meaningful stories and hold conversations with consumers in China & the wider APAC region. If you want to talk copywriting, reach out at firstname.lastname@example.org.
Editor: Joseph Leveque
Joseph is a partner and the head of digital strategy at 31Ten. He helps brands find the most efficient road online to reach ever more agile and informed Chinese consumers. Feel free to reach him at email@example.com if you have any questions regarding China’s Internet landscape, or if you simply enjoyed this piece and would like to talk more about it.