How to define metrics for augmented reality, mixed reality and virtual reality

virtual reality goggles

Alternative reality (AltR), including augmented reality (AR), mixed reality (MR) and virtual reality (VR) has presented some new challenges when recording metrics. Product managers, user experience designers and developers all have issues knowing how and what to capture.

Digital product development relies on iteration, learning from users reactions to make improvements. Aside from direct observation and questioning, our best learning comes from collecting large amounts of data as users experience the product.

While some data points overlap with website and app metrics, measuring activity in alternative reality experiences requires attention to a few new areas. AltR experiences present different challenges, more akin to games.

Grouping metrics

Learning from years of website and app data collection, we can bucket our metrics into similar logical groups.

Understanding entry points

One dataset, much like websites and apps, helps understand how the user found their way to the experience. This includes all the standard acquisition metrics:

  • Direct
  • Search
  • Social
  • Campaigns
  • Referrals

Understanding the audience

Also similar to traditional digital audience analytics we need to understand the user as much as possible:

  • Demographics
  • Interests
  • Geographic

As well as understand their equipment:

  • Hardware (computer, mobile, headset)
  • Software (OS, AltR source)

Understanding the environment

Unlike virtual reality, augmented and mixed reality have the added layer of mapping the territory. Capturing elements in the environment can enhance the dataset. Some technology only captures surfaces and shapes while others include image recognition to interpret the nature of the objects themselves. Depending on the data available the following metrics can be useful:

  • Objects mapped
  • People mapped
  • Locations mapped
  • Dimensions of the space
dashboard metrics

Understanding behaviour

Whichever form of AltR environment, the interaction models vary a lot from traditional digital properties.

Events

AltR presents the users with a number of options or activities usually understood as an event tree. Each action a user takes presents a suite of possible reactions by the system.

  • Passive events triggered (such as staying in a certain state)
  • Active events triggered (such as selecting an item or walking into a room)
  • Scenes loaded
  • Characters loaded

Flow

AltR follows storylines controlled by the user. Capturing in what order events take place and the amount of time spent in particular states form a core part of analysing user behaviour.

  • Sequence of events over time
  • Heatmap of user focus
  • Object focus over time (such as picking up, viewing from various angles)

Entrance/Exit

Analysts need to understand the state of the world when a user arrives and leaves. At present most experiences start in a single state, however, as they begin to connect and mobile augmented reality grows, users will enter in different states based on factors like location, demographics or interests.

  • Entrance state
  • Exit state

Interaction

In augmented reality, mixed reality and some virtual reality environments, users interact with physical objects. Capturing interaction with these objects really represents a linking of event data as outlined above with environment data, also above, created new compound metrics.

Current analytics landscape

Few native AltR analytics platforms exist at the moment. Retinad includes heat-maps and other AltR-specific data capture. But little else stands out in this nascent field.

A number of game analytics platforms, such as gameanalytics.com, also exist which cover unity and unreal engines among others. Teams can adapt these dashboards for AltR expereinces.

Alternatively, most web analytics platforms support the creation of events. These can be customised to relate to the necessary event states outlined here. Google Analytics covers all the basics and is extremely powerful for customisation as is Mixpanel.

Conclusion

Much as web analytics has grown and changed over time, AltR analytics will also evolve. For now, product managers, designers and engineers need to publish findings even as the inevitable new analytics software landscape emerges.


 

Free from screens: a sheep in wolf’s clothing — what to love about screentime

Over the past few months I have overheard or been embroiled in increasingly regular conversations with parents about the detriments of ‘screentime’. One friend recently shared an Atlantic article that even went so far as to declare a mental health crisis and claim that smartphones have destroyed a whole generation.

For the most part I agree with the article and the concerned parents — I have first hand experience of this with my own children, 12 and 6 years old. Significantly, the data tells a compelling story: since the introduction of the home computer and the iPhone, the number of teenagers going to hang out with friends has declined, fewer teens move out of their parents home, and happiness amongst 10–20 year olds has declined. Younger children play outside less, ask fewer questions, and default to screens over human interaction.

While I agree with the data, I found it jarring that both the parents and the article present a reduction in screen time as the only possible solution. Parents, experts, and the article’s author, barely even mention it as a conclusion — and just assume that the reader or listener would see banning and regulation as the only way to reverse the trend. I disagree.

I find the context examined in these arguments very limited, looking back at only the preceding two generations. To truly appreciate what’s going on, we need to look at it in a larger context.

A temporary drug

When we look back at the introduction of books into culture, we can imagine a similar scenario: “When I was a child I was running around in the woods at 6 and I was an apprentice bringing home money by 13. But look at young George, 14 years old, sitting by a candle reading books all day! We’re spoiling this generation!” This wasn’t the case — partly because the transition to widely available books happened slowly. We did not go from no books to ubiquitous books in a matter of months.

Modern technological advance happens so rapidly that our relatively rigid grey matter cannot adapt with enough self-consciousness to avoid the pitfalls of applying primitive desire for gratification to digital interactions: click here, get hit of dopamine, “Cool, I can do this all day.” Consider this a temporary state of affairs.

Smartphones will not last long in society. In 10 years they will be as familiar as VHS is to us now. We need to look at them as very temporary devices as we transition to integrated technology. At the moment the phone and other screens exist as portals to a lot more data, information and experiences than any individual has previously ever had access. Our slow brains with limited memory and an inability to have multiple conscious thoughts at once get overwhelmed and addicted. However as we — inevitably — integrate tech into our own bodies and extend our personal processing power, memory and even physical abilities, the brain will be able to catch up with quantity of input and be less overwhelmed.

The new senses

Aswe integrate technology, we will not need a lot of time focussing through a connected device. Trends — including augmented reality, artificial intelligence and blockchain technology — show that as everything becomes connected and every object begins to have a digital existence, we will begin to interact with the physical and digital versions of everyone and everything at the same time.

For example, think of your Nest thermostat hanging on your wall. You can currently reach out and touch the physical device. You can also reach out digitally through your phone and ‘touch’ the device. Imagine reaching out with your mind to change the temperature. The same for your best friend. You can meet them for a cup of tea in person or you can interact with their digital self by commenting on their Facebook post. Imagine sending them a thought message directly, from your brain to theirs. This experience will adapt to all people and items: your fridge, your grocery, the traffic light, your car, the swimming pool, the Amazon shop — we will interact with the world in a profoundly different way. Imagine opening the blinds with a thought or watching three TV streams at once, comprehending them all.

Technology has already begun to integrate into us. Look at the person with Air buds in their ears, with Snapchat glasses, and a smartphone in their hand. Amber Case in her Ted talk, ‘We are all cyborgs now,’ accepts the definition of a cyborg as “an organism to which exogenous components have been added for the purpose of adapting to new environments.” Next we’ll have contact lenses projecting info on our eyeballs.

Soon we can connect ourselves directly to our tech using a neural link between our brain and the internet of everything. Companies like Bryan Johnson’s Kernel and Elon Musk’s Neuralink hope to provide the interface between our physical self and our digital world. We can look at the smartphone as a lame simulacrum of this link that serves as the infant phase of the full transition to cyborg.

We need to guide our biological evolution to join pace and merge with our technological one until we cannot tell the difference. This young generation represents part of that transition which started a long time ago. Anything that stands in the way of this movement will get disrupted, adapted and often displaced, no matter how permanent it may seem. This includes how we see our very selves.

Education over regulation

A more interesting path explores ways to integrate digital interaction into our children’s lives, rather than limit them. While we progress to an enhanced technology future, how can we get children to balance the instant gratification of screens with the supposedly more beneficial analogue-world interaction?

I created a chart for my kids to help them develop healthy analogue and digital habits. I seek for them to have full freedom to use their screens whenever they want, but responsibly: taking care of situations in both analogue and digital life.

The chart hangs on the fridge, with a magnet marking how much access each child has to their screens at any time. Ideally, the child maintains consistenly positive choices, gaining full control forever and making the chart redundant. It may not work for everyone, but I know a lot of adults who could benefit from this exercise too. The chart helps the kids take control over their whole life without imposing the idea that analogue life somehow holds more importance or that we consider digital an unnatural state.

As Yuval Noah Harari says in his book, Sapiens: A Brief History of Humankind, “Culture tends to argue that it forbids only that which is unnatural. But from a biological perspective, nothing is unnatural. Whatever is possible is by definition also natural. A truly unnatural behaviour, one that goes against the laws of nature, simply cannot exist, so it would need no prohibition.”

Technology comes from the natural processes of the evolution of our brains and our culture. Seeing screens as some unnatural experience or an enemy of society that we need to limit, insults history and displays extreme short-sightedness.

What’s next in this new world?

We have not lost this new generation to smartphones. As parents, belonging to an older generation, we have been left behind as a new technological transition advances beyond our willigness to comprehend. This new world requires new skills which children crave and which we complain about just like our parents complained about TV or the PC.

We assume our solution means we have to get kids back to being ‘normal’, but they consider screen life the new normal. They need these skills to transition to what comes next. Like the generations before us, we can either get on board, get out of the way, or simply become irrelevant. You choose.


This is part of the Freedom series, looking at how near future technology will change civilisation.

Originally published on Medium

See also: Free energy will change your life