spatial computing - Innovation for Development

The COVID-19 pandemic has revealed the fragility of the networks linking the world together and challenged us to reimagine how we work together and maintain those connections. At the same time, it has presented us with a complex problem that crosses borders and raises economic, political, and social questions as well as medical ones. We need to enable experts to collaborate effectively and marshal enormous amounts of information as we formulate the best ways to limit the spread of the disease and to identify and protect the people most vulnerable to its effects.

We wanted to be very clear about what we as an organization want to achieve, how would our different future look like? And then walking it back in terms of if we want to achieve that, what do we need to change, and what are the steps that will get us there? Who are the people? What are the levels of change? What are the possible barriers? What is the resourcing behind that? How would we need to change our culture?

The pandemic has also accelerated changes in a world that is becoming more digitally connected. With smart cities and the Internet of Things, we are building digital infrastructure in which energy, sanitation, and health services are part of the same connected system. We have new sources of information about the world, through social media, crowd-sourced observation and open data repositories. Organizations are working, thinking and connecting differently. If we don’t spend time, energy and manpower exploring and co-developing pilots and prototypes then we are likely to be left behind and this will impact our competitiveness, and our ability to grow.

With these questions and issues in mind, we carried out a series of short-term, results-focused projects. We looked for the biggest opportunities, the places where spatial computing can contribute the most in the short term.

Virtual reality and spatial computing are visually impressive technologies, but we needed to identify use-cases where it could provide useful results, and understand in practical terms what it takes to implement them effectively.

When you hear Pokemon GO you know it’s Augmented Reality.

For virtual reality, think of the film The Matrix (minus killer robots!). The headset creates a full 360-degree environment that entirely replaces the real world. You can be standing in your office and seem to be on a tropical seashore or the top of a skyscraper. It’s powerfully immersive. The disadvantage is that, your sight is entirely blocked; you can’t see or hear the real world and the people around you (so watch your steps).

All these technologies are visually spectacular and entertaining, but what are the practical applications? We identified two initial problems that these technologies can address.

Telepresence and remote collaboration

Even before the pandemic, travel was one of UNDP’s challenges. It’s a significant part of our budget, and it has a large environmental impact. The pandemic introduces more difficulties, most importantly the risks posed to travellers. However, we need to be able to collaborate and connect with people far away, and bring expert knowledge to places where it is needed.

Current solutions haven’t entirely met our needs. Video conferencing, chat and document sharing can’t offer the faster communication, ease of rapport, spontaneity and sense of connection that in-person meetings do, and we’ve all experienced the fatigue and frustration of prolonged teleconferencing.

Spatial computing can be used for telepresence: allowing people in different locations to work together in a shared virtual space, almost as if they were in the same room, preserving the body language and spontaneity that come with face-to-face contact.

We explored this application with our Colombia country office by partnering with Spatial to create a shared meeting room, equipped with three-dimensional displays and interactive collaboration tools to allow policy-makers in remote locations to meet in a shared virtual work-space, to communicate and experience an immersive presentation about the development challenges and potential crisis management solutions surrounding the Amazon forest.

Travel also incurs risk, now more than ever. The danger of infection from COVID-19 has drastically limited the amount of travel we can do and has forced the cancelation of meetings and conferences such as Innovation Days and UNDP’s global RR meeting. Even without the coronavirus, political situations, emergent crises and other safety related issues regularly cause mission delays and cancelations.

Video conferencing, chat and document sharing can’t offer the faster communication, ease of rapport, spontaneity and sense of connection that in-person meetings do. Let’s not forget ZOOM fatigue.

What we’ve learned

We also learned valuable lessons about what it takes to implement these solutions effectively.

3D Content Creation: spatial computing happens in three dimensions, so making it effective requires the specialized work of 3D modelers and animators. Any spatial computing project needs allow for the need to find specialized providers, and the time and expense involved.

Changing the Culture: Even when a new product works better than the old, there are always inconveniences and cultural resistance to making changes. We worked with our colleagues at remote offices to understand what they need from those technologies, and to identify the obstacles to adopting new solutions.

This work is only the beginning! The real potential of spatial computing is much greater, and is part of a growing paradigm shift in our digital lives; how we work, how we live and connect to each other. The PAA project is devoted to using spatial computing to facilitate that change, not only within UNDP but outside it – to make UNDP a leader in the adoption and application of these technologies.

By Marina Petrovic, Innovative Finance and Technology Expert, UNDP Istanbul Regional Hub; Lejla Sadiku, Innovation team leader, UNDP Istanbul Regional Hub; Naomi Augustine-Yee, Technology Innovation Specialist, UNDP Istanbul Regional Hub; Austin Grossman, Interactive Augmented Reality Narrative Writer; UNDP Istanbul Regional Hub; Carolin Frankenhauser, Digital Analyst, UNDP, CDO; Frederic Repond, Senior Consultant, Monitor Deloitte

Reposted from UNDP website.

Zimbabwe’s informal food economy is the second largest in the world and some of the most vulnerable people depend on it for their livelihoods. It’s important to know how that economy works, and especially how it is being affected by the 2020 pandemic.

We took a data-driven approach to this challenge. We began by asking ourselves two questions. What are the factors determining the price of goods? Second, how has the COVID-19 pandemic changed those systems, and how has it stressed the economy’s weak points?

Like any informal economy, Zimbabwe’s is complicated — the price and availability of basic produce is determined by many intertwined social, economic and environmental factors. The amount of data collected can be overwhelming, and hard to interpret.

The Presence Anytime Anywhere (PAA) project is exploring solutions to this problem, using a new technology called spatial computing that supports 3D data visualization.

Remember the film Iron Man, and Tony Stark’s workshop? He uses holographic computer displays to work with information. He walks around it, moves it with his hands, makes it bigger or smaller. You can see how much easier it is to work with data when it’s something you can see in three dimensions.

That’s exactly what spatial computing helps us do. It uses computerized goggles to create a three-dimensional version of the digital world. Their display can take any digital information and display it in the room with you as a three-dimensional, interactive object made of light.

Edward Tufte, one of the pioneering scientists on data visualization, wrote that: “It is not how much information there is, but rather how effectively it is arranged.” A better visual representation of a dataset makes it easier to see the story that information contains. You might say, it eases the cognitive load of processing that information for your brain and lets your eyes do it much faster and more easily.

Spatial computing uses computerized goggles to create a three-dimensional version of the digital world.

UNDP is working with Flow Immersive (flow.gl) to create three-dimensional, immersive presentations. Unlike in a simple presentation, data can be charted along three different axes and viewed from all sides in a shared online space.

The Zimbabwe project is one of several pilot programmes being run by the PAA project to learn how to utilize spatial computing for UNDP’s digital initiatives. Additional applications of PAA storytelling with Flow were featured at the UN General Assembly, including as part of the SDG Moment with UNDP Administrator, Achim Steiner.

These pilot projects are part of PAA’s broader exploration of spatial computing’s potential as a tool for UNDP’s digital strategy, “the first of its kind in the UN system. We’re learning how data-driven storytelling can combine with the ability of people in different locations to work together as if they were in the same room.

As UNDP shifts to an integrated, data-driven approach (SparkBlue), new challenges will arise. Spatial computing can be a powerful tool as we take this next step, to help us see the data clearly, to see the patterns and correlations within it and to convey those lessons to the people who need them to make decisions.

Applications of PAA storytelling with Flow were featured at the UN General Assembly, including as part of the SDG Moment with UNDP Administrator, Achim Steiner.

What we have learned:

Cool is a scientific term — It’s fun to watch, and that’s part of helping people to understand what’s clearly a very complex undertaking.
Visualizing data on a greater scale — UNDP is using huge amounts of data, which makes it hard to visualize the impact and the real depth and complexity of what is happening. With many sensory modalities — visual, auditory, haptic, somatosensory and olfactory, 3D visualizations are creating perceptually enriched experiences and can help reveal hidden patterns and relationships that can be hard to recognize in complex data systems.
Empathic experience — 3D visualizations grab people’s attention in a similar way to a movie, TV show or a YouTube video does. They are more engaging than reports which are often ignored or fail to grab one’s attention.
The future of storytelling — 3D visualizations allow you to communicate much more data in a more compelling and interaction manner. They combat ‘Zoom fatigue.’
Recognizing new patterns — Immersing participants in the new 3D data reality can help to identify new patterns and perspectives. With new tools and techniques there is the potential to see connections that could not have been seen before, and chart an entirely new, more sustainable future.

Do you want to know more? Or create your own PAA 3D immersive storytelling — write to us: marina.petrovic@undp.org.

By Marina Petrovic, Innovative Finance and Technology Expert, UNDP Istanbul Regional Hub; Lejla Sadiku, Innovation team leader, UNDP Istanbul Regional Hub; Naomi Augustine-Yee, Technology Innovation Specialist, UNDP Istanbul Regional Hub; Austin Grossman, Interactive Augmented Reality Narrative Writer; UNDP Istanbul Regional Hub; Carolin Frankenhauser, Digital Analyst, UNDP, CDO; Babatunde Abidoye, Global Policy Advisor, UNDP; Frederic Repond, Senior Consultant, Monitor Deloitte

Reposted from UNDP’s Medium page.

The following piece was first published on the UN Development Programme and was originally published on September 30, 2019.

In the blockbuster science fiction movie ‘The Matrix’, after Neo takes the red pill offered by Morpheus, he not only learns the truth about reality, he also discovers how an ecological disaster has led to humans being harvested and enslaved by machines. At UNDP we wondered, what if we ‘matrix’ our way into tackling climate change, using emerging technologies to create urgency and kick start political action. After all, we’re in uncharted territory, or as writer James Bridle in ‘The New Dark Age’ calls it ‘peak knowledge.’

We went straight to the source — to Magic Leap, whose Senior Vice President for Creative Strategy, John Gaeta, created all the special effects in ‘The Matrix’. What did we learn?

Several studies indicate that being able to experience the impact of climate change allows people to connect emotionally with the problems and make them more likely to act on that feeling. We deployed this strategy when we were thinking through what the future of development might look like (check out our imaginary future of development organization — Mantis Systems — that we built with Nesta, SuperFlux and Bond). Our colleagues in the United Arab Emirates government went with similar tactics to create a visceral experience of air pollution by getting staff to breathe in air from 2050 that contained pollutants that would build in over 30+ years in case of business as usual development scenario. Imagine what breathing that air felt like, and what you might want to do to avoid it!

Spatial computing takes this one — well maybe several hundred — steps further. So what is it? Combine virtual, mixed and augmented reality and voila! you get emerging technology that brings information from 2D screens into a 3D space allowing you to interact directly with the digital world. It is a little bit like having an x-ray view into our world with the ability to play with it. On some level, it is the purest form to date of blending technology into the world.

If the definition sounds too techy, maybe a few examples might help. In some cases, this allows farmers to manipulate their physical environment (their farms) as they are affected by weather in real time in order to increase crop yields and reduce costs. Or consider the Hydrous project that gave public servants from Palau a visceral experience of the effects of climate change on the underwater world, which prompted them to do more to protect its coral reef.

The Hydrous project gave public servants from Palau a visceral experience of the effects of climate change on the underwater world, which prompted them to do more to protect its coral reef. ©Damsea/Shutterstock.com

Spatial computing can help reduce skill gap and errors in production by applying mixed reality to training employees without advanced technical experience or specialized equipment. It could also increase perception, engagement and overall experience of the world for the children with autism. And for one of our more favorite non-development applications of the technology, check out the experience of the speech from William Shakespeare’s ‘As You Like’ — as if you’re in the same room.

Working with Magic Leap, the German Space Agency and BadVR, we created an experiential visualization, Climate Change Spares No One, that brings to life visceral implications of the changing climate on the delta cities. Our intention is to enable anyone to feel, sense and see what life in +2 celsius world is like. The interactive experience was showcased during the 2019 Social Good Summit and during the UN General Assembly UN Action Zone.

We hope this is just a start of our partnership with emerging technology companies whose work can potentially increase the urgency of the scale and scope of climate emergency into our day to day lives, and inspire bold action to forestall the catastrophy the world is likely to experience if we don’t act.

This blog was co-written by UNDP’s Milica Begovic, Lejla Sadiku, Marina Petrovic, Robert Pasicko and Cassie Flynn, and from Magic Leap’s Andy Lanning, Prudence Fenton, Jad Madouchy, Suzanne Borders, Austin Grossman, Naomi Augustine-Yee, Graeme Devine and Quiddale O’Sullivan.