Posts Tagged ‘mobile’

Mobile storytelling: an evolving story

Charles and Ray Eames: Powers of Ten

Charles and Ray Eames: Powers of Ten

I was recently invited to speak at DUXcamp hosted by NPR and then again at Microsoft Research around the subject of Mobile and Storytelling. I created a rather stream of consciousness presentation, bringing together various thoughts about storytelling in the mobile space. Still very rough around the edges but a central theme is beginning to emerge: Mobile allow stories have scale.

Here’s the presentation deck:

2011-11-02 Mobile Storytelling (@ Microsoft Research)


With the arrival of smartphones, it’s amazing how much data we are collecting and consuming on our mobile devices. We tweet, checkin, google, blog, instagram, post status updates, yelp and a host of other things from our handheld devices. And somewhere on the internet this information is quietly collecting. In the ancient times, pharaohs had scribes that shadowed them, recording what they said. Now we have our mobile devices diligently collecting our data. There was once a time when people used to record their lives and thoughts in leather-bound diaries. Now we have smartphones, whose data, when strung together form a story of our lives.

Mobile Me

Mobile me: my story

My Story

I can’t remember when I heard it for the first time, but someone said, our identity is the story we repeat to ourselves. This is so true. I keep on telling my story of how I moved between the East and West, between physical environments (architecture, urban design) and virtual (web and mobile development and strategy), between technology and the humanities. I don’t have an identity grounded in an single culture, nation or land. At one time, I would have referred to this as being nomadic. Now I can just say I’m Mobile Me to borrow a term Apple has abandoned.

Our Stories

Today we have a wild abundance to the ways we collect our stories. Many of them track us automatically: Nike Plus tracks my run, tracks my finances and spending patterns, and Trip It neatly organizes my travel plans.

At the rate that memory capacity of devices are increasing, in a couple of years we will have an iPhone which would hold 256GB of data. Battery-life permitting, this would mean (albeit at a low resolution) you’d be able save your whole life by dangling your iPhone around your neck and recording every moment. This is often referred to as life caching or lifelogging. But what’s the point? When will you have the time to go back through hours of video to find and edit the interesting or meaningful parts. Jorge Luis Borges points out that 1:1 scale map is useless. Aren’t we doing just that when we don’t filter to good from the mundane?

Nicholas Felton has been obsessively collecting data about himself and publishes them in annual reports about himself since 2005. And now with his own iPhone app Daytum, you too can be as obsessive about your data as he is.

But Felton does provide us with a insightful clue. What data is meaningful? For his 2010 Annual Report he compiled and presented data around his father’s life. It is surprisingly moving. He masterfully abstracted meaningful data from the numbers and constructs a picture that pays a deeply personal and loving tribute.

Nicholas Felton: Annual Report 2010

Nicholas Felton: Annual Report 2010

An iPhone app called Memento compiles the data from your various disparate personal information repositories such as Twitter, Facebook and Flickr, and brings them back into a diary format, of all things. What used to be manual labor is automagic and becomes personal again. You can even add diary entries. What emerges is a story – your story. You see densities of information where you had memorable events, and long silences where you were buried in depression being dumped.

Other People’s Stories

We live in the age of Facebook. But Facebook is horrible when it comes to telling stories. It presents fragmented pieces of people’s lives that we are often forced to react to rather than engage. The timeline, in its quest to present ever growing amounts of information to us, become as fleeting as the stock ticker feed in Times Square, and belittles the personal importance of each post, by rendering it in the same small block, with the same small profile icon, in the same small font as everyone else. Some people are simply more important than others and we want to pay more heed to them. They are larger in our minds. Why are they the same size as the person whom I casually had a short conversation with at a conference I don’t even remember? Facebook is addressing this issue by adding filters, but with all the data crunching power that they use already around analyzing my relationship with my friends, shouldn’t they know who is important to me already?

Flipboard: Remembering Steve Jobs

Flipboard iPad app

Newspapers know how to present information. They’ve had enough years to refine their art. Typeface sizes matter. The fold matters. Sections matter. Photos matter. They bring your attention to what they deem important. Flipboard is an iPad app that tries to do that, by providing an illusion of priority through a tactful manipulation of layout, font sizes and images. It provides much needed difference and rhythm we are attracted to, over the often mind-numbing flat Twitter or Facebook feed.

Our Collective Stories

Daum Communication, a leading internet services provider in Korea offers a map service with a streetview option, much like Google Maps does in the States. As of Feb 2011 however, they have added a feature that goes a step beyond: streetview history. You can select from various past dates when the streetview camera captured the image. As one example, you can view the building where Daum is located now, under construction in 2008.

Daum Map showing history

Daum Map showing history

Maybe it’s possible to take this further by using tools like Photosynth to crowdsource forgotten images from people’s photo albums or maybe even add historic archival images, so that when you are viewing a certain place through the streetview tool, you can actually go back in time and take a historical journey through a neighborhood. Historians can narrate stories of a city’s development or you can tell your own story of fond childhood memories. What was once a personal memory can now build up a crowdsourced collective memory.

Curtis Wong of Microsoft Research has an wonderful presentation of Microsoft’s World Wide Telescope project where the tool for presenting the universe around us sets a stage for storytelling by allowing researchers and students alike to create a narrative through the interface. Something like an interactive version of Charles and Ray Eames’ masterpiece Powers of Ten.

World Wide Telescope

World Wide Telescope

Usahidi, an interactive map-based information collection tool was born out of a need to capture and report post-election violence during Kenya’s 2008 presidential elections. Usahidi means testimony. Since then it has been used widely to crowdsource data through mobile devices and present them dynamically on a map: from neighborhood snow removal updates to crime reporting. Most noteably it was deployed in the aftermath of the devastating earthquake in Haiti to crowdsource unsafe conditions and aid relief coordination.

How did you hear about Steve Jobs’ death? A lot of us heard through Twitter or from someone who heard it through Twitter, as a collective gasp went through the twitterverse at the news of his sooner-than-expected death. Tweets per second (TPS) is now a proxy for the velocity of the spread of news. When it comes to TPS, surprisingly Jobs’ death ranks #5. It’s the news of Beyonce’s pregnancy announced during the MTV Video Awards that takes the honor of #1. A newborn life wins over death.

Adding Our Life to Data

Jawbone, which produces high-performace mobile headsets, just came out with a very affordable health monitor bracelet called UP. Coupled with a smartphone, this bracelet tracks your eating, sleeping and exercise habits and “nudges” you to adopt better habits. You can imagine market-research groups like Nielsen paying people to don a device like this to track how people really react to what they are watching. Nike Plus gathers data about your run, but what would it be like if global events were tracked not just in the number of media reports but as bio-metric data? What kind of story would that tell? What would a collective “gasp” look like when people heard of Steve Job’s death or Beyonce’s pregnancy?

Jawbone UP

Jawbone UP

Interactive artist, Jonathan Harris is an amazing story teller. We Feel Fine is the project he is best known for. But his Whale Hunt is an incredible project in many ways. Here’s what he did:

I documented the entire experience with a plodding sequence of 3,214 photographs, beginning with the taxi ride to Newark airport, and ending with the butchering of the second whale, seven days later. The photographs were taken at five-minute intervals, even while sleeping (using a chronometer), establishing a constant “photographic heartbeat”. In moments of high adrenaline, this photographic heartbeat would quicken (to a maximum rate of 37 pictures in five minutes while the first whale was being cut up), mimicking the changing pace of my own heartbeat.

The Whale Hunt / A storytelling experiment / by Jonathan Harris

The Whale Hunt / A storytelling experiment / by Jonathan Harris

The result is very close to how our minds actually work – we capture more information and memories in relation to how intense our experience is. Time slows down because we are collecting more information (often for our survival).

This is exactly what happens in the way we collect data through our mobile devices. The more significant the event or location, the more photos, tweets, status updates, blog entries we create about it. You can see it on an individual level, but also on a greater collective level. If you were to represent this in a graphical way, you’ll see something analogous to World Wide Telescope’s universe, where you would have stories instead of stars. What would it mean to look at galaxies of stories across time and distance, zoom into individual shining stars of stories, or encounter black holes where a natural disaster abruptly muted thousands of voices in a single horrific event. You can almost imagine ripples of story supernova spreading at the speed of light as the news of the disaster spreads in its aftermath.

Scale of Stories = Scale of Identity

Recently, overcoming a freak October snowstorm in Washington DC, I went to the National Museum of the American Indian, and then to the National Museum of American History. There I witnessed two institutions telling stories. One of a frequently muted story of the American Indian, whose so many tribes are now forgotten because their stories did not survive the diseases, conflicts and forced migrations. In contrast I saw the victorious stories being told of a young nation who overcame colonial powers, native inhabitants and inner division, whose short story is still unfolding, and needs to be remembered and repeated because its identity and survival as a nation depends on it.

When I showed Google Earth for the first time to my dad on an iPad, the first thing he did was to look for the house he grew up in, deep in North Korea, having left it behind some 60 years ago during the Korean War. I saw the concentration and the emotion that poured over his face as he searched for his childhood home by scanning the geography but also his memory, desperately inferring its location through the landscape of streams, valleys and railroad tracks he remembered.

Zooming in, it’s my dad’s childhood story. Zooming out, it’s the tragic story of the Korean War and the subsequent division of Korea. Further out, it’s the historic story of the fear and ideological power struggle between the superpowers following World War II.

Our identity is the story we repeat to ourselves. If that is so, what is the story we repeat to ourselves as an individual, family, community, region, nation or as a humanity? For the first time in history, as we collect so much data about ourselves, we have the potential to simultaneously see our stories unfold dynamically at different scales. And maybe that can teach us something about ourselves.

Update (2011-11-09)

Microsoft Research has just posted the presentation online. It’s in 2 parts. See the second half.

Get Microsoft Silverlight

Life Caching on Mobile Phones

We\'ll soon be life caching on mobile devices

We'll soon be life caching on mobile devices

At some point in the near future, the term mobile "phone" will be too limiting to describe what we’ll be carrying around in our pockets.

Take the iPhone (or any smartphone) as an example. Currently there are 8GB and 16GB versions available, but at the rate memory is increasing and coming down in price, soon we’ll be getting 32GB, 64GB and 128GB versions in the next few years (or months?). What will it mean to carry that much capacity on a mobile phone.

All my music files are about 50GB, all my photos 30GB, my email 5GB, and another couple for all the movies files shot on my camera. That means I can be carrying all my digital possessions with me on my phone. The term "phone" refers to a communication device. With high-quality camera and movie capture capabilities along with massive storage, it is something more that a mere phone. At this point it become a life caching device.

Nokia and Samsung have already been busy exploring this concept, however they are still in very early stages of development. I always thought that Cyworld needs to move in this direction in order for it to remain relevant – i.e. provide a life-caching service closely coupled with mobile service, but I digress.

For a life caching mobile device/phone to be useful/usable, it needs to address some pretty fundamental challenges:

  • Powerful Search When you have so much stuff on such a small device you need something more close to Apple OS X’s Spotlight to find the stuff you are looking for.
  • Rapid Browsing Browsing photos on a traditional cell phone is pretty painful with the key-mapped interface. Touch interfaces (à la iPhone) with flicking provide faster access and browsing experience to photos, music, movies, email and message lists.
  • Logical Cross-Referecing It’s still a communications device after all, and it makes sense to be able to access content via people. When you find a person in your address book, you should be able to view all the content related to that person.
  • Easy Backup Heaven knows what will happen if (or is it a matter of when) you lose you life-cached possessions stored on your device.
  • QWERTY Keypad You’ll need to do a lot of typing to tag all the content coming into your device and well as for posting and sharing your content with others.
  • Web-PC-Device interoperability Your portable device is good for capturing precious moments, communicating and transporting content, but for sharing the web is still king. As for editing all the movies and photos, and backing up, the PC is still your best bet. Each device has its merits and content should be easily transferable between platforms.

Personalization and Mobile Phones

Custom wood case for iPhone by Miniot

Custom wood case for iPhone by Miniot

Mobile phones and most personal electronics devices have been made for durability. My Samsung phone is finished in stainless steel, plastic and glass. It is black and shiny. My iPod Nano is aluminum.

My wallet is made of leather. When I first bought by wallet, it was stiff, and uncomfortable. But at some point in time, it yielded and started to conform to the curve of my posterior. Same thing happened to my watch strap, also made of leather. It is has morphed to the size of my wrist. Shoes, jackets, baseball caps.. I can name numerous examples.

In the flood of hyper-niched marketing world, I am still surprised that very little effort is made in the personal electronics space to take advantage of this property of personal artifacts: that it registers the physical interaction between the artifact and user. Guitar frets have show well-worn usage by its owner. Yet phones resist this natural aging process.

Most aspects of our environment can be better personalized than our electronics. We can choose wallpaper or paint color for our apartments, adorn it with our personality over time. Personalization to mobile users usually means changing the background or ringtone or those little dangley phone accessories that you see all over Korea and Japan. Nothing that registers gradually over time. Why not a mobile phone made with leather or wood. Why not a iPod where a friend can scratch their message into the surface instead of having it laser engraved.

Bamboo, the degradable phone (via core77)

Bamboo, the degradable phone (via core77)

The Chute Smartphone (via Yanko Design)

“The Chute Smartphone (via Yanko Design)

It was refreshing to see a couple of example recently. Here are two concept phone examples, The Chute Smartphone and Bamboo phone, and iWood handcrafted iPhone case by Miniot made from high quality wood (commercially available).

Yet another argument for the use of natural materials in personal electronics is environmental. Massive amounts of mobile phones are consumed each year. The rates of mobile phone penetration is close to saturation in the US, Korea and in most developed countries. It’s rare that we find any recycling of mobile phones. Most people just throw away their phones when it is broken or when they switch carriers. Here’s where the mobile telecom industry can learn from the automotive industry. There is a whole secondary industry build around reclaiming, reusing, and recycling used car parts on one front, on another front there are movements to make more efficient cars. Why are there so few examples of environmental friendliness in the mobile phone industry? I think this is a marketing opportunity that begs to be tapped for both the consumer’s and industry’s benefit.

Geospatial Info + 3D Space + Web 2.0 + Mobile = ?

Geoblogging: The Gombe Chimpanzee Blog

Geoblogging: The Gombe Chimpanzee Blog

We all remember ooh’ed and aah’ed at Google Earth when it was first available in 2005. For the first time history, services such as Google Earth offer us a readily available, zoomable, navigable visualization of our physical world, the detail of which are ever increasing with new technologies being developed as showcased by Microsoft’s Virtual Earth and Photosynth projects.

But it’s not just a static representation of information, as maps have classically been. The convergence of the internet, mobile technology and geospatial representation of our physical world, presents an interesting intersection of technologies.

The internet contains an ever-expanding universe of knowledge and information, and with web 2.0 technologies, users are even more empowered to directly participate in that growth, and to share, aggregate, and find creative ways to seeks value in this information.

When information available on the web is combined with geospatial data what emerges is Geoweb. Geoweb presents yet another layer that information can be mapped or grounded to. It gives people an opportunity of assigning information, be it historical, commercial, social or existential to a given location.

Mobile technology has the two-fold function of being able to retrieve that information in real time at the location to which the information was associated to, as well as being able to record yet more information through text, photographic or motion input about the location.

The pressing issue now is not the availability of information but how to filter it to be meaningful?

Map have always been a filtered reprentation of selective information. A road map only maps roads for the purpose of guiding a user from point A to point B. So the challenge facing Geoweb is no longer one of technology, but one of selectivity and value. What does information presented in this way allow us to do?

It allows us to associate information on a scale and perspective that we were unable to do before. Classical maps show border, terrain, economic, or conflict information. Now we can map, aggregate, slice-and-dice all the atomized miscellaneous pieces of information geospatially. It allows us to associate information that was not possible or hard to do before. Oh joy.

We already see some examples of innovative use:

  • CARMA (Carbon Emissions Monitoring for Action): this s a project by Center for Global Development I was involved in at my old firm, Forum One Communications. It maps publicly available CO2 emission data of power plants and other polluting agents on to Google Maps, and encourages users to submit more data about polluter in their neighborhood.
  • Geoblogging: Jane Goodall Institute created the first geoblog: The Gombe Chimpanzee blog. It follows the activities and blog posts by Emily Wroblewski, a field researcher who is studying the Gombe Chimpanzees to coordinates on Google Earth.
  • Search for archeological sites: Scott Madry, an archaeologist at the University of North Carolina at Chapel Hill, has been pinpointing possible archaeological sites in France with the popular desktop program Google Earth.
  • Cannonball Run: Alex Roy, set a new record for driving across the American continent of under 32 hours, in the fall of 2006. He planned and practiced his run using Google Earth.
  • Metaverse Roadmap also shows us exciting possibilities of how 3D representation and the web may converge.

Other as yet unrealized examples are:

  • An amazing project Jonathan Harris and Sep Kamvar, We Feel Fine aggregates and visualizes th state of people’s emotions around the world. We may be able to map geospatially, in real time, the emotions around how a community reacts to tragedy or jubilation.
  • We could map the impossible path that Ernest Shackleton took to save the lives of the ill-fated crew of the Endurance in 1900. (UPDATE: It has already been done)
  • We can map the path of my UPS package as it travels from Amazon’s warehouse in Kentucky to my doorstep in real time, so we are not held hostage to the UPS man’s schedule.

* * *

Umberto Eco has a nice essay in How to Travel with a Salmon named “On the Impossibility of Drawing a Map of the Empire on a Scale of 1 to 1.” He quotes from Jorge Luis Borges who is in turn quoting Suarex Miranda:

…In that Empire, the craft of Cartography attained such Perfection that the Map of a Single province covered the space of an entire City, and the Map of the Empire itself an entire Province. In the course of Time, these Extensive maps were found somehow wanting, and so the College of Cartographers evolved a Map of the Empire that was of the same Scale as the Empire and that coincided with it point for point. Less attentive to the Study of Cartography, succeeding Generations came to judge a map of such Magnitude cumbersome, and, not without Irreverence, they abandoned it to the Rigours of sun and Rain. In the western Deserts, tattered Fragments of the Map are still to be found, Sheltering an occasional Beast or beggar; in the whole Nation, no other relic is left of the Discipline of Geography.

From Travels of Praiseworthy Men (1658) by J. A. Suarez Miranda

What we see Google Earth and Virtual Earth is the creation of such a map, mapping reality on to a mirrored world. We may actually be seeing something even more profound. A map that contains more information than even the 1-to-1 map.

[Update] Worldprocessor is a pre-Google Earth visualization of data on a globe. Very interesting nonetheless.

Random Thought: Twister for iPhone

iPhone Twister

iPhone Twister

It’s only a matter of time before someone comes up with a twister game for the iPhone.