Protagonist (2014)

In the Spring of 2014, We undertook practical research to ascertain the feasibility of building a prototype of a software engine that automatically creates a video life story from an individual’s social media content.

Protagonist is a practical attempt to make sense of our vast stashes of personal data in a human, emotional, narrative way. Using their own social media content, we want to create a short film memoir of an individual – with the output feeling meaningful, personalised, beautiful and potentially provocative. And we want to see if we can create this using an automated process.

Our ambition was that the Protagonist service would be a commercial, stand-alone, direct-to-consumer product, but would also be used within the Afterlife Retreat.

Our conclusion was that construction of narrative from online data poses an extremely complex computing problem, which is just starting to be explored by digital giants such as Google and Facebook. It would be very difficult to compete in this marketplace currently. We therefore feel it is not an avenue that is appropriate for us to pursue at this time.
However, we would be open to licensing the concept or working in partnership with a digital agency to develop the concept.
Using the learning from this Feasibility Study we may still pursue a smaller and simpler version of this concept as part of our planned Afterlife Retreat project.

‘Protagonist’ is a REACT Strategic Projects Feasibility Study. Funded by the Arts and Humanities Research Council (AHRC), REACT (Research and Enterprise in Arts and Creative Technology) is one of four UK Knowledge Exchange Hubs for the Creative Economy and is a collaboration between UWE Bristol (the University of the West of England), Watershed and the Universities of Bath, Bristol, Cardiff and Exeter.

‘The Rooms’ Sitting Room Session – When to say no to an idea

November 16, 2015 by

I was asked to speak about our REACT-funded project for 5 minutes as part of the Sitting Room Sessions at the Private View of REACT’s excellent ‘The Rooms’ festival in early November 2015.

Here is the text of the talk (minus the live asides of course, which made it a bit less dry!):

“I’m going to leave it actually.

Yes, its a great idea.
Yes, it seems there would be a market for it. The Funeral industry would be the most obvious place to start.
Yes, no-ones doing it yet, well they’ve started trying in a way, but their attempts are pretty poor quality, most people would agree with that.
Yes, I think there would be quite a lot of avenues to pursue in terms of start-up support and finance.

So, yes, I thought of it, I looked into the feasibility of it, I’ve got ideas about how it could work, what it could look like, who the partners and customers might be. I’ve even worked with people to explore exactly how we could produce it.

But I’m going to leave it there. I’m not going to take it forward.

These aren’t words I use a lot.
In fact, it took me a while to make the decision.
At first it definitely felt like a failure.
But now I know it was the best decision.
A positive exit from the project.
Better to say a considered no, than struggle on with something against your better judgement out of some kind of misplaced sense of duty, until it grinds to a bitter and messy halt. (a bit of melodrama there… did I mention I’m a theatre maker…)

In the Spring of 2014, with REACT Feasibility funding, I undertook research into the feasibility of building a software engine that automatically creates a video life story from an individual’s social media content. The project was called Protagonist.

Protagonist was an attempt to make sense of our vast stashes of personal data online in a human, emotional, narrative way. Using their own social media content, we wanted to create a short film memoir of an individual – with the output feeling meaningful and personalised. And we wanted to see if we could create this using an automated process.

Our ambition was that the Protagonist service would be a commercial, stand-alone, direct-to-consumer product.

I found that the construction of narrative from online data poses an extremely complex computing problem. Who knew?! Well I didn’t. As someone from an arts background its sometimes tricky to guess which seemingly impossible problems can be solved relatively simply by technology (or already have been) and which are actually basically impossible.

This particular area of algorithmically generated video is only just starting to be explored (with very limited success) by digital giants such as Google and Facebook.

So here’s why I’m saying no to this idea:

It would be very difficult to compete in this marketplace currently, if Google & Facebook with all their resources are struggling to make anything worth watching;
I’m a theatre maker, not a software engineer;
Life is short: I don’t want to spend at least the next 3 years setting up a software start-up, that won’t use my skills well;
I’ve got other ideas for other projects I want to make, so I’m going to use my skills and energy where they’ll be making the most impact, and get on with making those.

You can see what we’re up to at www.theotherwayworks.co.uk
I’m Katie Day, and I’m Artistic Director of The Other Way Works.
We’re a Birmingham-based theatre company making playful theatre that immerses our audiences in the story.”

Protagonist Feasibility Project: Related Activity

January 20, 2015 by

The Guardian: Facebook apologises over ‘cruel’ Year in Review clips

Facebook has apologised after learning, yet again, that not everything can be done algorithmically. Some things, it seems, need the human touch.
http://gu.com/p/44fgx/stw

 

BBC Your Story

This web app uses your Facebook profile (or manually entered personal info) to create your life story through the BBC News Archive.

http://yourstory.pilots.bbcconnectedstudio.co.uk/

It appears to filter the content displayed using your date of birth, and whether you like Music, Comedy, Politics or Sports.

Summary

August 14, 2014 by

Objective
To ascertain the feasibility of building a prototype of a software engine that automatically creates a video life story from an individual’s social media content.
Protagonist is a practical attempt to make sense of our vast stashes of personal data in a human, emotional, narrative way. Using their own social media content, we want to create a short film memoir of an individual – with the output feeling meaningful, personalised, beautiful and potentially provocative. And we want to see if we can create this using an automated process.
Our ambition was that the Protagonist service will be a commercial, stand-alone, direct-to-consumer product, but will also be used within the Afterlife Retreat (a large-scale residential theatre event The Other Way Works are planning for 2016).

Goals
To produce a design fiction film
To produce a technical specification for the software build

Outputs
An attractive design fiction film to illustrate the potential of the service. Working closely with an animator we have produced a film with a crafted, hand-made aesthetic, which is unique in the marketplace.
A Requirements document based on our research, to inform the Technology Feasibility Studies/Specifications.
Technology Feasibility Studies/Specifications from two independent software consultants/agencies demonstrating different approaches.
A summary of responses from our market research questionnaire.

Key Learning Outcomes
Building a prototype is feasible, and could be done for c£40,000.
Some adjustments would need to be made to the Specification, including reducing the range of social media feeds, and streamlining the processes that identify meaningfulness of the content to the user.
Music to soundtrack the video is a key component, with as much impact as the visual style. Rights and licensing cause a problem here, meaning that people can’t just have their favourite song to accompany the video. A selection of rights-free music would need to be offered, which mitigates the problem, but doesn’t solve it.
Construction of narrative from online data poses an extremely complex computing problem, which is just starting to be explored by digital giants such as Google and Facebook. It would be very difficult to compete in this marketplace currently.

Dr John Troyer

by

The Protagonist project is ahead of its time. The ideas are sound, the research relatively easy to complete, and the concept is strong. The problem is that current computer technology and proprietary ownership of that technology makes creating a Protagonist platform difficult and somewhat expensive. At a future date, and perhaps not that far in the future, this kind of cross-platform video creation will be possible. At that point the technology will be used by a wide array of businesses and individuals. From my own research, I know that the Protagonist platform would create the kind of video content that funeral directors and other organisations working on personal memorialisation projects would find quite useful. One of the other groups that will benefit from this technology is archivists, many of whom will encounter enormous difficulties in the near future when it comes to storing digital content and information for prolonged periods of time.

For me, the entire project, and working with Katie, pointed towards what kind of future these online content platforms are working towards. At this point and time it seems far-fetched that one single company or government entity will own and run all online content platforms, so these disparate groups could end up creating a Protagonist-like system that runs across the groups. I have my doubts that this will happen, but it could.

The project also opened up another point for me, which is this: Maybe it is completely acceptable that most of a person’s online content is eventually lost and ultimately deleted. We first world Homo sapiens currently live in a historical moment in which the concept and practice of information preservation is crucial for an individual’s identity. I’m no longer convinced that our future cousins will necessarily view a lifetime’s worth of digital content with such reverence. Indeed, the difficulties in maintaining accessibility to that information as computer technology changes is itself alone a fundamental issue. Many people fail to realise, I think, that internet years are significantly shorter than human years.

Dr. John Troyer
Deputy Director, Centre for Death and Society
University of Bath

Technical Feasibility Documents

by

We contracted two developers, independently of each other, to draw up a Technical Feasibility and Specification document for the build of a prototype of the Protagonist service. We wanted to get different ideas about how the problems could be solved, and get a sense of what approaches we could take.

We provided this Requirements Document as something to work from.

Here are their reports:

Tom Martin

The Swarm

Research and Market Research

by

Research

We undertook research into existing systems, ideas, companies and academic research to build our understanding of the area.
Pinterest Board with links to projects, people and services of interest to the project
http://www.pinterest.com/otherwayworks/protagonist/

Market Research

Focus GroupBees in a Tin Event, Birmingham, February 2014
An initial session to gauge potential user interest, and discover what kind of features would be appealing in a product like this.

Focus GroupPervasive Media Studio, Bristol, April 2014
A drop in session. Using printed versions of the online survey to provoke discussion and feedback. To gain specific feedback on video structure, which events were thought ‘important’, and highlight any concerns around privacy.

Online Survey
A survey was put together using Google Forms, and disseminated via online networks, to provide us with a wider range of feedback on our proposals. A summary of responses can be seen in Appendix.

Above is the graph of responses to the question “Which social networks hold your richest content?”:

Design fiction film

by

The test user for the film, Hannah Nicklin, has kindly agreed to our use of her content in this context to allow us to illustrate the Protagonist concept.

The test user was selected because of her active use of many social media platforms, which would provide us with a good range of content to work with.

The user’s content was selected manually, using the Specification as a guideline for the selection process.

The animation effects were created by animator Hazel O’Brien, using paper folding and green screen filming. The content used is in a digital format. These effects could easily become templates.

The music was composed and produced in response to the film by Mark Day.

Some brief initial feedback from the test user after seeing her film for the first time

It was nicely esoteric
I couldn’t tell how it had decided to choose some stuff
It was about 65%­70% on the money with what was significant (i.e. it didn’t know I was broken up with on graduation night!)
It was really nice to see family
The ‘on this date’ stuff was nice but it made me feel like an asshole I was complaining about having holes in my shoes on the day the first black president of the USA was elected.
The design was lovely
I was engaged the whole way through
I didn’t feel like it represented me
I did feel like it represented me online
I wanted there to be more Instagram because I like that place most
I wondered what other people were saying back to me, it felt odd for me to exist in a vacuum using content from such social space
I wanted it to choose some music from stuff I like on Soundcloud

Successful Elements

by

The aesthetic of the film received positive responses, and people appreciated the home-made look.

The ‘on this day in history’ snapshots had a good impact, and are technically the easiest to select user content for (the primary selection method being the datestamp, which is a universal item of metadata).

An example of this is shown in two screen shots from the design fiction film above.

Potential Markets & Routes to Market

by

Funeral Directors
In the future, when there will be more electronic, online documentation of peoples’ whole lives, the Funeral industry would be a strong market for this type of fully automated product. At this moment in time, manual selection of images from online and offline sources by friends and family using an image slideshow programme will produce the best effect.

Gift Experience
As a direct to consumer service Protagonist would be best pitched at the gift market. Our market research suggested that people would be more likely to buy the service for a friend or family member than for themselves (which would feel/be perceived to be vain or self-obsessed). This may work better on an annual model (showing the user’s year online, rather than their whole life), to allow for repeat custom.

Brand/Event Marketing Campaign
The service could be built for a one-off brand marketing campaign if commissioned.
A focus on an individual’s, or a group of friends’ experience of a specific event as documented on social media could be a good adaptation of the service for this market.

Competitors and other services

by

In the last 12 months there has been a burgeoning of services and tools exploring the challenge of drawing out a narrative from users’ recent and archived photo and video content. Many of these are single platform services, often developed by the platforms themselves.

Facebook LookBack Film
https://www.facebook.com/lookback / https://www.facebook.com/help/206982576163229/
A free automatically created 1 minute film, showing highlights from a user’s Facebook timeline. Uses chronology and number of likes to identify usable content. Only displays content uploaded by the user (not content that the user is tagged in). Latterly added a post-video manual editing function, which allowed users to swap photos in and out of the video within the existing structure.

Carousel by Dropbox
https://www.carousel.com/
Free display tool for Dropbox users to apply to their content on Dropbox.

Google+ Stories, Movies, and Auto Awesome
http://googleblog.blogspot.co.uk/2014/05/google-stories-and-movies-memories-made.html
Free tools for Google+ users to apply to a user’s content on Google+. Auto Awesome automatically creates daily video edits of photos and videos shot that day, with customisable options.

The Wilderness Downtown
http://www.thewildernessdowntown.com/
HTML5 video experience by Arcade Fire. The user enters the location of their childhood home or school, and the programme uses Google Maps and Streetview to customise a music video with this location.

Museum of Me
http://www.intel.com/museumofme/r/index.htm
HTML5 video experience by Intel that uses a Facebook App to create a virtual museum fly-through from your Facebook content.

Take this Lollipop
http://www.takethislollipop.com/
A ‘scare’ film in HTML5, highlighting the unexpected and unpleasant ways in which a Facebook user’s content may be viewed or used when they allow a Facebook App access to their account.

Animoto / Magisto
http://www.animoto.com / http://www.magisto.com
Cloud services that makes photos and video clips into rich videos. Magisto also auto-video edits.