Imagine a pair of reading glasses that allow you to read more immersively, more critically, more to your liking. Imagine an augmented reality experience that allows you to keep the best part of reading the newspaper: the smell of newsprint, the flop of the page in your hands, the neat columns and sections that help you focus on one facet of life at a time.

Combine the two together and you have an augmented reality experience that allows you to keep all the best parts of reading the paper while harnessing the power of the internet.

design process 

I took care to notate our design process. It’s not everyday that we develop (hack together!) an un-tethered virtual reality experience and then work backwards to consider the possible applications of the device. While I can’t diagram out a particular iterative process, I can say that three different processes added three different types of insight in the process. Making technical prototypes helped us understand the affordances and limitations of VR, the Leap Motion, and the processing capabilities of Unity running on Android. From these affordances and limitations, we were able to draw out a core mechanic for our interaction designs. Considering real world uses for the tool helped us identify activities with rubrics for use that align with the core mechanics and draw attention to real world problem spaces. Considering imagined worlds, whether through game design or design fiction, helped us understand what design features were successful or unsuccessful and imagine future uses.

After building two technical demos, we sat down and talked about what we learned. We expanded our discussion to consider using the video feed from the phone and building an AR experience. We listed the affordances of limitations of AR and gesture recognition and came to the conclusion that the core mechanic we wanted to pursue was concealing/revealing. We then brainstormed real world scenarios that might include concealing/revealing. I used those brainstorms to quickly and indiscriminately write some design fiction-inspired use-scenarios. You can read the scenarios and more about the brainstorming process here.

The scenarios I wrote were predictable, stereotyped, shallow and sometimes even preachy. But the objective wasn’t to be clever or even imaginative, it was to riff off a brainstorm and mine our mundane visions of the future. By pushing myself to come up with some use scenario for each item that we brainstormed, I was able to look across the scenarios and pull out what was necessary for a successful use-scenario:

Guidelines distilled from design fiction scenarios:

1. The scenario must feature one object.
2. The object and its use must have subtext or a backstory.
3. There must be an established gestural vocabulary and rubric for using the object.
4. You must be able to interact with the object close to your body, in the space immediately in front of your head and chest.
5. The interaction must be constrained to the rules of the physical world.

Based off of these guidelines, we brainstormed a number of interactions from taking a toaster apart, to caring for a pet, to examining a patient. I broke down each interaction according to the features above.

Considering the core mechanic: concealing and revealing, and the above criteria, we landed on reading the newspaper. Investigative journalism is all about uncovering the truth, yet the news is veiled by subjectivism, bias, and sometimes outright deceit. The mechanic and the subtext of the object seemed to be in alignment.

I then began breaking down the gestural vocabulary of both newspaper reading and the Leap Motion:

matching gesture with mechanics:

Then thought about the paper itself, breaking down the interactions–starting with concealing and revealing–capable when interacting with a fully printed paper or when interacting with a completely blank paper:

Landing on three different models for the reading the news:

I then created some design fiction scenarios, including drafting fictional testimonials for a few of the a-news reading scenarios:

Paradigm 1: truth telling

The news represents the journalists’ best attempt at the TRUTH. The problem that this causes are 1) the sheer overwhelming affect of the truth 2) the truth is subjective. Leading us to…

The news comes to you redacted. When you put on the glasses and point to a redacted box, a warning menu pops up…

You could then choose to swipe off the redactions and read the text as it was written. Of course, you could designate your own personal tolerance settings before reading the paper…

Now let’s hear from one of our fine customers…

Full testimonial: “Every morning I sit at my desk with my cup of coffee and read the news. I survey the state of the union, so to speak, the state of the world, actually. There’s something about that that just, puts me in the right frame of mind to start my day. Reminds me of my place in the world. Why I’m doing all this in the first place. But the news today, it’s gotten so…lurid. So, colored…I mean, what are we doing about Syria? Just spell it out. Then there’s all this millennial-speak, the newest do-dad and such. If I hear the word millennial one more time, I think I’ll pull my hair out. Kids always think they’re the first one to do everything, to invent anything, to have a coherent thought. So I just block it out. All of this nonsense about standing in the streets, in the cold, with your little poster boards. You think you invented that? My generation, we perfected that, and what did it lead to? Death, death of all of those young men, the undermining of our military, of our whole society if you ask me. Read a history book. Oh, look at me, this is no way to start the day…no way at all.”

Paradigm 2: pandering

In this model, we think of the newspaper as a commercial product that panders to the views and values of its readership. If we think of the newspaper  this way…

…then we might think that in the future, all that we retain from the paper is its organizational structure. The paper itself becomes a generic associated press feed, organized into columns, by importance, date and theme. All of the interpretive power of the news is customized to the reader. For example, our advanced machine learning algorithms can translate the news into the language, style or vernacular of your pleasing:

If you can imagine translating the language of the newspaper, you might also translate the visual style of the paper. For example, you might translate both the linguistic and visual style of the paper to the era of your choice (Do you really belong in the ’80’s?).

Permutations of paradigms 1 & 2:

While you’re annotating the news, why not share those annotations with others? Start your own subscription channel, perhaps.

Local newspapers are closing down left and right. Maybe their value is not in reporting the news but in interpreting the news in ways that make sense to their local readership. The job of the writer is not to be the authority, to have the authorial voice, but in our (post-)postmodern understanding, the writer stands beside the reader as a guide in the interpretation of global and personal events. It’s up to the writer to convince the reader that he is a reliable guide because of the life experience, political, religious, or national allegiances, beliefs, points of view, and humanity he shares with the reader.

Let’s here a testimonial from one of the subscribers to our annotation subscription service:

Full testimonial here: “And I say, “down with the authorial voice!” You can pretend to be more important than the rest of us. You can pretend to be smarter, more worldly, more worthy. But we all know that you lucked into your position, just like the rest of us fell into ours. You’re just another guy, probably another white guy, probably from Harvard. I meet guys like you all the time.

And you, local newspapers, stop pretending like you’re doing more than wrapping up AP reports and press releases with exclamation points and sweeping generalizations. I know they’ve slashed you to the quick. I know it’s not your fault, but you gotta let go.

I’m an informed citizen. I read the news. In fact I read the news five or six times a day. Sometimes I read the same article five or six ways, just to try to figure out what’s really going on, you know, under the surface. I mean who doesn’t trust Ira Glass’s commentary, and I can just plug in the ear buds and listen to him explain shit to me anytime I want. Then there are articles that I just Russell Brand gonna go ape-shit on with some kooky video, dressed in his pajamas, and I just have to…I mean have to watch at least some of it. I follow serious stuff too. Rebecca Solnit and DeRay’s annotations are always on point. Maggie Atwood, or even Miranda July if I’m feeling a little whimsical. The point is staying informed is work, and I can’t just close my eyes and turn off my brain and assume that you know what the fuck you’re talking about.”

Paradigm 3: investigative reading

All of us are capable of reading and writing, all we lack is access to information. If you start with a blank newspaper, add an article or topic of your interest, you could  fill the pages yourself with your investigation.

Let’s hear from one of our investigative news readers:

Full testimonial: “You’ve noticed it right? A phrase that sticks out. It sounds official, like it’s always been there, but has “gentle loner” always been a category, a subsection of the sect: loners? Was my mortgage always “subprime”? Does Rick Warren “radicalize” his followers? When did the “war on Christmas” begin? And then there’s the idioms, the turns of phrase “the lipstick on the pigs” tucked away in the “lock boxes” of our minds–if you will. And we all know the coded language like “thug,” or “athletic”. I mean is it a “riot” or an “uprising”? Who makes those things up? Where’s the spin machine? I mean it’s not like it’s one news channel or the White House or a particular lobbying group. How can I read the news, I mean actively, critically, read the news, if I don’t know where this shit comes from? Who said it first? Who is the person or agency or machine? I don’t just mean your name. I mean, I want to read your Twitter stream. I want to know who pays your salary. What’s your record on this issue? All that information is out there, if you dig a little. I can fill this whole paper, the whole damn thing tracing one turn of phrase back to it’s origins, and then figuring out the origins of the origin. That’s how you learn, you know. Question what you read.”

In conclusion:

As you read through these, I’m sure that some stuck out more than others. Some of them make more use of AR and/or a gestural vocabulary than others. Some would be more successful that others if used with our system.

But none of them are intended to be built. They are intended only to be provocations. To be an imagined future that allows us to discuss and evaluate the direction of news, of reading, and of mixed reality. If there is one thing that we’ve learned over the course of the project is that the imagined world, the creative content itself, is a sort of discounted prototyping tool for the future. It is only intended to last as long as the experiment, and then the prototype is to be destroyed and forgotten.

YouTube / Bloomberg Business – via Iframely

(This is less advanced, but you can see more of the interface:)

YouTube / Steven Henderson – via Iframely

…then, why not play the idea of an AR how-to that asks us to think about our world differently. Our set-up involves virtual touch. What does it mean to virtually touch something?

I am considering themes:

how to perform…

safety

resistance

respectability

tolerance

freedom (of speech)

In just the last week we’ve seen these performed with

football helmets, cameras, posters and sharpies, hand-holding, black clothing, poetry (Claudia Rankine’s “Citizen,”), not to mention horrible violence.

Thinking about how-to’s:

The Freedom Fighter's Manual (CIA pamphlet dropped over Nicaragua in 1983)

Make your own Newspaper Headlines

uniteyouthdublin.files.wordpress.com – via Iframely

capricorn.org – via Iframely

antenaantena.org – via Iframely

Then there are survival guides:

Pocket Survival Guide

Also thinking about James Pierce’s work on designing resistant interactions (having just seen his thesis presentation last week). And the instruction manual that accompanied the Obscura 1C Digital Camera:

jamesjpierce.com – via Iframely

Google Docs – via Iframely

I didn’t just see the horrible image once. The one of the child, Aylan Kurdi, lying there dead in the surf. I saw it again and again every time I opened Facebook or Twitter, every time I scrolled down the page. When faced with images of death and destruction, my first response is to turn away, to assure myself that I am aware of the evil in the world, that I’ve seen suffering, that it is not required that I feel the sorrow just this instant, just because Facebook says I should. But as the image was repeated, I dared myself to look longer, to see the details, the grooves on the soles of his shoes, his curled fingers. The more I looked, the less real it became, the more holes there were in my vision, the more aware I was of the limits of my screen. I was confronted with what I don’t know about Syria, about migration, about the other dead children.

To photograph is to appropriate the thing photographed. It means putting oneself into a certain relation to the world that feels like knowledge — and, therefore, like power. — Susan Sontag

In this sketch we will explore our desire to make an image real, to make it known and understood, and our frustration with the limits of that knowledge and experience. The audience member will approach a screen with an image (perhaps video) of a refugee camp, as he approaches the screen the image will fade as we zoom in on a 3-D rendering of the same image (created in agisoft). The closer to the screen the audience moves, both the more detailed and the more distorted the 3D image will appear.

We are asking the spectator to stand in our shoes, to look closely, to look from a distance, to choose their distance. We are relying on the limitations of the media (the 3-d rendering) to mediate (and complicate) the experience of the audience. The secondary audience, those watching the participant approach the screen, will have the opportunity to interpret the choices and responses of the participant, to judge them as they see fit. In this way, the secondary audience is watching a Brechtian piece of theatre. When it is his turn to participate in the experience, he will be able to judge his own choices and reactions against those he has watched experience it. But in the moment that he approaches the screen his participation, his freedom to choose, while not truly Artaudian, it does lie in the experience, in between thought and gesture.

Emoticam is an application that runs in the background of a user’s computer at all times, monitoring what they type. Anytime it detects an emoticon, or some other expression of real-world emoting, it takes a photo of the user’s face and uploads it to the project website. Visitors can view all images, or choose to filter to one specific term.

Emoticam

I chose this piece to represent two related skill sets, one abstract and one specific. Specifically, myself along with the MYNDWYRM collective of artists built this piece in Twilio, a web-based phone automation system. (We used OpenVBX for ease of use.) So I have experience building interactions and systems in Twilio/OpenVBX.

Secondly, this piece is one of about a dozen fully realized works I have made that focus on participation and interaction at the border of live gaming and performance.

This app was intended to imitate the core aspects of graffiti, but with sounds instead of visual art. Newer sounds will overwrite old sounds if dropped in the same location. This project was collaborative as I worked with another developer who created an android version.”