We Dwell Below

I was contacted by Gabrielle Jenks, Director of Abandon Normal Devices, in May 17′, about creating a VR experience for their biennial festival, which took place in September 17′ in Castleton, a village inside Derbyshire Peak District National Park of UK, with tons of caves! Together with Planeta, we created something crazy, unusual, and definitely push the boundaries of VR.

Concept

Inspired by the cave theme, I had an idea to transform normal people into cave dwellers. From an ordinary, clean, constraint person, to a bizarre, messy, loose wilder who acts freely! I imagine it’s the process of traversing the underground together, doing weird things together, that make the bounds between participants and shape whom they become.

So, we settled down the idea to transform participants as cave dwellers, but what weird behavior can we ask the participants to do?

Around that time, I was developing a Chewing Device for Google Cardboard for my Daily Life VR Ch. 2 Eat (see my instructables here!). Chewing Device is an analog mechanism to bring in the “chewing data” of real life, in order to reenact the chewing behavior in virtual world. Mouth apparently can do a lot of things, what if besides eating, it can also dig the ground? The image of people “chewing at the floor” together is so ridiculous that we all agree it is something worth of experiment.

Analog chewing device in action with real food!

Process: Interaction

Since we are bring in the chewing data, we need to use the most of the mouth / jaw movement. First of all, mouth is the only way and mean to travel down the ground, literally open & close mouth close to the floor, a.k.a. digging the ground.

We encourage participants to go wild and loose, so in experience participants can hear their voices in an echoey way. And the voice volume will generate gibberish, inspired by the 32 categorized symbols being used in Ice Age (source).

With mouth, participants can also eat stuff (meat), and puke stuff (hair ball, rock, etc). And when a participant opens mouth aiming at another participant, he/she will puke heart. And surprisingly that makes opening mouth as a major way to communicate! I’m so thrilled with this non-verbal communication, and really think it helps to shape the behaviors to be raw and primitive, and fun :)

Also, body contact is something we all love to encourage, so whenever participants touch others, random “twig” will be generated and planted on body where the contact point is.

Process: Content

Avatar – I used FinalIK library for creating VR avatar, and the result is quirky but at the same time very effective. The avatar looks like a possessed doll, and knowing in it is a real human being is a very strange and strong feeling. Throughout the journey, the participants will transform (both visually and mentally) from normal human to wilder, hence the shape and texture of the avatar will change every level down as well.

As for environments, it’s the part I probably spent the least time on because of the tight timeline. Luckily with Dan Brewster and Nick Dangerfield’s advice and encouragement, we settled on the heavy hand-drawn textured style.

↓ before

↓ after

Technology

We Dwell Below roughly has four parts involved: 1) Chewing Device, 2) Multiuser (networking) system in Unity, 3) Costumes, 4) Story.

Chewing Device

For this project, I need to redesign the one used with Google Cardboard + web browser completely, for it to work with HTC Vive + Unity3D. I laser-cut the base and used leather for strong strap in the end. Circular pressure sensor (FSR) for the input.

I made the prototype with FSR sensor and Arduino, and Joseph Saavedra helped on perfecting the hardware design and making! It uses Teensy board in the end. The wonderful chin strap is made by Kelsey LaSeur.

Multi-user / Networking VR in Unity

This is really a pain in the butt lol

I’ve spent a lot of time on the Unity Networking system. The documentation improved a lot but it’s still quite limited in 2017. Websocket is so much clear and transparent compared to this!

According to the latest news, Unity is going to re-write all its way to do networking (2019), so I guess all my efforts won’t be able to reuse anymore hahaha (cry cry).

Still, I could share some thoughts on multi-player in VR— it’s really fun to be able to interact with other people both virtually and physically. It is still awkward to interact with strangers, but with people you know, it feels like going through a great trip together.

Costumes

Suggested by AND Festival to have some elements of the scenography manufactured in material which was wearable like my project brain, we went a bit far :) Based on the cave theme and idea of participants having some sort of tribal ritual together, costume designer Kelsey designed 4 big coats and head-wears that are easy to take on and off. All 100% hand made!

To reduce infrared blocking for HTC Vive sensors as few as possible, we used mostly mesh fabric for the headwear.

For the 2018 Newcastle tour, we modified the helmet to make it stronger with more obvious face outlines. I used steal wire to make difference eye contours.

Final Installation

I am very lucky that Planeta, where I work at, was interested in co-producing the piece, or this would never happen in this short amount of time! We were all working hard until the last minutes, in the stone cottage near the installation site.

The project turned out to be a huge hit – thanks to the arrangement of AND Festival, we had a tent thus looked very secretive and seductive ;) Also the costume and weird behaviors with the Chewing Device, both make the experience not just interesting to try, but also entertaining to look at.

In 2018, we brought the project back to England again, this time a two day installation in The Great North Museum: Hancock in Newcastle. The participants are mostly kids this time! :D

Sleep Together

Sleep Together is a virtual reality group experience for people to use before going to sleep. In 60 seconds, participant (a.k.a. sleepyhead) will go through a series of breathing exercise, with other sleepyheads floating inside a big bird nest, breathing together.

I’m very lucky to make one of the 10 #veryveryshort projects, produced by NFB and ARTE, and co-commissioned by IDFA DocLab. The idea is to create something short yet sweet, because “Life is short, very very short.”

Continuing the idea of Daily Life VR – a WebVR series about life’s most basic and essential pursuits, I installed Sleep Together for the sleep aspect of life.

Storyboard

Moving Schematic

Extracts of Experience

And I’m also fortunate to share it with people in IDFA DocLab 2017 :)

Chewing Mask on instructables!

Yeah finally make one! Time to wrap up the Chapter 2 Eat of Daily Life VR!!!

www.instructables.com/id/Chewing-Mask

chew chew chew!

(so long, too long…)

MASK v2.0 – Unity!

Unity3D version of MASK!

Since it’s much more stable with App than with Webpage, maybe it’s worth a try to make a unity version of MASK collection. Hmm such a sweet dilemma!

Codes of accessing camera on the phone… omg so simple!

WebCamDevice[] devices = WebCamTexture.devices;
mCamera = new WebCamTexture ();

//for(var i=0; i<devices.Length; i++){
//	Debug.Log(devices[i].name);
//}

if (devices.Length > 0) {
	mCamera.deviceName = devices[0].name;
	displayPlan.GetComponent<Renderer>().material.mainTexture = mCamera;
	mCamera.Play ();
}

Summer Throwback Three: We Are What We Pretend To Be Inc. established!

The third major experiment I had this summer was a street car journey. At first it’s just another experiment with Unity, but then it leads me to an idea for the project I want to do this year, as the Research Fellow at ITP.

I decided to establish an incorporation, called WE ARE WHAT WE PRETEND TO BE Inc..

It’s inspired by the quotes of writer Kurt Vonnegut:

“We are what we pretend to be, so we must be careful about what we pretend to be.”

I discovered this when I was doing research for my thesis, and it inspired me a lot to shape the idea, and to make the MASK series.

WE ARE WHAT WE PRETEND TO BE Inc.(WAWPRE) is an entity for my series experiments about the relationship between our free will and unfree (uncontrollable) will. Can I change my unfree will by change my free will? How can I shift self identity by ultering my and also others perspective?

The first product of WE ARE WHAT WE PRETEND TO BE Inc. is MASK, a customized VR headset that helps the user become what they want to be.

The second service of WE ARE WHAT WE PRETEND TO BE Inc. is THE TRAM.

It’s an installation that provided customized virtual reality experience that users (aka passengers) can design their tram journey based on their mood, destination, favorite color, and cloth to wear. Also from the cutout hole on the cardboard box, which is usually covered with curtain, I will take a picture of the passengers, so the passengers will be able to see themselves in the VR.

WE ARE WHAT WE PRETEND TO BE Inc. THE TRAM takes you to wherever you want, fulfill whatever desire you want, therefore you become whomever you want. You set yourself free.

I joined Internet Yami-Ichi in New York this summer, and displayed my little handcraft Home Depot cardboard box as an installation + experience seller. For the software, I used Unity and started developing it on and off since August, since I was notified that I was selected to join the event. As for the box, I made it with recycled materials I found in junk shelf in one night before the event! Holy cow SO NERVOUS.

It’s really a successful experiments, for first ever user test! The users used it for a long time, average at least more than 1 minute! Which is a huge difference for me.  I think that’s because the tram keeps moving, the scenery outside the window keeps changing, plus there are ghost passengers getting on the bus when the tram stops at the tram station, the user is expecting things to happen, thus enjoys the VR experience for the longer time.

Screen captures with different customized setup

Contents development

It’s a subtraction process, because of limited time haha

You can escape into different spaces, under different circumstances.

Different customized options & graphic design

Ideas finalizing

Internet Yami-Ichi 

Thank you all my lovely first ever passengers<3 <3 <3

Let me thank you with this passengers all together gif <3<3<3 Thank you guys!!!