Leeuwarden, Fryslân – Valletta, Malta
An interactive experience for the Leeuwarden 2018 – Valletta 2018 European Capital of Culture.
Made for & with Leanne Wijnsma.
I was approached by artist Leanne Wijsma to work on a project for an upcoming exhibition in Leeuwarden. Leanne had scanned tunnels tunnels in Valletta and Leeuwarden, sister-cities and Captitals of Culture 2018. This idea was to join these two cities with an interactive application, in combination with a documentary on the history behind the tunnels in Valletta.
The project presented two challenges:
How to convert the Laserscan data into a real-time interactive application,
And how to do so in a limited amount of time.
Prerequisites
The data I had to start with was IR laserscan data, that was stitched together.
The end result had to be an application, “game”, that would be projected on a wall, that anyone could control.
I chose to develop the application in UE4.
The laserscan data would be processed in Meshlab, cleaned up in Blender, and brought into Unreal.
Processing
I brought the data into Meshlab.
Meshlab is fun, but very unforgiving. Its userinterface leaves much to be desired, it crashes without warning or debug information, and documentation comes in the form or research papers.
Once I had a workflow going, the processing the scan files went almost automated.
I used Screened Poisson Surface Reconstruction to generate a mesh. I reduced and unwrapped the mesh in Blender, and cut holes where doors would go.
Then, I baked the color of the pointcloud into a texture, along with a normalmap for lighting in UE4.
Gameplay
To keep the interaction as simple as possible, I designed some sort of waypoint system that would allow the person to move from A to B. A natural, but not too disorienting walking animation would help the user understand the space around them. The system works by planning out steps and animating between them.
I used a shader to project a straight line across the floor, to help show where the user would go.
I gave the user a flashlight to light add immersion, and a light barrel distortion gave me the option to use quite a wide FOV without serious distortion.