As I sat entranced by the movement of light particles across laser-scanned environments dancing across the contour of the domed screen, I knew I was experiencing something special. The sound resonated through the space and enveloped the audience in an electronic cinematic soundscape that matched every motion of the visuals on the dome. This experience is what this medium is about. This is an example of the magic and wonder fulldome art can exude on the world. This was Remote Sense by Jonny Knox.
Jonny Knox is a digital artist and designer who creates and builds immersive experiences and installations including stunning fulldome productions. With a background in music, architecture, and computer graphics he taps into his diverse skills and experiences to bring to life traditional narratives in a modern and digital light. Currently, he is working on his company Sublime Digital, a startup pioneering the future of digital realities including VR, AR, and shared immersion experiences like fulldome.
I experienced a live performance of Remote Sense at iX Symposium 2018 hosted by Montreal’s Société des arts technologiques (SAT). The piece utilizes Lidar (laser scanned) data of architecturally marvelous constructions as the basis for the visual experience. He teamed up with sound artist Darien Brito and the combination of the sound and visuals is truly breathtaking.
In this interview Jonny Knox discusses working in fulldome and the process behind creating Remote Sense.
How did you initially become inspired to create for fulldome and what led you to start creating for fulldome?
I remember as an architecture student, how frustrating it was trying to design in 3D software, the problem was is translation between 3D (actual) space and 2D (screen) space; I realised that really I needed to be in there with the 3D model! This got me thinking about what a native home for digital 3D space might be. Then in 2012 I was offered an opportunity to spend 12 months developing a creative idea with commercial potential, I chose fulldome! And now in 2018 we are a company (Sublime) with offices in Glasgow, London and Dubai designing and installing domes, spheres and experiences, all over the world. I love the freedom of the dome, it’s the geometry itself but also the magic it can instill is like no other medium.
I was lucky enough to see your performance 'Remote Sense' at the iX Symposium at SAT. What were the inspirations behind this performance?
Remote Sense was my opportunity to play, to develop something hopefully unique and to test some of the ideas and theories about immersion I have been developing professionally and personally. The idea was to create a live AV performance piece that had the cinematic quality of film, to test the boundaries between both mediums and to find a common ground, to discover a way of working which is both linear and non-linear at the same time.
Im fascinated by ancient artists, the cave artists, the shamans, how they worked, how they were inspired, what were the process's or states of the artists, and why did they make art in the first place! Remote Sense I guess is a homage to this.
The visual element of 'Remote Sense' is comprised solely of Lidar captured data. Would you give a brief description on what that means and discuss some of the processes you had to go through to be able to create the visuals for the performance?
Lidar capture, (or remote sensing), is the collection of precise x, y, z points in 3D space by pulsing laser light, the culminating millions or billions of points make up accurate models of buildings, landscapes and objects. I've been fascinated with the medium for years but it was always represented on a flat screen, and usually as a still image. Traditionally, if that is the right word to use for cutting edge technology, Lidar is extremely CPU intensive so I designed a system that converted cpu data to gpu data allowing for the rendering and blending of millions of lidar points in realtime in a dome. Each Lidar scan effectively becomes a single texture for position and a single texture for materiality, with this I work in a similar fashion, but in full 3D, to a VJ who blends and composites between 2D images and video sources.
I really enjoyed your use of light in the piece. I had previously described it as “shining a flashlight onto the walls suddenly revealing a magnificent structure that had been hidden in the darkness.” Could you discuss how you find light to be embodied in your piece?
Light, or illumination, is very interesting to me, the experiential interplay between dark and light has many layers of meaning on a consciousness and phenomenological level. The struggles we encounter, the battles we overcome, our fears and our strengths are all primordially pinned to our understanding of light and dark.
Remote Sense was as much a work of self therapy as anything else! I used the creation phase to express and to give light to ideas through my work. I’m not sure if it comes across but many of the scenes in Remote Sense are representations and working drawings of an inner transformative period of my life. So yes, light is very important to this piece!
And also, Lidar data, by its very nature, is derived from and exists as, light.
This is a live performance, what are the elements that you are controlling live?
Initially I was in full control of every aspect of Remote Sense, from timeline, to cameras, to all geometry, texture and material animations but I felt a loss of control, of cinematic framing, so I began to keyframe a backbone of scenes and cameras so I could concentrate on a more granular level, to spend more time looking up rather than down and to be more at one with the audience and the overall experience.
How would you describe the interaction between you as a performer and what is being projected onto the dome?
Im controlling everything on a midi controller, I arranged the keys and knobs in such a way as that I don’t have to look down so I can concentrate on the dome, the output that actually matters! Each dome is different, has its own nuances and so I also colour grade in realtime.
You also worked with sound artist Darien Brito. How did this collaboration come to fruition?
I had been experimenting with the basic Lidar realtime framework I had built when I decided it was time to bring audio into the mix and with the power of social media I met Darien on a forum! We worked collaboratively and remotely, I would send him visual material which he would work up sonically, and vice versa, sometimes Darien would extend the audio beyond the visual timeline inspiring me to design to his creations. It actually was a really great collaboration and working experience!
You are currently touring Remote Sense. How many different venues have you presented the piece in and what challenges do you find in taking this piece to a variety of different domes?
Remote Sense, has to date shown in FDUK (UK), Cryptic Nights (UK), Sonica (UK), AADN (FRANCE), EFFA (AUSTRALIA), The Satosphere (CANADA), TodaysArt (HOLLAND), and Mutek Dubai (UNITED ARAB EMIRATES)
Indeed, each dome is very different and with no standards in place I find live grading and tuning audio and visual essential. And with the system rendered in realtime I can quite quickly make adjustments for dome tilt, orientation and even FOV (as in the case of the Satosphere).
What are your hopes for the future of fulldome?
I believe we are seeing the beginning of an immersive revolution, spurred on in part by VR and AR, and now immersion is very much a part of the creative strategy of tomorrow. Fulldome has a bright future however it really needs to evolve with the times, we need it to become even fuller dome! Fulldome is concerned with looking up to the stars however spherical projection looks everywhere and this is what it need to become to embrace the many use cases in shared immersive media.
What do you want to say to people interested in this art form?
To the artists I would say go for it! This medium is such a rewarding environment to create for, there is literally nothing easy about it, and perhaps this is part of the fun but things are certainly getting easier with new tools, processors, better computers etc. When I first walked into the Satosphere in Montreal I felt as if I were walking into a modern day cathedral, and in this new cathedral the forms are temporal, the emotions ever changing, the atmosphere electric and the sense of being apart of something genuinely transformative and special, is absolutely alive!