CLOUDS is a documentary exploring creativity through the lens of code
Over the last year we have captured interviews with over 30 new media artists, curators, designers, and critics, using a new 3D cinema format called RGBD. CLOUDS presents a generative portrait of this digital arts community in a videogame-like environment. The artists inhabit a shared space with their code-based creations, allowing you to follow your curiosity through a network of stories.
What does it feel like to think with code? How can emerging technologies enable us to actualize our dreams? How has online sharing transformed the way artists collaborate?
A preview of the CLOUDS film, science fiction author Bruce Sterling speaks aloud about the art of code.
The creative practitioners featured in the documentary represent a new breed of interdisciplinary artists who combine software engineering, audiovisual design, and cultural engagement. CLOUDS explores the themes of creativity, aesthetics, simulation, and sharing articulated by these luminaries in the computational arts community.
Counter to the widespread perception of code as exclusively utilitarian, this generation of hacker-artists struggles to develop new forms of visual expression, expanding the vocabulary of technology to resonate at a deeper human level. In an increasingly data-driven culture, our subjects act as intermediaries who can help us to navigate and make sense of our evolving digital universe.
Storytellers featured in CLOUDS
The interview subjects in CLOUDS include Bruce Sterling, Casey Reas, Daniel Shiffman, Diederick Huijbers, Elliot Woods, Golan Levin, Greg Borenstein, Jer Thorp, Jesse Louis-Rosenberg, Jessica Rosenkrantz, Joel Gethin Lewis, Josh Nimoy, Julia Kaganskiy, Julian Oliver, Karolina Sobecka, Karsten “Toxi” Schmidt, Kyle Chayka, Kyle McDonald, Lindsay Howard, Marcus Wendt, Marius Watz, Nick Fox-Gieg, Paola Antonelli, Philip Whitfield, Rachel Binx, Regine Debatty, Satoru Higa, Shantell Martin, Sofy Yuditskaya, Theodore Watson, Vera Glahn, and Zachary Lieberman.
A documentary in a videogame environment
The CLOUDS experience is generated from a database of richly tagged interviews. For you, the viewer, the story begins with your curiosity. You input a question or search term as the starting point, setting the initial trajectory. Just as an everyday conversation will often flow through series of digressions, each subject you encounter offers branching paths to unexplored regions of the video collection. A unique narrative experience is generated just for you. You can sit back and watch the story conversation unfold, or let your interaction with the system guide you toward surprising outcomes.
The CLOUDS documentary comes in the form of an application for Mac or Windows, presenting a full-screen, immersive, interactive audio-visual experience.
Be the first to experience CLOUDS: own a collectible data capsule
Premiere screening at Eyebeam NYC
If you are in the NYC area, consider donating for a ticket to our launch at Eyebeam in Chelsea! There you’ll see the film presented as an installation, meet the filmmakers and catch a few CLOUDS participants in person.
In May we premiered Clouds: Beta, a pilot version of the project in a short video screened at Eyebeam. Since then, we have continued production with the intention of building the documentary in an interactive environment..
A film about creative technologies, using an emerging medium
CLOUDS is created with the RGBDToolkit -- an open-source software library we have developed for 3D filmmaking with the Kinect. Using a Microsoft XBOX Kinect depth sensor paired with an HD video camera, our toolkit makes possible the computer-graphic/video hybrid you see above. By supporting CLOUDS, you also support the development of the RGBDToolkit -- helping to make it better for everyone. We welcome you to try it out!
The toolkit package contains 3D printable mounts to securely fasten the cameras, and custom software for calibrating, recording data, and visualizing it as full color three dimensional forms. Check out the RGBD tumblr to see the spectacular projects by other filmmakers who have adopted this format.
Help support RGBDToolkit version 1.0
In addition to completing the film, we will be releasing version 1.0 of the RGBDToolkit application. This will include several new features requested by the community including support across Mac, Windows and Linux, integration with Maya and Cinema 4D, easier and more accurate camera calibration, and new effects!
If you’re interested in making your own movies using this technique, check out the "Filmmaker" reward or host your own depth filmmaking workshop.
How will you use the Kickstarter funds?
We need to expand our team to realize the complete interactive version of CLOUDS. Combining the workflows of documentary filmmaking and videogame production in a new format will involve coding, art direction, sound, editing, and interaction design. The majority of the funds will go to supplementing our two-person collaborative to a small team of co-creators.
We are working closely with several of the coders featured in the film to create new generative works embedded in the world of CLOUDS, allowing these artists to co-exist with their own creations in a computationally generated space! The Kickstarter’s success will ensure that we can compensate the artists as we collaborate to incorporate their work into the film.
It takes a community to make a film of this nature. To reach this stage in production, we have benefited from the help of friends and colleagues, including several key media arts organizations. In particular, we'd like to thank The Frank-Ratchye STUDIO for Creative Inquiry, Eyebeam Center for Art and Technology, YCAM InterLab, and NYU's ITP program for supporting us as we pursue the dream of an interactive documentary.
Your contribution to this project will ensure that the voices of these artists will be heard in a way never before possible.
James and Jonathan
Risks and challenges Learn about accountability on Kickstarter
Adopting a real time paradigm
The RGBD data is computationally intense to visualize. We've already achieved smooth frame rates using this format for real-time installations, such as “Spectacle of Change,” (http://jamesgeorge.org/works/spectacle.html) so we know it can work. But creating the full interactive experience required by CLOUDS, that can run on a modern laptop and includes visual design, generative storytelling, and user interaction will require advancement in our techniques. We’ve already begun experiments that move the computation to shaders on the GPU, and are excited to overcome these challenges using data pre-processing, optimization, and design.
Extracting compelling narratives
As this is a documentary, the story is crucial. An essential component of our process involves designing algorithms to explore a database of ideas and conversations. Taking a generative approach to storytelling runs the risk of producing sequences that feel disjointed or repetitive. Applying knowledge from natural language processing and recommendation software in conjunction with exhaustive tagging of our content, we can craft a system to customize the experience based on your current interests, leading you to encounter content that personally resonates. All this will result in a Story Engine which offers audiences many access points to this world of art and code.
Have a question? If the info above doesn't help, you can ask the project creator directly.