The Lifespan of Ephemera: Reflections on Collaborative Art and the Embodiment of Data

This chapter offers a retrospective account of an art project called “Knowing Together,” which was performed at Columbia University and later exhibited in both New York and Los Angeles in 2018-9. Although the project was conceived during a longstanding collaboration between the two authors of this chapter, artist Rosalie Yu and media scholar Charles Berret, the work was deeply collaborative in several other respects. The project showcased a novel imaging technique called collaborative photogrammetry, which was employed for the first time during a workshop involving twenty participants, and both this workshop and the exhibition that followed were collaborative efforts with a dedicated group of creative technologists at Columbia.

We, the authors, first began collaborating in 2016 at the Brown Institute for Media Innovation, a research center based between Columbia Journalism and Stanford Engineering. The Brown Institute was founded with a goal of developing new storytelling techniques, and its unique environment encouraged open exploration of creative projects by artists, journalists, computer scientists, and others interested in new forms of narrative. This collaboration grew out of mutual interests we discovered as we initially worked on separate projects at the Brown Institute. Rosalie’s practice centers on the experiential nature of art and technology, so Charles’s background in the history and philosophy of technology made for productive conversation. We both enjoyed asking questions about the nature of digital media that we didn’t know the answers to — questions that possibly didn’t have concrete or straightforward answers, but nevertheless rewarded the challenge of pursuing them. Just trying to answer those questions was essential in solidifying our key point of inquiry for this project: the entanglement of embodied experience in creating a collaborative dataset using the principles of data feminism (D’Ignazio C and Klein L, 2020).

We conceived of “Knowing Together” as an experiment testing the mundane limits of conventional 3D modeling techniques by centering embodied experience over objective realism. Photogrammetry is a means of creating 3D models by stitching together multiple photographs of an object from different angles to capture a static digital mapping of its surface (Debevec et al. 1996). To view an object from every angle in highly granular detail carries the promise of absolute fidelity, an ideal representation of the object to be viewed onscreen or printed. This is how photogrammetry stands between the domains of photography and sculpture, typically offering a convincing imitation of physical objects according to principles of photorealism. Museums often use photogrammetry to catalog and archive highly detailed, physical models of items from their collections. These 3D renderings are frozen in time, stored as ideal records of an object’s shape and contours, sometimes even used as digital building blocks for other applications. Departing from these conventional uses of photogrammetry, we wanted to push the limits of this highly representational artform and explore its deeper expressive potential beyond the scientific appeal to some stable mode of absolute truth. Our goal was to test whether collaborative photogrammetry could offer a means of capturing the momentary, fragile expression of embodied experience in three-dimensional art.

With “Knowing Together,” we thus inverted the conventional logic of photogrammetry by centering time and subjectivity, which are dynamic, situated factors usually excluded from 3D models of physical objects. We started by designing a workshop that would create an immersive, embodied experience for participants—not immersive in the artificial sense conventionally used for virtual reality applications, but rather experiential immersion in heightened awareness of one’s mind, senses, body, interpersonal relations, and surroundings. To this end, we imagined a workshop (described below), in which we designed an experiment that would challenge a group of people sharing an experience to create something more formally expressive and personally meaningful through photogrammetry than simple photorealism. While it’s easy to look at emerging technologies and technical skills as valuable in themselves, we wanted to teach the use of photogrammetry in a way that challenged the prevailing wisdom about how to use this technique and what we should value in its results. To this end, we designed a collective experience that could be captured and modeled through unconventional use of photogrammetry, rendering software, and 3D printers. It was important to us that the collective element of the experience be reflected in our fundamental critique in this project. A 3D model made through photogrammetry does not, in itself, convey the perspective of a single, universal observer. Instead, photogrammetry captures a multiplicity of viewpoints surrounding the object. Our collaborative approach to photogrammetry treats each participant as the source of a single perspective among the many needed to construct the sort of synoptic image a conventional 3D model appears to be, and yet still preserve the noise generated through the human idiosyncrasies of the image-gathering process in this workshop. In other words, we wanted to test the expressive limits of 3D scanning techniques using critical and creative approaches where conventional photorealism was not our core objective.

We found support for this project through EdLab, an organization similar to the Brown Institute in its focus on creative technology, but based across the Columbia University campus in Teachers College (TC), a school devoted to educational research and the training of teachers. EdLab’s mission is to “engage in conceptual development, demonstration projects, and new educational research to explore and document diverse possibilities for the future of education.” EdLab announced a call for funding from the Myers Foundation Art fellowship, a grant intended to bring art with educational significance to the students and staff of TC. We applied, they awarded the grant, and thus began a cross-campus collaboration combining art, media, and pedagogy. What appealed to EdLab about the proposal for “Knowing Together,” according to former lead designer Zoe Logan, was the project’s “beautiful melding of education, practical experience, and technology as an entry point for a more meaningful, personal connection. Art does not necessarily result from just using a 3D-scanner, although that can be an excellent draw. Such was the case with ‘Knowing Together,’ a piece that leveraged curiosity and enthusiasm for scanning technology and collaborative art in the service of meaningful interaction.”

A Collaborative Photogrammetry Workshop

The workshop was a day-long exercise exploring intimacy and vulnerability, in which groups of participants platonically embraced for about 10 minutes while the other participants collectively captured images of them by forming a circle and taking turns snapping photos with a single-lens reflex (SLR) digital camera. When combined, these photos yielded a 3D image of each performance. During these performances, a microphone between the subjects captured sounds that were not otherwise audible: breathing, heartbeats, and whispers.

The project was appealing to the TC EdLab because it centered on a workshop combining instruction in an advanced technological skill (photogrammetry, fig. 1) with a hands-on exploration of physical boundaries and intimacy (the workshop, fig. 2) for the sake of creating artworks (the sculptures, figs. 3, 4, 5). This offered an unconventional approach to each of these subjects, merging performance and digital image-making in a pedagogical setting that was inherently collaborative. The workshop was designed to teach participants an image capture tool and turn it into sculptures which the students could return and see in an exhibition. “Knowing Together” was conducted at a venue called the Learning Theater, an interactive education space located at TC’s Gottesman Library. The Learning Theater is a flexible space with moveable walls, and as each phase of the event progressed, participants were literally discovering new areas designed to facilitate their work.

We selected seven groups for the workshop, mostly pairs of strangers, who would each pose for collaborative photogrammetry and become the subjects of 3D-printed sculptures. The experience was meant to induce a natural feeling of discomfort as these groups figured out how to embrace one another, transgressing a basic boundary of physical intimacy, albeit in a safe setting. To make it feel safe, we developed means of establishing consent, discussing boundaries, and building trust. Before the image capture began, the workshop included an exercise in platonic intimacy, with one minute of eye contact and two minutes embracing a complete stranger, giving them your full attention. Rosalie and four EdLab staff were present during the workshop, and everyone was free to leave for any reason at any time. Despite these precautions to help participants feel comfortable, some level of discomfort was actually a key component in the experience curated for the workshop. Many participants noted the phenomenal strangeness of the performance itself, reporting that their embraces felt much longer than a few minutes, just as any unfamiliar experience can distort the perception of time. As the exercise progressed, many reported that their personal barriers fell as the experience moved from feeling awkward and contrived to becoming comfortable, meaningful, even powerful. For these participants, an initial feeling of discomfort was steadily replaced by empathic connection, streams of ideas, and even a feeling of physical disembodiment in a few cases.

“The act of intimacy, of consensually breaching personal space, shared and documented by participants, reached deeper resonance by requiring stillness as the camera was passed along the outer circle, a process which took several minutes,” Logan recalls. “The importance was clear to see: that moments of affection, of presence, and of inclusion are fleeting, but universal and precious, even among strangers.”

Another EdLab staff member, Ruta Kruliauskaite, also remembers the inclusive elements of the project as especially rare and important. “What was actually nice about artistic-academic collaboration, in my view, was to allow the participants into the art process,” Kruliauskaite said. “The workshop also allowed everyone to learn the more academic side of the art, which introduced them both to deeper, conceptual knowledge as well as understanding what photogrammetry is. Usually that’s limited to one paragraph next to the art piece in the museum.”

After the workshop, we asked participants to share written reflections on their experience that came to mind in the following weeks. These reflections helped us to better understand their thoughts and feelings during the workshop and the significance it would come to have in their later recollection. Selections from these participant reports were posted on the gallery walls alongside a full set of source photos, maps of the photo capture patterns, and the sculptures themselves (figs. 5, 6). Some reflections were especially poignant, displaying the participants’ willingness to be open and vulnerable not only with each other, but also with the unknown audience of the upcoming exhibition.

One participant, Jasper Lo, found that the workshop led to lasting philosophical questions. “Here’s something strange I’ve been thinking about: how long is the lifespan of the ephemera we create? When I embraced Jarret, or as we passed around our cameras, I wondered if we had prolonged the lifespan of our print with each extended minute.” This reflection suggests that the format of the workshop—at once technical, performative, and intimate—gave meaningful context to the otherwise mechanical activity of capturing images for photogrammetry. Rather than teaching the group to make a perfect 3D image of a teacup or other static object (as many photogrammetry exercises would), our workshop instead centered the human role in gathering data, and revealed that the data is inherently embodied, situational, and imperfect, but all the more meaningful for these reasons.

Another workshop participant recalls that during the embrace she began by closing her eyes, then pictured various scenes to calm herself, and eventually lost track of time even during this relatively brief exercise. When the pair began to pose, she said her legs were shaking with anxiety; by the end, she remembers her legs falling asleep. Indeed, for many participants, what began as an emotional challenge eventually became a physical one. In conventional photogrammetry, the object must remain completely still throughout the scanning process, but this is impossible for human subjects. Every participant moved as their image was captured. Some shifted their weight from leg to leg. Others squirmed or fidgeted. A few even slumped with fatigue or lost their balance, creating a highly revealing sort of noise in our image data that embraces the tactics of aesthetic dissent found in glitch feminism (Russell, 2020). As a result of this noise in the relatively small number of photographs for each performance, the final 3D scans present an expressive and highly revealing range of glitches that embody the group’s collective work as both performers and photographers. Even the room, its moving shadows, and ambient lighting created unintentional visual artifacts that are preserved in the sculptures for “Knowing Together.”

Exhibiting Our Images, Data, Writing, and Sculptures

Although we gathered a wide range of data during and after the workshop — digital images, maps of the camera’s movement, audio and video recordings, verbal and written reflections — it was not immediately apparent exactly what the creative output of the project should be. We knew we had a gallery space available in the Gottesman Library of Columbia University. We expected to create at least some form of 3D printed sculpture since this is a major focus of Rosalie’s ongoing artistic work. But would these sculptures take a familiar form despite the unconventional data we had gathered? Could our data even yield a model that a 3D printer could handle? And would a 3D sculpture alone attest to the meaning and significance that participants reported experiencing?

We stitched together various photos of each performance to generate 3D models, and full photo sets were exhibited alongside the sculptures as source material (Fig. 6). The exhibition at TC also included a booklet of images, writing, and participant reflections designed by Kimberly Gim. In both the booklet and exhibit, each piece depicts one of the group embraces as a model made from the various photos taken by other participants. As noted above, the noise in these datasets produced many visual anomalies that attest to the collective process of capturing these images. In the pose between Jasper and Jarret, the two broke their embrace halfway through, resulting in a telling gap or absence in the resulting 3D model (fig. 7). Rather than view this as a flaw, the scan and sculpture of their performance contains a void that attests to the process and its fragility. In Jasper’s case, this void takes on even greater significance as the source of his later reflections on the lifespan of ephemera.

“Every person who participated in the event had a stake in the gallery show,” Logan noted when we asked for her recollection of the project. “Some were recognizably the subjects of the sculptures, but everyone was an author of the work. In a world where the primary artist retains so much possession over a final piece, this first instance of ‘Knowing Together’ generously spread the recognition of collaborators by including their names, on the wall of the gallery and the exhibition booklet.”

After the first exhibition of “Knowing Together” in New York, we submitted the project to the ACM SIGGRAPH conference in Los Angeles, a venue for research and development in computer graphics. To our surprise, the organizers accepted both the sculptures for display in the conference art gallery (Wong et al., 2019), as well as an essay about the project for its art papers track (Yu and Berret, 2019). While it is rare for a project to be included in both venues, our work combined a concrete creative output and an academic component in the form of an experiment in the theory of digital images. In other words, this project had a greater volume of output than many other contributions to the conference due to the nature of the collaboration itself. This signals a strength inherent in many collaborations between academics, artists, and other cultural institutions: if you recognize the broad range of potential outputs in different domains touching your work, the overall yield of the work can extend beyond the initial conception, format, and expected audience of your project.

Conclusion: Glitches, Ephemera, Experiments

There’s wisdom in the old joke that discovery is a byproduct of waste, not vice versa. If we had to tabulate the hours spent working on “Knowing Together,” the resulting figure would inevitably conceal all of the time spent asking questions we never managed to answer, imagining projects we never managed to build, and talking through philosophical complexities that still remain mysterious to both of us. To recognize this incalculable volume of apparently wasted time reveals how productive it had been to honor our initial sources of curiosity and enduring sources of confusion, because this helped us to identify potent sites of artistic inquiry.

One of the most important lessons from this collaboration was to notice the difference between being experimental in your own field, and finding the opportunity to be experimental in a general sense, feeling unconfined by your background, expertise, status, or the roles typically assigned within your domain and profession. This experimental freedom is what we wanted our workshop collaborators to feel as they approached a complex, unfamiliar technique by questioning and inventing while they learned. In promoting future collaborations between scholars, artists, and cultural institutions, one of the most important lessons we want to share is this: insofar as it’s possible to dispense with people’s expected roles in a project and approach it with the basic curiosity that this opportunity brings to the surface for everyone involved, rare and unexpected new directions may emerge when this admittedly precarious approach yields its most striking rewards.

When Jasper, in particular, was led to reflect on the nature of ephemera, not just as a matter of capturing images but also of documenting individual and collective experience, this breaking down of roles was especially rewarding. The goal of our workshop was not so much to teach the practical, accepted use of an established technique (photogrammetry), but rather to create an environment free from those expectations where the technology, method, presentation, and its broader meaning would be radically open and could be confronted as whatever we make of it. Our collaborators have been more than just workshop facilitators and participants, but also agents, witnesses, and ultimately also creators of a shared experience captured in 3D scans and experimental sculptures. This project decenters the solitary observer, displaying a series of shared moments, collectively captured in images that aggregate a multiplicity of perspectives and experiences. The apparent glitches in the resulting sculptures are a testament to the messy, intensely human process of gathering an especially precarious dataset that is all the more revealing as a result.

Acknowledgments

The authors were funded for this project by the EdLab at Teachers College, Columbia University, and worked with EdLab staff including Zoe Logan, Angela Perrone, Ruta Kruliauskaite, Carlie Yutong Zhang, Trisha Barton, Jacqueline Heltz, and Katherine Hoskins. The authors also received support from the Brown Institute for Media Innovation while creating and exhibiting this project. The 35 workshop participants were: Loren Abdulezer, Ismeal Barry, Sam Bagdan, Yen Bui, Skyler Cheng, Tina Cheng, Glenn Contave, Atiya Dorsey, Jessica Durkin, Qiyao Fan, Joshua Friedman, Nieves Gruneiro, Brian Hart, Claire Jervert, Yuran Jia, Sangsuk Sylvia Kang, Autumn Kietponglert, Eunji Lee, June Lei, Jarret Leong, Qianlin Li, Lilian Yi-Hsuan Lin, Yijia Liu, Ziyu Liu, Jasper Lo, Kristin Lucas, Qun Ma, Yue Mu, Clara Santamaria, Mayu Yabe-Barry, Weidong Yang, Shuangting Yao, Xinhe Zhou, Lizhen Zhu and Fang Zou. Special thanks to Lior Zalmanson and LaGuardia Studio.

Full references and figures are in the linked PDF.