AIR: Glimpses of the kick off meetings with the selected artists
S+T+ARTS AIR is taking the concept of air and addressing two themes: resilient interspecies urban ecosystems, and human-AI ecosystems. During late 2023, we selected 10 artists forresidencies, who will develop their projects and collaborate with five main HUBS until September 2024.
All artists have been selected based on the Jury presentation, but in these first crucial interactions, there is the opportunity to explore and better understand the specific challenges and needs that are addressed. We organised kick off meetings,to make explicit what the core idea of the project is and what ambitions there are to which the team commits to achieve jointly, looking at scientific / technological, artistic, and societal / outreach objectives.
They serves as a benchmark, but also an analysis of what expertise is present or needs to be added, what new perspectives are explored in the project that would contribute to the overarching goals of AIR and fostering these multi-disciplinary collaborations, and to assess what would be feasible and in scope to manage expectations. Hence, creating a ground for fruitful collaboration and also throughout the project understanding the experimental nature.
Some interesting directions that come out of these projects: How digital acoustics (where recording and AI meet) can improve understanding of cities by seeking patterns/relations between animal sounds, city sounds and human sounds and add this to e.g. digital twins of cities or ambisonics experiences. How do mobility solutions ‘extend’ and ‘influence, limit, change’ the bodily expressions of everyday? How could we be capable of identifying and understanding multiple layers of significance of gestures? And would that enable the predicting of these next movements as a result of captured gestures?
The core team is the dedicated group from the AIR consortium and the artists that are linked to the specific project. We are proud that in AIR, mentoring and collaboration are a shared effort among the various partners. This means that each project has at least one scientific mentor (HUB), one art-driven innovation mentor, and one artistic mentor. This also leads to cross-pollination between the partners and possibilities to identify common needs and research challenges the artists have. Next to that, a monthly online “artist -café” is organized to stimulate serendipity and knowledge sharing.
+ Rome
During the kick-off meeting at Sony CSL – Rome, the resident artists had the opportunity to discuss their project ideas with the researchers. In particular, Filippo Gregoretti and Michail Ribakov, who refer to Sony CSL – Rome as the main hub, further outlined which research direction of Sony CSL they wished to collaborate with. While Gregoretti found the insights offered by Sony CSL – Rome Infosphere group to be valid for his work, Ribakov focused more on the collaboration with the Sustainable Cities research line.
However, the debate between artists Nathan Sinigaglia and Uncharted Limbo Collective, who refer to Sony CSL – Rome as a secondary hub, was also intense. During the kick-off, the expertise residing in the Augmented Creativity research line at Sony CSL – Rome was particularly helpful in guiding the two artistic entities from a technical standpoint.
Now that the meetings are continuing well beyond the kick-off, three major concepts are beginning to take shape as guidelines for collaborations between art and science: Visualizing the characteristics of information systems and the ability to interact with their flow, visualizing particular urban phenomena, and exploring human movement. The next steps will involve finding an artistic and expressive vocabulary for these three major guidelines.
+ Barcelona
Artists María Arnal, Filippo Nassetti, Richard Vijgen and scientists of the S.T.ARTS AIR Barcelona Hub got together at an online kick off meeting on December 18th. The idea was to get to know each other better and start building connections between the projects of the artists. We were excited to find out that there are many common interests within everyone in the group, way beyond what was on paper on the initial proposals. We also spent some time organizing the next steps and planning the first in person meetings.as the programming environment.
+ Stuttgart
First two residencies also started in HLRS HUB where Artists Hiền Hoàng and the Uncharted Limbo Collective work with HLRS visualization scientists on projects focusing on human-AI interaction, and ecology in cities.
An experiment in human-AI collaboration
At HLRS, Uncharted Limbo will conduct experiments with dancers and movement artists. An algorithm they are developing will generate an audiovisual environment in HLRS’s CAVE visualization facility to prompt the dancers’ motions. At the same time, they will use sensors to capture data characterizing the dancers’ movements and interior states such as heart rate and neural oscillations. These data will then be analyzed using deep neural networks, and the results will be fed back into the algorithm to train it to recognize and respond to patterns in the dancers’ movements. During the development of this method, Uncharted Limbo also plans to work with S+T+ARTS AIR partner Sony Computer Science Laboratories, who conduct research on augmented creativity. The team expects that by gathering and analyzing many hours of data, the algorithm should learn to anticipate, react to, and encourage human movements and emotions in an increasingly confident and deliberate way.
At the conclusion of the project, the team will stage a performance involving equal cooperation between one or more dancers and the resulting digital being — humans will react to audiovisual impulses while the algorithm responds to their movements, creating a performative dialogue between humans and a machine. In addition to testing new interactive technologies, Uncharted Limbo will use the project as a way to engage the public in thinking about issues related to artificial agency, digital presence, data ownership, and the omnipresence of digital technology. The results will also be available to be exhibited elsewhere in the future.
“S+T+ARTS Air offers us a unique opportunity to develop a project we’ve been discussing for a while — an urge to address the important questions of agency and emergent sentience of digital entities,” commented Eleana Polychronaki on behalf of Uncharted Limbo. “As artists with significant experience in coding and art-directing real-time visuals for dance performances and interactive installations, we found ourselves wondering how we can break the norm of the typical ‘accompanying visuals’ and ‘one-way reactivity’ seen in most New Media performances. We hope that by collaborating with the HLRS team and taking advantage of their expertise in innovative visualization and access to supercomputing power we will be able to contribute to the immense field of research that explores human-machine interaction and the ever-evolving dynamics of that relationship.”
The inner lives of city trees
Hoàng has already begun working with scientists at the TU Harburg to begin studying the physiology of trees and their biomechanics, including performing 3D and CT scans of trees to gain a dataset that will be the basis for the project. Working with HLRS and the Association for Culture and Education PiNA in Slovenia, she will develop an immersive multimedia installation that will combine such scanning data with other technologies such as data sonification and visualization in virtual reality.
Once completed, Hoàng intends Garden of Entanglement to offer viewers an emotionally resonant, immersive experience of moving through the inside of a tree as it is buffeted by the urban environment surrounding it. She suggests that this should promote environmental consciousness by providing a more holistic understanding of the relationship between urban development and the natural world.
“In the past, conversations with scientists have helped me to understand the biology of trees, but working as a fellow at HLRS is something special because it offers the chance to collaborate directly with experts in visualization and high-performance computing in realizing a project together,” Hoàng said during her first visit to the center. “I am looking forward to seeing how this develops and how it might help to develop my future artistic practice.”
+ Slovenia
Filippo Gregoretti during his residency delved into the intricate realms of human-AI interaction and emotional exploration. His project, “Samasana: Togetherness,” explores the multifaceted relationship between humans and artificial beings, with a focus on emotional and unconscious layers. Utilizing his diverse expertise as a visual artist, musician, performer, and engineer, Gregoretti aims to foster coexistence with benign artificial beings and create a co-creative emotional connection with algorithms. Through immersive spatialized sound environments and real-time algorithm-controlled experiences, Gregoretti aims to inspire an emotional connection with the algorithm, fostering profound awareness of our expression in the infosphere. At his residency in HEKA lab, he was working on the workflow of his project and agreed to use of Cycling74’s Max as the programming environment.
Jonathan Reus embarked on an innovative exploration at HEKA lab, delving into the intricacies of sound synthesis and spatial audio composition as part of his Voice Complexity project. During his on-site visit, Jonathan Reus conducted recording sessions with Slovenian sound artist Irena Tomažin at the anechoic chamber of Innorenew, an affiliate research facility of the S+T+ARTS AIR project. Additionally, he implemented research on applying artificial intelligence to develop a unique neural network, trained through unconventional voice usage. Central to Reus’s endeavors is the development of a versatile “glue” software, designed to revolutionize sound synthesis systems’ communication across various spatial audio platforms. This software introduces temporal gestures within 3D sound systems, enhancing the decoding and projection of sound in multi-dimensional arrays. Through intricate manipulation of real-time numeric inputs, it offers expanded creative freedom in sound design and introduces a novel approach to spatial audio composition.
Find out more soon.
