Isea 2022 - Possibles



June 15th

Short paper / Umwelten: Shifting agencies among the human, the non-human, and a machine learning algorithm

Authors: Puneet Jain

Work: A text writing platform/tool is developed where the font size of the letters, spacing between the letters, and color of the text dynamically changes based on the shifting of agencies between a human, a non-human, and a machine learning algorithm. ● Abstract (maximum of 300 words). This work is


Theme: non-human, senses, machine learning, umwelt, perception

Venue: CCCB

More information:

  • Abstract: This work is a research-creation project that invites a human, a non-human (a touch-me-not plant), and a machine-learning (ML) algorithm to question and re-formulate the relationship among themselves using the concept of ‘umwelt’ given by the German biologist Jacob Johann Von Uexküll. Through this project, the human and the non-human are brought in the same physical space (with different umwelts) to interact with one another and an ML algorithm, each having an agency to change others’ environment and hence, their umwelts. While the human changes the typography on the screen (using a text writing platform developed as a part of this project) through the typing speed and their choice of letters, both the plant and the (computer vision) machine learning algorithm alter this typography dynamically depending on the images clicked by a camera pointing to the plant. Such interactions manifested on the screen encapsulate a space where a non-linear and confusing typography symbolizes the overlapping, collision, and entanglement of the umwelts of these organisms – depicting the shifting of agencies among the human, the non-human and the machine learning algorithm over time.

  • Biography: Puneet’s work investigates new forms of communication and interventions between humans and the non-humans in a digital space. He follows both research based and an artistic approach for the creation of interfaces to open a dialogue between the human bodies and the non-humans (a machine, a plant, or sometimes even a slime mould) in the digital – questioning how sensory perceptions are morphed, enhanced, and augmented in such an entangled digital space. Puneet is currently pursuing his PhD at Concordia University, Montreal, Canada where his research lies at the intersection of human-computer interaction, STS, and disability studies. Prior to this, he worked as a research-fellow at I3D Lab, Indian Institute of Science (IISc), Bangalore where he built assistive technologies for people with sensori-motor (dis)abilities. Puneet has a masters (by research) in Computation and Data Sciences from IISc and has also worked as a Data Scientist at SAP Labs, India from 2016-2018.


  • CCCB

Montalegre, 5 - 08001 Barcelona

⟵ Return to 'Paper'

Sign up to make sure you never miss a deadline and have direct access to materials related to the ISEA2022 Conference.

Keep me updated

UOC CCCB Centre d'art Santa Mònica logo new art foundation Hac Te (Hub d'Art, Ciència i Tecnologia) Logo Macba Ajuntament de Barcelona Barcelona Capital Cultural i Científica logo institut ramon llull

Go to top