Isea 2022 - Possibles



June 13th

Full paper / AI Creativity: AI.R Taletorium, New Storytelling Realm

Authors: Predrag K. Nikolic, Ruiyang Liu, Giacomo Bertin


AI.R Taletorium is a collaborative storytelling system that consists of a neural story generator and a doodler-based fairytale visualizer. We design a character-centric bidirectional connection between the story generator and the visualizer, allowing kids to creatively influence the story generation by drawing objects they want to add to the storyline.


Theme: Artificial Intelligence Creativity, AI Automated Storytelling, Human-AI Interaction, Computer Vision, AI imagination

Venue: CCCB

More information:

  • Abstract: While stories provide an effective and concise way to share experiences, visual content acts as a more comprehensive and universal communication tool that transcends barriers between user groups with different cultural backgrounds and physical preferences. However, the meaningful learning process for coherent story visualization is a challenging task in AI studies. This paper proposes AI.R Taletorium, an intelligent artistic fairytale visualizer that interacts with users in real-time. Furthermore, language and mental development are key aspects of early childhood education.

    Our system is collaborative and enables common creative experiences between groups of children at remote locations. We aimed at offering an entertaining and imaginative platform for children to communicate with each other, improving their psychological health, especially under extreme circumstances such as the COVID- 19 pandemic. The system comprises two novel modules, a character-centric storyline generator and a visualizer in the form of doodlers. We connect the modules with characters and build a bidirectional link to allow kids to participate in the fairy tale creation process by simple adding their drawings.

    Our key contribution is as follows:

    – A novel multimodal fairytale co-creation interface based on interactive text-to-image transfer and vice-visa.

    – The system presents a novel fusion method between a learning-based language model and rule-based graph update, allowing for more flexibility in story generation and visualization with limited data.

    – Proposing a CLIP transformed story graph as an intermediate representation to transcend the barrier between digital storytelling content and insufficient illustration data (automatic AI fairy tales storytelling case).

    We fuse the rule-based language model with a learning-based generation model into a unified visualization system to allow for more flexibility during generation. AI.R Taletorium can visualize complex fairy tale scenes with stylized characters and rare relationships with our fused framework.

  • Biography: Dr Predrag K. Nikolić is an Associate Professor at the University of The Bahamas, Faculty of Liberal and Fine Arts in Nassau. He is an interactive media designer, media artist and researcher who holds a PhD in Digital Media and an MBA. He focuses on singularity, posthumanism, AI juxtaposition, and emancipation in his creative practice. His project Syntropic Counterpoints ( had numerous exposures worldwide together with other artworks such as Botorikko, Robosophy Philosophy, MindCatcher, InnerBody, Ciklosol, Before & Beyond, Vrroom, Digital Lolipop, In_Visible Island.
    Dr Predrag K Nikolic exhibited and presented his work at Ars Electronica, SIGGRAPH, SIGGRAPH Asia, ACM SIGGRAPH Sparks, Technarte USA, Technarte Spain, Singapore Science Center, Hong Kong – Shenzhen Design Biennial, Maison Shanghai, National Museum of Applied Arts in Belgrade, National Museum of Education in Belgrade.
    Professor Nikolić is Editor-In-Chief for EAI Endorsed Transactions on Creative Technologies Journal and European Alliance for Innovation (EAI) Fellow and founder of AI.R Lab.

    Ruiyang Liu is a PhD student at ShanghaiTech University working on computer vision, She worked with Professor Predrag K. Nikolić on artificial intelligence for digital art and design topics. In the five years of PhD study, she has published on several CV top conferences including CVPR, ECCV etc. Besides research, she also worked for top internet companies including Microsoft and Google as software engineering.

    Giacomo Bertin is a Physic of Data student at the University of Padova who holds a Bachelor’s degree in Physics with a thesis in computational methods for drug discovery.
    He is a machine learning engineer and artificial intelligence developer who is part of the Professor Nikolic’s Syntropic Counterpoints project and co-founder of AI.R Lab, focused on computer vision and natural language processing.


  • CCCB

Montalegre, 5 - 08001 Barcelona

⟵ Return to 'Paper'

Sign up to make sure you never miss a deadline and have direct access to materials related to the ISEA2022 Conference.

Keep me updated

UOC CCCB Centre d'art Santa Mònica logo new art foundation Hac Te (Hub d'Art, Ciència i Tecnologia) Logo Macba Ajuntament de Barcelona Barcelona Capital Cultural i Científica logo institut ramon llull

Go to top