Research Project

Sand Playground

Human-AI Collaboration for creative practices

Keywords

Creativity Support, Embodied Cognition, Tangible User Interfaces, Co-creativity

Publication

Sand Playground: Designing Human-AI physical Interface for
Co-creation in Motion ACM C&C22: PDF DOI

Team

Dina El-Zanfaly, Yiwei Huang, Yanwen Dong

Abstract

We created Sand Playground as a critical response to what is currently happening in the growing practice of leveraging generative Artificial Intelligence (AI) in creative activities such as design, music, and architecture. Several creativity support tools that enable human-AI co-creation in drawing activities were built for two-dimensional digital screens and surfaces. Sand Playground is an AI-mediated interface that consists of a sand bed and a metal ball that co-creates on sand depending on what the user draws using their finger. I wanted to investigate the role of embodiment in a human-AI co-creation process, specifically as a conversation in a physical medium such as sand. We first investigated the interactions needed between users and physical AI-mediated interfaces for co-creation. We also conducted a workshop study using Sand Playground to evaluate collaboration and the characteristics of the tangible interface. We identified findings responding to drawing strategies, multisensorial experience, and performative and ephemeral quality of creation. We published the work [8] at the ACM Creativity and Cognition Conference in 2022 (C&C 2022).

Second, through user studies, we investigated what mental models participants form when interacting with an embodied AI-mediated interface, and how these interfaces could enable people to understand how an AI agent works.

In Artistic Mimicry, the user leads the drawing sessions by directly drawing on the sand with one finger, and the interface responds synchronously by actuating the metal ball to be in the centrosymmetric position.

In Zen-garden, the user leads the drawing by placing a rock, or a group of rocks, on the sand. The interface recognizes the location and contour shapes and responds by actuating the metal ball to draw a series of concentric contours around the rocks.