Springe direkt zu Inhalt

Girls' Day 2023 Event Report

Understanding how chatbots work by developing dialogues

Girl's Day 2023

Girl's Day 2023
Image Credit: Julia Ullrich

On April 27, 2023, the Girls' Day took place as an in-person event for the first time since the pandemic. The HCC Research Group adapted last year's workshop concept, which we had to deploy in a nonverbal virtual format (you can read our event report here), as an in-person, hands-on format. 

The Girls' Day is a nationwide project for career and study orientation for girls. On this annual day of action, schoolgirls learn about professions or fields of study in which the proportion of women is less than 40 percent. Companies, businesses, and universities throughout Germany open their doors to girls on Girls' Day to introduce them to careers and courses of study in IT, crafts, science, and technology.

Prototype development

Prototype development
Image Credit: Julia Ullrich

The research group Human-Centered Computing offered the workshop "Hey Siri, how do I cook my favorite dish?" during which the participants looked behind the scenes of virtual assistants. After a short introduction to the history and technology behind virtual assistants, we talked about the girls' experiences with chatbots and virtual assistants. Most were familiar with voice assistants like Siri and Alexa. They shared their concerns that such devices actively listen to their surroundings and could be activated accidentally, even when the users didn't mean to. Although they had not used ChatGPT themselves, they had many stories to tell about relatives using it in various settings, for example, for generating drafts for emails.

Prototype development

Prototype development
Image Credit: Julia Ullrich

Following this phase of familiarization with the topic, we explored the questions of how chatbots work, what user data is important, and what requirements must be met to be able to train and program a chatbot in a hands-on interactive setting. We chose a simplified "Wizard-of-Oz" approach to interactively allow the participants to take a playful approach to developing simple dialogs for a recipe chatbot. For this, we divided the group into two: one group took on the role of the chatbot, and the other the role of the user. Each group tried to anticipate what the other actor could bring into the conversation. The chatbot group wrote down potential answers to hypothetical questions that the user could ask; the user group wrote down potential questions that they would expect a recipe chatbot to be able to answer. This initial phase of "training" the chatbot ended with a first test run: the user group would select an input card from their stack of cards which the chatbot group tried to respond to meaningfully by selecting a card from their stack that they believed could fit the user input. When the chatbot group realized, in this first round, that they had not anticipated the user's input that they were now confronted with, they reacted creatively: The users had asked for a recipe for vegetarian lasagna. The chatbot group only had a card with "meat" and with "vegetables" and decided to use these two cards in combination as the ingredient "meat-alternative," creating a recipe that includes "vegetable meat."

Evaluation

Evaluation
Image Credit: Julia Ullrich

Chatbot

Chatbot
Image Credit: Katrin Glinka

The chatbot group devised this strategy to create individual cards with information blocks that could be combined without any instruction. They expanded this strategy in the second round of "training" the chatbot. This gave them much more flexibility to react to the users asking for many different recipes. The chatbot group had not written an answer card with "noodles" but could present the users with a recipe consisting of 500 g four, eggs, and water. The girls in the user group that had prepared the input cards were actively and playfully trying to find the limits of the chatbot's knowledge. Both groups learned that planning for all possible human interactions is very difficult. The chatbot does not only need to know about cooking but also needs very good social skills to be able to communicate satisfactorily with the user.

Although the girls had not been familiar with the term "Computer Science" or the scope of this field before the workshop, they were very interested in learning what it entails and what they would learn in studying it. Their eagerness translated into very active participation: they were motivated to expand the chatbot dialogue and find humorous answers. After the workshop, the girls highlighted that they liked the hands-on method and were proud that they had managed to create a chatbot prototype on such short notice. They also mentioned that they enjoyed learning about the history of chatbots and that the workshop helped them understand why chatbots may not know everything.

The workshop was a fun and motivating experience for us. We particularly enjoyed communicating more directly with the girls in the in-person workshop, which also gave us much more opportunity to understand the girls' perspectives on our field of research and study.

Event report: Katrin Glinka, Ulrike Schäfer, Lars Sipos.