Perturbations in virtual reality (VR) lead to sensorimotor adaptation during exposure, but also to aftereffects once the perturbation is no longer present. An experiment was conducted to investigate the impact of different task instructions and body representation on the magnitude and the persistence of these aftereffects. Participants completed the paradigm of sensorimotor adaptation in VR. They were assigned to one of three groups: control group, misinformation group or arrow group. The misinformation group and the arrow group were each compared to the control group to examine the effects of instruction and body representation. The misinformation group was given the incorrect instruction that in addition to the perturbation, a random error component was also built into the movement. The arrow group was presented a virtual arrow instead of a virtual hand. It was hypothesised that both would lead to a lower magnitude and persistence of the aftereffect because the object identity between hand and virtual representation would be reduced, and errors would be more strongly attributed to external causes. Misinformation led to lower persistence, while the arrow group showed no significant differences compared to the control group. The results suggest that information about the accuracy of the VR system can influence the aftereffects, which should be considered when developing VR instructions. No effects of body representation were found. One possible explanation is that the manipulated difference between abstract and realistic body representation was too small in terms of object identity.

The secondary use of routinely collected patient data made possible by the broad consent form is seen as a prerequisite for developing data-driven health technologies. In Germany, relevant stakeholder groups (e.g., ethics committees and data protection authorities) specified the broad consent form; however, only one group of patient representatives was consulted, which may indicate asymmetries in engagement. This situation informed our research on medical data donation and emphasized foregrounding patient values. Drawing on participatory design, value sensitive design, and emerging research on value-led participation, we propose a method consisting of (1) a workshop concept for participatory value elicitation composed of four carefully coordinated phases and (2) an analysis procedure to examine the empirical data collected. This analysis allowed us to derive design requirements for medical data donation user interfaces. We conducted three workshops with patient advocates of vulnerable groups and patients in residential care of a psychosomatic unit. Our findings provide new directions to improve user interfaces for medical data donation: First, user interfaces need to enhance patients’ reflective thinking about the potential consequences of their data donation; second, a decision facilitator supporting patients’ value-based decision-making (e.g., by providing simple language or tailoring descriptions to patient needs); and finally, a data intermediary relieving patients’ decision-making and giving them control over their data after donation. Moreover, we emphasize the need to increase the use of participatory approaches in health technology development.

Ethics education in data science aims to teach aspiring data scientists a critical reflective data science practice. However, university courses must bridge the gap between theoretic knowledge of ethics and ethical practice. Towards this, our research aims to understand how we can promote a critical reflective practice through reflection. We, therefore, investigate how data science students start reflecting and what constitutes reflection-triggering contexts in data science education. For this, we introduce a reflective essay assignment and propose a reflection-sensitive inductive content analysis to analyze it. Our findings based on seven student reflective essays suggest that important reflection trigger characteristics in data science ethics education include students’ expectations, a new insight, motivators for reflection related to expectations, teaching formats, and emotions. Our reflection-sensitive analysis is suitable for explorative analysis and creates transparency about existing sensitizing concepts.

Data collections, such as those from citizen science projects, can provide valuable scientific insights or help the public to make decisions based on real demand. At the same time, the collected data might cause privacy risks for their volunteers, for example, by revealing sensitive information. Similar but less apparent trade-offs exist for data collected while using social media or other internet-based services. One approach to addressing these privacy risks might be to anonymize the data, for example, by using Differential Privacy (DP). DP allows for tuning and, consequently, communicating the trade-off between the data contributors' privacy and the resulting data utility for insights. However, there is little research that explores how to communicate the existing trade-off to users. % We contribute to closing this research gap by designing interactive elements and visualizations that specifically support people's understanding of this privacy-utility trade-off. We evaluated our user interfaces in a user study (N=378). Our results show that a combination of graphical risk visualization and interactive risk exploration best supports the informed decision, \ie the privacy decision is consistent with users' privacy concerns. Additionally, we found that personal attributes, such as numeracy, and the need for cognition, significantly influence the decision behavior and the privacy usability of privacy decision interfaces. In our recommendations, we encourage data collectors, such as citizen science project coordinators, to communicate existing privacy risks to their volunteers since such communication does not impact donation rates. %Understanding such privacy risks can also be part of typical training efforts in citizen science projects. %DP allows volunteers to balance their privacy concerns with their wish to contribute to the project. From a design perspective, we emphasize the complexity of the decision situation and the resulting need to design with usability for all population groups in mind. % We hope that our study will inspire further research from the human-computer interaction community that will unlock the full potential of DP for a broad audience and ultimately contribute to a societal understanding of acceptable privacy losses in specific data contexts.

Citizen science (CS) projects are conducted with interested volunteers and have already shown promise for large-scale scientific research. However, CS tends to cultivate the sharing of large amounts of data. Towards this, our research aims to understand better citizens‘ potential privacy concerns in such participation formats. We, therefore, investigate how meaningful public participation can be facilitated to foreground citizens‘ values regarding mobility data donation in CS. In this regard, we developed a two-step method: (1) a workshop concept for participatory value elicitation and (2) an analysis procedure to examine the empirical data collected systematically. Our findings based on three workshops provide new directions for improving data donation practices in CS.

Legal measures such as the GDPR aim to regulate the collection and use of personal data for scientific or commercial purposes. However, these measures might not be enough to protect individual privacy. Moreover, it is rarely possible for individuals to participate in and contribute to regulatory strategies. Informed by this situation, we were challenged on how responsible data collection can be achieved considering individuals‘ values and needs. Based on our ongoing research in healthcare and urban mobility, we developed a two-step method: first, a workshop concept for participatory values elicitation, and second, an analysis procedure to examine the empirical data collected systematically. Our findings from the workshops show how values can inform sociotechnical designs.

Consent interfaces are habitually designed to coerce people into sharing the maximum amount of data, rather than making decisions that align with their intentions and privacy attitudes, by leveraging cognitive biases to nudge users toward certain decision outcomes through interface design. Reflection and transparency have been proposed as two design dimensions of a choice architecture constituting a responsible nudge approach capable of counteracting these mechanisms by prompting reflected choice. In a crowdsourced experiment, we evaluate these capabilities of a proposed data-disclosure consent interface design deploying the responsible nudge approach within a realistic setting by exploiting a status quo bias during the sign-up of an online survey platform as a secondary task within a crowdsourcing context. Our results provide insights into a responsible design of consent interfaces, suggesting that prompting reflection significantly decreases the discrepancy between users’ privacy attitudes and decision outcomes. Meanwhile, making the presence of a nudge transparent had no significant effect on its influence. Furthermore, identifying individuals’ attitudes as a significant predictor of privacy behavior provides a promising direction for future research.

Just as other disciplines, the humanities explore how computational research approaches and tools can meaningfully contribute to scholarly knowledge production. Building on related work from the areas of CSCW and HCI, we approach the design of computational tools through the analytical lens of 'human-AI collaboration.' Such work investigates how human competencies and computational capabilities can be effectively and meaningfully combined. However, there is no generalizable concept of what constitutes 'meaningful' human-AI collaboration. In terms of genuinely human competencies, we consider criticality and reflection as guiding principles of scholarly knowledge production and as deeply embedded in the methodologies and practices of the humanities. Although (designing for) reflection is a recurring topic in CSCW and HCI discourses, it has not been centered in work on human-AI collaboration. We posit that integrating both concepts is a viable approach to supporting 'meaningful' human-AI collaboration in the humanities and other qualitative, interpretivist, and hermeneutic research areas. Our research, thus, is guided by the question of how critical reflection can be enabled in human-AI collaboration. We address this question with a use case that centers on computer vision (CV) tools for art historical image retrieval. Specifically, we conducted a qualitative interview study with art historians to explore a) what potentials and affordances art historians ascribe to human-AI collaboration and CV in particular, and b) in what ways art historians conceptualize critical reflection in the context of human-AI collaboration. We extended the interviews with a think-aloud software exploration. We observed and recorded participants' interaction with a ready-to-use CV tool in a possible research scenario. We found that critical reflection, indeed, constitutes a core prerequisite for 'meaningful' human-AI collaboration in humanities research contexts. However, we observed that critical reflection was not fully realized during interaction with the CV tool. We interpret this divergence as supporting our hypothesis that computational tools need to be intentionally designed in such a way that they actively scaffold and support critical reflection during interaction. Based on our findings, we suggest four empirically grounded design implications for 'critical-reflective human-AI collaboration': supporting reflection on the basis of transparency, foregrounding epistemic presumptions, emphasizing the situatedness of data, and strengthening interpretability through contextualized explanations.

Educating students from diverse disciplinary backgrounds is challenging. In this article, we report on our interdisciplinary course coding interaction and design (Coding IxD), which is designed for computer science and design students alike. This course has been developed over several years by consciously deliberating on existing hurdles within the educational concept. First, we motivate the need for Coding IxD and introduce the teaching principles that helped shape the course's general structure. Our teaching principles materialize in four method-based phases derived from research through design. Each phase consists of several methods that emerged to be suitable in an interdisciplinary context. Then, based on two selected student projects, we exemplify how interdisciplinary teams can arrive at novel interactive prototypes. We conclude by reflecting on our teaching practice as essential for a meaningful learning experience.

Clinical patient data is a valuable resource for data-driven medical research. However, discussions around personal data privacy highlight the urgency of designing user interfaces that communicate the possibilities and limitations of the data security used when sharing personal health data. To better understand patients’ values regarding medical data sharing, we developed a methodical approach for value-centered participatory workshops. This approach is inspired by two strains, value-sensitive design and reflective design, to reveal values related to a data donation process in the medical field. The data collected in the workshop (the first of three) will be used to derive design recommendations to improve data donation processes.

Participatory Design means recognizing that those who will be affected by a future technology should have an active say in its creation. Yet, despite continuous interest in involving people as future users and consumers into designing novel and innovative future technology, participatory approaches in technology design remain relatively underdeveloped in the German HCI community. This article brings together the diversity of voices, domains, perspectives, approaches, and methods that collectively shape Participatory Design in Germany. In the following, we (1) outline our understanding of participatory practice and how it is different from mere user involvement; (2) reflect current issues of participatory and fair technology design within the German Participatory Design community; and (3) discuss tensions relevant to the field, that we expect to arise in the future, and which we derived from our 2021 workshop through a speculative method. We contribute an introduction and an overview of current themes and a speculative outlook on future issues of Participatory Design in Germany. It is meant to inform, provoke, inspire and, ultimately, invite participation within the wider Computer Science community.

Routinely collected clinical patient data posits a valuable resource for data-driven medical innovation. Such secondary data use for medical research purposes is dependent on the patient’s consent. To gain an understanding of the patients’ values and needs regarding medical data donations, we developed a participatory workshop method, integrating approaches from value-sensitive and reflective design to explore patients’ values and translate them into hypothet- ical, ideal design solutions. The data gathered in the workshop are used to derive practicable design requirements for patient-oriented data donation technologies. In this paper, we introduce the work- shop process and evaluate its application.

Metaphorical design is a Participatory Design technique suitable for problem setting and concept development. The technique can be particularly constructive when designing (computer) systems in an already digitalized environment. In such contexts, designers might be tempted to draw on readily available technical solutions, thus hampering the discovery of new perspectives. Our use case is the development of a research data platform that aims to provide innovative functionality, especially for assessing and exploring digital resources. We developed a participatory workshop format adapting metaphorical design that first creates a shared understanding of the context and then guides participants to generate metaphors using a projective technique. We show how we used these metaphors to understand the participants’ model of the research data platform, to identify possible domains of activities, and to stimulate new viewpoints on the research data platform and its functionality. With this paper, we provide an application example of the adapted metaphorical design process, propose a metaphor evaluation matrix, and discuss the findings.

In this article, we reconsider elements of Agre’s critical technical practice approach (Agre, 1997) for critical technical practice approach for reflexive artificial intelligence (AI) research and explore ways and expansions to make it productive for an operationalization in contemporary data science. Drawing on Jörg Niewöhner’s co-laboration approach, we show how frictions within interdisciplinary work can be made productive for reflection. We then show how software development environments can be repurposed to infrastructure reflexivities and to make co-laborative engagement with AI-related technology possible and productive. We document our own co-laborative engagement with machine learning and highlight three exemplary critical technical practices that emerged out of the co-laboration: negotiating comparabilities, shifting contextual attention and challenging similarity and difference. We finally wrap up the conceptual and empirical elements and propose Reflexive Data Science (RDS) as a methodology for co-laborative engagement and infrastructured reflexivities in contemporary AI-related research. We come back to Agre’s ways of operationalizing reflexivity and introduce the building blocks of RDS: (1) organizing encounters of social contestation, (2) infrastructuring a network of anchoring devices enabling reflection, (3) negotiating timely matters of concern and (4) designing for reflection. With our research, we aim at contributing to the methodological underpinnings of epistemological and social reflection in contemporary AI research.

Every day, we have to decide multiple times, whether and how much personal data we allow to be collected. This decision is not trivial, since there are many legitimate and important purposes for data collection, for examples, the analysis of mobility data to improve urban traffic and transportation. However, often the collected data can reveal sensitive information about individuals. Recently visited locations can, for example, reveal information about political or religious views or even about an individual's health. Privacy-preserving technologies, such as differential privacy (DP), can be employed to protect the privacy of individuals and, furthermore, provide mathematically sound guarantees on the maximum privacy risk. However, they can only support informed privacy decisions, if individuals understand the provided privacy guarantees. This article proposes a novel approach for communicating privacy guarantees to support individuals in their privacy decisions when sharing data. For this, we adopt risk communication formats from the medical domain in conjunction with a model for privacy guarantees of DP to create quantitative privacy risk notifications. We conducted a crowd-sourced study with 343 participants to evaluate how well our notifications conveyed the privacy risk information and how confident participants were about their own understanding of the privacy risk. Our findings suggest that these new notifications can communicate the objective information similarly well to currently used qualitative notifications, but left individuals less confident in their understanding. We also discovered that several of our notifications and the currently used qualitative notification disadvantage individuals with low numeracy: these individuals appear overconfident compared to their actual understanding of the associated privacy risks and are, therefore, less likely to seek the needed additional information before an informed decision. The promising results allow for multiple directions in future research, for example, adding visual aids or tailoring privacy risk communication to characteristics of the individuals.

Mobility data is essential for cities and communities to identify areas for necessary improvement. Data collected by mobility providers already contains all the information necessary, but privacy of the individuals needs to be preserved. Differential privacy (DP) defines a mathematical property which guarantees that certain limits of privacy are preserved while sharing such data, but its functionality and privacy protection are difficult to explain to laypeople. In this paper, we adapt risk communication formats in conjunction with a model for the privacy risks of DP. The result are privacy notifications which explain the risk to an individual's privacy when using DP, rather than DP's functionality. We evaluate these novel privacy communication formats in a crowdsourced study. We find that they perform similarly to the best performing DP communications used currently in terms of objective understanding, but did not make our participants as confident in their understanding. We also discovered an influence, similar to the Dunning-Kruger effect, of the statistical numeracy on the effectiveness of some of our privacy communication formats and the DP communication format used currently. These results generate hypotheses in multiple directions, for example, toward the use of risk visualization to improve the understandability of our formats or toward adaptive user interfaces which tailor the risk communication to the characteristics of the reader.

Current robotic systems in surgery are telemanipulators, but the future will likely be more automated. Past and current developments literally put the robotic system at the center of the action, and force the surgical team to adapt to it. In addition to important advantages of robotic surgery, empirical studies identify serious disadvantages in sensory perception and team communication, leading to decreased situational awareness among the surgeon and the team. We therefore raise two interrelated questions: Which actors of a surgical team should be part of a controlled, semi-automated robotic assistance and how should the collaborative interaction between the actors (including the robot) be designed. Previous research has examined the situation awareness in robotic-assisted surgeries with bedside assistant, being either residents or specifically trained registered nurse first assistants, with advantages of one over the other. We built on this work by observing for the first time robotic-assisted surgeries with highly experienced bedside assistants, senior surgeons. We found that a senior surgeon in this role excelled once again, for example, through lively medical discussions and independent problem solving, and was more likely to give us clues about a thoughtful development of semi-autonomous, collaborative surgical robots. These new insights will form the basis for subsequent interviews in which surgical teams will reflect on their expectations of the robotic agency. Our overarching goal is then to translate the results into new user interface designs for robotic surgery through repeated cycles of participatory design workshops and expert evaluations.

During a research project in which we developed a machine learning (ML) driven visualization system for non-ML experts, we reflected on interpretability research in ML, computer-supported collaborative work and human-computer interaction. We found that while there are manifold technical approaches, these often focus on ML experts and are evaluated in decontextualized empirical studies. We hypothesized that participatory design research may support the understanding of stakeholders' situated sense-making in our project, yet, found guidance regarding ML interpretability inexhaustive. Building on philosophy of technology, we formulated explanation strategies as an empirical-analytical lens explicating how technical explanations mediate the contextual preferences concerning people's interpretations. In this paper, we contribute a report of our proof-of-concept use of explanation strategies to analyze a co-design workshop with non-ML experts, methodological implications for participatory design research, design implications for explanations for non-ML experts and suggest further investigation of technological mediation theories in the ML interpretability space.

The application of machine learning (ML) in the medical domain has recently received a lot of attention. However, the constantly growing need for data in such ML-based approaches raises many privacy concerns, particularly when data originate from vulnerable groups, for example, people with a rare disease. In this context, a challenging but promising approach is the design of privacy-preserving computation technologies (e.g. differential privacy). However, design guidance on how to implement such approaches in practice has been lacking. In our research, we explore these challenges in the design process by involving stakeholders from medicine, security, ML, and human-computer interaction, as well as patients themselves. We emphasize the suitability of reflective design in this context by considering the concept of privacy by design. Based on a real-world use case situated in the healthcare domain, we explore the existing privacy needs of our main stakeholders, i.e. medical researchers or physicians and patients. Stakeholder needs are illustrated within two scenarios that help us to reflect on contradictory privacy needs. This reflection process informs conceptional design rationales and our proposal for privacy-preserving explanation user interfaces. We propose that the latter support both patients’ privacy preferences for a meaningful data donation and experts’ understanding of the privacy-preserving computation technology employed.

Researchers and practitioners increasingly consider a human-centered perspective in the design of machine learning-based applications, especially in the context of Explainable Artificial Intelligence (XAI). However, clear methodological guidance in this context is still missing because each new situation seems to require a new setup, which also creates different methodological challenges. Existing case study collections in XAI inspired us; therefore, we propose a similar collection of case studies for human-centered XAI that can provide methodological guidance or inspiration for others. We want to showcase our idea in this workshop by describing three case studies from our research. These case studies are selected to highlight how apparently small differences require a different set of methods and considerations. With this workshop contribution, we would like to engage in a discussion on how such a collection of case studies can provide a methodological guidance and critical reflection.

The present study aims at evaluating how YouTube users understand, negotiate and appropriate science-related knowledge on YouTube. It is informed by the qualitative analysis of post-video discussions around visual scenarios of sea-level rise (SLR) triggered by climate change. On the one hand, the SLR maps have an exemplary status as contemporary visualizations of climate change risks, beyond traditional image categories such as scientific or popular imagery. YouTube, on the other hand, is a convenient media environment to investigate the situated appropriation of such visual knowledge, considering its increasing relevance as a navigational platform to provide, search, consume and debate science-related information. The paper draws on media practice theory and operationalizes digital methods and qualitative coding informed by Grounded Theory. It characterizes a number of communicative practices of articulated knowledge appropriation regarding climate knowledge. This includes “locating impacts,” “demanding representation,” “envisioning further,” “debating future action,” “relativizing the information,” “challenging the reality of anthropogenic climate change,” “embedding popular narratives,” “attributing to politics,” and “insulting others.” The article then discusses broader questions posed by the comments and related to the appropriation and discursive negotiation of knowledge within online video-sharing platforms. Ambiguity is identified as a major feature within the practice of science-related information retrieval and knowledge appropriation on YouTube. This consideration then serves as an opportunity to reconsider the relationship between information credibility and knowledge appropriation in the age of the digital. Findings suggest that ambiguity of information can have a positive impact on problem definition, future imagination and the discursive negotiation of climate change.

Ensuring the quality of the content provided in online settings is an important challenge today, for example, for social media or news. The Wikipedia community has ensured the high-quality standards for an online encyclopaedia from the beginning and has built a sophisticated set of automated, semi-automated, and manual quality assurance mechanisms over the last fifteen years. The scientific community has systematically studied these mechanisms but one mechanism has been overlooked --- edit filters. Edit filters are syntactic rules that assess incoming edits, file uploads or account creations. As opposed to many other quality assurance mechanisms, edit filters are effective before a new revision is stored in the online encyclopaedia. In the exploratory study presented, we describe the role of edit filters in Wikipedia's quality assurance system. We examine how edit filters work, describe how the community governs their creation and maintenance, and look into the tasks these filters take over. Our goal is to steer researchers' attention to this quality control mechanism by pointing out directions for future studies.

We present an exploratory case study describing the design and realisation of a ''pure mixed reality'' application in a museum setting, where we investigate the potential of using Microsoft's HoloLens for object-centred museum mediation. Our prototype supports non-expert visitors observing a sculpture by offering interpretation that is linked to visual properties of the museum object. The design and development of our research prototype is based on a two-stage visitor observation study and a formative study we conducted prior to the design of the application. We present a summary of our findings from these studies and explain how they have influenced our user-centred content creation and the interaction design of our prototype. We are specifically interested in investigating to what extent different constructs of initiative influence the learning and user experience. Thus, we detail three modes of activity that we realised in our prototype. Our case study is informed by research in the area of human-computer interaction, the humanities and museum practice. Accordingly, we discuss core concepts, such as gaze-based interaction, object-centred learning, presence, and modes of activity and guidance with a transdisciplinary perspective.

The global infrastructure of the Web, designed as an open and transparent system, has a significant impact on our society. However, algorithmic systems of corporate entities that neglect those principles increasingly populated the Web. Typical representatives of these algorithmic systems are recommender systems that influence our society both on a scale of global politics and during mundane shopping decisions. Recently, such recommender systems have come under critique for how they may strengthen existing or even generate new kinds of biases. To this end, designers and engineers are increasingly urged to make the functioning and purpose of recommender systems more transparent. Our research relates to the discourse of algorithm awareness, that reconsiders the role of algorithm visibility in interface design. We conducted online experiments with 105 participants using MTurk for the recommender system Recoin, a gadget for Wikidata. In these experiments, we presented users with one of a set of three different designs of Recoin's user interface, each of them exhibiting a varying degree of explainability and interactivity. Our findings include a positive correlation between comprehension of and trust in an algorithmic system in our interactive redesign. However, our results are not conclusive yet, and suggest that the measures of comprehension, fairness, accuracy and trust are not yet exhaustive for the empirical study of algorithm awareness. Our qualitative insights provide a first indication for further measures. Our study participants, for example, were less concerned with the details of understanding an algorithmic calculation than with who or what is judging the result of the algorithm.

Interpretability has become a key objective in the research, development and implementation of machine learning algorithms. However, existing notions of interpretability may not be conducive to how meaning emerges in algorithmic systems that employ ML algorithms. In this provocation, we suggest that hermeneutic analysis can be used to probe assumptions in interpretability. First, we propose three levels of interpretability that may be analyzed: formality, achievability, and linearity. Second, we discuss how the three levels have surfaced in prior work, in which we conducted an explicitation interview with a developer to understand decision-making in an algorithmic system implementation. Third, we suggest that design practice may be needed to move beyond analytic deconstruction, and showcase two design projects that exemplify possible strategies. In concluding, we suggest how the proposed approach may be taken up in future work and point to research avenues.

Using games to model, measure and increase creativity in children and adults would be a very engaging path to social impact and empirical progress. This paper proposes an approach to reframe the new and popular board game Codenames using associative creativity principles. The Remote Associates Test (RAT) is a test measuring creativity as a function of associative ability. comRAT-C is a previous computational cognitive system that can solve the RAT using associative and convergence principles. In this paper, we formalise Codenames using associative principles from comRAT-C. A way to computationally model and measure the difficulty of Codenames is proposed. We discuss whether Codenames or a future variant of it can be used in creativity research.

Cognitive science and cognitive psychology have long used creativity tests to measure and investigate the relationships between creativity, creative problem solving and other cognitive abilities. Implementing cognitive systems that can model and/or solve creativity tests can shed light on the cognitive process, and presents the possibility of building much more precise creativity measuring tools. This paper describes four cognitive AI systems related to the Remote Associates Test (RAT) and their contributions to creativity science. comRAT-C is a system that solves the RAT, correlating with human performance. comRAT-G reverse engineers this process to generate RAT queries with a high degree of parameter control. fRAT generates functional RAT queries, resurrecting a theoretical concept proposed by researchers many decades ago. The visual RAT takes advantage of the formal conceptualization necessary for computational implementation, to expand the RAT to the visual domain. All the cognitive systems and generated RAT queries have been successfully validated with human participants and have contributed in improving creativity modeling and measuring tools.

The present thesis offers an initial investigation of a previously unexplored by scientific research quality control mechanism of Wikipedia—edit filters. It is analysed how edit filters fit in the quality control system of English Wikipedia, why they were introduced, and what tasks they take over. Moreover, it is discussed why rule based systems like these seem to be still popular today, when more advanced machine learning methods are available. The findings indicate that edit filters were implemented to take care of obvious but persistent types of vandalism, disallowing these from the start so that (human) resources can be used more efficiently elsewhere (i.e. for judging less obvious cases). In addition to disallowing such vandalism, edit filters appear to be applied in ambiguous situations where an edit is disruptive but the motivation of the editor is not clear. In such cases, the filters take an “assume good faith” approach and seek via warning messages to guide the disrupting editor towards transforming their contribution to a constructive one. There are also a smaller number of filters taking care of haphazard maintenance tasks—above all tracking a certain bug or other behaviour for further investigation. Since the current work is just a first exploration into edit filters, at the end, a comprehensive list of open questions for future research is compiled.

The Remote Associates Test (RAT) is a measure of associative ability, which is often regarded as essential for creative thinking. The most commonly used version of the test is the compound RAT. However, many RAT items do not translate directly in different languages. Additionally, a linguistic measure cannot be used to measure visual associative ability. A visual measure for associative ability that is similar to the RAT would be a useful tool for cross-cultural investigations of creativity. The present study investigated the relationship between the linguistic and a newly developed visual version of RAT in Russian and Finnish native speakers (for both samples n = 67). Both linguistic and visual measures showed good internal reliabilities in both samples (Cronbach’s α = 0.73–0.84). The mean score in the visual task was slightly higher for the Finnish sample. The correlation between the two measures was stronger in the Russian sample (r = 0.56) compared to the Finnish sample (r = 0.28). These results are discussed in relation to linguistic and cultural differences between the samples.

Re-representation is a critical ability to (i) understanding human creative problem solving, and (ii) modeling computational cognitive systems able to support or perform creative problem solving tasks on their own. This paper proposes a unified multi-level cognitive approach to investigating re-representation: the study of sensory-based, concept-based and problem template based possible forms of re-representations in an integrated manner. Descriptions and explanations of each level prepare the ground for further computational modeling. A study is deployed in order to explore the relationship between the various tasks proposed to reflect re-representation. A significant correlation between the investigated tasks is discovered. Two previous studies from the literature are replicated. A new strong and significant relationship between the Pattern Meanings Test and the Alternative Uses Test is observed.

Problem-solving, creativity and spatial reasoning are high level abilities of cognitive systems with high potential for synergies. However, they have been treated separately by different fields. This special issue presents research work on these topics, aiming to observe their interrelations in order to create theoretical approaches, methodologies and computational tools to advance work on creativity and spatial problem-solving in cognitive systems.

The Remote Associates Test (RAT, CRA) is a classical creativity test used to measure creativity as a function of associative ability. The RAT has been administered in different languages. Nonetheless, because of how embedded in the language the test is, only a few items are directly translatable, and most of the time the RAT is created anew in each language. This process of manual (and in two cases computational) creation of RAT items is guided by the researchers’ understanding of the task. However, are the RAT items in different languages comparable? In this paper, different RAT stimuli datasets are analyzed qualitatively and quantitatively. Significant differences are observed between certain datasets in terms of solver performance. The potential sources of these differences are discussed, together with what this means for creativity psychometrics and computational vs. manual creation of stimuli.

The Remote Associates Test (RAT) is a classical creativity test developed by Mednick and Mednick in 1967. RAT problems and their norms so far exist only in a few languages, including English, Dutch, Japanese and Italian. In this paper, we describe our process of constructing a set of Remote Associates Test problem in Romanian. 63 native speaking Romanian participants have solved this set. The set of items shows high internal consistency. Normative data pertaining to each problem is provided, together with a description of RAT problems peculiarities in Romanian.

Human creativity is usually assessed with a variety of established creativity tests. One of this is the Remote Associates Test (RAT), which aims to measure the ability of reaching remote associates with linguistic stimuli. A well known variant of the RAT exists – the compound RAT, for which normative data and solvers have been proposed in the literature. However, a different type of RAT was proposed in 1971 by Worthen and Clark – a functional form which had the potential of measuring other types of associations. However, the few test items proposed by Worthen and Clark where lost during archive transport, and cannot be accessed. In this paper, we set to reconstruct an ample set of functional items in the spirit of Worthen and Clark’s idea, using information science techniques. Cognitive word associates are used as data. The process of a former computational solver of the RAT is repurposed to create rather than solve items. The approach of constructing queries is evaluated by getting human participants to solve both functional and compound items. In the process, a previous computational approach to solving the compound RAT is also validated in the functional RAT context.

The success of crowdsourcing systems such as Wikipedia relies on people participating in these systems. However, in this research we reveal to what extent human and machine intelligence is combined to carry out semi-automatic workflows of complex tasks. In Wikipedia, bots are used to realize such combination of human-machine intelligence. We provide an extensive overview on various edit types bots carry out in this regard through the analysis of 1,639 approved task requests. We classify existing tasks by an action-object-pair structure and reveal existing differences in their probability of occurrence depending on the investigated work context. In the context of community services, bots mainly create reports, whereas in the area of guidelines or policies bots are mostly responsible for adding templates to pages. Moreover, the analysis of existing bot tasks revealed insights that suggest general reasons, why Wikipedia’s editor community uses bots as well as approaches, how they organize machine tasks to provide a sustainable service. We conclude by discussing how these insights can prepare the foundation for further research.

To effectively support individual contributors in large scale ideation settings, we need a computational understanding of their cognitive processes. Although related work exists that references models from psychology, there are two shortcomings in existing approaches: First, they only analyse ideas on a statistical level. Second, they lack a notion of individual ideator differences. This work proposes a new user model for ideation, based on process-models from psychology and knowledge-graph based information extraction from submitted idea-texts. By building a computational model of ideation, this work aims to enable new graph-based analysis methods of cognitive ideation processes and new approaches to adaptive ideation support systems. These systems could be used to detect and counter fixation in individuals and to guide group efforts based on categories that should be explored.

Despite the significant benefits of crowd ideation platforms, these introduced two main challenges: (1) many ideas generated are basic and repetitive (2) the high number of ideas generated makes it practically impossible for ideation experts to review ideas one by one in order to select novel and useful ones. A key feature to overcome these issues resides in understanding the ideas. General Knowledge Graphs describe the meaning of domain-independent terms in an computationally understandable way and therefore represent a promising solution in obtaining such meaning. In this paper, we describe our research in understanding ideas and preliminary findings in application for crowd-powered innovation.

Interactive intelligent systems, i.e., interactive systems that employ AI technologies, are currently present in many parts of our social, public and political life. An issue reoccurring often in the development of these systems is the question regarding the level of appropriate human and computer contributions. Engineers and designers lack a way of systematically defining and delimiting possible options for designing such systems in terms of levels of automation. In this paper, we propose, apply and reflect on a method for human-computer configuration design. It supports the systematic investigation of the design space for developing an interactive intelligent system. We illustrate our method with a use case in the context of collaborative ideation. Here, we developed a tool for information extraction from idea content. A challenge was to find the right level of algorithmic support, whereby the quality of the information extraction should be as high as possible, but, at the same time, the human effort should be low. Such contradicting goals are often an issue in system development; thus, our method proposed helped us to conceptualize and explore the design space. Based on a critical reflection on our method application, we want to offer a complementary perspective to the value-centered design of interactive intelligent systems. Our overarching goal is to contribute to the design of so-called hybrid systems where humans and computers are partners.

With the surge of the application of machine learning (ML) systems in our daily life there is an increasing demand to make operation and results of these systems interpretable for people with different backgrounds (ML experts, non-technical experts etc.). A wide range of research exists, particular in ML research on specific interpretability techniques (e.g., extracting and displaying information from ML pipelines). However, often a background in machine learning or mathematics is required to interpret the results of the interpretability technique itself. Therefore there is an urgent lack of techniques which may help non-technical experts in using such systems. The grounding hypothesis of this analysis is that, especially for non-technical experts, context is an influential factor in how people make sense of complex algorithmic systems. Therefore an interaction between a user and an application assumed to be an interplay between a user and his historical context, the context of the situation in which the interaction is embedded and the algorithmic system. Interpretability techniques are the common link which bring all these different aspects together. In order to evaluate the assumption that most of the current interpretability research is tailored to a technical audience and gain an overview over existing interpretability techniques we conducted a literature mapping study studying the state of interpretability research in the field of natural language processing (NLP). The results of this analysis suggest that indeed most techniques are not evaluated in a context where a non-technical expert may use it and that even most publications lack a proper definition of interpretability. Keywords: Literature Mapping Study, Interpretability Research, Natural Language Processing.

Interpretation of machine learning results is a major challenge for non-technical experts, with visualization being a common approach to support this process. For instance, interpretation of clustering results is usually based on scatterplots that provide information about cluster characteristics implicitly through the relative location of objects. However, the locations and distances tend to be distorted because of artifacts stemming from dimensionality reduction. This makes interpretation of clusters difficult and may lead to distrust in the system. Most existing approaches that counter this drawback explain the distances in the scatterplot (e.g., error visualization) to foster the interpretability of implicit information. Instead, we suggest explicit visualization of the uncertainty related to the information needed for interpretation, specifically the uncertain membership of each object to its cluster. In our approach, we place objects on a grid, and add a continuous ''topography'' in the background, expressing the distribution of uncertainty over all clusters. We motivate our approach from a use case in which we visualize research projects, clustered by topics extracted from scientific abstracts. We hypothesize that uncertainty visualization can increase trust in the system, which we specify as an emergent property of interaction with an interpretable system. We present a first prototype and outline possible procedures for evaluating if and how the uncertainty visualization approach affects interpretability and trust.

Machine learning systems are ubiquitous in various kinds of digital applications and have a huge impact on our everyday life. But a lack of explainability and interpretability of such systems hinders meaningful participation by people, especially by those without a technical background. Interactive visual interfaces (e.g., providing means for manipulating parameters in the user interface) can help tackle this challenge. In this paper we present PreCall, an interactive visual interface for ORES, a machine learning-based web service for Wikimedia projects such as Wikipedia. While ORES can be used for a number of settings, it can be challenging to translate requirements from the application domain into formal parameter sets needed to configure the ORES models. Assisting Wikipedia editors in finding damaging edits, for example, can be realized at various stages of automatization, which might impact the precision of the applied model. Our prototype PreCall attempts to close this translation gap by interactively visualizing the relationship between major model metrics (recall, precision, false positive rate) and a parameter (the threshold between valuable and damaging edits). Furthermore, PreCall visualizes the probable results for the current model configuration to improve the human’sunderstanding of the relationship between metrics and outcome when using ORES. We describe PreCall’s components and present a use case that highlights the benefits of our approach. Finally, we pose further research questions we would like to discuss during the workshop.

Wikidata is one of the most edited knowledge bases which contains structured data. It serves as the data source for many projects in the Wikimedia sphere and beyond. Since its inception in October 2012, it has been increasingly growing in term of both its community and its content. This growth is reflected by an expanding number of research focusing on Wikidata. Our study aims to provide a general overview of the research performed on Wikidata through a systematic mapping study in order to identify the current topical coverage of existing research as well as the white spots which need further investigation. In this study, 67 peer-reviewed research from journals and conference proceedings were selected, and classified into meaningful categories. We describe this data set descriptively by showing the publication frequency, the publication venue and the origin of the authors and reveal current research focuses. These especially include aspects concerning data quality, including questions related to language coverage and data integrity. These results indicate a number of future research directions, such as, multilingualism and overcoming language gaps, the impact of plurality on the quality of Wikidata’s data, Wikidata’s potential in various disciplines, and usability of user interface.

Problem-solving, creativity and spatial reasoning are high level abilities of cognitive systems with high potential for synergies. However, they have been treated separately by different fields. This special issue presents research work on these topics, aiming to observe their interrelations in order to create theoretical approaches, methodologies and computational tools to advance work on creativity and spatial problem-solving in cognitive systems.

Project IKON aims to explore potentials for transferring knowledge generated in research projects at a major Berlin research institution, the Museum für Naturkunde (MfN, natural history museum). Knowledge transfer concerns both the exchange of knowledge among the employees (researchers as well as communicators or management staff), and with the broader public. IKON as a research project coincides with a continuous effort by the research institution to open itself up in terms of its activities and stored knowledge. To understand the specific requirements of IKON, we conducted semi-structured interviews with 5 employees of varying positions (researchers, management staff, employees) at the research institution. Our questions were conceived to understand: (1) what unites individual perspectives of knowledge transfer; (2) by whom and through which means (i.e., actors and infrastructures) is knowledge transfer carried out; (3) where relations between actors and infrastructures in the current state of knowledge transfer need support or intervention. From our results, we infer high-level implications for the design of an application that can support employees of the MfN in (1) collaborating with each other and (2) conceptualize Knowledge Transfer Activities based on semantically related research projects.

Insight problems are used in the study of human creativity problem solving to evaluate the creativity of the solver, and the process through which creativity problem solving is cognitively deployed. However, not many such problems exist, and the factors underlying their creation are not well controlled. The framework CreaCogs proposes ways in which cognitive AI systems could be used to solve diverse such problems using a small set of processes. In this paper, a previous approach for the creation of insight problems proposed in CreaCogs is implemented computationally. The initial experiments, results, limitations, perspectives and potential are reported upon.

@article{olteteanu_re-representation_2018, author = {Olteteanu, Ana-Maria and Indurkhya, Bipin}, title = {Re-representation in Cognitive Systems}, journal = {Frontiers in Psychology}, year = {2018}, url = {https://www.frontiersin.org/research-topics/7414/re-representation-in-cognitive-systems}, language = {english} }

Functional fixedness refers to a cognitive bias that prevents people from using objects in new ways and more abstractly from perceiving problems in new ways. Supporting people in overcoming functional fixedness could improve creative problem solving and capacities for creative design. A study was conducted to detect whether a relationship exists between participants' tendency to reorient objects presented as stimuli in an alternative uses test (AUT) and their creativity, also measured using the Wallach Kogan (WaKo) pattern meanings test. The AUT measures creativity as a function of identifying alternative uses for traditional objects. The WaKo pattern meanings test detects the ability to see an abstract pattern as different possible objects or scenes. Also studied is whether Kruglanski's need for closure (NFC) scale, a psychological measure, can predict the ability to incorporate reorientation cues when identifying uses. This study revealed highly significant, high correlations between reorientation and several creativity measures, and a correlation between reorientation and the predictability subscale of the NFC scale. A qualitative exploration of participants' responses reveals further metrics that may be relevant to assessing creativity in the AUT.

The Remote Associates Test (RAT) has been used to measure creativity, however few repositories or standardizations of test items exist, like the normative data on 144 items provided by Bowden and Jung-Beeman. comRAT is a computational solver which has been used to solve the compound RAT in linguistic and visual forms, showing correlation to human performance over the normative data provided by Bowden and Jung-Beeman. This paper describes using a variant of comRAT, comRAT-G, to generate and construct a repository of compound RAT items for use in the cognitive psychology and cognitive modeling community. Around 17 million compound Remote Associates Test items are created from nouns alone, aiming to provide control over (i) frequency of occurrence of query items, (ii) answer items, (iii) the probability of coming up with an answer, (iv) keeping one or more query items constant and (v) keeping the answer constant. Queries produced by comRAT-G are evaluated in a study in comparison with queries from the normative dataset of Bowden and Jung-Beeman, showing that comRAT-G queries are similar to the established query set.

Creative cognitive systems are rarely assessed with the same tools as human creativity. In this paper, an approach is proposed for building cognitive systems which can solve human creativity tests. The importance of using cognitively viable processes, cognitive knowledge acquisition and organization, and cognitively comparable evaluation when implementing creative problem-solving systems is emphasized. Two case studies of artificial cognitive systems evaluated with human creativity tests are reviewed. A general approach is put forward. The applicability of this general approach to other creativity tests and artificial cognitive systems, together with ways of performing cognitive knowledge acquisition for these systems are then explored.

A major goal of collaborative ideation is improving the creativity of the ideas generated. Recent approaches enhance creativity by showing users similar ideas during productive ideation and diverse ideas when they reach an impasse. However, related work either demands a higher mental effort from users to assess similarity or yields only a limited number of similarity values. Furthermore, idea relationship is only considered in one dimension (similarity). In our research in progress, we introduce a new approach called concept validation. It enables us to (1) capture the conceptualization of users' ideas and (2) assess multi-dimensional relationships between all ideas in near real-time. We conducted a study with 90 participants to validate the suitability of our approach. The results indicate that adding the extraction of semantic concepts to the ideation process has no negative impact on number and creativity of ideas generated. This signifies an important step towards our vision of an idea-based knowledge graph used by an interactive system to improve computer-supported human creativity.

In recent years, data publication workflows get more and more attention [1,2]. In order to obtain FAIR data [3], reviewers, data curators and other stakeholders have realized that not only the submitted data matter but also the underlying process to create that data within existing research practice. A better understanding of existing data publication practices in research workflows will help service providers such as data repositories (Pangaea [4], ENA [5], GenBank [6]) to support their users with more appropriate services and tools when submitting data, and otherwise, will sustain the role of data repositories in research practice. Such improved coordination will minimize the workload of researchers and data curators and will facilitate the review process of all stakeholders with respect to reproducibility. Furthermore, well-documented data publication workflows may improve data retrieval and finally data reuse in a long run. One obstacle towards comprehensible and properly described research workflows is the fact that data publication workflows in the life sciences are hard to define. Scholars have their very individual disciplinary background, research skills and experiences. In some domains such as biodiversity, scholars work from several weeks to years to collect and analyze often heterogeneous data from various sources, such as collections, environmental or molecular data repositories. Thus, reconstructing their work process after the project is finalized is very difficult if not impossible. However, our goal is to reveal the state of the art on how scholars manage their data in their research practices. We are in the process of setting up a survey whose general structure is organized according to the GFBio Data Lifecycle [7]. The results will allow us to reveal typical data practices workflows that can be used to evaluate the suitability of existing data repository portals, such as GFBio [8]. In our talk, we present the first insights of the survey. KEYWORDS: data publication workflows, data practices, biological and environmental data, green life sciences, biodiversity REFERENCES: 1. Dallmeier-Tiessen, S., Khodiyar, V., Murphy, F., Nurnberger, A., Raymond, L., Whyte, A., 2017. Connecting Data Publication to the Research Workflow: A Preliminary Analysis, International Journal of Digital Curation, 12, https://doi.org/10.2218/ijdc.v12i1.533. 2. González-Beltrán, A., Li, P., Zhao, J., Avila-Garcia, M. S., Roos, M., Thompson, M., van der Horst, E., Kaliyaperumal, R., Luo, R., Lee, T.-L., Lam, T., Edmunds, S.C., Sansone, S.-A., Rocca-Serra, P, 2015. From Peer-Reviewed to Peer-Reproduced in Scholarly Publishing: The Complementary Roles of Data Models and Workflows in Bioinformatics, PLOS ONE 10, 7, pp. 1–20, https://doi.org/10.1371/journal.pone.0127612. 3. Mark D. Wilkinson et al., 2016. The FAIR Guiding Principles for scientific data management and stewardship, Scientific Data 3. https://doi.org/10.1038/sdata.2016.18 4. Pangaea, https://www.pangaea.org 5. ENA, https://www.ebi.ac.uk/ena 6. GenBank, https://www.ncbi.nlm.nih.gov/genbank/ 7. GFBio Data Lifecycle, https://www.gfbio.org/training/materials/data-lifecycle 8. GFBio, https://www.gfbio.org

Recent research suggests that in visual analytics tasks, collaborative sensemaking relies on successful collaboration between humans and software agents. To advance the understanding of such collaboration, we consider that the latter possess a form of situational awareness which, when coordinated with humans, can enrich the collaborative sensemaking process. We propose a conceptual model for a coordinating agent that dynamically initiates interruptions, influenced by the analytic activities of humans. We provide possible designs for four coordinating strategies. In closing, we discuss plans for implementation, and how future studies can contribute to wider HCI and CSCW discourses.

Purpose The traditional means of pursuing research by having all the parameters and processes under one roof has given way to collaborative mechanisms of performing the same task. Collaborative work increases the quality of research and it is a big contributing factor to augment the growth of the scientific knowledge. This process leads to training of new and well-informed academicians and scientists. e-Research (Electronic Research) has gained significant amount of traction as technology serves as the backbone for undertaking collaborative research. The purpose of this paper is to provide a synoptic view of existing research surrounding e-Research and suggest a data lifecycle model that can improve the outcome of collaborative research. Design/methodology/approach Systematic literature review methodology has been employed to undertake this study. Using the outcome of the literature review and the analysis of the existing data lifecycle models, an improvised version of the data lifecycle model has been suggested. Findings This study has brought a conceptual model for data lifecycle for collaborative research. The literature review in the domain of e-Research has shown that the focus of these papers was on the following stages of data lifecycle model: concept and design, data collection, data processing, sharing and distribution of data and data analysis. Research limitations/implications In this paper, only journal papers have been considered and conference proceedings have not been included for literature review. Originality/value This paper suggests a conceptual model for the data lifecycle for collaborative research. This study can be useful for academic and research institutions to design their data lifecycle model.

In this interdisciplinary research project, I aim to combine philosophy of technology and computer science in order to develop a design methodology for algorithm awareness. The latter has become an emergent field of research due to the increasing influence of automated processes on everyday life. In this abstract, I show how current research is not yet exhaustive and that existing design theories fall short of accounting for the sociotechnical complexities that are at work in complex adaptive systems. Outlining my research apparatus, I illustrate how I will analyze specific case studies and apply results in use cases.

The evolution of semantic structured data, such as those behind the deep web or socialnetworks, requires mapping between sources to enable a high level integration. Sev-eral ontology matching systems have been developed to establish mappings betweenmultilingual ontologies, however, employing these systems in real world requires anassessment of the ontologies capability and performance which is conducted by theMultiFarm Track. Yet, this track still lacks of ontologies from different language fami-lies. In this paper, we contribute to the OAEI initiative with a Turkish dataset to extendthe coverage of languages for the matching systems.

Research-intensive organizations struggle to get an actionable overview of their research activities. We report on a preliminary architecture of a socio-technical system that aims to uncover the potential for internal transfer of knowledge and to facilitate this transfer in research-intensive organizations. We discuss two important roles in this human-powered system and its benefits in the context of a research museum.

The sensemaking process is a complex task in academic research practice. Scholars annotate on their research resources to reflect impressions, insights and facts, as well as reduce the cognitive load in their working memory as a prerequisite for carrying out such sensemaking processes. Given the data-intensive nature of research and the increased collaboration of often interdisciplinary working teams, a vast amount of annotations with a great extent of knowledge are produced daily. However, the annotation tools extant do not usually support sensemaking of those annotations. Using existing research as a foundation, we define a conceptual model for collaborative sensemaking that is based on human-machine collaboration. We introduce concepts to integrate machine intelligence into the collaborative sensemaking process by allowing users to interact with machine-recommended information in favor of exploring knowledge from collective intelligence through interactive visualizations of annotations.

Research questions in biodiversity are as diverse and heterogeneous as data are. Most metadata standards are mainly data-focused and pay little attention to the search perspective. In this work, we introduce a method to analyze the actual information need of biodiversity scholars based on two individual studies: (1) a series of workshops with domain experts and (2) an analysis of research and search questions collected in three different biodiversity projects. We finally present 12 high-level entities that appear in all kinds of biological data across the different sources evaluated.

While literature portals in the biomedical domain already enhance their search applications with ontological concepts, data portals offering biological primary data still use a classical keyword search. Similar to publications, biological primary data are described along meta information such as author, title, location and time which is stored in a separate file in XML format. Here, we introduce a semantic search for biological data based on metadata files. The search is running over 4.6 million datasets from GFBio - The German Federation for Biological Data (GFBio, https://www.gfbio.org), a national infrastructure for long-term preservation of biological data. The semantic search method used is query expansion. Instead of looking for originally entered keywords the search terms are expanded with related concepts from different biological vocabularies. Hosting our own Terminology Service with vocabularies that are tailored to the datasets, we demonstrate how ontological concepts are integrated into the search and how it improves the search result.

Enhancing creativity has been paid a lot of attention recently, especially with the emergence of online collaborative ideation. Prior work has shown that, in addition to the exposure of diverse and creative examples, visualising the solution space enables ideators to be inspired and, thus, arrive at more creative ideas. However, existing automated approaches which assess the diversity of a set of examples fail on unstructured short text due to their reliance on similarity computation. Furthermore, the conceptual divergence cannot be easily captured for such representations. This research in progress introduces an approach based on semantic annotation to overcome these issues. The solution proposed formalizes user ideas into a set of annotated concepts and a matching mechanism is then used to compute the similarity between users' ideas. We aim also to create a visualisation of the solution space based on the similarity matrix obtained by a matching process between all ideas.

Presenting a set of similar or diverse ideas during the idea generation process leads ideators to come-up with more creative and diverse ideas. However, to better assess the similarity between the ideas, we designed two matching systems, namely I-Match and OntoIdea. In the context of the idea generation process, each idea is represented by a set of instances from DBpedia describing the main concepts of the idea. Then, the developed matching systems are applied to compute the similarity between a set of instances that represent the ideas. The purpose of our participation at OAEI is to evaluate our designed instance matching algorithm in order to apply it to assess the similarity between ideas. The re- sults obtained for the first participation of I-Match and OntoIdea systems at OAEI 2017, on different instance matching tracks are so far quite promising.

A primary goal of a research infrastructure for data management should be to enable efficient data discovery and integration of heterogeneous data. The German Federation for Biological Data (GFBio) was guided by this goal. The basic component, that enables such interoperability and serves as a backbone for such a platform, is the GFBio Terminology Service (GFBio TS). This service acts as a semantic platform for accessing, developing and reasoning about terminological resources within the biological and environmental domains. A RESTful API gives access to these terminological resources in a uniform way, regardless of their degree of complexity and whether they are internally stored or externally accessed through web services. Additionally, a set of widgets with an intrinsic API connection are made available for easy integration in applications and web interfaces. Based on the requirements of GFBios partners, we describe the added value that is provided by the GFBio Terminology Service with practical scenarios as well as the challenges we still face. We conclude by describing our current activities and future developments.

The evolution of semantic structured data, such as those behind the deep web or socialnetworks, requires mapping between sources to enable a high level integration. Sev-eral ontology matching systems have been developed to establish mappings betweenmultilingual ontologies, however, employing these systems in real world requires anassessment of the ontologies capability and performance which is conducted by theMultiFarm Track. Yet, this track still lacks of ontologies from different language fami-lies. In this paper, we contribute to the OAEI initiative with a Turkish dataset to extendthe coverage of languages for the matching systems.

In many research projects, much more data are created than made publicly available. Keeping research data deliberately closed or publishing only selected subsections of the gathered data are unfortunately common practices in academia. Fortunately, such problems have been getting more and more attention in the past years. However, another issue that is still often overlooked concerns research data that are generated as part of a research project but that are generally not considered part of the primary research data. One example for such neglected research data are terminologies such as controlled vocabularies that are used to describe or classify primary research data. In this paper we will outline the process that is used by the Terminology Service of the German Federation for Biological Data (GFBio) to prepare and process terminologies so that they can be included in the GFBio Terminology Service where they are made available to researchers within and outside the original research project. We will also show how making such supplementary research data publicly available will benefit the researchers who share them as well as the scientific community as a whole.

The Semantic Web provides meaning to information resources in the form of machine-accessible structured data. Research in the Semantic Web field focuses commonly on tools and interfaces for technical experts leading to various usability problems. The complexity of Semantic Web technology makes it difficult especially for nontechnical experts to use these technologies. Existing research on Semantic Web usability considers mostly consumers of structured data that leave out the creation perspective. In this work, we focus on the usability of creating structured data from text, especially on the creation of relations between entities. We reviewed existing research and state of the art annotation tools to establish shortcomings and used our knowledge to propose an interaction design for the creation of relations. We conducted a user-study which showed that the proposed interaction design performed well, making it a contribution to enhance the overall usability in the field.

Building a shared understanding of a specific area of interest is of increasing importance in today’s information-centric world. A shared understanding of a domain can be realized by building a structured knowledge base about it collaboratively. Our research is driven by the goal to understand participation patterns over time in collaborative knowledge building efforts. Consequently, we focus our study on one representative project – Wikidata. Wikidata is a free, structured knowledge base that provides structured data to Wikipedia and other Wikimedia projects. This paper builds upon previous research, where we identified six common participation patterns, i.e. roles, in Wikidata. In the research presented here, we study the applicability of sequence analysis methods by analyzing the dynamics in users’ participation patterns. The sequence analysis is judged by its ability to answer three questions: (i) “Are there any preferable role transitions in Wikidata?”; (ii) “What are the dominant dynamic participation patterns?”; (iii) “Are users who join earlier more turbulent contributors?” Our data set includes participation patterns of about 20,000 users in each month from October 2012 to October 2014. We show that sequence analysis methods are able to infer interesting role transitions in Wikidata, find dominant dynamic participation patterns, and make statistical inferences. Finally, we also discuss the significance of these results with respect to the understanding of the participation process in Wikidata.

Motivation of free, libre and open source software developers has been widely studied over the years. The reasons people engage in this seemingly altruistic behavior have been elaborated and classified. The present work addresses a slightly different issue: what motivates individuals to participate in community network projects? Are the reasons similar to or quite distinct from these relevant to contributors to free software? Based on recently conducted interviews with community network activists from the Germany based project Freifunk and established FLOSS motivation research, we will analyse the specifics of the Freifunk project and the factors which spur its members to action. The obtained insights could then hopefully be used to understand the underlying group processes and help build sustainable communities.

Research has become more data-intensive over the last few decades. Sharing research data is often a challenge, especially for interdisciplinary collaborative projects. One primary goal of a research infrastructure for data management should be to enable efficient data discovery and integration of heterogeneous data. In order to enable such interoperability, a lot of effort has been undertaken by scientists to develop standards and characterize their domain knowledge in the form of taxonomies and formal ontologies. However, these knowledge models are often disconnected and distributed. The work presented here provides a promising approach for integrating and harmonizing terminological resources to serve as a backbone for a platform. The component developed, called the GFBio Terminology Service, acts as a semantic platform for access, development and reasoning over internally and externally maintained terminological resources within the biological and environmental domain. We highlight the utility of the Terminology Service by practical use cases of semantically enhanced components. We show how the Terminology Service enables applications to add meaning to their data by giving access to the knowledge that can be derived from the terminologies and data annotated by them.

The main goal of this work is to create a model of trust which can be considered as a reference for developing applications oriented on collaborative annotation. Such a model includes design parameters inferred from online communities operated on collaborative content. This study aims to create a static model, but it could be dynamic or more than one model depending on the context of an application. An analysis on Genius as a peer production community was done to understand user behaviors. This study characterizes user interactions based on the differentiation between Lightweight Peer Production (LWPP) and Heavyweight Peer Production (HWPP). It was found that more LWPP- interactions take place in the lower levels of this system. As the level in the role system increases, there will be more HWPP-interactions. This can be explained as LWPP-interacions are straightforward, while HWPP-interations demand more agility by the user. These provide more opportunities and therefore attract other users for further interactions.

Quality of Service (QoS) guarantees are usually regulated in a Service Level Agreement (SLA) between provider and consumer of services. Such guarantees are often violated, it may however be the case where the available services do not match exactly all required QoS, leading the system to grind to a halt. It would be better to look for an approximation for acceptable QoS and avoid the complete stopping of the running services. This paper aims at making dynamic QoS acceptability easy for service selection. The proposed model is based on an extension of existing probabilistic description logics reacting to QoS variations. The contributions made are twofold (1) a query description language to express the required QoS by means of a probabilistic description logic (2) a reasoning algorithm for decision making about the acceptability of QoS w.r.t the probabilistic description.

The reading of text resources in scholarly settings can take various forms. Each form provides scholars with different insights that complement each other. The first findings from an ongoing series of interviews on scholarly annotation practices suggest that users are aware of the various forms of reading, but they are reluctant to use automatic annotations and still rely on conventional tools. In this paper, we introduce a prototype of annotation software that aims to interrelate different types of reading synergistically by employing a mixed-initiative approach.

Wikidata promises to reduce factual inconsistencies across all Wikipedia language versions. It will enable dynamic data reuse and complex fact queries within the world's largest knowledge database. Studies of the existing participation patterns that emerge in Wikidata are only just beginning. What delineates most of the contributions in the system has not yet been investigated. Is it an inheritance from the Wikipedia peer-production system or the proximity of tasks in Wikidata that have been studied in collaborative ontology engineering? As a first step to answering this question, we performed a cluster analysis of participants' content editing activities. This allowed us to blend our results with typical roles found in peer-production and collaborative ontology engineering projects. Our results suggest very specialised contributions from a majority of users. Only a minority, which is the most active group, participate all over the project. These users are particularly responsible for developing the conceptual knowledge of Wikidata. We show the alignment of existing algorithmic participation patterns with these human patterns of participation. In summary, our results suggest that Wikidata rather supports peer-production activities caused by its current focus on data collection. We hope that our study informs future analyses and developments and, as a result, allows us to build better tools to support contributors in peer-production-based ontology engineering.

Most of the attention in previous research on the Wikipedia community has been devoted to the study of its production side: editors and their motivations, activity and roles. However, the value of the encyclopedia is also given by the millions of people who access it every day. In this work we focus on the - until now understudied - usage side of Wikipedia, investigating readers’ preference and behaviour as a precious source of information that can provide useful feedback to the editors’ community. One reason for the limited focus on Wikipedia readers in previous research might be how scholars consider the role of passive users, i.e. the readers, in online communities. Readers are often considered to not provide any visible contribution to the community, and have been referred to as “lurkers” or “free-riders” who are “more resource-taking than value-adding”. In our presentation, we want to show how information on reading patterns can be used by the Wikipedia community to support editorial work and to improve existing tools. We study users’ reading preference (what they read) and reading behavior (how they read) in Wikipedia. First, we observe that the most read articles do not necessarily correspond to the articles that are more frequently edited, suggesting some degree of non-alignment between users’ reading interests and authors’ editing interests. We then show that articles differ in the way they are read and that reading patterns can also change over time. For example, whereas some articles are read by many users for a short time, others attract less readers, but more time is spent reading them. In general, this means users show their interest in an article in different ways. Based on these results, we discuss how the study of reading patterns can provide valuable insights to the Wikipedia community and how these can help to build new tools or to adapt existing ones, such as the Article Feedback Tool or the SuggestBot. Information on users’ reading preference and behavior can be used to improve the structure and presentation of an article, as well as to decide which articles to edit next, bridging the gap between Wikipedia’s usage and production side.

Wikipedia is a collaboratively-edited online encyclopaedia that relies on thousands of editors to both contribute articles and maintain their quality. Over the last years, research has extensively investigated this group of users while another group of Wikipedia users, the readers, their preferences and their behavior have not been much studied. This paper makes this group and its %their activities visible and valuable to Wikipedia's editor community. We carried out a study on two datasets covering a 13-months period to obtain insights on users preferences and reading behavior in Wikipedia. We show that the most read articles do not necessarily correspond to those frequently edited, suggesting some degree of non-alignment between user reading preferences and author editing preferences. We also identified that popular and often edited articles are read according to four main patterns, and that how an article is read may change over time. We illustrate how this information can provide valuable insights to Wikipedia's editor community.

Research has shown the importance of a functioning governance system for the success of peer production communities. It particularly highlights the role of human coordination and communication within the governance regime. In this article, we extend this line of research by differentiating two categories of governance mechanisms. The first category is based primarily on communication, in which social norms emerge that are often formalized by written rules and guidelines. The second category refers to the technical infrastructure that enables users to access artifacts, and that allows the community to communicate and coordinate their collective actions to create those artifacts. We collected qualitative and quantitative data from Wikipedia in order to show how a community's consensus gradually converts social mechanisms into algorithmic mechanisms. In detail, we analyze algorithmic governance mechanisms in two embedded cases: the software extension "flagged revisions" and the bot "xqbot". Our insights point towards a growing relevance of algorithmic governance in the realm of governing large-scale peer production communities. This extends previous research, in which algorithmic governance is almost absent. Further research is needed to unfold, understand, and also modify existing interdependencies between social and algorithmic governance mechanisms.

In the literature on different forms of online collaboration, a growing variety of empirical phenomena is subsumed under labels such as “crowdsourcing” or “community”. Defining online collaboration as a form of organizing self-selected actors leading to a joint outcome, we try to clarify these concepts in form of a three-dimensional continuum. On one end, the crowd model is characterized by low task interdependence, central control and automatization as well as by a low level of interaction among its members. On the other end, the community model allows for high task interdependence, decentralized control and a high level of interaction among its members. We then locate ten cases discussed in the literature within this continuum and assess the implications of conceptually differentiating between crowds and communities for open innovation processes. We conclude that the innovation potential of the community model is greater than that of the crowd model, while being associated with a greater loss of control.