The primary papers are listed under "Course Outline" below. The complete reading list and additional material can be found on GitHub. Here are exemplary papers:

• Gillies, Marco, et al. "Human-centred machine learning." Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 2016.
• Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2015). Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine, 35(4), 105–120.
• Dudley, J. J., & Kristensson, P. O. (2018). A Review of User Interface Design for Interactive Machine Learning. ACM Transactions on Interactive Intelligent Systems (TiiS), 8(2), 8–37.

---

Human-Centered Machine Learning (HCML) is about closely linking the possibilities of human perception and intelligence with the computing capacity and performance of computers. However, there are two basic views of integrating humans in the ML process.

On the one hand, the human being is understood as a component of the technical system (human-in-the-loop), which can optimize the learning behavior of algorithms through its interactions with the system. Approaches in this area include reinforcement learning, preference learning, and active learning.

However, in this seminar, we will focus on another perspective. Human-Centered Machine Learning is based on the idea that ML should also be usable by non-technical experts, i.e., persons without background knowledge or experience in this field (algorithm-in-the-loop). User interface design is fundamental to the success of this perspective, but there is a lack of consolidated principles on how such interfaces should be designed. The transparency of such applications, for example, and interpretability of results are an essential prerequisite.

In this seminar, we will conduct a detailed review of existing approaches to human-centered machine learning from the perspective of interactive systems to contextualize them in the field of human-computer interaction. Based on an introduction of the various aspects of HCML, we learn about different approaches of integrating user interfaces in the ML process. Building on this, we open up this new field of research by a more reflective perspective on existing methods through student presentations.

Participants of this seminar are expected to prepare and present one given topic and to discuss the insights with the group. Based on the results of the discussion, participants will elaborate (paper) on their topic in more detail.

Course Organisation

The course is organized on GitLab (internal FU GitLab).

Course Outline

01 | 20.04.2020 — Introduction & Organisation 02 | 27.04.2020 — TOPIC 0: Perspectives in HCML

Paper Session

Gillies, Marco, et al. "Human-centred machine learning." Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 2016.

03 | 04.05.2020 — TOPIC 1: Interactive Machine Learning (iML) - Human-in-the-loop

Paper Session

Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2015). Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine, 35(4), 105–120.

04 | 11.04.2020 — TOPIC 2: Fairness, Accountability, Transparency, and Ethics (FATE) in ML

Paper Session and Guest Talk about "Fairness, Accountability, Transparency and Ethics (FATE)" by Dr. des. Simon David Hirsbrunner.

Crawford, Kate, and Vladan Joler. "Anatomy of an AI system: The Amazon Echo as an anatomical map of human labor, data, and planetary resources." AI Now Institute and Share Lab (2018).

05 | 18.05.2020 — TOPIC 3: Visual Analytics and HCML

Paper Session and Guest Talk about "Visual Analytics and HCML" by Dr.-Ing. Christoph Kinkeldey.

Sacha, Dominik, et al. "What you see is what you can change: Human-centered machine learning by interactive visualization." Neurocomputing 268 (2017): 164-175.

06 | 25.05.2020 — TOPIC 4: Guidelines for Human-AI Interaction

Paper Session

Amershi, Saleema, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, et al. 2019. “Guidelines for Human-AI Interaction.” In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–13. CHI ’19. Glasgow, Scotland Uk: Association for Computing Machinery. https://doi.org/10.1145/3290605.3300233.

01.06.2020 — Pfingsten 07 | 08.06.2020 — Q&A: LaTeX, Overleaf, and other Tools 08 | 15.06.2020 — TOPIC 5: Interactive Clustering - Human-in-the-loop

Paper Session and Guest Talk about "Interactive Clustering and HCML" by Michael Tebbe.

Smith, Alison, et al. "Closing the loop: User-centered design and evaluation of a human-in-the-loop topic modeling system." 23rd International Conference on Intelligent User Interfaces. 2018.

09 | 22.06.2020 — TOPIC 6: Evaluation of Explainable AI Systems

Paper Session

Mohseni, S., Zarei, N., & Ragan, E.D. (2020). A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems. arXiv: Human-Computer Interaction.

10 | 29.06.2020 — TOPIC 7: ML Interpretability I 

Paper Session and Guest Talk about "Machine Learning Interpretability" by Dr.-Ing. Christoph Kinkeldey.

Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019). Machine Learning Interpretability: A Survey on Methods and Metrics. Electronics, 8(8), 832–34. http://doi.org/10.3390/electronics8080832

11 | 06.07.2020 — TOPIC 7: ML Interpretability II 12 | 13.07.2020 —  Feedback Session