Supporting Effective AI-Augmented Decision-Making in Social Contexts
PI: Kenneth Holstein
Assistant Professor, Human-Computer Interaction Institute Carnegie Mellon University
Framework components: Interaction and Evaluation
This project will study how to support effective AI-augmented decision-making in the context of social work, where predictions regarding human behavior are fundamentally uncertain and where the “ground truth” labels upon which an AI system is trained—for example, whether an 
observed behavior is considered socially harmful—often represent imperfect proxies for the outcomes of which human decision-makers are interested. The goal is to develop an understanding of how expert decision-makers work with AI-based decision support (ADS) to inform social decisions in real-world contexts, and to develop new methods that support effective decision-making in these settings.
The project will investigate the following research questions: 1) How do human decision-makers currently work with existing ADS tools to support social decision-making? 2) How might novel training interfaces and human-AI feedback mechanisms support more effective integrations of human and AI judgment? We will take an iterative, human-centered design approach to explore and develop new interactions, interfaces, and algorithmic methods for human-AI decision-making, combining our team’s expertise across human-computer interaction, AI, psychology, learning sciences, statistics, and social computing.
Key Personnel
Alexandra Chouldechova
Associate Professor of Statistics and Public Policy
Carnegie Mellon University
Luke Guerdan
Graduate Student, Human-Computer Interaction Institute
Carnegie Mellon University
Anna Kawakami
Graduate Student, Human-Computer Interaction Institute
Carnegie Mellon University
Tzu-Sheng Kuo
Graduate Student, Human-Computer Interaction Institute
Carnegie Mellon University
Devansh Saxena
Presidential Postdoctoral Fellow, Human-Computer Interaction Institute
Carnegie Mellon University
Steven Wu
Assistant Professor of Computer Science and Societal Systems
Carnegie Mellon University
Haiyi Zhu
Associate Professor of Human-Computer Interaction
Carnegie Mellon University
Outcomes and Updates
- AI Failure Cards: Understanding and Supporting Grassroots Efforts to Mitigate AI Failures in Homeless Services
- The Situate AI Guidebook: Co-Designing a Toolkit to Support Multi-Stakeholder, Early-stage Deliberations Around Public Sector AI Proposals
- Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use
- Research Explores How a Wikipedia-Like Approach Could Improve AI Evaluation
- Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipedia
- Predictive Performance Comparison of Decision Policies Under Confounding
- Empowering Human Knowledge for More Effective AI-Assisted Decision-Making
- Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge
- A Taxonomy of Human and ML Strengths in Decision-Making to Investigate Human-ML Complementarity
- Award-Winning Research Highlights Challenges and Opportunities for More Reliable Human-AI Decision-Making
- AI Ethics Debate at Chicago Conference, Precursor to CASMI’s Next Workshop
- Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making
- Counterfactual Prediction Under Outcome Measurement Error
- Toward Supporting Perceptual Complementarity in Human-AI Collaboration via Reflection on Unobservables