Evaluation Framework Refinement in Predictive Policing
PI: Kris Hammond
Bill and Cathy Osborn Professor of Computer Science
Northwestern University
Framework component: Evaluation
As part of the precursor work to the formation of CASMI, an initial Framework was developed as a tool that could help designers and evaluators deconstruct ML systems into primitive parts that could be evaluated directly, in order to facilitate system-level evaluation with respect to human health and wellbeing impacts. To further appraise the Framework and to determine what changes and further tools might be necessary for it to be utilized in practice, a more substantive phase of field testing must be undertaken. Applying the Framework deeply in one or more contexts is the next step in assessing its effectiveness, refining the question sets, and developing a robust set of guiding materials for further dissemination. This project will test the effectiveness of the Framework as a tool by applying it to evaluate applications in the Predictive Policing space. Predictive policing is a complex and sensitive application area for ML technologies, which has led to significant societal concern and even calls for outright bans. Exercising the Framework in context could help satisfy two goals. It will help to articulate the specific challenges, impacts, constraints, and best practices within Predictive Policing while also helping to identify opportunities for improvement, expansion, and clarification of the Framework itself.
Key Personnel
Alexander Einarsson
Graduate Student, Computer Science and Artificial Intelligence
Northwestern University
Ryan Jenkins
Associate Professor of Philosophy
California Polytechnic State University
Outcomes and Updates
-
A Debiasing Technique for Place-based Algorithmic Patrol Management
- Ethical Framework Aims to Reduce Bias in Data-Driven Policing
-
A Machine Learning Evaluation Framework for Place-based Algorithmic Patrol Management
-
CASMI Hosts Workshop on Data-Driven Policing