The LCFI website uses cookies only for anonymised website statistics and for ensuring our security, never for tracking or identifying you individually. To find out more, and to find out how we protect your personal information, please read our privacy policy.

Agent Incentives: A Causal Perspective

Workshop Paper by Tom Everitt, Ryan Carey, Eric Langlois, Pedro A. Ortega, Shane Legg

Agent Incentives: A Causal Perspective. Proceedings of the AAAI 2021 Conference.

Abstract: We present a framework for analysing agent incentives using causal influence diagrams. We establish that a well-known criterion for value of information is complete. We propose a new graphical criterion for value of control, establishing its soundness and completeness. We also introduce two new concepts for incentive analysis: response incentives indicate which changes in the environment affect an optimal decision, while instrumental control incentives establish whether an agent can influence its utility via a variable X. For both new concepts, we provide sound and complete graphical criteria. We show by example how these results can help with evaluating the safety and fairness of an AI system.

Download Workshop Paper