The notion of individual fairness requires that similar people receive similar treatment. However, this is hard to achieve in practice since it is difficult to specify the appropriate similarity metric. In this work, we attempt to learn such similarity metrics from human annotated data. We gather a new dataset of human judgments on a criminal recidivism prediction (COMPAS) task. Assuming that people’s judgments encode the fairness metric they adhere to, we leverage prior work on metric learning and attempt to learn people’s similarity metrics from these judgments.