FrameworkArtificial intelligenceBook
A system optimises the objective you wrote, not the one you meant
Whatever you can measure and reward will be maximised literally, including the gaps between the measure and the goal.
AI product and risk discussions, designing any metric or bonus scheme, and evaluating a model's objective.
From The Alignment Problem · Brian Christian