Evaluation

Without a test set and a metric you have a demo, not a system.

• Artificial intelligence
• Software engineering

1 idea that carry it

• TechniqueArtificial intelligenceVideo located

  Without evaluation you have a demo, not a system

  Build the test set before the feature, measure every change against it, and keep a person in the loop where the metric cannot be trusted.

  Any AI product discussion, any system design that includes a model, and the question of how you know an AI feature works.

  From How to Systematically Set Up LLM Evals · Dave Ebbelaar