Evaluation Of Large Language Models — Toxicity

Large Language Models made quite a splash in the last year, and that's putting it mildly. AI in general made its way in everybody's lives almost unnoticed, but for the first time, an AI product made worldwide headlines and dazzled the world with human-like levels of question answering. No longer limited to specific topics, but being able to answer seemingly anything one can throw at it, LLMs stormed the scene and show no signs of slowing down.

With this kind of exposure comes intense scrutiny, and rightfully so. Defined.ai can help you build a better model and eliminate toxic responses. Our specialized crowd will make light work of annotating your model output in any locale.

The exact nature of the evaluation can be adapted to your needs. We can assess toxicity on a binary or Likert scale, but also make use of our highly adaptable platform to assess toxicity in a way that makes the most sense for your workflow and pipeline.

Example