AI & LLM Security Tools

Library for machine learning security that evaluates, defends, and verifies models against evasion, poisoning, extraction, and inference attacks.

Pytest-style evaluation framework for LLM applications with automated metrics for RAG accuracy, hallucination detection, and safety testing.

Generative AI vulnerability scanner and red-teaming tool that probes language models for prompt injections and data leaks.

Open-source testing framework for scanning LLMs and machine learning models for hallucinations, prompt injections, and biases.

Python framework for adding structural validation, schema enforcement, and output safety guardrails to language model responses.

Evaluation framework from the UK AI Security Institute and Meridian Labs for testing language models across reasoning, safety, and tool use.

Open-source toolkit from NVIDIA for adding programmable safety guardrails, topic controls, and policy filters to LLM apps.

CLI evaluation tool and test runner for LLM application security, automated red teaming, and CI/CD security pipeline gating.

Python automation framework from Microsoft for red teaming generative AI endpoints, agent workflows, and prompt defenses.