The single biggest gap in how teams build LLM applications is testing. People who would never dream of deploying a payments service without a test suite will happily ship a customer-facing chatbot on the strength of having tried twenty prompts by hand and thought the answers looked reasonable. Giskard exists to fix that, and its documentation teaches a discipline that this field badly needs. The core idea is that model behaviour should be tested the way software is tested.
You write scenarios and checks. A check asserts something about how the system should behave given an input: that it does not disclose information from another tenant, that it refuses a request outside its remit, that it does not invent a refund policy, that it stays on topic when a user tries to drag it elsewhere. These run in CI. They fail.
Someone has to fix them. Framed that way, it sounds obvious, and the reason it is not standard practice is that LLM outputs are non-deterministic and people conclude from that non-determinism that testing is impossible. It is not impossible, it is just probabilistic, and a check that a response never contains a competitor's name is perfectly well defined even when the response text varies. Giskard's documentation makes this argument clearly and that alone is worth the time.
The automated vulnerability scanning covers categories people forget: prompt injection, sensitive information disclosure, harmful content generation, robustness to rephrasing, and consistency across semantically identical inputs. The last one is quietly important. A system that answers a question correctly when asked one way and incorrectly when asked a slightly different way is unreliable in a manner that manual testing rarely surfaces, because the human tester asks it their way. The published research is what elevates Giskard above a tooling vendor.
Phare, RealHarm and RealPerformance are benchmarks with methodology and numbers attached, examining hallucination, factual accuracy, bias and harm across models. Whatever you think of any individual benchmark, a company publishing reproducible evaluation research rather than a leaderboard designed to flatter its own product is doing something useful for the field. The DeepLearning.AI collaboration on red teaming LLM applications is also a reasonable entry point if you want structured teaching rather than reference docs. Now the honest limitations.
This is documentation, not a course. It is well written developer reference with conceptual material scattered through it, and someone who wants a sequenced curriculum will have to assemble one. More importantly, automated scanning has a psychological hazard: a clean scan feels like assurance and is not. The scanner tests the vulnerability classes somebody thought to encode.
Your application's worst failure is probably specific to your domain, your data and your users, and no general purpose scanner will find it. The documentation is reasonably honest that automated detection complements rather than replaces domain-specific testing, but every team I have watched adopt a security scanner of any kind has gone through a phase of believing the green tick. Then there is the gap between detection and remediation. Giskard tells you the model can be prompted into disclosing its system instructions.
Fixing that is architecture work, and it may mean that the thing you wanted to build is not safely buildable in the form you wanted. That conversation is outside the tool's scope, reasonably, but it is where the difficulty actually lives. Pricing for the enterprise Hub is quote-based with nothing published, which is standard for the category and still irritating. Four point zero.
The library is free and good, the research is a real contribution, and the discipline it teaches is more valuable than the tool itself.