See exactly how model guardrails work, how they fail, and how to defend them. A controlled, no-weaponisation look at LLM jailbreaks and the mitigations that hold.
This course requires a signed disclaimer and government-ID verification to attend. Authorised, named attendees only. Delivered in a sealed local lab, nothing leaves the room.
Restricted course. Signed ethical-use disclaimer required before enrolment. Authorised red-team and research professionals only. Module order and grouping can be adjusted for private deliveries.
How reinforcement learning from human feedback shapes model behaviour during training. You walk through the reward model, the preference dataset and the optimisation loop so you understand exactly what RLHF is doing to the weights and where it leaves gaps.
How constitutional AI rules are baked into a model through self-critique and revision. You study the constitution, the critique loop and the resulting behavioural constraints so you can identify which rules are enforced at training time and which are enforced at inference time.
How system prompts set behavioural boundaries at inference time. You examine the instruction hierarchy from system prompt through developer prompt to user input and learn where each layer can be overridden by a lower one.
How output filters and content classifiers catch unsafe responses after the model generates them. You inspect classifier architectures, threshold tuning and the tradeoff between false positives and missed harmful content that every filter has to make.
Forcing the model into an unrestricted persona through role assignment. You craft persona prompts that override the system prompt and observe how the model's refusal behaviour changes when it believes it is playing a character.
Hiding harmful requests inside encoded text that the model decodes internally. You test base64, ROT13, Unicode substitution and mixed-language prompts to see which encodings slip past input filters and which the model can decode well enough to follow.
Using delimiters, formatting tricks and context framing to separate the harmful request from the safety trigger. You manipulate markdown, XML tags, code blocks and fictional framing to change how the model interprets the boundary between instruction and content.
Gradually escalating across multiple conversation turns so no single message triggers a refusal. You build multi-turn sequences that start innocuous and end with the model producing content it would refuse in a single prompt.
The crescendo technique in depth. You structure a conversation that slowly shifts topic and tone until the model crosses its own safety boundary, and you measure at which turn the guardrail fails.
Filling the context window with many examples of the desired unsafe behaviour so the model treats it as the expected pattern. You test how many shots are needed to override the safety training and how context length affects success rate.
Appending optimised token sequences that disable safety behaviour without changing the readable prompt. You study the Greedy Coordinate Gradient method, run suffix generation against a local model and observe how a nonsense string can flip the model from refusal to compliance.
Perturbing individual tokens to shift the model past its safety threshold. You swap, insert and delete tokens at the embedding level and measure how small perturbations change the probability of a refusal token appearing in the output.
Automating the search for adversarial suffixes at scale. You run search tools against a local model, tune the optimisation parameters and evaluate how transferable a suffix found on one model is when applied to another.
Stripping safety alignment from a model by fine-tuning on a few hundred carefully chosen examples. You fine-tune a local model, compare its behaviour before and after, and see how little data it takes to undo months of alignment work.
Using low-rank adaptation to remove guardrails with minimal compute. You train a LoRA adapter that targets the safety-relevant layers, apply it and measure the resulting safety regression against the base model.
Measuring the safety regression after each fine-tuning attack with a structured benchmark. You score the model on a safety evaluation suite before and after the attack so the damage is a number, not an anecdote.
Creating a benchmark that measures how well guardrails hold under repeated attack. You select categories, write test cases and define pass/fail criteria so the results are repeatable and comparable across model versions.
Scoring the safety regression after each attack technique using the benchmark you built. You run the full attack suite, collect the scores and produce a report that shows exactly which guardrail layers failed and by how much.
Monitoring guardrail effectiveness over time as models are updated, fine-tuned or retrained. You set up a tracking pipeline that runs the benchmark on every model update and flags when safety scores drop below your threshold.
Designing a safety architecture where no single guardrail layer is a single point of failure. You stack training-time alignment, system prompts, input filters, output classifiers and external moderation so an attacker who bypasses one layer still faces the next.
Using the attack data from this course to improve guardrails through adversarial training. You feed successful jailbreaks and suffix attacks back into the training loop and measure whether the guardrails become harder to bypass.
Setting up runtime monitoring that detects when guardrails are being probed or have started to degrade. You wire behavioural detection into the inference pipeline and configure alerts that fire before a guardrail fails completely.
A structured lab where you run every prompt-level and token-level attack against a local model. You work through role override, encoding bypass, multi-turn escalation, crescendo, many-shot priming and adversarial suffixes, recording which techniques succeed and which the guardrails catch.
A hands-on lab where you fine-tune a local model to remove its safety alignment, measure the regression quantitatively and then attempt to restore the guardrails. You see the full attack lifecycle from data preparation through training to evaluation.
A lab focused on building and running the robustness benchmark. You create your test suite, score the model before and after attacks, produce a guardrail drift report and present the findings as a red-team deliverable.
Hands-on with the CyberSecAI tooling, not slideware.
A sealed, local guardrail-removal environment used observe-only. You see a de-guardrailed model behave, never how it was done.
Every guardrail and bypass tied back to the standards.
Behavioural detection that flags jailbreak and abuse patterns at runtime.
Tamper-evident record of every lab session run under the disclaimer.
Not for anyone seeking to weaponise a model. The signed disclaimer and ID verification exist to keep this course to authorised professionals only.
This course counts toward the CyberSecAI AISVS assurance track. Assessment is practical and delivered under the disclaimer: you explain how the guardrail layers work and where they fail, assess a de-guardrailed model first-hand in the sealed lab, and show how you would harden and detect against the risk. On a passing result, the credit is recorded against your assurance track.
One day, hands-on, from how guardrails work to how you defend them, taught by the Founder and Principal AI Security Architect, CyberSecAI -- CISSP, CSSLP, FBCS and the author of the MCPS, ATTP and agent-payment-trust Internet-Drafts. A signed disclaimer and government-ID verification are required to attend. Places are limited.
Corporate & sovereign cohorts, and bespoke on-site delivery, on request.