Consistency Training

Consistency Training

new GDM research notes that both jailbreaking and sycophancy share a common cause — subtle changes in the prompt cause dramatic changes in output

they address it by training on small prompt changes, expecting the same result

deepmindsafetyresearch.medium.com/consistency-...

this is self-supervised learning

any prompt/response pair can be the seed, then use an LLM to rewrite

methods like this scale well because they can get ahead of new edge cases by simply training longer