This guide covers how to write content moderation policies that Large Language Models (LLMs) interpret well. LLMs offer significant flexibility, but effective policies follow practices tailored to how these models read instructions.

💡 Important disclaimer: These tips and strategies are based on current observations and performance with specific models. They may not generalize directly to new models or contexts and should always be validated through testing before being applied in production. LLM behavior can be nuanced and sometimes unpredictable. If you have questions, we’re here to help!

Setting up your first policy

There are four ways to start a policy in Manage Policies:

• Start from scratch. Click Create New Policy, name it, and add a description — a good place for a version number or date.
• Start from a preset. The Presets tab includes a simple default policy and a best-practices template you can build on.
• Convert an existing policy document. Click New Policy → Convert Policy to turn a policy you already have (for example, a written policy doc) into structured policy text.
• Duplicate an existing policy to use one of your current policies as a starting point.

Your policy should start with an H1 heading named "Policies" (# Policies). Under that, list your policy categories as H2 headings (e.g., ## Category Name) and include information about what is and isn't considered a violation of that category. Ensure category names follow the naming conventions (no special characters). Anything with a header format will populate a Moderation Label in the sidebar.

Save your policy draft periodically as you work. You can save your work as a new version, and restore a previous version from the version history if you need to roll back.

Writing clear policy definitions

The definitions within your policy categories need to be unambiguous for the LLM.

• Keep policies as simple as possible. Define terms clearly within or immediately after the category heading.
• Look for areas of repetition and consolidate definitions. But avoid excessive consolidation that results in run-on sentences packed with too many concepts, which confuses the LLM.
• Do not phrase policy definitions negatively (e.g., simply saying "No hate speech").

Example: When reviewing results for a policy that said, "No hate speech", we saw reasoning from the LLM for both:

• "The content does not contain hate speech, therefore it does violate the policy" and

• "The content does not contain hate speech, therefore it does not violate the policy."

Instead, phrase it positively or as a rule the content must not break (e.g., "Content violates this policy if it contains hate speech as defined below...").

In areas where your policy is unclear, vague, or doesn't explicitly cover a scenario, you will often observe the LLM giving inconsistent results for similar content. This is a helpful signal that the policy itself needs refinement to be more definitive in that scenario.