An important part of monitoring PolicyAI results is reviewing usage and decisions. This allows you to spot-check policy decisions and look at metrics to ensure your policies are performing well.

Decision History

Decision History is a great way to spot-check individual decisions and see how policies are performing. If you have a policy in production, it’s good practice to review recent decisions regularly.

To see your decision history, navigate to the Decision History tab under the Main menu on the sidebar in PolicyAI.

<aside> ℹ️

• In Dev, your Decision History will show results of CSV files or individual pieces of content you have tested against your policies.
• If you have a live policy in Production, your Decision History will show what decisions are being made in real-time from the API, as well as show historical information on past decisions if you select a date range. It will also show results from CSV files or individual pieces of content you have tested against your policies. </aside>

The Decision History will show the latest 100 results. You can sort the table of results by the following:

• Date Range (you can choose one specific day, or any range of dates)
• Policy (all policies, or just one)
• Decision Assessment (Safe, Unsafe, or both)
• Decision Category (custom to the policies you’re using- can choose all policies, or just one)

The table of results will show the following information for each policy:

• The time of the decision (in UTC)
• The policy against which the content was evaluated, including the AI model used.
• The content that was evaluated.
• The result of the evaluation, including:
  • How long the decision took to make
  • The policy label that was applied
  • The severity level
  • Reasoning for the decision
• The decision JSON (the { } button)

Usage

The Usage tab allows you to see how many decisions are being made in any given time period. This allows you to see at a glance which policy categories are most commonly being applied, how much content you’re sending to our API, and if there are any changes over time.