Automate your Review

Overview

The AI Review feature in Darwin helps you automatically check your annotation data for quality and consistency. You can create custom validation rules using natural language, tailored to your specific schema. The system uses the Darwin JSON as input, so rules are applied directly to your annotation data.

Key Features

• Custom Rule Creation: Write rules in plain English, referencing your schema’s "class" names, "property" names, "property value" options, and "attribute" names.
• Immediate Feedback: Annotations that do not conform to your rules are highlighted, and you can quickly navigate to the affected annotation, "class", "property", or even the specific video frame.
• Version Control: Rules can be updated and iterated on. All changes are tracked, and you can revert to previous rule versions at any time.
• Processing Time: Rules may take 4–8 minutes to process. Feel free to continue working elsewhere in the platform and return later to review the results.

How to Use AI Review

1. Describe Your Rule: Use natural language to describe the validation logic you need. Be specific about what you are referring to (e.g., "class", "property", "property value", or "attribute").

   • Example: Check that the “Location” property within each annotation does not contain both of the following combinations of values (on the same frame): “Outside” and “Barn” or “Inside” and “Grass”. If an annotation contains both values in either pair (e.g. “Location” = “Outside” and “Barn”), it should be flagged as invalid.

2. Automatic Rule Generation: The system interprets your instructions and creates validation rules for your data.

3. Review Results: Any annotation that fails a rule is flagged. You’ll see exactly which "class", "property", or "attribute" is affected, and can navigate directly to the issue.

   

4. Iterate and Improve: Refine your rules as needed. Use version control to manage changes and revert if necessary.

Workflow Automation with the Logic Stage

You can use the Logic Stage in Darwin to automate your workflow based on the results of the AI Review. Here are two common recommendations:

• Send Back for Correction:

  • If a callout is present, send the asset straight back to "Annotate" for correction.
  • If no callout is present, send the asset to "Review" for further quality assurance.

• Send to Review or Complete:

  • If a callout is present, send the asset to "Review" for manual inspection.
  • If no callout is present, send the asset to "Complete" to finish the workflow.

This branching logic ensures that only assets needing attention are routed for correction or review, while assets that pass all checks can move forward efficiently.

Best Practices

• Be Clear and Specific: Clearly state what you are referring to in your rule (e.g., "class", "property", "property value", or "attribute").
• Check Spelling: Ensure all schema elements are spelled correctly, as spelling errors will be flagged first.
• Test and Iterate: Rules can be sensitive to how they are phrased. Test on sample data and refine as needed.
• Allow Processing Time: Rules may take several minutes to process. You can continue working and return later to review flagged issues.

Example Rule Prompts

• Every 'Person' class must have an 'age' property with a value.
• All 'Event' classes must include a 'timestamp' attribute.

Troubleshooting

• If a rule isn’t working as expected, try rephrasing it for clarity and specificity.

• Consult version history to restore a previous rule set if needed.