Auto-Track

What is Auto-Track?

Auto-Track is an AI-powered video annotation tool that will streamline object labeling with Polygons and Bounding Boxes.

Auto-Track is particularly effective in handling scenarios that are not suited to linear interpolation such as:

Complex Object Movement
Irregular Motion
Shape Changes

As a result, Auto-Track significantly increases efficiency on your video labeling projects.

How to Use Auto-Track

Create Your Initial Annotation: Begin by creating a polygon or bounding box at the beginning point of the object within the video. We recommend creating your initial annotations with SAM or Auto-Annotate.
Adjust the Label Length & Create Further Keyframes: Creating additional keyframes before activation improves quality outcomes with Auto-Track, therefore we recommend as minimum that you create a keyframe at the beginning and end of the label.
Select Auto-Track Motion: Select your current annotation and click the "Auto-track motion" button to activate Auto-Track.
Track Progress: Monitor the progress indicator on the timeline to see Auto-Track in action.
Review with Edit Mode: Review the Auto-Track generated labels. If corrections are required, click the annotation centroid on the annotation to enter Edit Mode.
Optional: Re-Run Auto-Track: If you've created new keyframes these will help Auto-Track be more accurate! It may be faster to re-run Auto-Track instead of reviewing all your frames.

🚧
You can cancel the Auto-Track after initiation by clicking the "Auto-track motion" button and selecting "Stop auto-tracking motion"

Understanding Keyframes in Auto-Track

User-Generated Keyframes: These are the keyframes that users manually created or edited. These are represented by a diamond symbol on the video timeline, and serve as reference points for the Auto-Track feature.

Auto-Track-Generated Keyframes: Once Auto-Track is activated, it will create model-generated keyframes. These keyframes fill the gap between the user-generated keyframes and are differentiated as circle symbols on the video timeline.

User-Generated and Auto-Track Generated Keyframes are distinguished in the annotation exports for your downstream usage.

Handling Objects Moving In and Out of Frame

Auto-Track has the capability to identify when an object moves in and out of the video frame.

When the object is out of frame, Auto-Track will not place a keyframe on the annotation timeline.

Note: The polygon or bounding box will still be present in this release.

Feedback

Provide Feedback: We highly encourage you to share your thoughts, experiences, and suggestions. Please direct your feedback to your Customer Success Manager (CSM), or through support.

Direct Support: For immediate assistance or specific queries, our support team is ready to help. Contact us in-app!