Reporting Quick Start Guide

Overview

The reporting feature provides insights into annotator efficiency and performance on the platform.

• User-Friendly Interface for easy navigation.
• Downloadable Reports in CSV format.
• Custom Filters by user, dataset, and date range in UTC.
• Sortable Data: Click headers to sort in ascending or descending order.

Why Use This Feature?

• Identify Inefficiencies: Pinpoint slow points in workflows.
• Spot Top Performers: Recognize high-performing labelers to support team development.
• Accurate Payments: Track annotator time for precise payroll.
• Quality Improvement: Identify datasets with frequent rejections to improve data quality.
• Save Time: Get quick insights to streamline team performance.

Tabs and Slices

1. Annotator View: Track individual or team performance across datasets or specific stages.

   • Slices: Filter by User, or by Stage and User.

2. Dataset View: Compare performance across labelers on a dataset level.

   • Slices: Filter by Dataset, or by Dataset and Stage.

📘

Note: Slicing is available only when a single dataset or user is selected.

Main Metrics

• Total Time (Active Time): The tracked time during active work, excluding paused periods.
• Total Annotations: Count of annotations created within the selected time frame.
• Total Items: Number of items the user has engaged with.
• Time per Annotation: Average time spent per annotation.
• Time per File: Average time spent per file.
• Review Pass Rate: Percentage of items passing review on the first attempt.

Deeper Dive: Active Time and Pausing

Active Time represents the time a user is actively engaging in annotating or reviewing. It only includes periods of trackable interaction:

• Mouse clicks or scrolls within the platform (e.g., panning across an image or zooming).
• Keyboard activity (e.g., using hotkeys to navigate frames in a video).

Automatic Pausing happens when:

• The user switches to a different browser tab or window.
• There’s 60 seconds of inactivity in the platform. The 60 seconds before the pause are still counted as active time.

Manual Pausing:

• Users can manually pause time tracking by clicking on the timer, which turns gray to indicate that tracking has stopped. Click the timer again to resume tracking.

This approach ensures that active time accurately reflects periods of focused work, excluding idle or inactive time.

Additional Information

• Historical Data Access: Reports start from the date displayed in the bottom left corner. Use the link in the same location to access prior data.

  

• Timeout Handling: Broad searches on large datasets may time out. Refine your search by time period, user, or dataset to avoid this.

• API Access: All reports can be queried via API, enabling integration with external systems and custom reporting workflows.