Generate annotator reports

V7 offers two options for reporting on your team's performance. In this guide, we'll dive into the reporting insights that V7 offers, and how to access them.

The Annotators Report

The Annotators Report can be found in the lefthand side of any page in V7.

This report provides insights at an annotator level - It highlights:

Column Name

Description

Time spent annotating

The cumulative time an annotator spent working in a dataset based on the workview timer.

Total annotations created

The cumulative number of annotations created by the annotator.

Avg seconds per annotation

Total annotations created divided by time spent annotating.

Total images annotated

Total files that an annotator worked on. Videos or DICOM files = 1 image in this report.

Review pass rate

The percentage of files that an annotator worked on which were approved vs those that were rejected. Calculated by dividing approved files that were made in previous hours by the total number of files that were approved or rejected.

🚧

Hourly reporting

The Annotators Report is generated on an hourly basis. V7 fetches updated information from the database at the turn of each hour. This can have two impacts on this report's accuracy:

  • The report can be as much as 1 hour out of date.
  • If a file was worked on in 2 separate hours, it will be counted twice in the Total images annotated column.

This can be problematic for teams working with video files or DICOM series which may take more than 1 hour to annotate. For these teams, we'd recommend reading on for more info about the Items Report which is generated on an item-basis.

The Items Report

The Items Report can be generated from within the Overview tab of any dataset.

This report provides insights at an Item (file) level - It highlights:

Column Name

Description

filename

The filename of the item.

uploaded_date

When the item was imported to V7.

status

The current workflow stage of the item as of the report's generation

workflow_start_date

The timestamp of the moment the item entered the first stage of the workflow.

workflow_complete_date

The timestamp of the moment the item was marked as Complete.

number_of_frames

The number of frames of a video file and number of slices of a DICOM file.

folder

The folder or subfolder within the dataset to which the item has been moved.

time_spent_annotating_sec

The total time recorded by the timer when the item was in a Being Annotated stage.

time_spent_reviewing_sec

The total time recorded by the timer when the item was in the Review stage.

automation_time_annotating_sec

The equivalent in seconds of time saved using Auto Annotate while the file was in a Being Annotated stage (36 seconds).To derive the number of times Auto Annotate was used for an item, divide this number by 36.

automation_time_reviewing_sec

The equivalent in seconds of time saved using Auto Annotate while the file was in a Review stage (36 seconds).To derive the number of times Auto Annotate was used for an item, divide this number by 36.

annotators

The email addresses of the annotators to whom the item was assigned in a Being Annotated stage.

reviewers

The email addresses of the annotators to whom the item was assigned in a Review stage.

was_rejected_in_review

Whether the item was ever rejected at any stage of its workflow.

url

The link to the file in V7.