Generate annotator reports
V7 offers two options for reporting on your team's performance. In this guide, we'll dive into the reporting insights that V7 offers, and how to access them.
The Annotators Report
The Annotators Report can be found in the lefthand side of any page in V7.
This report provides insights at an annotator level - It highlights:
| Time spent annotating | The cumulative time an annotator spent working in a dataset based on the workview timer. |
| Total annotations created | The cumulative number of annotations created by the annotator. |
| Avg seconds per annotation | Total annotations created divided by time spent annotating. |
| Total images annotated | Total files that an annotator worked on. Videos or DICOM files = 1 image in this report. |
| Review pass rate | The percentage of files that an annotator worked on which were approved vs those that were rejected. Calculated by dividing approved files that were made in previous hours by the total number of files that were approved or rejected. |
Hourly reporting
The Annotators Report is generated on an hourly basis. V7 fetches updated information from the database at the turn of each hour. This can have two impacts on this report's accuracy:
- The report can be as much as 1 hour out of date.
- If a file was worked on in 2 separate hours, it will be counted twice in the Total images annotated column.
This can be problematic for teams working with video files or DICOM series which may take more than 1 hour to annotate. For these teams, we'd recommend reading on for more info about the Items Report which is generated on an item-basis.
The Items Report
The Items Report can be generated from within the Overview tab of any dataset.
This report provides insights at an Item (file) level - It highlights:
| Column name | Description |
|---|---|
| filename | The filename of the item. |
| uploaded_date | When the item was imported to V7. |
| status | The current workflow stage of the item as of the report's generation. |
| workflow_start_date | The timestamp of the moment the item entered the first stage of the workflow. |
| workflow_complete_date | The timestamp of the moment the item was marked as Complete. |
| number_of_frames | The number of frames of a video file and number of slices of a DICOM file. |
| folder | The folder or subfolder within the dataset to which the item has been moved. |
| time_spent_annotating_sec | The total time recorded by the timer when the item was in a Being Annotated stage. |
| time_spent_reviewing_sec | The total time recorded by the timer when the item was in the Review stage. |
| automation_time_annotating_sec | The equivalent in seconds of time saved using Auto Annotate while the file was in a Being Annotated stage (36 seconds).To derive the number of times Auto Annotate was used for an item, divide this number by 36. |
| automation_time_reviewing_sec | The equivalent in seconds of time saved using Auto Annotate while the file was in a Review stage (36 seconds).To derive the number of times Auto Annotate was used for an item, divide this number by 36. |
| annotators | The email addresses of the annotators to whom the item was assigned in a Being Annotated stage. |
| reviewers | The email addresses of the annotators to whom the item was assigned in a Review stage. |
| was_rejected_in_review | Whether the item was ever rejected at any stage of its workflow. |
| url | The link to the file in V7. |
Updated 5 months ago