Import annotations made outside of V7
In this guide, we'll take a look at how you can use Darwin's Python Library to import annotations made outside of V7.
Supported file formats
To import annotations to V7, make sure they're in one of the supported formats below:
- Darwin JSON
- COCO
- Pascal VOC
- Dataloop
- Labelbox
- Super-Annotate
- CSV (tags only)
- NIfTI (polygons only)
Getting started
If this is your first time using the Darwin Python Library, check out our Getting Started Guide to make sure you have the correct version of Python and our SDK installed. This guide will also show you how to generate an API key, which you'll want to hold onto for the steps below.
In addition to an API key, you will also want to gather:
1. The dataset identifier
The dataset identifier is the slugified version of your team name and dataset name (team-name/dataset-name). You can gather this using CLI commands. Start by entering the command below:
darwin authenticate
Enter your API key when prompted (and hold onto it, you'll need it again later on).
Once authenticated, enter the following command to pull up a list of your team's datasets and their identifiers:
darwin dataset remote
2. The annotation format name
The name of the format of the annotations you will be importing to V7. We currently support the following formats:
- Darwin JSON Format:
"darwin" - COCO:
"coco" - Pascal VOC:
"pascal_voc" - Dataloop:
"dataloop" - Labelbox:
"labelbox" - Super-annotate:
"superannotate" - CSV (for image tags):
"csv_tags" - CSV (for video tags):
"csv_tags_video" - NIfTI (for polygons):
"nifti"
3. The annotation paths
The paths to each of the annotation files you will be importing:
annotation_paths = [
"/path/to/annotation/1.json",
"/path/to/annotation/2.json",
"/path/to/annotation/3.json"
]
Import annotations
Now that we've gathered our API key, dataset identifier, format name, and annotation paths it's time to import annotations.
This starts by initialising the client using the API key:
import darwin.importer as importer
from darwin.client import Client
from darwin.importer import get_importer
client = Client.from_api_key(API_KEY)
From there, our dataset identifier can be used to target the dataset in V7:
dataset = client.get_remote_dataset(dataset_identifier=dataset_identifier)
Next, we can fetch the parser object needed to import annotations in the correct format by plugging the format name into the snippet below:
parser = get_importer(format_name)
importer.import_annotations(dataset, parser, annotation_paths)
It's also possible to specify the append argument. When append is set to True, the annotations are going to be added to the target items. When append is set to False, the target item is overwritten.
importer.import_annotations(dataset, parser, annotation_paths, append=True)
Appending annotations
Adding
append=Trueto the function call above will add the imported annotations, without overriding the existing ones!
Adding CSV Tags
In order to import tags via a CSV, you should format the .csv file as follows:
-The first column should be the name of the file you are uploading to in V7
-Additional columns should be tag names you wish to add to that file
-There should only be one row per item
e.g. If you wanted to add the tags "Blue", "Yellow Beak" and "Winter" to the "bird.jpeg" file and "Brown" and "Summer" to the "bear.jpeg" file then the CSV would have the following format:
bird.jpeg,Blue,Yellow Beak, Winter
bear.jpeg,Brown,Summer
Adding Annotation Confidence Scores
Darwin-py 0.8+ supports the ability to add confidence scores to your imported annotations. To do this, add you can add an inference field to the annotation payload
"inference": {
"confidence": 0.75,
"model": {
"id": "82b533b4-2637-468b-ba7b-9c9073f4f085",
"name": "Test",
"type": "external"
}
}
All fields are required:
confidence: the confidence score from 0 to 1model
-id: a UUID v4
-name: The name of your model
-type: if the model was trained outside of V7, this should beexternal
With annotation data, it would look something like the following:
{
"annotators": [
{
"email": "[email protected]",
"full_name": "Mark CS"
}
],
"bounding_box": {
"h": 50.0,
"w": 35.0,
"x": 1072.0,
"y": 618.0
},
"inference": {
"confidence": 0.75,
"model": {
"id": "82b533b4-2637-468b-ba7b-9c9073f4f085",
"name": "Test",
"type": "external"
}
},
"id": "15461689-5ba8-4eea-b600-07ad250b58e8",
"name": "Sign DV",
"polygon": {
"path": [
{
"x": 1090.0,
"y": 618.0
},
{
"x": 1089.0,
"y": 619.0
},
{
"x": 1089.0,
"y": 622.0
},
{
"x": 1086.0,
"y": 624.0
},
{
"x": 1086.0,
"y": 626.0
},
{
"x": 1085.0,
"y": 627.0
},
{
"x": 1085.0,
"y": 631.0
},
{
"x": 1081.0,
"y": 634.0
},
{
"x": 1079.0,
"y": 634.0
},
{
"x": 1075.0,
"y": 636.0
},
{
"x": 1072.0,
"y": 636.0
},
{
"x": 1072.0,
"y": 647.0
},
{
"x": 1074.0,
"y": 648.0
},
{
"x": 1075.0,
"y": 650.0
},
{
"x": 1076.0,
"y": 649.0
},
{
"x": 1077.0,
"y": 650.0
},
{
"x": 1078.0,
"y": 649.0
},
{
"x": 1079.0,
"y": 650.0
},
{
"x": 1078.0,
"y": 652.0
},
{
"x": 1075.0,
"y": 652.0
},
{
"x": 1073.0,
"y": 653.0
},
{
"x": 1072.0,
"y": 655.0
},
{
"x": 1074.0,
"y": 657.0
},
{
"x": 1074.0,
"y": 663.0
},
{
"x": 1075.0,
"y": 664.0
},
{
"x": 1086.0,
"y": 663.0
},
{
"x": 1088.0,
"y": 666.0
},
{
"x": 1091.0,
"y": 666.0
},
{
"x": 1095.0,
"y": 668.0
},
{
"x": 1105.0,
"y": 668.0
},
{
"x": 1107.0,
"y": 666.0
},
{
"x": 1106.0,
"y": 664.0
},
{
"x": 1106.0,
"y": 662.0
},
{
"x": 1107.0,
"y": 661.0
},
{
"x": 1107.0,
"y": 656.0
},
{
"x": 1106.0,
"y": 655.0
},
{
"x": 1107.0,
"y": 654.0
},
{
"x": 1106.0,
"y": 640.0
},
{
"x": 1105.0,
"y": 639.0
},
{
"x": 1105.0,
"y": 636.0
},
{
"x": 1104.0,
"y": 635.0
},
{
"x": 1104.0,
"y": 632.0
},
{
"x": 1103.0,
"y": 631.0
},
{
"x": 1102.0,
"y": 626.0
},
{
"x": 1101.0,
"y": 627.0
},
{
"x": 1100,
"y": 626.0
},
{
"x": 1100,
"y": 625.0
},
{
"x": 1098.0,
"y": 623.0
},
{
"x": 1098.0,
"y": 621.0
},
{
"x": 1097.0,
"y": 621.0
},
{
"x": 1095.0,
"y": 618.0
}
]
},
"reviewers": []
},
That's it! Check out the rest of our Darwin Python Library guides on how to upload images and video, create classes, and pull data.
Note on uploading Subannotations
If the class for the annotation you are trying to upload already exists, all you need to do is add the sub-annotation to the annotation file. You would then upload as normal.
If the class does not already exist (or does not include this subannotation), you will need to perform the upload twice. The first upload will create the annotation class and ignore subtypes. The second will add the subannotations.
Note on annotation types
There are small differences to note on how the path and filename fields are defined in different annotation file formats. For example, for the Darwin JSON format the path to the folder of the image files within V7 is defined in the 'path' field, while the filename itself is defined within the 'name' field. For a COCO format however, both the filename and path to the folder containing the files is defined within 'file_name' field.
An example of the fields in the Darwin JSON format and a COCO format are shown below respectively
Darwin JSON format
"name": "Brush_Training.jpeg",
"path": "/Set1",
COCO format
"file_name": "Set1/Brush_Training.jpeg",
Updated 30 days ago