Import annotations made outside of V7

In this guide, we'll take a look at how you can use Darwin's Python Library to import annotations made outside of V7.

📘

Supported file formats

To import annotations to V7, make sure they're in one of the supported formats below:

  • Darwin JSON
  • COCO
  • Pascal VOC
  • Dataloop
  • Labelbox
  • Super-Annotate
  • CSV (tags only)

Getting started

If this is your first time using the Darwin Python Library, check out our Getting Started Guide to make sure you have the correct version of Python and our SDK installed. This guide will also show you how to generate an API key, which you'll want to hold onto for the steps below.

In addition to an API key, you will also want to gather:

1. The dataset identifier

The dataset identifier is the slugified version of your team name and dataset name (team-name/dataset-name). You can gather this using CLI commands. Start by entering the command below:

darwin authenticate

Enter your API key when prompted (and hold onto it, you'll need it again later on).

Once authenticated, enter the following command to pull up a list of your team's datasets and their identifiers:

darwin dataset remote

2. The annotation format name

The name of the format of the annotations you will be importing to V7. We currently support the following formats:

  • Darwin JSON Format: "darwin"
  • COCO: "coco"
  • Pascal VOC: "pascal_voc"
  • Dataloop: "dataloop"
  • Labelbox: "labelbox"
  • Super-annotate: "superannotate"
  • CSV (for image tags): "csv_tags"
  • CSV (for video tags): "csv_tags_video"

3. The annotation paths

The paths to each of the annotation files you will be importing:

annotation_paths = [
  "/path/to/annotation/1.json",
  "/path/to/annotation/2.json",
  "/path/to/annotation/3.json"
]

Import annotations

Now that we've gathered our API key, dataset identifier, format name, and annotation paths it's time to import annotations.

This starts by initialising the client using the API key:

import darwin.importer as importer
from darwin.client import Client
from darwin.importer import get_importer

client = Client.from_api_key(API_KEY)

From there, our dataset identifier can be used to target the dataset in V7:

dataset = client.get_remote_dataset(dataset_identifier=dataset_identifier)

Next, we can fetch the parser object needed to import annotations in the correct format by plugging the format name into the snippet below:

parser = get_importer(format_name)
importer.import_annotations(dataset, parser, annotation_paths)

Note: The target item for the uploaded annotations is determined by the filename field in the annotation file.

It's also possible to specify the append argument. When append is set to True, the annotations are going to be added to the target items. When append is set to False, the target item is overwritten.

importer.import_annotations(dataset, parser, annotation_paths, append=True)

📘

Appending annotations

Adding append=True to the function call above will add the imported annotations, without overriding the existing ones.

Adding CSV Tags

In order to import tags via a CSV, you should format the .csv file as follows:
-The first column should be the name of the file you are uploading to in V7
-Additional columns should be tag names you wish to add to that file
-There should only be one row per item

e.g. If you wanted to add the tags "Blue", "Yellow Beak" and "Winter" to the "bird.jpeg" file and "Brown" and "Summer" to the "bear.jpeg" file then the CSV would have the following format:

bird.jpeg,Blue,Yellow Beak, Winter
bear.jpeg,Brown,Summer

Adding Annotation Confidence Scores

Darwin-py 0.8+ supports the ability to add confidence scores to your imported annotations. To do this, add you can add an inference field to the annotation payload

"inference": {
        "confidence": 0.75,
        "model": {
          "id": "82b533b4-2637-468b-ba7b-9c9073f4f085",
          "name": "Test",
          "type": "external"
        }
      }

All fields are required:

  • confidence: the confidence score from 0 to 1
  • model
    -id: a UUID v4
    -name: The name of your model
    -type: if the model was trained outside of V7, this should be external

With annotation data, it would look something like the following:

{
        "annotators": [
          {
            "email": "[email protected]",
            "full_name": "Mark CS"
          }
        ],
        "bounding_box": {
          "h": 50.0,
          "w": 35.0,
          "x": 1072.0,
          "y": 618.0
        },
        "inference": {
            "confidence": 0.75,
            "model": {
              "id": "82b533b4-2637-468b-ba7b-9c9073f4f085",
              "name": "Test",
              "type": "external"
            }
        },
        "id": "15461689-5ba8-4eea-b600-07ad250b58e8",
        "name": "Sign DV",
        "polygon": {
          "path": [
            {
              "x": 1090.0,
              "y": 618.0
            },
            {
              "x": 1089.0,
              "y": 619.0
            },
            {
              "x": 1089.0,
              "y": 622.0
            },
            {
              "x": 1086.0,
              "y": 624.0
            },
            {
              "x": 1086.0,
              "y": 626.0
            },
            {
              "x": 1085.0,
              "y": 627.0
            },
            {
              "x": 1085.0,
              "y": 631.0
            },
            {
              "x": 1081.0,
              "y": 634.0
            },
            {
              "x": 1079.0,
              "y": 634.0
            },
            {
              "x": 1075.0,
              "y": 636.0
            },
            {
              "x": 1072.0,
              "y": 636.0
            },
            {
              "x": 1072.0,
              "y": 647.0
            },
            {
              "x": 1074.0,
              "y": 648.0
            },
            {
              "x": 1075.0,
              "y": 650.0
            },
            {
              "x": 1076.0,
              "y": 649.0
            },
            {
              "x": 1077.0,
              "y": 650.0
            },
            {
              "x": 1078.0,
              "y": 649.0
            },
            {
              "x": 1079.0,
              "y": 650.0
            },
            {
              "x": 1078.0,
              "y": 652.0
            },
            {
              "x": 1075.0,
              "y": 652.0
            },
            {
              "x": 1073.0,
              "y": 653.0
            },
            {
              "x": 1072.0,
              "y": 655.0
            },
            {
              "x": 1074.0,
              "y": 657.0
            },
            {
              "x": 1074.0,
              "y": 663.0
            },
            {
              "x": 1075.0,
              "y": 664.0
            },
            {
              "x": 1086.0,
              "y": 663.0
            },
            {
              "x": 1088.0,
              "y": 666.0
            },
            {
              "x": 1091.0,
              "y": 666.0
            },
            {
              "x": 1095.0,
              "y": 668.0
            },
            {
              "x": 1105.0,
              "y": 668.0
            },
            {
              "x": 1107.0,
              "y": 666.0
            },
            {
              "x": 1106.0,
              "y": 664.0
            },
            {
              "x": 1106.0,
              "y": 662.0
            },
            {
              "x": 1107.0,
              "y": 661.0
            },
            {
              "x": 1107.0,
              "y": 656.0
            },
            {
              "x": 1106.0,
              "y": 655.0
            },
            {
              "x": 1107.0,
              "y": 654.0
            },
            {
              "x": 1106.0,
              "y": 640.0
            },
            {
              "x": 1105.0,
              "y": 639.0
            },
            {
              "x": 1105.0,
              "y": 636.0
            },
            {
              "x": 1104.0,
              "y": 635.0
            },
            {
              "x": 1104.0,
              "y": 632.0
            },
            {
              "x": 1103.0,
              "y": 631.0
            },
            {
              "x": 1102.0,
              "y": 626.0
            },
            {
              "x": 1101.0,
              "y": 627.0
            },
            {
              "x": 1100,
              "y": 626.0
            },
            {
              "x": 1100,
              "y": 625.0
            },
            {
              "x": 1098.0,
              "y": 623.0
            },
            {
              "x": 1098.0,
              "y": 621.0
            },
            {
              "x": 1097.0,
              "y": 621.0
            },
            {
              "x": 1095.0,
              "y": 618.0
            }
          ]
        },
        "reviewers": []
      },

That's it! Check out the rest of our Darwin Python Library guides on how to upload images and video, create classes, and pull data.

🚧

Annotation Formats

Some formats, like DICOM, work as VideoAnnotations. If you try to import annotations formatted as ImageAnnotations into a DICOM file, the action will fail. Instead, you need to format your annotations as VideoAnnotations.