Darwin JSON 1.0 is one of the annotation formats being supported on Darwin. Darwin JSON 1.0 is a legacy format which has now been replaced by Darwin JSON 2.0 as V7's default format.

When a Dataset or a subset of images and videos is exported from Darwin, an archive is generated. The archive includes a set of JSON files encoded in the Darwin JSON format. Each file references a single image or video, and contains the entirety of the annotations that have been made and approved within the platform.

Fields

Export

The content of each Darwin JSON file can be summarised by the following specifications.

type Export = {
  annotations: ImageAnnotation[] | VideoAnntoation[],
  dataset: String,
  image: Image | Video
}

An Export includes all the information about the exported resource. The image field includes information about the referenced resource, that is the image or video uploaded to Darwin; the dataset field simply encodes the Darwin Dataset name that resource belongs to; the annotations field lists all the annotations created on Darwin for that resource.

If the dataset field doesn't need further clarification, let's see what Image, Video, ImageAnnotation and VideoAnnotation look like.

The image field

The image field of an Export is either of type Image or Video.

Image

An Image describes an image resource uploaded to Darwin, including its original metadata.

type Image = {
  /** Internal filename on Darwin */
  filename: string,
  /** Height of the image */
  height: integer,
  /** Original filename of the uploaded file */
  original_filename: string,
  /** The path of the file within Darwin.
    * The path starts at root ("/"), and expands including the
    * folders in which the resource is stored.
    */
  path: string,
  /** The sequence number is a monotonic increasing number
    * for each file uploaded to a Darwin Dataset. The first
    * uploaded file is assigned a sequence number of 1,
    * the next one 2 and so on. 
    */
  seq: integer,
  /** The URL of the image thumbnail */
  thumbnail_url: string,
  /** The URL of the original image */
  url: string,
  /** Width of the image */
  width: integer,
  /** The URL of the image on Darwin.
    * It routes directly to the Darwin Workview
    */
  workview_url: string
}

Video

A Video describes a video resource uploaded to Darwin, including its original metadata.

type Video = {
  /** Internal filename on Darwin */
  filename: string,
  /** Frame-rate in which the video has been annotated on Darwin */
  fps: number,
  /** The number of frames generated from the original video */
  frame_count: integer,
  /** The URLs of the frames generated from the original video */
  frame_urls: string[],
  /** Original filename of the uploaded file */
  original_filename: string,
  /** The path of the file within Darwin.
    * The path starts at root ("/"), and expands including the
    * folders in which the resource is stored.
    */
  path: string,
  /** The sequence number is a monotonic increasing number
    * for each file uploaded to a Darwin Dataset. The first
    * uploaded file is assigned a sequence number of 1,
    * the next one 2 and so on. 
    */
  seq: number,
  /** The URL of the original image */
  url: string,
  /** The URL of the image on Darwin.
    * It routes directly to the Darwin Workview
    */
  workview_url: string
}

The annotations field

The annotations field lists all the annotations created on Darwin for an image or video, and its encoded respectively as an ImageAnnotation or VideoAnnotation. Let's see what they look like.

ImageAnnotation

An ImageAnnotation encodes the information of an annotation created on a image on Darwin. It includes the annotation data itself, but also the annotation class name and it may optionally include authorship information.

type ImageAnnotation = {
  /** An optional list of annotators of the image */
  annotators: Authorship[] | undefined,
  /** The annotation class name */
  name: string,
  /** An optional list of reviewers of the image */
  reviewers: Authorship[] | undefined,
  /** The actual data of the annotation */
  [annotation_type: MainAnnotationType]: AnnotationData,
  /** Optional sub annotation data **/
  ...[annotation_type: SubAnnotationType]: AnnotationData
}

To have a full overview and give an example of an ImageAnnotation, let's define the Authorship, MainAnnotationType, SubAnnotationType and AnnotationData

Authorship

An Authorship object encodes the information of a Darwin annotator or reviewer. It only includes their email and full name.

type Authorship = {
  /** Email of the annotator or reviewer on Darwin */
  email: string,
  /** Full name (first name + last name) of the annotator or reviewer */
  full_name: string
}

MainAnnotationType

A MainAnnotationType type simply encodes a string of one of the main annotation types available on Darwin. Note that a single annotation includes one and only one main annotation type.

type MainAnnotationType =
  'bounding_box' |
  'cuboid' |
  'ellipse' |
  'line' |
  'keypoint' |
  'polygon' |
  'skeleton' |
  'tag'

SubAnnotationType

A SubAnnotationType type encodes a simple string of one of the sub annotation types supported on Darwin. Note that a single annotation may include multiple sub annotation types, provided that the main annotation type supports them.

type SubAnnotationType =
  'attributes' |
  'directional_vector' |
  'instance_id' |
  'text'

AnnotationData

AnnotationData represents the core data of an annotation created on Darwin.

type AnnotationData =
  Attributes |
  BoundingBox |
  Cuboid |
  DirectionalVector |
  Ellipse |
  InstanceID |
  Line |
  Keypoint |
  Polygon |
  Skeleton |
  Tag |
  Text

type Attributes = {
  /** List of attribute names */
  attributes: string[]
}

type BoundingBox = {
  /** Height of the bounding box */
  h: float,
  /** Width of the bounding box */
  w: float,
  /** Left-most coordinate of the bounding box */
  x: float,
  /** Top-most coordinate of the bounding box */
  y: float
}

type Cuboid = {
  /** The back face of the cuboid */
  back: BoundingBox,
  /** The front face of the cuboid */
  front: BoundingBox
}

type DirectionalVector = {
  /** Orientation of the vector in radians */
  angle: float,
  /** Length of the vector */
  length: float
}

type Ellipse = {
  /** Orientation of the ellipse in radians */
  angle: float,
  /** The center point of the ellipse */
  center: Keypoint,
  /** The two radii of the ellipse, encoded as a point */
  radius: Keypoint
}

type InstanceID = {
  /** The positive integer value assigned to the annotation */
  value: number
}

type Line = {
  /** A list of adjacent points, to be considered as an open path */
  path: Keypoint[]
}

type Keypoint = {
  /** The horizontal coordinate of the keypoint */
  x: number,
  /** The vertical coordinate of the keypoint */
  y: number
}

type Polygon = {
  /** A list of adjacent points, to be considered as a closed path */
  path: Keypoint[]
}

type Skeleton = {
  /** A list of skeleton nodes */
  nodes: {
    /** The name of the node */
    name: string,
    /** A boolean value, which is true when the node is occluded,
      * or false when the node is visible
      */
    occluded: boolean,
    /** The horizontal coordinate of the keypoint */
    x: number,
    /** The vertical coordinate of the keypoint */
    y: number
  }[]
}

type Tag = {}

type Text = {
  /** The actual text */
  text: string
}

VideoAnnotation

A VideoAnnotation object encodes the information of an annotation created on a video on Darwin. It includes all the frame annotation data, but also the annotation class name, interpolation information, and it may optionally include authorship information.

type VideoAnnotation = {
  annotators: Authorship[] | undefined,
  frames: Frame[],
  interpolate_algorithm: string,
  interpolated: boolean,
  name: string,
  reviewers: Authorship[],
  segments: Segment[]
}

The core data of a VideoAnnotation can be found in the frames field. Each Frame can be described as an AnnotationData, but with additional data depending on the way the frame was created on Darwin (e.g. manually by an annotator, or programmatically by the interpolation algorithm).

Frames

A Frame object encodes the information of a frame annotation, either manually created by the annotators on Darwin, or algorithmically interpolated by the specified VideoAnnotation's interpolated_algorithm field.

type Frame: {
  [frame_key: string]: {
    keyframe: boolean,
    [annotation_type: MainAnnotationType]: AnnotationData,
    ...[annotation_type: SubAnnotationType]: AnnotationData
  }
}

{
  "image": {
    "seq": 1,
    "width": 1920,
    "height": 1080,
    "filename": "00000001.png",
    "original_filename": "Original.png",
    "url": "https://darwin.v7labs.com/api/images/-1/original",
    "thumbnail_url": "https://darwin.v7labs.com/api/images/-1/thumbnail",
    "path": "/",
    "workview_url": "https://darwin.v7labs.com/workview?dataset=-1&image=1"
  },
  "dataset": "Empty",
  "annotations": [
    {
      "annotators": [
        {
          "email": "andrea@v7labs.com",
          "full_name": "Andrea (V7 Support)"
        }
      ],
      "bounding_box": {
        "h": 247.89,
        "w": 301.45,
        "x": 102.98,
        "y": 125.81
      },
      "name": "Bounding Box",
      "reviewers": [],
      "text": {
        "text": "A simple box"
      }
    },
    {
      "annotators": [
        {
          "email": "andrea@v7labs.com",
          "full_name": "Andrea (V7 Support)"
        }
      ],
      "attributes": [
        "zig-zag"
      ],
      "line": {
        "path": [
          {
            "x": 728.3,
            "y": 300.21
          },
          {
            "x": 816.75,
            "y": 362.49
          },
          {
            "x": 881.52,
            "y": 262.84
          },
          {
            "x": 969.97,
            "y": 327.61
          },
          {
            "x": 1053.43,
            "y": 256.61
          }
        ]
      },
      "name": "Line",
      "reviewers": []
    },
    {
      "annotators": [
        {
          "email": "andrea@v7labs.com",
          "full_name": "Andrea (V7 Support)"
        }
      ],
      "cuboid": {
        "back": {
          "h": 226.71,
          "w": 227.96,
          "x": 1488.17,
          "y": 195.57
        },
        "front": {
          "h": 219.24,
          "w": 226.71,
          "x": 1430.87,
          "y": 249.13
        }
      },
      "name": "Cuboid",
      "reviewers": []
    },
    {
      "annotators": [
        {
          "email": "andrea@v7labs.com",
          "full_name": "Andrea (V7 Support)"
        }
      ],
      "ellipse": {
        "angle": 1.41,
        "center": {
          "x": 381.38,
          "y": 789.13
        },
        "radius": {
          "x": 141.61,
          "y": 156.71
        }
      },
      "name": "Ellipse",
      "reviewers": []
    },
    {
      "annotators": [
        {
          "email": "andrea@v7labs.com",
          "full_name": "Andrea (V7 Support)"
        }
      ],
      "keypoint": {
        "x": 453.01,
        "y": 747.4
      },
      "name": "Keypoint",
      "reviewers": []
    },
    {
      "annotators": [
        {
          "email": "andrea@v7labs.com",
          "full_name": "Andrea (V7 Support)"
        }
      ],
      "directional_vector": {
        "angle": -1.03,
        "length": 196.65
      },
      "instance_id": {
        "value": 2
      },
      "name": "Polygon",
      "polygon": {
        "path": [
          {
            "x": 1590.31,
            "y": 718.75
          },
          {
            "x": 1707.4,
            "y": 783.53
          },
          {
            "x": 1706.16,
            "y": 915.57
          },
          {
            "x": 1590.31,
            "y": 980.35
          },
          {
            "x": 1474.46,
            "y": 914.33
          },
          {
            "x": 1474.46,
            "y": 781.04
          }
        ]
      },
      "reviewers": []
    },
    {
      "annotators": [
        {
          "email": "andrea@v7labs.com",
          "full_name": "Andrea (V7 Support)"
        }
      ],
      "name": "Skeleton",
      "reviewers": [],
      "skeleton": {
        "nodes": [
          {
            "name": "1",
            "occluded": false,
            "x": 992.08,
            "y": 710.53
          },
          {
            "name": "2",
            "occluded": false,
            "x": 1078.18,
            "y": 711.8
          },
          {
            "name": "3",
            "occluded": false,
            "x": 1035.94,
            "y": 768.64
          },
          {
            "name": "4",
            "occluded": false,
            "x": 975.14,
            "y": 829.05
          },
          {
            "name": "5",
            "occluded": false,
            "x": 1035.34,
            "y": 831.64
          },
          {
            "name": "6",
            "occluded": false,
            "x": 1096.99,
            "y": 829.12
          }
        ]
      }
    }
  ]
}

{
  "dataset": "My Dataset",
  "image": {
    "seq": 1,
    "fps": 1.0,
    "filename": "00000001.mp4",
    "original_filename": "original_video.mp4",
    "url": "https://darwin.v7labs.com/api/videos/1/original",
    "path": "/",
    "workview_url": "https://darwin.v7labs.com/workview?dataset=1&image=11",
    "frame_count": 30,
    "frame_urls": [
      "https://darwin.v7labs.com/api/videos/1/frames/0",
      "https://darwin.v7labs.com/api/videos/1/frames/1",
      "https://darwin.v7labs.com/api/videos/1/frames/2",
      "https://darwin.v7labs.com/api/videos/1/frames/3",
      "https://darwin.v7labs.com/api/videos/1/frames/4",
      "https://darwin.v7labs.com/api/videos/1/frames/5",
      "https://darwin.v7labs.com/api/videos/1/frames/6",
      "https://darwin.v7labs.com/api/videos/1/frames/7",
      "https://darwin.v7labs.com/api/videos/1/frames/8",
      "https://darwin.v7labs.com/api/videos/1/frames/9",
      "https://darwin.v7labs.com/api/videos/1/frames/10",
      "https://darwin.v7labs.com/api/videos/1/frames/11",
      "https://darwin.v7labs.com/api/videos/1/frames/12",
      "https://darwin.v7labs.com/api/videos/1/frames/13",
      "https://darwin.v7labs.com/api/videos/1/frames/14",
      "https://darwin.v7labs.com/api/videos/1/frames/15",
      "https://darwin.v7labs.com/api/videos/1/frames/16",
      "https://darwin.v7labs.com/api/videos/1/frames/17",
      "https://darwin.v7labs.com/api/videos/1/frames/18",
      "https://darwin.v7labs.com/api/videos/1/frames/19",
      "https://darwin.v7labs.com/api/videos/1/frames/20",
      "https://darwin.v7labs.com/api/videos/1/frames/21",
      "https://darwin.v7labs.com/api/videos/1/frames/22",
      "https://darwin.v7labs.com/api/videos/1/frames/23",
      "https://darwin.v7labs.com/api/videos/1/frames/24",
      "https://darwin.v7labs.com/api/videos/1/frames/25",
      "https://darwin.v7labs.com/api/videos/1/frames/26",
      "https://darwin.v7labs.com/api/videos/1/frames/27",
      "https://darwin.v7labs.com/api/videos/1/frames/28",
      "https://darwin.v7labs.com/api/videos/1/frames/29"
    ]
  },
  "annotations": [
    {
      "annotators": [
        {
          "email": "andrea@v7labs.com",
          "full_name": "Andrea Azzini"
        }
      ],
      "frames": {
        "0": {
          "attributes": [
            "three"
          ],
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 258.59,
              "y": 196.06
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": true
        },
        "1": {
          "attributes": [
            "three"
          ],
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 258.59,
              "y": 196.06
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "10": {
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 568.45,
              "y": 268.17
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "11": {
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 568.45,
              "y": 268.17
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "12": {
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 568.45,
              "y": 268.17
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "13": {
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 568.45,
              "y": 268.17
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "14": {
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 568.45,
              "y": 268.17
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "15": {
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 568.45,
              "y": 268.17
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "16": {
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 568.45,
              "y": 268.17
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "17": {
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 568.45,
              "y": 268.17
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "2": {
          "attributes": [
            "three"
          ],
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 258.59,
              "y": 196.06
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "3": {
          "attributes": [
            "three"
          ],
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 258.59,
              "y": 196.06
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "4": {
          "attributes": [
            "three"
          ],
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 258.59,
              "y": 196.06
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "5": {
          "attributes": [
            "three"
          ],
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 258.59,
              "y": 196.06
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "6": {
          "attributes": [
            "three"
          ],
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 258.59,
              "y": 196.06
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "7": {
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 568.45,
              "y": 268.17
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": true
        },
        "8": {
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 568.45,
              "y": 268.17
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        },
        "9": {
          "ellipse": {
            "angle": 0.64,
            "center": {
              "x": 568.45,
              "y": 268.17
            },
            "radius": {
              "x": 84.96,
              "y": 84.96
            }
          },
          "keyframe": false
        }
      },
      "interpolate_algorithm": "linear-1.1",
      "interpolated": true,
      "name": "round object",
      "reviewers": [],
      "segments": [
        [
          0,
          18
        ]
      ]
    },
    {
      "annotators": [
        {
          "email": "andrea@v7labs.com",
          "full_name": "Andrea Azzini"
        }
      ],
      "frames": {
        "11": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": true
        },
        "12": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "13": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "14": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "15": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "16": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "17": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "18": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "19": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "20": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "21": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "22": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "23": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "24": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "25": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "26": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "27": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "28": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        },
        "29": {
          "bounding_box": {
            "h": 361.69,
            "w": 91.27,
            "x": 805.63,
            "y": 129.58
          },
          "keyframe": false
        }
      },
      "interpolate_algorithm": "linear-1.1",
      "interpolated": true,
      "name": "scale",
      "reviewers": [],
      "segments": [
        [
          11,
          30
        ]
      ]
    }
  ],
  "annotators": [
    {
      "email": "andrea@v7labs.com",
      "full_name": "Andrea Azzini"
    }
  ]
}