Registering new images or videos
After the external bucket is ready and you have populated it with images you need to notify Darwin which images Darwin should list, and for which dataset, this is done via a REST PUT request.
All REST endpoints in Darwin are using api keys for authentication, you can setup your own key here.
The following python scripts shows how to register a newly added image:
import requests
api_key = "<API_KEY HERE>"
team_slug = "your-team-slug-here"
dataset_slug = "your-dataset-slug-here"
storage_name = "your-storage-bucket-name-here"
headers = {
"Content-Type": "application/json",
"Accept": "application/json",
"Authorization": f"ApiKey {api_key}"
}
payload = {
"items": [
{
"path": "/",
"slots": [
{
"as_frames": "false",
"slot_name": "1",
"storage_key": "car_folder/cars.jpg",
"file_name": "cars.jpg"
}
],
"name": "cars.jpg"
}
],
"dataset_slug": dataset_slug,
"storage_slug": storage_name
}
response = requests.post(
f"https://darwin.v7labs.com/api/v2/teams/{team_slug}/items/register_existing",
headers=headers,
json=payload
)
body = response.json()
if response.status_code != 200:
print("request failed", response.text)
elif 'blocked_items' in body and len(body['blocked_items']) > 0:
print("failed to register items:")
for item in body['blocked_items']: print("\t - ", item)
if len(body['items']) > 0: print("successfully registered items:")
for item in body['items']: print("\t - ", item)
else:
print("success")
For more details, please refer to our API reference here.
Generating thumbnails
In cases where you prefer to generate thumbnails on your own, or when you are unable to give write access to V7 to your bucket, the payload changes slightly (notice thumbnail_key, width, height):
"items": [
{
"path": "/",
"slots": [
{
"type": "image",
"slot_name": "0",
"storage_key": "darwin/cars/2008_000074.jpg",
"storage_thumbnail_key": "darwin/2008_000074_thumbnail.jpg",
"width": 1920,
"height": 1080,
"file_name": "2008_000074.jpg"
}
],
"name": "cars.jpg"
}
]
We recommend using mogrify for thumbnail generation.
> mogrify -resize "356x200>" -format jpg -quality 50 -write thumbnail.jpg large.png
Don't use original images as thumbnails
It is strongly recommended that you don't use the originally registered image for your thumbnail. This can lead to CORS issues on some browsers.
Videos
Videos in v7 consist of a single item record, with multiple corresponding frames. Each Frame provides the high quality image that is used to annotate against. If the external storage configurations allows V7 write access; the frames and thumbnails can be extracted by V7 automatically after registering the items.
Below is a simple example with readwrite permissions:
import requests
api_key = "<API_KEY HERE>"
team_slug = "your-team-slug-here"
dataset_slug = "your-dataset-slug-here"
storage_name = "your-storage-bucket-name-here"
headers = {
"Content-Type": "application/json",
"Accept": "application/json",
"Authorization": f"ApiKey {api_key}"
}
payload = {
"items": [
{
"path": "/",
"slots": [
{
"as_frames": False,
"slot_name": "1",
"fps": 10,
"file_name": "seals.mp4",
"storage_key": "seals_folder/seals.mp4",
"type": "video"
}
],
"name": "Seals"
}
],
"dataset_slug": dataset_slug,
"storage_slug": storage_name
}
response = requests.post(
f"https://darwin.v7labs.com/api/v2/teams/{team_slug}/items/register_existing",
headers=headers,
json=payload
)
body = response.json()
if response.status_code != 200:
print("request failed", response.text)
elif 'blocked_items' in body and len(body['blocked_items']) > 0:
print("failed to register items:")
for item in body['blocked_items']: print("\t - ", item)
if len(body['items']) > 0: print("successfully registered items:")
for item in body['items']: print("\t - ", item)
else:
print("success")
The example payload below is for read only external storage; which requires the frames to be pre-extracted:
payload = {
"items": [
{
"path": "/",
"slots": [
{
"as_frames": "false",
"slot_name": "0",
"storage_key": "path/to/satellite-video.mp4",
"file_name": "satellite-video.mp4",
"fps": 2,
"sections": [
{
"height": 1200,
"section_index": 1,
"width": 1400,
"storage_hq_key": "path/to/hq_key_1.jpg",
"storage_lq_key": "path/to/lq_key_1.jpg"
},
{
"height": 1200,
"width": 1400,
"section_index": 2,
"storage_hq_key": "path/to/hq_key_2.jpg",
"storage_lq_key": "path/to/lq_key_2.jpg"
}
],
"type": "video",
"storage_thumbnail_key": "path/to/thumbnail_key.jpg",
},
],
"tags": {"Example_Tag_1": "Example 1", "Example_Tag_2": "Example 2"},
"name": "satellite-video.mp4"
}
],
"dataset_slug": dataset_slug,
"storage_slug": storage_name
}
DICOM and NIFTI files
When importing multiple DICOM or NIFTI files, they can be processed in one of two ways:
- Treat the files as separate items
- Treat the files as slices in a single series/sequence
To treat the files as separate items, specify multiple entries in the items key, e.g.
payload = {
"items": [
{
"path": "/",
"slots": [
{
"as_frames": "false",
"slot_name": "1",
"storage_key": "001.dcm",
"file_name": "001.dcm"
}
],
"name": "001.dcm"
},
{
"path": "/",
"slots": [
{
"as_frames": "false",
"slot_name": "1",
"storage_key": "002.dcm",
"file_name": "002.dcm"
}
],
"name": "002.dcm"
}
],
"dataset_slug": dataset_slug,
"storage_slug": storage_name
}
To treat the files as slices in a single series/sequence:
- 1: Specify multiple entries in the
slotskey, with the sameslot_name, e.g. - 2: Add
"options": {"ignore_dicom_layout": True}to the root of the payload
payload = {
"items": [
{
"path": "/",
"slots": [
{
"as_frames": "false",
"slot_name": "1",
"storage_key": "001/slice1.dcm",
"file_name": "slice1.dcm"
},
{
"as_frames": "false",
"slot_name": "1",
"storage_key": "001/slice2.dcm",
"file_name": "slice2.dcm"
}
],
"name": "001.dcm"
}
],
"dataset_slug": dataset_slug,
"storage_slug": storage_name,
"options": {"ignore_dicom_layout": True}
}
Multi-Planar View
To register files and extract axial, sagittal, and coronal views for a 3D volume:
- 1: Include the
"extract_views": "true" - 2: Make sure you specify
"slot_name": "0"
payload = {
"items": [
{
"path": "/",
"slots": [
{
"as_frames": "false",
"slot_name": "0",
"storage_key": "001/slice1.dcm",
"file_name": "slice1.dcm",
"extract_views": "true"
}
],
"name": "001.dcm"
}
],
"dataset_slug": dataset_slug,
"storage_slug": storage_name,
}
For more information, see our dedicated guide about slots.
Notes on Read Only Access
If you enable read only access then you must add certain fields in the S3 bucket depending:
For Images: thumbnail_key, height and width for each image.
For Videos/DICOMs: thumbnail_key at the item level, hq_key (high quality frames used for annotation when the video is paused) and lq_key (low quality frames used during playback) as well as the height and width for each frame.
Registration Endpoints
When registering read-only files, requests need to be sent to a different endpoint than read-write files:
Read-write
f"https://darwin.v7labs.com/api/v2/teams/{team_slug}/items/register_existing"Read-only
f"https://darwin.v7labs.com/api/v2/teams/{team_slug}/items/register_existing_readonly"
Tips on entering the key in your payload
The key should be the path where the file is stored. Storage keys are case sensitive, so take to ensure they match the filepaths in your external storage exactly.
For S3, exclude the bucket name, e.g. if the full path to your s3 bucket is
s3://example-bucket/darwin/sub_folder/example_image.jpgthen your key should be"darwin/sub_folder/example_image.jpg".For Azure, include the container name, e.g. if the full path to your blob is
https://myaccount.blob.core.windows.net/mycontainer/myblob.jpgthen your key should be"mycontainer/myblob.jpg".
Video Streaming
If you have video streaming enabled then, currently, you have to use readwrite permissions for it to work.
Registering Multiple Items at once
It is possible to enter a list of items to be registered in one API query rather than making an API query for each individual item.
Definitions
Listed below are some definitions to help you when registering your own files.
| Key | Meaning |
|---|---|
| slot_name | Name of the slot. You can read more about slots here but if you only have one item in an image then you can just enter this value as '0' |
| storage_key | The path to your file in your storage container e.g. city/new-york/busy-city.jpeg |
| file_name | Name of the file in a particular slot Note: This will only be relevant to those with multislotted items such as hanging DICOMs |
| path | Optional path to your file in V7 e.g. folder/subfolder. |
| name | Name of the file registered from your storage including extension |
| storage_slug | The slugified version of your storage "Name” from Settings > Storage. You can read more about slugs and identifiers here Note: This is the name and not the storage id |
| as_frames | Boolean value denoting whether you want a video registered as a video (False) or as frames (True) |
| fps | Integer value denoting desired frames per second of video. Enter 'native' for native frame rate |
| dataset_slug | Your dataset slug. You can read more about slugs and identifiers here |
| thumbnail_key | Path to the thumbnail image in your storage (Required for Read Only) |
| height | Height of the registered file (Required for Read Only) |
| width | Width of the registered file (Required for Read Only) |
| hq_key | Path to this high quality file for this frame in your storage (Required for Read Only) |
| lq_key | Path to this low quality file for this frame in your storage (Required for Read Only) |
Updated about 1 month ago