Migrating from V1 to V2

This guide takes you through the process of migrating from the V1 to V2 API. It is aimed at the technical aspects of V7 and will mostly not cover UI changes.

This guide will cover changes to API endpoints, the SDK/CLI, Darwin JSON/other export formats and more. It will also discuss continuity from V1 as well as brand new functionality present in V2.

Does this guide apply to me?

If you make use of the V7 API, SDK or CLI then it is vital that you read this guide to make sure that nothing breaks during migration.

Otherwise, refer to our guide on the UI changes.

If you are a newer customer who has always been on V2, then there is no need to take any action.

❗️

Some APIs and Identifiers have breaking changes

It is crucial to read through this document if you use any technical components. Some of our APIs and identifiers have breaking changes.

If you have any questions then please reach out.

What has changed?

The table below outlines each section which has changed. We'll dive deeper into these later in the guide:

How do I migrate a dataset from V1 to V2?

We will reach out with a proposed migration timeline where we will help support you through the process. Based on your V7 usage, we may suggest a bespoke plan if you have a more complicated set up.

Please make sure to read through the migration docs in full with plenty of time before the planned migration.

At an agreed time, we will migrate your datasets. For the duration of this migration, they will be temporarily unavailable for use. This process will take several minutes depending on the size of the dataset and the new dataset will be a 1 to 1 map of the previous workflow.

If you have a CSM, they will be more than happy to answer any questions you may have.

We look forward to migrating you onto the latest version of our platform!

🚧

Testing V2

Our recommendation for transitioning to V2 is the following:
-Create a V2 dataset. We can enable this for you if you reach out to your CSM or Support
-Rewrite your scripts to use V2 APIs to test functionality
-Migrate your V1 datasets and apply your new scripts to these making sure to use the new identifiers (dataset_ids, item_ids etc.)

Now we will take a deeper dive into the changes:

Changes to the Darwin-py SDK and CLI:

Our darwin-py and CLI are backwards compatible. The only thing that you need to change are ids such as but not limited to dataset_id, item_id, stage_id etc.

As always, you should ensure you are using the latest version of the darwin-py library - run pip install darwin-py --upgrade to install our latest version

Identifiers (Very Important):

The item and dataset identifiers will change between v1 and v2. It is critical that these are updated.

Slugs will not change and will also be included in the v2 export.

API endpoints with no changes from V1:

The below describes all API endpoints that are identical in V1 and V2. No changes are needed other than identifiers as previously mentioned

Annotation classes

Datasets (apart from anything related to items)

Async Reports

Memberships

External Storage (apart from registration)

Wind/Model

Items Report

Although many of the item APIs have changed, there is no change to the items report API and no change in the response format.

API endpoints with changes from V1 to V2:

The following covers endpoints which have changed between V1 and V2.

All items APIs:

Our item APIs have changed quite substantially. This includes registering items, updating items, listing items and more. You can see a full list of these APIs here.

One of the largest changes to consider is that the queries are now more flexible and can accommodate more than one dataset_id at a given time. This allows you to query items across a team with one request. We refer to this as itemization.

Another significant change is the introduction of slot information which allow users to render multiple items on the image canvas. You can read about this here.

Most queries have only had their endpoint changed and can include more query params but some will additionally require entering slot information. These include:
-Setting the layout of items by a provided filter
-Returning a pre-signed m3u8 index for streaming a requested slot
-Returning a list of slot sections

Note: Most of the time items will be single slotted and will need little change.

Workflows:

The workflows API has been changed substantially and reflects the introduction of the workflows wizard in the UI.

One of the biggest changes is that in V2 you are now able to create, update and delete workflows programmatically.

You can also now assign workflow stages to users using the workflow_id rather than retrieving the stage_id. Generally there will be no need to use the stage_id unless you want to set the stage of multiple items - simplifying these API requests.

Exports:

In exports, only the API endpoint has changed. The types of APIs, functionality and params are otherwise the same.

Registering Items in External Storage:

Some fairly major changes have been introduced to external storage item registration(read only is particularly different). You can see the new format on our V2 registration page and you can also see the API calls in our Imports API docs.

Items that were previously registered using V1 will be automatically migrated. However, new items will need to be registered using the new endpoint.

Webhooks

Along with other workflow stages, webhooks are now created using our workflows API.

Webhooks are not migrated from v1 to v2. In order to preserve webhooks, please refer to the ‘Manual Parts of Migration’ guide.

Tagging/Untagging:

The tagging/untagging endpoints have now been changed (along with other annotation APIs). We encourage using our SDK to handle importing/annotations annotations. Reach out to our CS or Support team if you have bespoke requirements.

Unique new V2 functionality:

There are some features in V2 that simply didn’t exist in V1. Here is a brief description with links to more detailed documentation.

Slots. An example would be an individual section of a hanging protocol. You can read more in our docs here.

Webhooks are now a part of workflow stages along with other stages such as annotation stage, review stage, code stage, etc.

-Additional flexibility when querying with itemization. This is described in our itemization docs.

Guide on Changing Workflows during/just after Migration:

Sometimes it will be necessary to change the automatically created workflow generated by the migration. For the sake of simplicity, this change will be performed in the UI. See our new workflows guide for details on how to set up your workflow and connect it to a dataset.

A workflow will be automatically created on migration. If you wish to change your workflow or certain stages aren’t included automatically (like webhooks), then you can create your desired workflow with no dataset attached. Once the dataset migration is complete then you can swap in this workflow for the automatically created one.

Note: The workflow itself will be replicated during the migration process to include the sections (such as annotate, review) which were in the V1 dataset. There is no need to create a new workflow if you are making no changes and do not use webhooks or consensus.

Changes to Darwin JSON:

With the introduction of slots, there are some changes to the format of Darwin JSON.

There will be a temporary option to export in either v1 or v2 format to allow a transition period if your data pipeline currently relies on the previous form of DarwinJSON. Versions of darwin-py 0.8.0 onwards will also allow you to convert between formats. You can read more details about this and upcoming Darwin JSON changes here.

Generally, if you don’t use much metadata in the exports, then there will be little that needs changing other than the addition of slot information.

The metadata section has changed quite substantially and it is worth reviewing our examples in our docs page.

🚧

Changes to Darwin JSON 2.0

It is critical that all teams review changes to Darwin JSON when migrating to using Darwin JSON V2.0.

Changes to other Export Formats:

Other export formats such as COCO are unchanged in V2 but currently only work for single slotted items. If you want to get exports for multi-slotted items then you will need to use Darwin JSON.

Annotator Reports:

Annotator reports will be migrated automatically during the migration and the format will be the same.

Parent/Child Teams:

There is no specific change that would impact parent/child teams. However, it is important that partners know about the changes to the API.

Manual Parts of the Migration:

The vast majority of the migration will be automatic but there are some sections that need manual intervention.

There are some parts of the system which won’t be automatically migrated. These include:

DICOMS with layout:

This will only apply to a few of customers and we will reach out with a bespoke plan.

Consensus stages:

Consensus is being changed and so will need to be reconfigured in the new workflow. However, the new version will be more powerful.

Webhooks:

Webhooks will now be a stage that can be added to any part of the workflow and will not just limited to being triggered when an item is in the complete stage.

Comments:

Reach out to us if you need to migrate comments. We plan to create the option to include comments in exports but, unfortunately, this will not be available during the migration period.

Both new consensus and webhook workflows can be set up before the dataset migration so that there is a transition onto the new workflow and disruption is minimized.

Extra Considerations:

The above guide applies to our publicly documented APIs. If you have reverse-engineered API requests then please reach out with a detailed description of these APIs and what you are using these for.

This can be used as an opportunity to optimize your current workflow to include new V2 features. We'll be happy to talk you through what the possibilities are.

Final Notes:

If in the unlikely event that there is an issue with the migration, the old dataset is archived rather than hard deleted. This means that we can safely rollback the changes in the case of an issue but we do not anticipate this.

If you have extensive dependencies on the V1 API, particularly those involving items and/or workflows then do reach out to our team for more bespoke migration advice. This can either be your CSM, CSE or a member of the Support team via [email protected]