Validate Annotations with Webhooks

🚧

An understanding of our webhook stage, and how to set up a Lambda function are needed for this doc.

Why is this useful

Webhooks allow you to relay information in real-time, allowing for higher efficiency and automation. It can be used as a QA step which to check for validation of annotations, or tags, and complete a certain number of steps based on the outcome of this. In this page, we will walk through an example of using the webhook stage, connected to an AWS Lambda function to validate the existence of annotations when they pass the annotation stage, and if no annotations exist then the items are moved back to the annotate stage with a comment. A further two example are outlined in the attached loom video at the end of the page.

Steps

1. In AWS, create a new Lambda function, with a layer to include the requests library. API Gateway needs to be used as the trigger, which will generate a webhook URL, which we will use in Darwin when we connect to the webhook stage. More information on webhooks can be found here.

Note: Documentation on setting up and using AWS API Gateway with a Lambda function is available here

2. Once the webhook URL is provided, we can focus on the Lambda code itself and what needs to be done within the code. We will focus on each section of the code step by step.

   1. Once the relevant libraries are imported, the API_KEY, stage_id, and workflow_id need to be defined for the particular dataset and attached workflow. These are needed for the stage API endpoint which will be discussed in more detail in part (ii). The API_KEY can be either manually entered or loaded from the environment.
      

   2. Next, we need to define a few separate functions to carry out the necessary steps. These are outlined in the image below.

      1. Firstly, we define a function which takes the dataset url and extracts the datasetid from this.
      2. Secondly, the main function to, in this instance, validate the existence of the annotations is defined. Here we extract the 'annotations' field from the json file and if this field is empty, meaning there is no annotations present, then this is appended to the initialised variable and returned from the function.
      3. The set_stage function uses the stage API endpoint to modify the stage of the item sent through the webhook. The variables defined at the beginning of the code are used here to move the files to the 'annotate' stage, within the given workflow.
      4. The final function is optional, however this is useful to give more information to the annotators. Using the 'comment_thread' endpoint, the function adds a comment to the image once it is set back to the annotate stage.

      

      

      

   3. The lambda_handler acts as the main body of the Lambda function, where the json file is loaded in, relevant fields extracted, and the above functions called on if there are found to be no annotations present.

   

3. With the code finalised, we can deploy and test the Lambda function, and finally run this through the Darwin workflow itself. When run successfully, any items which have no annotations present will be moved back to the annotate stage with a comment, while those with annotations will proceed to the review stage

Demo

Below is a tutorial of the use of webhooks for validating annotations, as shown for the example outlined above, in addition to two other example use cases also.