AWS S3 configuration

Suggest Edits

Using V7's external AWS integration, you can keep your data stored within a private Amazon S3 bucket. Check out the diagram here to see how it works, and if you're ready to get started follow our step-by-step instructions to create the integration.

🚧

The AWS integration is available on V7's Business and Enterprise plans. You can find out more about what each plan includes on our pricing page.

Read / Write access

To setup an external s3 account we first need to give our AWS role (arn:aws:iam::258327614892:role/external_s3 ) access:

  • Read via GetObject
  • Write via PutObject (optional)
{
	"Version": "2012-10-17",
	"Id": "PolicyForExternalAccess",
	"Statement": [
    {
      "Sid": "DarwinAccess",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::258327614892:role/external_s3"
      },
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::your-s3-bucket-name/*"
    }
  ]
}

If you don't need Darwin to process images after they are uploaded (e.g. generate thumbnails, split video frames etc), then you can leave out the Write access "s3:PutObject"

{
	"Version": "2012-10-17",
	"Id": "PolicyForExternalAccess",
	"Statement": [
    {
      "Sid": "DarwinAccess",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::258327614892:role/external_s3"
      },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::your-s3-bucket-name/*"
    }
  ]
}

If you already have a policy for your bucket, then you only need to add the Statement part.

📘

AWS policy type

Please make sure that the above mentioned policies are added directly to your S3 bucket (resource-based policy) not via IAM (role-based policy). In case of doubt we recommend following this AWS guide.

🚧

Required permissions for MIRAX file format

If you plan to register MIRAX files (.mrxs), you will need to also add s3:ListBucket permission to policy outlined above. This is due to fact that .mrxscome with a folder next to them with matching name, which contains additional image data across multiple files - Darwin platform needs to be able to list all those files to pull them correctly before processing.

CORS access

When annotators are requesting images to annotate, they will load them directly from your s3 bucket via a presigned url. However since that s3 bucket sits on a different domain than darwin.v7labs.com a CORS header needs to be configured.

You can find this under Permissions > CORS Configuration in the AWS S3 UI:

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "https://darwin.v7labs.com"
        ],
        "ExposeHeaders": []
    }
]

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
	<CORSRule>
		<AllowedOrigin>https://darwin.v7labs.com</AllowedOrigin>
		<AllowedMethod>GET</AllowedMethod>
	</CORSRule>
</CORSConfiguration>

Activation

When this is all setup, please message [email protected] with the following details:

  • S3 region
  • S3 bucket name
  • an optional prefix where we can upload thumbnails if needed (often /darwin/ )
  • your team name
    And we will turn on the external access for your team.

📘

Additional Storage Integrations

If your subscription includes additional storage integrations then these can be added by going to your 'Settings' -> 'Storage' and adding the details above to a 'New Storage Integration'.

These can be added without speaking to our Support team although we encourage you to speak with us if you have any questions.

🚧

KMS Encryption

If you are using KMS encryption then it is important that you give V7 permission to decrypt the relevant items. Otherwise, this will result in us being unable to render these items for annotation.

It's worth noting that you may need to review KMS permissions for objects as they retain the old permissions.

Note: V7 cannot support buckets where the KMS encryption is done with an AWS-managed key. In this case, please move your data to a new bucket and use a customer-managed key for KMS encryption.

You can see more details about S3 server side encryption in the AWS docs here.~

This doc on allowing users in other accounts to use KMS keys should also be useful.

❗️

S3 bucket name

Bucket name containing dots (.) will not work due to how AWS handles virtual-host-style HTTPS. Read more about it here https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html

📘

FAQs

For frequently asked questions on AWS S3 configurations, please refer to our FAQ page.

If you encounter any issues or have any questions feel free to contact us at [email protected]

Updated about 1 month ago

Next up