Uploading large files

In occasions, file sizes can be prohibitively large for uploading using the REST interface, as some services have file limits of 2GB or 4GB for a single file size.

In these cases, there are alternatives to consider:

If the files have not been compressed, consider placing them into a ZIP archive before uploading. The processor system will unpack the files.
If the entity type allows, consider splitting the spatial data into several files that under the required limit.
Bypass the REST upload altogether, and upload directly to the storage system, for instance AWS S3 or Azure Blob Storage.

The following sections cover how to upload data directly, depending on the storage system in use.

Uploading data directly to Azure Blob Storage

In order to upload data directly into Azure Blob storage, it is necessary to know how storage keys are composed in the system.

The keys in Azure are composed using the following syntax:

Where:

OrganizationID - Set to 2343243456678890

ProjectId - The ID of the project that contains the data entity

EntityId - The ID of the entity that will contain the file

FileName - Name for the file

Here is one particular examples of such a key:

2343243456678890@03684CA0F35947EC8243E52972CA3E70@3AC3A444AF414041A0668032536697AD@myblockmodel.zip

This key/blob pair should be created in the Blob Container that is used by the deployment.

Uploading data directly to AWS S3

In order to upload data directly into AWS S3, it is necessary to know how storage keys are composed in the system.