Azure Blob storage

Read records from files stored in Azure Blob storage and perform analysis in ArcGIS Velocity.

Examples

  • A researcher wants to load hundreds of delimited text files stored in Azure Blob storage and perform analysis on the data in Velocity.
  • A GIS department stores commonly used boundary shapefiles in Azure Blob storage and wants to load the county boundary shapefile into Velocity as an aggregation boundary.

Usage notes

  • All files identified in Azure Blob storage by the naming pattern in the dataset property must have the same schema and geometry type.
  • The account key will be encrypted the first time the analytic is saved and stored in an encrypted state.
  • When specifying the folder path, use forward slashes (/).
  • After configuring source connection properties, see Configure input data to learn how to define the schema and the key properties.

Parameters

ParameterExplanationData Type

Account key

The account access key to access Azure Blob storage.

Velocity will use the provided account access key to load specified data sources into the application.

The account access key will be encrypted the first time an analytic is saved and stored in an encrypted state.

String

Account name

The name of the Azure Storage Account which contains Azure Blob storage containers.

String

Endpoint suffix

The endpoint suffix used to access Azure Blob storage. For most users this will be: core.windows.net

String

Container name

The name of the Azure Blob storage container containing the files to load.

String

Folder path

The folder path of the folder containing the file(s) to be loaded into Velocity. For example:

  • If loading file(s) from the root level of Azure Blob storage, enter a single forward slash:
    • /
  • If loading file(s) from a folder in Azure Blob storage, enter a forward slash followed by the path to the folder:
    • /gis_data_folder/folder_containing_desired_dataset

String

Dataset

The name of the file to read if loading a single file, or a pattern indicating a set of files, followed by the file type extension.

To build a pattern indicating a set of files, use an asterisk (*) as a wildcard either on its own or in conjunction with a partial file name.

All files identified by the naming pattern must have the same schema and geometry type.

  • If loading a single file in a folder:
    • filename.csv
  • If loading all files in a folder:
    • *.shp
  • If loading select files in a folder:
    • sensor_data_201*.json

String

Load recent files only

Specifies whether the Azure Blob store source should load all files or only the files created or modified since the last run of the analytic.

  • By default, the parameter is set to false, meaning each time the analytic runs, all files in the specified bucket and path with the provided dataset name will be loaded.
  • When set to true, only files modified or created will be loaded by each run of the analytic.

The parameter can only be set to true for scheduled big data analytics.

For the first run of a scheduled big data analytic with the parameter set to true, big data analytics will not load any files and the analytic run will complete. Subsequent analytic runs will load files with a last modified date since the last scheduled run of the analytic.

Boolean

Considerations and limitations

  • All files identified in Azure Blob storage by the naming pattern in the dataset property must have the same schema and geometry type.
  • Ingesting JSON with an array of objects referenced by a root node is not currently supported for Amazon S3 or Azure Blob storage.