Azure Blob Storage

The Azure Blob Storage source type in ArcGIS Velocity reads records from files stored in Azure Blob Storage.

Examples

  • A researcher wants to load hundreds of delimited text files from Azure Blob Storage into Velocity to perform analysis on the data.
  • A GIS department stores commonly used boundary shapefiles in Azure Blob Storage and wants to load the county boundary shapefile into Velocity as an aggregation boundary.

Usage notes

  • All files identified in Azure Blob Storage by the naming pattern in the dataset property must have the same schema and geometry type.
  • The account key is encrypted the first time the analytic is saved and stored in an encrypted state.
  • When specifying the folder path, use forward slashes (//).
  • After configuring source connection properties, see Configure input data to learn how to define the schema and the key properties.

Parameters

ParameterDescriptionData type

Account key

The account access key for Azure Blob Storage.

Velocity uses the account access key to load specified data sources into the app.

The account access key is encrypted the first time an analytic is saved and stored in an encrypted state.

String

Account name

The name of the Azure Storage Account that contains Azure Blob Storage containers.

String

Endpoint suffix

The endpoint suffix used to access Azure Blob Storage. For most users this is core.windows.net.

String

Container name

The name of the Azure Blob Storage container containing the files to load.

String

Folder path

The folder path of the folder containing the files to load into Velocity. For example:

  • If you're loading files from the root level of Azure Blob Storage, enter a single forward slash (/).
  • If you're loading files from a folder in Azure Blob Storage, enter a forward slash followed by the path to the folder, for example, /gis_data_folder/folder_containing_desired_dataset.

String

Dataset

The name of the file to read if you're loading a single file, or a pattern indicating a set of files, followed by the file type extension.

To build a pattern indicating a set of files, use an asterisk (*) as a wildcard either on its own or in conjunction with a partial file name.

All files identified by the naming pattern must have the same schema and geometry type. The following are examples:

  • A single file in a folder: filename.csv
  • All files in a folder: *.shp
  • Select files in a folder: sensor_data_201*.json

String

Load recent files only

Specifies whether the Azure Blob Storage source loads all files or only the files created or modified since the last run of the analytic.

  • The default is false, which means that each time the analytic runs, all files in the specified source and path with the provided dataset name are loaded.
  • When set to true, only files modified or created are loaded at each run of the analytic.

The parameter can only be set to true for scheduled big data analytics.

For the first run of a scheduled big data analytic with the parameter set to true, big data analytics do not load any files and the analytic run will complete. Subsequent analytic runs load files with a last modified date since the last scheduled run of the analytic.

Boolean

Considerations and limitations

  • All files identified in Azure Blob Storage by the naming pattern in the dataset property must have the same schema and geometry type.
  • Ingesting JSON with an array of objects referenced by a root node is not currently supported for Amazon S3 or Azure Blob Storage.