Work with input data

Configure input dataIngest real-time dataIngest historical dataLocation parametersDate and time parametersFeed polling interval schedulingFeed to history feature layer

Feeds

Work with feedsFeed details and statusShare feedsAWS IoTAzure Event HubAzure Service BusCisco Edge IntelligenceGeotab (DeviceStatusInfo)Geotab (LogRecord)gRPCGTFSFeature LayerHTTP PollerHTTP ReceiverHTTP SimulatorKafkaMQTTRabbitMQRSSStream LayerVerizon Connect RevealWebSocket

Data sources

Work with data sourcesAmazon S3Azure Blob StorageAzure Cosmos DBFeature LayerFeature Layer (archive)HTTP PollerRSSStandard geographies

Data formats

Supported data formatsDelimitedJSONEsri JSONGeoJSONXML

Troubleshoot

Common ingestion issues and solutions
Back to Top

Azure Blob Storage

In this topic

  1. Examples
  2. Usage notes
  3. Parameters
  4. Considerations and limitations

The Azure Blob Storage source type in ArcGIS Velocity reads records from files stored in Azure Blob Storage.

Examples

The following are example uses of the Azure Blob Storage source:

  • A researcher wants to load hundreds of delimited text files from Azure Blob Storage into Velocity to perform analysis on the data.
  • A GIS department stores commonly used boundary shapefiles in Azure Blob Storage and wants to load the county boundary shapefile into Velocity as an aggregation boundary.

Usage notes

Keep the following in mind when working with the Azure Blob Storage source:

  • All files identified in Azure Blob Storage by the naming pattern in the Dataset parameter must have the same schema and geometry type. If specifying a folder name for the Dataset parameter, all files in the directories must have the same file type and schema.
  • The account key is encrypted the first time the analytic is saved and stored in an encrypted state.
  • When specifying the folder path, use forward slashes (//).
  • After configuring source connection parameters, see Configure input data to learn how to define the schema and the key parameters.

Parameters

ParameterDescriptionData type

Account key

The account access key for Azure Blob Storage.

Velocity uses the account access key to load specified data sources into the app.

The account access key is encrypted the first time an analytic is saved and stored in an encrypted state.

String

Account name

The name of the Azure Storage Account that contains Azure Blob Storage containers.

String

Endpoint suffix

The endpoint suffix used to access Azure Blob Storage. For most users, this will be the following: core.windows.net.

String

Container name

The name of the Azure Blob Storage container containing the files to load.

String

Folder path

The folder path of the folder containing the files to load into Velocity. The following are examples:

  • If you're loading files from the root level of Azure Blob Storage, enter a single forward slash (/).
  • If you're loading files from a folder in Azure Blob Storage, enter a forward slash followed by the path to the folder, for example, /gis_data_folder/folder_containing_desired_dataset.

String

Dataset

The name of the file to read if you are loading a single file, or a pattern indicating a set of files, followed by the file type extension.

To build a pattern indicating a set of files, use an asterisk (*) as a wildcard either on its own or in conjunction with a partial file name.

All files identified by the naming pattern must have the same schema and geometry type.

Alternatively, if loading multiple files and/or nested folders, you can also specify the containing folder name as the dataset name instead of a file name with extension. If specifying a containing folder name as the dataset, you cannot use wildcards or restrict file types. All files from the specified folder will be ingested and they should all have the same file type.

The following are examples:

  • A single file in a folder—filename.csv
  • All files in a folder—*.shp
  • Select files in a folder—sensor_data_201*.json
  • All files from a directory or a directory of directories (subdirectories)—containingFolderName

String

Load recent files only

Specifies whether the Azure Blob Storage source loads all files or only the files created or modified since the last run of the analytic.

  • The default is false, which means that each time the analytic runs, all files in the specified source and path with the provided dataset name are loaded.
  • When set to true, only files modified or created are loaded at each run of the analytic.

The parameter can only be set to true for scheduled big data analytics.

For the first run of a scheduled big data analytic with the parameter set to true, big data analytics do not load any files and the analytic run will complete. Subsequent analytic runs load files with a last modified date since the last scheduled run of the analytic.

Boolean

Considerations and limitations

There are several considerations and limitations to keep in mind when using the Azure Blob Storage source:

  • All files identified in Azure Blob Storage by the naming pattern in the dataset property must have the same schema and geometry type.
  • Ingesting JSON with an array of objects referenced by a root node is not currently supported for Amazon S3 or Azure Blob Storage.

Feedback on this topic?

In this topic
  1. Examples
  2. Usage notes
  3. Parameters
  4. Considerations and limitations