In ArcGIS Velocity, both real-time and big data analytics process or analyze records. The results from these analytics can be handled in a variety of ways by disseminating data to one or more outputs.
An output is a required component within an analytic. Outputs perform different actions including storing features, sending features to a stream layer, issuing an alert or notification, or actuating IoT device behavior through a cloud provider IoT hub.
Choose an output format
When writing to certain output types, you can choose the desired format for output features. Options can include delimited text, JSON, EsriJSON, GeoJSON, shapefile, or Parquet. Below are the output types that allow you to choose the desired format for output features:
Delimited | JSON | EsriJSON | GeoJSON | Shapefile | Parquet | |
---|---|---|---|---|---|---|
Amazon S3 | ||||||
Amazon SNS | ||||||
Amazon SQS | ||||||
Azure Blob Store | ||||||
Azure Event Hub | ||||||
Azure IoT Hub | ||||||
Kafka | ||||||
RabbitMQ |
Differences between real-time and big data analytic outputs
Real-time analytics emit each message to an output individually, as it is processed. This is because real-time analytics perform processing on each individual message as it is received. Consider, for example, a Feature Layer (new) output configured in a real-time analytic. Each time an event is ingested, processed, and sent to the output, a new feature is either added or used to update an existing feature in the feature layer.
Big data analytics emit messages to analytic outputs once processing for the entire analytic is complete. This is because big data analytics perform batch analysis and processing on stored data. Consider, for example, a Feature Layer (new) output configured in a big data analytic. Once processing is complete, typically all records are added or used to update existing features in the feature layer at once.
Implications of output rate and quantity
When emitting data from an analytic, it is important to consider the quantity and rate of data being sent to an output. Some output types are better suited to high velocity and high volumes of features than others based on their inherent function.
For example, a feed that is ingesting events at an average rate of 100 events per second and used in a real-time analytic to send email alerts might emit 100 emails every second. This could overload the capacity of your specified email server. The best practice is to use the Email output for incidents that are expected to occur infrequently.