Best practices for organization administrators

Plan for the capacity of your Analytics for IoT subscription

Your Analytics for IoT subscription comes with compute and storage capacity to ingest real-time observations, detect incidents of interest, store observations historically, and analyze historical data for patterns over time. It is important however to consider your organization's target use cases and evaluate which can be accommodated within your subscription plan.

The number of use cases that can be supported depend in many ways on the data velocity and the volume at which that data accumulates over time. The velocity of real-time data streams varies due to factors such as the type of data, the number of assets being tracked, the number of sensors deployed, and the frequency at which new observations are reported. Storm fronts, for example, are of interest to many organizations from the perspective of protecting critical infrastructure, but the number of storms active at any given time is low (depending on the study area) and updates do not typically come in at high velocities. On the other hand, a logistics or shipping company many have thousands of vehicles that report their position every few seconds.

The use cases applicable to Analytics for IoT vary by industry. For example, when putting Analytics for IoT into practice, transportation organizations may be interested in use cases like the below:

  • Fleet (vehicle) tracking
  • Equipment tracking
  • Anomaly detection (such as stalled cars on the roadway)
  • Digital messaging (such as alerts on weather events)
  • Work zone intrusion alerting
  • Connected car tracking

Many transportation organizations do not have more than a few hundred work vehicles or pieces of equipment to track, resulting in many of the above examples being lower velocity use cases. The Analytics for IoT subscription could for example simultaneously accommodate 3 to 4 of these use cases, other than connected cars. Connected car tracking typically involves high velocity data streams, and so would consume more capacity (as an individual use case) than the other examples.

The capacity of the Analytics for IoT subscription can be augmented with additional IoT Compute or Storage Units. Organization administrators are encouraged to work with their Esri account representatives to plan the right amount of capacity for their intended workflows.

Consider which users need real-time privileges

Analytics for IoT enables users to create feeds, real-time analytics, and big data analytics to work with IoT observations. Both feeds and real-time analytics are continuous or real-time tasks, meaning they are always running and consuming capacity within your Analytics for IoT subscription. You may wish to consider the key feeds and real-time analytics needed for your organizational workflows, and limit the privileges for these items to users who will manage those processes. For more information, see Create roles and assign users.

A common pattern is running a defined set of feeds and associated real-time analytics which process and store the incoming data in a feature layer. A broader set of users could then run ad-hoc big data analytics against the feature layer to answer different mission-related questions.

Encourage users to proactively manage their real-time items

As both feeds and real-time analytics are tasks that are always running and consuming capacity, it is important to proactively manage these items. Encourage users to stop feeds or real-time analytics that are not needed, or that have been set up largely for testing and development.

Review the actively running real-time items

On a periodic basis, it is recommended to review the real-time tasks being published with Analytics for IoT in case there are excess tasks running which are not needed. On any of the item list pages, simply choose to view Organization Content instead of My Content. When you view Organization Content, you can inspect certain details of user items such as a feed's schema or item logs and you can stop any running tasks. This allows you to free up compute capacity if needed. For more information, see Manage and monitor usage.

Apply shorter data retention time periods

When creating output feature layers, your users can apply data retention policies that range from 1 hour to 1 year. The best practice is to consider carefully the actual time period for which your data is relevant to your day-to-day workflows versus occasional analysis workflows. The time period for which you need data available for immediate exploration and visualization should be set as the data retention policy. If you need older data for occasional analysis, you can choose to export the data to the archive (cold store) prior to it being purged from the feature layer.

Applying shorter data retention policies for datasets that grow in real-time maximizes the remaining feature storage that will be available for analytical results. For more information, see About data retention.