Design a real-time analytic

In this lesson, you will create a real-time analytic on streaming data in ArcGIS Velocity. This lesson builds on the Create a feed lesson in which you created a new feed to collect AIS ship data. In this lesson, you will configure a real-time analytic to enrich the streaming AIS data with the U.S. Coast Guard District where each ship is located.

You will learn how to create a real-time analytic, add and configure real-time tools, add outputs that disseminate data, and store features in a feature layer.

This lesson is designed for beginners. You must have an ArcGIS Online account with access to ArcGIS Velocity. The estimated time to complete this lesson is 30 minutes.

Create a new real-time analytic

To begin, you will create a real-time analytic on the AIS feed you created in the Create a feed lesson.

  1. In a web browser, open the ArcGIS Velocity app and sign in with your ArcGIS Online credentials.

    For the best experience, use Google Chrome or Mozilla Firefox.

    Note:

    If you encounter issues signing in, contact your ArcGIS Online administrator. You may need to be assigned an ArcGIS Online role with privileges to use ArcGIS Velocity.

  2. From the main menu on the left, click Real-Time under ANALYTICS to access the Real-Time Analytics page.

    Real-Time Analytics page

    Here, you can view and manage existing real-time analytics as well as create real-time analytics.

  3. Click Create real-time analytic to open the analytic configuration wizard.
  4. On the Create a new real-time analytic page, click Existing feed.

    Create a real-time analytic using an existing feed.

  5. In the Select existing feed window, choose the ship_positions_simulation feed and click Confirm.

    Choose an existing feed to perform real-time analytics.

    Note:

    The ship_positions_simulation feed is available if you completed the Create a feed lesson. If it is not available, complete that lesson before continuing.

    The real-time analytic editor opens, where you can configure additional tools and outputs. By default, the editor displays the chosen feed and all processing elements in a linear, sequential layout.

    Real-time analytic editor

Create the analytic

Although the analytic only contains the input feed at this point, you will create it so you can return to it later if necessary.

  1. On the New Real-Time Analytic page, click Create analytic to create the new real-time analytic.
  2. In the Create Analytic window, configure Title and Summary as follows.
    1. For Title, type AIS Real Time Processing.
    2. For Summary, type Filter and modify incoming AIS data with streaming output and feature storage.
  3. Click Create analytic to create the analytic.

    Create the analytic

    Once the analytic is created, several new options are available in the analytic editor.

    Analytic editor after creating the real-time analytic

Add and configure an analytic tool

Next, you will add analytic tools to the AIS Real Time Processing analytic that will perform real-time analysis on the AIS data feed. With Velocity, you can build a sequence of successive analytic elements in a chain from inputs to outputs.

You will use the Join Features tool to enrich the inbound AIS data with the name of the U.S. Coast Guard District where each ship is located. You will perform a spatial join between the ship features and the U.S. Coast Guard District layer.

  1. Click Toggle to model view to switch the analytic editor to the model view.

    Real-time analytic editor in model view

    The analytic editor changes from a workflow view to a model view. The model view displays a graphical canvas for configuring an analytic similar to ModelBuilder in ArcGIS Desktop or the GeoEvent Service editor in ArcGIS GeoEvent Server.

  2. Next, you will add the Join Features tool to the AIS Real Time Processing analytic and connect it to the ship_positions_simulation feed.
    1. Click the Summarize Data folder and choose Join Features to add the tool to the canvas.
    2. Click the Join Features tool and drag it to the right of the ship_positions_simulation feed.
    3. Connect the ship_positions_simulation feed to the Join Features tool.

      Join Features tool connected to the feed

    Next, you will add the source of the enrichment data—the U.S. Coast Guard District feature layer—and connect it to the Join Features tool.

  3. Click the Sources folder, choose Feature Layer, and configure the feature layer source as follows:
    1. Click the My Content drop-down menu and choose By URL.
    2. In the Layer URL text box, type https://services2.arcgis.com/FiaPA4ga0iQKduv3/arcgis/rest/services/US_Coast_Guard_Districts_new/FeatureServer.
    3. Select the USCG_Districts (0) sublayer and click Next.

      Configure the data source to the feature layer used for enrichment.

    4. In the Filter Data step, accept the defaults and click Next.
    5. In the Confirm Schema step, accept the schema as sampled and click Next.

      Velocity will sample the data and generate a starting schema.

    6. In the Identify Key Fields step, under Tracking, choose Data does not have a Track ID.

      This step of the configuration wizard is used to define the construction of geometry, date and time, and tracking information. Since the data source is an ArcGIS feature layer, the geometry and date and time information is automatically configured.

    7. Click Complete to create the data source.

    With the new feature layer source added to the canvas, you will now connect it to the other components.

  4. Drag the USCG_Districts source below the ship_positions_simulation feed and connect it to the Join Features tool.

    Feed and data source configured in a real-time analytic

    The Join Features tool now has two inputs connected to it: the ship positions feed and the U.S. Coast Guard District feature layer data source. Next, you will configure the Join Features tool to perform a spatial join and add the name of the U.S. Coast Guard Districts to the inbound event data.

  5. Double-click the Join Features tool to open its properties and configure the tool as follows:
    1. For Join operation, click One-to-one.
    2. For Retain all features, click Only retain features that are joined.
    3. For Relationship, check the Spatial check box and choose Intersects from the drop-down menu for the type of spatial relationship.

      Join Features tool with spatial relationship options configured

      In this lesson, you are detecting when any of the ships intersect a U.S. Coast Guard District and adding, or joining, that contextual information to the ship data, which can then be used for increased situational awareness or further analysis. Next, you will configure the summary fields parameters.

    4. For Attribute, select DistrictName.
    5. For Statistic, select Any.
    6. For Output field name, delete DistrictName_Any and type DistrictName.
    7. Click Add to add the join operation.
    8. Leave the Change geometry of target feature to geometry of join feature set to No.
    9. Click Apply to apply the updates.

      Join Features tool with summary fields configured

  6. In the analytic editor, click Save to save the analytic.

Configure a feature layer output

With the necessary real-time processing elements and an enrichment data source added, you will now add two outputs to store and visualize the data. You will start by adding and configuring an output to store the data in a new feature layer. In the following section, you will add another output to send the data to a stream layer for visualization in a web map.

  1. Click the Outputs folder, choose Feature Layer (new), and configure the new feature layer output as follows:
    1. For Data storage method, select Keep latest feature.

      This retains only the latest observation for each track, in this case each ship. Subsequent observations will overwrite any previous observations for each track in the output feature layer.

      Note:

      The Add new features option retains all incoming data rather than only the most recently received features.

    2. For Each time the analytic runs, select Replace existing features and schema.

      Feature layer (new) output parameters configured

      With this configuration, each time the analytic is started (or restarted), all records in the output feature layer will be deleted and the schema of the output feature layer will be regenerated. This is useful when you are developing and testing a real-time analytic and adding, removing, or changing tools between runs.

      Note:

      In a production environment, you can edit the feature layer output and select Keep existing features and schema so existing data is retained if the analytic restarts.

    3. Click Next.
    4. In the Save step, for Feature layer name, type Ships_Inside_USCG_Districts.
    5. For Feature layer summary (optional), type Ships inside US Coast Guard Districts.

      Feature layer (new) save step

    6. Click Complete to save the new feature layer output.

    The new feature layer output, Ships_Inside_USCG_Districts, is added to the analytic editor.

  2. Connect the Join Features tool to the new Ships_Inside_USCG_Districts feature layer output.

    Join Features tool connected to the feature layer output

  3. Click Save to save the real-time analytic.

Configure a stream layer output

With the new feature layer output now created, you will now add a second output to send the features to a stream layer that you will visualized in a web map.

  1. Click the Outputs folder, choose Stream Layer, and configure the stream layer output as follows:
    1. Click Next to accept the defaults for the Stream Layer Configuration step.

      The Also publish a feature layer which keeps the latest observation for each Track ID and Select a related feature layer to provide geometry when stream layer is drawn in a map options are not required. When you publish a feature layer, you can publish a separate feature layer that contains only the latest observations. This is useful when you want to symbolize the current position of an asset, in this case a ship, differently in a web map. You can choose a related feature layer that can be used to enrich the output stream layer with additional fields.

    2. For Stream layer name, type Ships_Inside_USCG_Districts_stream.
    3. For Stream layer summary (optional), type Ships inside US Coast Guard Districts stream.

      Stream layer output configuration

  2. Click Complete to create the stream layer output.

    The new stream layer output, Ships_Inside_USCG_Districts_stream, is added to the analytic editor.

  3. Connect the Join Features tool to the Ships_Inside_USCG_Districts_stream stream layer output.

    Join Features tool connected to the new stream layer output

  4. Click Save to save the real-time analytic.

Start the real-time analytic

The real-time analytic now includes the necessary feeds, tool, and outputs and is ready to be started. When started, or running, the analytic will receive the simulated ship data, join to it the name of the U.S. Coast Guard District where each ship is currently located, and write the event data to the feature layer and stream layer outputs.

  1. In the upper right of the analytic editor, click Start.

    As with feeds, real-time analytics are long-running tasks and continue to run until they are stopped.

    The Start button text changes to Stop Initialization and then to Stop. This indicates that the real-time analytic is now running.

  2. Remember, the status of an analytic can be monitored from the Real-Time Analytics page in Velocity.

View the real-time analytic metrics

Metrics regarding running real-time analytics can be viewed in both the workflow and model views in Velocity. Metrics provide the average rate of events through each element and represent an average of the rate for each element for the past five minutes of run time. Additionally, while a real-time analytic is running, the metrics represent the compute utilization of the analytic. For details, see Real-time analytic metrics.

  1. In the AIS Real Time Processing real-time analytic, turn on the Metrics toggle button to view metrics for each element in the analytic.

    Turn on metrics for the real-time analytic

  2. Review the metrics for each element in the AIS Real Time Processing real-time analytic. Notice the events per second that each is processing, the total number of observations in and out, and the compute utilization.

    Metrics for the real-time analytic

Examine the output layer

With the real-time analytic now running and the feature and stream layer outputs available, you will add these output layers to a web map directly from the analytic.

  1. From the main menu, click Feeds under INPUT to access the Feeds page.
  2. Click Start to start the ship_positions_simulation feed if it is not already started.
  3. From the main menu, click Real-Time under ANALYTICS to access the Real-Time Analytics page.
  4. Click Start to start the AIS Real Time Processing real-time analytic if it is not already started.
  5. Once the AIS Real Time Processing real-time analytic is running, click the pencil icon to edit the analytic.
  6. Right-click the Ships_Inside_USCG_Districts feature layer output and select Open in Map Viewer.

    Open feature layer in the Map Viewer

    A new browser tab opens to Map Viewer in ArcGIS Online.

  7. Change the basemap to Dark Gray Canvas.

    Feature layer in web map

  8. Optionally, click the Add button and add the U.S. Coast Guard Districts feature layer using the URL https://services2.arcgis.com/FiaPA4ga0iQKduv3/arcgis/rest/services/US_Coast_Guard_Districts_new/FeatureServer.

    Feature layer in web map with U.S. Coast Guard Districts boundaries

  9. Click one of the ship position features to open the pop-up and explore its attributes. Each observation has been enriched with the name of the U.S. Coast Guard District (DistrictName) where the ship is currently located.

    Pop-up displaying information on selected ship

Next steps

You created a real-time analytic using ArcGIS Velocity and viewed the output features in a web map. Then you added the Join Features tool to perform spatial enrichment and associate contextual information with incoming observation data.

Explore other content as you continue working with Velocity including the Design a big data analytic quick lesson and if interested in using Arcade expressions, explore the Arcade expressions topic.