Create a feed

This quick lesson demonstrates how to create a feed in ArcGIS Velocity.

Feeds allow you to collect and visualize real-time data. Feeds typically connect to external sources of observational data such as Internet of Things (IoT) platforms, message brokers, or third-party APIs. Feeds are also a type of stream layer and can be added to ArcGIS Online web maps and apps.

In this lesson, you will work with Marine Cadastre Automatic Identification System (AIS) data. You will create a feed to simulate historical AIS data (ship position observations) from a file.

The configuration wizard you will use for creating a feed is used throughout Velocity. You'll use the configuration wizard whenever you configure feeds or output layers in Velocity.

This lesson is designed for beginners. You must have an ArcGIS Online account with access to ArcGIS Velocity. The estimated time to complete this quick lesson is 30 minutes.

Explore options for creating feeds

There are several places in ArcGIS Velocity where you can access the configuration wizard to create a feed.

  1. In a web browser, open the ArcGIS Velocity app and sign in with your ArcGIS Online credentials.

    For the best experience, use Google Chrome or Mozilla Firefox.

    Note:

    If you encounter issues signing in, contact your ArcGIS Online administrator. You may need to be assigned an ArcGIS Online role with privileges to use ArcGIS Velocity.

  2. Click Create feed on the Home page to create a feed.

    home page

  3. Alternatively, from the main menu on the left, click Feeds under INPUT to view the Feeds page.

    Here, you can create a feed as well as view existing feeds and their statuses.

    Feeds page

Create an HTTP Simulator feed

To begin, you will create an HTTP Simulator feed type using the feed configuration wizard.

  1. On either the Home page or Feeds page, click Create feed.

    The Select a feed type window appears. In this window, you have access to groupings of the various feed types available in Velocity.

    Select a feed type window

  2. Under Web and Messaging, click See all to access the feed types for web and messaging feeds.

    Web and messaging feed types

  3. Click HTTP to access the HTTP related feed types.

    Web and messaging HTTP feed types

    You can also use the Search text box to search for the word simulator.

  4. Click HTTP Simulator to choose this feed type.

Configure the HTTP Simulator feed

The first step in configuring a new feed is to define the connection parameters to the data source. This can include defining a connection to a server with authentication credentials or browsing to a feature layer in ArcGIS Online.

For HTTP Simulator feeds, you will connect to a simulation file available at a URL. Follow the steps below to configure the feed.

  1. In the Configure Simulation step, type the connection parameters as follows:
    1. For URL, enter https://a4iot-public.s3-us-west-2.amazonaws.com/simulations/AIS_MarineCadastre_SelectedVessels.csv.
    2. For Field separator (optional), accept the default comma (,), since the file you are connecting to is comma delimited.
    3. For Features per execution (optional), type 30.

      This simulation file contains a set of observations for 30 ships that are in time stamp order. By setting this property to 30, the data is extracted 30 rows at a time, meaning there will be a new observation for every unique ship in the dataset.

    4. For Interval for sending events (milliseconds) (optional), accept the default 1000.

      This parameter controls the rate at which the specified number of rows is simulated. In this case, you will simulate 30 observations (one for each ship) every 1000 milliseconds (or 1 second).

    5. For Repeat simulation, leave the check box checked.

      When checked, the HTTP Simulator feed loops the data when the end of the simulation file is reached.

    6. For Time field index (optional), type 1.

      This parameter sets the numerical index of the date field in the dataset, where the index starts at 0. For this dataset, the time stamp is in the second field, indicating that the index is 1.

    7. For Convert to current time, leave the check box checked.

      When checked, the time values in the data are converted to the current time as it is simulated. This allows apps to treat the data as real-time data.

      Feed connection parameters for the HTTP Simulator feed type

  2. Click Next.

    Velocity tests the connection to the data source and attempts to sample the data. If the connection is successful, it then attempts to derive the schema of the data, which you will explore next.

Confirm the data schema

When configuring a feed, it is important to define the schema of the data being received. Velocity defines the schema when it samples the data including estimating the data format, field delimiter, field type, and field name parameters.

  1. In the Confirm Schema step, confirm the parameters are similar to the illustration below.

    Confirm Schema step in the feed configuration wizard

  2. Review the field types returned from the sampled data under Field Type .

    For delimited data, Velocity identifies the field type based on the sampled records. It is important to review the field types and confirm that they are correct. For example, if a field is identified as a number but you know the values in the field represent coded values, it is a best practice to change the field type to a string. In this lesson, you will accept the default field types that were returned.

  3. Review the field names returned from the sampled data under Field Name.

    Velocity identified field names when it sampled the file. Field names are populated for delimited data when a header row is present. You can rename fields now and uncheck any unnecessary fields. In this lesson, you will accept the default field names that were returned.

  4. Review the sampled values returned from the sampled data under Sampled Values.
  5. Click Next to confirm the schema of the data without making changes.

Identify the key fields

Next, you will define parameters so Velocity can properly construct geometry, date and time information, and a use a unique identifier for each ship.

  1. In the Identify Key Fields step, configure the Location parameters as follows:
    1. For Location type, choose X/Y fields.
    2. For X (longitude), choose LON.
    3. For Y (latitude), choose LAT.
    4. For Z (altitude), choose None.
    5. For Spatial reference, choose GCS WGS 1984.

    Key fields location parameters in the feed configuration wizard

  2. Define the Date and time parameters as follows:
    1. For Does your data have date fields?, choose Yes.
    2. For Start time (optional), choose BaseDateTime.
    3. For End time (optional), leave it set to None.

      This parameter defines the end time for data where time is represented as intervals. For this lesson, the ship observations have a single time stamp.

    4. For Date format, choose Other (string).
    5. For Date Formatting String, type yyyy-MM-dd HH:mm:ss.S.

      Key fields date and time parameters in the feed configuration wizard

    The date and time parameters enable Velocity to properly construct dates from the string values provided in the BaseDateTime field.

  3. For the Track ID parameter under Tracking, choose MMSI.

    Key fields tracking parameter in the feed configuration wizard

    A Track ID is a unique identifier for an observation or feature in the data. It allows you to identify multiple records or events in the feed as being associated with a particular feature. In the AIS data feed, the unique identifier is the Maritime Mobile Service Identity (MMSI), which is unique to each ship.

  4. Click Next to confirm the key fields.

Save the feed

With the parameters of the feed configured, you will now assign a title and an optional summary, and save the new feed.

  1. For Feed title, type ship_positions_simulation.

    Note:

    Feed names must be unique across an organization.

  2. For Feed summary (optional), type Simulated AIS ship positions from the Marine Cadastre Automatic Identification System.

    Leave the Also publish a feature layer which keeps all observations check box unchecked.

  3. Click Complete to create the new feed.

    Save feed step in the feed configuration wizard

  4. If this is the first item you've created a feed in Velocity, a sign in screen appears where you verify your credentials. Provide your ArcGIS credentials and click Sign In. This verification is required and enables Velocity to maintain long-running tasks for you. This is the only time when you must sign in twice.

Review feed details

Once the feed is published, the feed Details page is available. You can review the details of the feed you just created as well as edit, clone, share, delete, view the data in a map, and so on.

Feed details page

The feed starts automatically and the status button changes from Initializing to Stop, indicating that the feed has started. If the feed does not start automatically, click Start. Once the Start button text changes from Initializing to Stop, the feed is running.

To view the new feed item details in your ArcGIS organization, click Item details. You can add more metadata or information as necessary.

To immediately view the data in a web map, click Open in Map Viewer.

Ship positions in a web map

Next steps

You configured a feed in ArcGIS Velocity and are now collecting and visualizing real-time data on a web map.

Next, explore the options for performing real-time analysis on this streaming data by creating a real-time analytic with the Design a real-time analytic quick lesson.