Create a feed

This lesson will demonstrate how to create a feed in ArcGIS Velocity.

Feeds allow you rapidly ingest and visualize real-time data. Feeds typically connect to external sources of observational data such as Internet of Things (IoT) platforms, message brokers, or third-party API's. Feeds are also a type of stream layer and can be easily added to ArcGIS Online web maps and other applications.

In this quick lesson, you will be working with Marine Cadastre Automatic Identification System (AIS) data. You will create a feed to simulate historical AIS data (ship position observations) from a file.

The configuration wizard you will use for creating a feed is a common one throughout the Velocity application. It is good to familiarize yourself with the process as you will be working with the configuration wizard whenever you are configuring input or output data components.

This lesson is designed for beginners. You must have an ArcGIS Online account with access to ArcGIS Velocity. Estimated time: 30 minutes.

Explore options for creating feeds

There are several places in the ArcGIS Velocity application you can access the configuration wizard to begin creating a feed.

  1. In a web browser, navigate to https://velocity.arcgis.com and sign in with your ArcGIS Online credentials.

    For the best experience, use Google Chrome or Mozilla Firefox.

    Note:

    If you encounter issues signing in, contact your ArcGIS Online administrator. You may need to be assigned to an ArcGIS Online role with privileges to use ArcGIS Velocity.

  2. Once you're signed in, you can create a new feed by clicking Create feed from the Home page.

    application home page

  3. In addition, by clicking Feeds on the menu on the left, you access the Feeds page where you can view existing feeds and their status as well as create new feeds.

    Feeds page in the

    In the next section, you will create and configure a new feed.

Create a new feed

To begin, you will create a new Simulator feed type using the feed configuration wizard.

  1. From either the Home page or Feeds page you explored above, click Create feed.
  2. In the Select a feed type window, you have access to groupings of the various feed types you can create in Velocity.

    Feed configuration wizard in the

  3. Click See all under Web and Messaging to access the feed types for web and messaging feeds. Similarly, you can search for simulator in the search.

    Web and messaging feed types

  4. Click HTTP to access the HTTP related feed types.

    Web and messaging HTTP feed types

  5. Click HTTP Simulator to choose this feed type.

Configure the HTTP Simulator feed

The first step in configuring a new feed is to define the connection properties to the data source. This can be, for example, defining a connection to a server with authentication credentials or simply browsing to a feature layer in ArcGIS Online.

For HTTP Simulator feeds, you will connect to a simulation file available at a URL.

  1. In the Configure Simulation step, enter the connection properties as follows:
    1. For URL, enter the following URL:

      https://a4iot-public.s3-us-west-2.amazonaws.com/simulations/AIS_MarineCadastre_SelectedVessels.csv

    2. For Field separator (optional), accept the default comma (,), since this file you are connecting to is comma-delimited.
    3. For Features per execution (optional), enter: 30

      This simulation file contains a set of observations for 30 ships, ordered by timestamp. By setting this property to 30, the data will be extracted 30 rows at a time, meaning there will be a new observation for every unique ship in the dataset.

    4. For Interval for sending events (milliseconds) (optional), accept the default 1000

      This parameter controls the rate at which the specified number of rows is simulated. In this case, you will simulate 30 observations (one for each ship) every 1000 milliseconds (or 1 second).

    5. For Repeat simulation, leave the checkbox checked.

      When checked, the HTTP Simulator feed will loop the data when the end of the simulation file is reached.

    6. For Time field index (optional), enter: 1

      This parameter sets the numerical index of the date field in the dataset, where the index starts at 0. For this dataset, the timestamp is in the second field, which means the index is 1.

    7. For Convert to current time, leave the checkbox checked.

      When checked, the time values in the data are converted to the current time as it is simulated. This allows applications to treat the data as current real-time data.

      Feed connection properties for the HTTP Simulator feed type

  2. Click Next to proceed to the next step.

    Velocity will test the connection to the data source and attempt to sample the data. If the connection is successful, it will then attempt to derive the schema of the data, which you will explore next.

Confirm the data schema

When configuring a feed, it is important to define the schema of the data you are receiving. Velocity makes a best attempt at defining that schema when it samples the data, estimating the data format, field delimiter, field type, and field name properties.

  1. In the Confirm Schema step, confirm the parameters are configured as follows:

    Confirm schema in the feed configuration wizard

  2. Explore the field types returned from the sampled data records.

    For delimited data, Velocity makes an estimation of the field type based on the sampled records. It is important to review the field types and confirm they are correct. For example, if a field is derived to be a number, but you know the values in the field represent coded values, it is a best practice to change the field type to a string. In this lesson, you will accept the default field types that were returned.

  3. Explore the field name values returned.

    Velocity has derived field names when it sampled the file. Field names will be populated for delimited data when a header row appears to be present. You can rename fields now as well as uncheck any unnecessary fields. In this lesson, you will accept the default field names that were returned.

  4. Click Next to confirm the schema of the data.

Identify the key fields

Next, you will configure some important fields so Velocity can properly construct geometry, date information, and a unique identifier for each ship.

  1. In the Identify Key Fields step, configure the Location parameters as follows:
    1. For Location type, choose X/Y Fields.
    2. For X (longitude), choose the LON field.
    3. For Y (latitude), choose the LAT field.
    4. For Spatial reference, choose GCS WGS 1984.

    Key fields location properties in the feed configuration wizard

  2. Define the Date and time parameters as follows:
    1. For Does your data have date fields?, choose Yes.
    2. For Start time (optional), choose BaseDateTime.
    3. For End time (optional), leave it set to None.

      This parameter defines the end time for data where time is represented as intervals. For this lesson, the ship observations only have a single timestamp.

    4. For Date format, choose Other (string).
    5. For Date formatting string, enter the following text:

      yyyy-MM-dd HH:mm:ss.S

      Key fields date and time properties in the feed configuration wizard

    The date and time properties allow Velocity to properly construct dates from the string values provided in the BaseDateTime field.

  3. For Tracking, choose MMSI for the Track ID.

    Key fields tracking properties in the feed configuration wizard

    A Track ID is a unique identifier for an observation or feature in the data. It allows you to identify multiple records or events in the feed as being associated with a particular feature. In the AIS data feed, the unique identifier is the Maritime Mobile Service Identity, or MMSI, which is unique to each ship.

  4. Click Next to confirm the key fields.

Save the feed

With the parameters of the feed configured, you will now assign a name, an optional summary, and save the new feed.

  1. For Feed name, enter the following text:

    ship_positions_simulation

    Note:

    Feed names must be unique across an organization.

  2. For Feed summary (optional), enter the following text:

    Simulated AIS ship positions from the Marine Cadastre Automatic Identification System

  3. Click Complete to save the new feed.

    Save feed in the feed configuration wizard

  4. If this is the first item you are creating in Velocity, when you click Complete, a login screen will appear to verify your credentials. Re-enter your ArcGIS credentials and click Sign In. This verification is required and enables Velocity to maintain long-running tasks on your behalf. You should only have to login a second time once.

Review feed details

Once the feed is published, the feed Details page loads. Here you can review the details of the feed you just created as well as edit, clone, share, delete, view the data in a map, and more.

Feed details page in the

The feed should start automatically and the status button will change from Initializing to Stop, indicating the feed has started. If the feed does not start automatically, click Start to start it. Once the Start button changes from Initializing to Stop, the feed is running.

To view the new feed item in the ArcGIS organization, click Item details. Here you can add any additional metadata or information that might be necessary.

You can also immediately view your data in a web map by clicking Open in Map Viewer.

Ship positions in a web map

Next steps

Congratulations! You have successfully configured a feed in ArcGIS Velocity and are now ingesting and visualizing real-time data in a web map.

Next, explore the options for performing real-time analysis on this streaming data by creating a real-time analytic. For details, see the Design a real-time analytic quick lesson.