Kafka

The Kafka feed type in ArcGIS Velocity subscribes to and consume messages from an externally accessible Kafka broker. Kafka is an open-source distributed streaming platform that has three key capabilities:

  • Publishing and subscribing to a stream of messages via topics.
  • Storing streams of messages in a fault-tolerant way.
  • Processing streams of messages.

For more information on Kafka, see the Apache Kafka documentation.

Example

  • A data analyst wants to ingest sensor data made available over Kafka.
  • A transportation department configures a Kafka feed to consume vehicle positions and attribute information.

Usage notes

  • The Kafka broker must be externally accessible on the Internet.

After configuring feed connection properties, see Configure input data to learn how to define the schema and the key properties.

Parameters

ParameterExplanationData Type

Broker

A comma-delimited list of Kafka broker(s) specified as the server:port. For example: machine1.domain.com:9092,machine2.domain.com:9092

String

Topic

The Kafka topic from which Velocity will consume messages.

String

Authentication

The type of authentication to access the HTTP endpoint. Authentication options include None and SASL/PLAIN.

String

Username

The username for accessing the Kafka broker.

This parameter only applies when Authentication is SASL/PLAIN.

String

Password

The password for accessing the Kafka broker.

This parameter only applies when Authentication is SASL/PLAIN.

String

Considerations and limitations

  • The Kafka broker must be externally accessible on the Internet.