Kafka

The Kafka feed type in ArcGIS Velocity subscribes to and consumes messages from an externally accessible Kafka broker. Kafka is an open-source distributed streaming platform that you can use to do the following:

  • Publish and subscribe to a stream of messages through topics.
  • Reliably store streams of messages.
  • Process streams of messages.

For more information about Kafka, see the Apache Kafka documentation.

Examples

  • A data analyst wants to ingest sensor data through a Kafka broker.
  • A transportation department configures a Kafka feed to consume vehicle positions and attribute information.

Usage notes

  • The Kafka broker must be externally accessible on the internet.
  • This feed type supports connectivity to Confluent Kafka.

After configuring feed connection parameters, see Configure input data to learn how to define the schema and the key parameters.

Parameters

ParameterDescriptionData type

Broker

A comma-delimited list of Kafka brokers specified as server:port, for example, machine1.domain.com:9092,machine2.domain.com:9092.

String

Topic

Specifies the Kafka topic from which Velocity will consume messages.

String

Consumer group ID

(optional)

Specifies the Kafka consumer group Velocity will join as a member.

String

Authentication

Specifies the type of authentication to access the HTTP endpoint. Authentication options are None and SASL/PLAIN.

String

Username

Specifies the user name for accessing the Kafka broker.

This parameter only applies when Authentication is set to SASL/PLAIN.

String

Password

Specifies the password for accessing the Kafka broker.

This parameter only applies when Authentication is set to SASL/PLAIN.

String

Considerations and limitations

The Kafka broker must be externally accessible on the internet.