Kafka

The Kafka output in ArcGIS Velocity can be used to send output features to a topic in an externally accessible Kafka broker. A Kafka message is sent for each output feature sent to the output.

For more information about Kafka, see Apache Kafka.

Examples

  • An analyst configures a real-time analytic to process and analyze events ingested by a feed and sends the resulting features to a Kafka topic.
  • After performing analysis in a big data analytic, a researcher wants to send the output spatial or tabular features to Kafka.

Usage notes

The Kafka broker must be externally accessible on the internet.

Parameters

ParameterDescriptionData type

Brokers

A comma-delimited list of Kafka brokers, specified as server:port—for example: machine1.domain.com:9092,machine2.domain.com:9092,machine3.domain.com:9092.

String

Topic

The Kafka topic to which the output will send messages.

String

Authentication

Specifies the type of authentication to access the HTTP endpoint. Authentication options are None, SASL/PLAIN, SASL/SCRAM-SHA-256, and SASL/SCRAM-SHA-512.

String

Use SSL

Species whether Velocity will use SSL (SASL_SSL) in attempting to connect to the Kafka broker.

Note:

This should match the configuration the broker has on a specified port. If disabled, Velocity will connect via PLAINTEXT.

Boolean

Username

Specifies the username for accessing the Kafka broker.

This parameter only applies when Authentication is set to SASL/PLAIN, SASL/SCRAM-SHA-256, or SASL/SCRAM-SHA-512.

String

Password

Specifies the password for accessing the Kafka broker.

This parameter only applies when Authentication is set to SASL/PLAIN, SASL/SCRAM-SHA-256, or SASL/SCRAM-SHA-512.

String

Output format name

The format for outgoing messages. Both delimited and JSON formats are supported.

String

Considerations and limitations

The Kafka broker must be externally accessible on the internet.