Kafka

The Kafka output in ArcGIS Velocity can be used to send output features to a topic in an externally accessible Kafka broker. A Kafka message is sent for each output feature sent to the output.

Learn more about Apache Kafka

Examples

The following are example use cases for the output:

• An analyst configures a real-time analytic to process and analyze events ingested by a feed and sends the resulting features to a Kafka topic.
• A financial analyst sends real-time stock trade updates to a Kafka topic, allowing instant data access.
• After performing analysis in a big data analytic, a researcher wants to send the output spatial or tabular features to Kafka.

Usage notes

Keep the following in mind when working with the output:

• The Kafka broker must be externally accessible on the internet.
• The following output data formats are available:
  • Delimited
  • JSON
  • GeoJSON
  • EsriJSON

Parameters

The following are the parameters for the output:

Parameter	Description	Data type
Brokers	A comma-delimited list of Kafka brokers, specified as server:port—for example, machine1.domain.com:9092,machine2.domain.com:9092,machine3.domain.com:9092.	String
Topic	The Kafka topic the output sends messages to.	String
Authentication	Specifies the type of authentication to access the HTTP endpoint. The following are the authentication options: None SASL/PLAIN SASL/SCRAM-SHA-256 SASL/SCRAM-SHA-512	String
Use SSL	Specifies whether Velocity uses SSL (SASL_SSL) in attempting to connect to the Kafka broker. Note: This should match the configuration the broker has on a specified port. If you this turn parameter off, Velocity connects with PLAINTEXT.	Boolean
Username	Specifies the username for accessing the Kafka broker. This parameter only applies when Authentication is set to SASL/PLAIN, SASL/SCRAM-SHA-256, or SASL/SCRAM-SHA-512.	String
Password	Specifies the password for accessing the Kafka broker. This parameter only applies when Authentication is set to SASL/PLAIN, SASL/SCRAM-SHA-256, or SASL/SCRAM-SHA-512.	String
Output format name	The format for outgoing messages. The following are the supported output format names: GeoJSON EsriJSON Delimited JSON	String
Write header rows	Specifies if the files write out the header row with field names. The default is false.	Boolean
Field delimiter	The field delimiter used by the file to separate fields. The default is a comma.	String