Kafka Inputs

George Alpizar
George Alpizar
  • Updated

Overview

This input type allows Edge Delta agent will collect events from a Kafka topic.

Review Sample Configuration

Review the following sample configuration:

  kafkas:
    - labels: "errorcheck"
      endpoint: "something"
      topic: "topic"
    - labels: "errorcheck-tls"
      endpoint: "something-tls"
      topic: "topic"
      tls:
        disable_verify: true
        ca_file: /var/etc/kafka/ca_file
        ca_path: /var/etc/kafka
        crt_file: /var/etc/kafka/crt_file
        key_file: /var/etc/kafka/keyfile
        key_password: p@ssword123
        client_auth_type: noclientcert 
        min_version: TLSv1_1
        max_version: TLSv1_3
    - labels: "my-kafka-events"
      endpoint: "something"
      topic: "topic"
      group_id: "my-group"
      sasl:
        username: kafka_username
        password: p@ssword123
        mechanism: PLAIN 
        optional: false
        field_mappings:
          namespace: "kubernetes.namespace"
          serviceName: "service"
          roleName: "user.role"
          systemType: "system"

Review Parameters

Review the following parameters that you can configure in the Edge Delta App. 

labels

Required

Enter a descriptive name for this input. 

When you create a workflow, you will use this label to enter your input into the workflow. 

Review the following example:

- labels: "my-kafka-events"

endpoint

Required

Enter your Kafka broker address.

Review the following example:

endpoint: "something-tls"

topic

Required

Enter your Kafka topic name.

Review the following example:

topic: "topic"

group_id

Required

Enter a consumer group within the specified topic name. 

Review the following example:

group_id: "my-group"

filters

Optional 

Enter an existing filter to add to this input. 

To learn more, see Filters.

Review the following example: 

filters: 
   - info

sasl: username

Optional 

Enter your Kafka SASL username.

Review the following example:

sasl:
  username: kafka_username

sasl: password

Optional 

Enter your Kafka SASL password.

Review the following example:

sasl:
  password: p@ssword123

sasl: mechanism

Optional 

Enter a Kafka SASL mechanism type to implement a secure authentication.

You can enter: 

  • PLAIN
  • SCRAM-SHA-256
  • SCRAM-SHA-512

Review the following example:

sasl:
  mechanism: PLAIN

tls: disable_verify

Optional 

To disable a TLS verification of a certificate, enter disable_verify: true.

To enable a TLS verification of the certificate, you can enter disable_verify: false or you can remove this line entirely. 

Review the following example: 

tls:
  disable_verify: true

tls: ca_path

Optional 

Enter the absolute file path to the CA certificate file.

Review the following example: 

tls:
  ca_path: /var/etc/kafka

tls: ca_file

Optional 

Enter the absolute path to scan the CA certificate file.

Review the following example: 

tls:
  ca_file: /certs/ca.pem

tls: crt_path

Optional 

Enter the absolute path to the certificate file. 

Review the following example: 

tls:
  crt_file: /certs/server-cert.pem

tls: key_file

Optional 

Enter the absolute path to the private key file. 

Review the following example: 

tls:
  key_file: /certs/server-key.pem

tls: key_password

Optional 

Enter the password for the key file.

Review the following example: 

tls:
  key_password: p@ssword123

tls: client_auth_type

Optional 

Enter a client authorization type. 

You can enter:

  • noclientcert
  • requestclientcert
  • requireanyclientcert
  • verifyclientcertifgiven
  • requireandverifyclientcert

The default setting is noclientcert.

Review the following example: 

tls:
  client_auth_type: noclientcert

tls: min_version

Optional 

Enter the minimum version of TLS to accept. 

Review the following example: 

tls:
  min_version: TLSv1_1

tls: max_version

Optional

Enter the maximum version of TLS to accept. 

Review the following example: 

tls:
  max_version: TLSv1_3

 

 

 

 

Share this document