Kafka fields

Kafka module

Broker Consumer Group Information have been read from (Broker handling the consumer group).

kafka.broker.id

Broker id

type: long

kafka.broker.address

Broker advertised address

type: keyword

kafka.topic.name

Topic name

type: keyword

kafka.topic.error.code

Topic error code.

type: long

kafka.partition.id

Partition id.

type: long

kafka.partition.topic_id

Unique id of the partition in the topic.

type: keyword

kafka.partition.topic_broker_id

Unique id of the partition in the topic and the broker.

type: keyword

Broker metrics from Kafka Broker JMX

kafka.broker.mbean

Mbean that this event is related to

type: keyword

kafka.broker.request.channel.queue.size

The size of the request queue

type: long

kafka.broker.request.produce.failed_per_second

The rate of failed produce requests per second

type: float

kafka.broker.request.fetch.failed_per_second

The rate of client fetch request failures per second

type: float

kafka.broker.request.produce.failed

The number of failed produce requests

type: float

kafka.broker.request.fetch.failed

The number of client fetch request failures

type: float

kafka.broker.replication.leader_elections

The leader election rate

type: float

kafka.broker.replication.unclean_leader_elections

The unclean leader election rate

type: float

kafka.broker.session.zookeeper.disconnect

The ZooKeeper closed sessions per second

type: float

kafka.broker.session.zookeeper.expire

The ZooKeeper expired sessions per second

type: float

kafka.broker.session.zookeeper.readonly

The ZooKeeper readonly sessions per second

type: float

kafka.broker.session.zookeeper.sync

The ZooKeeper client connections per second

type: float

kafka.broker.log.flush_rate

The log flush rate

type: float

kafka.broker.topic.net.in.bytes_per_sec

The incoming byte rate per topic

type: float

kafka.broker.topic.net.out.bytes_per_sec

The outgoing byte rate per topic

type: float

kafka.broker.topic.net.rejected.bytes_per_sec

The rejected byte rate per topic

type: float

kafka.broker.topic.messages_in

The incoming message rate per topic

type: float

kafka.broker.net.in.bytes_per_sec

The incoming byte rate

type: float

kafka.broker.net.out.bytes_per_sec

The outgoing byte rate

type: float

kafka.broker.net.rejected.bytes_per_sec

The rejected byte rate

type: float

kafka.broker.messages_in

The incoming message rate

type: float

Consumer metrics from Kafka Consumer JMX

kafka.consumer.mbean

Mbean that this event is related to

type: keyword

kafka.consumer.fetch_rate

The minimum rate at which the consumer sends fetch requests to a broker

type: float

kafka.consumer.bytes_consumed

The average number of bytes consumed for a specific topic per second

type: float

kafka.consumer.records_consumed

The average number of records consumed per second for a specific topic

type: float

kafka.consumer.in.bytes_per_sec

The rate of bytes coming in to the consumer

type: float

kafka.consumer.max_lag

The maximum consumer lag

type: float

kafka.consumer.zookeeper_commits

The rate of offset commits to ZooKeeper

type: float

kafka.consumer.kafka_commits

The rate of offset commits to Kafka

type: float

kafka.consumer.messages_in

The rate of consumer message consumption

type: float

consumergroup

kafka.consumergroup.id

Consumer Group ID

type: keyword

kafka.consumergroup.offset

consumer offset into partition being read

type: long

kafka.consumergroup.meta

custom consumer meta data string

type: keyword

kafka.consumergroup.consumer_lag

consumer lag for partition/topic calculated as the difference between the partition offset and consumer offset

type: long

kafka.consumergroup.error.code

kafka consumer/partition error code.

type: long

Assigned client reading events from partition

kafka.consumergroup.client.id

Client ID (kafka setting client.id)

type: keyword

kafka.consumergroup.client.host

Client host

type: keyword

kafka.consumergroup.client.member_id

internal consumer group member ID

type: keyword

partition

Available offsets of the given partition.

kafka.partition.offset.newest

Newest offset of the partition.

type: long

kafka.partition.offset.oldest

Oldest offset of the partition.

type: long

Partition data.

kafka.partition.partition.leader

Leader id (broker).

type: long

kafka.partition.partition.replica

Replica id (broker).

type: long

kafka.partition.partition.insync_replica

Indicates if replica is included in the in-sync replicate set (ISR).

type: boolean

kafka.partition.partition.is_leader

Indicates if replica is the leader

type: boolean

kafka.partition.partition.error.code

Error code from fetching partition.

type: long

Producer metrics from Kafka Producer JMX

kafka.producer.mbean

Mbean that this event is related to

type: keyword

kafka.producer.available_buffer_bytes

The total amount of buffer memory

type: float

kafka.producer.batch_size_avg

The average number of bytes sent

type: float

kafka.producer.batch_size_max

The maximum number of bytes sent

type: long

kafka.producer.record_send_rate

The average number of records sent per second

type: float

kafka.producer.record_retry_rate

The average number of retried record sends per second

type: float

kafka.producer.record_error_rate

The average number of retried record sends per second

type: float

kafka.producer.records_per_request

The average number of records sent per second

type: float

kafka.producer.record_size_avg

The average record size

type: float

kafka.producer.record_size_max

The maximum record size

type: long

kafka.producer.request_rate

The number of producer requests per second

type: float

kafka.producer.response_rate

The number of producer responses per second

type: float

kafka.producer.io_wait

The producer I/O wait time

type: float

kafka.producer.out.bytes_per_sec

The rate of bytes going out for the producer

type: float

kafka.producer.message_rate

The producer message rate

type: float