Loading

Amazon MQ

Version 0.5.0 beta:[] (View all)
Compatible Kibana version(s) 8.16.5 or higher
9.0.0 or higher
Supported Serverless project types
What's this?
Security
Observability
Subscription level
What's this?
Basic
Level of support
What's this?
Elastic

Amazon MQ is a fully managed message broker service that supports Apache ActiveMQ Classic and RabbitMQ, making it easy to set up and operate message brokers in the cloud. It provides automated broker provisioning, maintenance, and scaling, allowing you to focus on building applications without managing the underlying infrastructure. With built-in security, reliability, and support for industry-standard messaging protocols, Amazon MQ enables seamless communication between distributed applications.

The Amazon MQ integration allows you to efficiently collect and monitor broker performance, queue activity, and message throughput by ingesting CloudWatch metrics into Elastic. This integration helps you gain deep insights into broker health, optimize message flow, and troubleshoot issues in real time.

Important

Extra AWS charges on API requests will be generated by this integration. Check API Requests for more details.

To enable the activemq_general_logs integration, you must configure your ActiveMQ broker to publish general logs to Amazon CloudWatch Logs. Follow these steps:

  1. Assign Necessary Permissions: Ensure the IAM user creating or managing the broker has the logs:CreateLogGroup permission. This allows Amazon MQ to create the required log groups in CloudWatch.

  2. Set Up a Resource-Based Policy: Configure a policy that permits Amazon MQ to publish logs to your CloudWatch log groups. This involves granting logs:CreateLogStream and logs:PutLogEvents permissions.

  3. Enable Logging on the Broker:

    • Navigate to the Amazon MQ console.
    • During broker creation or by editing an existing broker, expand the Additional settings section.
    • In the Logs section, select the option to publish General logs to Amazon CloudWatch Logs.

For detailed instructions, refer to the Amazon MQ Developer Guide.

This integration presently supports Amazon MQ for Apache ActiveMQ and RabbitMQ metrics.

The Amazon MQ integration collects metrics and logs from Apache ActiveMQ and metrics from RabbitMQ.

Data streams:

  • activemq_general_logs: Collects ActiveMQ general logs, including system events, warnings, and errors, which are published to a designated Amazon CloudWatch log group.
  • activemq_metrics: Collects broker metrics and destination (queue and topic) metrics.
  • rabbitmq_metrics: Collects broker, queue and node metrics.

You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware.

Before using any Amazon MQ integration you will need:

  • AWS Credentials to connect with your AWS account.
  • AWS Permissions to make sure the user you're using to connect has permission to share the relevant data.

For more details about these requirements, check the AWS integration documentation.

  • Elastic Agent must be installed. For detailed guidance, follow these instructions.
  • You can install only one Elastic Agent per host.
  • Elastic Agent is required to stream data from the S3 bucket and ship the data to Elastic, where the events will then be processed through the integration's ingest pipelines.

When general logging is enabled for your Amazon MQ ActiveMQ broker, it publishes the activemq.log file at the default INFO logging level to a designated log group. Please note that DEBUG logging is not supported.

Amazon MQ for ActiveMQ provides a range of broker and queue metrics that help monitor system performance, resource utilization, and message flow. These metrics can be used for various use cases, including:

  • Tracking broker resource utilization, such as compute, memory, and storage.
  • Monitoring message throughput and queue performance.
  • Identifying connection patterns and active consumers for optimizing messaging workloads.

The following metrics are related to Amazon MQ quotas. You can disable their collection by turning off Collect ActiveMQ quota metrics under Advanced Options:

  • AmqpMaximumConnections
  • MqttMaximumConnections
  • OpenwireMaximumConnections
  • StompMaximumConnections
  • WsMaximumConnections

Amazon MQ for RabbitMQ offers a variety of broker and queue metrics to monitor system performance, resource utilization, and message flow. These metrics are essential for:

  • Assessing broker resource usage, including CPU, memory, and storage.
  • Tracking message rates and queue depths to ensure efficient message processing.
  • Analyzing connection counts and consumer activity to optimize messaging workloads.