Sending data to Elastic Cloud Hosted

Elastic Cloud Hosted simplifies safe, secure communication between Logstash and Elasticsearch. When you configure the Elasticsearch output plugin to use cloud_id with either the cloud_auth option or the api_key option, no additional SSL configuration is needed.

Examples:

  • output {elasticsearch { cloud_id => "<cloud id>" cloud_auth => "<cloud auth>" } }
  • output {elasticsearch { cloud_id => "<cloud id>" api_key => "<api key>" } }

Note that the value of the api_key option is in the format id:api_key, where id and api_key are the values returned by the Create API key API.

Elastic Cloud Hosted is available on AWS, GCP, and Azure, and you can try it for free at https://cloud.elastic.co/registration.

Warning

If an Elastic Cloud Hosted deployment is protected by an AWS PrivateLink VPC filter, you can't use cloud_id. The Cloud ID encodes the public Elasticsearch endpoint, which is not available after the filter is associated. Find the private Elasticsearch URL, then connect using that URL and an API key:

output {elasticsearch { hosts => ["ELASTICSEARCH_ENDPOINT_URL"] api_key => "<api key>" } }

Logstash uses the Cloud ID, found in the Elastic Cloud web console, to build the Elasticsearch and Kibana hosts settings. It is a base64 encoded text value of about 120 characters made up of upper and lower case letters and numbers. If you have several Cloud IDs, you can add a label, which is ignored internally, to help you tell them apart. To add a label, prefix your Cloud ID with a label and a : separator in this format "<label>:<cloud-id>".

Cloud Auth is optional. Construct this value by following this format "<username>:<password>". Use your Cloud username for the first part. Use your Cloud password for the second part, which is given once in the Cloud UI when you create a cluster. If you change your Cloud password in the Cloud UI, remember to change it here, too.

The Elasticsearch input, output, and filter plugins, as well as the Elastic_integration filter plugin, support cloud_id and cloud_auth in their configurations.

These settings in the logstash.yml config file can help you get set up to use Central Pipeline Management in Elastic Cloud:

  • xpack.management.elasticsearch.cloud_id
  • xpack.management.elasticsearch.cloud_auth

You can use the xpack.management.elasticsearch.cloud_id setting as an alternative to xpack.management.elasticsearch.hosts.

You can use the xpack.management.elasticsearch.cloud_auth setting as an alternative to both xpack.management.elasticsearch.username and xpack.management.elasticsearch.password. The credentials you specify here should be for a user with the logstash_admin role, which provides access to .logstash-* indices for managing configurations.