Elastic
Start free trial Contact Sales
Loading

Elastic Serverless Forwarder input plugin

  • Plugin version: v2.0.0
  • Released on: 2024-12-23
  • Changelog

For other versions, see the Versioned plugin docs.

Getting help

For questions about the plugin, open a topic in the Discuss forums. For bugs or feature requests, open an issue in Github. For the list of Elastic supported plugins, please consult the Elastic Support Matrix.

Description

Using this input you can receive events from Elastic Serverless Forwarder over http(s) connections to the configured port.

SSL Enabled SSL Disabled
<br>input {<br> elastic_serverless_forwarder {<br> port => 8080<br> ssl_certificate => "/path/to/logstash.crt"<br> ssl_key => "/path/to/logstash.key"<br> }<br>}<br>
 <br>input {<br> elastic_serverless_forwarder {<br> port => 8080<br> ssl_enabled => false<br> }<br>}<br>

This input provides minimal enrichment on events, and avoids including information about itself, the client from which it received the data, or about the original event as-decoded from the request.

Note

Senders are advised to use care with respect to fields that are reserved in Logstash. ESF sends the Logstash-required @timestamp field by default, but if this value is missing it will be populated with the current time.

This plugin has SSL on-by-default.

At a minimum, you will need to either configure the plugin to present its identity, or disable SSL.

Additionally, you may wish to authenticate clients using SSL client authentication, and/or authenticate requests using HTTP Basic authentication as described below.

SSL Identity

In order to establish SSL with a client, this input plugin will need to present an SSL certificate that the client trusts, and have access to the associated key. These are configurable with ssl_certificate, ssl_key, and optionally ssl_key_passphrase.

SSL Client Authentication

By default, this plugin does not request certificates from clients during SSL negotiation.

It can be configured to either request or require client certificates using ssl_client_authentication, which often also requires configuring it with a list of ssl_certificate_authorities to trust. When validating a certificate that is presented, ssl_verification_mode controls how certificates are verified.

Note

ESF does not currently support presenting client certificates, so requesting or requiring clients to present identity is only useful when combined with an SSL-terminating proxy.

SSL Advanced Configuration

This plugin exposes several advanced SSL configurations:

HTTP Basic Authentication

You can configure this plugin to authenticate requests using HTTP Basic authentication by configuring auth_basic_username and auth_basic_password.

Note

Basic Authentication is not a substitute for SSL, as it provides neither secrecy nor security on its own. When used with SSL disabled, HTTP Basic credentials are transmitted in effectively clear-text and can be easily recovered by an adversary.

Here are some tips for configuring the Elastic Serverless Forwarder input to work with the elasticsearch output:

  • Set the document_id in the output configuration when you use the Elastic Serverless Forwarder input with an Elasticsearch output plugin.

    output {
  elasticsearch {
      ...
      document_id => "%{[@metadata][_id]}"
      ...
  }
}

  • Starting from version 1.10.0 of Elastic Serverless Forwarder, configuring document_id as shown in the example above is sufficient (the _id field is no longer available, and instead, Logstash now receives the @metadata._id field).

  • For Elastic Serverless Forwarder v1.9.0 and earlier, rename the field _id to @metadata._id with a filter:

    filter {
    # support ESF < 1.10
    if [_id] and ![@metadata][_id] {
      mutate { rename => { "_id" => "[@metadata][_id]" } }
    }
}

This plugin supports the following configuration options plus the Common options described later.

Note

As of version 2.0.0 of this plugin, a previously deprecated SSL setting has been removed. Please check out Elasticsearch Output Obsolete Configuration Options for details.

Setting Input type Required
auth_basic_username string No
auth_basic_password password No
host string No
port number No
ssl_certificate a valid filesystem path No
ssl_certificate_authorities array No
ssl_client_authentication string, one of ["none", "optional", "required"] No
ssl_cipher_suites array No
ssl_enabled boolean No
ssl_handshake_timeout number No
ssl_key a valid filesystem path No
ssl_key_passphrase password No
ssl_supported_protocols array No
ssl_verification_mode string, one of ["certificate"] No

Also see Common options for a list of options supported by all input plugins.

  • Value type is password
  • There is no default value for this setting.

Password for HTTP basic authorization. Requires auth_basic_username.

  • Value type is string
  • There is no default value for this setting.

Username for basic authorization. Requires auth_basic_password.

  • Value type is string
  • Default value is "0.0.0.0" (all available interfaces)

The host or ip to bind

  • Value type is number
  • Default value is 8080

The TCP port to bind to

  • Value type is path
  • There is no default value for this setting.

SSL certificate to use. This certificate MUST be PEM-formatted, and MAY contain a chain of certificates starting with the certificate that identifies itself, followed by zero or more ordered intermediates optionally ending with the root signing authority. Providing a complete chain allows clients to trust our certificate if their configuration allows them to trust one of our intermediates.

  • Value type is array
  • Default value is []

Validate client certificates against these authorities. You can define multiple files or paths. All the certificates will be read and added to the trust store.

If you wish to perform client authentication, you need to set ssl_client_authentication to optional or required.

  • Value type is array
  • Default value is ['TLS_AES_256_GCM_SHA384', 'TLS_AES_128_GCM_SHA256', 'TLS_CHACHA20_POLY1305_SHA256', 'TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384', 'TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384', 'TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256', 'TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256', 'TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256', 'TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256', 'TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384', 'TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384', 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256', 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256']

The list of cipher suites to use, listed by priorities.

This is an advanced SSL configuration.

This default list applies for OpenJDK 11.0.14 and higher. For older JDK versions, the default list includes only suites supported by that version. For example, the ChaCha20 family of ciphers is not supported in older versions.

  • Value can be any of:

    • none: do not request client’s certificate, or validate certificates that are presented
    • optional: request client’s certificate, and validate it against our trust authorities if-and-only-if it is presented
    • required: require a valid certificate from the client that is signed by a trusted certificate authority

  • Default value is "none"

By default the server doesn’t do any client authentication. This means that connections from clients are private when SSL is enabled, but that this input will allow SSL connections from any client. If you wish to configure this plugin to reject connections from untrusted hosts, you will need to configure this plugin to authenticate clients, and may also need to configure it with a list of ssl_certificate_authorities.

  • Value type is boolean
  • Default value is true

Events are, by default, sent over SSL, which requires configuring this plugin to present an identity certificate using ssl_certificate and key using ssl_key.

You can disable SSL with +ssl_enabled => false+.

  • Value type is number
  • Default value is 10000

Time in milliseconds for an incomplete ssl handshake to timeout

This is an advanced SSL configuration.

  • Value type is path
  • There is no default value for this setting.

SSL key to use.

Note

This key need to be in the PKCS8 format, you can convert it with OpenSSL for more information.

  • Value type is password
  • There is no default value for this setting.

SSL key passphrase to use.

  • Value type is array
  • Allowed values are: 'TLSv1.1', 'TLSv1.2', 'TLSv1.3'
  • Default depends on the JDK being used. With up-to-date Logstash, the default is ['TLSv1.2', 'TLSv1.3']. 'TLSv1.1' is not considered secure and is only provided for legacy applications.

List of allowed SSL/TLS versions to use when establishing a connection to the HTTP endpoint.

This is an advanced SSL configuration.

For Java 8 'TLSv1.3' is supported only since 8u262 (AdoptOpenJDK), but requires that you set the LS_JAVA_OPTS="-Djdk.tls.client.protocols=TLSv1.3" system property in Logstash.

Note

If you configure the plugin to use 'TLSv1.1' on any recent JVM, such as the one packaged with Logstash, the protocol is disabled by default and needs to be enabled manually by changing jdk.tls.disabledAlgorithms in the $JDK_HOME/conf/security/java.security configuration file. That is, TLSv1.1 needs to be removed from the list.

  • Value type is string

  • There is only one currently-supported mode:

    • certificate: verifies that a certificate provided by the client is signed by a trusted authority (CA), is within its valid date range, and that the client has possession of the associated key, but does not perform hostname validation.

  • The default value is certificate.

When ssl_client_authentication causes a client to present a certificate, this setting controls how that certificate is verified.

Note

Client identity is not typically validated using SSL because the receiving server only has access to the client’s outbound-ip, which is not always constant and is frequently not represented in the certificate’s subject or subjectAltNames extensions. For more information, see RFC2818 § 3.2 (HTTP over TLS — Client Identity)

Warning

As of version 2.0.0 of this plugin, some configuration options have been replaced. The plugin will fail to start if it contains any of these obsolete options.

Setting Replaced by
ssl ssl_enabled

These configuration options are supported by all input plugins:

Setting Input type Required
add_field hash No
enable_metric boolean No
id string No
tags array No
type string No
  • Value type is hash
  • Default value is {}

Add a field to an event

  • Value type is boolean
  • Default value is true

Disable or enable metric logging for this specific plugin instance by default we record all the metrics we can, but you can disable metrics collection for a specific plugin.

  • Value type is string
  • There is no default value for this setting.

Add a unique ID to the plugin configuration. If no ID is specified, Logstash will generate one. It is strongly recommended to set this ID in your configuration. This is particularly useful when you have two or more plugins of the same type, for example, if you have 2 elastic_serverless_forwarder inputs. Adding a named ID in this case will help in monitoring Logstash when using the monitoring APIs.

input {
  elastic_serverless_forwarder {
    id => "my_plugin_id"
  }
}
Note

Variable substitution in the id field only supports environment variables and does not support the use of values from the secret store.

  • Value type is array
  • There is no default value for this setting.

Add any number of arbitrary tags to your event.

This can help with processing later.

  • Value type is string
  • There is no default value for this setting.

Add a type field to all events handled by this input.

Types are used mainly for filter activation.

The type is stored as part of the event itself, so you can also use the type to search for it in Kibana.

If you try to set a type on an event that already has one (for example when you send an event from a shipper to an indexer) then a new input will not override the existing type. A type set at the shipper stays with that event for its life even when sent to another Logstash server.