---
title: Connect to Amazon S3 with static credentials for ES|QL Data Federation
description: Set up Amazon S3 static credentials for ES|QL Data Federation so Elasticsearch can read your private bucket with an access key and secret key.
url: https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-static-credentials
products:
  - Elasticsearch
applies_to:
  - Elastic Cloud Serverless: Unavailable
  - Elastic Stack: Experimental in 9.5
---

# Connect to Amazon S3 with static credentials for ES|QL Data Federation
Static credentials let Elasticsearch read a private Amazon S3 data source using an AWS access key and secret key. You grant an IAM identity read-only access to your objects, generate a long-lived access key for it, and enter that key when you connect the data source.
Setup involves steps in both AWS and Elastic: create the IAM identity and access key in AWS, then enter the key when connecting the data source in Elastic.
You can use this page in two ways:
- Work through the following steps to understand each AWS resource and how the pieces fit together.
- Jump to the [complete AWS CLI example](#complete-aws-cli-example) to set it up hands-on and learn it by doing.

Refer to the [AWS IAM documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) as the authoritative reference for the commands shown here.

## Before you begin

To follow this guide, you need:
- An Elastic project or deployment with ES|QL Data Federation available.
- An AWS account with permissions to create IAM policies, users, and access keys.
- An S3 bucket containing the file or files you want to query.
- A role with the cluster manage privilege, or a `global.data_source` privilege, to create the data source. Refer to [Manage access](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-security).


## Connect to AWS with static credentials

Follow these steps to create and use dedicated credentials for use with Data Federation.
<stepper>
  <step title="Create a read-only IAM policy">
    Elasticsearch reads your objects through an IAM identity, so first create an IAM policy that grants read-only access to only the objects you want to query in AWS. This policy is the part specific to this integration.The following policy allows reading your objects with `s3:GetObject`, and listing the bucket with `s3:ListBucket` and `s3:GetBucketLocation` for prefix or glob queries:
    ```json
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [ "s3:GetObject" ],
          "Resource": [ "arn:aws:s3:::<bucket-name>/<path>/*" ] 
        },
        {
          "Effect": "Allow",
          "Action": [ "s3:ListBucket", "s3:GetBucketLocation" ],
          "Resource": [ "arn:aws:s3:::<bucket-name>" ] 
        }
      ]
    }
    ```

    <dropdown title="Example: create the policy with the AWS CLI">
      Refer to the AWS IAM documentation for the authoritative steps and for console-based setup.
      ```shell
      POLICY_ARN=$(aws iam create-policy \
        --policy-name parquet-sample-policy \
        --policy-document file://permissions-policy.json \
        --query 'Policy.Arn' --output text) 
      ```
    </dropdown>
  </step>

  <step title="Attach the policy to an IAM identity">
    Elasticsearch authenticates to S3 as an IAM user. Attach the policy from the previous step to a dedicated user you create for Elasticsearch, or to an existing identity. Refer to the [AWS IAM documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users.html) for guidance on managing IAM users.
    <dropdown title="Example: create a user and attach the policy with the AWS CLI">
      Create a user for Elasticsearch and attach the policy from the previous step:
      ```shell
      # Create a dedicated IAM user for Elasticsearch
      aws iam create-user --user-name esql-user

      # Attach the read-only policy to the user
      aws iam attach-user-policy \
        --user-name esql-user \
        --policy-arn "${POLICY_ARN}" 
      ```
    </dropdown>
  </step>

  <step title="Create an access key">
    Generate a long-lived access key for the user you authorized.
    ```shell
    aws iam create-access-key --user-name esql-user
    ```
    The response includes an **access key ID** and a **secret access key**. The secret is shown only once, so copy it now. These credentials do not expire on their own. They stay valid until you deactivate or delete them in AWS. This is the access key and secret key pair you enter in Elastic.
  </step>

  <step title="Connect the data source and create a dataset">
    Back in Elastic:**Step 1.** Connect the S3 data source with the **Access and Secret Keys** method.
    <tab-set>
      <tab-item title="UI">
        In Kibana:
        1. Go to **Data management** > **ES|QL Data Federation**.
        2. Click **Connect data source**.
        3. Set **Data source type** to **Amazon S3**.
        4. Under **Authentication**, select **Access and Secret Keys**.
        5. Enter the **access key** and **secret key** from the previous step.
        For the full field reference, refer to [Connect external data sources](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-sources).
        ![The Authentication section of the Connect data source flyout with Access and Secret Keys selected, showing the access key and secret key fields](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/images/data-federation/access-and-secret-keys.png)
      </tab-item>

      <tab-item title="Console">
        ```json

        {
          "type": "s3",
          "settings": {
            "region": "eu-north-1",
            "auth": "static_credentials",
            "access_key": "<AWS_ACCESS_KEY_ID>", <1>
            "secret_key": "<AWS_SECRET_ACCESS_KEY>"
          }
        }
        ```
      </tab-item>

      <tab-item title="curl">
        ```bash
        curl -X PUT "${ELASTICSEARCH_URL}/_query/data_source/prod_s3_static" \
          -H "Authorization: ApiKey ${API_KEY}" \
          -H "Content-Type: application/json" \
          -d '{
          "type": "s3",
          "settings": {
            "region": "eu-north-1",
            "auth": "static_credentials",
            "access_key": "<AWS_ACCESS_KEY_ID>",
            "secret_key": "<AWS_SECRET_ACCESS_KEY>"
          }
        }'
        ```
      </tab-item>
    </tab-set>
    **Step 2.** [Create a dataset](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-datasets) that points at your files, for example `s3://amzn-s3-demo-bucket/some/sample.parquet` in **Parquet** format.You can now query the remote data with ES|QL.
  </step>
</stepper>


## Complete AWS CLI example

The preceding steps explain each AWS resource on its own. The following is an end-to-end AWS example of that setup, using sample values for one scenario. It is illustrative, not a script to run as-is: replace the example bucket, file, and names with your own before you run it. As with the individual steps, AWS is the authoritative reference for these commands.
<dropdown title="Show the complete AWS CLI example">
  This example grants read access to a single Parquet file at `s3://amzn-s3-demo-bucket/some/sample.parquet` and creates a long-lived access key for a dedicated IAM user. Run the commands in order in [AWS CloudShell](https://docs.aws.amazon.com/cloudshell/latest/userguide/welcome.html) or any shell with the AWS CLI configured.**Step 1.** Set the variables for your environment:
  ```shell
  export BUCKET_NAME="amzn-s3-demo-bucket"
  export FILE_NAME="some/sample.parquet"
  export POLICY_NAME="parquet-sample-policy"
  export IAM_USER="esql-user"
  ```
  **Step 2.** Create a read-only IAM policy scoped to your objects, and capture the policy ARN it returns:
  ```shell
  POLICY_ARN=$(aws iam create-policy \
    --policy-name "${POLICY_NAME}" \
    --policy-document "$(cat <<EOF
  {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": [ "s3:GetObject" ],
        "Resource": [ "arn:aws:s3:::${BUCKET_NAME}/${FILE_NAME}" ]
      },
      {
        "Effect": "Allow",
        "Action": [ "s3:ListBucket", "s3:GetBucketLocation" ],
        "Resource": [ "arn:aws:s3:::${BUCKET_NAME}" ]
      }
    ]
  }
  EOF
  )" \
    --query 'Policy.Arn' --output text)
  ```
  **Step 3.** Create a dedicated IAM user for Elasticsearch:
  ```shell
  aws iam create-user --user-name "${IAM_USER}"
  ```
  **Step 4.** Attach the policy to the user:
  ```shell
  aws iam attach-user-policy \
    --user-name "${IAM_USER}" \
    --policy-arn "${POLICY_ARN}"
  ```
  **Step 5.** Generate a long-lived access key. Copy the access key ID and secret access key from the output, because the secret is shown only once:
  ```shell
  aws iam create-access-key --user-name "${IAM_USER}"
  ```
</dropdown>


## Next steps

- [Query your data](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-querying) with `FROM`, including metadata columns and current limitations.
- [Create and manage datasets](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-datasets) to add more datasets over this data source, and configure file formats and settings.
- [Manage credentials and privileges](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-security) to control who can access your data sources and datasets.