Setting up machine learning

Requirements overview ¶

To use the Elastic Stack {ml-features}, you must have:

  • the appropriate subscription level or the free trial period activated
  • xpack.ml.enabled set to its default value of true on every node in the cluster (refer to Machine learning settings in Elasticsearch)
  • ml value defined in the list of node.roles on the machine learning nodes
  • machine learning features visible in the Kibana space
  • security privileges assigned to the user that:
    • grant use of machine learning features, and
    • grant access to source and destination indices.

Tip

The fastest way to get started with machine learning features is to start a free 14-day trial of Elasticsearch Service in the cloud.

Security privileges ¶

Assigning security privileges affects how users access machine learning features. Consider the two main categories:

  • Elasticsearch API user: uses an Elasticsearch client, cURL, or Kibana Dev Tools to access machine learning features via Elasticsearch APIs. It requires Elasticsearch security privileges.
  • Kibana user: uses the machine learning features in Kibana and does not use Dev Tools. It requires either Kibana feature privileges or Elasticsearch security privileges and is granted the most permissive combination of both. Kibana feature privileges are recommended if you control job level visibility via Spaces. Machine learning features must be visible in the relevant space. Refer to Feature visibility in Spaces for configuration information.

You can configure these privileges

  • under Security. To open Security, find Stack Management in the main menu or use the global search field.
  • via the respective Elasticsearch security APIs.

Elasticsearch API user ¶

If you use machine learning APIs, you must have the following cluster and index privileges:

For full access:

  • machine_learning_admin built-in role or the equivalent cluster privileges
  • read and view_index_metadata on source indices
  • read, manage, and index on destination indices (for '{dataframe} analytics jobs' only)

For read-only access:

  • machine_learning_user built-in role or the equivalent cluster privileges
  • read index privileges on source indices
  • read index privileges on destination indices (for '{dataframe} analytics jobs' only)

Important

The machine_learning_admin and machine_learning_user built-in roles give access to the results of all anomaly detection jobs, irrespective of whether the user has access to the source indices. You must carefully consider who is given these roles, as anomaly detection job results may propagate field values that contain sensitive information from the source indices to the results.

Kibana security ¶

Important

Granting All or Read Kibana feature privilege for Machine Learning will also grant the role the equivalent feature privileges to certain types of Kibana saved objects, namely index patterns, dashboards, saved searches, and visualizations as well as machine learning job, trained model and module saved objects.

Feature visibility in Spaces ¶

In Kibana, the machine learning features must be visible in your space. To manage which features are visible in your space, go to Stack Management > Kibana* > *Spaces or use the global search field to locate Spaces directly.

Manage spaces in {kib}

In addition to index privileges, source data views must also exist in the same space as your machine learning jobs. You can configure these under Data Views. To open Data Views*, find *Stack Management > Kibana in the main menu, or use the global search field.

Each machine learning job and trained model can be assigned to all, one, or multiple spaces. This can be configured in Machine Learning. To open Machine Learning*, find *Stack Management > Alerts and Insights in the main menu, or use the global search field. You can edit the spaces that a job or model is assigned to by clicking the icons in the Spaces column.

Assign machine learning jobs to spaces

Kibana user ¶

Within a Kibana space, for full access to the machine learning features, you must have:

  • Machine Learning: All Kibana privileges
  • Data Views Management: All Kibana feature privileges
  • read, and view_index_metadata index privileges on your source indices
  • data views for your source indices
  • data views, read, manage, and index index privileges on destination indices (for '{dataframe} analytics jobs' only)

Within a Kibana space, for read-only access to the machine learning features, you must have:

  • Machine Learning: Read Kibana privileges
  • data views for your source indices
  • read index privilege on your source indices
  • data views and read index privileges on destination indices (for '{dataframe} analytics jobs' only)

Important

A user who has full or read-only access to machine learning features within a given Kibana space can view the results of all anomaly detection jobs that are visible in that space, even if they do not have access to the source indices of those jobs. You must carefully consider who is given access to machine learning features, as anomaly detection job results may propagate field values that contain sensitive information from the source indices to the results.

Note

Data views can be automatically created when creating a '{dataframe} analytics job'.

For access to use machine learning APIs via Dev Tools in Kibana, set the Elasticsearch security privileges and grant access to machine_learning_admin or machine_learning_user built-in roles.

Data Visualizer feature ¶

Within a Kibana space, to upload and import files in the Data Visualizer, you must have:

  • Machine Learning: Read or Discover: All Kibana feature privileges
  • Data Views Management: All Kibana feature privileges
  • ingest_admin built-in role, or manage_ingest_pipelines cluster privilege
  • create, create_index, manage and read index privileges for destination indices

For more information, see Security privileges and Kibana privileges.