Elastic Cloud on Kubernetes breaking changes

ECK 3.4.0 sets seccompProfile to RuntimeDefault on managed pods. This causes a rolling restart of nearly all ECK-managed pods (Elasticsearch, Kibana, APM Server, Enterprise Search, Logstash, Elastic Maps Server, and Elastic Package Registry) during the operator upgrade. Beats and Elastic Agent pods are not affected. This rolling restart only occurs when the operator flag set-default-security-context is set to auto-detect (the default) or true. For more information, check PR #9012.

Impact
Upgrading to ECK 3.4.0 will trigger a rolling restart of most managed workloads. Plan the upgrade during a maintenance window.

Action
Schedule the ECK operator upgrade during a maintenance window to account for the rolling restart. If you need to prevent the restart, set the operator flag set-default-security-context to false before upgrading, though this is not recommended as the new default improves security posture.

ECK 3.4.0 sets a default security context on the Kibana init container, which will cause Kibana pods to rolling restart during the operator upgrade. For more information, check PR #9218.

Impact
Kibana pods will be restarted as part of the operator upgrade.

Action
No action required. Be aware that Kibana pods will restart during the upgrade.

ECK 3.4.0 adds client certificate authentication support for Elasticsearch. This changes the pre-stop hook and readiness probe scripts embedded in the Elasticsearch pod spec to handle client certificates when available, which causes a rolling restart of all Elasticsearch pods during the operator upgrade. For more information, check PR #9229 and PR #9375.

Impact
All Elasticsearch pods will be restarted as part of the operator upgrade.

Action
No action required. Be aware that Elasticsearch pods will restart during the upgrade. Plan the upgrade during a maintenance window.

ECK 3.4.0 increases the default Kibana memory limit from 1Gi to 2Gi. Kibana 9.4.x increased its V8 heap limit from 60% to 75% of container memory, but with 1Gi containers the resulting ~750MB of heap is not enough headroom for plugin initialization, leading to OOM crashes. This change will cause Kibana pods to rolling restart during the operator upgrade. For more information, check PR #9328.

Impact
Kibana pods that do not have explicit memory limits set will be restarted as part of the operator upgrade, and each pod will consume up to 2Gi of memory instead of 1Gi.

Action
Ensure that cluster nodes have sufficient memory to accommodate the increased default. If you have explicitly set a memory limit in the Kibana podTemplate, this change does not affect you. However, if you have set a memory limit lower than 2Gi, be aware that Kibana 9.4.0+ may experience OOM crashes due to the increased V8 heap usage.

The standalone Enterprise Search, App Search and Workplace Search products remain available in maintenance mode and are no longer recommended for new search experiences. We recommend transitioning to our actively developed Elastic Stack tools to build new semantic and AI powered search experiences. There will be no standalone Enterprise Search 9.x image to update to. For more information, check Migrating to 9.x from Enterprise Search 8.x versions.

Impact
The upgrade to version 9.0.0 is not possible for standalone Enterprise search resources.

Action
Migrate away from Enterprise Search following this guide. Only once the standalone Enterprise Search resources have been deleted attempt the upgrade of the Elastic Stack to version 9.0.0.