Elastic APM Python Agent release notes

Review the changes, fixes, and more in each version of Elastic APM Python Agent.

To check for security updates, go to Security announcements for the Elastic stack.

Release date: July 30, 2024

• Make published Docker images multi-platform with the addition of linux/arm64 #2080

• Fix handling consumer iteration if transaction not sampled in kafka instrumentation #2075
• Fix race condition with urllib3 at shutdown #2085
• Fix compatibility with setuptools>=72 that removed test command #2090

Release date: June 10, 2024

• Fix outcome in ASGI and Starlette apps on error status codes without an exception #2060

Release date: May 20, 2024

• Fix CI release workflow #2046

Release date: May 17, 2024

• Relax wrapt dependency to only exclude 1.15.0 #2005

Release date: April 3, 2024

• Add ability to override default JSON serialization #2018

Release date: March 19, 2024

• Fix urllib3 2.0.1+ crash with many args #2002

Release date: March 8, 2024

• Fix artifacts download in CI workflows #1996

Release date: March 7, 2024

• Fix artifacts upload in CI build-distribution workflow #1993

Release date: March 7, 2024

• Fix CI release workflow #1990

Release date: March 6, 2024

• Fix starlette middleware setup without client argument #1952
• Fix blocking of gRPC stream-to-stream requests #1967
• Always take into account body reading time for starlette requests #1970
• Make urllib3 transport tests more robust against local env #1969
• Clarify starlette integration documentation #1956
• Make dbapi2 query scanning for dollar quotes a bit more correct #1976
• Normalize headers in AWS Lambda integration on API Gateway v1 requests #1982

Release date: January 10, 2024

• Async support for dbapi2 (starting with psycopg) #1944
• Add object name to procedure call spans in dbapi2 #1938
• Add support for python 3.10 and 3.11 lambda runtimes

• Fix asyncpg support for 0.29+ #1935
• Fix dbapi2 signature extraction to handle square brackets in table name #1947

Release date: October 11, 2023

• Add Python 3.12 support
• Collect the configured_hostname and detected_hostname separately, and switch to FQDN for the detected_hostname. #1891
• Improve postgres dollar-quote detection to be much faster #1905

• Fix url argument fetching in aiohttp_client instrumentation #1890
• Fix a bug in the AWS Lambda instrumentation when event["headers"] is None #1907
• Fix a bug in AWS Lambda where metadata could be incomplete, causing validation errors with the APM Server #1914
• Fix a bug in AWS Lambda where sending the partial transaction would be recorded as an extra span #1914

Release date: July 25, 2023

• Add support for grpc aio server interceptor #1870

• Fix a bug in the Elasticsearch client instrumentation which was causing loss of database context (including statement) when interacting with Elastic Cloud #1878

Release date: July 3, 2023

• Add server_ca_cert_file option to provide custom CA certificate #1852
• Add include_process_args option to allow users to opt-in to collecting process args #1867

• Fix a bug in the GRPC instrumentation when reaching the maximum amount of spans per transaction #1861

Release date: June 12, 2023

• Fix compatibility issue with older versions of OpenSSL in lambda runtimes #1847
• Add latest tag to docker images #1848
• Fix issue with redacting user:pass in URLs in Python 3.11.4 #1850

Release date: June 6, 2023

• Fix release process for docker and the lambda layer #1845

Release date: June 5, 2023

• Add lambda layer for instrumenting AWS Lambda functions #1826
• Implement instrumentation of Azure Functions #1766
• Add support for Django to wrapper script #1780
• Add support for Starlette to wrapper script #1830
• Add transport_json_serializer configuration option #1777
• Add S3 bucket and key name to OTel attributes #1790
• Implement partial transaction support in AWS lambda #1784
• Add instrumentation for redis.asyncio #1807
• Add support for urllib3 v2.0.1+ #1822
• Add service.environment to log correlation #1833
• Add ecs_logging as a dependency #1840
• Add support for synchronous psycopg3 #1841

• Fix spans being dropped if they don’t have a name #1770
• Fix AWS Lambda support when event is not a dict #1775
• Fix deprecation warning with urllib3 2.0.0 pre-release versions #1778
• Fix activation_method to only send to APM server 8.7.1+ #1787
• Fix span.context.destination.service.resource for S3 spans to have an "s3/" prefix. #1783

Note: While this is considered a bugfix, it can potentially be a breaking change in the Kibana APM app: It can break the history of the S3-Spans / metrics for users relying on context.destination.service.resource. If users happen to run agents both with and without this fix (for same or different languages), the same S3-buckets can appear twice in the service map (with and without s3-prefix).

• Fix instrumentation to not bubble up exceptions during instrumentation #1791
• Fix HTTP transport to not print useless and confusing stack trace #1809

Release date: March 6, 2023

• Fix issue with botocore instrumentation creating spans with an incorrect service.name #1765
• Fix a bug in the GRPC instrumentation when the agent is disabled or not recording #1761

Release date: February 16, 2023

• Add service.agent.activation_method to the metadata #1743

• Small fix to underlying Starlette logic to prevent duplicate Client objects #1735
• Change server_url default to http://127.0.0.1:8200 to avoid ipv6 ambiguity #1744
• Fix an issue in GRPC instrumentation with unsampled transactions #1740
• Fix error in async Elasticsearch instrumentation when spans are dropped #1758

Release date: January 30, 2023

• GRPC support #1703
• Wrapper script Flask support (experimental) #1709

• Fix an async issue with long elasticsearch queries #1725
• Fix a minor inconsistency with the W3C tracestate spec #1728
• Fix a cold start performance issue with our AWS Lambda integration #1727
• Mark **kwargs config usage in our AWS Lambda integration as deprecated #1727

Release date: November 17, 2022

• Fix error in Elasticsearch instrumentation when spans are dropped #1690
• Lower log level for errors in APM Server version fetching #1692
• Fix for missing parent.id when logging from a DroppedSpan under a leaf span #1695

Release date: November 3, 2022

• Fix elasticsearch instrumentation for track_total_hits=False #1687

Release date: October 26, 2022

• Add support for Python 3.11
• Add backend granularity data to SQL backends as well as Cassandra and pymongo #1585, #1639
• Add support for instrumenting the Elasticsearch 8 Python client #1642
• Add *principal* to default sanitize_field_names configuration #1664
• Add docs and better support for custom metrics, including in AWS Lambda #1643
• Add support for capturing span links from AWS SQS in AWS Lambda #1662

• Fix Django’s manage.py check when agent is disabled #1632
• Fix an issue with long body truncation for Starlette #1635
• Fix an issue with transaction outcomes in Flask for uncaught exceptions #1637
• Fix Starlette instrumentation to make sure transaction information is still present during exception handling #1674

Release date: September 7, 2022

• Add redis query to context data for redis instrumentation #1406
• Add AWS request ID to all botocore spans (at span.context.http.request.id) #1625

• Differentiate Lambda URLs from API Gateway in AWS Lambda integration #1609
• Restrict the size of Django request bodies to prevent APM Server rejection #1610
• Restrict length of exception.message for exceptions captured by the agent #1619
• Restrict length of Starlette request bodies #1549
• Fix error when using elasticsearch(sniff_on_start=True) #1618
• Improve handling of ignored URLs and capture_body=off for Starlette #1549
• Fix possible error in the transport flush for Lambda functions #1628

Release date: August 9, 2022

• Added lambda support for ELB triggers #1605

Release date: August 9, 2022

• Fixed an issue with non-integer ports in Django #1590
• Fixed an issue with non-integer ports in Redis #1591
• Fixed a performance issue for local variable shortening via varmap() #1593
• Fixed elasticapm.label() when a Client object is not available #1596

Release date: June 30, 2022

• Fix an issue with Kafka instrumentation and unsampled transactions #1579

Release date: June 22, 2022

• Add instrumentation for aiobotocore #1520
• Add instrumentation for kafka-python #1555
• Add API for span links, and implement span link support for OpenTelemetry bridge #1562
• Add span links to SQS ReceiveMessage call #1575
• Add specific instrumentation for SQS delete/batch-delete #1567
• Add trace_continuation_strategy setting #1564

• Fix return for opentelemetry.Span.is_recording() #1530
• Fix error logging for bad SERVICE_NAME config #1546
• Do not instrument old versions of Tornado > 6.0 due to incompatibility #1566
• Fix transaction names for class based views in Django 4.0+ #1571
• Fix a problem with our logging handler failing to report internal errors in its emitter #1568

Release date: March 30, 2022

• Fix otel_attributes-related regression with older versions of APM Server (<7.16) #1510

Release date: March 29, 2022

• Add OpenTelemetry API bridge #1411
• Change default for sanitize_field_names to sanitize *auth* instead of authorization #1494
• Add span_stack_trace_min_duration to replace deprecated span_frames_min_duration #1498
• Enable exact_match span compression by default #1504
• Allow parent celery tasks to specify the downstream parent_span_id in celery headers #1500

• Fix Sanic integration to properly respect the capture_body config #1485
• Lambda fixes to align with the cross-agent spec #1489
• Lambda fix for custom service_name #1493
• Change default for stack_trace_limit from 500 to 50 #1492
• Switch all duration handling to use datetime.timedelta objects #1488

Release date: March 9, 2022

• Fix exit_span_min_duration and disable by default #1483

Release date: February 22, 2022

• use "unknown-python-service" as default service name if no service name is configured #1438
• add transaction name to error objects #1441
• don’t send unsampled transactions to APM Server 8.0+ #1442
• implement snapshotting of certain configuration during transaction lifetime #1431
• propagate traceparent IDs via Celery #1371
• removed Python 2 compatibility shims #1463

Note: Python 2 support was already removed with version 6.0 of the agent, this now removes unused compatibilit shims.

• fix span compression for redis, mongodb, cassandra and memcached #1444
• fix recording of status_code for starlette #1466
• fix aioredis span context handling #1462

Release date: December 7, 2021

• fix AttributeError in sync instrumentation of httpx #1423
• add setting to disable span compression, default to disabled #1429

Release date: November 29, 2021

• fix an issue with Sanic exception tracking #1414
• asyncpg: Limit SQL queries in context data to 10000 characters #1416

Release date: November 17, 2021

• Add support for Sanic framework #1390

• fix compatibility issues with httpx 0.21 #1403
• fix span_compression_exact_match_max_duration default value #1407

Release date: November 15, 2021

• fix an issue with metrics_sets configuration referencing the TransactionMetricSet removed in 6.6.2 #1397

Release date: November 10, 2021

• Fix an issue where compressed spans would count against transaction_max_spans #1377
• Make sure HTTP connections are not re-used after a process fork #1374
• Fix an issue with psycopg2 instrumentation when multiple hosts are defined #1386
• Update the User-Agent header to the new spec #1378
• Improve status_code handling in AWS Lambda integration #1382
• Fix aiohttp exception handling to allow for non-500 responses including HTTPOk #1384
• Force transaction names to strings #1389
• Remove unused http.request.socket.encrypted context field #1332
• Remove unused transaction metrics (APM Server handles these metrics instead) #1388

Release date: November 2, 2021

• Fix some context fields and metadata handling in AWS Lambda support #1368

Release date: October 18, 2021

• Add experimental support for AWS lambda instrumentation #1193
• Add support for span compression #1321
• Auto-infer destination resources for easier instrumentation of new resources #1359
• Add support for dropped span statistics #1327

• Ensure that Prometheus histograms are encoded correctly for APM Server #1354
• Remove problematic (and duplicate) event.dataset from logging integrations #1365
• Fix for memcache instrumentation when configured with a unix socket #1357

Release date: October 4, 2021

• Add instrumentation for Azure Storage (blob/table/fileshare) and Azure Queue #1316

• Improve span coverage for asyncpg #1328
• aiohttp: Correctly pass custom client to tracing middleware #1345
• Fixed an issue with httpx instrumentation #1337
• Fixed an issue with Django 4.0 removing a private method #1347

Release date: August 31, 2021

• Rename the experimental log_ecs_formatting config to log_ecs_reformatting #1300
• Add support for Prometheus histograms #1165

• Fixed cookie sanitization when Cookie is capitalized #1301
• Fix a bug with exception capturing for bad UUIDs #1304
• Fix potential errors in json serialization #1203
• Fix an issue with certain aioredis commands #1308

Release date: July 14, 2021

• ensure that the elasticsearch instrumentation handles DroppedSpans correctly #1190

Release date: July 7, 2021

• Fix handling of non-http scopes in Starlette/FastAPI middleware #1187

Release date: July 7, 2021

• Fix issue with Starlette/FastAPI hanging on startup #1185

Release date: July 6, 2021

• Add additional context information about elasticsearch client requests #1108
• Add use_certifi config option to allow users to disable certifi #1163

• Fix for Starlette 0.15.0 error collection #1174
• Fix for Starlette static files #1137

Release date: June 28, 2021

• suppress the default_app_config attribute in Django 3.2+ #1155
• bump log level for multiple set_client calls to WARNING #1164
• fix issue with adding disttracing to SQS messages when dropping spans #1170

Release date: June 7, 2021

• Fix an attribute access bug introduced in 6.2.0 #1149

Release date: June 3, 2021

• catch and log exceptions in interval timer threads #1145

Release date: May 31, 2021

• Added support for aioredis 1.x #2526
• Added support for aiomysql #1107
• Added Redis pub/sub instrumentation #1129
• Added specific instrumentation for AWS SQS #1123

• ensure metrics are flushed before agent shutdown #1139
• added safeguard for exceptions in processors #1138
• ensure sockets are closed which were opened for cloud environment detection #1134

Release date: April 28, 2021

• added destination information to asyncpg instrumentation #1115
• fixed issue with collecting request meta data with Django REST Framework #1117
• fixed httpx instrumentation for newly released httpx 0.18.0 #1118

Release date: April 14, 2021

• fixed issue with empty transaction name for the root route with Django #1095
• fixed on-the-fly initialisation of Flask apps #1099

Release date: April 8, 2021

• fixed a validation issue with the newly introduced instrumentation for S3, SNS and DynamoDB #1090

Release date: March 31, 2021

• Add global access to Client singleton object at elasticapm.get_client() #1043
• Add log_ecs_formatting config option #1058 #1063
• Add instrumentation for httplib2 #1031
• Add better instrumentation for some AWS services (S3, SNS, DynamoDB) #1054
• Added beta support for collecting metrics from prometheus_client #1083

• Fix for potential capture_body: error hang in Starlette/FastAPI #1038
• Fix a rare error around processing stack frames #1012
• Fix for Starlette/FastAPI to correctly capture request bodies as strings #1041
• Fix transaction names for Starlette Mount routes #1037
• Fix for elastic excepthook arguments #1050
• Fix issue with remote configuration when resetting config values #1068
• Use a label for the elasticapm Django app that is compatible with Django 3.2 validation #1064
• Fix an issue with undefined routes in Starlette #1076

Release date: February 1, 2021

• Fix for GraphQL span spamming from scalar fields with required flag #1015