--- title: ECS Logging with Winston description: This Node.js package provides a formatter for the winston logger, compatible with Elastic Common Schema (ECS) logging. In combination with the Filebeat... url: https://www.elastic.co/elastic/docs-builder/docs/3028/reference/ecs/logging/nodejs/winston products: - ECS Logging - ECS Logging Node.js --- # ECS Logging with Winston This Node.js package provides a formatter for the [winston](https://github.com/winstonjs/winston#readme) logger, compatible with [Elastic Common Schema (ECS) logging](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/ecs/logging/intro). In combination with the [Filebeat](https://www.elastic.co/beats/filebeat) shipper, you can [monitor all your logs](https://www.elastic.co/log-monitoring) in one place in the Elastic Stack. `winston` 3.x versions >=3.3.3 are supported. ## Setup ### Step 1: Install ```cmd $ npm install @elastic/ecs-winston-format ``` ### Step 2: Configure ```js const winston = require('winston'); const { ecsFormat } = require('@elastic/ecs-winston-format'); const logger = winston.createLogger({ format: ecsFormat(/* options */), transports: [ new winston.transports.Console() ] }); logger.info('hi'); logger.error('oops there is a problem', { err: new Error('boom') }); ``` ### Step 3: Configure Filebeat The best way to collect the logs once they are ECS-formatted is with [Filebeat](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/beats/filebeat): 1. Follow the [Filebeat quick start](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/beats/filebeat/filebeat-installation-configuration) 2. Add the following configuration to your `filebeat.yaml` file. For Filebeat 7.16+ ```yaml filebeat.inputs: - type: filestream paths: /path/to/logs.json parsers: - ndjson: overwrite_keys: true add_error_key: true expand_keys: true processors: - add_host_metadata: ~ - add_cloud_metadata: ~ - add_docker_metadata: ~ - add_kubernetes_metadata: ~ ``` For Filebeat < 7.16 ```yaml filebeat.inputs: - type: log paths: /path/to/logs.json json.keys_under_root: true json.overwrite_keys: true json.add_error_key: true json.expand_keys: true processors: - add_host_metadata: ~ - add_cloud_metadata: ~ - add_docker_metadata: ~ - add_kubernetes_metadata: ~ ``` 1. Make sure your application logs to stdout/stderr. 2. Follow the [Run Filebeat on Kubernetes](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/beats/filebeat/running-on-kubernetes) guide. 3. Enable [hints-based autodiscover](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/beats/filebeat/configuration-autodiscover-hints) (uncomment the corresponding section in `filebeat-kubernetes.yaml`). 4. Add these annotations to your pods that log using ECS loggers. This will make sure the logs are parsed appropriately. ```yaml annotations: co.elastic.logs/json.overwrite_keys: true co.elastic.logs/json.add_error_key: true co.elastic.logs/json.expand_keys: true ``` 1. Make sure your application logs to stdout/stderr. 2. Follow the [Run Filebeat on Docker](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/beats/filebeat/running-on-docker) guide. 3. Enable [hints-based autodiscover](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/beats/filebeat/configuration-autodiscover-hints). 4. Add these labels to your containers that log using ECS loggers. This will make sure the logs are parsed appropriately. ```yaml labels: co.elastic.logs/json.overwrite_keys: true co.elastic.logs/json.add_error_key: true co.elastic.logs/json.expand_keys: true ``` For more information, see the [Filebeat reference](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/beats/filebeat/configuring-howto-filebeat). You might like to try out our tutorial using Node.js ECS logging with winston: [Ingest logs from a Node.js web application using Filebeat](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/manage-data/ingest/ingesting-data-from-applications/ingest-logs-from-nodejs-web-application-using-filebeat). ## Usage ```js const winston = require('winston'); const { ecsFormat } = require('@elastic/ecs-winston-format'); const logger = winston.createLogger({ level: 'info', format: ecsFormat(/* options */), transports: [ new winston.transports.Console() ] }); logger.info('hi'); logger.error('oops there is a problem', { foo: 'bar' }); ``` Running this script (available [here](https://github.com/elastic/ecs-logging-nodejs/blob/main/packages/ecs-winston-format/examples/basic.js)) will produce log output similar to the following: ```cmd % node examples/basic.js {"@timestamp":"2023-10-14T02:14:17.302Z","log.level":"info","message":"hi","ecs.version":"8.10.0"} {"@timestamp":"2023-10-14T02:14:17.304Z","log.level":"error","message":"oops there is a problem","ecs.version":"8.10.0","foo":"bar"} ``` The formatter handles serialization to JSON, so you don’t need to add the [json](https://github.com/winstonjs/logform#json) formatter. As well, a timestamp is automatically generated by the formatter, so you don’t need to add the [timestamp](https://github.com/winstonjs/logform#timestamp) formatter. ## Error logging By default, the formatter will convert an `err` meta field that is an Error instance to [ECS Error fields](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/ecs/ecs-error). For [example](https://github.com/elastic/ecs-logging-nodejs/blob/main/packages/ecs-winston-format/examples/error.js): ```js const winston = require('winston'); const { ecsFormat } = require('@elastic/ecs-winston-format'); const logger = winston.createLogger({ format: ecsFormat(), transports: [ new winston.transports.Console() ] }); const myErr = new Error('boom'); logger.info('oops', { err: myErr }); ``` will yield (pretty-printed for readability): ```cmd % node examples/error.js | jq . { "@timestamp": "2021-01-26T17:25:07.983Z", "log.level": "info", "message": "oops", "error": { "type": "Error", "message": "boom", "stack_trace": "Error: boom\n at Object. (..." }, "ecs.version": "8.10.0" } ``` Special handling of the `err` meta field can be disabled via the `convertErr: false` option: ```js ... const logger = winston.createLogger({ format: ecsFormat({ convertErr: false }), ... ``` ## HTTP Request and Response Logging With the `convertReqRes: true` option, the formatter will automatically convert Node.js core [request](https://nodejs.org/api/http.md#http_class_http_incomingmessage) and [response](https://nodejs.org/api/http.md#http_class_http_serverresponse) objects when passed as the `req` and `res` meta fields, respectively. ```js const http = require('http'); const winston = require('winston'); const { ecsFormat } = require('@elastic/ecs-winston-format'); const logger = winston.createLogger({ level: 'info', format: ecsFormat({ convertReqRes: true }), transports: [ new winston.transports.Console() ] }); const server = http.createServer(handler); server.listen(3000, () => { logger.info('listening at http://localhost:3000') }); function handler (req, res) { res.setHeader('Foo', 'Bar'); res.end('ok'); logger.info('handled request', { req, res }); } ``` This will produce logs with request and response info using [ECS HTTP fields](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/ecs/ecs-http). For [example](https://github.com/elastic/ecs-logging-nodejs/blob/main/packages/ecs-winston-format/examples/http.js): ```cmd % node examples/http.js | jq . ... # run 'curl http://localhost:3000/' { "@timestamp": "2023-10-14T02:15:54.768Z", "log.level": "info", "message": "handled request", "http": { "version": "1.1", "request": { "method": "GET", "headers": { "host": "localhost:3000", "user-agent": "curl/8.1.2", "accept": "*/*" } }, "response": { "status_code": 200, "headers": { "foo": "Bar" } } }, "url": { "path": "/", "full": "http://localhost:3000/" }, "client": { "address": "::ffff:127.0.0.1", "ip": "::ffff:127.0.0.1", "port": 49538 }, "user_agent": { "original": "curl/8.1.2" }, "ecs.version": "8.10.0" } ``` ## Log Correlation with APM This ECS log formatter integrates with [Elastic APM](https://www.elastic.co/apm). If your Node app is using the [Node.js Elastic APM Agent](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/apm/agents/nodejs), then a number of fields are added to log records to correlate between APM services or traces and logging data: - Log statements (e.g. `logger.info(...)`) called when there is a current tracing span will include [tracing fields](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/ecs/ecs-tracing)—`trace.id`, `transaction.id`, `span.id`. - A number of service identifier fields determined by or configured on the APM agent allow cross-linking between services and logs in Kibana—`service.name`, `service.version`, `service.environment`, `service.node.name`. - `event.dataset` enables [log rate anomaly detection](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/solutions/observability/logs/inspect-log-anomalies) in the Elastic Observability app. For example, running [examples/http-with-elastic-apm.js](https://github.com/elastic/ecs-logging-nodejs/blob/main/packages/ecs-winston-format/examples/http-with-elastic-apm.js) and `curl -i localhost:3000/` results in a log record with the following: ```cmd % node examples/http-with-elastic-apm.js | jq . ... "service.name": "http-with-elastic-apm", "service.version": "1.4.0", "service.environment": "development", "event.dataset": "http-with-elastic-apm" "trace.id": "7fd75f0f33ff49aba85d060b46dcad7e", "transaction.id": "6c97c7c1b468fa05" } ``` These IDs match trace data reported by the APM agent. Integration with Elastic APM can be explicitly disabled via the `apmIntegration: false` option, for example: ```js const logger = winston.createLogger({ format: ecsFormat({ apmIntegration: false }), // ... }) ``` ## Limitations and Considerations The [ecs-logging spec](https://github.com/elastic/ecs-logging/tree/main/spec) suggests that the first three fields in log records should be `@timestamp`, `log.level`, and `message`. As of version 1.5.0, this formatter does **not** follow this suggestion. It would be possible but would require creating a new Object in `ecsFields` for each log record. Given that ordering of ecs-logging fields is for **human readability** and does not affect interoperability, the decision was made to prefer performance. ## Reference ### `ecsFormat([options])` - `options` `{type-object}` The following options are supported: - `convertErr` `{type-boolean}` Whether to convert a logged `err` field to ECS error fields. **Default:** `true`. - `convertReqRes` `{type-boolean}` Whether to logged `req` and `res` HTTP request and response fields to ECS HTTP, User agent, and URL fields. **Default:** `false`. - `apmIntegration` `{type-boolean}` Whether to enable APM agent integration. **Default:** `true`. - `serviceName` `{type-string}` A "service.name" value. If specified this overrides any value from an active APM agent. - `serviceVersion` `{type-string}` A "service.version" value. If specified this overrides any value from an active APM agent. - `serviceEnvironment` `{type-string}` A "service.environment" value. If specified this overrides any value from an active APM agent. - `serviceNodeName` `{type-string}` A "service.node.name" value. If specified this overrides any value from an active APM agent. - `eventDataset` `{type-string}` A "event.dataset" value. If specified this overrides the default of using `${serviceVersion}`. Create a formatter for winston that emits in ECS Logging format. This is a single format that handles both [`ecsFields([options])`](#winston-ref-ecsFields) and [`ecsStringify([options])`](#winston-ref-ecsStringify). The following two are equivalent: ```js const { ecsFormat, ecsFields, ecsStringify } = require('@elastic/ecs-winston-format'); const winston = require('winston'); const logger = winston.createLogger({ format: ecsFormat(/* options */), // ... }); const logger = winston.createLogger({ format: winston.format.combine( ecsFields(/* options */), ecsStringify() ), // ... }); ``` ### `ecsFields([options])` - `options` `{type-object}` The following options are supported: - `convertErr` `{type-boolean}` Whether to convert a logged `err` field to ECS error fields. **Default:** `true`. - `convertReqRes` `{type-boolean}` Whether to logged `req` and `res` HTTP request and response fields to ECS HTTP, User agent, and URL fields. **Default:** `false`. - `apmIntegration` `{type-boolean}` Whether to enable APM agent integration. **Default:** `true`. - `serviceName` `{type-string}` A "service.name" value. If specified this overrides any value from an active APM agent. - `serviceVersion` `{type-string}` A "service.version" value. If specified this overrides any value from an active APM agent. - `serviceEnvironment` `{type-string}` A "service.environment" value. If specified this overrides any value from an active APM agent. - `serviceNodeName` `{type-string}` A "service.node.name" value. If specified this overrides any value from an active APM agent. - `eventDataset` `{type-string}` A "event.dataset" value. If specified this overrides the default of using `${serviceVersion}`. Create a formatter for winston that converts fields on the log record info object to ECS Logging format. ### `ecsStringify([options])` Create a formatter for winston that stringifies/serializes the log record to JSON. This is similar to `logform.json()`. They both use the `safe-stable-stringify` package to produce the JSON. Some differences: - This stringifier skips serializing the `level` field, because it is not an ECS field. - Winston provides a `replacer` that converts bigints to strings The argument **for** doing so is that a **JavaScript** JSON parser looses precision when parsing a bigint. The argument against is that a BigInt changes type to a string rather than a number. For now this stringifier does not convert BitInts to strings.