--- title: Source serialization description: Source serialization refers to the process of (de)serializing POCO types in consumer applications as source documents indexed and retrieved from Elasticsearch... url: https://www.elastic.co/elastic/docs-builder/docs/3016/reference/elasticsearch/clients/dotnet/source-serialization products: - Elasticsearch - Elasticsearch .NET Client - Elasticsearch Client --- # Source serialization Source serialization refers to the process of (de)serializing POCO types in consumer applications as source documents indexed and retrieved from Elasticsearch. A source serializer implementation handles serialization, with the default implementation using the `System.Text.Json` library. As a result, you may use `System.Text.Json` attributes and converters to control the serialization behavior. - [Modelling documents with types](#modeling-documents-with-types) - [Default behavior](#default-behaviour) - [Using Elasticsearch types in documents](#elasticsearch-types-in-documents) - [Customizing source serialization](#customizing-source-serialization) - [Using `System.Text.Json` attributes](#system-text-json-attributes) - [Configuring custom `JsonSerializerOptions`](#configuring-custom-jsonserializeroptions) - [Registering custom `System.Text.Json` converters](#registering-custom-converters) - [Creating a custom `Serializer`](#creating-custom-serializers) - [Native AOT](#native-aot) - [Vector data serialization](#vector-data-serialization) - [Opt‑in on document properties](#optin-on-document-properties) - [Configure encodings globally](#configure-encodings-globally) ## Modeling documents with types Elasticsearch provides search and aggregation capabilities on the documents that it is sent and indexes. These documents are sent as JSON objects within the request body of a HTTP request. It is natural to model documents within the Elasticsearch .NET client using [POCOs (*Plain Old CLR Objects*)](https://en.wikipedia.org/wiki/Plain_Old_CLR_Object). This section provides an overview of how types and type hierarchies can be used to model documents. ### Default behaviour The default behaviour is to serialize type property names as camelcase JSON object members. We can model documents using a regular class (POCO). ```csharp public class MyDocument { public string StringProperty { get; set; } } ``` We can then index the an instance of the document into Elasticsearch. ```csharp using Elastic.Clients.Elasticsearch; var document = new MyDocument { StringProperty = "value" }; var indexResponse = await Client.IndexAsync(document); ``` The index request is serialized, with the source serializer handling the `MyDocument` type, serializing the POCO property named `StringProperty` to the JSON object member named `stringProperty`. ```javascript { "stringProperty": "value" } ``` ### Using Elasticsearch types in documents There are various cases where you might have a POCO type that contains an `Elastic.Clients.Elasticsearch` type as one of its properties. For example, consider if you want to use percolation; you need to store Elasticsearch queries as part of the `_source` of your document, which means you need to have a POCO that looks like this: ```csharp using Elastic.Clients.Elasticsearch.QueryDsl; using Elastic.Clients.Elasticsearch.Serialization; public class MyPercolationDocument { [JsonConverter(typeof(RequestResponseConverter))] public Query Query { get; set; } public string Category { get; set; } } ``` Failure to strictly use `RequestResponseConverter` for `Elastic.Clients.Elasticsearch` types will most likely result in problems with document (de-)serialization. ## Customizing source serialization The built-in source serializer handles most POCO document models correctly. Sometimes, you may need further control over how your types are serialized. The built-in source serializer uses the [Microsoft `System.Text.Json` library](https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/overview) internally. You can apply `System.Text.Json` attributes and converters to control the serialization of your document types. ### Using `System.Text.Json` attributes `System.Text.Json` includes attributes that can be applied to types and properties to control their serialization. These can be applied to your POCO document types to perform actions such as controlling the name of a property or ignoring a property entirely. Visit the [Microsoft documentation for further examples](https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/overview). We can model a document to represent data about a person using a regular class (POCO), applying `System.Text.Json` attributes as necessary. ```csharp using System.Text.Json.Serialization; public class Person { [JsonPropertyName("forename")] public string FirstName { get; set; } [JsonIgnore] public int Age { get; set; } } ``` We can then index an instance of the document into Elasticsearch. ```csharp var person = new Person { FirstName = "Steve", Age = 35 }; var indexResponse = await Client.IndexAsync(person); ``` The index request is serialized, with the source serializer handling the `Person` type, serializing the POCO property named `FirstName` to the JSON object member named `forename`. The `Age` property is ignored and does not appear in the JSON. ```javascript { "forename": "Steve" } ``` ### Configuring custom `JsonSerializerOptions` The default source serializer applies a set of standard `JsonSerializerOptions` when serializing source document types. In some circumstances, you may need to override some of our defaults. This is achievable by creating an instance of `DefaultSourceSerializer` and passing an `Action`, which is applied after our defaults have been set. This mechanism allows you to apply additional settings or change the value of our defaults. The `DefaultSourceSerializer` includes a constructor that accepts the current `IElasticsearchClientSettings` and a `configureOptions` `Action`. ```csharp public DefaultSourceSerializer(IElasticsearchClientSettings settings, Action? configureOptions = null); ``` Our application defines the following `Person` class, which models a document we will index to Elasticsearch. ```csharp public class Person { public string FirstName { get; set; } } ``` We want to serialize our source document using Pascal Casing for the JSON properties. Since the options applied in the `DefaultSouceSerializer` set the `PropertyNamingPolicy` to `JsonNamingPolicy.CamelCase`, we must override this setting. After configuring the `ElasticsearchClientSettings`, we index our document to Elasticsearch. ```csharp using System.Text.Json; using Elastic.Transport; using Elastic.Clients.Elasticsearch; using Elastic.Clients.Elasticsearch.Serialization; static void ConfigureOptions(JsonSerializerOptions o) { o.PropertyNamingPolicy = null; } var nodePool = new SingleNodePool(new Uri("http://localhost:9200")); var settings = new ElasticsearchClientSettings( nodePool, sourceSerializer: (defaultSerializer, settings) => new DefaultSourceSerializer(settings, ConfigureOptions)); var client = new ElasticsearchClient(settings); var person = new Person { FirstName = "Steve" }; var indexResponse = await client.IndexAsync(person); ``` The `Person` instance is serialized, with the source serializer serializing the POCO property named `FirstName` using Pascal Case. ```javascript { "FirstName": "Steve" } ``` As an alternative to using a local function, we could store an `Action` into a variable instead, which can be passed to the `DefaultSouceSerializer` constructor. ```csharp Action configureOptions = o => { o.PropertyNamingPolicy = null; } ``` ### Registering custom `System.Text.Json` converters In certain more advanced situations, you may have types which require further customization during serialization than is possible using `System.Text.Json` property attributes. In these cases, the recommendation from Microsoft is to leverage a custom `JsonConverter`. Source document types serialized using the `DefaultSourceSerializer` can leverage the power of custom converters. For this example, our application has a document class that should use a legacy JSON structure to continue operating with existing indexed documents. Several options are available, but we’ll apply a custom converter in this case. Our class is defined, and the `JsonConverter` attribute is applied to the class type, specifying the type of a custom converter. ```csharp using System.Text.Json.Serialization; [JsonConverter(typeof(CustomerConverter))] public class Customer { public string CustomerName { get; set; } public CustomerType CustomerType { get; set; } } public enum CustomerType { Standard, Enhanced } ``` When serializing this class, rather than include a string value representing the value of the `CustomerType` property, we must send a boolean property named `isStandard`. This requirement can be achieved with a custom JsonConverter implementation. ```csharp using System; using System.Text.Json; using System.Text.Json.Serialization; public class CustomerConverter : JsonConverter { public override Customer Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) { var customer = new Customer(); while (reader.Read() && reader.TokenType != JsonTokenType.EndObject) { if (reader.TokenType == JsonTokenType.PropertyName) { if (reader.ValueTextEquals("customerName")) { reader.Read(); customer.CustomerName = reader.GetString(); continue; } if (reader.ValueTextEquals("isStandard")) { reader.Read(); var isStandard = reader.GetBoolean(); if (isStandard) { customer.CustomerType = CustomerType.Standard; } else { customer.CustomerType = CustomerType.Enhanced; } continue; } } } return customer; } public override void Write(Utf8JsonWriter writer, Customer value, JsonSerializerOptions options) { if (value is null) { writer.WriteNullValue(); return; } writer.WriteStartObject(); if (!string.IsNullOrEmpty(value.CustomerName)) { writer.WritePropertyName("customerName"); writer.WriteStringValue(value.CustomerName); } writer.WritePropertyName("isStandard"); if (value.CustomerType == CustomerType.Standard) { writer.WriteBooleanValue(true); } else { writer.WriteBooleanValue(false); } writer.WriteEndObject(); } } ``` We can then index a customer document into Elasticsearch. ```csharp var customer = new Customer { CustomerName = "Customer Ltd", CustomerType = CustomerType.Enhanced }; var indexResponse = await Client.IndexAsync(customer); ``` The `Customer` instance is serialized using the custom converter, creating the following JSON document. ```javascript { "customerName": "Customer Ltd", "isStandard": false } ``` ### Creating a custom `Serializer` Suppose you prefer using an alternative JSON serialization library for your source types. In that case, you can inject an isolated serializer only to be called for the serialization of `_source`, `_fields`, or wherever a user-provided value is expected to be written and returned. Implementing `Elastic.Transport.Serializer` is technically enough to create a custom source serializer. ```csharp using Elastic.Transport; public class VanillaSerializer : Serializer { public override object? Deserialize(Type type, Stream stream) => throw new NotImplementedException(); public override T Deserialize(Stream stream) => throw new NotImplementedException(); public override ValueTask DeserializeAsync(Type type, Stream stream, CancellationToken cancellationToken = default) => throw new NotImplementedException(); public override ValueTask DeserializeAsync(Stream stream, CancellationToken cancellationToken = default) => throw new NotImplementedException(); public override void Serialize(object? data, Type type, Stream stream, SerializationFormatting formatting = SerializationFormatting.None, CancellationToken cancellationToken = default) => throw new NotImplementedException(); public override void Serialize(T data, Stream stream, SerializationFormatting formatting = SerializationFormatting.None) => throw new NotImplementedException(); public override Task SerializeAsync(object? data, Type type, Stream stream, SerializationFormatting formatting = SerializationFormatting.None, CancellationToken cancellationToken = default) => throw new NotImplementedException(); public override Task SerializeAsync(T data, Stream stream, SerializationFormatting formatting = SerializationFormatting.None, CancellationToken cancellationToken = default) => throw new NotImplementedException(); } ``` Registering up the serializer is performed in the `ConnectionSettings` constructor. ```csharp using Elastic.Transport; using Elastic.Clients.Elasticsearch; var nodePool = new SingleNodePool(new Uri("http://localhost:9200")); var settings = new ElasticsearchClientSettings( nodePool, sourceSerializer: (defaultSerializer, settings) => new VanillaSerializer()); var client = new ElasticsearchClient(settings); ``` There are various cases where you might have a POCO type that contains an `Elastic.Clients.Elasticsearch` type as one of its properties (see [Elasticsearch types in documents](#elasticsearch-types-in-documents)). The `SourceSerializerFactory` delegate provides access to the default built-in serializer so you can access it when necessary. For example, consider if you want to use percolation; you need to store Elasticsearch queries as part of the `_source` of your document, which means you need to have a POCO that looks like this. ```csharp using Elastic.Clients.Elasticsearch.QueryDsl; using Elastic.Clients.Elasticsearch.Serialization; public class MyPercolationDocument { [JsonConverter(typeof(RequestResponseConverter))] public Query Query { get; set; } public string Category { get; set; } } ``` A custom serializer would not know how to serialize `Query` or other `Elastic.Clients.Elasticsearch` types that could appear as part of the `_source` of a document. Therefore, your custom `Serializer` would need to store a reference to our built-in serializer and delegate serialization of Elastic types back to it. Depending on the use-case, it might be easier to instead derive from `SystemTextJsonSerializer` and/or create a custom `IJsonSerializerOptionsProvider` implementation. ## Native AOT To use the Elasticsearch client in a [native AOT](https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot) application, `System.Text.Json` must be configured to use [source generation](https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/source-generation). Source generation is always for internal `Elastic.Clients.Elasticsearch` types, but additional steps are required to also enable source generation for user defined types. ```csharp using System.Text.Json.Serialization; [JsonSerializable(typeof(Person), GenerationMode = JsonSourceGenerationMode.Default)] [JsonSerializable(typeof(...), GenerationMode = JsonSourceGenerationMode.Default)] public sealed partial class UserTypeSerializerContext : JsonSerializerContext { } ``` ```csharp using Elastic.Transport; using Elastic.Clients.Elasticsearch; using Elastic.Clients.Elasticsearch.Serialization; var nodePool = new SingleNodePool(new Uri("http://localhost:9200")); var settings = new ElasticsearchClientSettings( nodePool, sourceSerializer: (_, settings) => new DefaultSourceSerializer(settings, UserTypeSerializerContext.Default)); var client = new ElasticsearchClient(settings); var person = new Person { FirstName = "Steve" }; var indexResponse = await client.IndexAsync(person); ``` As an alternative, the `UserTypeSerializerContext` can as well be set when configuring the `JsonSerializerOptions` as described in [Configuring custom `JsonSerializerOptions`](#configuring-custom-jsonserializeroptions): ```csharp static void ConfigureOptions(JsonSerializerOptions o) { o.TypeInfoResolver = UserTypeSerializerContext.Default; } ``` ## Vector data serialization Efficient ingestion of high-dimensional vectors often benefits from compact encodings rather than verbose JSON arrays. The client provides opt‑in converters for vector properties in your source documents that serialize to either hexadecimal or `base64` strings, depending on the vector type and the Elasticsearch version you target. - Float vectors can use `base64` starting from Elasticsearch 9.3.0. - Byte/bit vectors can use hexadecimal strings starting from Elasticsearch 8.14.0 and `base64` starting from Elasticsearch 9.3.0. - The legacy representation (JSON arrays) remains available for backwards compatibility. Base64 is the preferred format for high‑throughput indexing because it minimizes payload size and reduces JSON parsing overhead. ### Opt‑in on document properties Vector encodings are opt‑in. Apply a `System.Text.Json` `JsonConverter` attribute on the vector property of your POCO. For best performance, model the properties as `ReadOnlyMemory`. ```csharp using System; using System.Text.Json.Serialization; using Elastic.Clients.Elasticsearch.Serialization; public class ImageEmbedding { [JsonConverter(typeof(FloatVectorDataConverter))] public ReadOnlyMemory Vector { get; set; } } public class ByteSignature { [JsonConverter(typeof(ByteVectorDataConverter))] public ReadOnlyMemory Signature { get; set; } } ``` Without these attributes, vectors are serialized using the default source serializer behavior. ### Configure encodings globally When the opt‑in attributes are present, you can control the actual wire encoding globally via `ElasticsearchClient` settings on a per‑type basis: - `FloatVectorDataEncoding`: controls float vector encoding (legacy arrays or `base64`). - `ByteVectorDataEncoding`: controls byte/bit vector encoding (legacy arrays, hexadecimal, or `base64`). These settings allow a single set of document types to work against mixed clusters. For example, a library using the 8.19.x client can talk to both 8.x and 9.x servers and dynamically opt out of `base64` on older servers without maintaining duplicate POCOs (with/without converter attributes). Set the encoding based on your effective server version: - Float vectors: use `base64` for 9.3.0+; otherwise use legacy arrays. - Byte/bit vectors: prefer `base64` for 9.3.0+; use hexadecimal for 8.14.0–9.2.x; otherwise use legacy arrays.