Optimizing text for searches with text analyzers

OpenSearch provides several text analyzers to convert your structured text into the format that works best for your searches.

OpenSearch supports the following text analyzers:

1. Standard analyzer – Parses strings into terms at word boundaries per the Unicode text segmentation algorithm. It removes most, but not all, punctuation. It converts strings to lowercase. You can remove stop words if you turn on that option, but it does not remove stop words by default.
2. Simple analyzer – Converts strings to lowercase and removes non-letter characters when it splits a string into tokens on any non-letter character.
3. Whitespace analyzer – Parses strings into terms between each whitespace.
4. Keyword analyzer – Receives a string as input and outputs the entire string as one term.
5. Pattern analyzer – Splits strings into terms using regular expressions and supports converting strings to lowercase. It also supports removing stop words.
6. Language analyzer – Provides analyzers specific to multiple languages.
7. Fingerprint analyzer – Creates a fingerprint to use as a duplicate detector.

If you want to use a text analyzer, specify the name of the analyzer for the analyzer field: standard, simple, whitespace, stop, keyword, pattern, fingerprint, or language.

Each analyzer consists of one tokenizer and zero or more token filters. Different analyzers have different character filters, tokenizers, and token filters. To pre-process the string before the tokenizer is applied, you can use one or more character filters.

Example: Specify the standard analyzer in a simple query

To use the analyzer when you map an index, specify the value within your query. For example, to map your index with the French language analyzer, specify the value for the analyzer field:

Sample Request

The following query maps an index with the language analyzer set to french: