Truncate token filteredit
Truncates tokens that exceed a specified character limit. This limit defaults to
10 but can be customized using the length parameter.
For example, you can use the truncate filter to shorten all tokens to
3 characters or fewer, changing jumping fox to jum fox.
This filter uses Lucene’s TruncateTokenFilter.
Exampleedit
The following analyze API request uses the truncate filter
to shorten tokens that exceed 10 characters in
the quinquennial extravaganza carried on:
GET _analyze
{
"tokenizer" : "whitespace",
"filter" : ["truncate"],
"text" : "the quinquennial extravaganza carried on"
}
The filter produces the following tokens:
[ the, quinquenni, extravagan, carried, on ]
Add to an analyzeredit
The following create index API request uses the
truncate filter to configure a new
custom analyzer.
PUT custom_truncate_example
{
"settings" : {
"analysis" : {
"analyzer" : {
"standard_truncate" : {
"tokenizer" : "standard",
"filter" : ["truncate"]
}
}
}
}
}
Configurable parametersedit
-
length -
(Optional, integer)
Character limit for each token. Tokens exceeding this limit are truncated.
Defaults to
10.
Customizeedit
To customize the truncate filter, duplicate it to create the basis
for a new custom token filter. You can modify the filter using its configurable
parameters.
For example, the following request creates a custom truncate filter,
5_char_trunc, that shortens tokens to a length of 5 or fewer characters:
PUT 5_char_words_example
{
"settings": {
"analysis": {
"analyzer": {
"lowercase_5_char": {
"tokenizer": "lowercase",
"filter": [ "5_char_trunc" ]
}
},
"filter": {
"5_char_trunc": {
"type": "truncate",
"length": 5
}
}
}
}
}