elasticsearch中设置index.highlight.max_analyzed_offset
Caused by: java.lang.IllegalArgumentException:
The length of [message] field of [VqCNQHkBFO-4K4CU-7tq] doc of [xxxx-node02-xxxx-2021.05.06]
index has exceeded [1000000] - maximum allowed to be analyzed for highlighting.
This maximum can be set by changing the [index.highlight.max_analyzed_offset] index level setting.
For large texts, indexing with offsets or term vectors is recommended!
浏览官网可知：
突出显示没有偏移或术语向量的索引文本，需要在搜索请求期间实时分析内存中的该文本。对于大文本，此分析可能需要大量时间和内存。为了防止这种情况，将分析的最大字符数限制为 1000000。可以使用 index 设置为特定索引更改此默认限制index.highlight.max_analyzed_offset。
max_analyzed_offset 用于高亮请求的可分析最大字符数量，仅对无offsets和term vectors的文本高亮请求有效。默认值1000000
1、通过kibana进行配置：
PUT /_settings
{
"index" : {
"highlight.max_analyzed_offset" : 100000000
}
}
2、通过java进行配置
完整的配置类,我这里采用的是更新设置的请求方法，也可使用新增，主要得根据自己的情况。
因为我的创建索引是在实体类中以@Document(indexName = INDEXNAME,createIndex = true)来创建。所以我设置以更新的方式UpdateSettingsRequest.
import org.apache.http.HttpHost;
import org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.settings.Settings;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;
import java.io.IOException;
import java.time.Duration;
/**
* es配置
*/
@Component
public class ESConfig {
@Value("${es.ip}")
private String ip;
@Value("${es.port}")
private int port;
@Value("${es.indexHighlightMaxAnalyzedOffset}")
private Long indexHighlightMaxAnalyzedOffset;
@Bean
public RestHighLevelClient restHighLevelClient() throws IOException {
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(
new HttpHost(ip, port, "http")
)
.setHttpClientConfigCallback(httpClientBuilder -> {
httpClientBuilder.setKeepAliveStrategy(((httpResponse, httpContext) -> Duration.ofMinutes(5).toMillis()));
httpClientBuilder.disableAuthCaching();
return httpClientBuilder;
})
);
//索引偏移量设置/更新 index.highlight.max_analyzed_offset
UpdateSettingsRequest updateSettingsRequest = new UpdateSettingsRequest(INDEXNAME);
updateSettingsRequest.settings(Settings.builder().put("index.highlight.max_analyzed_offset",indexHighlightMaxAnalyzedOffset));
client.indices().putSettings(updateSettingsRequest, RequestOptions.DEFAULT);
return client;
}
}
index.highlight.max_analyzed_offset的大小虽然不能过小，但建议也不要过大，会对内存带来负担。
