Apache HttpClient 未显示响应的 Content-Length 和 Content-Encoding 标头

我安装了Apache httpcomponents-client-5.0.x https://hc.apache.org/httpcomponents-client-5.0.x/index.html在查看 http 响应的标头时，我很惊讶它没有显示Content-Length and Content-Encodingheaders，这是我用于测试的代码

import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
import org.apache.hc.client5.http.impl.classic.CloseableHttpResponse;
import org.apache.hc.client5.http.impl.classic.HttpClients;
import com.sun.net.httpserver.Headers;

CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet request = new HttpGet(new URI("https://www.example.com"));
CloseableHttpResponse response = httpclient.execute(request);
Header[] responseHeaders = response.getHeaders();
for(Header header: responseHeaders) {               
    System.out.println(header.getName());
}
// this prints all the headers except 
// status code header
// Content-Length
// Content-Encoding

无论我尝试什么，我都会得到相同的结果，就像这样

Iterator<Header> headersItr = response.headerIterator();
while(headersItr.hasNext()) {
    Header header = headersItr.next();
    System.out.println(header.getName());
}

Or this

HttpEntity entity = response.getEntity();
System.out.println(entity.getContentEncoding()); // NULL
System.out.println(entity.getContentLength());   // -1

根据这个问题 https://stackoverflow.com/questions/18726892/apache-httpclient-response-content-length-returns-1这个问题是 6 年前提出的，即使对于旧版本的 Apache HttpClient，这似乎也是一个老问题。

当然，服务器实际上返回了 Wireshark 确认的那些标头，并且 Apache HttpClient 会自行记录

2020-04-03 07:59:09,106 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << HTTP/1.1 200 OK
2020-04-03 07:59:09,106 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Content-Encoding: gzip
2020-04-03 07:59:09,106 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Accept-Ranges: bytes
2020-04-03 07:59:09,107 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Age: 451956
2020-04-03 07:59:09,107 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Cache-Control: max-age=604800
2020-04-03 07:59:09,107 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Content-Type: text/html; charset=UTF-8
2020-04-03 07:59:09,107 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Date: Fri, 03 Apr 2020 05:59:09 GMT
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Etag: "3147526947+gzip"
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Expires: Fri, 10 Apr 2020 05:59:09 GMT
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Server: ECS (dcb/7EEB)
2020-04-03 07:59:09,108 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Vary: Accept-Encoding
2020-04-03 07:59:09,109 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << X-Cache: HIT
2020-04-03 07:59:09,109 DEBUG [org.apache.hc.client5.http.headers] http-outgoing-0 << Content-Length: 648

BTW, java.net.http图书馆称为JDK HttpClient https://docs.oracle.com/en/java/javase/11/docs/api/java.net.http/java/net/http/HttpClient.html效果很好并显示所有标题。

我是否做错了什么，或者我应该报告多年来一直存在的错误？

HttpComponents 提交者在这里...

你没有仔细注意Dave G所说的话。默认情况下，HttpClientBuilder将启用透明解压缩，并且您不再看到某些标头的原因是here https://github.com/apache/httpcomponents-client/blob/92100e13a6aeb9ce04d23df2561c01b330cb961a/httpclient5/src/main/java/org/apache/hc/client5/http/impl/classic/ContentCompressionExec.java#L141-L145:

if (decoderFactory != null) {
  response.setEntity(new DecompressingEntity(response.getEntity(), decoderFactory));
  response.removeHeaders(HttpHeaders.CONTENT_LENGTH);
  response.removeHeaders(HttpHeaders.CONTENT_ENCODING);
  response.removeHeaders(HttpHeaders.CONTENT_MD5);
} ...

对于JDK HttpClient，它不会执行任何透明解压缩，因此您看到的是压缩流的长度。需要你自己解压。

卷曲提交者在这里...

I have 提出了一个问题 https://github.com/curl/curl/issues/5182 too.

更新：23 年 2 月 3 日禁用自动解压的密码是：

CloseableHttpClient httpclient = HttpClients.createSimple();
// OR
CloseableHttpClient httpclient = HttpClients.custom().disableContentCompression().build();