从 Tomcat 7 升级到 8 时,我遇到了同样的问题:持续出现大量有关缓存的日志警告。
1.简答
将其添加到Context https://tomcat.apache.org/tomcat-9.0-doc/config/context.html#Attributes你的 xml 元素$CATALINA_BASE/conf/context.xml
:
<!-- The default value is 10240 kbytes, even when not added to context.xml.
So increase it high enough, until the problem disappears, for example set it to
a value 5 times as high: 51200. -->
<Resources cacheMaxSize="51200" />
所以默认是10240
(10 MB),因此设置比此更大的大小。然后调整到警告消失的最佳设置。
请注意,在流量较高的情况下,警告可能会再次出现。
1.1 原因(简述)
该问题是由于 Tomcat 无法达到其目标缓存大小而导致的,因为缓存条目小于这些条目的 TTL。因此,Tomcat 没有足够的缓存条目可能会过期,因为它们太新鲜了,因此它无法释放足够的缓存,从而输出警告。
这个问题在 Tomcat 7 中没有出现,因为 Tomcat 7 在这种情况下根本不输出警告。 (导致你和我在没有得到通知的情况下使用糟糕的缓存设置。)
与缓存的大小和 TTL 相比,在相对较短的时间内接收到相对大量的资源 HTTP 请求(通常是静态的)时,就会出现此问题。如果缓存达到最大值(默认为 10mb),并且超过其大小的 95% 包含新鲜缓存条目(新鲜意味着缓存中的时间少于 5 秒),那么您将收到 Tomcat 尝试的每个 webResource 的警告消息加载到缓存中。
1.2 可选信息
如果您需要在正在运行的服务器上调整cacheMaxSize而不重新启动它,请使用JMX。
最快的修复方法是完全禁用缓存:<Resources cachingAllowed="false" />
,但这并不是最理想的,因此请按照我刚才的描述增加cacheMaxSize。
2. 长答案
2.1 背景信息
A 网络资源 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/WebResource.java是 Web 应用程序中的文件或目录。出于性能原因,Tomcat 可以缓存 WebSource。这静态资源缓存的最大值 https://tomcat.apache.org/tomcat-8.0-doc/config/resources.html#Common_Attributes(所有资源总计)默认为 10240 kbyte (10 MB)。当请求 webResource 时(例如加载静态图像时),webResource 会加载到缓存中,然后将其称为缓存条目。
每个缓存条目都有一个TTL https://tomcat.apache.org/tomcat-8.0-doc/config/resources.html#Common_Attributes(time to live),即允许缓存条目在缓存中停留的时间。当 TTL 到期时,缓存条目就可以从缓存中删除。 cacheTTL 的默认值为 5000 毫秒(5 秒)。
关于缓存还有更多内容要讲,但这与问题无关。
2.2 产生原因
以下代码来自缓存类 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java详细展示了缓存策略:
152 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#152 // Content will not be cached but we still need metadata size
153 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#153 long delta = cacheEntry.getSize http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/CachedResource.java#CachedResource.getSize%28%29();
154 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#154 size.addAndGet http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7u40-b43/java/util/concurrent/atomic/AtomicLong.java#AtomicLong.addAndGet%28long%29(delta);
156 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#156 if (size.get http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7u40-b43/java/util/concurrent/atomic/AtomicLong.java#AtomicLong.get%28%29() > maxSize) {
157 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#157 // Process resources unordered for speed. Trades cache
158 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#158 // efficiency (younger entries may be evicted before older
159 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#159 // ones) for speed since this is on the critical path for
160 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#160 // request processing
161 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#161 long targetSize =
162 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#162 maxSize * (100 - TARGET_FREE_PERCENT_GET) / 100;
163 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#163 long newSize = evict http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#Cache.evict%28long%2Cjava.util.Iterator%29(
164 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#164 targetSize, resourceCache.values http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7u40-b43/java/util/Map.java#Map.values%28%29().iterator http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7u40-b43/java/util/Collection.java#Collection.iterator%28%29());
165 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#165 if (newSize > maxSize) {
166 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#166 // Unable to create sufficient space for this resource
167 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#167 // Remove it from the cache
168 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#168 removeCacheEntry http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#Cache.removeCacheEntry%28java.lang.String%29(path);
169 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#169 log.warn http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-juli/8.0.24/org/apache/juli/logging/Log.java#Log.warn%28java.lang.Object%29(sm.getString http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-util/8.0.24/org/apache/tomcat/util/res/StringManager.java#StringManager.getString%28java.lang.String%2Cjava.lang.Object%5B%5D%29("cache.addFail", path));
170 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#170 }
171 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#171 }
加载 webResource 时,代码会计算缓存的新大小。如果计算出的大小大于默认的最大大小,则必须删除一个或多个缓存条目,否则新的大小将超过最大值。因此,代码将计算“targetSize”,这是缓存希望保持的大小(作为最佳值),默认情况下为最大值的 95%。为了达到此目标大小,必须从缓存中删除/逐出条目。这是使用以下代码完成的:
215 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#215 private long evict(long targetSize, Iterator http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7u40-b43/java/util/Iterator.java#Iterator<CachedResource http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/CachedResource.java#CachedResource> iter) {
217 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#217 long now = System.currentTimeMillis http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7u40-b43/java/lang/System.java#System.currentTimeMillis%28%29();
219 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#219 long newSize = size.get http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7u40-b43/java/util/concurrent/atomic/AtomicLong.java#AtomicLong.get%28%29();
221 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#221 while (newSize > targetSize && iter.hasNext http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7u40-b43/java/util/Iterator.java#Iterator.hasNext%28%29()) {
222 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#222 CachedResource http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/CachedResource.java#CachedResource resource = iter.next http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7u40-b43/java/util/Iterator.java#Iterator.next%28%29();
224 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#224 // Don't expire anything that has been checked within the TTL
225 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#225 if (resource.getNextCheck http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/CachedResource.java#CachedResource.getNextCheck%28%29() > now) {
226 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#226 continue;
227 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#227 }
229 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#229 // Remove the entry from the cache
230 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#230 removeCacheEntry http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#Cache.removeCacheEntry%28java.lang.String%29(resource.getWebappPath http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/CachedResource.java#CachedResource.getWebappPath%28%29());
232 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#232 newSize = size.get http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7u40-b43/java/util/concurrent/atomic/AtomicLong.java#AtomicLong.get%28%29();
233 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#233 }
235 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#235 return newSize;
236 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#236 }
因此,当缓存条目的 TTL 过期且尚未达到 targetSize 时,该缓存条目将被删除。
在尝试通过逐出缓存条目来释放缓存后,代码将执行以下操作:
165 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#165 if (newSize > maxSize) {
166 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#166 // Unable to create sufficient space for this resource
167 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#167 // Remove it from the cache
168 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#168 removeCacheEntry http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#Cache.removeCacheEntry%28java.lang.String%29(path);
169 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#169 log.warn http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-juli/8.0.24/org/apache/juli/logging/Log.java#Log.warn%28java.lang.Object%29(sm.getString http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-util/8.0.24/org/apache/tomcat/util/res/StringManager.java#StringManager.getString%28java.lang.String%2Cjava.lang.Object%5B%5D%29("cache.addFail", path));
170 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/8.0.24/org/apache/catalina/webresources/Cache.java#170 }
因此,如果尝试释放缓存后,大小仍然超过最大值,则会显示无法释放的警告消息:
cache.addFail=Unable to add the resource at [{0}] to the cache for web application [{1}] because there was insufficient free space available after evicting expired cache entries - consider increasing the maximum size of the cache
2.3 问题
正如警告消息所说,问题是
驱逐过期的缓存条目后可用空间不足 - 考虑增加缓存的最大大小
如果您的 Web 应用程序在短时间内(5 秒)加载大量未缓存的 WebResources(大约最大缓存,默认为 10mb),那么您将收到警告。
令人困惑的部分是 Tomcat 7 没有显示警告。这仅仅是由以下 Tomcat 7 代码引起的:
1606 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ProxyDirContext.java#1606 // Add new entry to cache
1607 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ProxyDirContext.java#1607 synchronized (cache) {
1608 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ProxyDirContext.java#1608 // Check cache size, and remove elements if too big
1609 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ProxyDirContext.java#1609 if ((cache.lookup http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ResourceCache.java#ResourceCache.lookup%28java.lang.String%29(name) == null) && cache.allocate http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ResourceCache.java#ResourceCache.allocate%28int%29(entry.size)) {
1610 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ProxyDirContext.java#1610 cache.load http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ResourceCache.java#ResourceCache.load%28org.apache.naming.resources.CacheEntry%29(entry);
1611 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ProxyDirContext.java#1611 }
1612 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ProxyDirContext.java#1612 }
结合:
231 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ResourceCache.java#231 while (toFree > 0) {
232 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ResourceCache.java#232 if (attempts == maxAllocateIterations) {
233 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ResourceCache.java#233 // Give up, no changes are made to the current cache
234 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ResourceCache.java#234 return false;
235 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.tomcat/tomcat-catalina/7.0.0/org/apache/naming/resources/ResourceCache.java#235 }
因此,当 Tomcat 7 无法释放缓存时,它根本不会输出任何警告,而 Tomcat 8 将输出警告。
因此,如果您使用具有与 Tomcat 7 相同的默认缓存配置的 Tomcat 8,并且您在 Tomcat 8 中收到警告,那么您(和我的)Tomcat 7 的缓存设置在没有警告的情况下表现不佳。
2.4 解决方案
有多种解决方案:
- 增加缓存(推荐)
- 降低 TTL(不推荐)
- 禁止缓存日志警告(不推荐)
- 禁用缓存
2.4.1.增加缓存(推荐)
正如这里所描述的:http://tomcat.apache.org/tomcat-8.0-doc/config/resources.html http://tomcat.apache.org/tomcat-8.0-doc/config/resources.html
通过增加<Resources cacheMaxSize="XXXXX" />
内Context
元素在$CATALINA_BASE/conf/context.xml
,其中“XXXXX”代表增加的缓存大小,以千字节为单位指定。默认值为 10240(10 MB),因此请设置比该值更高的大小。
您必须调整以获得最佳设置。请注意,当流量/资源请求突然增加时,问题可能会再次出现。
为了避免每次想要尝试新的缓存大小时都必须重新启动服务器,您可以使用 JMX 更改它而无需重新启动。
To 启用 JMX https://tomcat.apache.org/tomcat-8.0-doc/config/listeners.html#JMX_Remote_Lifecycle_Listener_-_org.apache.catalina.mbeans.JmxRemoteLifecycleListener,将其添加到$CATALINA_BASE/conf/server.xml
内Server
元素:<Listener className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener" rmiRegistryPortPlatform="6767" rmiServerPortPlatform="6768" />
并下载catalina-jmx-remote.jar
from https://tomcat.apache.org/download-80.cgi https://tomcat.apache.org/download-80.cgi并把它放进去$CATALINA_HOME/lib
。
然后使用 jConsole(默认随 Java JDK 附带)通过 JMX 连接到服务器,并查看设置以在服务器运行时增加缓存大小。这些设置的更改应立即生效。
2.4.2.降低 TTL(不推荐)
降低cacheTtl
值低于 5000 毫秒并调整为最佳设置。
例如:<Resources cacheTtl="2000" />
这实际上归结为在内存中拥有并填充缓存而不使用它。
2.4.3.禁止缓存日志警告(不推荐)
配置日志记录以禁用记录器org.apache.catalina.webresources.Cache
.
有关登录 Tomcat 的更多信息:http://tomcat.apache.org/tomcat-8.0-doc/logging.html http://tomcat.apache.org/tomcat-8.0-doc/logging.html
2.4.4.禁用缓存
您可以通过设置禁用缓存cachingAllowed http://tomcat.apache.org/tomcat-8.0-doc/config/resources.html to false
.
<Resources cachingAllowed="false" />
尽管我记得在 Tomcat 8 的 beta 版本中,我使用 JMX 来禁用缓存。 (不确定具体原因,但通过 server.xml 禁用缓存可能会出现问题。)