Creating a large number of threads1 and accepting a large number of requests doesn't mean that your server will be able to process the requests
如果你有 N 个线程并且只有 M 个物理处理器/核心,那么每个线程将获得 1 个处理器,如果M >= N
和平均数M / N
处理器如果M < N
。假设您有 N 个请求,每个请求都在 1 个线程上运行,并且每个请求需要R
秒的CPU时间。平均经过时间T
运行一个请求所采取的是T = Min(R, R * N / M) seconds
。很明显,随着你的增加N
(活动线程和活动请求的数量)平均运行时间T
对于每个单独的请求按比例增加。
除此之外,如果您有很多线程,它们都将使用内存,并且所有线程都将竞争对共享数据结构……或数据库的访问。所有这些额外的资源使用和争用都以各种方式增加了整个系统的开销。
所以,我怀疑正在发生的事情是,每个线程都尝试同时处理请求的数量,时间T
开始接近客户端或服务器端请求超时。 (并请注意,调度程序等的变幻莫测意味着任何给定请求的实际时间可能小于或显着大于平均值。)当请求超时时,这反过来会降低获得的请求的吞吐量。已完成,因为对每个超时请求执行的工作(通常)被浪费了。
Unless the requests entail talking to slow external services, I'd advise you to REDUCE the number of threads to no more than 200 ... the Tomcat default2. I expect that this will increase the system throughput. It won't necessarily let you process all of those 1000 requests that were launched in that period, but I predict that it will increase the number of requests that are successfully processed.
1 - Indeed, increasing the number of threads to 1000 doesn't even mean that you will be able to accept 1000 requests. If you have hundreds of threads in RUNNABLE state, it is likely that Tomcat's listener thread (the one that calls ServerSocket.accept()
) will be CPU starved and won't be able to keep up with the request arrival rate.
2 - You will need to do some performance tuning on your system, but I wouldn't be surprised if reducing it even further improved things even more. It will depend on your hardware, your application and (I expect) your backend database.