Python greenthread 和 requests 模块每次只处理 10 个请求?

2024-04-02

我正在使用Python 2.7.5。并尝试使用协程绿色线程(Pythoneventlet)和Pythonrequests模块来加速我的 REST API 请求。

我知道Pythonrequests https://requests.readthedocs.io/en/latest/user/advanced/模块用途poolmanager对象(来自urllib3module) 来维护连接,我设置了 poolmanagerDEFAULT_POOLSIZE = 1000.

然后,monkeypatched 我的 python 模块位于__init__.py,并生成 1000 个 greenthread 以将 PATCH 请求(REST API)发送到我的设备。我的设备(F5 bigip device)可以处理50req/s。

这是日志,我发现问题所在。

2022-07-13 09:54:31.893 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 1
2022-07-13 09:54:31.893 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.894 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 2
2022-07-13 09:54:31.895 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.895 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 3
2022-07-13 09:54:31.896 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.896 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 4
2022-07-13 09:54:31.897 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.897 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 5
2022-07-13 09:54:31.898 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.898 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 6
2022-07-13 09:54:31.899 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.900 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 7
2022-07-13 09:54:31.900 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.901 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 8
2022-07-13 09:54:31.901 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.902 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 9
2022-07-13 09:54:31.902 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.903 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 10
2022-07-13 09:54:31.903 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.904 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 11
2022-07-13 09:54:31.904 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.905 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 12
2022-07-13 09:54:31.906 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.906 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 13
2022-07-13 09:54:31.907 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.907 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 14
2022-07-13 09:54:31.908 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.909 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 15
2022-07-13 09:54:31.909 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.910 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 16
2022-07-13 09:54:31.910 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.911 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 17
2022-07-13 09:54:31.911 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.912 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 18
2022-07-13 09:54:31.912 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.913 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 19
2022-07-13 09:54:31.913 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.914 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 20
2022-07-13 09:54:31.914 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.915 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 21
2022-07-13 09:54:31.915 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.916 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 22
2022-07-13 09:54:31.916 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.917 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 23
2022-07-13 09:54:31.917 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.918 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 24
2022-07-13 09:54:31.918 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.919 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 25
2022-07-13 09:54:31.919 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.920 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 26
2022-07-13 09:54:31.920 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.921 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 27
2022-07-13 09:54:31.921 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.922 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 28
2022-07-13 09:54:31.922 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.923 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 29
2022-07-13 09:54:31.923 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.924 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 30
2022-07-13 09:54:31.925 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.925 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 31
2022-07-13 09:54:31.926 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.927 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 32
2022-07-13 09:54:31.927 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.928 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 33
2022-07-13 09:54:31.928 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.929 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 34
2022-07-13 09:54:31.929 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.930 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 35
2022-07-13 09:54:31.930 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:31.931 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse running green thread is 36
2022-07-13 09:54:31.931 31706 INFO f5_openstack_agent.lbaasv2.drivers.bigip.vs_connection [req-f7db08a8-432a-413f-b4c9-f9eb90de9613 bcd6de7aaddc49e386a664d0a20efcff 8a2d7296ae9b4bd4a412eb3cb9aa680e - - -] time elapse waiting  green thread is 0
2022-07-13 09:54:32.207 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:00.273908 sec
2022-07-13 09:54:32.209 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 0.276766 sec
2022-07-13 09:54:32.302 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:00.337186 sec
2022-07-13 09:54:32.303 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 0.340059 sec
2022-07-13 09:54:32.329 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:00.381034 sec
2022-07-13 09:54:32.331 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 0.384191 sec
2022-07-13 09:54:32.365 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:00.390816 sec
2022-07-13 09:54:32.367 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 0.394396 sec
2022-07-13 09:54:32.393 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:00.458491 sec
2022-07-13 09:54:32.396 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 0.461856 sec
2022-07-13 09:54:32.419 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:00.457652 sec
2022-07-13 09:54:32.421 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 0.460368 sec
2022-07-13 09:54:32.480 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:00.507334 sec
2022-07-13 09:54:32.481 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 0.510402 sec
2022-07-13 09:54:32.519 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:00.569053 sec
2022-07-13 09:54:32.521 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 0.571782 sec
2022-07-13 09:54:32.555 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:00.587162 sec
2022-07-13 09:54:32.558 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 0.590332 sec
2022-07-13 09:54:32.592 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:00.637019 sec
2022-07-13 09:54:32.595 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 0.640877 sec





2022-07-13 09:54:38.395 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:06.436821 sec
2022-07-13 09:54:38.396 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 6.439384 sec
2022-07-13 09:54:38.466 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:06.512065 sec
2022-07-13 09:54:38.468 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 6.515319 sec
2022-07-13 09:54:38.531 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:06.553081 sec
2022-07-13 09:54:38.533 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 6.556024 sec
2022-07-13 09:54:38.583 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:06.631171 sec
2022-07-13 09:54:38.585 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 6.634816 sec
2022-07-13 09:54:38.612 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:06.632594 sec
2022-07-13 09:54:38.614 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 6.635825 sec
2022-07-13 09:54:38.659 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:06.700147 sec
2022-07-13 09:54:38.661 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 6.703008 sec
2022-07-13 09:54:38.686 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:06.749487 sec
2022-07-13 09:54:38.688 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 6.753057 sec
2022-07-13 09:54:38.749 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:06.799416 sec
2022-07-13 09:54:38.750 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 6.801891 sec
2022-07-13 09:54:38.775 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:06.814294 sec
2022-07-13 09:54:38.776 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 6.817045 sec
2022-07-13 09:54:38.809 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:06.824674 sec
2022-07-13 09:54:38.811 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 6.827580 sec




2022-07-13 09:54:43.222 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:11.283620 sec
2022-07-13 09:54:43.224 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 11.287391 sec
2022-07-13 09:54:43.259 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:11.296450 sec
2022-07-13 09:54:43.261 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 11.299664 sec
2022-07-13 09:54:44.608 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:12.668820 sec
2022-07-13 09:54:44.610 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 12.672017 sec
2022-07-13 09:54:44.656 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:12.711311 sec
2022-07-13 09:54:44.658 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 12.714831 sec
2022-07-13 09:54:44.724 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:12.777329 sec
2022-07-13 09:54:44.725 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 12.780314 sec
2022-07-13 09:54:44.779 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:12.803660 sec
2022-07-13 09:54:44.781 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 12.806557 sec
2022-07-13 09:54:44.834 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:12.862764 sec
2022-07-13 09:54:44.836 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 12.865921 sec
2022-07-13 09:54:44.861 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:12.895232 sec
2022-07-13 09:54:44.864 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 12.899029 sec
2022-07-13 09:54:44.904 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:12.921001 sec
2022-07-13 09:54:44.906 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 12.924201 sec
2022-07-13 09:54:44.947 31706 INFO requests.sessions [-] send request {"connectionLimit": 1388} time elapse is 0:00:13.005864 sec
2022-07-13 09:54:44.949 31706 INFO f5.bigip.tm.ltm.virtual [-] method modify time elapse is 13.008865 sec
...

代码是这样的

                pool = eventlet.greenpool.GreenPool()
                for vs in vss:
                    try:
                        # vs.modify(connectionLimit=limit)
                        pool.spawn(vs.modify, connectionLimit=limit)
                        nums = pool.running()
                        waits = pool.waiting()
                        LOG.info("time elapse running green thread is %d" % nums)
                        LOG.info("time elapse waiting  green thread is %d" % waits)
                    except Exception as ex:
                        LOG.error(
                            "Fail to refresh virtual server %s"
                            " connection limit %s." % (vs.name, limit)
                        )

如上面的日志所示,需要处理 36 个请求。所有 36 个请求似乎都是一次性发出的。

我也计时requests.sessions发送方法。看起来每 10 个请求作为一个批次进行响应,并且每个批次比前一批延迟大约 6 秒。

起初,我认为我没有对我的模块进行猴子修补,但我检查了init.py 文件,以及eventlet猴子补丁就在那里。

我有点困惑,看起来是非阻塞的,但是每10个请求都有响应并被阻塞。另外,poolmanager连接数是1000,比10大很多。

为什么以这种方式回复请求,任何人都可以给我提示吗?这可能是我的设备问题吗?

--------更新一些信息--------

正如我提到的,我改变了poolmanager DEFAULT_POOLSIZE = 1000这影响了pool_maxsize。然后我再次进行测试,我将 pool_maxsize 更改为1.

虽然日志显示2022-07-18 08:19:52.423 15412 WARNING requests.packages.urllib3.connectionpool [-] Connection pool is full, discarding connection: 100.123.73.244: Full,过程变得更快。

重建与设备的连接似乎比维护连接池并重用它们更有效。

class HTTPAdapter(BaseAdapter):
    """The built-in HTTP Adapter for urllib3.

    Provides a general-case interface for Requests sessions to contact HTTP and
    HTTPS urls by implementing the Transport Adapter interface. This class will
    usually be created by the :class:`Session <Session>` class under the
    covers.

    :param pool_connections: The number of urllib3 connection pools to cache.
    :param pool_maxsize: The maximum number of connections to save in the pool.
    :param max_retries: The maximum number of retries each connection
        should attempt. Note, this applies only to failed DNS lookups, socket
        connections and connection timeouts, never to requests where data has
        made it to the server. By default, Requests does not retry failed
        connections. If you need granular control over the conditions under
        which we retry a request, import urllib3's ``Retry`` class and pass
        that instead.
    :param pool_block: Whether the connection pool should block for connections.

    Usage::

      >>> import requests
      >>> s = requests.Session()
      >>> a = requests.adapters.HTTPAdapter(max_retries=3)
      >>> s.mount('http://', a)
    """
    __attrs__ = ['max_retries', 'config', '_pool_connections', '_pool_maxsize',
                 '_pool_block']

    def __init__(self, pool_connections=DEFAULT_POOLSIZE,
                 pool_maxsize=DEFAULT_POOLSIZE, max_retries=DEFAULT_RETRIES,
                 pool_block=DEFAULT_POOLBLOCK):
        pool_connections = 1
        # I change this pool_maxsize value from 1000 to 1
        # pool_maxsize = 1000
        pool_maxsize = 1
        if max_retries == DEFAULT_RETRIES:
            self.max_retries = Retry(0, read=False)
        else:
            self.max_retries = Retry.from_int(max_retries)
        self.config = {}
        self.proxy_manager = {}

        super(HTTPAdapter, self).__init__()

        self._pool_connections = pool_connections
        self._pool_maxsize = pool_maxsize
        self._pool_block = pool_block

        self.init_poolmanager(pool_connections, pool_maxsize, block=pool_block)

我很好奇为什么它会这样。是F5 REST服务器问题还是我的代码问题...


我想我终于找到了这个历史问题的答案。这几天找到了BigIP的这个配置。它确实有连接限制,如下图所示。

本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)

Python greenthread 和 requests 模块每次只处理 10 个请求? 的相关文章

随机推荐

  • 将字符串编码为十六进制

    我有将字符串转换为十六进制的函数 function encode str str encodeURIComponent str split join return str toLowerCase example 守护村子 alert enc
  • 在vim中搜索并替换选定的文件路径

    我经常处理内部充满文件路径的文件 我希望能够在可视模式下快速选择我的文件路径并将其替换为其他文件路径 例如我有这样的文件 balvadsd mnt Windows Documents and Settings stuff file exe
  • VBA - CallByName 不接受变体参数

    解决方案 只需在 CallByName 语句中的 Value 两边加上括号即可强制对其求值 Ex CallByName MobClass TargetData vbLet Value 感谢来自另一篇文章的 Rory 我可能会删除该文章 因为
  • 使用蒙特卡罗模拟计算 π 的局限性

    我问过一个与此非常相似的问题 所以我会在最后提到以前的解决方案 我有一个website https monte carlo sjorsvanholst nl使用客户端的 CPU 计算 同时将其存储在服务器上 到目前为止我已经得到 701 7
  • 在客户端 SAPUI5 中仅过滤 sap.m.List

    我正在寻找仅在客户端过滤列表 并让其他类似表等在服务器端过滤 是否有任何选项可以添加到列表中以在客户端进行过滤 问候 您可以使用操作模式v2 ODataModel 的参数 该参数可以设置为模型级别 https openui5 hana on
  • 如何应用 django 补丁

    我想对这个错误应用补丁 http code djangoproject com ticket 13095 http code djangoproject com ticket 13095 但我以前从未这样做过 我不知道从哪里开始 谁能给我指
  • 如何使用 Spring JPA 仅获取实体的选定属性?

    我在我的项目中使用 Spring Boot 1 3 3 RELEASE 和 Hibernate JPA 我的实体看起来像这样 Data NoArgsConstructor Entity Table name rule public clas
  • 读取 jar 文件中的 zip 文件

    之前我们的 Web 应用程序中有一些 zip 文件 我们想要解析 zip 文件中的特定文本文档 这不是问题 URL url getClass getResource zipfile ZipFile zip new ZipFile url g
  • 序列或批处理项目 DataGridView

    我有一个包含 940000 行的大型 DataGridView 哎呀 通过解析 csv 文件填充 DataGridView 有一个名为序列的列 编号为 1 到 940000 我试图做的是重新编号序列以溢出到序列中DataGridView 中
  • Angular Reactive Form 提交并明确验证

    我有一个反应形式
  • 如何监控SciPy.odeint的进程?

    SciPy 可以通过 scipy integrate odeint 或其他包求解 ode 方程 但它在函数完全求解后给出结果 但是 如果 ode 函数非常复杂 程序将花费大量时间 一两天 才能给出整个结果 那么我如何监控求解方程的步骤 当方
  • 将动态添加到 IList 失败

    在下面的代码示例中调用l Add s and c Add s 是成功的 但对于泛型时会失败IList
  • com.jcraft.jsch.JSchException:java.net.ConnectException:连接被拒绝:连接

    我知道有重复的 gt gt gt 从重复的 gt gt gt 复制 只要您的本地计算机有一个正在运行的 SSH 服务器 它说 只要你的本地机器有运行的 SSH 服务器 但我不知道如何运行 SSH 服务器 我打开我的腻子 双击它 不确定这是否
  • utf-8特殊字符不显示[重复]

    这个问题在这里已经有答案了 我将网站从本地测试服务器移至 NameCheap 共享主机 现在我遇到了问题 某些页面无法正确显示 utf 8 特殊字符 而是显示问号 所有页面均采用 utf 8 编码 所有数据库表也是如此 奇怪的是 有些页面可
  • 无法加载 libgdx 的共享库 box2d

    我有一个使用 Maven 构建的 libGDX 项目 它过去运行良好 但最近它停止工作 因为 libGDX 将 box2d 移动为扩展 我将扩展作为依赖项添加到项目的核心 就像任何其他依赖项一样
  • 在 Python 中将 USB 视频捕获设备友好名称与 OpenCV 端口号相关联

    我想在 Windows 平台上使用 Python 获取外部 USB 视频捕获设备的友好名称和 USB 端口号 我正在使用 OpenCV 从 USB 捕获设备捕获视频 OpenCV 将 USB 端口称为 1 找到的第一个工作摄像头 0 对我来
  • 为Buildbot添加自定义功能

    我用 python 编写了一个函数 我想让 Buildbot 来执行这个函数 当它收到 构建 命令时 我之前使用过 factory addStep 通过命令行添加新命令 但我不确定如何向 Buildbot 添加 python 函数 谢谢 如
  • Symfony2:如何禁用表单级联验证?

    我有一个带有一个实体表单字段的表单 当我打电话时 form gt isValid symfony 验证与该字段关联的对象 我知道问题是fixed http github com symfony symfony commit 0c70a410
  • Android 模拟器上的布局看起来不像我在 Android 应用程序项目中所做的布局

    Android 模拟器上的布局看起来不像我在 Android 应用程序项目中所做的布局 我在 Eclipse 中做了一个带有两个单选按钮和两个旋转器的布局 但是当我在 Android 模拟器上运行它时 旋转器在箭头符号旁边各有两个额外的单选
  • Python greenthread 和 requests 模块每次只处理 10 个请求?

    我正在使用Python 2 7 5 并尝试使用协程绿色线程 Pythoneventlet 和Pythonrequests模块来加速我的 REST API 请求 我知道Pythonrequests https requests readthe