我遇到了 aiohttp 的问题,出现以下错误,但不确定修复它的最佳方法:
Traceback (most recent call last):
File "/app/app/services/file_ingestion_utils.py", line 110, in send_api_request
async with session.post(url, headers=self.headers, data=payload) as response:
File "/usr/local/lib/python3.8/site-packages/aiohttp/client.py", line 1117, in __aenter__
self._resp = await self._coro
File "/usr/local/lib/python3.8/site-packages/aiohttp/client.py", line 544, in _request
await resp.start(conn)
File "/usr/local/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 905, in start
self._continue = None
File "/usr/local/lib/python3.8/site-packages/aiohttp/helpers.py", line 656, in __exit__
raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
我看到从here https://stackoverflow.com/questions/54132962/why-asyncio-timeouterror-is-raised这些错误可能有点模糊,但我至少想了解至少在我的情况下导致这些错误的原因。我的实现如下所示。
async def async_request(self, df, entity):
api_request_records = []
...some logic to prepare records...
@backoff.on_exception(backoff.expo, aiohttp.ClientError, max_tries=2)
async def send_api_request(payload, session):
url = <some_url>
try:
async with session.post(url, headers=self.headers, data=payload) as response:
...some response handling logic...
except asyncio.TimeoutError:
self.logger.exception(f"Asyncio TimeoutError on {url} and payload {payload}")
async with aiohttp.ClientSession() as session:
await asyncio.gather(
*[send_api_request(api_request_record, session)) for api_request_record in api_request_records])
Q1:当长度api_request_records
很小,该方法工作正常,但是当它很大时,我更有可能得到 TimeoutError。为什么?
Q2:设置 ClientSession(timeout=...) 参数是这里的秘密,因为根据here https://stackoverflow.com/questions/53049523/how-to-prevent-raise-asyncio-timeouterror-and-continue-the-loop,可能会有帮助吗?但是,我觉得这个响应可能有点过时,因为它指出现在使用 ClientTimeout 对象而不是 int。相关地,根据官方文档here https://docs.aiohttp.org/en/stable/client_reference.html#clienttimeout,默认的 ClientTimeout 似乎已经是无限时间了total=None
那么这也适用于 ClientSession 吗?
总的来说,希望得到一些帮助和推荐的方法。谢谢!