reonokiy/swh-lister
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Use generic HTTP retry policy by default and rename dedicated decorator

Browse source 
Instead of retrying HTTP requests only for 429 status code by default,
prefer to use the generic retry policy enabling to also retry for status
codes >= 500 but also on ConnectionError exceptions.

Rename throttling_retry decorator to http_retry to reflect this change.
This commit is contained in:
Antoine Lambert 2022-09-21 16:56:34 +02:00
parent 9b3e565cf7
commit 9c55acd286
23 changed files with 71 additions and 62 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/tutorial.rst
View file

@ -317,7 +317,7 @@ We generally recommend logging every unhandleable error with the response conten
then immediately stop the listing by doing an equivalent of
:py:meth:`Response.raise_for_status` from the ``requests`` library. As for rate-limiting
errors, we have a strategy of using a flexible decorator to handle the retrying for us.
It is based on the ``tenacity`` library and accessible as :py:func:`throttling_retry` from
It is based on the ``tenacity`` library and accessible as :py:func:`http_retry` from
:py:mod:`swh.lister.utils`.
Pagination