reonokiy/swh-lister
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

lister: Add utility decorator to ease HTTP requests rate limit handling

Browse source 
Add swh.lister.utils.throttling_retry decorator enabling to retry a
function that performs an HTTP request who can return a 429 status code.

The implementation is based on the tenacity module and it is assumed
that the requests library is used when querying an URL.

The default wait strategy is based on exponential backoff.

The default max number of attempts is set to 5, HTTPError exception
will then be reraised.

All tenacity.retry parameters can also be overridden in client code.
This commit is contained in:
Antoine Lambert 2021-01-15 12:25:45 +01:00
parent c782275296
commit d1fbccd988
3 changed files with 174 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

3
requirements.txt
View file

@ -7,4 +7,5 @@ xmltodict
iso8601
beautifulsoup4
pytz
launchpadlib
launchpadlib
tenacity