Commit graph

10 commits

Author SHA1 Message Date
Antoine Lambert
9c55acd286 Use generic HTTP retry policy by default and rename dedicated decorator
Instead of retrying HTTP requests only for 429 status code by default,
prefer to use the generic retry policy enabling to also retry for status
codes >= 500 but also on ConnectionError exceptions.

Rename throttling_retry decorator to http_retry to reflect this change.
2022-09-26 10:48:40 +02:00
Antoine Lambert
f1a1b30fd1 arch: Set log level to debug for URL requests 2022-09-13 12:09:13 +02:00
Antoine Lambert
a55f171ed5 arch: Use tempfile module to create temporary directory
It ensures created temporary directories will be removed once they
are no longer needed.
2022-09-13 12:08:02 +02:00
Franck Bret
0acf5b0f4f Arch: Add throttling retry for scrapping and resources download 2022-08-30 09:50:29 +02:00
Valentin Lorentz
766fbbcc91 arch: Un-nest long method 2022-08-25 09:41:44 +02:00
Valentin Lorentz
b7ec6cb120 tests: Simplify origin comparison and improve pytest diff on failure
By using a single equality instead of checking len() then zip()
to check one by one, pytest can find the common/missing elements
and print them nicely when the two lists are unequal.
2022-08-24 17:21:24 +02:00
Valentin Lorentz
4b511b4181 arch: Use lazy interpolation in logging statements 2022-08-23 13:43:07 +02:00
Valentin Lorentz
dde7865ac4 arch: Fix broken ref 2022-08-19 19:07:55 +02:00
Franck Bret
7dd412e553 arch: Extra_loader_arguments consistency + documentation
Split extraloader_arguments artifacts to artifacts and arch_metadata
Add lister documentation at module level

Related T4233
2022-08-19 15:43:58 +02:00
Franck Bret
1bf11aa26d Add arch lister module (origins from archives).
After a first attempt with D7812 this one use a different strategy to
retrieve origins.

Fetch and extract "core.files.tar.gz", "extra.files.tar.gz" and "community.files.tar.gz" from archives.archlinux.org. That step ensure that we have a list of "official" packages.
Parse metadata from 'desc' file to build origins url.
Scrap the origin url to get artifacts metadata that list all versions of a package.

It also fetch and extract unofficial 'arm' packages from archlinuxarm.org but in this case we can not get all versions of an arm package.

Related T4233
2022-06-15 09:11:57 +02:00