swh-lister/swh/lister/tests
Antoine Lambert af24960bc2 Add save-bulk lister to check origins prior their insertion in database
This new and special lister enables to verify a list of origins to archive
provided by users (for instance through the Web API).

Its purpose is to avoid polluting the scheduler database with origins that
cannot be loaded into the archive.

Each origin is identified by an URL and a visit type. For a given visit type
the lister is checking if the origin URL can be found and if the visit type
is valid.

The supported visit types are those for VCS (bzr, cvs, hg, git and svn) plus
the one for loading a tarball content into the archive.

Accepted origins are inserted or upserted in the scheduler database.

Rejected origins are stored in the lister state.

Related to #4709
2024-09-04 10:42:23 +02:00
..
__init__.py swh.lister.gitlab: Improve headers extraction 2018-07-11 18:09:18 +02:00
test_cli.py Add save-bulk lister to check origins prior their insertion in database 2024-09-04 10:42:23 +02:00
test_lister_packages.py Harmonize listers parameters and add test to check mandatory ones 2023-09-06 11:55:34 +02:00
test_pattern.py lister.pattern: Make batch record parametric and test it 2023-08-01 15:06:21 +02:00
test_utils.py Use http_retry decorator from swh.core.retry module 2023-04-13 14:19:57 +02:00
utils.py Use http_retry decorator from swh.core.retry module 2023-04-13 14:19:57 +02:00