swh-lister/requirements.txt
Antoine Lambert af24960bc2 Add save-bulk lister to check origins prior their insertion in database
This new and special lister enables to verify a list of origins to archive
provided by users (for instance through the Web API).

Its purpose is to avoid polluting the scheduler database with origins that
cannot be loaded into the archive.

Each origin is identified by an URL and a visit type. For a given visit type
the lister is checking if the origin URL can be found and if the visit type
is valid.

The supported visit types are those for VCS (bzr, cvs, hg, git and svn) plus
the one for loading a tarball content into the archive.

Accepted origins are inserted or upserted in the scheduler database.

Rejected origins are stored in the lister state.

Related to #4709
2024-09-04 10:42:23 +02:00

19 lines
262 B
Text

beautifulsoup4
breezy >= 3.3.1, < 3.3.5 # use versions with available binary wheels
dateparser
dulwich
iso8601
launchpadlib
looseversion
lxml
mercurial
psycopg2
pyreadr
python_debian
repomd
requests
setuptools
subvertpy
tenacity >= 8.4.2
testing.postgresql
toml