swh-lister/swh/lister
Antoine Lambert 95714f6f37 rpm: Turn fedora lister into a generic Red Hat based distribution one
As Red Hat based linux distributions share the same type of package repository,
rework the fedora lister into a generic one to list RPM source packages and
their versions from numerous distributions.

For a given distribution, the RPM lister will fetch packages metadata from a
list of release identifiers and a list of software components. Source packages
are then processed and relevant info are extracted to be sent to the RPM loader.
When all releases and components were processed, the lister collected all versions
for each package name and send those info to the scheduler that will create RPM
loading tasks afterwards.

Nevertheless, as there is no generic way to list all releases and components for
a given distribution but also to guess the right URL to retrieve packages metadata
from, those info need to be manually provided to the lister as input parameters.
Some examples of those parameters for various distributions can be found in the
config directory of the lister.

Regarding the produced origin URLs, as there is no way to find valid HTTP ones
for all distributions, the same behavior as with the debian lister is used and
they have the following form: rpm://{instance}/packages/{package_name} where
the instance variable corresponds to the name of the listed distribution such
as Fedora, CentOS, or openSUSE.

Related to swh/meta#5011.
2023-08-16 13:25:23 +00:00
..
arch Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
aur Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
bitbucket Use http_retry decorator from swh.core.retry module 2023-04-13 14:19:57 +02:00
bower Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
cgit cgit: Allow url to be optional 2023-05-23 11:47:51 +02:00
conda Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
cpan Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
cran Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
crates crates: Don't extract unused files 2023-06-20 16:06:21 +02:00
debian debian: refactor inner loop slightly to help mypy 2023-06-21 13:57:27 +02:00
gitea gogs, gitea: Fix task execution to pass along extra kwargs 2022-12-14 16:09:56 +01:00
github github: Fix fixtures use in tests 2023-01-02 18:06:26 +01:00
gitiles Add gitiles lister 2023-07-13 10:30:51 +02:00
gitlab lister: Allow lister to build url out of the instance parameter 2023-05-19 15:03:49 +02:00
gitweb Add Gitweb lister 2023-07-10 16:50:41 +02:00
gnu Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
gogs gogs: Ensure to list all repositories 2023-06-26 15:16:48 +00:00
golang Use http_retry decorator from swh.core.retry module 2023-04-13 14:19:57 +02:00
hackage Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
hex hex: Update loader arguments 2023-03-22 08:45:41 +00:00
launchpad Use http_retry decorator from swh.core.retry module 2023-04-13 14:19:57 +02:00
maven mypy: Bump to 1.0.1 and fix new typing errors 2023-02-17 17:56:07 +01:00
nixguix nixguix/lister: Rename listed origin visit type to tarball-directory 2023-06-08 11:24:38 +02:00
npm Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
nuget Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
opam opam: Fix 'opam init' error when relisting an opam instance 2023-06-29 17:49:21 +02:00
packagist packagist: Yield pages of origins to regularly record origins 2023-08-04 11:09:58 +02:00
pagure pagure/tasks: Add missing docstring for list_pagure task function 2023-06-23 14:29:17 +02:00
phabricator Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
pubdev Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
puppet Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
pypi pypi: Allow passing configuration arguments to task 2023-08-04 15:34:02 +02:00
rpm rpm: Turn fedora lister into a generic Red Hat based distribution one 2023-08-16 13:25:23 +00:00
rubygems Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
sourceforge sourceforge: Allow passing configuration arguments to task 2023-08-04 15:30:09 +02:00
stagit Add stagit lister 2023-07-13 11:50:51 +02:00
tests rpm: Turn fedora lister into a generic Red Hat based distribution one 2023-08-16 13:25:23 +00:00
tuleap Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
__init__.py Add support for more tarball recognition based on extensions 2022-10-25 09:50:31 +02:00
cli.py Fix crash of 'swh lister run' when called without -l 2023-05-10 10:19:26 +02:00
pattern.py packagist: Yield pages of origins to regularly record origins 2023-08-04 11:09:58 +02:00
py.typed typing: minimal changes to make a no-op mypy run pass 2019-10-28 15:35:21 +01:00
utils.py Use http_retry decorator from swh.core.retry module 2023-04-13 14:19:57 +02:00