reonokiy/swh-lister
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

pattern: Ensure accurate origin counts returned by run method

Browse source 
Previously, the run method was returning the total count of ListedOrigin
objects sent to scheduler database.

However, some listers can send multiple ListedOrigin objects for a given
origin URL during the listing process, for instance when an origin is
contained in multiple pages (e.g. gogs listing) or when the listing
is gathering multiple versions of an origin spread across multiple
pages (e.g. maven listing).

This changes ensures an accurate count of listed origins by maintaining
a set of origin URLs associated to the sent ListedOrigin objects.
This commit is contained in:
Antoine Lambert 2022-09-29 11:14:08 +02:00
parent 3928fc9ee9
commit 8d85b2e4e8
4 changed files with 32 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

2
swh/lister/conda/tests/test_lister.py
View file

@ -13,7 +13,7 @@ def test_conda_lister_free_channel(datadir, requests_mock_datadir, swh_scheduler
res = lister.run()
assert res.pages == 3
assert res.origins == 14
assert res.origins == 11
def test_conda_lister_conda_forge_channel(