reonokiy/swh-lister
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
swh-lister/swh/lister
History
Antoine R. Dumont (@ardumont) e3c856b5ee
utils.split_range: Split into not overlapping ranges 
Existing listers use the `is_within_bound` [1] method from the base lister.
This method uses inclusive boundaries in all cases.

As some "range" task listers [2] [3] are using `split_range` function to create
"overlapping" ranges, this can cause concurrent insert issues down the line [4].

This commit adapts the function `split_range` to make the generated ranges no
longer overlap.

[1]
https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/core/lister_base.py$194-199

[2]
https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/gitlab/tasks.py$37-41

[3]
https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/gitea/tasks.py$36-41

Related to T2577
2020-09-10 11:01:44 +02:00
..
bitbucket tests: Separate lister instantiations 2020-09-02 12:49:00 +02:00
cgit tests: Separate lister instantiations 2020-09-02 12:49:00 +02:00
core pytest: Define plugin and declare it in the root conftest 2020-09-02 12:25:15 +02:00
cran tests: Separate lister instantiations 2020-09-02 12:49:00 +02:00
debian tests: Separate lister instantiations 2020-09-02 12:49:00 +02:00
gitea utils.split_range: Split into not overlapping ranges 2020-09-10 11:01:44 +02:00
github tests: Separate lister instantiations 2020-09-02 12:49:00 +02:00
gitlab utils.split_range: Split into not overlapping ranges 2020-09-10 11:01:44 +02:00
gnu tests: Separate lister instantiations 2020-09-02 12:49:00 +02:00
launchpad launchpad.tasks: Update copyright headers 2020-09-08 14:42:37 +02:00
npm tests: Separate lister instantiations 2020-09-02 12:49:00 +02:00
packagist tests: Separate lister instantiations 2020-09-02 12:49:00 +02:00
phabricator tests: Separate lister instantiations 2020-09-02 12:49:00 +02:00
pypi tests: Separate lister instantiations 2020-09-02 12:49:00 +02:00
tests utils.split_range: Split into not overlapping ranges 2020-09-10 11:01:44 +02:00
__init__.py Enable black 2020-04-08 16:31:22 +02:00
cli.py Enable black 2020-04-08 16:31:22 +02:00
py.typed typing: minimal changes to make a no-op mypy run pass 2019-10-28 15:35:21 +01:00
pytest_plugin.py tests: Separate lister instantiations 2020-09-02 12:49:00 +02:00
utils.py utils.split_range: Split into not overlapping ranges 2020-09-10 11:01:44 +02:00