swh.lister.cgit: Remove repo page visit step
Remove the need to visit every page and extract the origin url by introducing a parameter url_prefix. The origin url is in format <prefix>/<repo_name> where The prefix is same for all the repos for a particular cgit instance.
This commit is contained in:
parent
7e3c79bb1d
commit
0bf24469b7
9 changed files with 215 additions and 223 deletions
|
@ -214,7 +214,11 @@ import logging
|
|||
from swh.lister.cgit.tasks import cgit_lister
|
||||
|
||||
logging.basicConfig(level=logging.DEBUG)
|
||||
cgit_lister(base_url='http://git.savannah.gnu.org/cgit/')
|
||||
# simple cgit instance
|
||||
cgit_lister(url='https://git.kernel.org/')
|
||||
# cgit instance whose listed repositories differ from the base url
|
||||
cgit_lister(url='https://cgit.kde.org/',
|
||||
url_prefix='https://anongit.kde.org/')
|
||||
```
|
||||
|
||||
Licensing
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue