reonokiy/swh-lister
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

indexing_lister: Allow to define flush packet size

Browse source 
Prior to this commit, indexing lister instances were flushing every packet of
20. This can now be defined per sub classes.
This commit is contained in:
Antoine R. Dumont (@ardumont) 2019-06-22 00:14:34 +02:00
parent 5ec3067b0d
commit 6662ae8db5
No known key found for this signature in database
GPG key ID: 52E2E9840D10C3B8
2 changed files with 6 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

3
swh/lister/bitbucket/lister.py
View file

@ -30,6 +30,9 @@ class BitBucketLister(IndexingHttpLister):
if per_page != DEFAULT_BITBUCKET_PAGE:
self.PATH_TEMPLATE = '%s&pagelen=%s' % (
self.PATH_TEMPLATE, per_page)
# to stay consistent with prior behavior (20 * 10 repositories then)
self.flush_packet_db = int(
(self.flush_packet_db * DEFAULT_BITBUCKET_PAGE) / per_page)
def get_model_from_repo(self, repo):
return {