reonokiy/swh-lister
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
swh-lister/swh/lister
History
Antoine R. Dumont (@ardumont) 92d494261f
lister: Make sure lister that requires github tokens can use it 
Deploying the nixguix lister, I realized that even though the credentials configuration
is properly set for all listers, the listers actually requiring github origin
canonicalization do not have access to the github credentials. It's lost during the
constructor to only focus on the lister's credentials. Which currently translates to
listers being rate-limited.

This commit fixes it by pushing the self.github_session instantiation in the constructor
when the lister explicitely requires the github session. Hence lifting the rate limit
for maven, packagist, nixguix, and github listers.

Related to infra/sysadm-environment#4655
2022-10-26 17:23:40 +02:00
..
arch Send package artifact checksums to loaders when info is available 2022-09-30 18:44:11 +02:00
aur Update instructions for running a lister in docker 2022-09-29 11:26:40 +02:00
bitbucket Refactor and deduplicate HTTP requests code in listers 2022-09-26 10:48:40 +02:00
bower Update instructions for running a lister in docker 2022-09-29 11:26:40 +02:00
cgit cgit/tests: Rename readme.md to readme 2022-09-26 13:22:10 +02:00
conda conda: Yield listed origins after all artifacts in a page are processed 2022-10-25 10:49:52 +02:00
cpan cpan: Fix module version extraction for some edge cases 2022-10-11 15:24:34 +02:00
cran Send package artifact checksums to loaders when info is available 2022-09-30 18:44:11 +02:00
crates Crates.io: Add last_update for each version of a crate 2022-10-05 17:10:28 +02:00
debian debian: Remove no longer needed code to get accurate origins count 2022-09-29 11:14:42 +02:00
gitea gogs/lister: Allow public gogs instance listing 2022-10-21 18:21:18 +02:00
github lister: Make sure lister that requires github tokens can use it 2022-10-26 17:23:40 +02:00
gitlab Update value of User-Agent HTTP request header used by listers 2022-09-26 10:48:40 +02:00
gnu nixguix: Add lister 2022-10-03 18:26:36 +02:00
gogs gogs/lister: Allow public gogs instance listing 2022-10-21 18:21:18 +02:00
golang Refactor and deduplicate HTTP requests code in listers 2022-09-26 10:48:40 +02:00
hackage Hackage: List origins from hackage.haskell.org, The Haskell Package Repository 2022-09-27 14:22:03 +02:00
launchpad Use generic HTTP retry policy by default and rename dedicated decorator 2022-09-26 10:48:40 +02:00
maven lister: Make sure lister that requires github tokens can use it 2022-10-26 17:23:40 +02:00
nixguix lister: Make sure lister that requires github tokens can use it 2022-10-26 17:23:40 +02:00
npm Update value of User-Agent HTTP request header used by listers 2022-09-26 10:48:40 +02:00
nuget Nuget: Lister for NuGet the package manager for .NET 2022-09-27 14:56:36 +02:00
opam python: Reformat code with black 22.3.0 2022-04-08 15:15:09 +02:00
packagist lister: Make sure lister that requires github tokens can use it 2022-10-26 17:23:40 +02:00
phabricator Update value of User-Agent HTTP request header used by listers 2022-09-26 10:48:40 +02:00
pubdev Update instructions for running a lister in docker 2022-09-29 11:26:40 +02:00
puppet Puppet: Switch artifacts from dict to list 2022-10-25 14:49:09 +02:00
pypi Use generic HTTP retry policy by default and rename dedicated decorator 2022-09-26 10:48:40 +02:00
rubygems rubygems: Fix debug log 2022-10-13 16:40:35 +02:00
sourceforge Update value of User-Agent HTTP request header used by listers 2022-09-26 10:48:40 +02:00
tests nixguix: Register task 2022-10-03 18:26:36 +02:00
tuleap Refactor and deduplicate HTTP requests code in listers 2022-09-26 10:48:40 +02:00
__init__.py Add support for more tarball recognition based on extensions 2022-10-25 09:50:31 +02:00
cli.py python: Reformat code with black 22.3.0 2022-04-08 15:15:09 +02:00
pattern.py lister: Make sure lister that requires github tokens can use it 2022-10-26 17:23:40 +02:00
py.typed typing: minimal changes to make a no-op mypy run pass 2019-10-28 15:35:21 +01:00
utils.py Use generic HTTP retry policy by default and rename dedicated decorator 2022-09-26 10:48:40 +02:00