reonokiy/swh-lister
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
swh-lister/swh/lister
History
Antoine Lambert 323e277482 gitea, gogs: Ensure query parameters are not duplicated in API URLs 
Gitea API return next pagination link with all query parameters provided
to an API request.

As we were also passing a dict of fixed query parameters to the page_request
method, some query parameters ended up having multiple instances in the URL
for fetching a new page of repositories data. So each time a new page was
requested, new instances of these parameters were appended to the URL which
could result in a really long URL if the number of pages to retrieve is high
and make the request fail.

Also remove a debug log already present in http_request method.
2024-06-05 15:27:58 +02:00
..
arch Use beautifulsoup4 CSS selectors to simplify code and type checking 2024-04-16 11:22:51 +02:00
aur python: Fix black formatting after bump to 23.1.0 in pre-commit 2023-12-05 10:33:07 +01:00
bioconductor Use beautifulsoup4 CSS selectors to simplify code and type checking 2024-04-16 11:22:51 +02:00
bitbucket python: Fix black formatting after bump to 23.1.0 in pre-commit 2023-12-05 10:33:07 +01:00
bower Harmonize listers parameters and add test to check mandatory ones 2023-09-06 11:55:34 +02:00
cgit Use beautifulsoup4 CSS selectors to simplify code and type checking 2024-04-16 11:22:51 +02:00
conda Harmonize listers parameters and add test to check mandatory ones 2023-09-06 11:55:34 +02:00
cpan python: Fix black formatting after bump to 23.1.0 in pre-commit 2023-12-05 10:33:07 +01:00
cran cran: Use pyreadr instead of rpy2 to read a RDS file from Python 2023-11-14 17:09:42 +01:00
crates python: Fix black formatting after bump to 23.1.0 in pre-commit 2023-12-05 10:33:07 +01:00
debian python: Fix black formatting after bump to 23.1.0 in pre-commit 2023-12-05 10:33:07 +01:00
dlang Remove spurious space 2023-09-21 09:18:54 +02:00
elm Elm stateful lister 2024-01-09 14:05:56 +01:00
gitea gogs, gitea: Fix task execution to pass along extra kwargs 2022-12-14 16:09:56 +01:00
github Harmonize listers parameters and add test to check mandatory ones 2023-09-06 11:55:34 +02:00
gitiles Add gitiles lister 2023-07-13 10:30:51 +02:00
gitlab Ensure HTTPError.response is not None 2023-10-18 10:41:57 +02:00
gitweb Use beautifulsoup4 CSS selectors to simplify code and type checking 2024-04-16 11:22:51 +02:00
gnu python: Fix black formatting after bump to 23.1.0 in pre-commit 2023-12-05 10:33:07 +01:00
gogs gitea, gogs: Ensure query parameters are not duplicated in API URLs 2024-06-05 15:27:58 +02:00
golang Harmonize listers parameters and add test to check mandatory ones 2023-09-06 11:55:34 +02:00
hackage Harmonize listers parameters and add test to check mandatory ones 2023-09-06 11:55:34 +02:00
hex python: Fix black formatting after bump to 23.1.0 in pre-commit 2023-12-05 10:33:07 +01:00
julia Stateful Julia lister 2023-12-18 16:02:22 +01:00
launchpad launchpad, npm: Port code to updated swh-scheduler API 2024-05-22 17:44:00 +02:00
maven Use beautifulsoup4 CSS selectors to simplify code and type checking 2024-04-16 11:22:51 +02:00
nixguix nixguix: Ensure to not use a redirection URL as an origin URL 2024-04-24 14:25:48 +02:00
npm launchpad, npm: Port code to updated swh-scheduler API 2024-05-22 17:44:00 +02:00
nuget Use beautifulsoup4 CSS selectors to simplify code and type checking 2024-04-16 11:22:51 +02:00
opam opam: Fix 'opam init' error when relisting an opam instance 2023-06-29 17:49:21 +02:00
packagist Harmonize listers parameters and add test to check mandatory ones 2023-09-06 11:55:34 +02:00
pagure pagure/tasks: Add missing docstring for list_pagure task function 2023-06-23 14:29:17 +02:00
phabricator python: Fix black formatting after bump to 23.1.0 in pre-commit 2023-12-05 10:33:07 +01:00
pubdev python: Fix black formatting after bump to 23.1.0 in pre-commit 2023-12-05 10:33:07 +01:00
puppet python: Fix black formatting after bump to 23.1.0 in pre-commit 2023-12-05 10:33:07 +01:00
pypi Harmonize listers parameters and add test to check mandatory ones 2023-09-06 11:55:34 +02:00
rpm python: Fix black formatting after bump to 23.1.0 in pre-commit 2023-12-05 10:33:07 +01:00
rubygems Use beautifulsoup4 CSS selectors to simplify code and type checking 2024-04-16 11:22:51 +02:00
sourceforge Use beautifulsoup4 CSS selectors to simplify code and type checking 2024-04-16 11:22:51 +02:00
stagit Use beautifulsoup4 CSS selectors to simplify code and type checking 2024-04-16 11:22:51 +02:00
tests Harmonize listers parameters and add test to check mandatory ones 2023-09-06 11:55:34 +02:00
tuleap Hook up recently introduced options to all listers 2022-12-05 16:33:45 +01:00
__init__.py Add support for more tarball recognition based on extensions 2022-10-25 09:50:31 +02:00
cli.py cli: Print lister stats at the end of the run command 2023-11-07 19:00:53 +01:00
pattern.py packagist: Yield pages of origins to regularly record origins 2023-08-04 11:09:58 +02:00
py.typed typing: minimal changes to make a no-op mypy run pass 2019-10-28 15:35:21 +01:00
utils.py Use http_retry decorator from swh.core.retry module 2023-04-13 14:19:57 +02:00