Commit graph

19 commits

Author SHA1 Message Date
Antoine Lambert
8ffd8dadef cli: Fix initialization for all listers
Prior to this commit, initializing all listers was failing after the
debian lister processing because of global insert_minimum_data init

Related T1629
2019-04-10 11:33:55 +02:00
Antoine R. Dumont (@ardumont)
262c297a5e
lister.cli: Fix spelling typo 2019-02-14 10:47:28 +01:00
David Douard
0720c8e12e Kill (useless) --create-tables and --with-data cli command options 2019-02-06 15:38:11 +01:00
David Douard
4f2580a97c Make the --lister option of the cli tool a variadic argument
and add a 'all' possibel value for it, so that one can initialize all the
database for all listers at once.
2019-02-06 10:19:26 +01:00
David Douard
72658ff720 cli: fix debian lister so it also uses config overrides 2018-12-20 15:01:18 +01:00
Antoine Lambert
ffe4ac9a3c swh.lister.npm: Add an incremental npm lister
This new lister enables to get only new or updated npm packages
since the last listing operation.

Related T1378
Closes T1398
2018-12-03 17:58:27 +01:00
Antoine Lambert
605a67f51d swh.lister.npm : Add a lister of all available packages in the npm registry
Related T1378
Closes T1380
2018-11-26 11:04:13 +01:00
Antoine R. Dumont (@ardumont)
ed64d24634
pypi.lister: Normalize pypi name to PyPI
Related T422
2018-09-14 13:24:48 +02:00
Antoine R. Dumont (@ardumont)
6ff3b90859
swh.lister.pypi: Add a pypi lister implementation using xmlprc api
Based solely on pypi's deprecated xmlrpc api [1].  No other way of listing
pypi.org is referenced (except for parsing an html page through a
legacy api [2])

[1] https://warehouse.readthedocs.io/api-reference/xml-rpc/#pypi-s-xml-rpc-methods

[2] https://pypi.python.org/simple/

Related T422
2018-08-01 10:25:21 +02:00
Antoine R. Dumont (@ardumont)
b6b588dbbb
lister.cli: Insert optional flag to permit post insert data
That's needed for example for having the minimum necessary to make the
debian lister run.
2018-07-27 11:13:13 +02:00
Antoine R. Dumont (@ardumont)
2c69b586bc
Revert "lister.cli: Insert optional flag to permit post insert data"
This reverts commit e61512afc7.

This commit contains one section not supposed to be there yet (pypi is
not ready)
2018-07-27 11:12:01 +02:00
Antoine R. Dumont (@ardumont)
e61512afc7
lister.cli: Insert optional flag to permit post insert data
That's needed for example for having the minimum necessary to make the
debian lister run.
2018-07-27 11:06:22 +02:00
Antoine R. Dumont (@ardumont)
726d45b182
swh.lister.cli: Factorize supported listers 2018-07-27 10:21:38 +02:00
Antoine R. Dumont (@ardumont)
6c54b64a8f
swh.lister.cli: Add debian lister to the list of supported listers 2018-07-27 10:19:48 +02:00
Antoine R. Dumont (@ardumont)
63cff5b337
lister.cli: Fix broken imports 2018-07-17 15:48:48 +02:00
Antoine R. Dumont (@ardumont)
4c4aa0ead2
swh.lister: Make LISTER_NAME a class attribute
swh.lister.gitlab: make the 'instance' a constructor parameter
2018-07-11 17:43:41 +02:00
Antoine R. Dumont (@ardumont)
581028cfc5
swh.lister.cli: Fix cli docstring 2018-07-11 15:56:33 +02:00
Antoine R. Dumont (@ardumont)
3e62bc867e
swh.lister.cli: Simplify cli 2018-07-11 09:45:51 +02:00
Antoine R. Dumont (@ardumont)
afcd6997c4
swh.lister.cli: Add a basic cli to deal with create/drop db actions 2018-07-03 15:49:52 +02:00