Archit Agrawal
a9a37a85bf
swh.lister.cran
...
Add a lister to list all the CRAN packages .
It uses the build-in API in R language to list the packages
and get their metadata.
Closes T1709
2019-06-11 21:26:31 +05:30
Archit Agrawal
7c6245e663
swh.lister.gnu: Add function to check for file extension.
...
Added a function which will derive the extension from filename
and check if the fie extension match the type of file that is to be
archived.
2019-06-11 15:12:53 +05:30
Archit Agrawal
709ba8a6e5
swh.lister.gnu: Add functionality to list all the tarballs for a package.
...
As discussed in T1389 to ingest all packages using base loader, it need
a list of all the tarballs for a pakage.
Hence modifified lister to recursively list all the tarballs for a
package with their last updated time.
2019-06-08 21:56:00 +05:30
Archit Agrawal
151f6cd223
swh.lister.gnu
...
Implement first pass of gnu lister to list all the
packages present in https://ftp.gnu.org/
Add GNU lister in README and cli.py
Closes T1722
2019-06-08 21:56:00 +05:30
archit
fedfd73c8e
swh.lister.phabricator
...
Add a lister of all hosted repositories on a Phabricator instance
Closes T808
2019-05-15 19:54:33 +05:30
Antoine Lambert
7d192a2f1b
README.md: Fix outdated instructions and improve formatting
2019-05-14 10:57:17 +02:00
Archit Agrawal
4b27f9d9c4
Updated toplevel function names in README
2019-03-24 10:54:12 +05:30
Archit Agrawal
26232db926
Removed Extra blank space
2019-03-20 00:54:42 +05:30
Archit Agrawal
fa91132364
Removed extra space from the README
2019-03-20 00:42:18 +05:30
Archit Agrawal
5acb1fefc1
Updated README.md for listers
2019-03-19 15:57:29 +05:30
Antoine R. Dumont (@ardumont)
ed64d24634
pypi.lister: Normalize pypi name to PyPI
...
Related T422
2018-09-14 13:24:48 +02:00
Antoine R. Dumont (@ardumont)
cba22b7d19
doc: Fix typos according to review
2018-09-06 14:58:03 +02:00
Antoine R. Dumont (@ardumont)
bd4cf81c12
README: Reference pypi lister
2018-08-01 10:25:28 +02:00
Antoine R. Dumont (@ardumont)
6ff3b90859
swh.lister.pypi: Add a pypi lister implementation using xmlprc api
...
Based solely on pypi's deprecated xmlrpc api [1]. No other way of listing
pypi.org is referenced (except for parsing an html page through a
legacy api [2])
[1] https://warehouse.readthedocs.io/api-reference/xml-rpc/#pypi-s-xml-rpc-methods
[2] https://pypi.python.org/simple/
Related T422
2018-08-01 10:25:21 +02:00
Antoine R. Dumont (@ardumont)
e62fea6def
README.md: Update to demo the lister debian run
2018-07-27 11:28:44 +02:00
Antoine R. Dumont (@ardumont)
364786a2da
lister/gitlab: Allow to define the per page elements to read
2018-07-20 13:41:25 +02:00
Antoine R. Dumont (@ardumont)
2dfa8bdb26
README.md: Add gitlab to the lister implementations referenced
2018-07-18 14:25:07 +02:00
Antoine R. Dumont (@ardumont)
f7eb3e529e
README.md: Simplify the readme to use the default configuration
...
Related T1138
2018-07-18 12:22:47 +02:00
Antoine R. Dumont (@ardumont)
7fadb1f609
README.md: Align github sample run
2018-07-17 15:48:49 +02:00
Antoine R. Dumont (@ardumont)
d88f1b60c9
core/lister: Make the tasks take an explicit lister_args argument
...
Avoid eating *all* arbitrary arguments and passing them along to the
new_lister method.
2018-07-17 15:48:48 +02:00
Antoine R. Dumont (@ardumont)
a527977b6c
README.md: Add a section to demo how to run the gitlab lister
...
Related T989
2018-07-11 15:56:32 +02:00
Antoine R. Dumont (@ardumont)
1fd44207eb
README: Add a run section to demonstrate how to run a github lister
2018-07-11 11:26:19 +02:00
Antoine R. Dumont (@ardumont)
3e62bc867e
swh.lister.cli: Simplify cli
2018-07-11 09:45:51 +02:00
Antoine R. Dumont (@ardumont)
afcd6997c4
swh.lister.cli: Add a basic cli to deal with create/drop db actions
2018-07-03 15:49:52 +02:00
Antoine R. Dumont (@ardumont)
1540c989b5
README: Use the right default configuration sample
2018-07-03 12:17:46 +02:00
Antoine R. Dumont (@ardumont)
27f3eaf9e9
README: Fix and document how to boostrap github lister model
2018-06-27 09:52:55 +02:00
Antoine R. Dumont (@ardumont)
050b927f5d
README.md: Update documentation to current state
...
Related T989
2018-06-26 15:45:22 +02:00
Antoine R. Dumont (@ardumont)
1f0f169a85
README.md: Rename main readme
2018-06-26 10:47:37 +02:00
