reonokiy/swh-lister
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
150 commits 1 branch 0 tags 16 MiB
Commit graph

150 commits

This branch
This branch
All branches
Author SHA1 Message Date
Antoine R. Dumont (@ardumont)
1540c989b5
README: Use the right default configuration sample 2018-07-03 12:17:46 +02:00
Antoine R. Dumont (@ardumont)
f4fe1b058b
swh.lister.*: Formatting 2018-07-03 12:17:46 +02:00
Antoine R. Dumont (@ardumont)
27f3eaf9e9
README: Fix and document how to boostrap github lister model 2018-06-27 09:52:55 +02:00
Antoine R. Dumont (@ardumont)
53c12d54c3
etc/crontab: Remove dead code 
This is now running through the swh scheduler
 2018-06-26 15:48:35 +02:00
Antoine R. Dumont (@ardumont)
d868ea69a1
TODO: Remove no longer relevant file 2018-06-26 15:45:23 +02:00
Antoine R. Dumont (@ardumont)
050b927f5d
README.md: Update documentation to current state 
Related T989
 2018-06-26 15:45:22 +02:00
Antoine R. Dumont (@ardumont)
d9a25865a2
docs/tutorials: Format sentences 2018-06-26 10:48:12 +02:00
Antoine R. Dumont (@ardumont)
4b351321d8
docs/tutorials: Remove blank spaces 2018-06-26 10:48:03 +02:00
Antoine R. Dumont (@ardumont)
1f0f169a85
README.md: Rename main readme 2018-06-26 10:47:37 +02:00
Stefano Zacchiroli
4f2361ab0e fix typo in docstrings/comments (tnx codespell) 2018-03-12 12:02:30 +01:00
Nicolas Dandrimont
16e40151dd core.tasks: use proper TaskType import 2018-01-19 14:24:17 +01:00
Stefano Zacchiroli
b8ad57fc05 docs: integrate swh-lister tutorial 
originally appeared on the Software Heritage blog at
https://www.softwareheritage.org/2017/03/24/list-the-content-of-your-favorite-forge-in-just-a-few-steps/
by Avi Kelman

Closes T762
 2017-11-02 17:53:30 +01:00
Stefano Zacchiroli
fe4725d383 docs: add absolute anchor to documentation index 2017-11-02 11:09:07 +01:00
Nicolas Dandrimont
9b58ecbd6d Make setup.py use nose for tests 2017-10-30 17:11:25 +01:00
Nicolas Dandrimont
082f415952 swh.storage is the requirement with the schemata stuff 2017-10-30 17:06:02 +01:00
Nicolas Dandrimont
77585de20c Force utf-8 encoding for tests 2017-10-30 17:05:18 +01:00
Nicolas Dandrimont
2922b68570 Clean up dependencies to enable tests on build 2017-10-30 17:04:49 +01:00
Nicolas Dandrimont
e477a46c60 Add missing __init__.py files 
Helps with tests autodetection
 2017-10-30 16:38:27 +01:00
Nicolas Dandrimont
6a7b0f802e swh.lister.debian.tasks: add task_queue attribute 2017-10-30 15:08:37 +01:00
Nicolas Dandrimont
458a9e6733 Add a new DebianListerTask 2017-10-30 14:19:43 +01:00
Nicolas Dandrimont
6b2863f125 force SQLAlchemy >= 1.0 for package builds 2017-10-13 12:12:36 +02:00
Nicolas Dandrimont
2857dcf926 Cleanup packaging 2017-10-12 17:16:49 +02:00
Nicolas Dandrimont
751b64bc45 Remove the swh/loader/_version.py file on clean 2017-10-11 17:56:08 +02:00
Nicolas Dandrimont
a0d35c4a25 Update debian/rules for tests during build 2017-10-11 17:46:46 +02:00
Nicolas Dandrimont
3aaa2d1ed8 Update debian/control for new dependencies 2017-10-11 17:44:02 +02:00
Nicolas Dandrimont
c6e455ce9b debian.lister: use get_packages() method on the snapshot to list packages 
Allows to list the packages outside of lister context (e.g. for loader tests)
 2017-10-10 16:37:15 +02:00
Nicolas Dandrimont
d2a71ac980 debian.lister: make file size an integer rather than a string 2017-10-10 16:36:58 +02:00
Nicolas Dandrimont
3efd0e33bf lister_transports: load version by using vcversioner's export module 2017-10-10 16:36:38 +02:00
Sushant Sushant
83ebb95705 Add a lister for Debian-like package archives 
This work is based on Sushant's internship and D229.
 2017-10-04 12:43:09 +02:00
Nicolas Dandrimont
6c8e16aae9 lister_transports: allow overriding the parameters to requests 2017-09-11 15:23:17 +02:00
Nicolas Dandrimont
52b5c140c1 lister_base: use the softwareheritage-scheduler-dev db by default 2017-09-11 15:21:56 +02:00
Nicolas Dandrimont
7476f5d533 lister_base: make the default filter return all entries 2017-09-11 15:21:56 +02:00
Stefano Zacchiroli
8a519af8f5 docstring: drop useless heading ":py" domain in crossrefs 2017-09-06 20:27:34 +02:00
Stefano Zacchiroli
e3c59f8fa9 sanitize docstrings for sphinx 2017-09-05 17:15:58 +02:00
Nicolas Dandrimont
5a08bb7278 lister_trasnports: send a custom http user agent 2017-09-05 15:21:55 +02:00
Nicolas Dandrimont
5eca777c75 lister_transports: filter known http response codes 2017-09-05 15:21:17 +02:00
Nicolas Dandrimont
de01552ac5 indexing_lister: actually filter objects by start and end of range 2017-09-05 14:51:35 +02:00
Nicolas Dandrimont
cf3220d1fb github.models: handle the fork argument 2017-09-05 11:38:27 +02:00
Stefano Zacchiroli
164922afe2 requirements.txt: add missing dependency on "arrow" 2017-09-05 10:54:54 +02:00
Nicolas Dandrimont
4b56b6037c github.models: add fork information to repos 2017-09-04 19:40:45 +02:00
Nicolas Dandrimont
f6f077b789 github.tasks: the github api is rooted at api.github.com 2017-09-04 17:42:20 +02:00
Nicolas Dandrimont
04c22dd56e lister_base: when storing responses for debugging, use a timestamp as filename 2017-09-04 17:41:54 +02:00
Nicolas Dandrimont
3fe5619bcc lister_base: use timezone-aware datetimes everywhere 2017-09-04 17:41:36 +02:00
Nicolas Dandrimont
a85f55dbf9 lister_transports: add headers to debug data 2017-09-04 17:41:06 +02:00
Nicolas Dandrimont
eb47600461 lister_transports: requests credentials are a tuple (username, password) 2017-09-04 17:11:00 +02:00
Nicolas Dandrimont
7c6ac9df0d lister_base: cache all responses 2017-09-04 17:11:00 +02:00
Nicolas Dandrimont
3c7cfe7d78 lister_base: prepare_folders takes its arguments flattened 2017-09-04 17:11:00 +02:00
Stefano Zacchiroli
67a4c62a64 docs/: add sphinx apidoc generation skeleton 
change cherry picked from python module template
commit 71b117ba0cf9f1251b1cac26d0994df03a4c787d
 2017-08-30 12:25:37 +02:00
Nicolas Dandrimont
b6bdab70c6 lister_base: create_tasks returns a list of tasks 2017-06-28 16:44:52 +02:00
Nicolas Dandrimont
c677ad6067 indexing_lister: use task_ids to disable tasks 2017-06-28 16:44:27 +02:00