reonokiy/swh-lister
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
164 commits 1 branch 0 tags 16 MiB
Commit graph

164 commits

This branch
This branch
All branches
Author SHA1 Message Date
Antoine R. Dumont (@ardumont)
db36c499fe
swh.lister.gitlab: Do not store information we cannot have 2018-07-11 11:26:18 +02:00
Antoine R. Dumont (@ardumont)
d9d582f7eb
swh.lister.gitlab: Change the default sort order using id 
Related T989
 2018-07-11 11:26:18 +02:00
Antoine R. Dumont (@ardumont)
b84db26bee
swh.lister.gitlab: Make the lister's task instance parametric 
Related T989
 2018-07-11 11:26:18 +02:00
Antoine R. Dumont (@ardumont)
3760708897
swh.lister.gitlab: Use one configuration for the gitlab lister 
Related T989
 2018-07-11 11:19:06 +02:00
Antoine R. Dumont (@ardumont)
8ad70b3d60
swh.lister.gitlab: Make the lister instance parametric 
This impacts:
- the credentials reading which needs to be indexed per instance
- the models since a new instance column needs to referenced and
  indexed

Related T989
 2018-07-11 11:16:42 +02:00
Antoine R. Dumont (@ardumont)
9c5963f015
swh.lister.gitlab: Respect camel case convention 
Related T989
 2018-07-11 09:46:20 +02:00
Antoine R. Dumont (@ardumont)
f8055a34af
swh.lister.gitlab: Add notes around current limitations 2018-07-11 09:45:54 +02:00
Antoine R. Dumont (@ardumont)
0e517b6143
swh.lister.gitlab: Use http_url_to_repo key to build the origin url 2018-07-11 09:45:54 +02:00
Antoine R. Dumont (@ardumont)
7a0e3a2379
gitlab.lister: Override filtering as it's not possible as is 2018-07-11 09:45:53 +02:00
Antoine R. Dumont (@ardumont)
1335a5b25f
swh.lister.gitlab: Use gitlab's index identifier (page id) 
And not 'project' id
 2018-07-11 09:45:53 +02:00
Antoine R. Dumont (@ardumont)
3e62bc867e
swh.lister.cli: Simplify cli 2018-07-11 09:45:51 +02:00
Antoine R. Dumont (@ardumont)
afcd6997c4
swh.lister.cli: Add a basic cli to deal with create/drop db actions 2018-07-03 15:49:52 +02:00
Antoine R. Dumont (@ardumont)
0f9ba7bd9b
swh.lister.gitlab: Bootstrap gitlab lister 
Related T989
 2018-07-03 12:23:11 +02:00
Antoine R. Dumont (@ardumont)
c350434127
swh.lister.core.lister_transports: Fix typo and pep8 violations 2018-07-03 12:17:46 +02:00
Antoine R. Dumont (@ardumont)
1540c989b5
README: Use the right default configuration sample 2018-07-03 12:17:46 +02:00
Antoine R. Dumont (@ardumont)
f4fe1b058b
swh.lister.*: Formatting 2018-07-03 12:17:46 +02:00
Antoine R. Dumont (@ardumont)
27f3eaf9e9
README: Fix and document how to boostrap github lister model 2018-06-27 09:52:55 +02:00
Antoine R. Dumont (@ardumont)
53c12d54c3
etc/crontab: Remove dead code 
This is now running through the swh scheduler
 2018-06-26 15:48:35 +02:00
Antoine R. Dumont (@ardumont)
d868ea69a1
TODO: Remove no longer relevant file 2018-06-26 15:45:23 +02:00
Antoine R. Dumont (@ardumont)
050b927f5d
README.md: Update documentation to current state 
Related T989
 2018-06-26 15:45:22 +02:00
Antoine R. Dumont (@ardumont)
d9a25865a2
docs/tutorials: Format sentences 2018-06-26 10:48:12 +02:00
Antoine R. Dumont (@ardumont)
4b351321d8
docs/tutorials: Remove blank spaces 2018-06-26 10:48:03 +02:00
Antoine R. Dumont (@ardumont)
1f0f169a85
README.md: Rename main readme 2018-06-26 10:47:37 +02:00
Stefano Zacchiroli
4f2361ab0e fix typo in docstrings/comments (tnx codespell) 2018-03-12 12:02:30 +01:00
Nicolas Dandrimont
16e40151dd core.tasks: use proper TaskType import 2018-01-19 14:24:17 +01:00
Stefano Zacchiroli
b8ad57fc05 docs: integrate swh-lister tutorial 
originally appeared on the Software Heritage blog at
https://www.softwareheritage.org/2017/03/24/list-the-content-of-your-favorite-forge-in-just-a-few-steps/
by Avi Kelman

Closes T762
 2017-11-02 17:53:30 +01:00
Stefano Zacchiroli
fe4725d383 docs: add absolute anchor to documentation index 2017-11-02 11:09:07 +01:00
Nicolas Dandrimont
9b58ecbd6d Make setup.py use nose for tests 2017-10-30 17:11:25 +01:00
Nicolas Dandrimont
082f415952 swh.storage is the requirement with the schemata stuff 2017-10-30 17:06:02 +01:00
Nicolas Dandrimont
77585de20c Force utf-8 encoding for tests 2017-10-30 17:05:18 +01:00
Nicolas Dandrimont
2922b68570 Clean up dependencies to enable tests on build 2017-10-30 17:04:49 +01:00
Nicolas Dandrimont
e477a46c60 Add missing __init__.py files 
Helps with tests autodetection
 2017-10-30 16:38:27 +01:00
Nicolas Dandrimont
6a7b0f802e swh.lister.debian.tasks: add task_queue attribute 2017-10-30 15:08:37 +01:00
Nicolas Dandrimont
458a9e6733 Add a new DebianListerTask 2017-10-30 14:19:43 +01:00
Nicolas Dandrimont
6b2863f125 force SQLAlchemy >= 1.0 for package builds 2017-10-13 12:12:36 +02:00
Nicolas Dandrimont
2857dcf926 Cleanup packaging 2017-10-12 17:16:49 +02:00
Nicolas Dandrimont
751b64bc45 Remove the swh/loader/_version.py file on clean 2017-10-11 17:56:08 +02:00
Nicolas Dandrimont
a0d35c4a25 Update debian/rules for tests during build 2017-10-11 17:46:46 +02:00
Nicolas Dandrimont
3aaa2d1ed8 Update debian/control for new dependencies 2017-10-11 17:44:02 +02:00
Nicolas Dandrimont
c6e455ce9b debian.lister: use get_packages() method on the snapshot to list packages 
Allows to list the packages outside of lister context (e.g. for loader tests)
 2017-10-10 16:37:15 +02:00
Nicolas Dandrimont
d2a71ac980 debian.lister: make file size an integer rather than a string 2017-10-10 16:36:58 +02:00
Nicolas Dandrimont
3efd0e33bf lister_transports: load version by using vcversioner's export module 2017-10-10 16:36:38 +02:00
Sushant Sushant
83ebb95705 Add a lister for Debian-like package archives 
This work is based on Sushant's internship and D229.
 2017-10-04 12:43:09 +02:00
Nicolas Dandrimont
6c8e16aae9 lister_transports: allow overriding the parameters to requests 2017-09-11 15:23:17 +02:00
Nicolas Dandrimont
52b5c140c1 lister_base: use the softwareheritage-scheduler-dev db by default 2017-09-11 15:21:56 +02:00
Nicolas Dandrimont
7476f5d533 lister_base: make the default filter return all entries 2017-09-11 15:21:56 +02:00
Stefano Zacchiroli
8a519af8f5 docstring: drop useless heading ":py" domain in crossrefs 2017-09-06 20:27:34 +02:00
Stefano Zacchiroli
e3c59f8fa9 sanitize docstrings for sphinx 2017-09-05 17:15:58 +02:00
Nicolas Dandrimont
5a08bb7278 lister_trasnports: send a custom http user agent 2017-09-05 15:21:55 +02:00
Nicolas Dandrimont
5eca777c75 lister_transports: filter known http response codes 2017-09-05 15:21:17 +02:00