Commit graph

20 commits

Author SHA1 Message Date
Léni Gauffier
58ef08b083 Added LaunchpadLister
Summary:
Related to T1734

From abandonned D2799

Reviewers: ardumont

Reviewed By: ardumont

Differential Revision: https://forge.softwareheritage.org/D2974
2020-04-12 01:00:12 +02:00
David Douard
93a4d8b784 Enable black
- blackify all the python files,
- enable black in pre-commit,
- add a black tox environment.
2020-04-08 16:31:22 +02:00
Nicolas Dandrimont
62dc4dc257 Use pkg_resources to get the package version instead of vcversioner 2019-11-22 15:49:23 +01:00
Antoine R. Dumont (@ardumont)
85d001067a
setup.py: Kill deprecated swh-lister command
Prior to this commit, the pip activation environment failed because the old cli
name no longer exists, it's named 'lister' now.
2019-09-20 11:11:27 +02:00
David Douard
e3c0ea9d90 implement listers as plugins
Listers are declared as plugins via the `swh.workers` entry_point.

As such, the registry function is expected to return a dict with the
`task_modules` field (as for generic worker plugins), plus:

- `lister`: the lister class,
- `models`: list of SQLAlchemy models used by this lister,
- `init` (optionnal): hook (callable) used to initialize the lister's state
  (typically, create/initialize the database for this lister).
  If not set, the default implementation creates database tables (after
  optionally having deleted exisintg ones) according to models declared in
  the `models` register field.

There is no need for explicitely add lister task modules in the main
`conftest` module, but any new/extra lister to be tested must be registered
(the tested lister module must be properly installed in the test environment).

Also refactor a bit the cli tools:
- add support for the standard --config-file option at the 'lister' group
  level,
- move the --db-url to the 'lister' group,
- drop the --lister option for the `swh lister db-init` cli tool:
  initializing (especially with --drop-tables) the database for a single
  lister is unreliable, since all tables are created using a sibgle MetaData
  (in the same namespace).
2019-09-03 15:02:24 +02:00
Antoine R. Dumont (@ardumont)
c507948da8
bin: Drop dead code 2019-06-28 18:17:15 +02:00
David Douard
d6169c7141 cli: register the 'lister' cli subcommand
also add a cli group named 'lister' for the sake of consistency with
other swh packages and rename the command as 'db-init', like:

  swh lister db-init LISTER [...]
2019-05-22 13:36:57 +02:00
David Douard
b8381d8a1b Add an entry point for the cli command 2019-02-06 10:21:50 +01:00
David Douard
9da0bd26eb setup: kill remaining nose whims to be around 2018-10-30 12:17:25 +01:00
David Douard
788efd31aa Fix a typo in setup.py 2018-10-23 19:09:24 +02:00
David Douard
9a205b1938 setup: prepare for pypi upload
related to T1242
2018-10-08 11:27:54 +02:00
Antoine Pietri
f65a3672bb Add requirements-test.txt 2018-10-05 12:23:15 +02:00
Nicolas Dandrimont
9b58ecbd6d Make setup.py use nose for tests 2017-10-30 17:11:25 +01:00
Nicolas Dandrimont
2922b68570 Clean up dependencies to enable tests on build 2017-10-30 17:04:49 +01:00
Nicolas Dandrimont
2857dcf926 Cleanup packaging 2017-10-12 17:16:49 +02:00
Nicolas Dandrimont
3efd0e33bf lister_transports: load version by using vcversioner's export module 2017-10-10 16:36:38 +02:00
Nicolas Dandrimont
a71dc779d1 setup.py: update for swh.lister rename 2017-06-12 18:14:08 +02:00
Antoine Pietri
ede9e5048c requirements: split internal and external requirements in two separate files 2017-02-09 14:32:02 +01:00
Nicolas Dandrimont
9809deec00 setup.py: install ghlister script 2016-10-20 17:28:22 +02:00
Nicolas Dandrimont
c20bbab4ef Add python packaging metadata 2016-03-17 18:02:45 +01:00