No description
Find a file
Avi Kelman (fiendish) 68d77fd43f Refactor lister code
Streamline production of new listers by aggressively moving core
functionality into progressively inherited (A->B->C) base classes
with the transport layer abstracted.
This should make common individual forge listers straightforward to
produce with minimal customization. Github and Bitbucket listers
can be used as examples of the indexing type.
2017-03-06 12:35:49 +01:00
bin lister.py: class-ify the GitHub lister 2016-10-20 14:22:31 +02:00
debian Update storage configuration reading 2016-12-15 19:07:02 +01:00
etc Revert to the pre-qless refactoring version 2016-09-13 14:57:26 +02:00
sql Revert to the pre-qless refactoring version 2016-09-13 14:57:26 +02:00
swh/lister Refactor lister code 2017-03-06 12:35:49 +01:00
.gitignore Add python packaging metadata 2016-03-17 18:02:45 +01:00
ACKNOWLEDGEMENTS add ACKNOWLEDGEMENTS 2015-04-26 15:54:25 +02:00
LICENSE add license information 2015-04-26 16:24:32 +02:00
Makefile Makefile: add from swh-py-template 2015-10-27 14:35:54 +01:00
MANIFEST.in requirements: split internal and external requirements in two separate files 2017-02-09 14:32:02 +01:00
README Revert to the pre-qless refactoring version 2016-09-13 14:57:26 +02:00
requirements-swh.txt requirements: split internal and external requirements in two separate files 2017-02-09 14:32:02 +01:00
requirements.txt Refactor lister code 2017-03-06 12:35:49 +01:00
setup.py requirements: split internal and external requirements in two separate files 2017-02-09 14:32:02 +01:00
TODO Revert to the pre-qless refactoring version 2016-09-13 14:57:26 +02:00

Licensing
=========

This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.  See the GNU General Public License for more details.

See top-level LICENSE file for the full text of the GNU General Public License
along with this program.


Dependencies
============

- python3
- python3-psycopg2
- python3-requests
- python3-sqlalchemy


Deployment
==========

1. git clone under $GHLISTER_ROOT (of your choosing)
2. mkdir ~/.config/swh/ ~/.cache/swh/lister-github/
3. edit $GHLISTER_ROOT/etc/crontab and customize GHLISTER_ROOT
4. crontab $GHLISTER_ROOT/etc/crontab
5. create configuration file ~/.config/swh/lister-github.ini

Sample configuration file
-------------------------

cat ~/.config/swh/lister-github.ini

        [main]
        db_url = postgres:///github
          # see http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls
        cache_dir = /home/zack/.cache/swh/lister-github
        log_dir =   /home/zack/.cache/swh/lister-github
        username = foobar  # github username
        password = quux    # github password