get rid of driver shell script, to simplify deployment

update README and crontab to match
This commit is contained in:
Stefano Zacchiroli 2015-09-15 08:57:16 +02:00
parent 8711fad9fa
commit 35a855ba32
3 changed files with 19 additions and 36 deletions

25
README
View file

@ -19,20 +19,29 @@ Dependencies
============
- python3
- python3-sqlalchemy
- python3-psycopg2
- python3-requests
- python3-sqlalchemy
Configuration file
==================
Deployment
==========
1. git clone under $GHLISTER_ROOT (of your choosing)
2. mkdir ~/.config/swh/ ~/.cache/swh/lister-github/
3. edit $GHLISTER_ROOT/etc/crontab and customize GHLISTER_ROOT
4. crontab $GHLISTER_ROOT/etc/crontab
5. create configuration file ~/.config/swh/lister-github.ini
Sample configuration file
-------------------------
cat ~/.config/swh/lister-github.ini
[main]
# see http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls
db_url = postgres:///github
cache_dir = cache
log_dir = log
username = foobar
password = quux
# see http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls
cache_dir = /home/zack/.cache/swh/lister-github
log_dir = /home/zack/.cache/swh/lister-github
username = foobar # github username
password = quux # github password

View file

@ -1,26 +0,0 @@
#!/bin/bash
# PYTHONPATH and PATH (for ghlister) should be set via cron
# export PYTHONPATH=$HOME/src/swh-lister-github/
log_dir=$(grep log_dir ~/.config/swh/lister-github.ini | cut -f 3 -d' ')
test -d "$log_dir" || mkdir -p "$log_dir"
logfile="${log_dir}/$(date +%Y%m%d).log"
# catchup with recent changes
ghlister catchup &> "$logfile"
# XXX: computing the following stats via this script forced to have double
# configuration for shell-vs-python. Also it's unneeded: we can compute the
# same information (number of repos created per day) from the DB, via the
# last_seen column.
# dbservice=lister-github
# dbconn="service=${dbservice}"
# psql="psql --no-psqlrc --pset t --pset format=unaligned ${dbconn}"
# # compute and store current totals
# all_repos=$(echo "select count(*) from repos" | $psql)
# fork_repos=$(echo "select count(*) from fork_repos" | $psql)
# orig_repos=$(echo "select count(*) from orig_repos" | $psql)
# echo "insert into repos_history(repos, fork_repos, orig_repos) values (${all_repos}, ${fork_repos}, ${orig_repos})" | $psql > /dev/null

View file

@ -1,4 +1,4 @@
GHLISTER_ROOT=$HOME/src/swh-lister-github
GHLISTER_ROOT=~/src/swh-lister-github
# m h dom mon dow command
0 8 * * * PATH=$GHLISTER_ROOT/bin:$PATH PYTHONPATH=$GHLISTER_ROOT update-github-list
0 8 * * * PYTHONPATH=$GHLISTER_ROOT $GHLISTER_ROOT/bin/ghlister catchup >> ~/.cache/swh/lister-github/$(date +\%Y\%m\%d).log 2>&1