reonokiy/swh-lister
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Ignore psqlrc when loading the rubygems database dump

Browse source 
The SQL dump contains ownership instructions that can't be run if you
don't have the right users in your database clusters. When someone has a
psqlrc with ON_ERROR_STOP, this fails the load of the dump.

Use the opportunity to trigger an exception when psql returns a non-zero
exit code, rather than continue with an empty/inconsistent database.
This commit is contained in:
Nicolas Dandrimont 2022-12-05 13:43:32 +01:00
parent f4aafe026b
commit a66e24bfa2
1 changed files with 17 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

18
swh/lister/rubygems/lister.py
View file

@ -119,9 +119,13 @@ class RubyGemsLister(StatelessLister[RubyGemsListerPage]):
dump_tar.extractall(temp_dir)
logger.debug("Populating rubygems database with dump %s", dump_id)
# FIXME: make this work with -v ON_ERROR_STOP=1
psql = subprocess.Popen(
["psql", "-q", db_url],
["psql", "--no-psqlrc", "-q", db_url],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
# passing value of gzip.open as stdin of subprocess.run makes the process
@ -133,6 +137,18 @@ class RubyGemsLister(StatelessLister[RubyGemsListerPage]):
psql.stdin.close() # type: ignore
psql.wait()
if psql.returncode != 0:
assert psql.stdout
for line in psql.stdout.readlines():
logger.warning("psql out: %s", line.decode().strip())
assert psql.stderr
for line in psql.stderr.readlines():
logger.warning("psql err: %s", line.decode().strip())
raise ValueError(
"Loading rubygems dump failed with exit code %s.",
psql.returncode,
)
def get_pages(self) -> Iterator[RubyGemsListerPage]:
# spawn a temporary postgres instance (require initdb executable in environment)
with Postgresql() as postgresql: