~helmut/debian-dedup.git - Unnamed repository; edit this file 'description' to name the repository.

Age	Commit message (Collapse)	Author
2016-05-23	remove curl dependency	Helmut Grohne
	Teach importpkg how to download urls using urlopen and thus remove the need for invoking curl.
2016-05-23	autoimport: add option to skip hash checking	Helmut Grohne
	For variations of dedup, that do not consume the data.tar member, this option can save significant bandwidth.
2016-05-22	autoimport: stream package list and use generic decompressor	Helmut Grohne
	* streaming means that we do not need to hold the entire package list in memory (but the pkgs dict will become large anyway). * The decompress utility allows easily switching to e.g. xz which is the only compression format for the dbgsym suites.
2016-05-21	move from deprecated optparse to argparse	Helmut Grohne

2016-05-01	use same Python version for autoimport and importpkg	Helmut Grohne
	The autoimport tool runs the Python interpreter explicitly. Instead of invoking just "python" and thus calling whatever the current default is, use sys.executable which is the interpreter used to run autoimport, thus locking both to the same Python version.
2016-04-08	urlopen moved from urllib to urllib.request in py3k	Helmut Grohne

2015-04-16	tempfile.mkdtemp does not like bytes in py3k	Helmut Grohne

2015-04-16	unquote moved from urllib to urllib.parse in py3k	Helmut Grohne

2014-04-21	autoimport: support protocols besides http	Helmut Grohne

2013-09-02	add option -d --database for db path to all scripts	Helmut Grohne

2013-09-02	autoimport: avoid hard coded temporary directory	Helmut Grohne

2013-07-26	verify package hashes when importing via http	Helmut Grohne

2013-07-10	schema: reference package table by integer key	Helmut Grohne
	One approach to improve performance is to reduce the database size. A package name takes up 15 bytes in average. A number of a package takes up two bytes. Multiply that difference with the number of references and it should be noticeably. A small test set show a reduction by 10%.
2013-06-11	autoimport: don't fork for readyaml	Helmut Grohne
	This appears to be a huge performance boost.
2013-06-11	autoimport: support processing individual files	Helmut Grohne
	This gets back the original functionality of importpkg.py.
2013-06-10	split the import phase to a yaml stream	Helmut Grohne
	importpkg.py now emits a yaml stream instead of updating the database. The acutual updating now happens in readyaml.py. In this process autoimport.py was significantly reworked to import packages in parallel.
2013-03-07	use "ON DELETE CASCADE" clauses	Helmut Grohne

2013-03-07	enable enforcing foreign keys	Helmut Grohne

2013-02-25	autoimport: this is not how foreign key constraints work	Helmut Grohne

2013-02-23	autoimport: log which packages are dropped	Helmut Grohne

2013-02-22	autoimport: fix version check to actually work	Helmut Grohne
	Don't fail on new packages and skip versions already processed again.
2013-02-22	autoimport: skip old versions entirely	Helmut Grohne
	Presumably this is responsible for the blocking curl processes, since importpkg will terminate early when processing an old version.
2013-02-22	autoimport: first wait on the import	Helmut Grohne
	Otherwise the import zombifies and curl blocks.
2013-02-21	rename test.py to importpkg.py	Helmut Grohne

2013-02-20	reduce memory usage of autoimport	Helmut Grohne

2013-02-20	implement autoimport	Helmut Grohne