~helmut/debian-dedup.git - Unnamed repository; edit this file 'description' to name the repository.

Age	Commit message (Collapse)	Author
2014-07-22	store filenames as blobsbinaryfilename	Helmut Grohne
	To facilitate this importpkg.py decodes filenames using iso-8859-1. While this is not the encoding used for most filenames, decoding will never fail. Conversely, readyaml.py encodes to iso-8859-1 to undo the above effect. webapp.py cannot currently cope with the above change and is broken by this commit.
2013-09-02	add option -d --database for db path to all scripts	Helmut Grohne

2013-07-24	readyaml: cache the whole function table	Helmut Grohne
	This should reduce the query bandwidth to the rdbms.
2013-07-23	schema: reference hash functions by integer key	Helmut Grohne
	This already worked quite well for package.id. On a test data set of 5% size this transformation reduces the database size by about 4%.
2013-07-10	schema: reference package table by integer key	Helmut Grohne
	One approach to improve performance is to reduce the database size. A package name takes up 15 bytes in average. A number of a package takes up two bytes. Multiply that difference with the number of references and it should be noticeably. A small test set show a reduction by 10%.
2013-06-11	autoimport: don't fork for readyaml	Helmut Grohne
	This appears to be a huge performance boost.
2013-06-10	split the import phase to a yaml stream	Helmut Grohne
	importpkg.py now emits a yaml stream instead of updating the database. The acutual updating now happens in readyaml.py. In this process autoimport.py was significantly reworked to import packages in parallel.