summaryrefslogtreecommitdiff
path: root/autoimport.py
AgeCommit message (Collapse)Author
2016-05-21move from deprecated optparse to argparseHelmut Grohne
2016-05-01use same Python version for autoimport and importpkgHelmut Grohne
The autoimport tool runs the Python interpreter explicitly. Instead of invoking just "python" and thus calling whatever the current default is, use sys.executable which is the interpreter used to run autoimport, thus locking both to the same Python version.
2016-04-08urlopen moved from urllib to urllib.request in py3kHelmut Grohne
2015-04-16tempfile.mkdtemp does not like bytes in py3kHelmut Grohne
2015-04-16unquote moved from urllib to urllib.parse in py3kHelmut Grohne
2014-04-21autoimport: support protocols besides httpHelmut Grohne
2013-09-02add option -d --database for db path to all scriptsHelmut Grohne
2013-09-02autoimport: avoid hard coded temporary directoryHelmut Grohne
2013-07-26verify package hashes when importing via httpHelmut Grohne
2013-07-10schema: reference package table by integer keyHelmut Grohne
One approach to improve performance is to reduce the database size. A package name takes up 15 bytes in average. A number of a package takes up two bytes. Multiply that difference with the number of references and it should be noticeably. A small test set show a reduction by 10%.
2013-06-11autoimport: don't fork for readyamlHelmut Grohne
This appears to be a huge performance boost.
2013-06-11autoimport: support processing individual filesHelmut Grohne
This gets back the original functionality of importpkg.py.
2013-06-10split the import phase to a yaml streamHelmut Grohne
importpkg.py now emits a yaml stream instead of updating the database. The acutual updating now happens in readyaml.py. In this process autoimport.py was significantly reworked to import packages in parallel.
2013-03-07use "ON DELETE CASCADE" clausesHelmut Grohne
2013-03-07enable enforcing foreign keysHelmut Grohne
2013-02-25autoimport: this is not how foreign key constraints workHelmut Grohne
2013-02-23autoimport: log which packages are droppedHelmut Grohne
2013-02-22autoimport: fix version check to actually workHelmut Grohne
Don't fail on new packages and skip versions already processed again.
2013-02-22autoimport: skip old versions entirelyHelmut Grohne
Presumably this is responsible for the blocking curl processes, since importpkg will terminate early when processing an old version.
2013-02-22autoimport: first wait on the importHelmut Grohne
Otherwise the import zombifies and curl blocks.
2013-02-21rename test.py to importpkg.pyHelmut Grohne
2013-02-20reduce memory usage of autoimportHelmut Grohne
2013-02-20implement autoimportHelmut Grohne