Age | Commit message (Collapse) | Author |
|
No explicit "import sqlite3" left. It's still a bit rough around the
corners, particularly since sqlalchemy's support for executemany is
totally broken.
|
|
|
|
One approach to improve performance is to reduce the database size. A
package name takes up 15 bytes in average. A number of a package takes
up two bytes. Multiply that difference with the number of references and
it should be noticeably. A small test set show a reduction by 10%.
|
|
This appears to be a huge performance boost.
|
|
This gets back the original functionality of importpkg.py.
|
|
importpkg.py now emits a yaml stream instead of updating the database.
The acutual updating now happens in readyaml.py. In this process
autoimport.py was significantly reworked to import packages in parallel.
|
|
|
|
|
|
|
|
|
|
Don't fail on new packages and skip versions already processed again.
|
|
Presumably this is responsible for the blocking curl processes, since
importpkg will terminate early when processing an old version.
|
|
Otherwise the import zombifies and curl blocks.
|
|
|
|
|
|
|