summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2013-03-26Merge branch schemachangeHelmut Grohne
2013-03-20webapp: report correct sizesHelmut Grohne
2013-03-20webapp: remove broken assertHelmut Grohne
Fails on long inputs.
2013-03-18dedup.image: mask errors from PILHelmut Grohne
2013-03-12dedup.arreader: missing bytes markerHelmut Grohne
2013-03-12move ArReader from importpkg to dedup.arreaderHelmut Grohne
Also document it.
2013-03-10README: update queries to match content table splitHelmut Grohne
2013-03-09split content table to a hash tableHelmut Grohne
In the old content table (package, filename, size) would be the same for multiple hash functions. Now the schema represents that each file has precisely one size, but multiple hashes.
2013-03-09webapp: drop unused function compute_sharedstatsHelmut Grohne
The sharing table works great and I don't want to adapt it for the next step in the schema change.
2013-03-07use "ON DELETE CASCADE" clausesHelmut Grohne
2013-03-07enable enforcing foreign keysHelmut Grohne
2013-03-07schema.sql: remove unsatisfiable foreign keyHelmut Grohne
In the dependency table we will insert dependencies on packages which are not tracked. This happens during initial import and for virtual packages. Therefore the "required" column cannot be a foreign key.
2013-03-07schema.sql: annotat foreign keys of sharingHelmut Grohne
2013-03-07integrate the source table into the package tableHelmut Grohne
2013-03-07README: explain queriesHelmut Grohne
2013-03-06README: added interesting queryHelmut Grohne
2013-03-05webapp: added /source/<pkg> pageHelmut Grohne
2013-03-05webapp: helper function function_combinationHelmut Grohne
2013-03-05importpkg: source header may contain a versionHelmut Grohne
2013-03-04webapp: fix index templateHelmut Grohne
Apparently not all browsers understand <a ... /> in all rendering modes.
2013-03-04webapp: use caching table "shared" for /binary pageHelmut Grohne
2013-03-04webapp: generate /comparison pages in constant-spaceHelmut Grohne
2013-03-04importpkg: record the source package relationshipHelmut Grohne
2013-03-02update_sharing: wrong database nameHelmut Grohne
2013-03-02add sharing tableHelmut Grohne
The sharing table is a cache for the /binary web pages. It essentially contains the numbers presented. This caching table is not automatically populated. It needs to be reconstructed after every (group of) package imports.
2013-03-02update READMEHelmut Grohne
* Tell about schema.sql. * Explain WAL.
2013-03-02move fetchiter from webapp to dedup.utilsHelmut Grohne
2013-03-02move sql schema to a separate fileHelmut Grohne
2013-03-02added html form to main pageHelmut Grohne
Thanks to Jan Luehr for doing the work.
2013-02-25webapp: open database cursor lazilyHelmut Grohne
Makes things more correct when using Application in multiprocessing context.
2013-02-25webapp: pass database to Application classHelmut Grohne
2013-02-25README: another interesting queryHelmut Grohne
2013-02-25Merge branch 'crosshash'Helmut Grohne
Conflicts in webapp.py: * The fetchall -> fetchiter change caused big conflicts. * New hash combination (image_sha512, image_sha512) added.
2013-02-25webapp: complete cross hash supportHelmut Grohne
2013-02-25autoimport: this is not how foreign key constraints workHelmut Grohne
2013-02-24hash image contentsHelmut Grohne
2013-02-24README: fix mistakeHelmut Grohne
2013-02-23importpkg: ignore filenames with encoding errorsHelmut Grohne
2013-02-23autoimport: log which packages are droppedHelmut Grohne
2013-02-22autoimport: fix version check to actually workHelmut Grohne
Don't fail on new packages and skip versions already processed again.
2013-02-22autoimport: skip old versions entirelyHelmut Grohne
Presumably this is responsible for the blocking curl processes, since importpkg will terminate early when processing an old version.
2013-02-22webapp: add caching headersHelmut Grohne
2013-02-22webapp: stream responsesHelmut Grohne
Maybe this gets memory usage down for large responses.
2013-02-22webapp: attempt to reduce memory usageHelmut Grohne
2013-02-22webapp: support matching sha512 against gzip_sha512Helmut Grohne
This covers only the /binary page. The comparison may still be empty.
2013-02-22autoimport: first wait on the importHelmut Grohne
Otherwise the import zombifies and curl blocks.
2013-02-21move compression functions to module dedup.compressionHelmut Grohne
2013-02-21do not track byted compiled python filesHelmut Grohne
2013-02-21move hashing functions to module dedup.hashingHelmut Grohne
2013-02-21include maintainer informationHelmut Grohne