[EXP] Lazy signature loading classes that use manifests #1619
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## latest #1619 +/- ##
==========================================
+ Coverage 82.74% 90.51% +7.76%
==========================================
Files 122 92 -30
Lines 13200 9085 -4115
Branches 1780 1800 +20
==========================================
- Hits 10922 8223 -2699
+ Misses 2014 597 -1417
- Partials 264 265 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
This was referenced Jun 26, 2021
commit 0f03178 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jul 4 13:56:06 2021 -0700 add check for parent & internal locations in MultiIndex commit 5e6d26f Author: C. Titus Brown <titus@idyll.org> Date: Sun Jul 4 13:40:32 2021 -0700 add test for MultiIndex re filenames commit d6257a6 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jul 4 13:31:24 2021 -0700 emit full location of .sig files under directories commit f4b75fe Author: C. Titus Brown <titus@idyll.org> Date: Sun Jul 4 13:30:49 2021 -0700 update describe output test for full filename stuff commit 12c3424 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jul 4 13:23:07 2021 -0700 slight refactor of SignatureLoadingProgress commit 5b02dce Author: C. Titus Brown <titus@idyll.org> Date: Sun Jul 4 08:02:17 2021 -0700 more comment commit 445f131 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jul 4 07:55:49 2021 -0700 update docstrings commit 267895f Author: C. Titus Brown <titus@idyll.org> Date: Sun Jul 4 07:55:40 2021 -0700 add test for DirectoryIndex commit 512bbf2 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jul 4 07:21:14 2021 -0700 add directory load w/manifest into _load_databases commit 7c4949f Author: C. Titus Brown <titus@idyll.org> Date: Sun Jul 4 07:03:42 2021 -0700 add DirectoryIndex commit 9726be9 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jul 4 06:34:47 2021 -0700 rename 'locations' to 'index_locations' for ManifestOfManifests commit aeaf63c Author: C. Titus Brown <titus@idyll.org> Date: Sat Jul 3 16:00:30 2021 -0700 simplify database load list commit 8d1f9a7 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jul 3 15:54:17 2021 -0700 update _signatures_with_internal commit c24772d Author: C. Titus Brown <titus@idyll.org> Date: Sat Jul 3 15:32:24 2021 -0700 retain parent origin for MultiIndex directory loading commit fa3a071 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jul 3 15:15:39 2021 -0700 disable failing tests for now commit 6d51ae4 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jul 3 15:15:34 2021 -0700 revert relpath behavior commit 5c3f5fc Author: C. Titus Brown <titus@idyll.org> Date: Sat Jul 3 12:16:49 2021 -0700 make manifest internal_location relative for MultiIndex commit f7e755f Merge: cb1594f 21f5e63 Author: C. Titus Brown <titus@idyll.org> Date: Fri Jul 2 05:40:40 2021 -0700 Merge branch 'latest' of https://github.com/sourmash-bio/sourmash into add/manifest_lazy_sigfile commit cb1594f Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 27 08:52:21 2021 -0700 add manifest_of_manifests commit 3a25095 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 27 08:04:31 2021 -0700 change from sigfile to lca file for no manifests :) commit cc3b643 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 27 07:58:24 2021 -0700 some basic tests of the lazy loading indices commit 61c1b24 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 27 07:38:31 2021 -0700 doc strings, comments, etc commit 02c485d Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 26 09:24:44 2021 -0700 restore LazyLoadedSigfile, add comments/docstrings commit 75ba72b Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 26 09:12:52 2021 -0700 upgrade debugging output on _load_database commit cc44288 Merge: a60438e e386085 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 26 09:03:09 2021 -0700 Merge branch 'latest' of https://github.com/sourmash-bio/sourmash into add/manifest_lazy_sigfile commit a60438e Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 24 12:39:17 2021 -0700 fix manifest code commit b633db1 Merge: 6e17c1a 9dbd8b5 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 24 11:27:23 2021 -0700 Merge branch 'latest' of https://github.com/sourmash-bio/sourmash into add/manifest_lazy_sigfile commit 6e17c1a Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 05:39:08 2021 -0700 re-add LazyMultiIndex here commit da381a1 Merge: 0a982cb 99199ee Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 05:35:01 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/manifest_lazy_sigfile commit 0a982cb Merge: 6ebec9c 9bb6a9b Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 05:33:43 2021 -0700 Merge branch 'add/picklist_manifests_sbt' into add/manifest_lazy_sigfile commit 99199ee Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 05:32:52 2021 -0700 move manifest stuff to manifest.py commit 0adee52 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 05:28:54 2021 -0700 remove print commit 83e387e Merge: fe83b68 9bb6a9b Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 18:15:48 2021 -0700 Merge branch 'add/picklist_manifests_sbt' into add/picklist_zf_manifests commit fe83b68 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 18:07:23 2021 -0700 revert collection to multiindex commit 60a6eec Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 14:08:34 2021 -0700 change LoadedCollection back over to MultiIndex; remove LazyMultiIndex commit 6ebec9c Merge: 287cb7b c3f1a3d Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:53:58 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/manifest_lazy_sigfile commit 9bb6a9b Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:53:40 2021 -0700 fix header writing commit 7486871 Merge: 4221fc9 c3f1a3d Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:53:03 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/picklist_manifests_sbt commit c3f1a3d Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:42:38 2021 -0700 reverse order of adding to seen set commit 71b81ed Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:41:33 2021 -0700 add docstring commit ed5fb7a Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:40:43 2021 -0700 rename matches_siginfo to matches_manifest_row commit 2756e7d Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:28:37 2021 -0700 add save/load test commit ba2e53c Merge: c243b0e c04f137 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 11:12:07 2021 -0700 Merge branch 'latest' of github.com:dib-lab/sourmash into add/picklist_zf_manifests commit c243b0e Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 11:10:20 2021 -0700 add manifest tests commit e301645 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 10:07:35 2021 -0700 add a test for sig manifest commit e315c90 Merge: d95813e 0814bcc Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 09:58:29 2021 -0700 Merge branch 'latest' of github.com:dib-lab/sourmash into add/picklist_zf_manifests commit 287cb7b Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 19 17:47:04 2021 -0700 a lazy signature loading class, using manifests commit d95813e Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 19 13:52:50 2021 -0700 add manifest versions commit 4221fc9 Merge: d4a9a2e 31018df Author: C. Titus Brown <titus@idyll.org> Date: Fri Jun 18 05:47:28 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/picklist_manifests_sbt commit 31018df Merge: 9e46ff8 74de59a Author: C. Titus Brown <titus@idyll.org> Date: Fri Jun 18 05:44:38 2021 -0700 Merge branch 'latest' of github.com:dib-lab/sourmash into add/picklist_zf_manifests commit d4a9a2e Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 14:01:48 2021 -0700 fix test for manifests commit 2da0085 Merge: a7e153a 9e46ff8 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 14:01:27 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/picklist_manifests_sbt commit 9e46ff8 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 13:55:21 2021 -0700 cleanup of comments etc. commit e1e367a Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 13:49:21 2021 -0700 remove @ctb comments commit 5cad5ff Merge: 54ea3f9 8812142 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 12:22:46 2021 -0700 Merge branch 'add/picklist_selectors' into add/picklist_zf_manifests commit 8812142 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 12:20:45 2021 -0700 further attempt to fix test commit 54ea3f9 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 12:17:27 2021 -0700 only match picklist at end of 'select' commit 122d043 Merge: f697ec4 de6f3c4 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 11:40:39 2021 -0700 Merge branch 'add/picklist_selectors' into add/picklist_zf_manifests commit de6f3c4 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 11:38:35 2021 -0700 remove order dependence from test commit f697ec4 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 09:31:18 2021 -0700 fix coltypes commit 7937292 Merge: bba101c 4d156e9 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 09:24:38 2021 -0700 Merge branch 'add/picklist_selectors' into add/picklist_zf_manifests commit 4d156e9 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 09:13:52 2021 -0700 add docs commit ab286cf Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 08:50:57 2021 -0700 remove debugging print commit c965648 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 08:44:32 2021 -0700 add a test for using prefetch CSV as picklist commit ca6ea4f Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 08:34:08 2021 -0700 add picklist test that checks indexing-and-then-search == index commit bba101c Merge: 39abe57 ba5c8bc Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 08:13:05 2021 -0700 Merge branch 'add/picklist_selectors' into add/picklist_zf_manifests commit ba5c8bc Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 07:47:13 2021 -0700 block multiple picklists on SBTs and LCAs, for now commit a074127 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 07:38:45 2021 -0700 add picklists to lca index commit a0335a3 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 07:32:10 2021 -0700 add picklists to sourmash compare commit c0e5781 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 07:25:59 2021 -0700 add picklists to prefetch commit 7a30b20 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 07:12:17 2021 -0700 add picklists and tests for search, gather, index commit ced72d2 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 06:25:01 2021 -0700 add picklist args throughout, eek. commit 984a557 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 06:14:23 2021 -0700 fix space commit fddf141 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 06:14:09 2021 -0700 move picklist reporting into sourmash_args commit b3c6bb9 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 06:09:40 2021 -0700 move picklist.py from sourmash.sig into sourmash commit 21ce4b7 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 15:41:51 2021 -0700 fix tests for new SignaturePicklist commit b8f4bb8 Merge: 8e5fb8d b787b75 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 15:40:10 2021 -0700 Merge branch 'latest' of github.com:dib-lab/sourmash into add/picklist_selectors commit 8e5fb8d Merge: 5ac4671 04c209c Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 12:05:31 2021 -0700 Merge branch 'add/picklist' into add/picklist_selectors commit 04c209c Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 11:27:21 2021 -0700 remove comment commit 14b87d4 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 11:18:32 2021 -0700 trap errors and be nice to users commit 4f8e20c Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 11:16:49 2021 -0700 cover untested code with tests commit 8f65f22 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 10:17:50 2021 -0700 test with --md5 selector commit 9d60e32 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 10:15:06 2021 -0700 documentation commit 3d23d87 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 09:39:59 2021 -0700 add --picklist-require-all &c commit 14a88a7 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 09:34:43 2021 -0700 verify output commit 207a813 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 09:31:22 2021 -0700 more picklist tests commit 9b50748 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 09:19:40 2021 -0700 fix tests :) commit aaa4548 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 09:18:08 2021 -0700 update comments, constructor, etc. commit a7e153a Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 07:22:39 2021 -0700 fix tests commit c356842 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 06:12:24 2021 -0700 done, I think? commit 75dc079 Merge: 1dd8170 39abe57 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 05:25:38 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/picklist_manifests_sbt commit 39abe57 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 05:23:42 2021 -0700 CSV output function commit 1dd8170 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 05:20:35 2021 -0700 add manifests to SBTs commit 72d8497 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 04:23:48 2021 -0700 move manifest stuff into manifest class commit a4057e6 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 10:33:09 2021 -0700 create LazyMultiIndex commit 730a717 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 09:40:45 2021 -0700 more cleanup and docs commit 230c793 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 09:28:09 2021 -0700 cleanup and simplification of ZipFile stuff commit 8a8c3b2 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 08:54:23 2021 -0700 shift signature metadata matching from manifests over to picklist commit ab0fc0e Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 08:45:06 2021 -0700 misc cleanup commit c3b6fc0 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 08:36:08 2021 -0700 more cleanup commit 509eb45 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 08:23:41 2021 -0700 remove MultiIndex commit af5eb86 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 08:09:53 2021 -0700 fix test names for new LoadedCollection commit c6cb1af Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 08:08:46 2021 -0700 fix all the tests commit 915f847 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 07:48:35 2021 -0700 cleanup/simplification of LoadedCollection commit be9ef77 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 07:41:48 2021 -0700 create LoadedCollection to replace MultiIndex non-lazy loading commit 3c0c9cf Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 07:17:08 2021 -0700 try making manifests obligatory for MultiIndex commit 23c1531 Merge: 67a9be1 5ac4671 Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 13:35:20 2021 -0700 Merge branch 'add/picklist_selectors' into add/picklist_zf_manifests commit 5ac4671 Merge: a88b66d 031522c Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 13:35:04 2021 -0700 Merge branch 'add/picklist' into add/picklist_selectors commit 031522c Merge: 3c05f95 ff75ec0 Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 13:34:37 2021 -0700 Merge branch 'latest' of github.com:dib-lab/sourmash into add/picklist commit 67a9be1 Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 11:46:07 2021 -0700 more comment commit 1d7e0cf Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 11:41:22 2021 -0700 update comment about picklist.found commit 2f2269b Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 11:33:56 2021 -0700 work through manifests for MultiIndex commit cb8e28d Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 06:34:34 2021 -0700 get started adding manifests to MultiIndex commit 01d33fc Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 06:21:05 2021 -0700 provide 'select' more generically on manifests commit 17b9576 Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 06:12:39 2021 -0700 build out a manifest class a bit commit b2547f3 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 13 20:20:28 2021 -0700 add missing manifest CLI file commit 14a5ee1 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 13 08:59:00 2021 -0700 hacky but functional manifest support commit 6593a42 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 13 08:37:05 2021 -0700 try out manifests commit e205e64 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 13 07:39:47 2021 -0700 special case md5 prefixes, for prefetch commit b57b2b3 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 13 07:23:46 2021 -0700 support special picklist interactions with zipfile collections commit a88b66d Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 13 06:32:38 2021 -0700 factor out picklist checks to 'passes_all_picklists' fn commit 54407a3 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 11:16:02 2021 -0700 test 'Index.find' on picklists for SBTs and LCAs commit 03cc61b Merge: de6fc06 3c05f95 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:48:59 2021 -0700 Merge branch 'add/picklist' into add/picklist_selectors commit 3c05f95 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:48:15 2021 -0700 split column_type out of SignaturePicklist a bit commit 1bdf88e Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:28:23 2021 -0700 split pickfile out a little bit commit de6fc06 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:48:30 2021 -0700 picklist tests for .signatures() methods on Index classes commit def1933 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:48:15 2021 -0700 split column_type out of SignaturePicklist a bit commit a817843 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:28:23 2021 -0700 split pickfile out a little bit commit b1fc982 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:01:56 2021 -0700 add picklists to selectors commit 74f31f5 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 09:21:58 2021 -0700 track found etc commit 505b04f Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 09:11:24 2021 -0700 basic tests for picklist functionality commit 3ecfb48 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 08:55:28 2021 -0700 integrate picklists into sourmash sig extract commit bb794ec Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 08:38:24 2021 -0700 initial picklist implementation commit 3a583a9 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 07:36:48 2021 -0700 clean up sourmash.sig submodule commit 66b0599 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 07:23:13 2021 -0700 cleanup flakes errors commit 0997834 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 07:17:44 2021 -0700 various cleanups of sourmash_args
0f03178
to
79c0b86
Compare
ctb
added a commit
that referenced
this pull request
Jul 5, 2021
* port docstrings and MultiIndex improvements over from #1619 * remove unused import * add tests for MultiIndex manifests, now that they work * comment and parent cleanup
…o add/manifest_lazy_sigfile
…o add/manifest_lazy_sigfile
1 task
…o add/lazy_load_index
…o add/manifest_lazy_sigfile
ctb
added a commit
that referenced
this pull request
Oct 26, 2021
… to use `ZipStorage` (#1598) * port docstrings and MultiIndex improvements over from #1619 * remove unused import * add tests for MultiIndex manifests, now that they work * comment and parent cleanup * Squashed commit of the following: commit 3268907 Author: C. Titus Brown <titus@idyll.org> Date: Fri Jul 2 05:38:56 2021 -0700 remove leftover merge code commit 3b53de9 Merge: 0f7dc81 21f5e63 Author: C. Titus Brown <titus@idyll.org> Date: Fri Jul 2 05:35:54 2021 -0700 Merge branch 'latest' of https://github.com/sourmash-bio/sourmash into add/zipfile_use_storage commit 0f7dc81 Author: C. Titus Brown <titus@idyll.org> Date: Fri Jun 25 10:45:34 2021 -0700 fix error message commit fc0c6fe Merge: 65646fb a5a52b1 Author: C. Titus Brown <titus@idyll.org> Date: Fri Jun 25 10:17:33 2021 -0700 Merge branch 'latest' of https://github.com/sourmash-bio/sourmash into add/zipfile_use_storage commit 65646fb Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 24 11:35:55 2021 -0700 fix merge commit dec537a Merge: c039fd6 9dbd8b5 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 24 11:31:47 2021 -0700 Merge branch 'latest' of https://github.com/sourmash-bio/sourmash into add/zipfile_use_storage commit c039fd6 Merge: 89fad20 8cc96cd Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 17:48:50 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/zipfile_use_storage commit 8cc96cd Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 15:43:19 2021 -0700 fix tests for a CLEAN test-data/prot/ directory commit 5e49336 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 10:32:49 2021 -0700 update docstring commit 2438d90 Merge: 873592d 0ff54e7 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 10:34:23 2021 -0700 Merge branch 'latest' of https://github.com/sourmash-bio/sourmash into add/picklist_zf_manifests commit 89fad20 Merge: c6a8ad7 0ff54e7 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 10:33:13 2021 -0700 Merge branch 'latest' of https://github.com/sourmash-bio/sourmash into add/zipfile_use_storage commit c6a8ad7 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 10:32:49 2021 -0700 update docstring commit bd753d2 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 09:04:56 2021 -0700 fix a few more things commit 86ac7ad Merge: 41438a6 873592d Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 09:03:35 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/zipfile_use_storage commit 873592d Merge: b6d5547 1992de9 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 08:48:31 2021 -0700 Merge branch 'latest' of https://github.com/sourmash-bio/sourmash into add/picklist_zf_manifests commit b6d5547 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 08:47:02 2021 -0700 add test for multiple selects commit 8ebac0d Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 08:37:25 2021 -0700 remove print statements commit 701878b Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 08:36:45 2021 -0700 update test files to have manifest, update tests commit faad6ee Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 08:17:40 2021 -0700 don't test manifest content commit 44aba07 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 08:08:58 2021 -0700 more refactor zipfile select commit 61ce0f2 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 07:37:53 2021 -0700 refactor zipfile select commit 5879ff2 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 07:37:28 2021 -0700 check compatibility in MinHash.intersection_and_union commit e1c44a6 Merge: 6c1f9da d473199 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 07:14:38 2021 -0700 Merge branch 'latest' of http://github.com/sourmash-bio/sourmash into add/picklist_zf_manifests commit 6c1f9da Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 07:12:58 2021 -0700 more manifest testing for zipfiles commit 1b2cf73 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 07:04:47 2021 -0700 add use_manifest fixture, refactor manifest loading commit 38ec792 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 06:50:13 2021 -0700 add sig manifest tests for other file types commit 6905d40 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 06:42:51 2021 -0700 update sig manifest to error when manifests cannot be generated commit fa47667 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 06:19:51 2021 -0700 rename signatures_with_internal to _signatures_with_internal commit 096b141 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 05:46:32 2021 -0700 add manifests to default zip collection output commit 99199ee Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 05:32:52 2021 -0700 move manifest stuff to manifest.py commit 0adee52 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 23 05:28:54 2021 -0700 remove print commit 83e387e Merge: fe83b68 9bb6a9b Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 18:15:48 2021 -0700 Merge branch 'add/picklist_manifests_sbt' into add/picklist_zf_manifests commit fe83b68 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 18:07:23 2021 -0700 revert collection to multiindex commit 60a6eec Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 14:08:34 2021 -0700 change LoadedCollection back over to MultiIndex; remove LazyMultiIndex commit 9bb6a9b Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:53:40 2021 -0700 fix header writing commit 7486871 Merge: 4221fc9 c3f1a3d Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:53:03 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/picklist_manifests_sbt commit 41438a6 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:52:32 2021 -0700 fix header writing commit 4026855 Merge: 6b18439 c3f1a3d Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:48:33 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/zipfile_use_storage commit c3f1a3d Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:42:38 2021 -0700 reverse order of adding to seen set commit 71b81ed Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:41:33 2021 -0700 add docstring commit ed5fb7a Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:40:43 2021 -0700 rename matches_siginfo to matches_manifest_row commit 2756e7d Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 12:28:37 2021 -0700 add save/load test commit ba2e53c Merge: c243b0e c04f137 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 11:12:07 2021 -0700 Merge branch 'latest' of github.com:dib-lab/sourmash into add/picklist_zf_manifests commit c243b0e Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 11:10:20 2021 -0700 add manifest tests commit e301645 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 10:07:35 2021 -0700 add a test for sig manifest commit e315c90 Merge: d95813e 0814bcc Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 22 09:58:29 2021 -0700 Merge branch 'latest' of github.com:dib-lab/sourmash into add/picklist_zf_manifests commit d95813e Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 19 13:52:50 2021 -0700 add manifest versions commit 4221fc9 Merge: d4a9a2e 31018df Author: C. Titus Brown <titus@idyll.org> Date: Fri Jun 18 05:47:28 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/picklist_manifests_sbt commit 6b18439 Merge: 9ff0eab 31018df Author: C. Titus Brown <titus@idyll.org> Date: Fri Jun 18 05:46:39 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/zipfile_use_storage commit 31018df Merge: 9e46ff8 74de59a Author: C. Titus Brown <titus@idyll.org> Date: Fri Jun 18 05:44:38 2021 -0700 Merge branch 'latest' of github.com:dib-lab/sourmash into add/picklist_zf_manifests commit 9ff0eab Merge: 9c530a5 9e46ff8 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 14:02:08 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/zipfile_use_storage commit d4a9a2e Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 14:01:48 2021 -0700 fix test for manifests commit 2da0085 Merge: a7e153a 9e46ff8 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 14:01:27 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/picklist_manifests_sbt commit 9e46ff8 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 13:55:21 2021 -0700 cleanup of comments etc. commit e1e367a Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 13:49:21 2021 -0700 remove @ctb comments commit 5cad5ff Merge: 54ea3f9 8812142 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 12:22:46 2021 -0700 Merge branch 'add/picklist_selectors' into add/picklist_zf_manifests commit 8812142 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 12:20:45 2021 -0700 further attempt to fix test commit 54ea3f9 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 12:17:27 2021 -0700 only match picklist at end of 'select' commit 122d043 Merge: f697ec4 de6f3c4 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 11:40:39 2021 -0700 Merge branch 'add/picklist_selectors' into add/picklist_zf_manifests commit de6f3c4 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 11:38:35 2021 -0700 remove order dependence from test commit f697ec4 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 09:31:18 2021 -0700 fix coltypes commit 7937292 Merge: bba101c 4d156e9 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 09:24:38 2021 -0700 Merge branch 'add/picklist_selectors' into add/picklist_zf_manifests commit 4d156e9 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 09:13:52 2021 -0700 add docs commit ab286cf Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 08:50:57 2021 -0700 remove debugging print commit c965648 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 08:44:32 2021 -0700 add a test for using prefetch CSV as picklist commit ca6ea4f Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 08:34:08 2021 -0700 add picklist test that checks indexing-and-then-search == index commit bba101c Merge: 39abe57 ba5c8bc Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 08:13:05 2021 -0700 Merge branch 'add/picklist_selectors' into add/picklist_zf_manifests commit ba5c8bc Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 07:47:13 2021 -0700 block multiple picklists on SBTs and LCAs, for now commit a074127 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 07:38:45 2021 -0700 add picklists to lca index commit a0335a3 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 07:32:10 2021 -0700 add picklists to sourmash compare commit c0e5781 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 07:25:59 2021 -0700 add picklists to prefetch commit 7a30b20 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 07:12:17 2021 -0700 add picklists and tests for search, gather, index commit ced72d2 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 06:25:01 2021 -0700 add picklist args throughout, eek. commit 984a557 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 06:14:23 2021 -0700 fix space commit fddf141 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 06:14:09 2021 -0700 move picklist reporting into sourmash_args commit b3c6bb9 Author: C. Titus Brown <titus@idyll.org> Date: Thu Jun 17 06:09:40 2021 -0700 move picklist.py from sourmash.sig into sourmash commit 21ce4b7 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 15:41:51 2021 -0700 fix tests for new SignaturePicklist commit b8f4bb8 Merge: 8e5fb8d b787b75 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 15:40:10 2021 -0700 Merge branch 'latest' of github.com:dib-lab/sourmash into add/picklist_selectors commit 8e5fb8d Merge: 5ac4671 04c209c Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 12:05:31 2021 -0700 Merge branch 'add/picklist' into add/picklist_selectors commit 04c209c Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 11:27:21 2021 -0700 remove comment commit 14b87d4 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 11:18:32 2021 -0700 trap errors and be nice to users commit 4f8e20c Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 11:16:49 2021 -0700 cover untested code with tests commit 8f65f22 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 10:17:50 2021 -0700 test with --md5 selector commit 9d60e32 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 10:15:06 2021 -0700 documentation commit 3d23d87 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 09:39:59 2021 -0700 add --picklist-require-all &c commit 14a88a7 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 09:34:43 2021 -0700 verify output commit 207a813 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 09:31:22 2021 -0700 more picklist tests commit 9b50748 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 09:19:40 2021 -0700 fix tests :) commit aaa4548 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 09:18:08 2021 -0700 update comments, constructor, etc. commit a7e153a Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 07:22:39 2021 -0700 fix tests commit 9c530a5 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 07:12:46 2021 -0700 add comment about Storage encapsulation commit 48fd900 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 07:07:17 2021 -0700 all tests pass, w00t commit d6a48c1 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 07:01:34 2021 -0700 refactor ZipFileLinearIndex to use ZipStorage underneath commit 5a185bb Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 06:45:42 2021 -0700 change internal zipfile writing to use ZipStorage commit c356842 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 06:12:24 2021 -0700 done, I think? commit 75dc079 Merge: 1dd8170 39abe57 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 05:25:38 2021 -0700 Merge branch 'add/picklist_zf_manifests' into add/picklist_manifests_sbt commit 39abe57 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 05:23:42 2021 -0700 CSV output function commit 1dd8170 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 05:20:35 2021 -0700 add manifests to SBTs commit 72d8497 Author: C. Titus Brown <titus@idyll.org> Date: Wed Jun 16 04:23:48 2021 -0700 move manifest stuff into manifest class commit a4057e6 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 10:33:09 2021 -0700 create LazyMultiIndex commit 730a717 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 09:40:45 2021 -0700 more cleanup and docs commit 230c793 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 09:28:09 2021 -0700 cleanup and simplification of ZipFile stuff commit 8a8c3b2 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 08:54:23 2021 -0700 shift signature metadata matching from manifests over to picklist commit ab0fc0e Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 08:45:06 2021 -0700 misc cleanup commit c3b6fc0 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 08:36:08 2021 -0700 more cleanup commit 509eb45 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 08:23:41 2021 -0700 remove MultiIndex commit af5eb86 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 08:09:53 2021 -0700 fix test names for new LoadedCollection commit c6cb1af Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 08:08:46 2021 -0700 fix all the tests commit 915f847 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 07:48:35 2021 -0700 cleanup/simplification of LoadedCollection commit be9ef77 Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 07:41:48 2021 -0700 create LoadedCollection to replace MultiIndex non-lazy loading commit 3c0c9cf Author: C. Titus Brown <titus@idyll.org> Date: Tue Jun 15 07:17:08 2021 -0700 try making manifests obligatory for MultiIndex commit 23c1531 Merge: 67a9be1 5ac4671 Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 13:35:20 2021 -0700 Merge branch 'add/picklist_selectors' into add/picklist_zf_manifests commit 5ac4671 Merge: a88b66d 031522c Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 13:35:04 2021 -0700 Merge branch 'add/picklist' into add/picklist_selectors commit 031522c Merge: 3c05f95 ff75ec0 Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 13:34:37 2021 -0700 Merge branch 'latest' of github.com:dib-lab/sourmash into add/picklist commit 67a9be1 Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 11:46:07 2021 -0700 more comment commit 1d7e0cf Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 11:41:22 2021 -0700 update comment about picklist.found commit 2f2269b Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 11:33:56 2021 -0700 work through manifests for MultiIndex commit cb8e28d Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 06:34:34 2021 -0700 get started adding manifests to MultiIndex commit 01d33fc Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 06:21:05 2021 -0700 provide 'select' more generically on manifests commit 17b9576 Author: C. Titus Brown <titus@idyll.org> Date: Mon Jun 14 06:12:39 2021 -0700 build out a manifest class a bit commit b2547f3 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 13 20:20:28 2021 -0700 add missing manifest CLI file commit 14a5ee1 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 13 08:59:00 2021 -0700 hacky but functional manifest support commit 6593a42 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 13 08:37:05 2021 -0700 try out manifests commit e205e64 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 13 07:39:47 2021 -0700 special case md5 prefixes, for prefetch commit b57b2b3 Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 13 07:23:46 2021 -0700 support special picklist interactions with zipfile collections commit a88b66d Author: C. Titus Brown <titus@idyll.org> Date: Sun Jun 13 06:32:38 2021 -0700 factor out picklist checks to 'passes_all_picklists' fn commit 54407a3 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 11:16:02 2021 -0700 test 'Index.find' on picklists for SBTs and LCAs commit 03cc61b Merge: de6fc06 3c05f95 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:48:59 2021 -0700 Merge branch 'add/picklist' into add/picklist_selectors commit 3c05f95 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:48:15 2021 -0700 split column_type out of SignaturePicklist a bit commit 1bdf88e Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:28:23 2021 -0700 split pickfile out a little bit commit de6fc06 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:48:30 2021 -0700 picklist tests for .signatures() methods on Index classes commit def1933 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:48:15 2021 -0700 split column_type out of SignaturePicklist a bit commit a817843 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:28:23 2021 -0700 split pickfile out a little bit commit b1fc982 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 10:01:56 2021 -0700 add picklists to selectors commit 74f31f5 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 09:21:58 2021 -0700 track found etc commit 505b04f Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 09:11:24 2021 -0700 basic tests for picklist functionality commit 3ecfb48 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 08:55:28 2021 -0700 integrate picklists into sourmash sig extract commit bb794ec Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 08:38:24 2021 -0700 initial picklist implementation commit 3a583a9 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 07:36:48 2021 -0700 clean up sourmash.sig submodule commit 66b0599 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 07:23:13 2021 -0700 cleanup flakes errors commit 0997834 Author: C. Titus Brown <titus@idyll.org> Date: Sat Jun 12 07:17:44 2021 -0700 various cleanups of sourmash_args * fix test by raising proper error * update comments * add relevant tests * fix tests and new, exciting bug * add tests for hand-created zipfile with, and without, manifests * remove outdated comment
…o add/manifest_lazy_sigfile
…o add/manifest_lazy_sigfile_upd
5 tasks
|
note,
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR extends manifest functionality:
DirectoryIndexthat makes use of manifests.LazyMultiIndexthat provisionally supports full lazy loading of signatures from indexes using manifests.See #1096 (comment) for originating motivation.
There is no particular urgency in moving towards making this merge-ready until we have more use cases. Things to consider:
DirectoryIndexto useFSStorageunderneath;_directoryindex_loadcause any potential problems?src/sourmash/sig/__main__.pyinto their own PR;LazyMultiIndexfromDirectoryIndex; I'm still not sure if the latter is actually useful!