Opened 9 months ago

Last modified 8 months ago

#447 new feature

Add an index to Shimmer store files so we can lookup decls in constant-ish time.

Reported by: benl Owned by:
Priority: normal Milestone: memoizer
Component: Shimmer Version: none
Keywords: Cc:

Description

The existing Shimmer binary format stores all the top-level declarations in one big list. Add a proper database-like index to the binary format so that we can lookup declarations in constant-ish time.

For the memoization project we'll store memoized results by appending them to the end of the file, so it may be better to keep the index in a separate file rather than interleave index data with payload data in a single file.

Eventually we'll want to incrementally update the index when new expressions are added to the store, however for the first cut it would be ok to periodically rebuild the index in batch mode and just do a linear scan for recently added declarations.

Change History (1)

comment:1 Changed 8 months ago by benl

  • Summary changed from Add a database-like index so we can lookup decls in binary Shimmer store files. to Add an index to Shimmer store files so we can lookup decls in constant-ish time.
Note: See TracTickets for help on using tickets.