Skip to content

support key-value backends
Compare
Choose a tag to compare
@xiuliren xiuliren released this 03 Mar 20:42
· 219 commits to master since this release
v0.5.0
20e9205

version based on Julia 0.5

features

once an Associate type was defined and values were retrievable using Dict indexing, it could be used as a backend of BigArray. So it is highly extensible.

current backends include:

  • google cloud storage
  • aws s3

test

It was used for pinky40 dataset processing as a big test.

problems

speed

the speed is pretty slow due to single processing reading, should be extensible using multiple threads in the future. Current Julia do not support threading iteration of tuples!

compatibility of neuroglancer

current binary chunks were compressed using blosclz, and config file is config.json while neuroglancer uses a json file called info. The basic idea is the same, but implementation is different, so should be strait forward to modify for compatibility of neuroglancer.

Assets 2
Loading