enhancement: multiple block stores with usage policies

[elitak]

Type: Enhancement

Severity: Medium

Description:

If I have a very fast SSD and a very slow magnetic archive disk, I'd like some way to direct the ipfs daemon to use them according to some flags I can set through ipfs config and ipfs pin.

First, there would need to be a way to designate different block stores in separate directories, leaving it to the user to mount whatever disks there appropriately. In the config, each store could be marked with a relative performance value (positive integer; ∈ Z-+).

Second, when setting pins, there could be a similar --availability option to assign a similar value to pinned objects. The initial ipfs add would try to use a store that had a performance value at least as high as the availability value. When garbage collecting, objects could be rearranged in a way that prioritized storing higher-availability objects on faster storage, including: swapping out pinned objects that have had that value lowered; and accounting for storage limits that have changed or been hit, reclaiming more performant space as necessary.

[whyrusleeping]

Very interesting idea. I remember @travisperson talking about a similar thing a while back.

[Kubuxu]

This could mean creating structure that would hold info about blocks, which might be very useful for other things too.

[magik6k]

This might fit into new interfaces proposed in ipfs/notes#255

[Stebalien]

This might fit into new interfaces proposed in ipfs/notes#255

It does.

[travisperson]

There isn't anything in the issue, but it's #403. Here are the IRC logs when I first asked about this for context: https://botbot.me/freenode/ipfs/2014-10-10/?msg=23269334&page=2