Prepare log-index lookup for sharding

[lkatalin]

The log-index lookup should resolve the virtual index to the treeID + index for a specific shard.

[lkatalin]

Since GetLogEntryByIndexHandler() instantiates a trillian client to query based on the NewTrillianClient() command, which takes the treeID/logID[1] from a default in api during server set-up in serve.go, it seems that the necessary change (if any) would be not in the GetLogEntryByIndexHandler() function itself, but in the way the TreeID is set during API configuration.

Currently this seems to be set once via a flag when the Rekor server is first run, which made sense when there was only one tree but probably won't make sense anymore.[2] I think the Rekor server set-up would have to be decoupled from instantiating new trillian clients (as it's one client per treeID from what I can tell). Does this sound right?

[1] Am I reading correctly that the API's logID is equivalent to a trillian TreeID? Should these terms be merged?

[2] The logRangesMap also appears to be passed in via a flag as if the user is supposed to provide this. What is the way this is intended to be used?

Tagging @dlorenc @bobcallaway for any guidance - thanks!

[lkatalin]

[1] Am I reading correctly that the API's logID is equivalent to a trillian TreeID? Should these terms be merged?

I was able to grab @dlorenc for a brief meeting and he confirmed these two terms do refer to the same thing.

[2] The logRangesMap also appears to be passed in via a flag as if the user is supposed to provide this. What is the way this is intended to be used?

My current question is still around how the logRangesMap flag is meant to be used - I was thinking of logRanges being a global var that gets updated when a new artifact is uploaded (adds to active tree len) or when a new shard is created (freezes last tree, adds new active tree). I've grabbed another slot with @dlorenc on Thurs morning, but if someone is able to comment on this beforehand that would help me get to working on it faster. Thanks!

[lkatalin]

Ex. is sending a command to the Rekor server that includes this logRangesMap flag + value supposed to create a new TrillianClient with updated treeID?

[lkatalin]

Notes from my meeting with @dlorenc :

• The server is going to restart every time we shard. That's when this logRangesMap flag is passed in with new info.

• I'm not really worrying about calculating the correct thing to pass in via the logRangesMap flag - this is largely manual.

• Don't currently need to edit config/rekor.yaml

• The todo is creating or choosing the correct trillian client based on the logRangesMap:

  • For a write, get the active index and talk to it
  • For a read, we can create multiple trillian clients (at startup or on the fly) -> I think it's best to do it on the fly in the handler