support multiple namespaces in one layer

[liangchenye]

Signed-off-by: liang chenye liangchenye@huawei.com

[liangchenye]

Hi @jzelinskie @Quentin-M , the PR follows the suggestion from you in #150, PTAL.

It makes the following changes:

1. change *namespace to []namespace in struct Layer
2. return 0~N namespaces while detecting a layer and let the underling feature detector to decide which namespace does it support
3. add a new table - LayerNamespace
   remove namespace_id from talbe Layer and add the related soi/search/delete SQL accordingly.
   When insert a layer into the database, merge its own namespace with its parent's namespace
   and insert them into LayerNamespace.
4. modify the related test cases

[Quentin-M]

Why aren't layer.Parent.Namespaces already loaded? We already make the assumption that the provided Parent layer has been retrieved from the database (see L255).

[liangchenye]

The layer.Parent.Namespaces are loaded in work.go.
But in layer_test, it is not. In FindLayer (around L55), the parent's ID and name are loaded, but parent's namespace is not.

As we already make the assumption that the provided Parent layer has been retrieved.
So either we modify the related test cases or load parent's namespace in FindLayer.
My second commits did the later choice, it is easier for developers who use database interface directly but it will be time consuming.

[liangchenye]

in new PR, I'll update test cases. So will not reload parent's namespaces.

[jzelinskie]

NamespaceNames

[jzelinskie]

When complete, this change will also need a database migration to move all current installations to the new schema.

[liangchenye]

Add 'version' field to Namespace. (commit: add 'version' to namespace')

I have two questions:

1. Since it breaks V1, should there be a V2 branch?
   (commit: testing commit: modify V1 by add Namespace.Version)
2. about the database migration, is that mean should we keep the origial 2015*.sql and anther sql file? Also, should we add a .go file to migrate v1 data to this new version?

[jzelinskie]

This should be rebased now that it cannot be merged anymore and then reviewed again.

Master is going to be the branch for "v2" work, we're managing the branches similar to etcd where we have branches for each minor release.

The idea behind migrations is to always move forward by adding new files. This includes writing code (not just SQL) to upgrade from the old data schemas.

[liangchenye]

Rebased in my branch https://github.com/liangchenye/clair/tree/multiple-detector.

I don't know why it becomes an 'unknown' repository. Once the upgrading code was done, I'll add a new PR and close this one.

[liangchenye]

Thanks @jzelinskie , some create/alter table work could be done by SQL, but the sequence between SQL and code matters. So now I'm working on adding a pure migration code.

[liangchenye]

Hi @jzelinskie , I don't know if you have ever meet the same problem in goose.
Seems goose does not work well with 'vendor' ?

And another thing is, a user has to have goose lib under his/her GOPATH directory.
The goose golang migration is quite different with goose sql migration, it genereates .go files under a /tmp/goose* directory and then 'go run' them. So Clair will depend on a go-dev environment , is it acceptable?

[liangchenye]

As comment above, .go migration is not well supported by goose. (to my knowledge).
I change to use 'pure' sql way.
It works good and pass tests :) https://github.com/liangchenye/clair/tree/multiple-detector.
I'll review that thoroughly and submit again next week.

[liangchenye]

close and work on #193

[Quentin-M]

Hi,

Thank you again for your work on this.

liamstask/goose is clearly problematic. Issue #93 tracks it. There is a terrible lack of solid solution for database migration in Go. As far as I know, none of them except liamstask/goose support doing migrations using go code and none can do it without having a go env. That is a huge drawback for any Go app that use databases and that can pretend to be ran in production.