RHEL namespaces are not getting created, RHEL fetcher says no updates

[brianwcook]

I am seeing on multiple test instances of clair that there are no RHEL namespaces. The fetcher is reporting there are no updates for RHEL.

clair[32415]: 2016-06-15 13:16:55.005371 I | updater/fetchers/rhel: fetching Red Hat vulnerabilities
clair[32415]: 2016-06-15 13:16:57.090650 D | updater/fetchers/debian: no Debian update
clair[32415]: 2016-06-15 13:16:59.168916 D | updater/fetchers/rhel: no Red Hat update.

From API and database I see centos, debian and ubuntu namespaces. The net result is scanning RHEL based images that are known to have vulnerabilities report no vulnerabilities.

[bartockbat]

Note:

curl http://localhost:6060/v1/namespaces | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   376  100   376    0     0  64571      0 --:--:-- --:--:-- --:--:-- 75200
{
    "Namespaces": [
        {
            "Name": "debian:7"
        },
        {
            "Name": "debian:8"
        },
        {
            "Name": "debian:unstable"
        },
        {
            "Name": "debian:9"
        },
        {
            "Name": "centos:7"
        },
        {
            "Name": "centos:6"
        },
        {
            "Name": "server:7"
        },
        {
            "Name": "rhel:7.2"
        },
        {
            "Name": "ubuntu:16.04"
        },
        {
            "Name": "ubuntu:12.04"
        },
        {
            "Name": "ubuntu:15.04"
        },
        {
            "Name": "ubuntu:14.10"
        },
        {
            "Name": "ubuntu:14.04"
        },
        {
            "Name": "ubuntu:15.10"
        },
        {
            "Name": "ubuntu:13.04"
        },
        {
            "Name": "ubuntu:12.10"
        }
    ]
}

However, when I look at the particular namespace:

curl  http://10.18.1.130:6060/v1/namespaces/rhel%3A7.2/vulnerabilities?limit=10
{"Vulnerabilities":null}

But:

curl  http://10.18.1.130:6060/v1/namespaces/ubuntu%3A12.04/vulnerabilities?limit=1 | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   951  100   951    0     0  10452      0 --:--:-- --:--:-- --:--:-- 10566
{
    "NextPage": "gAAAAABXYaeh8FU6kq1LemPQLXfrfZMD9fkkRSB_fWXTpXbPCadqiyH3mIRvm-Ip-VRwH6RxWBUE8PlH0-8T1bZAOaCxAOrwDA==",
    "Vulnerabilities": [
        {
            "Description": "Multiple cross-site scripting (XSS) vulnerabilities in phpMyAdmin 4.0.x before 4.0.10.15, 4.4.x before 4.4.15.5, and 4.5.x before 4.5.5.1 allow remote attackers to inject arbitrary web script or HTML via (1) a crafted Host HTTP header, related to libraries/Config.class.php; (2) crafted JSON data, related to file_echo.php; (3) a crafted SQL query, related to js/functions.js; (4) the initial parameter to libraries/server_privileges.lib.php in the user accounts page; or (5) the it parameter to libraries/controllers/TableSearchController.class.php in the zoom search page.",
            "Link": "http://people.ubuntu.com/~ubuntu-security/cve/CVE-2016-2560",
            "Metadata": {
                "NVD": {
                    "CVSSv2": {
                        "Score": 4.3,
                        "Vectors": "AV:N/AC:M/Au:N/C:N/I:P"
                    }
                }
            },
            "Name": "CVE-2016-2560",
            "NamespaceName": "ubuntu:12.04",
            "Severity": "Medium"
        }
    ]
}

[Quentin-M]

Hi,

Red Hat vulnerabilities are currently listed under the CentOS namespaces [1].
Could you try the following?

curl "http://10.18.1.130:6060/v1/namespaces/centos:7/vulnerabilities?limit=10"

[bartockbat]

That works - however, we want to be 100% certain that it is picking up the RHEL7.x vulnerabilities.
How can we be certain?
What bugs me is that if I scan a RHEL based Docker image, I never see any vulnerabilities.
I can't believe that I am creating perfect Docker images.
G

[brianwcook]

When we scan rhel images with vulnerabilities that show up in openscap,
Clair returns 'no vulnerabilities'
On Jun 15, 2016 2:25 PM, "One Whose Name We Cannot Speak" <
notifications@github.com> wrote:

That works - however, we want to be 100% certain that it is picking up the
RHEL7.x vulnerabilities.
How can we be certain?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#207 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/ADg4JycTxgaqaRBiI8_YoOtmuPrDJEDwks5qMFG4gaJpZM4I2mFO
.

[Quentin-M]

Are you sure you are analyzing all the layers of your image, with the proper parents?

You can request the last layer of your image with https://github.com/coreos/clair/tree/ad11393/api/v1#get-layersname and verify that:

• the NamespaceName is set to centos:X (the OS is properly detected),
• the ParentName is present (the layer tree is properly pushed),
• there are Features (the packages are properly detected).

[bartockbat]

Quentin - I'm simply using the compiled analyze-local-images:

./analyze-local-images 012345689abcdef

2016-06-15 15:35:42.580693 I | Saving a17f0446090e to local disk (this may take some time)
2016-06-15 15:36:50.881041 I | Retrieving image history
2016-06-15 15:36:51.601475 I | Analyzing 11 layers...
2016-06-15 15:36:51.601496 I | Analyzing c453594215e4370541ba0a2a238c9429026de1d1deedf5e5b7442778e428c60f
2016-06-15 15:36:52.901727 I | Analyzing 1c088b280181c7f5de0880130d041876ceb47cf7ae24ce490d603a1dc9cc9853
2016-06-15 15:36:52.905610 I | Analyzing 3299a732dcf62604a4b741c5fcf6a17362fb27beadf790f4085287d16fe2dd7b
2016-06-15 15:36:52.909107 I | Analyzing 2faecc7bbbf2cd8402b1d1b814f0809b45ce8f9fdeeb981182ba658ab9f6c5b4
2016-06-15 15:36:52.912239 I | Analyzing 74fe90baaa38486b6152fd278cf8baf38ab6fc5c424c832f423279424969e40b
2016-06-15 15:36:52.915544 I | Analyzing 488a0d2a48190aec4bd0d83a6b087e7936f5e8325e6dbe6b2ed3691b9ffeb480
2016-06-15 15:36:52.919531 I | Analyzing 116a70ba00a30469511a8a844ccdafeaf8ef688928095d5385d6896e3b81dd2a
2016-06-15 15:36:52.923336 I | Analyzing 14a35a6974b949e0f7ff68a1218eb8513d75329eede91b29fde140b2d75b52ab
2016-06-15 15:36:52.925569 I | Analyzing ae79ef37bb632a61dbf0764c31b5b9eb9909acce7e3d7d9df0af0216553b0bb3
2016-06-15 15:36:52.954643 I | Analyzing 4686d75d666f1713709f524beb952313f9d432702f44d7cdfe8fb17eaf3c7880
2016-06-15 15:36:52.958019 I | Analyzing a17f0446090e972c5facce31393ba70afbbe2566ec0860060aae9b12071ae2c1
2016-06-15 15:36:52.960431 I | Retrieving image's vulnerabilities
Clair report for image a17f0446090e (2016-06-15 19:36:53.90739734 +0000 UTC)
Success! No vulnerabilities were detected in your image

[Quentin-M]

Got it. If you are sure that you ran analyze-local-images after the initial update had ran (update finished), then could you please give us the result of the following command to get the raw response, telling us what Clair thinks to know about your image?

curl "http://localhost:6060/v1/layers/a17f0446090e972c5facce31393ba70afbbe2566ec0860060aae9b12071ae2c1?features&vulnerabilities"

[bartockbat]

Quentin - I have a copy of the output below.
It looks like the proper name space is there - "NamespaceName": "rhel:7.2",

Perhaps you can elaborate on what we did?
Thanks - Glen

Results.tar.gz

[brianwcook]

it looks like the namespace is detected as rhel 7.2, and that namespace contains no vulnerability metadata. Is it supposed to be detected as RHEL or Centos? Either way it doesn't detect any vulnerabilities in the layers.

[Quentin-M]

Hi,

Thanks for the report. I now understand the issue. As discussed on IRC, the confusion came from the fact that you actually analyze RHEL images and not CentOS images. As it is right now, Clair only supports CentOS images.

I see few ways it can be done:

• Modify the rhel fetcher to write vulnerabilities' namespace as rhel instead of centos (one line change) and update the redhatrelease namespace detector to tag the images as rhel instead of centos. Ideally, we could exploit support multiple namespaces in one layer #193 to tag CentOS images with both namespaces. The tricky part is making sure every production database still work as expected (and still get updated) after that change. It would require a migration of some sort.
• Modify the namespace detector to tag the RHEL images as centos (easiest way, but hacky).

Unfortunately, I am afraid I won't find the time in the coming weeks to implement this change. Hopefully it could come from an external contributor or another maintainer if anyone is interested in doing so.

Edit: Also, the detector has to detect rhel:7 instead of rhel:7.2

[brianwcook]

okay, I implemented option 2 just to get things working locally. I hardcoded the detector so anything that it finds it names centos:something and now scanning is working. I compared the new output with oscap-docker and the results match.

I am not particularly skilled at golang but I may give option 1 a try. If I do get it working I might need some assistance getting it in shape to be pulled.

[Quentin-M]

Sure! That would really really be awesome! Thanks!

[brianwcook]

@Quentin-M, can you just clarify for me - You are expecting the change the vulnerabilities so they are written as rhel namespace instead of centos, but then how will centos vulnerabilities map to the rhel namespace? Seems like you would then have the same problem in reverse plus needing to modify both the vulnerabilities table and the layers table data in the db.

What if I added another regexp to the centos detector specifically for rhel and had it tag layers as centos? Then when #193 is merged in we can add code to also tag rhel layers as rhel for reporting purposes. This avoids database migrations and gets functionality in now without waiting on #193.

-Brian

[brianwcook]

@Quentin-M could you respond to previous comment? I am blocked not sure how to proceed.

[Quentin-M]

@brianwcook My apologies for the late answer. Your suggestion sounds fine to me 👍. Looks like you already have a patch then as suggested in #207 (comment) then?

[brianwcook]

Hi Quentin, please see - #229

[jzelinskie]

Closing this due to inactivity. If it's still relevant please remake the issue. Thanks.