LE end-entity returned as non-trusted

[jvehent]

When testing invoicer.securing-devops.com, the end-entity shows as not trusted by any browser:

--- Trust ---
Mozilla Microsoft Apple Android
   ✘        ✘       ✘      ✘
--- Trust paths ---
CN=invoicer.securing-devops.com (id=186748317)
├──C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3 (id=1707992)
│  └──O=Digital Signature Trust Co., CN=DST Root CA X3 (id=38)
└──C=US, O=Let's Encrypt, CN=Let's Encrypt Authority X3 (id=1818345)
   └──C=US, O=Internet Security Research Group, CN=ISRG Root X1 (id=842653)

This is obviously incorrect. The DST root is trusted by all browsers, and the ISRG one is trusted by Mozilla.

I think this might be an issue with how the trust table is managed. First off, I'm seeing 25 current trust entries for the DST path when there should only be one:

observatory=> select count(*), cert_id, issuer_id from trust where cert_id=1707992 and is_current=true group by cert_id, issuer_id;
 count | cert_id | issuer_id 
-------+---------+-----------
    25 | 1707992 |        38

Secondly, two of those (339310112 and 336867947) show the intermediate was not trusted by mozilla, microsoft, apple and android, which is obviously wrong.

observatory=> select id, timestamp, cert_id, issuer_id, trusted_ubuntu::text || trusted_mozilla::text || trusted_microsoft::text || trusted_apple::text || trusted_android::text from trust where cert_id=1707992 and is_current=true;
    id     |         timestamp          | cert_id | issuer_id |         ?column?         
-----------+----------------------------+---------+-----------+--------------------------
 339890605 | 2018-05-27 12:57:15.867882 | 1707992 |        38 | truetruetruetruefalse
 339869968 | 2018-05-20 02:25:48.384477 | 1707992 |        38 | truetruetruetruefalse
 339791931 | 2018-05-19 01:07:53.002878 | 1707992 |        38 | truetruetruetruefalse
 339784021 | 2018-05-18 22:34:43.0978   | 1707992 |        38 | truetruetruetruefalse
 339784020 | 2018-05-18 22:34:43.09475  | 1707992 |        38 | truetruetruetruefalse
 339777083 | 2018-05-18 20:24:35.891523 | 1707992 |        38 | truetruetruetruefalse
 339741532 | 2018-05-18 09:40:12.393152 | 1707992 |        38 | truetruetruetruefalse
 339672928 | 2018-05-17 12:28:55.767134 | 1707992 |        38 | truetruetruetruefalse
 339561278 | 2018-05-15 13:48:48.963577 | 1707992 |        38 | truetruetruetruefalse
 339310112 | 2018-04-16 19:55:49.106908 | 1707992 |        38 | truefalsefalsefalsefalse
 338346257 | 2018-03-29 07:14:26.674185 | 1707992 |        38 | truetruetruetruefalse
 338294550 | 2018-03-29 07:14:22.50969  | 1707992 |        38 | truetruetruetruefalse
 338269854 | 2018-03-29 07:14:22.037713 | 1707992 |        38 | truetruetruetruefalse
 338224344 | 2018-03-29 07:14:21.747098 | 1707992 |        38 | truetruetruetruefalse
 338224343 | 2018-03-29 07:14:21.587228 | 1707992 |        38 | truetruetruetruefalse
 338082379 | 2018-03-29 07:14:20.617758 | 1707992 |        38 | truetruetruetruefalse
 337991227 | 2018-03-22 16:31:17.562493 | 1707992 |        38 | truetruetruetruefalse
 337977179 | 2018-03-22 12:22:27.711672 | 1707992 |        38 | truetruetruetruefalse
  32329540 | 2018-03-10 21:20:29.000613 | 1707992 |        38 | truetruetruetruefalse
   3197855 | 2018-03-10 21:19:57.54188  | 1707992 |        38 | truetruetruetruefalse
   5130776 | 2018-03-10 21:18:47.68008  | 1707992 |        38 | truetruetruetruefalse
   3197879 | 2018-03-10 21:18:36.653985 | 1707992 |        38 | truetruetruetruefalse
   3213296 | 2018-03-10 21:17:17.2512   | 1707992 |        38 | truetruetruetruefalse
 336867947 | 2018-03-10 21:16:45.220115 | 1707992 |        38 | truefalsefalsefalsefalse
 336866686 | 2018-03-10 21:01:02.921893 | 1707992 |        38 | truetruetruetruefalse

So I think we have a couple bugs in trust handling.