`Added to Fleet` set to `Never` when enrolling manually on MDM after installing fleetd first #20059

lucasmrod · 2024-06-27T17:08:02Z

Fleet version: latest (bug introduced in fleet-v4.51.0)

💥 Actual behavior

Installing fleetd on macOS and then enrolling to MDM manually causes the host's Added to Fleet in the host details to be Never.

Expected behavior

The Added to Fleet should not be reset (set to Never) when enrolling the host to Fleet MDM manually.

🧑‍💻 Steps to reproduce

Install fleetd on macOS
Install MDM profile via My Device (aka manual MDM enroll)

QA notes

Before applying 4.54.0 reproduce issue as described above for a macOS device, let's call it device X.
Before applying 4.54.0 add a good macOS device (without the issue described here), let's call this one device Y.
Apply 4.54.0, which will run a migration that will fix the issue for the macOS device X and will leave device Y untouched.
Test enrolling DEP device works (macOS, iOS, iPadOS)

check the Added to Fleet date is accurate.

Test enrolling macOS manually works (first install fleetd, then enroll via MDM via My device)

check the Added to Fleet date is accurate.

The text was updated successfully, but these errors were encountered:

JoStableford · 2024-06-28T13:30:15Z

Related to a Slack conversation

noahtalerman · 2024-06-28T13:47:56Z

Hey @lucasmrod and @sharon-fdm!

I'm not sure why this ended up on the drafting board...is the expected behavior unclear? Something else?

The "Added to Fleet" timestamp is when the host enrolls to Fleet. When a host shows up in Fleet for the first time. More context here: https://fleetdm.com/handbook/company/why-this-way#why-does-fleet-use-mdm-on-off-instead-of-mdm-enrolled-unenrolled

lucasmrod · 2024-06-28T13:49:23Z

Sorry, I incorrectly added the :product label. This is a bug. Let us know how big of a priority is fixing this (vs other bugs/features).

sharon-fdm · 2024-06-28T14:10:55Z

Thanks @lucasmrod.
As always, let's keep on our board and estimate soon.

sharon-fdm · 2024-06-28T15:37:46Z

Hey team! Please add your planning poker estimate with Zenhub @getvictor @jacobshandling @lucasmrod @mostlikelee @RachelElysia

zayhanlon · 2024-07-01T14:00:04Z

@sharon-fdm adding a p2 label here because the 'workaround' option of deleting the host to get this updated with the correct date does not work for customer-preston. they're unable to delete the hosts. their entire enrollment status and workflow is based on this 'enrolled date' field

jacobshandling · 2024-07-01T18:39:06Z

@zayhanlon got it. Sharon is out today, but we saw the P2 label and have prioritized accordingly.

sharon-fdm · 2024-07-02T13:47:53Z

Thanks @zayhanlon.
I bumped it the top of our bug list and will get this looked at asap.

zayhanlon · 2024-07-02T13:52:11Z

Thank you all!

lucasmrod · 2024-07-02T20:54:36Z

adding a p2 label here because the 'workaround' option of deleting the host to get this updated with the correct date does not work for customer-preston. they're unable to delete the hosts. their entire enrollment status and workflow is based on this 'enrolled date' field

@zayhanlon @noahtalerman
I have a PR in review that solves the issue moving forward (for new devices added to Fleet).
Let's discuss what we should do with hosts that already are in this state (Added to Fleet = Never). What enroll date do we set on them?

zayhanlon · 2024-07-03T00:53:52Z

@lucasmrod guessing that there's no way for us to figure out the actual enroll date?

lucasmrod · 2024-07-03T16:26:34Z

I'll be taking a look at where we can get the enrolled date from. AFAICS it won't be exact, most likely an approximation (deduced from other MySQL table row timestamps).

lucasmrod · 2024-07-03T19:17:04Z

@zayhanlon @noahtalerman

Added to Fleet is the date the osquery agent enrolled (different than the MDM enroll time or the time that the host was ingested via DEP).

I did some digging and we can use the host_disks.created_at which most of the cases is close to the hosts.last_enrolled_at. In some cases it's many days apart (e.g. if you uninstall+install fleetd again on a device, then last_enrolled_at will be updated and will be different than host_disks.created_at)

Here's a dump from dogdood (46 hosts have 0 day difference, and 14 have a non-0 difference):

select datediff(hd.created_at, h.last_enrolled_at) as days_between_last_enrolled_at_and_host_disks from host_disks hd join hosts h on hd.host_id=h.id join host_mdm hmdm on hmdm.host_id=h.id where h.platform = 'darwin' and h.last_enrolled_at !;
+----------------------------------------------+
| days_between_last_enrolled_at_and_host_disks |
+----------------------------------------------+
|                                           -8 |
|                                            0 |
|                                            0 |
|                                          -92 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                           -2 |
|                                         -231 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                         -147 |
|                                         -130 |
|                                          -91 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                           -6 |
|                                            0 |
|                                            0 |
|                                          -70 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                          -52 |
|                                          -96 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                          -12 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                           -9 |
|                                          -10 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
|                                            0 |
+----------------------------------------------+
60 rows in set (0.20 sec)

Let me know if we are a-ok going this route. (Via a migration in 4.54.0 to fix these "Never" hosts using host_disks.created_at)

zayhanlon · 2024-07-03T19:23:24Z

Seems like it makes sense but I'll wait for @noahtalerman

zayhanlon · 2024-07-08T14:00:26Z

@lucasmrod @noahtalerman is there NO other way for us to capture the actual enrollment date vs this option^ ? The diff on some of these is really high

lucasmrod · 2024-07-08T21:41:35Z

is there NO other way for us to capture the actual enrollment date vs this option^ ? The diff on some of these is really high

I ran out of ideas. Let me check with the team.

valentinpezon-primo · 2024-07-09T13:17:25Z

@lucasmrod - thanks for looking into this !

Do we know why the date is "Never" even tho the agent is installed ? 🤔

We have some internal flow that triggers if the enrollment date change, in our minde this date should never changed, and if it is, it means the device was re-enrolled

if the host_disks.created_at fix is going to apply only for device that has "Never" date, I think it's acceptable

noahtalerman · 2024-07-09T14:41:53Z

Hey @lucasmrod thanks for digging into this!

Installing fleetd on macOS and then enrolling to MDM manually causes the host's Added to Fleet in the host details to be Never.

Looking at the bug description, my understanding is that customer-preston is doing the opposite. Their workflow:

Turn on MDM manually (installed enrollment profile)
Install fleetd

@valentinpezon-primo please correct me if I'm wrong.

Do we know why the date is "Never" even tho the agent is installed ?

I had the same question...

Don't we have a "created at" timestamp in the hosts table in the Fleet DB that gets created when fleetd is installed and host enrolls? Maybe this the bug?

Meaning, assuming I'm right about customer-preston's workflow, we're not populating that "created at" timestamp for hosts that turn on MDM manually and then have fleetd installed.

I did some digging and we can use the host_disks.created_at which most of the cases is close to the hosts.last_enrolled_at. In some cases it's many days apart (e.g. if you uninstall+install fleetd again on a device, then last_enrolled_at will be updated and will be different than host_disks.created_at)

@lucasmrod, are there other cases that we know of in which the dates are far apart?

If I'm understanding correctly, customer-prestons hosts that have "Never" haven't had fleetd uninstalled and then re-installed yet.

@valentinpezon-primo is that right?

If that's right, and assuming we don't have a "created at" timestamp for hosts that turn on MDM manually and then have fleetd installed, it sounds like host_disks.created_at will be accurate.

valentinpezon-primo · 2024-07-09T16:16:09Z

Yes @noahtalerman , we do this (profile then fleetd is installed by fleet)

But this behavior is new (the "never" date), was properly populated before !

lucasmrod · 2024-07-09T16:32:39Z

Hi folks!

Do we know why the date is "Never" even tho the agent is installed ?

The bug we introduced in v4.51.0 caused MDM enrolls/re-enrolls to set the Added to Fleet field to Never. (We are fixing this in the next release.)

Looking at the bug description, my understanding is that customer-preston is doing the opposite. Their workflow:
Turn on MDM manually (installed enrollment profile)
Install fleetd

Gotcha. AFAICS the Never would happen in these scenarios:
A. fleetd installed first, manual MDM enroll after.
B. MDM re-enroll (e.g. if the user unenrolls from MDM manually by uninstalling the enrollment profile and then re-enrolling to MDM by installing the enrollment profile again)
C. During DEP migration flow (where fleetd is installed first then MDM turned on in Fleet).

I'm guessing for customer-preston these "Never" happen because of either (B) or (C)?

lucasmrod · 2024-07-09T16:34:46Z

Don't we have a "created at" timestamp in the hosts table in the Fleet DB that gets created when fleetd is installed and host enrolls? Maybe this the bug?

created_at timestamp is set when the host is enrolled through DEP (so it's different than last_enrolled_at, which is set when fleetd enrolls via orbit/osquery)

lucasmrod · 2024-07-09T16:35:20Z

if the host_disks.created_at fix is going to apply only for device that has "Never" date, I think it's acceptable

Yes, that would be the plan.

…0173) #20059 - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [X] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [X] Added/updated tests - [X] Manual QA for all new/changed functionality

lucasmrod · 2024-07-11T16:16:41Z

@xpkoala @PezHub I've added QA notes to the issue.

PezHub · 2024-07-11T17:06:35Z

Thanks @lucasmrod and good timing. I literally encountered this yesterday after enrolling my device via DEP, removing the profile then manually re-enrolling. I'll be sure to test the other scenarios listed above as well.

PezHub · 2024-07-11T20:55:17Z

I had to test scenario B for another mdm ticket today and confirm this fixes the issue (screenshot I posted above)

fleet-release · 2024-07-17T22:58:08Z

Enrolling manually,
Fleet's date now accurate,
Calm as cloud city's view.

lucasmrod added bug Something isn't working as documented #g-mdm MDM product group :incoming New issue in triage process. :product Product Design department (shows up on 🦢 Drafting board) #g-endpoint-ops Endpoint ops product group and removed #g-mdm MDM product group labels Jun 27, 2024

sharon-fdm assigned noahtalerman Jun 27, 2024

zayhanlon added the customer-preston label Jun 28, 2024

lucasmrod added :release Ready to write code. Scheduled in a release. See "Making changes" in handbook. and removed :product Product Design department (shows up on 🦢 Drafting board) labels Jun 28, 2024

noahtalerman assigned sharon-fdm and unassigned noahtalerman Jun 28, 2024

lukeheath added the ~released bug This bug was found in a stable release. label Jun 28, 2024

zayhanlon added the P2 Prioritize as urgent label Jul 1, 2024

jacobshandling assigned lucasmrod and unassigned sharon-fdm Jul 1, 2024

jacobshandling removed the :incoming New issue in triage process. label Jul 1, 2024

lucasmrod mentioned this issue Jul 2, 2024

Fix last_enrolled_at for macOS devices when re-enrolling to MDM #20173

Merged

8 tasks

lukeheath added this to the 4.54.0-tentative milestone Jul 3, 2024

sharon-fdm added the :ai Request AI PR review label Jul 9, 2024

lukeheath modified the milestones: 4.54.0, 4.55.0-tentative Jul 9, 2024

sharon-fdm modified the milestones: 4.55.0-tentative, 4.54.0 Jul 10, 2024

PezHub mentioned this issue Jul 11, 2024

Improve backend determination of mdm.connected_to_fleet to account for cases where Fleet doesn't receive the MDM "Checkout" message for Apple devices #20057

Closed

lukeheath closed this as completed Jul 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`Added to Fleet` set to `Never` when enrolling manually on MDM after installing fleetd first #20059

`Added to Fleet` set to `Never` when enrolling manually on MDM after installing fleetd first #20059

lucasmrod commented Jun 27, 2024 •

edited

Loading

JoStableford commented Jun 28, 2024

noahtalerman commented Jun 28, 2024

lucasmrod commented Jun 28, 2024

sharon-fdm commented Jun 28, 2024

sharon-fdm commented Jun 28, 2024

zayhanlon commented Jul 1, 2024 •

edited

Loading

jacobshandling commented Jul 1, 2024

sharon-fdm commented Jul 2, 2024

zayhanlon commented Jul 2, 2024

lucasmrod commented Jul 2, 2024

zayhanlon commented Jul 3, 2024

lucasmrod commented Jul 3, 2024

lucasmrod commented Jul 3, 2024

zayhanlon commented Jul 3, 2024

zayhanlon commented Jul 8, 2024

lucasmrod commented Jul 8, 2024

valentinpezon-primo commented Jul 9, 2024

noahtalerman commented Jul 9, 2024

valentinpezon-primo commented Jul 9, 2024

lucasmrod commented Jul 9, 2024

lucasmrod commented Jul 9, 2024

lucasmrod commented Jul 9, 2024

lucasmrod commented Jul 11, 2024

PezHub commented Jul 11, 2024

PezHub commented Jul 11, 2024

fleet-release commented Jul 17, 2024

Added to Fleet set to Never when enrolling manually on MDM after installing fleetd first #20059

Added to Fleet set to Never when enrolling manually on MDM after installing fleetd first #20059

Comments

lucasmrod commented Jun 27, 2024 • edited Loading

💥 Actual behavior

Expected behavior

🧑‍💻 Steps to reproduce

QA notes

JoStableford commented Jun 28, 2024

noahtalerman commented Jun 28, 2024

lucasmrod commented Jun 28, 2024

sharon-fdm commented Jun 28, 2024

sharon-fdm commented Jun 28, 2024

zayhanlon commented Jul 1, 2024 • edited Loading

jacobshandling commented Jul 1, 2024

sharon-fdm commented Jul 2, 2024

zayhanlon commented Jul 2, 2024

lucasmrod commented Jul 2, 2024

zayhanlon commented Jul 3, 2024

lucasmrod commented Jul 3, 2024

lucasmrod commented Jul 3, 2024

zayhanlon commented Jul 3, 2024

zayhanlon commented Jul 8, 2024

lucasmrod commented Jul 8, 2024

valentinpezon-primo commented Jul 9, 2024

noahtalerman commented Jul 9, 2024

valentinpezon-primo commented Jul 9, 2024

lucasmrod commented Jul 9, 2024

lucasmrod commented Jul 9, 2024

lucasmrod commented Jul 9, 2024

lucasmrod commented Jul 11, 2024

PezHub commented Jul 11, 2024

PezHub commented Jul 11, 2024

fleet-release commented Jul 17, 2024

`Added to Fleet` set to `Never` when enrolling manually on MDM after installing fleetd first #20059

`Added to Fleet` set to `Never` when enrolling manually on MDM after installing fleetd first #20059

lucasmrod commented Jun 27, 2024 •

edited

Loading

zayhanlon commented Jul 1, 2024 •

edited

Loading