Add fleetctl and Fleet UI support for remote flags

[noahtalerman]

This feature requires Orbit, the Fleet agent manager: https://fleetdm.com/announcements/introducing-orbit-your-fleet-agent-manager

If, at some point, you revoked an old enroll secret, this feature won't work for hosts that were added to Fleet using this old enroll secret. This is because Orbit uses the enroll secret to receive new flags from Fleet. For these hosts, all existing features will work as expected.

To update the flags on these hosts, we recommend deploying a new package. This will update each host's enroll secret. Here's how to deploy a new package:

1. First check which hosts need a new enroll secret by running the following query:

SELECT * FROM orbit_info WHERE enrolled = false

Note that hosts that don't have Orbit installed will return an error because the orbit_info table doesn't exist. These errors can be ignored.

2. In Fleet, head to the Hosts page and select Add hosts to find the fleetctl package command with an active enroll secret.
3. Copy and run the fleetctl package command to create a new package. Distribute this package to the hosts that returned results in step (1).
4. Done!

Instead of deploying a new package, the administrator can push the active enroll secret to the hosts returned in step (1):

• macOS: Push the enroll secret to /opt/orbit/secret.txt.
• Windows: Push the enroll secret to C:\Program Files\Orbit\secret.txt.
• Linux: Update the enroll secret set in the configuration file /etc/default/orbit.

Problem

Currently, whenever admins want to change the flags on osquery, they have to deploy the new flag to the end user via MDM or a Munki-equivalent. This results in a lot of effort and time and risk.

We want to make this deployment step unnecessary. Instead, the admin should be able to specify new osquery flags to Fleet, and the Orbit instances installed should re-start osquery with the appropriate flags.

Parent epic

• Manage flags for osquery remotely via Orbit #6851

Requirements

• The agent_options key in the config YAML supports a new command_line_flags key.
• The agent_options key in the team YAML supports a new command_line_flags key.
• The Settings > Organization settings > Agent options page supports a new command_line_flags key.
• The Settings > Teams > Team details > Agent options page supports a new command_line_flags key.
• Keys under command_line_flags are validated (keys match real keys and values are right type of data) based on the latest osquery (https://osquery.readthedocs.io/en/latest/installation/cli-flags/).

Design

When there are no configs for command_line_flags, then there is a blank command_line_flags: {} # requires Orbit setting in second position after the config setting.

When the command_line_flags config is populated:

Note that overrides is hidden in all cases.

Tasks

1

• Support a new command_line_flags top-level key in the organization settings' agent options YAML and JSON (AppConfig)
• The overrides setting does not accommodate command_line_flags, which is why it is only allowed at the top-level (and not inside config or every override)

2

• Support a new command_line_flags top-level key in the team's agent options YAML and JSON
• The overrides setting does not accommodate command_line_flags, which is why it is only allowed at the top-level (and not inside config or every override)

3

• The Settings > Organization settings > Agent options page supports a new command_line_flags key.
• The UI should show the existence of a flags object, even if blank
• When there are no configs for command_line_flags, then there is a blank command_line_flags: {} # requires Orbit setting in second position after the config setting.
• When the command_line_flags config is populated, it displays those settings
• When there is no overrides being specified by the user, do not show a blank overrides section

4

• The Settings > Teams > Team details > Agent options page supports a new command_line_flags key
• Same behaviour as for the Organization Settings (point 3)

5

• Bump react-query from 3.34.16 to 3.39.2 #7692

[noahtalerman]

@zhumo do we want to add this requirement? I think yes.

"Keys under command_line_flags are validated (keys match real keys and values are right type of data) based on the latest osquery."

Both the config and team YAML document (where command_line_flags live) will soon have this validation.

Also, it looks like we're adding command_line_flags to the UI by default. How do you want to handle the case in which folks with plain osquery use command_line_flags? (I think command_line_flags require Orbit)

I suggest adding a requirement like...

"Clearly document that command_line_flags requires Orbit."

[zhumo]

Hey! Thanks for those suggestions. Yes, I agree with all of them.

In addition, I might say, for a blank command_line_flags option, it should look like

command_line_flags: { } # requires Orbit

I will update the images and requirements.

[zhumo]

btw validation is for all of these: https://osquery.readthedocs.io/en/latest/installation/cli-flags/

[noahtalerman]

@lukeheath can you please file the child issues for this epic and move them into the "Designed" column?

[lukeheath]

@mna I'm assigning you this ticket from the roadmap to begin the spec process. Please spend some time researching the parent epic and related issues to understand goals and context.

Once you have a good understanding of the feature request, please update the specs as necessary to reflect implementation details.

Typically this would be separated into two issues (one for fleetctl+api changes, the other for the UI.) However, since the UI changes are just changing the YAML, I thought it might be something we could handle in a single issue. What do you think?

If you have any ideas, questions, or concerns, please tag the appropriate people in a comment on this issue. Once you feel the issue is ready to estimate, please move it to the "Specified" column on the roadmap board. Thanks!

[mna]

One major question: will orbit authentication/enrollment already be done before this ticket? E.g. #7109 . Will orbit already make requests and receive agent options from fleet (i.e. distributed read/write)? This part is a big undertaking and will significantly affect the estimation.

@lukeheath @noahtalerman

[mna]

One major question: will orbit authentication/enrollment already be done before this ticket? E.g. #7109 . Will orbit already make requests and receive agent options from fleet (i.e. distributed read/write)?

The answer is yes, this is part of the draft PR being worked on here: #7246 and confirmed by @zhumo here: #7109 (comment). So this ticket should not have to modify the agent at all.

[sharvilshah]

Just a heads up @noahtalerman @mna @zhumo -- both @lucasmrod and I think that command_line_flags is confusing, so I have changed it to command_line_startup_flags to make it easier to understand the intent

[noahtalerman]

@lucasmrod and I think that command_line_flags is confusing

@sharvilshah and @lucasmrod why do you think command_line_flags is confusing?

cc @zhumo

[zhumo]

@sharvilshah @lucasmrod We chose command_line_flags because it matches the title of what's in the osquery readthedocs site. https://osquery.readthedocs.io/en/latest/installation/cli-flags/. I think we should hew to being consistent here unless there's some reason it's confusing to be done this way in Fleet but not in osquery.

[noahtalerman]

@zhumo do we no longer need these frontend changes?

[zhumo]

@sharvilshah is it the case that the requirements above will "just work" based on work you've done on the agent side? Or do we still need some interface work to meet the requirements above?

[sharvilshah]

@zhumo yep, it just works, there are no interface changes required...

[noahtalerman]

@sharvilshah just double checking here.

3

• The Settings > Organization settings > Agent options page supports a new command_line_flags key.
• The UI should show the existence of a flags object, even if blank
• When there are no configs for command_line_flags, then there is a blank command_line_flags: {} # requires Orbit setting in second position after the config setting.
• When the command_line_flags config is populated, it displays those settings
• When there is no overrides being specified by the user, do not show a blank overrides section

4

• The Settings > Teams > Team details > Agent options page supports a new command_line_flags key
• Same behaviour as for the Organization Settings (point 3)

Did your work cover the UI changes in number (3) and (4) under the "Tasks" section? (from this issue's description)

[sharvilshah]

@noahtalerman, no my work does not cover the (3) and (4) UI changes --

To clarify: If you add command_line_flags and populate it to the YAML (either global/team agent options one), it will pick up those things, but there is not default When there are no configs for command_line_flags, then there is a blank command_line_flags: {} # requires Orbit setting in second position after the config setting. in the UI

[noahtalerman]

no my work does not cover the (3) and (4) UI changes --

Got it. Thanks!

@sharvilshah your changes didn't cover the validation specified by (5) right?

[sharvilshah]

@noahtalerman, no it doesn't cover (5) -- we didn't spec it out when outlined the changes to orbit and Fleet

[noahtalerman]

Got it! I'm trying to figure out what's left.

@zhumo FYI the UI changes (3 and 4 under "Tasks") and validation (5) specified in this ticket are not finished.

Currently, it's unlikely the interface team will get these changes in this release. Should we loop in an engineer on the platform team?

Also, it may make sense to only prioritize the UI changes. I think we can push the validation to the next release.

[zhumo]

@sharvilshah the config options will still work even without the UI making them appear right?

If so, @noahtalerman I think we can ship the orbit work in this release, and wait on 3,4,5.

[sharvilshah]

@sharvilshah the config options will still work even without the UI making them appear right?

@zhumo yes, exactly

[noahtalerman]

ship the orbit work in this release, and wait on 3,4,5.

Sounds good 👍

[noahtalerman]

Ah, I might have overlooked 1 and 2 in this ticket as well. That is, the ability to set flags using the config and teams YAML files and fleetctl apply

ability to set flags using the config and teams YAML files and fleetctl apply

@sharvilshah a user won't be able to do this yet, correct?

[noahtalerman]

cc @zhumo ^^

[sharvilshah]

@sharvilshah a user won't be able to do this yet, correct?

@noahtalerman they will, the way it's implemented means it is transparent and it should not matter whether Fleet UI is used or fleetctl apply with YAML files is used

Unless of course there is a bug :)

[noahtalerman]

should not matter whether Fleet UI is used or fleetctl apply with YAML files is used

Got it! @sharvilshah I marked 1 and 2 under tasks as done. Please let me know if this doesn't make sense.

cc @mna

[mna]

@lukeheath I have a PR out for the backend validation of the new flags section (#7979 ), and I will take a shot at the frontend changes in a separate PR, but note that this is not just static text changes - it requires some code to remove the overrides key from the yaml if empty, and to add the comment next to the command_line_flags if empty. Should not be crazy-hard, but it's more that I'm not familiar with many of our frontend technologies and tooling, so I'll rely heavily on code review to right the ship :)

[lukeheath]

@mna Sounds good! Thanks for handling the frontend aspect and feel free to ping @frontend with any questions :)