Watcher does not consider build hash when determining if upgrade was rolled back

[AndersonQ]

With the introduction of the tracking of UPG_ROLLBACK state the watcher compares the new version of the agent with the previous one. However when comparing it only uses the major.minor.path[-SNAPSHOT], ignoring the build hash.

If the agent is upgraded from an SNAPSHOT version to another the watcher considers they as being the same version and therefore it marks the upgrade as UPG_ROLLBACK and neither logs why it changed the upgrade status nor adds any information to the new state.
This leads to an successful upgrade, a healthy agent but marked as UPG_ROLLBACK as shown below. Without any log or information explaining why the agent is on an UPG_ROLLBACK state.

The culprit seems to be here:

elastic-agent/internal/pkg/agent/application/upgrade/marker_watcher.go

Lines 117 to 123 in d58ac9c

	// If the marker exists but the version of Agent we're running right
	// now is the same as the prevVersion recorded in the marker, it means
	// the upgrade was rolled back. Ideally, this UPG_ROLLBACK state would've
	// been recorded in the marker's upgrade details field but, in case it
	// isn't for some reason, we fallback to explicitly setting that state as
	// part of the upgrade details in the marker.
	if marker.PrevVersion == currentVersion {

The UpgradeMarker has both hashes, it just need to be used when comparing the versions

elastic-agent/internal/pkg/agent/application/upgrade/step_mark.go

Lines 27 to 43 in d58ac9c

	type UpdateMarker struct {
	// Hash agent is updated to
	Hash string `json:"hash" yaml:"hash"`
	//UpdatedOn marks a date when update happened
	UpdatedOn time.Time `json:"updated_on" yaml:"updated_on"`
	// PrevVersion is a version agent is updated from
	PrevVersion string `json:"prev_version" yaml:"prev_version"`
	// PrevHash is a hash agent is updated from
	PrevHash string `json:"prev_hash" yaml:"prev_hash"`
	// Acked is a flag marking whether or not action was acked
	Acked bool `json:"acked" yaml:"acked"`
	Action *fleetapi.ActionUpgrade `json:"action" yaml:"action"`
	Details *details.Details `json:"details,omitempty" yaml:"details,omitempty"`
	}

For confirmed bugs, please report:

• Version: main
• Operating System: all
• Discuss Forum URL: N/A
• Steps to Reproduce: Upgrade from a snapshot build to another from the same version.

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[pchila]

Reopening as it needs fixes from PRs #3827 #3837