75$ - Haxe - Fix HMM git bullshit

[ninjamuffin99]

Introductory Information:

hmm is a small helper for haxelib that allows you to specify, install, and update project dependencies using lib.haxe.org libraries, git, mercurial, or dev (local path reference) libraries.

This exists because haxelib does not yet support specifying git, mercurial, or other non haxelib-based project dependencies in the haxelib.json, .hxml, etc. files.

We use this in Friday Night Funkin' to allow for easier swapping of branches that may potentially require slightly different lib versions, as well as the CI type bullshit where it can easily install things with a script, and it downloads all the exact correct versions.

hmm has a feature where you can get haxelib bullshit from git hmm git [library] and then hmm lock that version. That will spit out a git ref into the hmm.json file. hmm also has a feature where you can call hmm reinstall and it will run through everything in hmm.json, and if your currently installed versions don't match the hmm.json versions, it will reinstall it. This is especially helpful when switching branches, where you can often have slightly different versions of things.

However, that shit is busted when you install from a specific git ref. When you attempt to reinstall / update a git based hmm.json listing, it won't properly update to whatever ref you have set in the hmm.json. You need to force reinstall using hmm reinstall -f, which would either redownload every haxelib, unless you specify the library hmm reinstall -f flixel hscript.

The Project:

Make it so when you call hmm reinstall it properly goes through and checks the git refs, and reinstalls if the git refs of the currently downloaded haxelib don't match the git commit refs of the hmm.json file.

Prerequisites:

hmm is written in Haxe, so you'll need to know your way around it and the weirdo command aspects of it. Also you knowing your way around GIT COMMAND bullshit as well as haxelib knowhow. Seems like hmm is pretty straight forward in it's code, so if you're generally decent at Haxe, I don't see it being too trivial!

How Much Can ninjamuffin Help?

• I am decent with Haxe, and know my way around how hmm works (on a user side), as we use it for FNF pretty damn extensively. If you are new to hmm, give me a shout, but also... hmm is pretty simple to use!

Budget/Timeline:

My dipshit estimate is that this can probably be done in a short afternoon or so. I just am too lazy lol! I really mean it when I say it seems straightforward! But also I am naive. Currently have 75$ USD allocated, but can prob toss you bit more if it takes longer than an hour or two.

No rush or timeline on this though, it's a pretty quality of life feature!

[steviegt6]

Having trouble reproducing this issue myself, installing mithril @ master (the example given on hmm's repo for convenience sake) and running hmm lock works as expected.

Modifying the hmm.json file to use a different ref (this time the commit right before the latest on the repo at the time of writing) and then running hmm reinstall updates as expected:

I haven't been able to reproduce issues with mismatched git refs.

[ninjamuffin99]

let me retry some things, might be with specific repos, might even be OS specific (I use windows, I believe the issue also happens on Mac as well, which I will test too) and it looks like you're using Linux of some sort!

[ninjamuffin99]

Ahh, actually I figured it out, what I had before was actually a misinterpretation of the issue we were having.

Changing the commit DID work. It was actually changing the REMOTE that should be weird / work unintended.

I'll update the main post in a sec, but ACTUALLY the issue was:

say we are on branch audio-fix of our GAME and that branch is using flixel from https://github.com/haxeflixel/flixel on the dev branch. Then we swap to visuals-fix branch of our GAME, which is using flixel from https://github.com/mastereric/flixel on the dev branch. Even if they might be totally different, I believe that hmm doesn't properly tell the difference between the two. Will get back with more findings too hehe.

[steviegt6]

let me retry some things, might be with specific repos, might even be OS specific (I use windows, I believe the issue also happens on Mac as well, which I will test too) and it looks like you're using Linux of some sort!

On Windows, I just prefer Git Bash, so here's to hoping that we won't encounter issues based on preferred software lol

Ahh, actually I figured it out, what I had before was actually a misinterpretation of the issue we were having.

Changing the commit DID work. It was actually changing the REMOTE that should be weird / work unintended.

I'll update the main post in a sec, but ACTUALLY the issue was:

say we are on branch audio-fix of our GAME and that branch is using flixel from https://github.com/haxeflixel/flixel on the dev branch. Then we swap to visuals-fix branch of our GAME, which is using flixel from https://github.com/mastereric/flixel on the dev branch. Even if they might be totally different, I believe that hmm doesn't properly tell the difference between the two. Will get back with more findings too hehe.

Gotcha. I'll play around with that as well to see if I can get it to work (break).

[steviegt6]

Alright, I see the issue in hmm's implementation, they don't check the remote at all. I'll confirm this behavior myself and then probably implement a quick fix.

EDIT: On second thought, that isn't an exact solution. Need to figure out why it can't realize it's out-of-date based on the commit alone.

[steviegt6]

hmm seems to actually have this covered since git rev-parse will fail accordingly.

[steviegt6]

Looking at the responsible code, I can't work out a situation where this code would fail and completely ignore updating a repo. Do your CI runs have any logs to go off of or anything? Could also try making a minimal reproducible repo or archive to test with as well.
hmm's handler there will reinstall if rev-parse fails to find the ref or if HEAD and the current ref don't match (should practically cover differences between remotes¹). If rev-parse fails to find the revision for the ref and/or HEAD then it'll reinstall as well.

I could try making a quick patch anyway that checks the remote, but I cannot reproduce this issue myself. It may help regardless, though.

Footnotes

1. Alright so technically, if you're working with remote b that forks remote a but your ref exists on remote a and remote b, hmm won't care. This is fine, though, since you'll still end up with the same repository for the most part. ↩

[ninjamuffin99]

Yeah I think I have a small repo, I'll toss it up / make it public in a sec, eating dinner right now teehee

[steviegt6]

Yeah I think I have a small repo, I'll toss it up / make it public in a sec, eating dinner right now teehee

Sounds good, just ate as well lol

[steviegt6]

Alright well I think I have something working on a fork, haxelib git hmm https://github.com/steviegt6/hmm tomat/git-remote-fix --global. Lemme know if this solves the issue

[steviegt6]

Yeah everything is in working order. Had a mini heart attack when smth seemingly broke but then I realized it was because I swapped remotes without swapping refs so it tried to resolve something that didn't exist lol

All good here 👌

Also realizing I never bothered to explicitly say I'm claiming this bounty, so, uh, I am claiming this bounty.

[EliteMasterEric]

Does it work if you switch from haxeflixel/flixel dev to mastereric/flixel dev, assuming that the later has a different head commit?

[steviegt6]

Does it work if you switch from haxeflixel/flixel dev to mastereric/flixel dev, assuming that the later has a different head commit?

Without locking, yes.

[ninjamuffin99]

figuring out closer to what it is

if you change the file name in the hmm.json file directly, it doesn't seem to register it

in between compiles, i changed the flixel url in hmm.json to be mastereric/flixel, and it registers it as "already installed"

in fact, I can change it to anything, even busted gits that dont exist

[steviegt6]

figuring out closer to what it is

if you change the file name in the hmm.json file directly, it doesn't seem to register it

in between compiles, i changed the flixel url in hmm.json to be mastereric/flixel, and it registers it as "already installed"

in fact, I can change it to anything, even busted gits that dont exist

Could you try with my fork (haxelib git hmm https://github.com/steviegt6/hmm tomat/git-remote-fix --global)? I made it so changes to the URL force it to reinstall, so it should resolve this (even if there may potentially be a more elegant way).

Also I think I've worked out the issue behind this - since you don't change the ref, it doesn't trigger an update. As I established earlier, hmm was completely ignoring the URL value when checking for updates. My fork resolves this, so it would work as expected now. Since upstream hmm doesn't check for changes in the URL, the ref value not changing basically indicates that nothing has changed to it. Updates would only be triggered if the branch doesn't exist or the commit doesn't exist (which there may be some cases of, but that isn't super common when dealing with forks).

[ninjamuffin99]

Nice seems to work and be fixed! I tossed you a friend request on discord to discuss payment shenanigans, if not, toss me an email (linked on my github)

I will mark this as CLOSED!!! Good shit!

[ninjamuffin99]

also maybe you can toss a PR to the hmm main library see how that goes heheh