No Go depedency

[PXshadow]

Removes Go as a dependency!
Prebuilt binaries of the Go part of the compiler and auto commits them into the repo.
When the compiler is ran (in Haxe) checks if go4hx (Go part of the compiler) exec exists if it doesn't it tries to build it,
If for any reason it cannot be built (missing Go, issue in build process etc) it will look for the OS and ARCH of a pre built binary.

For Haxe devs with no connection to Go this allows using the compiler with no install of Go necessary! Big win for over coming the barrier to entry of trying out the compiler and working on it.

[PXshadow]

Failing because of Haxe compiler error for some reason: HaxeFoundation/haxe#12008

[PXshadow]

I've run into a problem. The compiler relies on running 2 Go cmd commands go get and go mod init. These commands allow very smart version control of version controlled repos with caching when the versions are the same.

The trouble is that the functionality to do this is hidden away in src/cmd/go/internal: https://github.com/golang/go/blob/master/src/cmd/go/internal/modget/get.go

It is not trivial to pull out, and adding the Go compiler as a dependency defeats the point of this PR.

So there are a few options left:

1. Spend time trying to get around the internal restriction and pull the code out from the aforementioned package.
2. Rewrite our own functionality that is compatible with how Go does versioning.
3. Use vendoring and don't cache depedencies between builds and instead hold the code locally.
4. Use a hybrid approach with one of the other approaches, where if Go is detected to be installed use it's system otherwise use no Go system (might be burdensome to maintain)
5. Simple git clone (with or without depth?) for all links and always point go2hx to the directory that way.

[elliott5]

I’m not sure what approach you’ve decided to take @PXshadow. The audience that wants to just try go2hx out, probably on a small trial go program, without installing the go compiler proper, would be happy for the stdgo dependencies to be vendored. But as soon as they wanted to create their own library the problem would start again. So the question is, how much time would it take to create a go get etc specialist program, and would that be time well spent?

[PXshadow]

So the question is, how much time would it take to create a go get etc specialist program, and would that be time well spent?

A day or two, yes it would be well spent. I should be able to pull the internal package and place it in the go2hx repo and then I will have access to it. I can keep the whole cmd chunk at first and then trim what is not being used.

[mikaib]

I'll just give my opinion below (please do ignore if it's bad or unnecessary)

First of all: I think that having go removed as a dependency is a great thing!
But: I think that the best way going forward would be to prevent adding any more code that depends on go to then at a later date go thru (no pun intended) and remove any dependency on go of existing code. My reasoning for this is simple:

• Adding go get to the Haxe side of things adds an extra layer of moving parts (which may or may not take longer than expected to get working "well")
• Personally I classify this as QOL, so it would be better to work on something else instead (perhaps create a milestone of all QOL related issues?)

At the end of the day it will be time well spent, but if it's worth it to work on now is debatable in my honest opinion.

[PXshadow]

@mikaib Sorry for the late reply. go get being classified as a QOL is true, but also misses in my view the necessary of something like go get needing to be inside of the go2hx compiler in one way or another. This is because go get handles the dependency problem of 3rd party code having dependencies which in turn have more dependencies, without this most 3rd party code will not work. So that leaves the option to either reinvent the wheel and parse the go.mod/go.sum files ourselves (a bad idea) or use what the Go compiler uses.

Finally removing Go as a dependency at this stage means compiling multiple binaries of all of the Go code present in the go2hx repo as a single statically linked binary. With this approach, even if more Go code is added into the compiler, it won't cause go2hx to have Go as a dependency again.

[mikaib]

Yeah that seems like a good way to go :D

[PXshadow]

I've been working on this and how to pull out the go get and package manager behavior from Go.
One thing I forgot about is the stdlib. When Go is installed it also pulls in the stdlib.

I think the best approach now might be to follow what @jeremyfa game engine ceramic does:
"Ceramic is mostly self-contained. It embeds its own versions of haxe, node & electron so that it won’t break if you update something on your computer."

We can embed a Go compiler into the project for the user, and even embed the Haxe nightly version and not have to wait for a 5.0 release.

This is a tradeoff between setup time vs reliability of tooling.
Ceramic is also using git modules, which go2hx could also take advantage of in order to pull in the stdlib from Go.

Current idea all together

1. Run on the various os and arch matrix:

go build cmd/go

2. Commit the binaries into a separate repo called go-binary.
3. add go2hx/go-binary, go2hx/tools as gitmodules.
4. On setup, build the Go part of the compiler with the correct go-binary (handled with Haxe code)
5. For now, just use the prebaked stdlib created every passing commit from go2hx.
6. When a repo is not in the stdlib and go get would be called use the one from go-binary, same with go mod init.

Still potential issues

1. If go2hx ever stops prebaking go's stdlib the go repo would need to be cloned which is quite a heavy repo.
2. Pick when to do shallow vs unshallow clones and convert between them.
3. It would be awesome to allow current Haxe users, using the stable version to use the nightly version to use HXB which is really crucial for a large stdlib like go2hx creates, when -lib go2hx is called. (Don't know how that should look like from a dev experience)
4. Using Go's package manager is less benifical if go2hx compiles the same library for every project. Should look into auto making haxelibs.

[PXshadow]

Even more fun to think about! https://go.dev/doc/manage-install

[PXshadow]

Trying a version 2 of this approach, by handling how to install Go and use the correct version. #239