Investigate Appimage

[pierreporte]

For a really long time, installation of software under Linux was possible by using distribution-specific package manager or by compiling the whole programme. The package managers work really well but if the desired programme is not present or is too old, the compilation was the only way and it may be really long and not easy for someone not really into computers. For several months now, two thing came out to simplify the installation of software under Linux: Flatpak (formerly known as XDG-Apps) for all distributions and Snap for Ubuntu only. Flatpak and Snap packages are distributed like Windows installers and provide sandboxed applications. For now, according to the Flatpak website, it is available in Archlinux, Debian, Fedora, Mageia and Ubuntu.

I think it would be good to provide Flatpak packages for Otter. Flatpak being available to Ubuntu, Snap could be avoided. Users could install each weekly or beta version without having to compile or wait that the package maintainers do it. With Flatpak only, virtually all distributions providing Flatpak can be targeted and it could even be official packages.

The process of building a Flatpak package seems quite easy and fast so the additional work should be minimal after the first time.

Personally, even if I compile Otter directly from the git repository (from the AUR package in Archlinux), I will use the Flatpak package if it is made available.

[Emdek]

@pierreporte, well, since it will take distributions some time to adopt QtWebKit NG it could make sense to prepare such packages to not let users wait.

[annulen]

If all you need is standalone binaries + GUI installer, flatpack is insane overkill. I propose to use more lightweight solutions like QtIFW (see CPackIFW), AppImage, or something in this vein

[pierreporte]

@annulen I don’t get the “overkillness” of Flatpak by reading the documentation of both Flatpak and AppImage. Could you do a fast little summary of the pro and cons of Flatpak, Snappy, Appimage and QtIFW?

Besides this, we also need to consider which packaging system is the most used. It would be a pity that the user would have to install a whole packaging system for one program.

[annulen]

There is no need to install anything to use Qt IFW. You download installer and execute it. That's all.
I was using IntallJammer in the past, there was no Qt IFW those days. But it's probably dead by now.

I don’t get the “overkillness” of Flatpak

It runs application in a chroot with almost complete OS, which is distrributed within the package, while right approach is to pack just Qt and a few other libraries, and use libc, X11 and all other stuff fronm the host system.

[probonopd]

right approach is to pack just Qt and a few other libraries, and use libc, X11 and all other stuff fronm the host system

Which you can do with AppImage @annulen

[pierreporte]

Can AppImage packages be integrated to the system like any other application? That is, Otter will automatically appear in the right menus and could be set as the default browser? Also, if someone installs Otter with the package manager provided by the distribution and with the AppImage package, what happens?

[probonopd]

An AppImage is just a self-mounting ISO that contains an application and everything it needs to run which cannot be assumed to be part of the base system. Hence, the application inside the AppImage can copy a .desktop file into the system to register itself with the system (e.g., to show itself in the menus). There is a desktopintegration helper script which can be used for this purpose.

If someone installs Otter with the package manager provided by the distribution and with the AppImage package, then both are available and can be used.

See appimage.org for information; a large number of example AppImages (including some browsers) are available for testing on https://bintray.com/probono/AppImages.

[pierreporte]

It’s great then. We could then have classic packages for stable releases and AppImage packages for weekly and beta without any interference.

[probonopd]

Yes, for updating e.g., from weekly to weekly, binary delta updates could be used (so that users only download the bytes that have changed, making updates quick and efficient).

[pierreporte]

According to Jesse Smith’s post Opinion on DistroWatch, AppImage clearly seems to be the better packaging system.

[ahoneybun]

I managed to get a snap working in --devmode, it does not have flash working at the moment. Working on those issues now.

[pierreporte]

I was willing to propose a Chocolatery package for Windows, since Firefox and Chrome are already available, and it has very similar features to the package managers on Linux (installation, un-installation, update, dependencies). However, someone already did one but it is clearly outdated. In fact, it is made by @adgellida who clearly states that his Chocolatery packages are unmaintained. Should we consider to ask him to give us control on the package?

Creating Chocolatey Packages

[mariokamenjak]

Doesn't Otter have auto update on Windows?

[Emdek]

@mariokamenjak, we have support for auto update, but currently it is used only as update notification.
I'm considering moving Windows installer to Qt IFW, since it would allow to split main package into modules and do only partial updates (usually only main exe gets changed anyway), using 7Z for archives, plus would allow to install other modules and could be used as update tool.
I guess that it would be best to create semi offline installer that bundles common libs, QtWebKit backend and selected translations, allowing to fetch more locales and other modules (like QtWebEngine backend or development files) during installation.

[annulen]

In this case it makes sense to provide Qt IFW packages for all platforms

[mariokamenjak]

Well, I am glad that a snap package is in the pipeline though.

[probonopd]

Is there something I could help you with regarding the AppImage?

[Emdek]

@probonopd, I'm going to give it a try for this beta (which means this week, in worst case after beta itself), this way Linux users could make use of revived QtWebKit which is here even bigger update than under Windows. ;-)

[pierreporte]

Great ! I recently tried the AppImage file for OpenShot 2.1 (the new stable version after 3 years of work!) and it didn’t work. Can’t wait to test the Otter package.

[Emdek]

While beta got delayed after all I'm still planning to try to create Appimage for one of upcoming weeklies.

[probonopd]

Generated an experimental AppImage based on the trusty ppa, please try:
https://bintray.com/probono/AppImages/Otter/_latestVersion#files
I did not test extensively, so it is expected that some fine-tuning might still be needed.

Here is the recipe that was used to produce it:
https://github.com/probonopd/AppImages/blob/master/recipes/otter/Recipe

[pierreporte]

@probonopd I tried your AppImage and it doesn’t work. It asks me if I want do create a .desktop file, but right after it quits saying that libwrap.so.0 is missing.

/home/thomas/.cache/thumbnails/normal/6c7214fca28c029ee7a412cce876ceea.png is missing. Probably not running ./bin//otter-browser.wrapper from within an AppImage.
Hence falling back to using .DirIcon
/tmp/.mount_O8mH5U/usr/bin/otter-browser: error while loading shared libraries: libwrap.so.0: cannot open shared object file: No such file or directory

I use an up to date Archlinux where Otter runs usually fine.

[Emdek]

@Frenzie, so, does it actually work for you? ;-)

[probonopd]

setting up CentOS6 to be able to build revived QtWebKit was a pain

Maybe it would be easier to do using debian oldstable, Ubuntu trusty or something along these lines?

[Emdek]

@probonopd, maybe, but that CentOS instance is finally ready to use. ;-)
I may consider choosing something different if I'll need to produce new one.

[annulen]

You don't need OS instance, you need provisioning script (or puppet manifest, or whatever else you are familiar with)

[annulen]

BTW, wheezy does not provide gstreamer 1.0. You should really consider building with jhbuild, it requires only very basic system dependencies

[Frenzie]

Sorry, I haven't actually tried it yet. Spending time with family. :)

…

On Dec 27, 2016 9:11 AM, "Michał" ***@***.***> wrote: @Frenzie <https://github.com/Frenzie>, so, does it actually work for you? ;-) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1148 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMYBQYiNuosQ65quGEbO5t1cck8MZzGks5rMMgTgaJpZM4Izi4K> .

[Emdek]

@annulen, maybe later, so far I've invested too much time into current solution. ;-)

@probonopd, is there some way to force linuxdeployqt to deploy GStreamer plugins?

[c-cesar]

Testing in Manjaro full updated in the unstable branch:

This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".
Available platform plugins are: xcb.
Reinstalling the application may fix this problem.

As it can be fixed appending -platform xcbto launch script, I tried to manually mount the appimage as iso, but failed. Maybe @probonopd can help.

[probonopd]

Posting this from http://otter-browser.org/tmp/otter-browser-0.9.12.dev-x86_64.AppImage on ubuntu-16.04-desktop-amd64.iso. Did not have to add -platform xcb. Running well for me! (One thing that drives me nuts immediately is that ctrl-leftarrow does not jump the cursor one word back, but goes to the previous page in the browser history...)

@c-cesar: Do you see any error messages when you launch it in the command line?

[probonopd]

@c-cesar this is a squashfs-based type 2 AppImage. To mount:

./otter-browser-0.9.12.dev-x86_64.AppImage --appimage-offset
# 112664
sudo mount ./otter-browser-0.9.12.dev-x86_64.AppImage /mnt -o loop,offset=112664
cp -r /mnt otter-browser-0.9.12.dev-x86_64.AppDir 
# Now you can edit it there

There is no launch script however; but there is a qt.conf. This specifies that plugins should be loaded from the plugins/ directory. In plugins/platforms there is `libxcb.so` (as it should). I don't know why it apparently is not picked up by your system from there, or what the `-platform xcb`switch changes, as it should load the xcb platform anyway. Is it trying to load another platform plugin on your system by default?

[pierreporte]

@probonopd I just tested your AppImage. It refused to start because of missing “language dictionaries”:

No language dictionaries for the language: "fr"
./otter-browser-0.9.12.dev-x86_64.AppImage: symbol lookup error: ./otter-browser-0.9.12.dev-x86_64.AppImage: undefined symbol: _ZN12QWebSettings17pluginSearchPathsEv

I am running an up-to-date Archlinux.

[Emdek]

@probonopd, is there any generator for launch scripts?

@pierreporte, it seems that for some reason system QtWebKit got picked up.

I've managed to create version with bundled GStreamer, not that video playback might be choppy (I've heard that another branch contains updated player):
http://otter-browser.org/tmp/otter-browser-0.9.12.dev156-x86_64.AppImage

[probonopd]

@Emdek, linuxdeployqt does not use launch scripts. But if you want to use one, you should be able to replace the AppDir symlink inside the AppDir with a bash script, and then run the AppDir through appimagetool manually. I assume you want to use this to set some LD_PRELOAD environment variable? In this case, you can make use of $APPDIR which will point to the mount point at which the AppImage is mounted at runtime.

[Emdek]

@probonopd, I'm more interested in that dialog shown by your experimental AppImage linked earlier, is this that one?
https://github.com/probonopd/AppImageKit/blob/master/desktopintegration
I could use it also to set GStreamer search path for plugins, currently set by application itself.
Can it be used to prevent issues with picking up system libraries instead of bundled?

[probonopd]

Well, I use the desktopintegration script for my non-linuxdeployqt-generated AppImages, but I think you could make it work with the linuxdeployqt-generated AppDir as well. To see how I use it, it's best to download a simple AppImage and look at AppRun, *.desktop, usr/bin/*.wrapper:

wget -c "https://bintray.com/probono/AppImages/download_file?file_path=Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage"
chmod a+x ./download_file\?file_path\=Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage 
./download_file\?file_path\=Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage --appimage-extract
find ./squashfs-root/
file squashfs-root/AppRun # This AppRun file parses the Exec= line of the *.desktop file next to it
cat squashfs-root/leafpad.desktop | grep Exec=  # Look at this
cat ./squashfs-root/usr/bin/leafpad.wrapper # This is the (renamed) desktopintegration script you mentioned

Instead of the compiled AppRun file it is also possible to use a script. I have seen AppImages where (a modified version of) the desktopintegration script was used as AppRun. I think the electron-builder generated AppImages work this way.

---

As for linuxdeployqt, it uses patchelf to make it load libraries from inside the AppImage rather than from the base system, if available. Turns out this is still a bit buggy (PRs welcome!).

[c-cesar]

@probonopd there is no launch script, but there is a Exec field in otter-browser.desktop where i can put the flag. I wanted to append it because the error message listed the platform plugin. But the cause is the ambiguity of the message. It could not find or load. So, I run the ldd in xcbplugin and find my problem: libselinux.so.1 => not found.

SELinux is not in Arch or Manjaro repos, but is installable. After I installed libselinux I can start Otter.
The new WebKkit is loaded and reach 347 vs 342 points of older webkit in https://html5test.com. I don't have the time to test more extensively now, but I should say that SELinux is something that all distros related to RedHat have, but is fairly rare in others. As the host was Cent OS the QPA plugin was linked against SELinux, but I don't even have this in my default repositories. So, It's a good idea to include his lib in the recipe. Their dependencies are fairly common, minus libpcre, that should be included too.

ldd /usr/lib/libselinux.so.1 
        linux-vdso.so.1 (0x00007ffdc3bf2000)
        libpcre.so.1 => /usr/lib/libpcre.so.1 (0x00007fe677807000)  <-- I don't have this either in repositories 
        libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fe677603000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007fe677265000)
        /usr/lib64/ld-linux-x86-64.so.2 (0x000055d3c00ac000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fe677048000)

[probonopd]

Agree, would be best to compile without SELinux.
AppImageCommunity/pkg2appimage#83

[Emdek]

@probonopd, thanks for the tip, I'll experiment with that script later.
I've used patchelf already to "fix" GStreamer plugins and their dependencies.
BTW, what about some option to set path to extra libs to be deployed?
Some option to set libs to exclude could be also useful (like useless GStreamer 0.10 libs in my case).

Custom Qt build might be a problem...
It seems that krita simply includes libselinux.so.1 but I don't see libpcre.so.1 in their archive.

@c-cesar, try that newer build, it includes GStreamer 1.10 so it gets more points for multimedia support.

[Emdek]

Here is another attempt, linked against GStreamer 1.10 but without bundling it (bundled can lead to no audio, most likely thanks to Pulse Audio) and including both libselinux.so.1 and libpcre.so.1:
http://otter-browser.org/tmp/otter-browser-0.9.12.dev156.2-x86_64.AppImage

[c-cesar]

The new AppImage runs, but started a blank profile, instead of using my usual, like the otehrs.
html5test failed with this message: "Test has failed: undefined is not an object (evaluating 'MediaSource.isTypeSupported')"
All three videos from this page works. http://www.quirksmode.org/html5/tests/video.html
Youtube html5 test reports all formats are supported, but MSE is not.

[Emdek]

@c-cesar, fresh profile? That is weird.
You can check which folder is used using Diagnostic Report in Help menu.

That test is borked, it assumes that MSE is available, to get results you need to enable it in about:config (QtWebKitBackend/EnableMediaSource) but I suggest to disable it after running test, since it might cause issues with video playback on YT.

[c-cesar]

I launched it from a terminal session logged with another user. my bad.
But I'm hit for another bug. I cant write diacritics (character stress marks) in VK.com text boxes. Not related with app image, I think. But good to you know.

[Emdek]

@c-cesar, sounds like #1205.

Anyway, this one is done, I'll later experiment with that extra desktop integration script, but main goal was reached, there is first official AppImage on sf.net. ;-)

[probonopd]

@Emdek just out of curiosity, why not on https://github.com/OtterBrowser/otter-browser/releases?

[Emdek]

@probonopd, while sf.net has own set of issues it is still superior to github when it comes to distributing binaries (SFTP access, statistics etc.).
Plus I still remember times when github offered this feature in slightly different form and then simply removed it...