Appimage built on Ubuntu 18.04.6 works fine on 20.04 but not on 18.04

[akontsevich]

Errors related to GStreamer as well. We have successfully built Appimage in docker based on Ubuntu 18.04.6 LTS: #8 and it works fine on 20.04 but not on 18.04 for some strange reason.

Errors are the following:

/tmp/.mount_MyAppUSQNkG/MyApp: /usr/lib/x86_64-linux-gnu/libp11-kit.so.0: 
no version information available (required by /tmp/.mount_MyAppUSQNkG/usr/lib/libgnutls.so.30)

(gst-plugin-scanner:5568): GStreamer-WARNING **: 21:31:52.038: Failed to load plugin 
'/tmp/.mount_MyAppUSQNkG/usr/lib/gstreamer-1.0/libgstassrender.so': 
/tmp/.mount_MyAppUSQNkG/usr/lib/libass.so.9: undefined symbol: FcWeightToOpenTypeDouble

(gst-plugin-scanner:5640): GVFS-WARNING **: 21:31:53.083: Error creating proxy: 
Error calling StartServiceByName for org.gtk.vfs.Daemon: 
GDBus.Error:org.freedesktop.systemd1.UnitMasked: 
Unit gvfs-daemon.service is masked. (g-io-error-quark, 36)

(gst-plugin-scanner:5669): GStreamer-WARNING **: 21:31:53.490: 
Failed to load plugin '/tmp/.mount_MyAppUSQNkG/usr/lib/gstreamer-1.0/libgstlibav.so': 
/tmp/.mount_MyAppUSQNkG/usr/lib/libass.so.9: undefined symbol: FcWeightToOpenTypeDouble

FcWeightToOpenTypeDouble is something about fontconfig. What could cause this error? How to fix? Any ideas?

[akontsevich]

Ah, I see: bionic has libfontconfig1 version 2.12.6 and ffmpeg PPA has 2.13.1 and FcWeightToOpenTypeDouble() function was introduced since 2.12.92 version

Think need to put latest libfontconfig.so.1 from the PPA to the bundle, will try...

[akontsevich]

Ah, I see: bionic has libfontconfig1 version 2.12.6 and ffmpeg PPA has 2.13.1 and FcWeightToOpenTypeDouble() function was introduced since 2.12.92 version

Think need to put latest libfontconfig.so.1 from the PPA to the bundle, will try...

No, does not work: seems requires all fontconfig packages like fontconfig-config. How to put them in, @TheAssassin? In appimage-builder it is easy to do - just mention package name and it will be included. What to do here?

[TheAssassin]

https://github.com/AppImage/pkg2appimage/blob/master/excludelist#L102

fontconfig is on the excludelist and is not supposed to be bundled, it always causes issues. You need to build on a newer release of Ubuntu or an alternative distro if you need a newer version. The PPA apparently does not just provide ffmpeg packages built against the stuff 18.04 provides, it also ships newer dependencies...

[akontsevich]

https://github.com/AppImage/pkg2appimage/blob/master/excludelist#L102

fontconfig is on the excludelist and is not supposed to be bundled, it always causes issues. You need to build on a newer release of Ubuntu or an alternative distro if you need a newer version. The PPA apparently does not just provide ffmpeg packages built against the stuff 18.04 provides, it also ships newer dependencies...

Vicious circle: to make Appimage work need to upgrade gstreamer, which brings new dependencies, which requires newest distro or downgrade gstreamer. Any other solutions?!

[TheAssassin]

Well, your initial claim is wrong. You said you were building on some Ubuntu 18.04. But in fact you have modified that distro significantly with packages from some PPA that update an excludelisted dependency. You are not building on Ubuntu 18.04, really.

There are PPAs like Stephan Binner's Qt PPAs that are built against the distro-provided packages only and thus can be deployed properly. These don't provide any backported dependencies. However, that also means that when Qt does not support building on a distro any more, it won't be supported by the PPA either.

Targeting 20.04 or newer is an acceptable trade-off. Debian stable, which was released in 2019, might also be a suitable target, depending on your requirements.

By the way, this is not really a plugin issue. The excludelist is standard to AppImage creation.

Please stop making comparisons with this appimage-builder project. It violates some of the original principles of AppImage which linuxdeploy still respect. Whataboutism à la "it works with appimage-builder" is not helpful and therefore not welcome.

[akontsevich]

Well, your initial claim is wrong. You said you were building on some Ubuntu 18.04. But in fact you have modified that distro significantly with packages from some PPA that update an excludelisted dependency. You are not building on Ubuntu 18.04, really.

No, above is wrong! It is not the full story and not my initial claim. Initial problem was this error: #8 (comment) with libx264.so.152 and x264-10bit/libx264.so.152 loading on Ubuntu 18.04. And I was able to solve it only by installing the mentioned PPA and upgrade to the latest gstreamer, bad thing it upgrades not only gstreamer packages.

There are PPAs like Stephan Binner's Qt PPAs that are built against the distro-provided packages only and thus can be deployed properly. These don't provide any backported dependencies. However, that also means that when Qt does not support building on a distro any more, it won't be supported by the PPA either.

I do not need Qt here, we build it on our own - version 5.15.2 as only this version works with QGroundControl now. Does this PPA contains gstreamer as well? What is its URL?

Targeting 20.04 or newer is an acceptable trade-off. Debian stable, which was released in 2019, might also be a suitable target, depending on your requirements.

Our requirement was to support 18.04 for now, may be we'll switch to 20.04.

By the way, this is not really a plugin issue. The excludelist is standard to AppImage creation.

Yes, for this particular problem - it is not a plugin issue, but the whole chains of problems I'm trying to solve in #8 and #9 - yes it is! And this chains of problems actually is the one - make video and gstreamer work from within Appimge, for now we forced it to work only for 20.04.

Please stop making comparisons with this appimage-builder project. It violates some of the original principles of AppImage which linuxdeploy still respect. Whataboutism à la "it works with appimage-builder" is not helpful and therefore not welcome.

I'm not comparing, just trying to say that saying that principle to put whole package fully preserving its directory structure is a very convenient way, and limitation to put all the libraries inside single /usr/lib does not always work and has very limited application especially when need to put other files than libraries to a bundle. And asking if linuxdeploy allows something similar?

[TheAssassin]

The initial comment states:

We have successfully built Appimage in docker based on Ubuntu 18.04.6 LTS

Only the second comment mentions PPA:

#9 (comment)

#8 mentions some PPA only in the last few comments.

This is really exhausting. You keep providing breadcrumbs only, add more and more issues that are unrelated to the initial problem, don't provide a (minimal) example project that can be used to reproduce the issue but instead expect/demand from people who have already noted they don't have lots of time to set up their own environment using instructions from various different pages setting up some project they have never seen before, ....
You (continue to?) use -l in a way it is not designed for (which was noted in #8 (comment)).
You do not provide any build artifacts (AppImages) that could be tested in a VM either, because your project is apparently proprietary.

I do not think I will be able to help without investing at least an entire day into just setting up the environment. And that is for sure way too much time I am able to or even want to spend on this alpha-grade plugin.

If you want to have your project be debugged by me, please provide a repository (as requested in #8 (comment)) that contains the build scripts that result in an AppImage that shows the problems you mention. Please also make sure these issues show up in a vanilla Ubuntu 18.04.6 VM. That will allow me to figure out what you do, where the issues are, and whether there are things that can be fixed in either the plugin or linuxdeploy itself.

What you are doing there looks relatively experimental (all the -l of random libraries, drivers, ...). If any of this makes sense, it will need to be integrated in this plugin. Then, it can also be maintained.

[akontsevich]

I provided tons of info about the problems and my actions I did trying to solve. I do not know what can I say more here. We can screenshare so I may help You setting up build env to reproduce.

If you want to have your project be debugged by me, please provide a repository (as requested in #8 (comment)) that contains the build scripts that result in an AppImage that shows the problems you mention.

I provided it in the last comment: #8 (comment) as You requested: https://github.com/airmap/qgroundcontrol/pull/13/files - did You looked into it?

[TheAssassin]

You linked to a PR, not a repository. Do you want me to go to the repository and branch that PR is based on and try that? Why not point to that branch directly?

Are there build artifacts publicly available? As it is a PR, I suppose the job has not uploaded files to GitHub yet?

[akontsevich]

You linked to a PR, not a repository. Do you want me to go to the repository and branch that PR is based on and try that? Why not point to that branch directly?

Why not? Copy paste branch name from the repo. I thought PR with explanation and diff is much more convenient for understanding. Branch: https://github.com/airmap/qgroundcontrol/tree/linuxdeploy_appimage

Are there build artifacts publicly available?

What do you mean? It is is available.

[TheAssassin]

Then please provide a link. I spent another five minutes on searching for it myself, because this kind of back and forth asking you for such information is quite annoying, honestly.

Building myself is not trivial, because that GitHub actions workflow uses lots of third-party actions to pull in dependencies. GitHub does not provide a CLI tool (unlike e.g., Drone) that can be used to run such workflows offline.

I'd have to fork your repository and run everything myself. This is the kind of annoyance I mean: you don't make it really easy for the person on the other end, they either have to ask you many questions until they are at a state they can work with, or invest time to do lots of setup work oneself.

Off topic: this is the reason I always avoid such dependencies, and also build CI stuff in a way that is entirely reproducible locally. See for instance https://github.com/linuxdeploy/linuxdeploy/blob/master/.github/workflows/main.yml (all commands are reproducible in a Docker container if needed). Another example is AppImageLauncher, where all builds are run in Docker containers using code contained in the repository (https://github.com/TheAssassin/AppImageLauncher/tree/master/ci) so that all builds can be run on virtually any CI system as well as on any workstation that has Docker installed (the corresponding workflow is therefore relatively trivial as well, https://github.com/TheAssassin/AppImageLauncher/blob/master/.github/workflows/main.yml).

[akontsevich]

Then please provide a link.

What link?! I provided all the links: to the repo, PR, branch, changed files - what else?

Off topic: this is the reason I always avoid such dependencies, and also build CI stuff in a way that is entirely reproducible locally.

Sure, problem is QGC CI and our CI are very different. I do not know how they build.

[TheAssassin]

Are there build artifacts publicly available?

resulted in

What do you mean? It is is available.

Let me recap. I asked for build artifacts. You said there were some "available" (whatever that means). Where are the build artifacts?

Is this a language barrier problem? Do you know the term "build artifact"? In the context of software engineering and CI, build artifacts are files produced during a CI run that can be downloaded. So where are they?

[akontsevich]

Initially You asked me for a repo and branch with appimage build script with my modifications:here they are:
https://github.com/airmap/qgroundcontrol/pull/13/files#diff-8fa72783f6ef987f20d1d6d1c46ffda80b242449a87611337945ea3077d0fee3

No, I do not have build artifacts for QGroundControl as we do not use their CI, so I can only provide the build script above.

[akontsevich]

Just checked: QGroundControl does not work with Gstreamer 1.14 on Ubuntu 18.04 at all. Version 1.16 (Ubuntu 20.04) or latest 1.18.5 (like QGroundControl mentiones) from ffmpeg4 PPA work fine.

So I think we'll follow Your suggestion and Upgrade to 20.04 as well as QGroundControl based on it now. Sorry to trouble You. We may close #8 and #9 tickets now.