[Arch] Backintime fails to build #1333

afettouhi · 2022-10-13T10:17:51Z

Hi, I have an up to date Arch install and I am trying install the git version of backintime via the AUR using yay. But the install fails during unittesting for reason. Here is the output when it fails

OK
python3 -m unittest -b test/test_backintime.py
E..

ERROR: test_diagnostics_arg (test.test_backintime.TestBackInTime)

Traceback (most recent call last):
File "/home/af/.cache/yay/backintime-git/src/backintime/common/test/test_backintime.py", line 181, in test_diagnostics_arg
diagnostics = json.loads(output)
File "/usr/lib/python3.10/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Ran 3 tests in 3.040s

aryoda · 2022-10-14T17:39:03Z

Thanks for reporting this, I am already aware of this bug in another issue (no real duplicate):
See #1233 (comment)

The current work-around to install from AUR is described here:
#921 (comment)
It would be great if you report back here if it works or something else is missing...

aryoda · 2022-10-16T00:58:14Z

Fix submitted via PR #1335 (also fixes the unit test and updates the AUR installation section in README incl. a link for a work-around to this issue)

afettouhi · 2022-10-22T09:31:40Z

Still fails to build for me on Arch. Though the error is different now

Traceback (most recent call last):
  File "/home/af/.cache/yay/backintime-git/src/backintime/common/backintime.py", line 1190, in <module>
    startApp()
  File "/home/af/.cache/yay/backintime-git/src/backintime/common/backintime.py", line 507, in startApp
    args = argParse(None)
  File "/home/af/.cache/yay/backintime-git/src/backintime/common/backintime.py", line 568, in argParse
    args, unknownArgs = mainParser.parse_known_args(args)
  File "/usr/lib/python3.10/argparse.py", line 1859, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib/python3.10/argparse.py", line 2068, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/usr/lib/python3.10/argparse.py", line 2008, in consume_optional
    take_action(action, args, option_string)
  File "/usr/lib/python3.10/argparse.py", line 1936, in take_action
    action(self, namespace, argument_values, option_string)
  File "/home/af/.cache/yay/backintime-git/src/backintime/common/backintime.py", line 742, in __call__
    diagnostics = collect_diagnostics()
  File "/home/af/.cache/yay/backintime-git/src/backintime/common/diagnostics.py", line 80, in collect_diagnostics
    result['host-setup']['locale'] = ', '.join(locale.getlocale())
TypeError: sequence item 0: expected str instance, NoneType found
E.s
======================================================================
ERROR: test_diagnostics_arg (test.test_backintime.TestBackInTime)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/af/.cache/yay/backintime-git/src/backintime/common/test/test_backintime.py", line 191, in test_diagnostics_arg
    output = subprocess.check_output(["./backintime", "--diagnostics"])
  File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['./backintime', '--diagnostics']' returned non-zero exit status 1.

----------------------------------------------------------------------
Ran 3 tests in 2.966s

FAILED (errors=1, skipped=1)
make: *** [Makefile:570: unittest] Error 1
make: Leaving directory '/home/af/.cache/yay/backintime-git/src/backintime/common'```

emtiu · 2022-10-22T09:47:11Z

Thanks for reporting, I'm re-opening the ticket and assigning to @aryoda :)

aryoda · 2022-10-22T12:35:20Z

@afettouhi I did not test it with yay but differently. Could you please describe the exact steps to reproduce this (mainly theyay call with all arguments)?

afettouhi · 2022-10-22T17:36:06Z

I am just trying to build is this way with yay

yay -S backintime-cli-git backintime-git

aryoda · 2022-10-22T23:45:18Z

The error tracebacks are exactly reproducible!

afettouhi · 2022-10-23T04:09:25Z

So, is this an issue with yay then?

aryoda · 2022-10-23T15:46:45Z

So, is this an issue with yay then?

No it isn't at all. I could reproduce the issue by cloning BiT from the master branch on a fresh Arch installation.

I am preparing a fix for this and will explain the details then here

buhtz · 2022-10-23T18:38:38Z

Do I see it correct that locale.getlocale() return None on an Arch system?

Can you show us the output of locale on your shell please? May look like this

LANG=de_DE.UTF-8
LANGUAGE=
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=

Just a quick shoot. Modify that line

result['host-setup']['locale'] = ', '.join(locale.getlocale())

into

result['host-setup']['locale'] \
    = ', '.join(locale.getlocale() or ['(not determinable)'])

afettouhi · 2022-10-23T20:21:41Z

[af@andre ~]$ locale
LANG=da_DK.UTF-8
LC_CTYPE="da_DK.UTF-8"
LC_NUMERIC="da_DK.UTF-8"
LC_TIME="da_DK.UTF-8"
LC_COLLATE="da_DK.UTF-8"
LC_MONETARY="da_DK.UTF-8"
LC_MESSAGES="da_DK.UTF-8"
LC_PAPER="da_DK.UTF-8"
LC_NAME="da_DK.UTF-8"
LC_ADDRESS="da_DK.UTF-8"
LC_TELEPHONE="da_DK.UTF-8"
LC_MEASUREMENT="da_DK.UTF-8"
LC_IDENTIFICATION="da_DK.UTF-8"
LC_ALL=

aryoda · 2022-10-23T20:47:14Z

Sorry to revise my first statement that I can reproduce the exact traceback. I have different ones in nearly the same locations.

I am still working on a fix for those but the originally reported traceback I cannot reproduce:

File "/home/af/.cache/yay/backintime-git/src/backintime/common/diagnostics.py", line 80, in collect_diagnostics
result['host-setup']['locale'] = ', '.join(locale.getlocale())
TypeError: sequence item 0: expected str instance, NoneType found

@buhtzz Are you working on that (just to be sure we don't do the same work)? I think the diagnostics code requires some more try-catches for such edge cases...

The currently known reasons to me for the other failing unit tests are:

Missing dependencies on fresh install in PKGBUILD:
sudo pacman -Sy python-packaging (required by BiT) + sudo pacman -Sy python-pyfakefs (required for a unit test)
-> I would like to contact all maintainers of ARCH BiT packages since this problem looks universal
Error in implementation of our new feature --diagnostics:
RTE when using new rsync version that supports -VV to write version information as JSON
because BiT tries a string trim on a dict (created from JSON) in that case instead of trimming a version string
-> fix already prepared but not yet published
Unit test bug that I myself have introduced by changing the log output (write all log to stderr, incl. INFO and WARNING)
Write all log output to stderr (do not pollute stdout) #1337
One unit test now fails in case of a first-time installation because the BiT serviceHelper D-Bus daemon
is not yet installed and BiT throws a different sequence of warnings than the unit test is ignoring
(the regexp ignores any output at the beginning and end of the log output, but not within the expected output)

These warnings cause the unit test test_local_snapshot_is_successful in test_backintime.py to fail:

WARNING: Failed to connect to Udev serviceHelper daemon via D-Bus: org.freedesktop.DBus.Error.ServiceUnknown
WARNING: D-Bus message: The name net.launchpad.backintime.serviceHelper was not provided by any .service files

-> I will prepare a fix the next days to make the expected output regexp more tolerant

Edit: Inserted 4 as result of #1333 (comment)

locale.getlocale() returns (None, None) if the locale is invalid (eg. tweaked for make test).
Use a sensible default then instead of an RTE in:

File "/home/af/.cache/yay/backintime-git/src/backintime/common/diagnostics.py", line 80,
in collect_diagnostics result['host-setup']['locale'] = ', '.join(locale.getlocale())`

The reason why the BiT installation from AUR succeeded in my tests a few days ago are clear now to me:

The old rsync version was still installed where the version number is still a string (not a JSON via -VV).
I should have updated all my Arch packages first...
New unit test bugs were introduced after my successful test via the "log to stderr" PR which was merged later
Edit: I have tested with another AUR package that did not manipulate the locale settings (that causes a unit test to fail).
To reproduce it on any distro just use: env LC_ALL=C make test (from within the common folder)

buhtz · 2022-10-23T21:37:17Z

@buhtzz Are you working on that (just to be sure we don't do the same work)?

No I don't.

I think the diagnostics code requires some more try-catches for such edge cases...

Maybe. But even the getlocal() should return something. If the return value is None then something is very wrong with the underlying system.
We can't catch every possible error.

Missing dependencies on fresh install in PKGBUILD:
...
-> I would like to contact all maintainers of ARCH BiT packages since this problem looks universal

Why there are multiple maintainers on one distro?

We need to clearly separate the responsibilities and resources between upstream and distros.

buhtz · 2022-10-24T08:36:00Z

[af@andre ~]$ locale LANG=da_DK.UTF-8

Strange. There is a locale but while PKGBUILD there is no locale.

Can you show me the output of python3 -c "import locale;print(locale.getlocale())" please?

aryoda · 2022-10-24T08:49:10Z

Strange. There is a locale but while PKGBUILD there is no locale.

PKGBUILD may possibly run as sudo or even root so @afettouhi could you please also check the locale as sudo (or whatever privileges yay is started...

afettouhi · 2022-10-24T08:50:36Z

[af@andre ~]$ sudo locale
[sudo] adgangskode for af:
LANG=da_DK.UTF-8
LC_CTYPE="da_DK.UTF-8"
LC_NUMERIC="da_DK.UTF-8"
LC_TIME="da_DK.UTF-8"
LC_COLLATE="da_DK.UTF-8"
LC_MONETARY="da_DK.UTF-8"
LC_MESSAGES="da_DK.UTF-8"
LC_PAPER="da_DK.UTF-8"
LC_NAME="da_DK.UTF-8"
LC_ADDRESS="da_DK.UTF-8"
LC_TELEPHONE="da_DK.UTF-8"
LC_MEASUREMENT="da_DK.UTF-8"
LC_IDENTIFICATION="da_DK.UTF-8"
LC_ALL=

aryoda · 2022-10-24T08:59:51Z

And what does python3 -c "import locale; print(locale.getlocale())" say? I'd expect None to be consistent with the unit test failure...

afettouhi · 2022-10-24T09:02:35Z

[af@andre ~]$ python3 -c "import locale; print(locale.getlocale())"
('da_DK', 'UTF-8')

aryoda · 2022-10-24T09:06:58Z

And

python3 -c "import locale; print(locale.getlocale(locale.LC_MESSAGES))"
python3 -c "import locale; print(locale.getlocale(locale.LC_CTYPE))"

afettouhi · 2022-10-24T09:07:53Z

[af@andre ~]$ python3 -c "import locale; print(locale.getlocale(locale.LC_MESSAGES))" (None, None)

[af@andre ~]$ python3 -c "import locale; print(locale.getlocale(locale.LC_CTYPE))" ('da_DK', 'UTF-8')

buhtz · 2022-10-24T09:32:46Z

Thanks to that thread in the arch forum
https://bbs.archlinux.org/viewtopic.php?pid=2063725#p2063725

We knew that the PKGBUILD "script" does set LC_ALL=C explicit before calling make.
Now I understand. And it is OK for me. It isn't a "bug" in the PKGBUILD script.

aryoda · 2022-10-24T11:15:19Z

We knew that the PKGBUILD "script" does set LC_ALL=C explicit before calling make test

Hero :-) Now I see it clear too, it was all just in front of our (PKGBUILD) eyes:

> python3 -c "import locale; print(locale.getlocale())"
('en_US', 'UTF-8')
> env LC_ALL=C python3 -c "import locale; print(locale.getlocale())"
(None, None)

And the reason I didn't see a failing unit test was because I have tested my other fix with a different package
that does not set/change the locale:

backintime: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=backintime
vs backintime-git: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=backintime-git

I will add a fix for this issue too in my upcoming PR.

@afettouhi Thanks for your stickiness and cooperation to hunt this bug 🥇

Good team work :-)

afettouhi · 2022-10-24T11:17:15Z

Glad I could help.

aryoda · 2022-11-04T16:57:14Z

I have fixed all problems (I hope) and contacted the AUR package maintainer to indicate open issues at the packaging side.

@afettouhi Once the PR is merged into BiT you could try to install it again and it should work now (or I have work again ;-)

afettouhi · 2022-11-04T17:51:56Z

Looking forward to the update.

* getlocale() returns (None, None) causing join() to throw this RTE: TypeError: sequence item 0: expected str instance, NoneType found

… to #1333) * Bug was introduced with #1337 (write all log output to stderr)

willemw12 · 2022-11-06T06:39:52Z

All tests in https://aur.archlinux.org/pkgbase/backintime-git are passing now.

emtiu added Bug Testing labels Oct 13, 2022

aryoda mentioned this issue Oct 16, 2022

Fix bug: BiT does not start (DBus exception NameHasNoOwner) #1335

Merged

emtiu closed this as completed in ad4e91c Oct 21, 2022

emtiu reopened this Oct 22, 2022

emtiu assigned aryoda Oct 22, 2022

emtiu added Distro-Specific only for certain distributions, desktop environments or display servers Reproduced labels Oct 23, 2022

emtiu changed the title ~~Backintime fails to build from in Arch~~ [Arch] Backintime fails to build Oct 23, 2022

aryoda mentioned this issue Nov 4, 2022

Fix #1333: arch Linux build fails #1353

Merged

aryoda closed this as completed in #1353 Nov 5, 2022

aryoda added a commit that referenced this issue Nov 5, 2022

Fix bug (related to #1333): RTE in --diagnostics with LC_ALL=C

43f6e18

* getlocale() returns (None, None) causing join() to throw this RTE: TypeError: sequence item 0: expected str instance, NoneType found

aryoda added a commit that referenced this issue Nov 5, 2022

Fix failing unit test on first-time installation from source (related…

b95a8e0

… to #1333) * Bug was introduced with #1337 (write all log output to stderr)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Arch] Backintime fails to build #1333

[Arch] Backintime fails to build #1333

afettouhi commented Oct 13, 2022

aryoda commented Oct 14, 2022

aryoda commented Oct 16, 2022 •

edited

Loading

afettouhi commented Oct 22, 2022 •

edited by emtiu

Loading

emtiu commented Oct 22, 2022

aryoda commented Oct 22, 2022

afettouhi commented Oct 22, 2022 •

edited

Loading

aryoda commented Oct 22, 2022

afettouhi commented Oct 23, 2022

aryoda commented Oct 23, 2022 •

edited

Loading

buhtz commented Oct 23, 2022 •

edited

Loading

afettouhi commented Oct 23, 2022

aryoda commented Oct 23, 2022 •

edited

Loading

buhtz commented Oct 23, 2022

buhtz commented Oct 24, 2022 •

edited

Loading

aryoda commented Oct 24, 2022

afettouhi commented Oct 24, 2022

aryoda commented Oct 24, 2022

afettouhi commented Oct 24, 2022

aryoda commented Oct 24, 2022

afettouhi commented Oct 24, 2022 •

edited

Loading

buhtz commented Oct 24, 2022

aryoda commented Oct 24, 2022 •

edited

Loading

afettouhi commented Oct 24, 2022

aryoda commented Nov 4, 2022

afettouhi commented Nov 4, 2022

willemw12 commented Nov 6, 2022

[Arch] Backintime fails to build #1333

[Arch] Backintime fails to build #1333

Comments

afettouhi commented Oct 13, 2022

OK python3 -m unittest -b test/test_backintime.py E..

ERROR: test_diagnostics_arg (test.test_backintime.TestBackInTime)

aryoda commented Oct 14, 2022

aryoda commented Oct 16, 2022 • edited Loading

afettouhi commented Oct 22, 2022 • edited by emtiu Loading

emtiu commented Oct 22, 2022

aryoda commented Oct 22, 2022

afettouhi commented Oct 22, 2022 • edited Loading

aryoda commented Oct 22, 2022

afettouhi commented Oct 23, 2022

aryoda commented Oct 23, 2022 • edited Loading

buhtz commented Oct 23, 2022 • edited Loading

afettouhi commented Oct 23, 2022

aryoda commented Oct 23, 2022 • edited Loading

buhtz commented Oct 23, 2022

buhtz commented Oct 24, 2022 • edited Loading

aryoda commented Oct 24, 2022

afettouhi commented Oct 24, 2022

aryoda commented Oct 24, 2022

afettouhi commented Oct 24, 2022

aryoda commented Oct 24, 2022

afettouhi commented Oct 24, 2022 • edited Loading

buhtz commented Oct 24, 2022

aryoda commented Oct 24, 2022 • edited Loading

afettouhi commented Oct 24, 2022

aryoda commented Nov 4, 2022

afettouhi commented Nov 4, 2022

willemw12 commented Nov 6, 2022

OK
python3 -m unittest -b test/test_backintime.py
E..

aryoda commented Oct 16, 2022 •

edited

Loading

afettouhi commented Oct 22, 2022 •

edited by emtiu

Loading

afettouhi commented Oct 22, 2022 •

edited

Loading

aryoda commented Oct 23, 2022 •

edited

Loading

buhtz commented Oct 23, 2022 •

edited

Loading

aryoda commented Oct 23, 2022 •

edited

Loading

buhtz commented Oct 24, 2022 •

edited

Loading

afettouhi commented Oct 24, 2022 •

edited

Loading

aryoda commented Oct 24, 2022 •

edited

Loading