[bug] Conan doesn't resolve reference to latest binary package when old sqlite3 is installed #14945

BobIsOnFire · 2023-10-16T11:48:48Z

Environment details

Operating System+version: Oracle Linux 6.10 (glibc 2.12)
Compiler+version: gcc 8.2
Conan version: 2.0.13
Python version: 3.6

Steps to reproduce

Trying to build & install boost from source on old system (glibc 2.12), b2/4.10.1 resolves to a binary package on CCI, which is built against a newer glibc:

$ conan install --requires=boost/1.82.0 --build=missing
======== Computing necessary packages ========
Requirements
    boost/1.82.0#7c9cc60d65d6942a56d4806ab85bda7c:eb34317ab81722ab55d97692af0d62754e8851dd - Build
    bzip2/1.0.8#411fc05e80d47a89045edc1ee6f23c1d:75807e7575ecc0f6141344f7c4f5e86106b92ec2 - Build
    libbacktrace/cci.20210118#ec1aa63bbc10145c6a299e68e711670c:897e6fde718e72637d92b0b8ed929893a1976238 - Build
    zlib/1.3#06023034579559bb64357db3a53f88a4:897e6fde718e72637d92b0b8ed929893a1976238 - Build
Build requirements
    b2/4.10.1#8dc3df1cc73ad65d86cbdfd31fdb011f:63fead0844576fc02943e16909f08fcdddd6f44b#a31a98f757dcf4d3f03ed629ccde26b7 - Download (conancenter)
...
-------- Downloading 1 package --------
b2/4.10.1: Retrieving package 63fead0844576fc02943e16909f08fcdddd6f44b from remote 'conancenter'
b2/4.10.1: Package installed 63fead0844576fc02943e16909f08fcdddd6f44b
b2/4.10.1: Downloaded package revision a31a98f757dcf4d3f03ed629ccde26b7
...
-------- Installing package boost/1.82.0 (5 of 5) --------
boost/1.82.0: Building from source
...
boost/1.82.0: RUN: b2 -q numa=on target-os=linux architecture=x86 [...]
b2: /lib64/libc.so.6: version `GLIBC_2.17' not found (required by b2)
b2: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by b2)

Easy to solve -- build b2 from source:

$ conan install --requires=boost/1.82.0 --build=missing --build='b2/*'
======== Computing necessary packages ========
b2/4.10.1: Forced build from source
Requirements
    boost/1.82.0#7c9cc60d65d6942a56d4806ab85bda7c:eb34317ab81722ab55d97692af0d62754e8851dd - Build
    bzip2/1.0.8#411fc05e80d47a89045edc1ee6f23c1d:75807e7575ecc0f6141344f7c4f5e86106b92ec2#d7a79e2c40e526c412f2db99564eadfd - Cache
    libbacktrace/cci.20210118#ec1aa63bbc10145c6a299e68e711670c:897e6fde718e72637d92b0b8ed929893a1976238#a7e3c00cc7faaa29f0b441cc0abab184 - Cache
    zlib/1.3#06023034579559bb64357db3a53f88a4:897e6fde718e72637d92b0b8ed929893a1976238#2e82fa1c9727666c9d482aa0d2d8abb4 - Cache
Build requirements
    b2/4.10.1#8dc3df1cc73ad65d86cbdfd31fdb011f:63fead0844576fc02943e16909f08fcdddd6f44b - Build
...
-------- Installing package b2/4.10.1 (1 of 5) --------
b2/4.10.1: Building from source
...
b2/4.10.1: Created package revision 39eb34d0777dc531a81abd80f0466622
b2/4.10.1: Package '63fead0844576fc02943e16909f08fcdddd6f44b' created
b2/4.10.1: Full package reference: b2/4.10.1#8dc3df1cc73ad65d86cbdfd31fdb011f:63fead0844576fc02943e16909f08fcdddd6f44b#39eb34d0777dc531a81abd80f0466622
...
-------- Installing package boost/1.82.0 (5 of 5) --------
boost/1.82.0: Building from source
...
boost/1.82.0: Package 'eb34317ab81722ab55d97692af0d62754e8851dd' created
boost/1.82.0: Full package reference: boost/1.82.0#7c9cc60d65d6942a56d4806ab85bda7c:eb34317ab81722ab55d97692af0d62754e8851dd#e9d2b818be766aad4f94f0bd94f75e24

So now I have two package revisions of b2:

$ conan list 'b2/4.10.1:*#*'
Local Cache
  b2
    b2/4.10.1
      revisions
        8dc3df1cc73ad65d86cbdfd31fdb011f (2023-07-22 06:10:05 UTC)
          packages
            63fead0844576fc02943e16909f08fcdddd6f44b
              revisions
                a31a98f757dcf4d3f03ed629ccde26b7 (2023-08-11 15:24:55 UTC) <<< Binary pkg from CCI, broken on my system
                39eb34d0777dc531a81abd80f0466622 (2023-10-16 09:50:58 UTC) <<< My binary pkg
              info
                settings
                  arch: x86_64
                  os: Linux

Bui if I decide to build boost again (e.g. with different settings), conan reproducibly picks b2 from CCI:

$ conan install --requires=boost/1.82.0 -s build_type=Debug --build=missing
======== Computing necessary packages ========
Requirements
    boost/1.82.0#7c9cc60d65d6942a56d4806ab85bda7c:01e0916e01c9f024a112bfa3e77583644218b92c - Build
    bzip2/1.0.8#411fc05e80d47a89045edc1ee6f23c1d:4e3d72a07bfbfcf8bcc3e2a4a24a200d8acbceec - Build
    libbacktrace/cci.20210118#ec1aa63bbc10145c6a299e68e711670c:295d4ee81415d35a09959c89a57c51b82825332e - Build
    zlib/1.3#06023034579559bb64357db3a53f88a4:295d4ee81415d35a09959c89a57c51b82825332e - Build
Build requirements
    b2/4.10.1#8dc3df1cc73ad65d86cbdfd31fdb011f:63fead0844576fc02943e16909f08fcdddd6f44b#a31a98f757dcf4d3f03ed629ccde26b7 - Cache
...
-------- Installing package boost/1.82.0 (5 of 5) --------
boost/1.82.0: Building from source
...
boost/1.82.0: RUN: b2 -q numa=on target-os=linux architecture=x86 [...]
b2: /lib64/libc.so.6: version `GLIBC_2.17' not found (required by b2)
b2: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by b2)

Am I missing some sort of logic here? Shouldn't latest package revision be picked?

Currently I am working around by running conan remove -c 'b2/*' and building the package from source on the next run so that there is only one package in cache and it's mine, which is OK from me, but the behaviour is still weird.

The text was updated successfully, but these errors were encountered:

memsharded · 2023-10-16T12:02:16Z

Hi @BobIsOnFire

Thanks for your report, that certainly doesn't look right, let me investigate it.

memsharded · 2023-10-16T12:23:45Z

First initial test seems to be working fine, the latest package-revision is picked from the cache:

def test_package_revision_latest():
    c = TestClient()
    c.save({"tool/conanfile.py": GenConanfile("tool", "0.1").with_package_file("file.txt",
                                                                               env_var="MYVAR"),
            "pkg/conanfile.py": GenConanfile("pkg", "0.1").with_tool_requires("tool/0.1")})
    with environment_update({"MYVAR": "MYVALUE1"}):
        c.run("create tool")
    prev1 = c.created_package_revision("tool/0.1")
    with environment_update({"MYVAR": "MYVALUE2"}):
        c.run("create tool")
    prev2 = c.created_package_revision("tool/0.1")

    c.run("create pkg")
    assert prev2 in c.out  # latest package-revision is used
    assert prev1 not in c.out # not the older one

    c.run("install --requires=pkg/0.1 --build=pkg*")
    assert prev2 in c.out   # latest package-revision is used
    assert prev1 not in c.out   # not the older one

Are you sure that nothing is happening between the conan list 'b2/4.10.1:*#*' and the conan install --requires=boost/1.82.0? No other command or operation? If you do repeat the conan install --requires=boost/1.82.0 at the end of the conan install command that is picking the old package-revision, do you still get the same output?

BobIsOnFire · 2023-10-16T14:12:22Z

Hi @memsharded! Yes, I don't think I have run anything else...

I tried to minimize my setup and commands that I executed + some extra info that could be useful: https://github.com/BobIsOnFire/conan-14945-repro. And here are the full logs for the docker commands:

Now I can reproduce this very easily. Let me know what else I can run to help you!

BobIsOnFire · 2023-10-16T14:27:44Z

If you do repeat the conan install --requires=boost/1.82.0 at the end of the conan install command that is picking the old package-revision, do you still get the same output?

Sorry, I missed this. Yes, conan install --requires=boost/1.82.0 --build='boost/*' (without extra settings) fails for me even after I have built my own b2 package. My b2 isn't picked up at all.

Repro: BobIsOnFire/conan-14945-repro@ddd56eb

memsharded · 2023-10-16T14:52:20Z

I am quite baffled by this...
What happens if you remove the ConanCenter revision from the cache and runs without --update the second time? Is Conan able to pick your locally built one?

conan-io/conan#14945 (comment)

BobIsOnFire · 2023-10-16T15:24:04Z

Yes, after I remove ConanCenter binary, it successfully resolves to the only binary left in the cache: BobIsOnFire/conan-14945-repro@65dbc7b

Log: docker-run.log

memsharded · 2023-10-16T16:21:06Z

Ok, I'll investigate further with your repo docker setup, thanks for putting it together.

BobIsOnFire · 2023-10-16T16:26:10Z

I have one more insight: see these timestamps from conan list commands:

$ conan list 'b2/*#*:*'
Local Cache
  b2
    b2/4.10.1
      revisions
        8dc3df1cc73ad65d86cbdfd31fdb011f (2023-07-22 06:10:05 UTC)
          packages
            63fead0844576fc02943e16909f08fcdddd6f44b
              revisions
                a31a98f757dcf4d3f03ed629ccde26b7 (2023-08-11 15:24:55 UTC)
                694a7a070fa6e9329ac35e86105da771 (2023-10-16 15:34:21 UTC)
              info
                settings
                  arch: x86_64
                  os: Linux

$ conan list 'b2/*:*#latest'
Local Cache
  b2
    b2/4.10.1
      revisions
        8dc3df1cc73ad65d86cbdfd31fdb011f (2023-07-22 06:10:05 UTC)
          packages
            63fead0844576fc02943e16909f08fcdddd6f44b
              revisions
                a31a98f757dcf4d3f03ed629ccde26b7 (2023-10-16 15:34:21 UTC)
              info
                settings
                  arch: x86_64
                  os: Linux

The 'latest' query picked CCI revision and put a timestamp for the local build on it!

BobIsOnFire · 2023-10-16T16:29:59Z

Yeah, looks like this query is broken on my OS:

conan/conan/internal/cache/db/packages_table.py

Lines 130 to 143 in c271142

    
           query = f'SELECT {self.columns.reference}, ' \ 
        
                   f'{self.columns.rrev}, ' \ 
        
                   f'{self.columns.pkgid}, ' \ 
        
                   f'{self.columns.prev}, ' \ 
        
                   f'{self.columns.path}, ' \ 
        
                   f'MAX({self.columns.timestamp}), ' \ 
        
                   f'{self.columns.build_id} ' \ 
        
                   f'FROM {self.table_name} ' \ 
        
                   f'WHERE {self.columns.rrev} = "{pref.ref.revision}" ' \ 
        
                   f'AND {self.columns.reference} = "{str(pref.ref)}" ' \ 
        
                   f'AND {self.columns.pkgid} = "{pref.package_id}" ' \ 
        
                   f'{check_prev} ' \ 
        
                   f'AND {self.columns.prev} IS NOT NULL ' \ 
        
                   f'GROUP BY {self.columns.pkgid} '

This is what is sent to DB: SELECT reference, rrev, pkgid, prev, path, MAX(timestamp), build_id FROM packages WHERE rrev = "8dc3df1cc73ad65d86cbdfd31fdb011f" AND reference = "b2/4.10.1" AND pkgid = "63fead0844576fc02943e16909f08fcdddd6f44b" AND prev IS NOT NULL GROUP BY pkgid

I have put it into sqlite3 session (see my DB: cache.sqlite3.zip):

SQLite version 3.6.20
sqlite> .headers on
sqlite> select * from packages;
reference|rrev|pkgid|prev|path|timestamp|build_id
bzip2/1.0.8|411fc05e80d47a89045edc1ee6f23c1d|75807e7575ecc0f6141344f7c4f5e86106b92ec2|1c386a2fea96eec9e9e65835a6f79a04|b/bzip2f9d984ac77655|1697469989.61755|
libbacktrace/cci.20210118|ec1aa63bbc10145c6a299e68e711670c|897e6fde718e72637d92b0b8ed929893a1976238|ca38bf8d4684b42e3296d7800caaff58|b/libba77b38a904c024|1697469993.1005|
zlib/1.3|06023034579559bb64357db3a53f88a4|897e6fde718e72637d92b0b8ed929893a1976238|96c596c29d61b108409b27a1e954023c|b/zlib360bd96370a59|1697469996.00854|
boost/1.82.0|7c9cc60d65d6942a56d4806ab85bda7c|eb34317ab81722ab55d97692af0d62754e8851dd|b2398d46bbee97d783cf9cd5a97ff77a|b/boost54b7382a381c2|1697470226.28899|
b2/4.10.1|8dc3df1cc73ad65d86cbdfd31fdb011f|63fead0844576fc02943e16909f08fcdddd6f44b|a31a98f757dcf4d3f03ed629ccde26b7|b2ead845df79f5b|1691767495.215|
b2/4.10.1|8dc3df1cc73ad65d86cbdfd31fdb011f|63fead0844576fc02943e16909f08fcdddd6f44b|694a7a070fa6e9329ac35e86105da771|b/b233222725a8c34|1697470461.27146|
sqlite> SELECT reference, rrev, pkgid, prev, path, MAX(timestamp), build_id FROM packages WHERE rrev = "8dc3df1cc73ad65d86cbdfd31fdb011f" AND reference = "b2/4.10.1" AND pkgid = "63fead0844576fc02943e16909f08fcdddd6f44b"  AND prev IS NOT NULL GROUP BY pkgid
   ...> ;
reference|rrev|pkgid|prev|path|MAX(timestamp)|build_id
b2/4.10.1|8dc3df1cc73ad65d86cbdfd31fdb011f|63fead0844576fc02943e16909f08fcdddd6f44b|a31a98f757dcf4d3f03ed629ccde26b7|b2ead845df79f5b|1697470461.27146|

So, it picks up all data from the first entry, except for timestamp, which is picked from second one.

memsharded · 2023-10-16T16:41:26Z

That is very interesting. I am still wondering why it doesn't fail in other cases, I will send my test to CI to validate it in other platforms too. Maybe it is some subtle difference in the sqlite3 behavior in that plaform?

BobIsOnFire · 2023-10-16T17:20:55Z

Maybe it is some subtle difference in the sqlite3 behavior in that plaform?

It could be that my sqlite3 version (3.6.20) is quite old. I googled a little bit, looks like bare columns in aggregate queries (i.e. the ones not mentioned in either GROUP BY or aggregate functions) are not defined in SQL, but SQLite has an extension for min/max aggregates: https://www.sqlite.org/lang_select.html#bareagg. Maybe my old version doesn't have that extension yet 🤔

See also:

memsharded · 2023-10-16T18:01:42Z

From https://www.sqlite.org/chronology.html, it seems sqlite 3.6.20 is from 2009!! 😱 😱 😱
We are talking 14 years, I'd say that there is high chance that this is the reason, and that sqlite hasn't still implemented behavior that is assumed to work in the same way.

Do you think you could upgrade a bit that sqlite in your system and try?

Side note, I am trying to build your docker, but hitting:

 => ERROR [2/5] RUN yum -y install oracle-softwarecollection-release-el6 &&     yum -y install         rh-python36         devtoolset-8-gcc-c++ &&

I guess that this is not freely available?

BobIsOnFire · 2023-10-16T18:31:40Z

Yeah, "quite old" is an understatement 😅 Oracle Linux 6 itself is a 14-year-old distribution, but I am locked in with it for a while. I will try building a fresher version of sqlite3 tomorrow, and see if issue reproduces!

Side note, I am trying to build your docker, but hitting:

Sorry, it looks like the error message was cut out. Could you copy it one more time?

memsharded · 2023-10-16T18:37:21Z

 => ERROR [2/5] RUN yum -y install oracle-softwarecollection-release-el6 &&     yum -y install         rh-python36         devtoolset-8-gcc-c++ &&       0.6s ------
 > [2/5] RUN yum -y install oracle-softwarecollection-release-el6 &&     yum -y install         rh-python36         devtoolset-8-gcc-c++ &&     yum clean all && rm -rf /var/cache/yum:
------
executor failed running [/bin/sh -c yum -y install oracle-softwarecollection-release-el6 &&     yum -y install         rh-python36         devtoolset-8-gcc-c++ &&     yum clean all && rm -rf /var/cache/yum]: exit code: 139

BobIsOnFire · 2023-10-16T18:43:35Z

Are you using WSL, by any chance? There is a known issue which I also hit: microsoft/WSL#4694 (comment) (requires WSL VM restart)

memsharded · 2023-10-16T20:46:30Z

I am using docker desktop for windows (I think on wsl2), probably same issue, yes. Not sure I am able to change the WSL config, this is a company provisioned computer.

BobIsOnFire · 2023-10-17T08:54:30Z

Hi @memsharded,

I have built a newer sqlite3 version (3.43.2) from source, and the problem is gone now! Sorry if I took too much of your time yesterday 😅

I also found the commit where the extension was added: https://www.sqlite.org/src/info/fa13edd39c3b8ec7, which got into sqlite3 version 3.7.11 (2012-03-20 release date...) -- https://www.sqlite.org/releaselog/3_7_11.html. I think there is a way to write this SQL query via JOINs, which doesn't use any extensions and is more db-agnostic.

I know that rewriting a query to accomodate such old releases would be too much to ask, but do you think it makes sense to add an exception (or at least a warning) when user is using a too old sqlite3 version? Now that we know which versions are too old.

memsharded · 2023-10-17T10:09:21Z

I have built a newer sqlite3 version (3.43.2) from source, and the problem is gone now! Sorry if I took too much of your time yesterday 😅

Not a problem, this was a fun one to investigate 😄 , it was driving me crazy!

Yes, I think failing if the sqlite version is too old would be nice.
Checking if

>>> import sqlite3
>>> sqlite3.sqlite_version

is the right way to check the system sqlite version

memsharded · 2023-10-18T22:27:52Z

I am adding the check for sqlite version in PR #14950

BobIsOnFire · 2023-10-19T06:03:58Z

Thank you!

memsharded self-assigned this Oct 16, 2023

memsharded added the stage: triaging label Oct 16, 2023

BobIsOnFire pushed a commit to BobIsOnFire/conan-14945-repro that referenced this issue Oct 16, 2023

Reproduce without extra build_type

beb521f

conan-io/conan#14945 (comment)

BobIsOnFire pushed a commit to BobIsOnFire/conan-14945-repro that referenced this issue Oct 16, 2023

Remove offending package

4f2db3f

conan-io/conan#14945 (comment)

BobIsOnFire pushed a commit to BobIsOnFire/conan-14945-repro that referenced this issue Oct 16, 2023

Reproduce without extra build_type

2fd3ec4

conan-io/conan#14945 (comment)

BobIsOnFire pushed a commit to BobIsOnFire/conan-14945-repro that referenced this issue Oct 16, 2023

Remove offending package

c9154a8

conan-io/conan#14945 (comment)

BobIsOnFire added a commit to BobIsOnFire/conan-14945-repro that referenced this issue Oct 16, 2023

Reproduce without extra build_type

ddd56eb

conan-io/conan#14945 (comment)

BobIsOnFire added a commit to BobIsOnFire/conan-14945-repro that referenced this issue Oct 16, 2023

Remove offending package

65dbc7b

conan-io/conan#14945 (comment)

memsharded mentioned this issue Oct 16, 2023

test package revision order #14950

Merged

BobIsOnFire changed the title ~~[bug] Conan doesn't resolve reference to latest binary package~~ [bug] Conan doesn't resolve reference to latest binary package when old sqlite3 is installed Oct 17, 2023

czoido closed this as completed in #14950 Oct 19, 2023

memsharded added this to the 2.0.14 milestone Oct 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[bug] Conan doesn't resolve reference to latest binary package when old sqlite3 is installed #14945

[bug] Conan doesn't resolve reference to latest binary package when old sqlite3 is installed #14945

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

memsharded commented Oct 16, 2023

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 17, 2023 •

edited

Loading

memsharded commented Oct 17, 2023

memsharded commented Oct 18, 2023

BobIsOnFire commented Oct 19, 2023

[bug] Conan doesn't resolve reference to latest binary package when old sqlite3 is installed #14945

[bug] Conan doesn't resolve reference to latest binary package when old sqlite3 is installed #14945

Comments

BobIsOnFire commented Oct 16, 2023 • edited Loading

Environment details

Steps to reproduce

memsharded commented Oct 16, 2023

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023 • edited Loading

BobIsOnFire commented Oct 16, 2023 • edited Loading

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023 • edited Loading

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023 • edited Loading

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023 • edited Loading

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023 • edited Loading

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 16, 2023 • edited Loading

memsharded commented Oct 16, 2023

BobIsOnFire commented Oct 17, 2023 • edited Loading

memsharded commented Oct 17, 2023

memsharded commented Oct 18, 2023

BobIsOnFire commented Oct 19, 2023

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

BobIsOnFire commented Oct 16, 2023 •

edited

Loading

BobIsOnFire commented Oct 17, 2023 •

edited

Loading