update outdated dependencies #829

remi-dupre · 2021-12-02T15:18:40Z

This takes parts of a larger effort to unify all dependencies in mimirsbrunn, a few notes here:

latest version of chrono appears to have a vulnerability: https://rustsec.org/advisories/RUSTSEC-2020-0159, I don't exactly understand why but it seems that it doesn't matter for a normal usage: The call to localtime_r may be unsound chronotope/chrono#499 (comment)
minidom wasn't updated: [feature] update to 'minidom:0.13.0' #746
replaced unmaintained failure with anyhow

datanel · 2021-12-02T15:40:36Z

The tests fail because we use proj 7.2.1 but the crate proj:0.24 requires proj >= 8.1.0

Should we upgrade the version of proj in the docker image ? @woshilapin @pbougue

src/gtfs/read.rs

woshilapin · 2021-12-02T16:37:32Z

The tests fail because we use proj 7.2.1 but the crate proj:0.24 requires proj >= 8.1.0

Should we upgrade the version of proj in the docker image ? @woshilapin @pbougue

I think we can take the opportunity to upgrade to Proj:8.1.0 (which is the version for the crate proj:0.24.0). I'm trying to update https://github.com/CanalTP/ci-images for that. I'll keep you posted how it goes.

woshilapin · 2021-12-02T17:35:27Z

Might need https://github.com/CanalTP/ci-images/pull/32 to be merged and released first but this should help us land your PR with a upgraded version of proj.

woshilapin · 2021-12-02T18:17:45Z

New docker images for the CI have been updated and released with Proj:8.1.0 inside. I've made a small PR on top of your branch that should make transit_model aware of these new images (I hope I didn't forget anything).

Note that by a happy coincidence, the new Docker image was build just after rust:1.57.0 as released. This mean you might have new clippy warnings showing up to fix (our CI check for clippy warnings).

datanel · 2021-12-03T10:05:26Z

@remi-dupre there is still some tests failing but I can push a fix if you want

remi-dupre · 2021-12-03T10:27:40Z

I should be fined, I initially missed the all-targets --all-features parameters and a few system dependencies (proj, clang) as I've just received a new laptop for work.

datanel · 2021-12-03T10:36:06Z

I should be fined, I initially missed the all-targets --all-features parameters and a few system dependencies (proj, clang) as I've just received a new laptop for work.

Just execute make test

remi-dupre · 2021-12-03T10:37:28Z

I still have some failing tests on my side, I wonder if it comes from the update of proj?

---- test_write_netex_france_from_gtfs stdout ----
thread 'test_write_netex_france_from_gtfs' panicked at 'assertion failed: `(left == right)`

Diff < left / right > :
 [
     "<?xml version=\"1.0\" encoding=\"UTF-8\"?>",
     "<PublicationDelivery version=\"1.09:FR-NETEX_ARRET-2.1-1.0\" xmlns=\"http://www.netex.org.uk/netex\" xmlns:core=\"http://www.govtalk.gov.uk/core\" xmlns:gml=\"http://www.opengis.net/gml/3.2\" xmlns:ifopt=\"http://www.ifopt.org.uk/ifopt\" xmlns:siri=\"http://www.siri.org.uk/siri\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://www.netex.org.uk/netex\">",
     "\t<PublicationTimestamp>2019-04-03T17:19:00+00:00</PublicationTimestamp>",
     "\t<ParticipantRef>Participant</ParticipantRef>",
     "\t<dataObjects>",
     "\t\t<GeneralFrame id=\"FR:GeneralFrame:NETEX_ARRET:\" version=\"any\">",
     "\t\t\t<members>",
     "\t\t\t\t<Quay id=\"FR:Quay:GDLR:\" version=\"any\">",
     "\t\t\t\t\t<Name>Gare de Lyon (RER)</Name>",
     "\t\t\t\t\t<Centroid>",
     "\t\t\t\t\t\t<Location>",
<    "\t\t\t\t\t\t\t<gml:pos srsName=\"EPSG:2154\">653983.726554971 6860704.890453682</gml:pos>",
>    "\t\t\t\t\t\t\t<gml:pos srsName=\"EPSG:2154\">653983.726554971 6860704.890453683</gml:pos>",
     "\t\t\t\t\t\t</Location>",
     "\t\t\t\t\t</Centroid>",
     "\t\t\t\t\t<AccessibilityAssessment id=\"FR:AccessibilityAssessment:GDLR_0:\" version=\"any\">",
     "\t\t\t\t\t\t<MobilityImpairedAccess>partial</MobilityImpairedAccess>",
[...]
     "\t\t\t\t\t<Centroid>",
     "\t\t\t\t\t\t<Location>",
<    "\t\t\t\t\t\t\t<gml:pos srsName=\"EPSG:2154\">650223.8079440364 6860484.10406642</gml:pos>",
>    "\t\t\t\t\t\t\t<gml:pos srsName=\"EPSG:2154\">650223.8079440364 6860484.104066422</gml:pos>",
     "\t\t\t\t\t\t</Location>",
     "\t\t\t\t\t</Centroid>",
     "\t\t\t\t\t<TransportMode>bus</TransportMode>",
     "\t\t\t\t\t<StopPlaceType>busStation</StopPlaceType>",
     "\t\t\t\t</StopPlace>",
     "\t\t\t</members>",
     "\t\t</GeneralFrame>",
     "\t</dataObjects>",
     "</PublicationDelivery>",
 ]

', /home/remi/code/navitia/transit_model/src/test_utils.rs:125:9

datanel · 2021-12-03T10:38:43Z

I still have some failing tests on my side, I wonder if it comes from the update of proj?

---- test_write_netex_france_from_gtfs stdout ----
thread 'test_write_netex_france_from_gtfs' panicked at 'assertion failed: `(left == right)`

Diff < left / right > :
 [
     "<?xml version=\"1.0\" encoding=\"UTF-8\"?>",
     "<PublicationDelivery version=\"1.09:FR-NETEX_ARRET-2.1-1.0\" xmlns=\"http://www.netex.org.uk/netex\" xmlns:core=\"http://www.govtalk.gov.uk/core\" xmlns:gml=\"http://www.opengis.net/gml/3.2\" xmlns:ifopt=\"http://www.ifopt.org.uk/ifopt\" xmlns:siri=\"http://www.siri.org.uk/siri\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://www.netex.org.uk/netex\">",
     "\t<PublicationTimestamp>2019-04-03T17:19:00+00:00</PublicationTimestamp>",
     "\t<ParticipantRef>Participant</ParticipantRef>",
     "\t<dataObjects>",
     "\t\t<GeneralFrame id=\"FR:GeneralFrame:NETEX_ARRET:\" version=\"any\">",
     "\t\t\t<members>",
     "\t\t\t\t<Quay id=\"FR:Quay:GDLR:\" version=\"any\">",
     "\t\t\t\t\t<Name>Gare de Lyon (RER)</Name>",
     "\t\t\t\t\t<Centroid>",
     "\t\t\t\t\t\t<Location>",
<    "\t\t\t\t\t\t\t<gml:pos srsName=\"EPSG:2154\">653983.726554971 6860704.890453682</gml:pos>",
>    "\t\t\t\t\t\t\t<gml:pos srsName=\"EPSG:2154\">653983.726554971 6860704.890453683</gml:pos>",
     "\t\t\t\t\t\t</Location>",
     "\t\t\t\t\t</Centroid>",
     "\t\t\t\t\t<AccessibilityAssessment id=\"FR:AccessibilityAssessment:GDLR_0:\" version=\"any\">",
     "\t\t\t\t\t\t<MobilityImpairedAccess>partial</MobilityImpairedAccess>",
[...]
     "\t\t\t\t\t<Centroid>",
     "\t\t\t\t\t\t<Location>",
<    "\t\t\t\t\t\t\t<gml:pos srsName=\"EPSG:2154\">650223.8079440364 6860484.10406642</gml:pos>",
>    "\t\t\t\t\t\t\t<gml:pos srsName=\"EPSG:2154\">650223.8079440364 6860484.104066422</gml:pos>",
     "\t\t\t\t\t\t</Location>",
     "\t\t\t\t\t</Centroid>",
     "\t\t\t\t\t<TransportMode>bus</TransportMode>",
     "\t\t\t\t\t<StopPlaceType>busStation</StopPlaceType>",
     "\t\t\t\t</StopPlace>",
     "\t\t\t</members>",
     "\t\t</GeneralFrame>",
     "\t</dataObjects>",
     "</PublicationDelivery>",
 ]

', /home/remi/code/navitia/transit_model/src/test_utils.rs:125:9

yes it comes from proj, good luck 😜

datanel · 2021-12-03T10:41:43Z

@remi-dupre I already have the fix about proj, I can push it if want

remi-dupre · 2021-12-03T10:43:16Z

@remi-dupre I already have the fix about proj, I can push it if want

Sure! Go for it, I'm not used to work with proj so I may have struggled (I was at the step of manually installing proj as latest available version from arch repositories is still 8.0).

remi-dupre · 2021-12-03T11:24:02Z

Thanks for the help with the rough corners!

woshilapin · 2021-12-03T11:54:19Z

@remi-dupre I already have the fix about proj, I can push it if want

Sure! Go for it, I'm not used to work with proj so I may have struggled (I was at the step of manually installing proj as latest available version from arch repositories is still 8.0).

With almost every update we made with proj we had to fix our tests. If you don't know proj, it's basically for playing with and converting geolocations from one system of projection to another. But it seems that with proj updates comes some improvement in their conversion system (more precise, adjusting for evolution of tectonic or whatever): therefore, some conversion change results slightly (and break our tests).

woshilapin

Seems good. Thank you very much for pulling out al this work.

remi-dupre added 2 commits December 2, 2021 16:12

update outdated dependencies

72905dd

bump version to 0.42.2

5b0a081

replace unmaintained crate failure with anyhow

5591fb7

woshilapin reviewed Dec 2, 2021

View reviewed changes

src/gtfs/read.rs Show resolved Hide resolved

woshilapin and others added 3 commits December 3, 2021 09:06

fix: update docker images for proj:8.1.0 (#1)

5ce5654

fix clippy lints

1e8cb08

use anyhow for netex_france

0a5df6b

fix iter_chain in main files

417f2e0

David Quintanel added 2 commits December 3, 2021 11:46

Fix proj precision in fixtures

d500860

bump subcrate version

4287c15

datanel approved these changes Dec 3, 2021

View reviewed changes

woshilapin approved these changes Dec 3, 2021

View reviewed changes

woshilapin merged commit c95cddd into hove-io:master Dec 3, 2021

datanel mentioned this pull request Jan 26, 2023

Remove failure #442

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update outdated dependencies #829

update outdated dependencies #829

remi-dupre commented Dec 2, 2021 •

edited

Loading

datanel commented Dec 2, 2021

woshilapin commented Dec 2, 2021

woshilapin commented Dec 2, 2021 •

edited

Loading

woshilapin commented Dec 2, 2021 •

edited

Loading

datanel commented Dec 3, 2021

remi-dupre commented Dec 3, 2021

datanel commented Dec 3, 2021

remi-dupre commented Dec 3, 2021

datanel commented Dec 3, 2021

datanel commented Dec 3, 2021

remi-dupre commented Dec 3, 2021

remi-dupre commented Dec 3, 2021

woshilapin commented Dec 3, 2021

woshilapin left a comment

update outdated dependencies #829

update outdated dependencies #829

Conversation

remi-dupre commented Dec 2, 2021 • edited Loading

datanel commented Dec 2, 2021

woshilapin commented Dec 2, 2021

woshilapin commented Dec 2, 2021 • edited Loading

woshilapin commented Dec 2, 2021 • edited Loading

datanel commented Dec 3, 2021

remi-dupre commented Dec 3, 2021

datanel commented Dec 3, 2021

remi-dupre commented Dec 3, 2021

datanel commented Dec 3, 2021

datanel commented Dec 3, 2021

remi-dupre commented Dec 3, 2021

remi-dupre commented Dec 3, 2021

woshilapin commented Dec 3, 2021

woshilapin left a comment

Choose a reason for hiding this comment

remi-dupre commented Dec 2, 2021 •

edited

Loading

woshilapin commented Dec 2, 2021 •

edited

Loading

woshilapin commented Dec 2, 2021 •

edited

Loading