[VTK 8.2.0/HDF5 1.10.5] Upgrade VTK and HDF5 #5574

Neumann-A · 2019-03-06T23:17:33Z

Ran out of time to test the build but I think it is a good start.

Rastaban · 2019-03-07T00:06:50Z

Rastaban · 2019-03-07T00:57:58Z

Thanks for taking this on! Here are the results I currently see in the CI system. We are currently seeing flakeyness in our CI system with all ports that depend on qt5-base due to the way it caches its results, so if you are not able to reproduce the osg-qt failures locally they can probably be ignored for now. Let me know if you would like me to attach any of the failure logs.

 Processing arm-uwp                        411 vs 411
 Processing arm64-windows                  470 vs 470
 Processing x64-linux                      606 vs 610
     alembic                                                     Skip vs Fail
     hdf5                          **regression**                Fail vs Pass
     highfive                                                    Skip vs Fail
     itk                                                         Skip vs Fail
     kealib                                                      Skip vs Pass
     matio                                                       Skip vs Fail
     netcdf-c                                                    Skip vs Pass
     netcdf-cxx4                                                 Skip vs Pass
     vtk                                                         Skip vs Fail
 Processing x64-osx                        597 vs 604
     alembic                                                     Skip vs Fail
     hdf5                          **regression**                Fail vs Pass
     highfive                                                    Skip vs Pass
     itk                                                         Skip vs Fail
     kealib                                                      Skip vs Pass
     matio                                                       Skip vs Fail
     netcdf-c                                                    Skip vs Pass
     netcdf-cxx4                                                 Skip vs Pass
     vtk                                                         Skip vs Pass
     vtk-dicom                                                   Skip vs Pass
 Processing x64-uwp                        439 vs 439
 Processing x64-windows                    877 vs 881
     osg-qt                        **regression**                Fail vs Pass
     pcl                                                         Skip vs Pass
     vtk                           **regression**                Fail vs Pass
     vtk-dicom                                                   Skip vs Pass
 Processing x64-windows-static             773 vs 776
     pcl                                                         Skip vs Pass
     vtk                           **regression**                Fail vs Pass
     vtk-dicom                                                   Skip vs Pass
 Processing x86-windows                    856 vs 860
     osg-qt                        **regression**                Fail vs Pass
     pcl                                                         Skip vs Pass
     vtk                           **regression**                Fail vs Pass
     vtk-dicom                                                   Skip vs Pass

Neumann-A · 2019-03-07T21:24:30Z

Current problem: Get VTK to link ogg (or Ogg? maybe a name conflict?)

Ok got it to work but had to rename a OGG to Ogg in Thirdparty/ogg/CMakeLists.txt. Wondering what the correct libraryname for ogg is.

Neumann-A · 2019-03-07T23:44:53Z

@Rastaban: I need to know where the regressions are after the last commit and what happend. I succesfully build x64-windows but times up for today.

Neumann-A · 2019-03-08T10:46:26Z

ok i'll give up with trying to test pcl with vtk 8.2.0 on linux.
Hitting the following while compiling flann with gcc 7.3 :
c++: internal compiler error: Killed (program cc1plus).

So on x64-windows PCL builds fine. On linux vtk 8.2 builds fine. Currently testing x64-windows-static but I don't expect any surprises. So if there are still regression please let me know and attach the logs.

Neumann-A · 2019-03-08T11:48:36Z

I spoke too soon but the last commit fixed the x64-windows-static build. On my machine it build successfully but failed to copy to installed/x64-windows-static because I ran out of disk space.
Nevertheless it reported: Installing package vtk[core]:x64-windows-static... done
So this PR should be wrapped up... lets wait what CI tells us

Rastaban · 2019-03-09T07:22:11Z

Sorry for the delay, here is what I see now. As mentioned earlier, any failure that has a dependancy on qt5-base can be ignored for now because of #5566. Let me know if you need any of the failure logs.

processing arm-uwp                        410 vs 411
     qhull                         **regression**                Fail vs Pass
 Processing arm64-windows                  470 vs 471
     qhull                         **regression**                Fail vs Pass
 Processing x64-linux                      615 vs 614
     highfive                                                    Pass vs Fail
     kealib                        **regression**                Fail vs Pass
     qhull                                                       Pass vs Fail
 Processing x64-osx                        609 vs 607
     pcl                                                         Pass vs Skip
     qhull                                                       Pass vs Fail
 Processing x64-uwp                        438 vs 439
     qhull                         **regression**                Fail vs Pass
 Processing x64-windows                    881 vs 884
     osg-qt                        **regression**                Fail vs Pass
     pcl                                                         Skip vs Pass
     qhull                         **regression**                Fail vs Pass
 Processing x64-windows-static             774 vs 778
     pcl                                                         Skip vs Pass
     qhull                         **regression**                Fail vs Pass
     vtk                           **regression**                Fail vs Pass
     vtk-dicom                                                   Skip vs Pass
 Processing x86-windows                    861 vs 864
     osg-qt                        **regression**                Fail vs Pass
     pcl                                                         Skip vs Pass
     qhull                         **regression**                Fail vs Pass

Neumann-A · 2019-03-09T13:24:14Z

kealib regression Fail vs Pass

Error:

/vcpkg/buildtrees/kealib/src/kealib-1.4.7/trunk/include/libkea/KEAAttributeTableInMem.h:38:10: fatal error: H5Cpp.h: No such file or directory
 #include "H5Cpp.h"

Can somebody report upstream that they need to use #include <H5Cpp.h> everywhere to correctly include this header? Maybe: @miurahr?

Neumann-A · 2019-03-09T13:36:15Z

 Processing x64-windows-static             774 vs 778
     pcl                                                         Skip vs Pass
     qhull                         **regression**                Fail vs Pass
     vtk                           **regression**                Fail vs Pass
     vtk-dicom                                                   Skip vs Pass

@Rastaban: I cannot reproduce the vtk regression in x64-windows-static

Building package vtk[core]:x64-windows-static...
-- Downloading https://github.com/Kitware/VTK/archive/v8.2.0.tar.gz...
-- Extracting source D:/vcpkg_test/vcpkg/downloads/Kitware-VTK-v8.2.0.tar.gz
-- Applying patch fix-find-lz4.patch
-- Applying patch fix_ogg_linkage.patch
-- Using source at D:/vcpkg_test/vcpkg/buildtrees/vtk/src/v8.2.0-4dfb703163
-- Configuring x64-windows-static
-- Building x64-windows-static-dbg
-- Building x64-windows-static-rel
-- Performing post-build validation
-- Performing post-build validation done
Building package vtk[core]:x64-windows-static... done
Installing package vtk[core]:x64-windows-static...
Installing package vtk[core]:x64-windows-static... done
Elapsed time for package vtk:x64-windows-static: 20.81 min

Neumann-A · 2019-03-09T23:12:52Z

@Rastaban I would like to know the regression of 07b0183 and if af06207 introduced any new regressions.
If 07b0183 still has the regressions in x64-windows-static then I also need the log files because I couldn't reproduce those.

PhoebeHui · 2019-03-12T09:25:06Z

@Neumann-A, check the latest results and attach the failure logs:

 Processing arm-uwp                        412 vs 412
 Processing arm64-windows                  472 vs 472
 Processing x64-linux                      619 vs 617
     highfive                                                    Pass vs Fail
     qhull                                                       Pass vs Fail
 Processing x64-osx                        612 vs 610
     pcl                                                         Pass vs Skip
     qhull                                                       Pass vs Fail
 Processing x64-uwp                        440 vs 440
 Processing x64-windows                    878 vs 886
     gdal                                                        Skip vs Pass
     netcdf-c                      **regression**                Fail vs Pass
     netcdf-cxx4                                                 Skip vs Pass
     osg                                                         Skip vs Pass
     osg-qt                                                      Skip vs Pass
     osgearth                                                    Skip vs Pass
     pdal                                                        Skip vs Pass
     pdal-c                                                      Skip vs Pass
 Processing x64-windows-static             778 vs 780
     pcl                                                         Skip vs Pass
     qhull                         **regression**                Fail vs Pass
 Processing x86-windows                    858 vs 866
     gdal                                                        Skip vs Pass
     netcdf-c                      **regression**                Fail vs Pass
     netcdf-cxx4                                                 Skip vs Pass
     osg                                                         Skip vs Pass
     osg-qt                                                      Skip vs Pass
     osgearth                                                    Skip vs Pass
     pdal                                                        Skip vs Pass
     pdal-c                                                      Skip vs Pass

failureLogs.zip

Neumann-A · 2019-03-12T11:53:32Z

@PhoebeHui
The qhull failure raises my eyebrows:
LINK : fatal error LNK1201: error writing to program database 'C:\vsts\_work\2\s\buildtrees\qhull\x64-windows-static-rel\qhull.pdb'; check for insufficient disk space, invalid path, or insufficient privilege
But I can also reproduce it locally with the same error message. Any idea where that might come from? I see that the folder in windows is partially write protected for some strange reason.

Seems to be a ninja issue...

Rastaban · 2019-03-12T18:55:36Z

In the latest test run the only regressions are qhull on arm64-windows and x86-windows with the same LNK1201 error

Neumann-A · 2019-03-12T20:09:43Z

@Rastaban: Hmm tried it at home for x86-windows and it worked the first time. Removed it and updated Visual Studio 2017 to newest version -> LNK 1201. Seems to be a regression in the build tools not correctly releasing the lock on the file or something similar.

Neumann-A · 2019-03-12T20:12:23Z

and to top it off.... if I readd PREFER_NINJA it works again o.O. How should we treat that? Certainly not a vcpkg issue

miurahr · 2019-03-13T02:46:31Z

Can somebody report upstream that they need to use #include <H5Cpp.h> everywhere to correctly include this header? Maybe: @miurahr?

This commit maybe by @gillins

gillins · 2019-03-13T03:42:55Z

Is the HDF5 C++ library still being built? FYI might be worth upgrading to kealib 1.4.11 - many improvements to the build system have been made.

Rastaban · 2019-03-13T22:41:41Z

@Neumann-A you may be right about the toolset regression. Looking through the internal MSVC bug database there have been a few issues related to LNK1201 that have been fixed recently.

Rastaban · 2019-03-13T23:26:32Z

Other than the qhull failure on x64-windows-static everything else passes the tests. Are there any other outstanding issue that need to be resolved before this is merged?

see discussion in microsoft#5574

Neumann-A · 2019-03-14T14:10:34Z

@Rastaban: should i clean up the commit history or do you want to merge it the way it is? There are probably other portfiles which can remove extra code changing *targets.cmake due to commit 0b5a576 but that could be done in another PR. The problem was that the release version of the targets file was missing the debug information due to the fact that cmake was simply not finding the debug versions due to the missing path. (folly is such an example I at least know of.)

Rastaban · 2019-03-14T18:01:36Z

ports/qhull/portfile.cmake

@@ -7,7 +7,20 @@ vcpkg_from_github(
    SHA512 8f5177ea45f82fa28f13e95105497e7e29086d7301e1cb8d3860fff09ebf8d0f01cfcb0f044c422f0ac0ba94b845bba223232e5eeb613bf671f65a569b8766d0
    HEAD_REF master
 )
-
+if(${VCPKG_TARGET_TRIPLET} STREQUAL "x64-windows-static") 


this should be ${TARGET_TRIPLET}

fixed in
1feb0dc

Rastaban · 2019-03-14T18:06:51Z

I was going to do a Squash and Merge, but if you are willing to cleanup the commit history then I can just merge it when you are done.

extra code comments. Maybe the test should actually try to link targets from the package instead of just testing find_package

Also now builds dynamic and static libs in dynamic build due to the targets exported by hdf5. (Revert to default hdf5 build behavior)

HDF5_USE_STATIC_LIBRARIES must be set in static builds so that find_package(HDF5) finds the static hdf5 libraries

Neumann-A mentioned this pull request Mar 7, 2019

[libogg] Add package config module #5384

Merged

Neumann-A changed the title ~~[WIP] Upgrade Vtk 8.2.0~~ [VTK 8.2.0/HDF5 1.10.5] Upgrade VTK and HDF5 Mar 8, 2019

Neumann-A mentioned this pull request Mar 8, 2019

Update HDF5 to 1.10.5 and build both static and shared hdf5 libs #4777

Closed

Neumann-A mentioned this pull request Mar 8, 2019

Fix double-conversion issue in ITK #5115

Merged

luzpaz mentioned this pull request Mar 8, 2019

Openscenegraph: Update to 3.6.3 #5443

Closed

Rastaban self-assigned this Mar 9, 2019

Neumann-A mentioned this pull request Mar 11, 2019

TODO: Test if VTK can use vcpkg_fixup_cmake_targets #5612

Closed

Neumann-A added a commit to Neumann-A/vcpkg that referenced this pull request Mar 14, 2019

workaround for toolset regression in qhull

5ad730e

see discussion in microsoft#5574

Rastaban reviewed Mar 14, 2019

View reviewed changes

Neumann-A added 13 commits March 14, 2019 22:24

fix missing debug references in cmake generated (release) target files.

1c7972a

cmake sometimes requires a generator if it is not defined beforehand.

bdc5d53

extra code comments. Maybe the test should actually try to link targets from the package instead of just testing find_package

default szip to also build static libraries in shared builds

9f41fed

update szip control

69bacbb

updated hdf5 to 1.10.5.

49f59fd

Also now builds dynamic and static libs in dynamic build due to the targets exported by hdf5. (Revert to default hdf5 build behavior)

kealib update to 1.4.11 and correctly include hdf5

4de9cb0

update netcdf-c tp 4.6.2 and fixed curl linkage

0e898fe

fixed flann for linux

7419fc9

fix highfive in static builds.

9ac08b3

HDF5_USE_STATIC_LIBRARIES must be set in static builds so that find_package(HDF5) finds the static hdf5 libraries

fixes microsoft#5562

6c3c673

correct hdf5 usage in netcdf-cxx4

1ee65c1

workaround for qhull regression in static build see microsoft#5574

1feb0dc

vtk update to 8.2.0

d5fbb5e

Neumann-A force-pushed the VTK_8.2.0 branch from 5ad730e to d5fbb5e Compare March 14, 2019 21:39

Rastaban merged commit d72e380 into microsoft:master Mar 15, 2019

Rastaban mentioned this pull request Mar 15, 2019

[vtk] update to 8.2.0 #5462

Closed

vicroms mentioned this pull request Mar 15, 2019

Make sure vcpkg_test_cmake uses the correct generator #5635

Closed

Neumann-A mentioned this pull request Mar 27, 2019

[lzma] Locate correct release/debug libraries in static build #5625

Closed

myd7349 mentioned this pull request May 23, 2019

What is the magic behind vcpkg_test_cmake()? #5249

Closed

Neumann-A deleted the VTK_8.2.0 branch June 3, 2019 21:11

GiulioRomualdi mentioned this pull request Jun 18, 2020

Implement the ConvexHullHelper class ami-iit/bipedal-locomotion-framework#51

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[VTK 8.2.0/HDF5 1.10.5] Upgrade VTK and HDF5 #5574

[VTK 8.2.0/HDF5 1.10.5] Upgrade VTK and HDF5 #5574

Neumann-A commented Mar 6, 2019

Rastaban commented Mar 7, 2019

Rastaban commented Mar 7, 2019

Neumann-A commented Mar 7, 2019 •

edited

Loading

Neumann-A commented Mar 7, 2019

Neumann-A commented Mar 8, 2019 •

edited

Loading

Neumann-A commented Mar 8, 2019

Rastaban commented Mar 9, 2019

Neumann-A commented Mar 9, 2019

Neumann-A commented Mar 9, 2019

Neumann-A commented Mar 9, 2019

PhoebeHui commented Mar 12, 2019

Neumann-A commented Mar 12, 2019 •

edited

Loading

Rastaban commented Mar 12, 2019

Neumann-A commented Mar 12, 2019

Neumann-A commented Mar 12, 2019

miurahr commented Mar 13, 2019

gillins commented Mar 13, 2019

Rastaban commented Mar 13, 2019 •

edited

Loading

Rastaban commented Mar 13, 2019

Neumann-A commented Mar 14, 2019

Rastaban Mar 14, 2019

Neumann-A Mar 14, 2019

Rastaban commented Mar 14, 2019

[VTK 8.2.0/HDF5 1.10.5] Upgrade VTK and HDF5 #5574

[VTK 8.2.0/HDF5 1.10.5] Upgrade VTK and HDF5 #5574

Conversation

Neumann-A commented Mar 6, 2019

Rastaban commented Mar 7, 2019

Rastaban commented Mar 7, 2019

Neumann-A commented Mar 7, 2019 • edited Loading

Neumann-A commented Mar 7, 2019

Neumann-A commented Mar 8, 2019 • edited Loading

Neumann-A commented Mar 8, 2019

Rastaban commented Mar 9, 2019

Neumann-A commented Mar 9, 2019

Neumann-A commented Mar 9, 2019

Neumann-A commented Mar 9, 2019

PhoebeHui commented Mar 12, 2019

Neumann-A commented Mar 12, 2019 • edited Loading

Rastaban commented Mar 12, 2019

Neumann-A commented Mar 12, 2019

Neumann-A commented Mar 12, 2019

miurahr commented Mar 13, 2019

gillins commented Mar 13, 2019

Rastaban commented Mar 13, 2019 • edited Loading

Rastaban commented Mar 13, 2019

Neumann-A commented Mar 14, 2019

Rastaban Mar 14, 2019

Choose a reason for hiding this comment

Neumann-A Mar 14, 2019

Choose a reason for hiding this comment

Rastaban commented Mar 14, 2019

Neumann-A commented Mar 7, 2019 •

edited

Loading

Neumann-A commented Mar 8, 2019 •

edited

Loading

Neumann-A commented Mar 12, 2019 •

edited

Loading

Rastaban commented Mar 13, 2019 •

edited

Loading