Make the Windows tests run reliably #775
Assignees
Labels
Feature: engineering
related to internal engineering work: tests, refactoring, etc.
Milestone
Comments
KoeMai
added a commit
to KoeMai/vscode-cmake-tools
that referenced
this issue
Nov 10, 2019
bobbrow
pushed a commit
that referenced
this issue
Jan 8, 2020
* Add driver code model interface - Create driver code model interfac The cms client class has a code model. This code model is defined by the cmake cmake-server(7). The cmake-tools-extension needs a code model, which does not have to be identical to the cmake server. For more flexibility in future, a separate model is created. The first version contains all information required for the extension (e.g. for cpptools, code model tree) and is based on the cmake-server(7) model. - Separate code mode driver support from CMS driver version * Add first code model test and fix typos and a small bug * Fix unix build by separate test for single configuration generators * Fix node.js version problem with streamfilter * Add tests for project and target information * Fix linux build and reformat code model tests * Extend tests for shared library and utilities - Add tests for interface parts of cpptools - Add tests for utilities parts of tree view - Extend CMake test Project by required calls * Fix missing isGenerated in cms-client * Fix wrong library filename * Fix MacOS build and disable windows test for mingw (#775) * Fix regex * Update codemodel driver interface .documentation * Clean up tests and add sysroot test Clean up - Add helper for code model generation - Restructure test - Fix spelling Add missing sysroot test * Fix instable tests - Different CMake versions generate different code model outputs. - macOS test fails. I am unable to test the problem, but it seems the sysroot variable is required to build on macOS. * Remove test filter
bobbrow
pushed a commit
that referenced
this issue
Jan 27, 2020
* Replace workspace path read from vscode to constructor variable - The workspace is read from vscode, this makes it hard to refactor cms-driver to test compiler integration on a lower level. The workspace needed to be set by a higher level, to make low level driver independent of vscode ui and data structures. - Make non public function protected. * Replaced pickGenerator by variable for generator The cms client normally started a pick call in the UI. This modification moves the questions infront of the start. This removes the required callback from cms client. * Clean up CMake Build Start Problem UI interaction The CMake Driver has many dependencies to UI. This modification moves the dependencies from CMakeDriver into cmake-tools. This allows to test the driver without UI interaction. - Move quickstart and missing source dir warning - Remove not required configuration setting `autoRestartBuild` I was not able to bring the driver into a state where two builds run in parallel. The `CMakePreconditionProblemSolver` never signals `busy` CMake. - Change visability of some do methods in cms-driver. * Fix working path getter * Fix parallel clean and configure on driver level The parallel execution of cmake configure is allowed by the driver. This modification will forbid a parallel configuration. * Move selection of prefered generator The prefered generators are extracted by the cmake drive from configuration, kit files or settings. This modification moves the aggregation of different prefered generator out of the driver into the cmake tools. This allows to defines exact preferred generators for testing (no hidden dependency). The driver only checks the prefered generator list, if there is more then one then the first existing generator is selected. - Fix wrong preferred generator definition in test - Extend debug information * Remove toolset dependency in cms-client The CMS Server client had a dependency to configuration reader to read the toolset. For my this is a inconsistent information. Only the toolset was from config file but not the platform. * Add cms driver test for linux * Fix problem with invalid preferred generator * Add tests for reconfiguration and generator switch * Harmonize cleanConfigure with configure in driver. * Remove Workspace state dependency from driver This modification reduces the dependency of the cmake driver layer to the workspace context. This modification allows to setup low level CMake driver tests, so that the driver code could be tested with different cmake versions. The UI parts can then tested with a driver mock, or by system tests. - Replace DirectoryContext by ConfigurationReader - Moved access to DirectoryContext state by setter funcions (setVarianOptions). This allows to set variant in a consistent way. - Reduces fakes from tests * Clean up and fix variant naming * Extract cmake flags generation out of configure method for readability * Stabilize tests and reformat code * Set timeout for teardown of driver * Fix some errors * Restructure drivers * Move cmake server client driver part * Add first parts of cmake file api implementation * Clean up cmake file api code * Add workaround for preconfigured projects Added a workaround to get the preconfigured generator and its settings from cache file. * Implement target listing and restructure code - Fix test for switching kit files - Implement target map for configurations - Add workaround for recover already configured cmake projects - Add file api helpers and interface * Fix wrong generator configuration in tests and assign default value of generator in driver. * Fix problems on generator errors or missing query response - CI: Add cmake to search path - Fix FileAPI driver compatibility on missing generator - Fix FileAPI driver problem with missing reply file - Add test for configuration behavior on configuration error of cmake * Generate api query files on start of cmake configuration The generation of the api query files should although be done when a "unspecified kit" is used. This requires to create the file on each configuration, not only on init of a new build directory or an clean up before build. * CI loads wrong cmake version * Fix cmake version problem for unix global * Try to fix travis mac os problem with bad executable * Restructure codemodel api This modification extracts the code model from the UI elements (tree view) and compile information into a code model API. It is necessary to implement two driver variants - CMake Server CodeModel - CMake FileAPI Code Model The code model support is detected by a special driver family (`CMakeCodeModelDriver`), which provides a property for `onCodeModelChanged`. The driver api contains one required information from cmake code model for generation of cpptool compiler information and for the cmake tree view. * Implement code model parts for the tree view. - Inherit from `CMakeCodeModelDriver` instead of `CMakeDriver` - Implement FileAPI translation functions for Extension Driver_API * Add first implementation for cpptool required compile information - Replace cmake server depencencies by driver_api - Extend API py missing language information - Extend CMake FileAPI interface for CompileGroups - Implement required convert functions * Fix access on map from undefined object * Add missing "all" target and "install" target and fix problems on switch between MsBuild and ninja * Fix some path resolving problems in code model * Fix cmake tree view - open file and missing headers * Fix/harmonize cms-driver and cmfileapi-driver tests - Fix `Test generator switch` Now it is a more realistic test case. This test used the same driver methods like the pre-configured cmake build folder test. This makes it difficult to detect pre-configured build folders. The cmake-tool class(extension front-end) use a different way to switch the kits on a running cmake driver instance used. - Add cmake configuration fail test to cms-driver. * Fix problem with invalid active kit * Add driver code model interface - Create driver code model interfac The cms client class has a code model. This code model is defined by the cmake cmake-server(7). The cmake-tools-extension needs a code model, which does not have to be identical to the cmake server. For more flexibility in future, a separate model is created. The first version contains all information required for the extension (e.g. for cpptools, code model tree) and is based on the cmake-server(7) model. - Separate code mode driver support from CMS driver version * Add first code model test and fix typos and a small bug * Fix unix build by separate test for single configuration generators * Fix node.js version problem with streamfilter * Add tests for project and target information * Fix linux build and reformat code model tests * Extend tests for shared library and utilities - Add tests for interface parts of cpptools - Add tests for utilities parts of tree view - Extend CMake test Project by required calls * Fix missing isGenerated in cms-client * Fix wrong library filename * Fix MacOS build and disable windows test for mingw (#775) * Fix regex * Update codemodel driver interface .documentation * Clean up tests and add sysroot test Clean up - Add helper for code model generation - Restructure test - Fix spelling Add missing sysroot test * Fix instable tests - Different CMake versions generate different code model outputs. - macOS test fails. I am unable to test the problem, but it seems the sysroot variable is required to build on macOS. * Remove test filter * Refactor test suite generation for driver test This modification allows reusing driver tests for all driver types. The tests check information and behavior to be according to the driver interface. We know then that the CMake tools frontend see the same behavior from the driver. * Added execution of driver tests to FileAPI Driver * Clean up description and reformat code * Update driver creation with new configuration setting * Implement sysroot mapping for FileApi * Fix lint and style * Review comments - Move setting description - Replace cmake version for CI tests - Move convertion of `useCMakeServer` to `cmakeCommunicationMode` - Create missing interface types - Search for newest index file * Insert review comments - Update setting behavior for automatic mode - Fix spelling - Remove not required query content - Add version check for code model and cache (index file has no version) * Fix merge conflicts * Change code model load behavior on init of driver * Insert localization to warnings.
|
the kits can test is failing single-root-UI test as well (The other tests are updated with the new API and working). This specific test was being ignored for a period of time, so it never ran to be failed (we were showing outputs only when a test failed, now there will be some outputs to make sure we are not skipping any test, we can revert to hiding outputs when we are sure all test are running by modifying this line). The tests under |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
The PR Review tests only on Travis. There are windows tests configured for AppVeyor. These tests are disabled, they are run slower. I execute them on my fork.
Since last merge the tests are instabile, last successful build.
Broken test:
Detect an MinGW compiler file on windows:
AssertionError: object tested must be an array, a map, an object, a set, a string, or a weakset, but undefined given
at Context.test (C:\projects\vscode-cmake-tools-6ek1x\test\unit-tests\kit-scan.test.ts:115:49)
The text was updated successfully, but these errors were encountered: