Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Framework: PR testing changed packages #3218

Merged
merged 10 commits into from
Aug 5, 2018
Merged

Framework: PR testing changed packages #3218

merged 10 commits into from
Aug 5, 2018

Conversation

william76
Copy link
Contributor

@william76 william76 commented Aug 1, 2018

@trilinos/framework

Description

Update to PR Autotester scripts to use the Tribits based changed-packages utility to create a packageEnables.cmake file.

Motivation and Context

The current auto-tester defaults to a more conservative model of selecting packages to enable in testing. This can result in long test times for trivial changes such as modifying some ctest drivers, etc. when we would not need to retest everything in that situation.

This update will reduce some of these testing times by using the Tribits/CMake infrastructure to more accurately determine which packages need to be built based on the files that are changed between develop and the source branch.

Relates to #3133

How Has This Been Tested?

I ran these tests on Jenkins to test the changes for the GCC 4.9.3 compiler:

  1. Tested a change to a file in Tpetra that kicked off testing.
  2. Tested a change to a file in cmake/ctest/drivers/dorksaber to test that the new script will not enable packages for testing.

@william76 william76 added the Framework tasks Framework tasks (used internally by Framework team) label Aug 1, 2018
@bartlettroscoe bartlettroscoe added the stage: in progress Work on the issue has started label Aug 1, 2018
@william76
Copy link
Contributor Author

The # --- BEGIN SCAFFOLDING --- ... # --- END SCAFFOLDING --- blocks are just left commented for now in case we need to do more testing. Once @trilinos/framework team approves us to merge in, I'll remove those sections and put in an update to this PR.

@william76 william76 changed the title Pr testing changed packages Framework: PR testing changed packages Aug 1, 2018
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request.

Copy link
Contributor

@mhoemmen mhoemmen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yo delete the "nurgleflurgle" we gotta be boring in Trilinos ;-)

@@ -134,6 +127,18 @@ then
echo "There was an issue loading the intel environment. The error code was: $ierror"
exit $ierror
fi
# --- BEGIN SCAFFOLDING ---
# nurgleflurgle... name match! can we change this to a regex or something to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"nurgleflurgle"?

elif [ "Trilinos_pullrequest_gcc_4.9.3" == "${JOB_BASE_NAME:?}" ]; then
CONFIG_SCRIPT=PullRequestLinuxGCC4.9.3TestingSettings.cmake
# --- BEGIN SCAFFOLDING ---
# nurgleflurgle... name match! can we change this to a regex or something to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"nurgleflurgle"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hah... as noted in my 1st comment -- all the stuff in scaffolding blocks will be removed before merging. This was just in there because the testing scripts do a name-match on the Jenkins job... once the framework team is ready to merge this in, I'll remove those comment blocks.

mhoemmen
mhoemmen previously approved these changes Aug 1, 2018
Copy link
Contributor

@mhoemmen mhoemmen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok "nurgleflurgle" is harmless never mind :)

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 1241
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 57be48f
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 935
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 57be48f
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

  • Build Num: 486
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 57be48f
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a

Using Repos:

Repo: TRILINOS (william76/Trilinos)
  • Branch: PRTesting-ChangedPackages
  • SHA: 57be48f
  • Mode: TEST_REPO

Pull Request Author: william76

@trilinos-autotester
Copy link
Contributor

NOTICE: The AutoTester has encountered an internal error (usually a Communications Timeout), testing will be restarted, previous tests may still be running but will be ignored by the AutoTester...

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 1242
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 57be48f
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 936
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 57be48f
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

  • Build Num: 487
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 57be48f
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a

Using Repos:

Repo: TRILINOS (william76/Trilinos)
  • Branch: PRTesting-ChangedPackages
  • SHA: 57be48f
  • Mode: TEST_REPO

Pull Request Author: william76

# - One of the SEMS modules will load CMake 3.4.x also,
# so this will pull in the SEMS cmake 3.10.3 version
# for Trilinos compatibility.
module load sems-cmake/3.10.3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Loaded CMake module twice.

jwillenbring
jwillenbring previously approved these changes Aug 1, 2018
Copy link
Member

@jwillenbring jwillenbring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just saw the one double module load. Other than that, these changes look good.

@william76 william76 dismissed stale reviews from jwillenbring and mhoemmen via 014ab60 August 1, 2018 18:26
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Error: Jenkins Jobs - A user has commited a change to the PR before testing completed. The original testing SHA = 57be48f Does not match the current commit SHA = 014ab60. The Jenkins Jobs will be shutdown; Testing of this PR must occur again.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED

Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.

Pull Request Auto Testing has FAILED (click to expand)

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 1242
  • Status: ERROR

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 57be48f
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 936
  • Status: ERROR

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 57be48f
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

  • Build Num: 487
  • Status: ERROR

Jenkins Parameters

Parameter Name Value
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 57be48f
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a


CDash Test Results for PR# 3218.

@william76
Copy link
Contributor Author

@jwillenbring removed the duplicate sems-cmake/3.10.3 load from the GCC 4.8.4 driver script.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 1244
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 014ab60
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 938
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 014ab60
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

  • Build Num: 489
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 014ab60
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a

Using Repos:

Repo: TRILINOS (william76/Trilinos)
  • Branch: PRTesting-ChangedPackages
  • SHA: 014ab60
  • Mode: TEST_REPO

Pull Request Author: william76

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_pullrequest_gcc_4.9.3

  • Build Num: 1244
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.9.3
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 014ab60
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a

Build Information

Test Name: Trilinos_pullrequest_gcc_4.8.4

  • Build Num: 938
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
COMPILER_MODULE sems-gcc/4.8.4
JENKINS_BUILD_TYPE Release
JENKINS_COMM_TYPE MPI
JENKINS_DO_COMPLEX OFF
JENKINS_JOB_TYPE Experimental
MPI_MODULE sems-openmpi/1.8.7
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 014ab60
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a

Build Information

Test Name: Trilinos_pullrequest_intel_17.0.1

  • Build Num: 489
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
PULLREQUESTNUM 3218
TEST_REPO_ALIAS TRILINOS
TRILINOS_SOURCE_BRANCH PRTesting-ChangedPackages
TRILINOS_SOURCE_REPO https://github.com/william76/Trilinos
TRILINOS_SOURCE_SHA 014ab60
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 5059d8a


CDash Test Results for PR# 3218.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
THE LAST COMMIT TO THIS PULL REQUEST HAS BEEN REVIEWED, BUT NOT ACCEPTED OR REQUIRES CHANGES

@trilinos-autotester
Copy link
Contributor

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ mhoemmen ]!

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - Master Automerge is disabled (in .cfg file)

Copy link
Member

@jwillenbring jwillenbring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks for making the minor change.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - Master Automerge is disabled (in .cfg file)

2 similar comments
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - Master Automerge is disabled (in .cfg file)

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - Master Automerge is disabled (in .cfg file)

@mhoemmen mhoemmen merged commit c4c4c13 into trilinos:develop Aug 5, 2018
@bartlettroscoe bartlettroscoe removed the stage: in progress Work on the issue has started label Aug 5, 2018
@william76
Copy link
Contributor Author

@mhoemmen why did you merge this in!?

From my comments above:

hah... as noted in my 1st comment -- all the stuff in scaffolding blocks will be removed before
merging. This was just in there because the testing scripts do a name-match on the Jenkins
job... once the framework team is ready to merge this in, I'll remove those comment blocks.

In this case, the intent of the framework team, since 3 of us were gone late last week, was to review this PR on Monday and decide to merge it in after our stand up meeting if we liked it, and after I pulled the scaffolding comments out first. We wanted the PR in so we could look at it and review.

In the future, please hold back on merging Framework team PR's since we generally have a specific timing in mind for when we click the button to make sure that we're around and paying close attention to testing, etc. in case things woof. ;)

@mhoemmen
Copy link
Contributor

mhoemmen commented Aug 6, 2018

@william76 Sorry, I was on autopilot. I'll let y'all do this next time. My bad :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Framework tasks Framework tasks (used internally by Framework team)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants