Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

invoke colcon instead of ament_tools #132

Merged
merged 15 commits into from
Apr 21, 2018
Merged

invoke colcon instead of ament_tools #132

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
b6742d1
invoke colcon instead of ament_tools
dirk-thomas Mar 15, 2018
303ffac
remove obsolete escaping logic to deal with trailing double dashes
dirk-thomas Apr 11, 2018
d6c25e0
update package commands
dirk-thomas Apr 16, 2018
ae3602e
fix --ros1-path position
dirk-thomas Apr 16, 2018
1482d88
test results
dirk-thomas Apr 17, 2018
865872b
display console output for colcon tests as well (#150)
mikaelarguedas Apr 18, 2018
47bf9cd
remove obsolete comment
dirk-thomas Apr 18, 2018
28ff9e6
add cmake args into existing argument if it exists
dirk-thomas Apr 19, 2018
6dc14da
remove ament from (almost) everywhere
dirk-thomas Apr 19, 2018
caf7aa9
linter argument
dirk-thomas Apr 20, 2018
d36d993
run test-result twice, once with / once without --all
dirk-thomas Apr 20, 2018
605115c
place event handlers before (#153)
mikaelarguedas Apr 20, 2018
df76f5d
use -j1 for bridge
dirk-thomas Apr 20, 2018
4fe9cfd
determine colcon path on Windows
dirk-thomas Apr 21, 2018
78d6cab
unify section naming
dirk-thomas Apr 21, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions create_jenkins_job.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,8 @@ def main(argv=None):
'use_fastrtps_default': 'true',
'use_opensplice_default': 'false',
'use_isolated_default': 'true',
'ament_build_args_default': '--parallel --cmake-args -DSECURITY=ON --',
'ament_test_args_default': '--retest-until-pass 10',
'build_args_default': '--event-handler console_cohesion+ --cmake-args " -DSECURITY=ON"',
'test_args_default': '--event-handler console_cohesion+ --retest-until-pass 10',
'enable_c_coverage_default': 'false',
'dont_notify_every_unstable_build': 'false',
'turtlebot_demo': False,
Expand Down Expand Up @@ -190,7 +190,7 @@ def create_job(os_name, job_name, template_file, additional_dict):
'cmake_build_type': 'None',
'time_trigger_spec': PERIODIC_JOB_SPEC,
'mailer_recipients': DEFAULT_MAIL_RECIPIENTS,
'ament_test_args_default': '--retest-until-fail 20 --ctest-args -LE linter --',
'test_args_default': '--retest-until-fail 20 --ctest-args " -LE" linter',
})

# configure turtlebot jobs on Linux only for now
Expand Down
80 changes: 14 additions & 66 deletions job_templates/ci_job.xml.em
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@
use_opensplice_default=use_opensplice_default,
use_isolated_default=use_isolated_default,
cmake_build_type=cmake_build_type,
ament_build_args_default=ament_build_args_default,
ament_test_args_default=ament_test_args_default,
build_args_default=build_args_default,
test_args_default=test_args_default,
enable_c_coverage_default=enable_c_coverage_default,
))@
</properties>
Expand Down Expand Up @@ -94,8 +94,8 @@ supplemental_repos_url: ${build.buildVariableResolver.resolve('CI_ROS2_SUPPLEMEN
use_whitespace: ${build.buildVariableResolver.resolve('CI_USE_WHITESPACE_IN_PATHS')}, <br/>
isolated: ${build.buildVariableResolver.resolve('CI_ISOLATED')}, <br/>
cmake_build_type: ${build.buildVariableResolver.resolve('CI_CMAKE_BUILD_TYPE')}, <br/>
ament_build_args: ${build.buildVariableResolver.resolve('CI_AMENT_BUILD_ARGS')}, <br/>
ament_test_args: ${build.buildVariableResolver.resolve('CI_AMENT_TEST_ARGS')}, <br/>
build_args: ${build.buildVariableResolver.resolve('CI_BUILD_ARGS')}, <br/>
test_args: ${build.buildVariableResolver.resolve('CI_TEST_ARGS')}, <br/>
coverage: ${build.buildVariableResolver.resolve('CI_ENABLE_C_COVERAGE')}\
""");]]>
</script>
Expand Down Expand Up @@ -150,43 +150,15 @@ fi
if [ "$CI_ENABLE_C_COVERAGE" = "true" ]; then
export CI_ARGS="$CI_ARGS --coverage"
fi
if [ -n "${CI_AMENT_BUILD_ARGS+x}" ]; then
ESCAPE=""
for arg in ${CI_AMENT_BUILD_ARGS}; do
# These are arguments that the user wants to pass through to ament.
# Before they get there, they will first be passed through and parsed by
# run_ros2_batch.py. In order to retain their original meaning, make
# sure to "escape" lone dash sequences that the user put onto the line.
# This ensures that these make it through to ament in the way the user intended.
case "${arg}" in
*-- )
arg="${arg}-"
;;
esac
ESCAPE="${ESCAPE}${arg} "
done
export CI_ARGS="${CI_ARGS} --ament-build-args ${ESCAPE%?} --"
fi
if [ -n "${CI_AMENT_TEST_ARGS+x}" ]; then
ESCAPE=""
for arg in ${CI_AMENT_TEST_ARGS}; do
# These are arguments that the user wants to pass through to ament.
# Before they get there, they will first be passed through and parsed by
# run_ros2_batch.py. In order to retain their original meaning, make
# sure to "escape" lone dash sequences that the user put onto the line.
# This ensures that these make it through to ament in the way the user intended.
case "${arg}" in
*-- )
arg="${arg}-"
;;
esac
ESCAPE="${ESCAPE}${arg} "
done
export CI_ARGS="${CI_ARGS} --ament-test-args ${ESCAPE%?} --"
fi
@[if os_name in ['linux', 'linux-aarch64'] and turtlebot_demo]@
export CI_ARGS="$CI_ARGS --ros1-path /opt/ros/kinetic"
@[end if]@
if [ -n "${CI_BUILD_ARGS+x}" ]; then
export CI_ARGS="$CI_ARGS --build-args $CI_BUILD_ARGS"
fi
if [ -n "${CI_TEST_ARGS+x}" ]; then
export CI_ARGS="$CI_ARGS --test-args $CI_TEST_ARGS"
fi
echo "Using args: $CI_ARGS"
echo "# END SECTION"

Expand Down Expand Up @@ -289,35 +261,11 @@ if "!CI_CMAKE_BUILD_TYPE!" == "Debug" (
if "!CI_ENABLE_C_COVERAGE!" == "true" (
set "CI_ARGS=!CI_ARGS! --coverage"
)
if "%CI_AMENT_BUILD_ARGS%" NEQ "" (
set ESCAPE=
set PARSER=%CI_AMENT_BUILD_ARGS%
:build-loop
for /f "tokens=1* delims= " %%a in ("!PARSER!") do (
set substring=%%a
if "!substring:~-2!" EQU "--" (
set substring=%%a-
)
set "ESCAPE=!ESCAPE!!substring! "
set PARSER=%%b
)
if defined PARSER goto build-loop
set "CI_ARGS=!CI_ARGS! --ament-build-args !ESCAPE:~0,-1! --"
if "!CI_BUILD_ARGS!" NEQ "" (
set "CI_ARGS=!CI_ARGS! --build-args !CI_BUILD_ARGS!"
)
if "%CI_AMENT_TEST_ARGS%" NEQ "" (
set ESCAPE=
set PARSER=%CI_AMENT_TEST_ARGS%
:test-loop
for /f "tokens=1* delims= " %%a in ("!PARSER!") do (
set substring=%%a
if "!substring:~-2!" EQU "--" (
set substring=%%a-
)
set "ESCAPE=!ESCAPE!!substring! "
set PARSER=%%b
)
if defined PARSER goto test-loop
set "CI_ARGS=!CI_ARGS! --ament-test-args !ESCAPE:~0,-1! --"
if "!CI_TEST_ARGS!" NEQ "" (
set "CI_ARGS=!CI_ARGS! --test-args !CI_TEST_ARGS!"
)
echo Using args: !CI_ARGS!
echo "# END SECTION"
Expand Down
4 changes: 2 additions & 2 deletions job_templates/ci_launcher_job.xml.em
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@
use_opensplice_default=use_opensplice_default,
use_isolated_default=use_isolated_default,
cmake_build_type=cmake_build_type,
ament_build_args_default=ament_build_args_default,
ament_test_args_default=ament_test_args_default,
build_args_default=build_args_default,
test_args_default=test_args_default,
enable_c_coverage_default=enable_c_coverage_default,
))@
</properties>
Expand Down
91 changes: 35 additions & 56 deletions job_templates/packaging_job.xml.em
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
default_repos_url=default_repos_url,
supplemental_repos_url=supplemental_repos_url,
cmake_build_type=cmake_build_type,
ament_build_args_default=ament_build_args_default,
build_args_default=build_args_default,
))@
<hudson.model.BooleanParameterDefinition>
<name>CI_USE_FASTRTPS</name>
Expand All @@ -41,9 +41,9 @@
<defaultValue>@(test_bridge_default)</defaultValue>
</hudson.model.BooleanParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>CI_AMENT_TEST_ARGS</name>
<description>Additional arguments passed to 'ament test' if testing the bridge.</description>
<defaultValue>@(ament_test_args_default)</defaultValue>
<name>CI_TEST_ARGS</name>
<description>Additional arguments passed to the 'test' verb if testing the bridge.</description>
<defaultValue>@(test_args_default)</defaultValue>
</hudson.model.StringParameterDefinition>
</parameterDefinitions>
</hudson.model.ParametersDefinitionProperty>
Expand Down Expand Up @@ -102,9 +102,9 @@ repos_url: ${build.buildVariableResolver.resolve('CI_ROS2_REPOS_URL')}, <br/>
use_fastrtps: ${build.buildVariableResolver.resolve('CI_USE_FASTRTPS')}, <br/>
use_opensplice: ${build.buildVariableResolver.resolve('CI_USE_OPENSPLICE')}, <br/>
cmake_build_type: ${build.buildVariableResolver.resolve('CI_CMAKE_BUILD_TYPE')}, <br/>
ament_build_args: ${build.buildVariableResolver.resolve('CI_AMENT_BUILD_ARGS')}, <br/>
build_args: ${build.buildVariableResolver.resolve('CI_BUILD_ARGS')}, <br/>
test_bridge: ${build.buildVariableResolver.resolve('CI_TEST_BRIDGE')}, <br/>
ament_test_args: ${build.buildVariableResolver.resolve('CI_AMENT_TEST_ARGS')}\
test_args: ${build.buildVariableResolver.resolve('CI_TEST_ARGS')}\
""");]]>
</script>
<sandbox>false</sandbox>
Expand Down Expand Up @@ -134,36 +134,20 @@ export CI_ARGS="$CI_ARGS --repo-file-url $CI_ROS2_REPOS_URL"
if [ "$CI_TEST_BRIDGE" = "true" ]; then
export CI_ARGS="$CI_ARGS --test-bridge"
fi
if [ "${CI_CMAKE_BUILD_TYPE}" != "None" ]; then
export CI_ARGS="$CI_ARGS --cmake-build-type $CI_CMAKE_BUILD_TYPE"
fi
if [ -n "${CI_AMENT_BUILD_ARGS+x}" ]; then
case $CI_AMENT_BUILD_ARGS in
*-- )
# delimiter is already appended
;;
* )
CI_AMENT_BUILD_ARGS="$CI_AMENT_BUILD_ARGS --"
;;
esac
export CI_ARGS="$CI_ARGS --ament-build-args $CI_AMENT_BUILD_ARGS"
fi
if [ -n "${CI_AMENT_TEST_ARGS+x}" ]; then
case $CI_AMENT_TEST_ARGS in
*-- )
# delimiter is already appended
;;
* )
CI_AMENT_TEST_ARGS="$CI_AMENT_TEST_ARGS --"
;;
esac
export CI_ARGS="$CI_ARGS --ament-test-args $CI_AMENT_TEST_ARGS"
fi
@[if os_name in ['linux', 'linux-aarch64']]@
export CI_ARGS="$CI_ARGS --ros1-path /opt/ros/kinetic"
@[else]@
export CI_ARGS="$CI_ARGS --ros1-path /Users/osrf/kinetic/install_isolated"
@[end if]@
if [ "${CI_CMAKE_BUILD_TYPE}" != "None" ]; then
export CI_ARGS="$CI_ARGS --cmake-build-type $CI_CMAKE_BUILD_TYPE"
fi
if [ -n "${CI_BUILD_ARGS+x}" ]; then
export CI_ARGS="$CI_ARGS --build-args $CI_BUILD_ARGS"
fi
if [ -n "${CI_TEST_ARGS+x}" ]; then
export CI_ARGS="$CI_ARGS --test-args $CI_TEST_ARGS"
fi
echo "Using args: $CI_ARGS"
echo "# END SECTION"

Expand Down Expand Up @@ -202,47 +186,42 @@ echo "# BEGIN SECTION: Run packaging script"
echo "# END SECTION"
@[end if]@
@[elif os_name == 'windows']@
setlocal enableDelayedExpansion
rmdir /S /Q ws workspace

echo "# BEGIN SECTION: Determine arguments"
set "PATH=%PATH:"=%"
set "PATH=!PATH:"=!"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Can you clarify why we need to use ! everywhere now ? and how this plays with the delayed expansion?

Thanks!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using % didn't work for me. I didn't spend too much effort trying to figure out why. Using ! made it work.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I was wondering if this didnt work before you enabled the delayed expansion but would have worked after enabling it.

That's fine, I'm not planning on looking into it to find out

set "CI_ARGS=--packaging --force-ansi-color"
if "%CI_BRANCH_TO_TEST%" NEQ "" (
set "CI_ARGS=%CI_ARGS% --test-branch %CI_BRANCH_TO_TEST%"
if "!CI_BRANCH_TO_TEST!" NEQ "" (
set "CI_ARGS=!CI_ARGS! --test-branch !CI_BRANCH_TO_TEST!"
)
if "%CI_USE_FASTRTPS%" == "true" (
set "CI_ARGS=%CI_ARGS% --fastrtps"
if "!CI_USE_FASTRTPS!" == "true" (
set "CI_ARGS=!CI_ARGS! --fastrtps"
)
if "%CI_USE_OPENSPLICE%" == "true" (
set "CI_ARGS=%CI_ARGS% --opensplice"
if "!CI_USE_OPENSPLICE!" == "true" (
set "CI_ARGS=!CI_ARGS! --opensplice"
)
if "%CI_ROS2_REPOS_URL%" EQU "" (
if "!CI_ROS2_REPOS_URL!" EQU "" (
set "CI_ROS2_REPOS_URL=@default_repos_url"
)
set "CI_ARGS=%CI_ARGS% --repo-file-url %CI_ROS2_REPOS_URL%"
if "%CI_TEST_BRIDGE%" == "true" (
set "CI_ARGS=%CI_ARGS% --test-bridge"
set "CI_ARGS=!CI_ARGS! --repo-file-url !CI_ROS2_REPOS_URL!"
if "!CI_TEST_BRIDGE!" == "true" (
set "CI_ARGS=!CI_ARGS! --test-bridge"
)
if "%CI_CMAKE_BUILD_TYPE%" NEQ "None" (
set "CI_ARGS=%CI_ARGS% --cmake-build-type %CI_CMAKE_BUILD_TYPE%"
if "!CI_CMAKE_BUILD_TYPE!" NEQ "None" (
set "CI_ARGS=!CI_ARGS! --cmake-build-type !CI_CMAKE_BUILD_TYPE!"
)
if "%CI_AMENT_BUILD_ARGS%" NEQ "" (
if "%CI_AMENT_BUILD_ARGS:~-2%" NEQ "--" (
set "CI_AMENT_BUILD_ARGS=%CI_AMENT_BUILD_ARGS% --"
)
set "CI_ARGS=%CI_ARGS% --ament-build-args %CI_AMENT_BUILD_ARGS%"
if "!CI_BUILD_ARGS!" NEQ "" (
set "CI_ARGS=!CI_ARGS! --build-args !CI_BUILD_ARGS!"
)
if "%CI_AMENT_TEST_ARGS%" NEQ "" (
if "%CI_AMENT_TEST_ARGS:~-2%" NEQ "--" (
set "CI_AMENT_TEST_ARGS=%CI_AMENT_TEST_ARGS% --"
)
set "CI_ARGS=%CI_ARGS% --ament-test-args %CI_AMENT_TEST_ARGS%"
if "!CI_TEST_ARGS!" NEQ "" (
set "CI_ARGS=!CI_ARGS! --test-args !CI_TEST_ARGS!"
)
echo Using args: %CI_ARGS%
echo Using args: !CI_ARGS!
echo "# END SECTION"

echo "# BEGIN SECTION: Run packaging script"
python -u run_ros2_batch.py %CI_ARGS%
python -u run_ros2_batch.py !CI_ARGS!
echo "# END SECTION"
@[else]@
@{ assert False, 'Unknown os_name: ' + os_name }@
Expand Down
8 changes: 4 additions & 4 deletions job_templates/snippet/property_parameter-definition.xml.em
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
default_repos_url=default_repos_url,
supplemental_repos_url=supplemental_repos_url,
cmake_build_type=cmake_build_type,
ament_build_args_default=ament_build_args_default,
build_args_default=build_args_default,
))@
<hudson.model.BooleanParameterDefinition>
<name>CI_USE_WHITESPACE_IN_PATHS</name>
Expand Down Expand Up @@ -56,9 +56,9 @@ This tests the robustness to whitespace being within the different paths.</descr
<defaultValue>@(enable_c_coverage_default)</defaultValue>
</hudson.model.BooleanParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>CI_AMENT_TEST_ARGS</name>
<description>Additional arguments passed to 'ament test'.</description>
<defaultValue>@(ament_test_args_default)</defaultValue>
<name>CI_TEST_ARGS</name>
<description>Additional arguments passed to the 'test' verb.</description>
<defaultValue>@(test_args_default)</defaultValue>
</hudson.model.StringParameterDefinition>
</parameterDefinitions>
</hudson.model.ParametersDefinitionProperty>
6 changes: 3 additions & 3 deletions job_templates/snippet/property_parameter-definition_common.xml.em
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ choices.remove(cmake_build_type)
</choices>
</hudson.model.ChoiceParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>CI_AMENT_BUILD_ARGS</name>
<description>Additional arguments passed to 'ament build'.</description>
<defaultValue>@(ament_build_args_default)</defaultValue>
<name>CI_BUILD_ARGS</name>
<description>Additional arguments passed to the 'build' verb.</description>
<defaultValue>@(build_args_default)</defaultValue>
</hudson.model.StringParameterDefinition>
Loading