Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix included nested model expansion in SDF generation #768

Merged
merged 12 commits into from
May 11, 2021

Conversation

jennuine
Copy link
Contributor

@jennuine jennuine commented Apr 15, 2021

🦟 Bug fix

Part of #479

Summary

This PR resolves the issue of included nested models (e.g., //model/include) being expanded even though the option "Expand include tags" is disabled.

Before PR

  1. Load model_nested_include.sdf in Ignition
  2. Click on the top left menu > Save world as...
  3. Save the world (with "Expand include tags" disabled)
  4. Open the saved sdf file and see the included models are expanded

After PR
Run the above steps again except the saved sdf file will have //include tags instead

Checklist

  • Signed all commits for DCO
  • Added tests
  • Updated documentation (as needed)
  • Updated migration guide (as needed)
  • codecheck passed (See contributing)
  • All tests passed (See test coverage)
  • While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>
Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>
@jennuine jennuine added bug Something isn't working 🏢 edifice Ignition Edifice labels Apr 15, 2021
@jennuine jennuine requested a review from azeey April 15, 2021 00:26
@jennuine jennuine requested a review from chapulina as a code owner April 15, 2021 00:26
@jennuine jennuine self-assigned this Apr 15, 2021
@github-actions github-actions bot added the 🏯 fortress Ignition Fortress label Apr 15, 2021
Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>
@codecov
Copy link

codecov bot commented Apr 15, 2021

Codecov Report

Merging #768 (73b7110) into ign-gazebo5 (d3ee64a) will increase coverage by 0.06%.
The diff coverage is 72.72%.

❗ Current head 73b7110 differs from pull request most recent head 32dd57a. Consider uploading reports for the commit 32dd57a to get more accurate results
Impacted file tree graph

@@               Coverage Diff               @@
##           ign-gazebo5     #768      +/-   ##
===============================================
+ Coverage        65.32%   65.39%   +0.06%     
===============================================
  Files              240      240              
  Lines            17624    17679      +55     
===============================================
+ Hits             11513    11561      +48     
- Misses            6111     6118       +7     
Impacted Files Coverage Δ
.../plugins/component_inspector/ComponentInspector.cc 7.04% <0.00%> (-0.17%) ⬇️
src/rendering/RenderUtil.cc 42.10% <0.00%> (-0.05%) ⬇️
src/systems/physics/Physics.cc 69.44% <83.01%> (ø)
src/SdfGenerator.cc 94.77% <89.28%> (-1.23%) ⬇️
src/systems/physics/EntityFeatureMap.hh 93.65% <90.90%> (ø)
src/systems/diff_drive/DiffDrive.cc 86.91% <100.00%> (+0.94%) ⬆️
src/Server.cc 83.43% <0.00%> (+0.63%) ⬆️
src/SimulationRunner.cc 93.77% <0.00%> (+1.03%) ⬆️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a9cfa0d...32dd57a. Read the comment docs.

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>
Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>
@chapulina chapulina removed the 🏯 fortress Ignition Fortress label Apr 19, 2021
ASSERT_NE(nullptr, uri);
ASSERT_NE(nullptr, uri->GetText());
EXPECT_EQ(
"https://fuel.ignitionrobotics.org/1.0/OpenRobotics/models/Coke Can",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Saving the Fuel version number doesn't seem to work for me. Can you add a test to verify?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I modified the previous test, what do you think? 57d5833

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>
Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>
if (_saveFuelVersion && uriMapIt != _includeUriMap.end())
{
// find fuel model version from file path
std::string version = common::parentPath(e->FilePath());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this is the same as modelDir.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 73b7110

if (isNumber(version))
{
sdf::ElementPtr uriElem = e->GetIncludeElement()->GetElement("uri");
std::string uri = uriElem->GetValue()->GetAsString();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tip: This can be simplified as e->GetIncludeElement()->Get<std::string>("uri")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines 481 to 482
_elem->InsertElement(e->GetIncludeElement());
e->RemoveFromParent();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed another thing. This changes the order of models. I'm testing with model_nested_include.sdf and the original has:

      <model name="M2">
        <include>
          <uri>sphere</uri>
          <pose>0 2 2 0 0 0</pose>
        </include>

        <model name="M3">
          <include>
            <name>coke</name>
            <uri>https://fuel.ignitionrobotics.org/1.0/OpenRobotics/models/Coke Can</uri>
            <pose>2 2 2 0 0 0</pose>
          </include>
        </model>
      </model>

The saved file has

      <model name='M2'>
        <model name='M3'>
          <include>
            <name>coke</name>
            <uri>https://fuel.ignitionrobotics.org/1.0/OpenRobotics/models/Coke Can</uri>
            <pose>2 2 2 0 -0 0</pose>
          </include>
        </model>
        <include>
          <uri>sphere</uri>
          <pose>0 2 2 0 -0 0</pose>
        </include>
      </model>

(with <plugin> removed). The order in the original is sphere, M3, whereas in the saved file M3,sphere. The two files will have different results when sdf::Model::ModelByIndex is called. I think can preserve the order if we do this:

Suggested change
_elem->InsertElement(e->GetIncludeElement());
e->RemoveFromParent();
e ->Copy(e->GetIncludeElement());

This is the first time I've used sdf::Element::Copy and I now see its usefulness.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for noticing that! 73b7110

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm now getting the `//include/@name attribute set.

      <include name='include_nested_new_name'>
        <uri>include_nested</uri>
        <name>include_nested_new_name</name>
        <pose>1 2 3 0 -0 0</pose>
      </include>

I believe it's because e is a model and has a name attribute and sdf::Element::Copy doesn't seem to be clearing the attributes when copying. I think this is something we should fix in libsdformat. Since I don't know the reason for why attributes are not cleared in https://github.com/osrf/sdformat/blob/25004f0bad227cf11854cc208e74be08e8623359/src/Element.cc#L218-L220, I think it might be best to create another function to clear attributes instead of modifying sdf::Element::Copy.

Copy link
Contributor Author

@jennuine jennuine Apr 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we create both a sdf::Element::RemoveAllAttributes() as well as sdf::Element::RemoveAttribute(string key)? Just to have both options for users

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that sounds good to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the PR that adds these methods: gazebosim/sdformat#555

This will be forward ported to dome & edifice once merged

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new attribute removal methods have been ported to sdf11 and with b5a521c the attributes are cleared before copying the include element

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>
Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>
Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>
@jennuine jennuine added the needs upstream release Blocked by a release of an upstream library label Apr 30, 2021
@jennuine jennuine mentioned this pull request Apr 30, 2021
7 tasks
@jennuine jennuine removed the needs upstream release Blocked by a release of an upstream library label May 3, 2021
@jennuine
Copy link
Contributor Author

jennuine commented May 3, 2021

@osrf-jenkins run tests please

@jennuine jennuine added the needs upstream release Blocked by a release of an upstream library label May 3, 2021
@jennuine
Copy link
Contributor Author

jennuine commented May 3, 2021

Requires #758 to fix macOS build failure

@jennuine jennuine removed the needs upstream release Blocked by a release of an upstream library label May 5, 2021
@jennuine
Copy link
Contributor Author

jennuine commented May 5, 2021

@osrf-jenkins run tests please

Copy link
Contributor

@azeey azeey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Thanks for iterating.

@jennuine jennuine merged commit dca877a into ign-gazebo5 May 11, 2021
@jennuine jennuine deleted the jennuine/sdf_gen branch May 11, 2021 22:49
scpeters added a commit that referenced this pull request May 19, 2021
* 🎈 3.8.0 (#688)

Signed-off-by: Louise Poubel <louise@openrobotics.org>

* Make it so joint state publisher is quieter (#696)

Signed-off-by: Michael Carroll <michael@openrobotics.org>

* [BULLET] Making GetContactsFromLastStepFeature optional in Collision Features (#690)

* GetContactsFromLastStepFeature made optional

Signed-off-by: Tomas Lorente <jtlorente@ekumenlabs.com>

Co-authored-by: Addisu Z. Taddese <addisu@openrobotics.org>

* Add test for thermal object temperatures below 0 kelvin (#621)

Signed-off-by: Ashton Larkin <ashton@openrobotics.org>

* Scenebroadcaster sensors (#698)

* Add sensors to scene broadcaster

Signed-off-by: Nate Koenig <nate@openrobotics.org>

* Update src/systems/scene_broadcaster/SceneBroadcaster.cc

Co-authored-by: Michael Carroll <michael@openrobotics.org>

* Fix codecheck

Signed-off-by: Nate Koenig <nate@openrobotics.org>

Co-authored-by: Nate Koenig <nate@openrobotics.org>
Co-authored-by: Michael Carroll <michael@openrobotics.org>

* Fix diffuse and ambient values for ackermann example (#707)

Signed-off-by: Ammaar Solkar <asketch8@gmail.com>

* 🎈 5.0.0 (#731)

Signed-off-by: Louise Poubel <louise@openrobotics.org>

* Support configuring particle scatter ratio in particle emitter system (#674)

* set particle scatter ratio through sdf

Signed-off-by: Ian Chen <ichen@osrfoundation.org>

* address feedback

Signed-off-by: Ian Chen <ichen@osrfoundation.org>

* add todo note about merging forward

Signed-off-by: Ian Chen <ichen@osrfoundation.org>

Co-authored-by: Ashton Larkin <42042756+adlarkin@users.noreply.github.com>

* Update PlaybackScrubber description (#733)

Signed-off-by: Ammaar Solkar <asketch8@gmail.com>

* Iterate through changed links only in UpdateSim (#678)

Signed-off-by: Ashton Larkin <ashton@openrobotics.org>

* Do not pass -Wno-unused-parameter to MSVC compiler (#716)

Signed-off-by: Silvio Traversaro <silvio.traversaro@iit.it>

* Use Protobuf_IMPORT_DIRS instead of PROTOBUF_IMPORT_DIRS for compatibility with Protobuf CMake config (#715)

Signed-off-by: Silvio Traversaro <silvio.traversaro@iit.it>

* Fix component inspector shutdown crash (#724)

Signed-off-by: Louise Poubel <louise@openrobotics.org>

Co-authored-by: Alejandro Hernández Cordero <ahcorde@gmail.com>

* Validate step size and RTF parameters (#740)

Only set them if they are strictly positive.

Signed-off-by: Luca Della Vedova <luca@openrobotics.org>

* Fix compute_rtfs arguments (#737)

Signed-off-by: Caio Amaral <caioaamaral@gmail.com>

* Fixed collision visual bounding boxes (#746)

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>

* Fix CMakelists.txt merge

Signed-off-by: Nate Koenig <nate@openrobotics.org>

* ECM's ChangedState gets message with modified components (#742)

* ecm's ChangedState to contain modified components

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>

* updated log_system test

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>

* removed unnecessary calls

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>

Co-authored-by: Ian Chen <ichen@osrfoundation.org>

* fixed particle emitter forward playback (#745)

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>

* Merge pull request #730 from ignitionrobotics/particle_emitter

Particle emitter based on SDF

* 4 7 0 prep (#755)

* Prepare for 4.7.0

Signed-off-by: Nate Koenig <nate@openrobotics.org>

* Added placeholder

Signed-off-by: Nate Koenig <nate@openrobotics.org>

Co-authored-by: Nate Koenig <nate@openrobotics.org>

* Fix 'invalid animation update data' msg for actors (#754)

Signed-off-by: Ashton Larkin <ashton@openrobotics.org>

* Update benchmark comparison instructions (#766) (#766)

Signed-off-by: Ashton Larkin <ashton@openrobotics.org>

* [DiffDrive] add enable/disable (#772)

* add enable/disable diffdrive

Signed-off-by: Guillaume Doisy <guillaume.doisy@wyca.fr>

* remove debug

Signed-off-by: Guillaume Doisy <guillaume.doisy@wyca.fr>

* do not subscribe to enable if topic is empty

Signed-off-by: Guillaume Doisy <guillaume.doisy@wyca.fr>

* add test

Signed-off-by: Guillaume Doisy <guillaume.doisy@wyca.fr>

* lint and style

Signed-off-by: Guillaume Doisy <guillaume.doisy@wyca.fr>

* change enable type to bool and renamed to enabled

Signed-off-by: Guillaume Doisy <guillaume.doisy@wyca.fr>

* Add odometry publisher system (#547)

* Create Initial Odometry Publisher system plugin

Add code for initial plugin that gets position from Pose component and
calculates velocities based on rolling mean from displacement data.

Signed-off-by: Maganty Rushyendra <mrushyendra@yahoo.com.sg>

* Remove Linear and Angular Velocity components

Also renames frames in Odometry msg to include model name, and makes
various style changes.

Signed-off-by: Maganty Rushyendra <mrushyendra@yahoo.com.sg>

* Get World pose instead of pose of robot base frame

Signed-off-by: Maganty Rushyendra <mrushyendra@yahoo.com.sg>

* Add documentation for variables and functions

Includes minor stylistic changes.

Signed-off-by: Maganty Rushyendra <mrushyendra@yahoo.com.sg>

* Check for valid odomTopic and update copyright year

Signed-off-by: Maganty Rushyendra <mrushyendra@yahoo.com.sg>

* Add tests for OdometryPublisherSystem and fix velocity calculation bug

Swap X and Y linear velocities when calculating odometry velocities
relative to robotBaseFrame.

Signed-off-by: Maganty Rushyendra <mrushyendra@yahoo.com.sg>

Co-authored-by: ahcorde <ahcorde@gmail.com>

* Patch particle emitter2 service (#777)

* Patch particle emitter2 service

Signed-off-by: Nate Koenig <nate@openrobotics.org>

* Remove condition variable

Signed-off-by: Nate Koenig <nate@openrobotics.org>

* Set emitter frame and relative pose

Signed-off-by: Nate Koenig <nate@openrobotics.org>

Co-authored-by: Nate Koenig <nate@openrobotics.org>

* Preparing for 4.8.0 release (#780)

Signed-off-by: Nate Koenig <nate@openrobotics.org>

Co-authored-by: Nate Koenig <nate@openrobotics.org>

* 👩‍🌾 Enable Focal CI (#646)

Signed-off-by: Louise Poubel <louise@openrobotics.org>

Co-authored-by: Michael Carroll <michael@openrobotics.org>

* [TPE] Support setting individual link velocity  (#427)

Signed-off-by: claireyywang <22240514+claireyywang@users.noreply.github.com>
Signed-off-by: Ian Chen <ichen@osrfoundation.org>

Co-authored-by: Ian Chen <ichen@osrfoundation.org>
Co-authored-by: Louise Poubel <louise@openrobotics.org>

* Don't store duplicate ComponentTypeId in ECM (#751)

Signed-off-by: Louise Poubel <louise@openrobotics.org>

* Feature/hydrodynamics (#749)

Implement hydrodynamics and thruster plugin.

Signed-off-by: Arjo Chakravarty <arjo@openrobotics.org>
Co-authored-by: Mabel Zhang <mabel@openrobotics.org>
Co-authored-by: Carlos Agüero <caguero@openrobotics.org>

* Fix macOS build: components::Name in benchmark (#784)

Signed-off-by: Louise Poubel <louise@openrobotics.org>

Co-authored-by: Steve Peters <scpeters@openrobotics.org>

* Fix ColladaExporter submesh index bug (#763)

Signed-off-by: Jorge Perez <jjperez@ekumenlabs.com>

* 👩‍🌾 Fix Windows build and some warnings (#782)

Signed-off-by: Louise Poubel <louise@openrobotics.org>

Co-authored-by: Alejandro Hernández Cordero <ahcorde@gmail.com>

* Prevent crash on Plotting plugin with mutex (#747)

Signed-off-by: Louise Poubel <louise@openrobotics.org>

* Bump ign-physics version to 3.2 (#792)

Signed-off-by: Louise Poubel <louise@openrobotics.org>

* Bump to ign-msgs 7.1 / sdformat 11.1, Windows fixes (#758)

Signed-off-by: Louise Poubel <louise@openrobotics.org>

* Util: Use public API from libsdformat for detecting non-file source (#794)

Signed-off-by: Eric Cousineau <eric.cousineau@tri.global>

* Fix included nested model expansion in SDF generation (#768)

* fixed included nested model expansion

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>

* added resource path to test

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>

* use orig URIs & support multi level nesting

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>

* save fuel version when enabled

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>

* retrieve uri from map

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>

* copy included element

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>

* clear attributes before copying include element

Signed-off-by: Jenn Nguyen <jenn@openrobotics.org>

* Map canonical links to their models (#736)

Signed-off-by: Ashton Larkin <ashton@openrobotics.org>

* ColladaExporter, export submesh selected (#802)

* Export only submesh if selected
* Add test case for the PR
* Attempting a unified solution

Signed-off-by: Jorge Perez <jjperez@ekumenlabs.com>
Co-authored-by: Nate Koenig <nate@openrobotics.org>

Co-authored-by: Michael Carroll <michael@openrobotics.org>
Co-authored-by: Jose Tomas Lorente <jtlorente@ekumenlabs.com>
Co-authored-by: Addisu Z. Taddese <addisu@openrobotics.org>
Co-authored-by: Ashton Larkin <ashton@openrobotics.org>
Co-authored-by: Nate Koenig <nkoenig@users.noreply.github.com>
Co-authored-by: Nate Koenig <nate@openrobotics.org>
Co-authored-by: Ammaar Solkar <asketch8@gmail.com>
Co-authored-by: Ian Chen <ichen@osrfoundation.org>
Co-authored-by: Ashton Larkin <42042756+adlarkin@users.noreply.github.com>
Co-authored-by: Silvio Traversaro <pegua1@gmail.com>
Co-authored-by: Alejandro Hernández Cordero <ahcorde@gmail.com>
Co-authored-by: Luca Della Vedova <luca@openrobotics.org>
Co-authored-by: Caio Amaral <caioaamaral@gmail.com>
Co-authored-by: Jenn Nguyen <jenn@openrobotics.org>
Co-authored-by: G.Doisy <doisyg@users.noreply.github.com>
Co-authored-by: Rushyendra Maganty <mrushyendra@yahoo.com.sg>
Co-authored-by: Claire Wang <22240514+claireyywang@users.noreply.github.com>
Co-authored-by: Arjo Chakravarty <arjo129@gmail.com>
Co-authored-by: Mabel Zhang <mabel@openrobotics.org>
Co-authored-by: Carlos Agüero <caguero@openrobotics.org>
Co-authored-by: Steve Peters <scpeters@openrobotics.org>
Co-authored-by: Jorge Perez <j.j.perez13@hotmail.com>
Co-authored-by: Eric Cousineau <eric.cousineau@tri.global>
Co-authored-by: Jorge Perez <jjperez@ekumenlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working 🏢 edifice Ignition Edifice
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants