Fix rendering of OGRE meshes with materials and textures (ros-pkg ticket #5421)

[hershwg]

OGRE meshes as robot parts (opposed to collada) are not rendered correctly but all red with no shading or texture applied. The problem is that their .material files and textures are only read from one global path, "rviz/ogre_media". This requires copying the materials of all robot parts to this global location (undesirable or even impossible as a regular user with a system-wide installed ROS).

Gazebo fixed this by allowing packages to export resource lookup paths. Two different patches can be applied to RViz to fix the behavior and correctly render OGRE meshes, attached below (last tested with electric released version, patches by Stefan Osswald, University of Freiburg).

The patches are relatively straight-forward, local, and don't affect any other behavior so it would be great to get one of them in for the Fuerte release.

trac data:

• Owner: hersh
• Reporter: ahornung
• Reported at: Wed Apr 11 11:20:10 2012
• URL: http://code.ros.org/trac/ros-pkg/ticket/5421

[hershwg]

[ahornung] Both patches extend the OGRE resource package path.

1 implements the same behavior as Gazebo plugins: packages can export their resource path with

  <depend packge="rviz"/>
  <export>
    <rviz rviz_media_path="${prefix}/Media/materials/scripts:${prefix}/Media/textures"/>
  </export>

This requires to depend on rviz (and thus rx) due to the plugin lookup implementation. For URDF description packages, this pulls in unwanted dependencies.

2 on the other hand relies on a special tag in the URDF which specifies resource paths for all meshes:

<robot xmlns:xacro="http://ros.org/wiki/xacro">
  <resource dirname="package://robot_description/Media/textures"/>
  <resource dirname="package://robot_description/Media/materials/scripts"/>
...
</robot>

This avoids the problems of #1, at the "cost" of an additional URDF tag.

No idea what's the best solution, but two looks cleaner to me.

[hershwg]

[ahornung] Forgot to add a downside of #2: It only works for robot meshes from a URDF, but not for visualization_msgs::Marker::MESH_RESOURCE. Maybe both patches make sense to be applied, one for robot meshes and one for "true" rviz extensions which then need to depend on it?

[ahornung]

Any update on this? I guess it will never be merged into Fuerte, but what about Hydro?

The patches did not seem to get converted properly from code.ros.org, I'll post them below.

Patch No. 1:

===================================================================
--- src/rviz/robot/robot.cpp (Revision 38598)
+++ src/rviz/robot/robot.cpp (Arbeitskopie)
@@ -48,6 +48,7 @@
 #include <OGRE/OgreResourceGroupManager.h>

 #include <ros/console.h>
+#include <ros/package.h>

 namespace rviz
 {
@@ -195,6 +196,29 @@
     links_category_ = property_manager_->createCategory( "Links", name_, parent_property_, this );
   }

+  // set rviz media paths by adding all packages that exports "rviz_media_path" for rviz
+  std::vector<std::string> rviz_media_paths;
+  ros::package::getPlugins("rviz","rviz_media_path",rviz_media_paths);
+  std::string delim(":");
+  for (std::vector<std::string>::iterator iter=rviz_media_paths.begin(); iter != rviz_media_paths.end(); iter++)
+  {
+    if (!iter->empty()) 
+    {
+      int pos1 = 0;
+      int pos2 = iter->find(delim);
+      while (pos2 != (int)std::string::npos)
+      {
+        Ogre::ResourceGroupManager::getSingleton().addResourceLocation(iter->substr(pos1,pos2-pos1), "FileSystem", ROS_PACKAGE_NAME);
+        pos1 = pos2+1;
+        pos2 = iter->find(delim,pos2+1);
+      }
+      Ogre::ResourceGroupManager::getSingleton().addResourceLocation(iter->substr(pos1,iter->size()-pos1), "FileSystem", ROS_PACKAGE_NAME);
+    }
+  }
+
+  Ogre::ResourceGroupManager::getSingleton().initialiseAllResourceGroups();
+
+
   typedef std::vector<boost::shared_ptr<urdf::Link> > V_Link;
   V_Link links;
   descr.getLinks(links);

This allows packages to export their resource path with

  <depend packge="rviz"/>
  <export>
    <rviz rviz_media_path="${prefix}/Media/materials/scripts:${prefix}/Media/textures"/>
  </export>

...but requires to depend on rviz (and thus rx) due to the plugin lookup implementation. For URDF description packages, this pulls in unwanted dependencies.

Patch No. 2:

Index: src/rviz/robot/robot.cpp
===================================================================
--- src/rviz/robot/robot.cpp (Revision 38598)
+++ src/rviz/robot/robot.cpp (Arbeitskopie)
@@ -48,6 +48,7 @@
 #include <OGRE/OgreResourceGroupManager.h>

 #include <ros/console.h>
+#include <ros/package.h>

 namespace rviz
 {
@@ -195,6 +196,43 @@
     links_category_ = property_manager_->createCategory( "Links", name_, parent_property_, this );
   }

+  for (TiXmlElement *n = root_element->FirstChildElement("resource"); n != NULL; n = n->NextSiblingElement("resource")) {
+        const std::string *filename = n->Attribute(std::string("dirname"));
+        if (filename) {
+            std::string fullname = *filename;
+              if (fullname.find("package://") == 0)
+              {
+                fullname.erase(0, strlen("package://"));
+                size_t pos = fullname.find("/");
+                if (pos == std::string::npos)
+                {
+                  ROS_FATAL("Could not parse package:// format for [%s]",filename->c_str());
+                }
+
+                std::string package = fullname.substr(0, pos);
+                fullname.erase(0, pos);
+                std::string package_path = ros::package::getPath(package);
+
+                if (package_path.empty())
+                {
+                  ROS_FATAL("%s Package[%s] does not exist",filename->c_str(),package.c_str());
+                  continue;
+                }
+
+                fullname = package_path + fullname;
+              } else if (fullname.find("file://") == 0) {
+                fullname.erase(0, strlen("file://"));
+              }
+            ROS_INFO("Adding resource location %s", fullname.c_str());
+            Ogre::ResourceGroupManager::getSingleton().addResourceLocation(fullname, "FileSystem", ROS_PACKAGE_NAME);
+        } else {
+            ROS_ERROR("Resource node: dirname attribute missing");
+        }
+    }
+
+  Ogre::ResourceGroupManager::getSingleton().initialiseAllResourceGroups();
+
+
   typedef std::vector<boost::shared_ptr<urdf::Link> > V_Link;
   V_Link links;
   descr.getLinks(links);

This relies on a special tag in the URDF which specifies resource paths for all meshes:

<robot xmlns:xacro="http://ros.org/wiki/xacro">
  <resource dirname="package://robot_description/Media/textures"/>
  <resource dirname="package://robot_description/Media/materials/scripts"/>
...
</robot>

Looks a lot cleaner, but has the "cost" of an additional URDF tag.

[hershwg]

Sorry for ignoring this for a year and a half, I agree it looks important to fix.

I hope we can make a fix that is a little smoother than either suggestion above though... it seems to me that mesh files are usually stored in the same or a nearby directory as their associated textures and other resources. How about inside loadMeshFromResource() we strip the final filename off of the mesh file path and add the resultant directory to OGRE's resource path? Would that do it? Would we need to recurse down from there into other directories? Would we need to go up from there into parent directories?

Can you post or link to an example that includes the .urdf, .mesh, and material or texture files that are currently ignored?

[ahornung]

True, I would assume the textures to be close to the meshes. Assuming the same directory would be a good start, but I think recursing into subdirectories could also be required. I don't know the usual structure or conventions of an Ogre mesh export, though... "Media/materials" seems to be a common path for them:
https://github.com/jorge-d/zappy/tree/master/graphic_dir/datafiles/media/materials
https://ipvs.informatik.uni-stuttgart.de/roboearth/repos/pre-release/tags/workshop2/re_gazebo_sim/Media/materials/
https://bitbucket.org/sinbad/ogre/src/75648638061a85b8c9210633b24575bea46984c0/Samples/Media/materials?at=v1-9

The meshes we're working with are not publicly available unfortunately, but I could send you an example via email.

[hershwg]

Example via email is fine, but better would be a public example that
matches the directory layout you need.
On Jul 31, 2013 1:59 AM, "Armin Hornung" notifications@github.com wrote:

True, I would assume the textures to be close to the meshes. Assuming the
same directory would be a good start, but I think recursing into
subdirectories could also be required. I don't know the usual structure or
conventions of an Ogre mesh export, though... "Media/materials" seems to be
a common path for them:

https://github.com/jorge-d/zappy/tree/master/graphic_dir/datafiles/media/materials

https://ipvs.informatik.uni-stuttgart.de/roboearth/repos/pre-release/tags/workshop2/re_gazebo_sim/Media/materials/

https://bitbucket.org/sinbad/ogre/src/75648638061a85b8c9210633b24575bea46984c0/Samples/Media/materials?at=v1-9

The meshes we're working with are not publicly available unfortunately,
but I could send you an example via email.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/507#issuecomment-21848881
.

[ahornung]

OK, sent you a mail.

There are models that consist of multiple part (partly reused) so mesh files can be in a subtree of a folder. In that case adding every directory does not make much sense. In line with rviz, what about adding ogre_media of the package containing the meshes to the resource path?

[hershwg]

Well, I'm still not sure.

Ogre::ResourceGroupManager::addResourceLocation() has a "recursive" option, so it would be easy to implement something like the ogre_media convention you describe. Could go ahead and open it up to the whole package containing the mesh file though.

What I worry about with a convention like this is that when it works it will seem kind of magical and when it doesn't it will be pretty unclear as to why it doesn't work.

I like the <export> tag idea a lot, but the fact that it means that urdf packages then need to depend on rviz is a problem. It occurred to me that we could use the "urdf" package as a stand-in for the owner of the plugin, like so:

<export>
   <urdf media_path="${prefix}/Media"/>
</export>

But since the actual "urdf" package wouldn't actually be related, and could conceivably go away or have its name changed and then break this seems like a problem.

In that direction, we could make an empty package called "media_exports" which just has some documentation in it explaining what it is for. Then rviz and nao_description could both depend on that, and we would use:

<export>
  <media_export path="${prefix}/Media"/>
</export>

I don't have a feel for the "cost" of adding a tag to URDF. Since it doesn't address the problem of Ogre meshes in marker messages, maybe it is not really in the running.

[hershwg]

In fact, since this issue of needing to depend on a package named by the export tag exists generally, we could call the new package "misc_export". Then anyone could depend on it and thus read kind of arbitrary stuff out of the export section.

<export>
   <misc_export media_path="${prefix}/Media"/>
   <misc_export favorite_color="blue"/>
</export>

[hershwg]

A benefit of the "media_export" package idea is that you could put common materials in a separate package from the .mesh files that needed them. If we used the scheme where it added the $package/ogre_media when loading a .mesh file from $package, it would not be able to see the materials in the common package.

[wjwwood]

Could the functionality of this new media_export package be integrated into an existing common dependency for these description packages, possibly one of the robot_model packages?

[hershwg]

Well, I was thinking we could use "urdf", since that is a dependency of anything that has a urdf file. However, rviz has a feature where you can send "mesh marker" messages which refer to mesh files, which may in turn need to load texture and material files (and thus need this media-loading behavior). At that point, urdf is not particularly related.

[wjwwood]

Then maybe not, just a thought.

Otherwise, making a new package seems like the best option.

[dgossow]

Guys, correct me if I'm wrong, but from what I understand, there are two separate issues here:

1. Finding an Ogre mesh referenced from a URDF
2. Finding assets (textures) referenced from an Ogre mesh (both for visualizing a URDF and mesh markers)

For (1), you can specify the mesh filename as a package:// URI, which can be resolved into an absolute path, as done in the urdf tutorials on ros.org. Don't know how Gazebo handles that though.

For (2), the path to the asset in the mesh file should be relative to the file itself, so adding the path where the mesh file resides as a resource location should do the job. You could allow package:// URIs there as well.

[hershwg]

This issue is about (2). (1) is working fine.

The problem is that with Ogre .mesh files, you can't specify a material
file name. You specify a "material resource name" or similar. It is not
part of a directory hierarchy like a filename, it is just a single,
hopefully-unique identifier. Separately, you need to tell Ogre about all
the directories in which it might find resources. Sigh. Since we know the
full path to the Ogre .mesh file, we could add that very directory to the
resource path, that's fine. But in the examples Armin sent me, the meshes
and materials are all arranged under separate directories:
media/materials/scripts/.material, media/materials/textures/.png,
media/meshes/*.mesh, etc. So there would need to be either a big recursive
inclusion of lots of irrelevant stuff, or we would need to establish a
convention which says rviz will silently look for a folder named "media" or
something as it walks up the tree from the .mesh file.

So that's why I thought the explicit thing would be nice, because
it would sort of put the information right out there, without making people
hunt through source code for the magic directory name.

On Thu, Aug 1, 2013 at 6:31 PM, David Gossow notifications@github.comwrote:

Guys, correct me if I'm wrong, but from what I understand, there are two
separate issues here:

1. Finding an Ogre mesh referenced from a URDF
2. Finding assets (textures) referenced from an Ogre mesh (both for
   visualizing a URDF and mesh markers)

For (1), you can specify the mesh filename as a package:// URI, which can
be resolved into an absolute path, as done in the urdf tutorials on
ros.org. Don't know how Gazebo handles that though.

For (2), the path to the asset in the mesh file should be relative to the
file itself, so adding the path where the mesh file resides as a resource
location should do the job. You could allow package:// URIs there as well.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/507#issuecomment-21981508
.

[dgossow]

Thanks for the clarification!

In that case, my vote would go to creating a media_export package and using the export tag in the package.xml to specify your media paths.

[ahornung]

Sounds good to me too!

[hershwg]

I have a fix for this in 17f5520. You'll also need https://github.com/ros/media_export in your workspace to try it out. See the README.md in media_export for descriptions of what you need to add to your packages which want to export ogre_media_path s.

[hershwg]

Thanks to @ahornung for the code that reads the paths into Ogre!

[ahornung]

I can confirm that it's working great here with RViz from the hydro-devel branch, thanks!

By the way, the original code for the paths above was contributed by Stefan Osswald (University of Freiburg).