ShapeFrame and ShapeNode

[jslee02]

This is a work-in-progress pull request for implementing ShapeFrame/ShapeNode as discussed in #394.

[mkoval]

I don't think it is necessary to have both a const char* overload and a const std::string& overload. Won't the const char* implicitly convert to an std::string?

[mxgrey]

I'm not sure about this particular case, but I've seen cases where having an overloaded function where one of the overloads takes a std::string, the compiler won't always implicitly convert it when you pass in a const char*. I've especially seen this for templated functions, which the next one down is. (I don't know about this specific case, since @jslee02 wrote it, but I'm guessing it's a similar situation here.)

[jslee02]

As @mxgrey said, the implicit converting sometimes didn't work. I'm still not sure exactly when is the case. I just tested this function and BodyNode::createEndEffector(const char*), and the implicit converting worked for both of them. So we can remove these functions.

Edit: My test went wrong. The conversion doesn't work so we need the overloaded functions. This might be the case of templated function case.

[mxgrey]

I think the way it works is that it will attempt to use a templated function instead of performing an implicit conversion, but then the templated function fails because it wasn't meant for the case of a const char*.

[jslee02]

I see. The process of type deduction for templated function doesn't check if the passed-in parameter type can be implicitly converted to one of possible types. Good to know!

[mxgrey]

Do shapes have Color anymore? I thought Shapes were now purely geometric information, and that color information would be in the VisualAddon of the ShapeNode. Or is this just WIP code that's going to be removed later?

[jslee02]

I thought so too, but I realized MeshShape has color properties (colors for the vertices) in it. This function gets called when the color of VisualAddon is changed so that MeshShape can handle it. By default, it does nothing for other shapes. This function is a compromised solution, so I'm open to any better way for this case.

[mxgrey]

There has been some discussion about the possibility of MeshShape being changed into something that only holds geometric data. If we do that, then we could have it so that the texture and color data gets automatically put into the VisualAddon as it's parsed with assimp. I think this would be also be nice for moving away from assimp being a core dependency (instead it would only be an io dependency).

That said, I don't know what the timeline should be for implementing something like that. We could probably hold off on it until version 7.1.

[jslee02]

Yeah, the change sounds good to me. If I remember correctly, there is ongoing work of Mesh class (for moving away from assimp) by @mkoval. Once it's done we can remove this function. Maybe a note for this here would be good.

[jslee02]

Okay, I didn't remember correctly. 😭 @mklingen was working on it at here. Ref: #453.

[mkoval]

I very much do not want DART in the business of defining a canonical set of material properties. I am cautiously optimistic about @PyryM's suggestion:

I think materials should just be key/value stores that map string keys to [vec4 | texture | string] values.

It is straightforward to allow the user to store arbitrary vertex properties. We could simply allow the user to store a std::vector<T> with size() equal to the number of vertices. I am less sure about providing an API to store textures. Hopefully @PyryM can clarify a few points:

• Assimp supports spherical, cylindrical, and uv mapping. Are all three used in practice?
• I also found some references to uvw-mapping. How does this differ from uv mapping?
• Are uv-coordinates stored as separate vertex properties or attached to the image asset?
• How are uv-coordinates associated with the corresponding assert?
• Are all textures 24-bit RGBA? Is it common to use a different bit depth or number of channels?

My only other concern is that VisualAddon is attached to ShapeFrame, not Shape. These properties only apply to MeshShape and are intimately related to the mesh, e.g. arrays of vertex properties must have the same length as the number of vertices in the mesh.

Alternatively, we could take the simpler route I suggested in #453:

MeshNode should contain the bare minimum amount of information necessary to perform collision checking and a dynamical simulation. Additionally, DART should provide an Uri to the original resource that can be loaded for visualization. The user is responsible for loading any additional information, e.g. textures and material properties, that is necessary only for visualization.

This has two downsides. First, every mesh is loaded from disk twice: once for collision detection and once for visualization. Second, we cannot programmatically construct a textured or vertex-colored mesh. This latter limitation bothers me more than the former - @mklingen and I independently ran into this in our research when rendering a color map.

[psigen]

Regarding the method in #453:

The user is responsible for loading any additional information, e.g. textures and material properties, that is necessary only for visualization.

Consider loading an object from a file URI but then changing the object on disk (I was trying to adjust some objects in a scene while I had it open in rviz). There is zero guarantee that each loading operation will have the same result, especially if assets are lazily-loaded, which can lead to out-of-sync visuals or even more pathological consistency issues.

This might even specifically be intentional in cases where an object is being dynamically updated, such as serving a generated model that results from perception or some other process at an http URI.

We also currently have to run an HTTP proxy server and rewrite URIs to implement dart_rviz, because we have the problem that URIs that are resolvable in the originating DART process aren't necessarily resolvable in the scope of the visualization (because it can be run over a network).

So, while it is very semantically clean, it does have some significant downsides.

[PyryM]

Assimp supports spherical, cylindrical, and uv mapping. Are all three used in practice?

Spherical and cylindrical are ways of automatically assigning uv coordinates. I wouldn't bother trying to directly support them since any 3d editing program can compute these mappings and export them as regular uv coordinates.

Are uv-coordinates stored as separate vertex properties or attached to the image asset?
I also found some references to uvw-mapping. How does this differ from uv mapping?

Texture coordinates are vertex attributes, they're part of the mesh data like vertex positions and normals. Typical texture mapping, to simplify a lot, works like color = my_texture[u,v]. You can have 3d textures (a 3d array compared to a 2d array for a normal texture) in which case color = my_texture[u,v,w]. But the names and meanings of all the vertex attributes are just conventions, and shaders are free to interpret the attributes however they want.

Are all textures 24-bit RGBA? Is it common to use a different bit depth or number of channels?

32 bit RGBA8/BGRA8 is the most common, and probably the only format that's likely to be encountered in this application. It might be worth supporting float textures (R32F or RGBA32F) to make it convenient to visualize procedurally generated data without having to squash or encode it into RGBA8.

[mkoval]

This discussion moved to #612. In the meantime, should we remove this function?

[jslee02]

I think we should keep this function as far as MeshShape contains color data and we want to use and update it.

[mkoval]

I'm not sure if this overload is necessary. It only seems helpful in the case where you want to create a ShapeNode with several AddOns, but you don't need to set any properties on them. Any other case would be better served by adding a getOrCreate function to `AddonManager, e.g.:

ShapeNode* node = bodyNode->createShapeNode();
node->getOrCreate<VisualAddon>()->mColor = Orange();

[jslee02]

This function was added exactly for the case you mentioned. One might want to create a ShapeNode with several addons as soon as it gets created. Here is one example case. Without this function, we need multiple lines for this like:

auto shapeNode = bodyNode->createShapeNode(shape);
shapeNode->createVisualAddon();
shapeNode->createCollisionAddon();
shapeNode->createDynamicsAddon();

I don't object to remove this function, but this would be useful for the above use case.

[mxgrey]

I like the function, although I wonder if we can have it accept Properties arguments for each Addon which will use their default constructor if one is not provided. I can't think of a way to do that at the moment, but I suspect it may be possible with enough cleverness.

[mkoval]

This is only useful if you want to create the Addon, but not set any properties on it. Is this something that we expect to happen frequently?

[mxgrey]

It looks like this function has been left undefined. I'm not exactly sure how we should approach defining it. Should the ShapeNode be given default properties? I'm inclined to say that we don't really need this function, since we already have the templated version of createShapeNode that allows the collision addon to be created simultaneously.

[jslee02]

Yeah, I also intended to remove this function but forgot to remove the declaration. Removing.

[mxgrey]

I've finished reviewing. All of my comments which are relevant to this pull request have been addressed. I've also gone ahead and made some changes to the API for accessing and creating ShapeNodes. In particular, I've added With to all functions that operate on a subset of ShapeNodes that contain a particular Addon type. I've also removed the AddonAdder struct and replaced it with two void-returning variadic functions which are now located in common/AddonManager.h.

I only have one remaining concern: It seems that soft bodies are very very broken right now. If we're okay with soft bodies being broken on master, then I'm fine with merging in this pull request. I do think we should try to get them at least looking believable before the major release, though.

[jslee02]

Could you take a look at osgDart as well? I made it just work (I believe ) but there are some stopgaps. For example, EntityNode is replaced by ShapeFrameNode but WorldNode is still using EntityNode.

I'm looking at soft bodies now.

[mxgrey]

The drag and drop features in osgDart seem to be working fine. Unfortunately, the diff for those files isn't showing up for me on Github. If anyone can recommend a good git diff viewing desktop application, I would appreciate it. Otherwise, I can comb through the raw code some time soon.

[psigen]

On Ubuntu, I generally use meld.

[mxgrey]

I use meld for merging, but I don't like looking back and forth from side to side when it comes to comparing two files. I really like the linear way that Github lays it out. The standard console tool lays it out similarly, but isn't very pretty. I probably won't be able to find what I want, so I'll just have to settle for meld or the console. It would be nice if Github offered an offline version of their viewer.

[jslee02]

It would be nice if Github offered an offline version of their viewer.
👍

I created two dummy branches (with and without osgDart changes) to help reviewing osgDart here. It doesn't include commit-wise changes of the original history, but you can see the final diff.

Also, I resolved the instability of the soft bodies in the last commits.

[psigen]

@mxgrey you can possibly try the vertical diff mode in vimdiff?
http://stackoverflow.com/a/244766

I actually find it hard to use the unified vertical diff, because it makes it really tricky to compare large block diffs, but I think several difftools have flags to go into that mode.

[mxgrey]

I've pushed some fixes to the way Addons are cloned, and I've restructured osgDart. Soft bodies seem to be working again after @jslee02 patched it.

I think this pull request is good to merge now 👍

[jslee02]

👍 looks good to me too. Will merge this once #607 is merged.