Demo scene seems a bit broken

[pwab]

I tried your provided demo scene and was confused about the cylinder behavior. Here is a short gif:

The reason could be the broken CylinderCollisionShape in Godot 3.2.

[pwab]

One thing that has to be mentioned is that the CollisionShape of the cans is a ConvexPolygonShape right now:

https://github.com/BastiaanOlij/gdleapmotion/blob/358b2df4238dd35da6d45411ce83e1325d9d8952/demo/can.tscn#L64

When changing this to a CylinderCollisionShape with radius 0.02, height 0.05 and margin 0.04 (standard value) then you get this:

Even by adapting the margin you aren't getting something useful. And the same happens when you change to CapsuleShape. If you use a SphereShape you're getting other behavior:

The only one that worked was the BoxShape. So to me it seems the CylinderShape is not the only shape with a problem. It might be more related to the problem that Godot is not good at colliding small things (see godotengine/godot#2092 and godotengine/godot#16505).

[pwab]

It might be more related to the problem that Godot is not good at colliding small things

That was a bit fast. Change from Bullet to Godot Physics and you can clearly see that SphereShape and CapsuleShape are working there perfectly. Still the CylinderShape does not. So it boils down to this:

• The Bullet Engine has a problem with CollisionShapes other than the BoxShape when the scale is very small
• The CylinderShape is broken in both systems

EDIT:
And this was partly my fault:

E 0:00:02.262   shape_create: CylinderShape is not supported in GodotPhysics. Please switch to Bullet in the Project Settings.
  <C++-Fehler>  Method failed. Returning: RID()
  <C++-Quellcode>servers/physics/physics_server_sw.cpp:73 @ shape_create()

[pouleyKetchoupp]

The collision margin is an extra distance at which things collide, so it won't work well if this margin is larger than the size of your shapes. The default value of 0.04 is meant to be used for objects of scale around 1.

There are two things you can do to fix the problem:

• The ideal would be to scale the whole scene up (x20) to make sure it's closer to standard scale, because the physics engine works best when things are around a scale of 1.
• You can try to set all the shape margins to the minimum (0.001). I'm not sure how the hand works, but if it has collision shapes too, they need to be set to the same low margin as well.

[pwab]

Hey @pouleyKetchoupp thanks for your suggestions. Just some remarks on that:

The ideal would be to scale the whole scene up (x20) to make sure it's closer to standard scale, because the physics engine works best when things are around a scale of 1.

We already discussed this with @BastiaanOlij at BastiaanOlij/gdleapmotion-asset#1. You are totally right that this would much improve the physics performance but it is not the way VR systems - and therefore also the leap motion - work. Using units as meters is the standard here. Apart from the possible offsets in perception Bastiaan mentioned in the discussion you also have the problem that external tools like Blender or the common CAD packages used to create objects for real world scenarios work with units as meters. You would have to scale every asset then.

You can try to set all the shape margins to the minimum (0.001). I'm not sure how the hand works, but if it has collision shapes too, they need to be set to the same low margin as well.

I did not think about the hand margins - thank you. But in the comment above you can see that just the demo scene (without hands) has issues that are not affected by the hands margins - it did help to lower the margin but not that much. The problem might be that the minimum margin you mentioned isn't small enough when working with units as meters.

I'll try to look into the shapes and margins again but I'm quite sure the problems can only be solved by allowing smaller margins and using the (currently not working) CCD.

[pouleyKetchoupp]

I understand you need real world scale for your objects, but scale is relative. You can apply a global scale on the root of your whole scene that affects the camera and all objects inside, without changing all of your assets. The user wouldn't be able to tell the difference. Problems would occur only if some scripts you use have hardcoded values based on real world scale, but even then that shouldn't be a huge problem to fix. Of course I might be missing something specific to VR though :)

Making the physics engine work with smaller scale using CCD is overkill for this use case imo.

[BastiaanOlij]

The confusing thing is that 1 unit = 1 meter and thus 0.04 is 4 centimeters is no longer just a standard in VR but often applied to normal games as well.

I don't know yet whether this is a hard limit in the bullet physics engine that is not designed to work at this scale or if the precision configured in Godot is just ill chosen.

But this is definitely a problem that needs to be fixed upstream.

Godot should allow values with higher precision when set from code so one thing to try is to add a script to our cans and in its _ready function set the margin to 0.001 and see if that improves things

[pouleyKetchoupp]

Yes, 1 unit = 1 meter is the standard in most games and physics engines as well. The problem here is caused by the fact this scene works with objects around 1 cm, and Bullet is not configured to work well at this scale.

Using smaller margins for all objects is worth trying (including the hand if it uses rigid bodies, otherwise it won't help), but I don't know if it will solve the problem completely.

That's why I was proposing another simple solution, which doesn't require extra changes in the engine. Just put the whole scene in a Spatial node with a uniform scale of 20. It's easy to do and could work.

I'm sure the physics engine supports uniform scale, and it works better with large scale than small scale. So in a use case where you mix 1 cm objects with 1 m objects, it's safer to scale everything up in physics than working with real world scale.

[pouleyKetchoupp]

Oh, I forgot that for the test with scaling the scene, you would need to scale gravity as well by the same amount. So that adds a little bit more to do :)

[BastiaanOlij]

While this works fine for a desktop game, scaling stuff up in VR provides all sorts of problems. While the ARVRServer is able to deal with most of those by applying a world scale I think its the wrong solution.

It's all relative, I need to find out why bullet can't work with small units while it can when you scale it all up by 20x. That doesn't make mathematical sense, the exact same should be achievable with making all margins 20x smaller.

The reality in VR is that you will often work with centimeters. Objects you pick up are that size

[pwab]

That doesn't make mathematical sense, the exact same should be achievable with making all margins 20x smaller.

Well in mechanical lectures you are confronted with scaling effects that occur when you scale a structure and try to use the same structural principles. A construction could fail because if length is scaled linear, area and volume do not - and so some terms in equations grow unproportionally. But this phenomenom is more likely to see in materials engineering and I'm not sure if rigid body dymanics are affected in any way - especially when using uniform scaling for the world and all including objects.

I'm not sure but isn't it possible that quadratic and exponential terms in equations could lead to accuracy issues if small floats are used as parameters? Like a cylinder collision shape with radius of 0.001 would be followed by a inertia tensor (which is calculated by bullet) with the quadratic radius of 0.000001 - and so maybe other parameters with higher exponents lead to floating point problems?

---

Anyway. I'm investigating the Bullet documentation and forum about units, scale and scaling effects.

I found a 5 year old Bullet_User_Manual for version 2.83 in the bullet3 repository. It could be outdated but I'm sure it still could be quite informative. Some excerpts:

Collision Margin
[...]
Bullet uses a small collision margin for collision shapes, to improve performance and reliability of the collision detection. It is best not to modify the default collision margin, and if you do use a positive value: zero margin might introduce problems. By default this collision margin is set to 0.04, which is 4 centimeter if your units are in meters (recommended).

Dependent on which collision shapes, the margin has different meaning. Generally the collision margin will expand the object. This will create a small gap. To compensate for this, some shapes will subtract the margin from the actual size. For example, the btBoxShape subtracts the collision margin from the half extents. For a btSphereShape, the entire radius is collision margin so no gap will occur. Don’t override the collision margin for spheres. For convex hulls, cylinders and cones, the margin is added to the extents of the object, so a gap will occur, unless you adjust the graphics mesh or collision size.

(Quoted from page 15)

• Using units as meters is recommended
• The margins effect is different depending on the shape

Also btCollisionShape::setMargin uses a btScalar as parameter which is documented here and seems to be able to switch between single and double point precision:

The btScalar type abstracts floating point numbers, to easily switch between double and single floating point precision.

In a forum thread it is stated that:

You can reduce the margin in exchange for performance and "accuracy" -- with a lower margin you might end up in penetration more often and then the penetration resolution code will kick in -- it will push the shapes apart but does so in a non-physical manner so it tends to wander from correctness.

Alternatively you could leave the margin as-is but choose your units to be centimeters instead of meters, however you'd have to compensate the meaning of your timestep, mass properties, and maybe also think about how to modify tune friction and restitution. Another way to think about this is: you're scaling your contraption's dimensions up by some factor (2x, 5x, 10x) and reducing the effective flow of time by a corresponding inverse factor.

This is also explained here . In the manual I also found the following General Tips:

Avoid very small and very large collision shapes

The minimum object size for moving objects is about 0.2 units, 20 centimeters for Earth gravity. If smaller objects or bigger gravity are manipulated, reduce the internal simulation frequency accordingly, using the third argument of btDiscreteDynamicsWorld::stepSimulation. By default it is 60Hz. For instance, simulating a dice throw (1cm-wide box with a gravity of 9.8m/s2) requires a frequency of at least 300Hz (1./300.). It is recommended to keep the maximum size of moving objects smaller then about 5 units/meters.

Avoid large mass ratios (differences)

Simulation becomes unstable when a heavy object is resting on a very light object. It is best to keep the mass around 1. This means accurate interaction between a tank and a very light object is not realistic.

(Quoted from page 35)

From the before mentioned forum thread you will get the following tips:

Configure Bullet to use double precision.
Use fixed-length substeps.
Step at least 240Hz, but know: rumor has it somewhere around 1000Hz strange bugs start to show up.

Just a sidenote about CCD:

Have you enabled continuous collision detection (CCD) on the small moving objects? It definitely helps solve the tunnelling problem but my experience has been: it doesn't fix it completely.

Other topics about world scale:

• https://pybullet.org/Bullet/phpBB3/viewtopic.php?f=9&t=11736&p=39448
• https://pybullet.org/Bullet/phpBB3/viewtopic.php?p=36844&f=&t=10979
• https://pybullet.org/Bullet/phpBB3/viewtopic.php?f=9&t=11920&p=40151

---

This is quite interesting. If those tips are still valid then we should investigate those points:

• Is bullet (or can it be) configured to use double precision? Is this supported by Godot before the PR about double-precision support for physics?
• Is bullet (or can it be) configured to use fixed-length substeps? Is the PR about semi-fixed timestep helping?
• Does the collision iterations PR be a solution if it is transferred to 3D as well to fix CCD issues?

[BastiaanOlij]

@pwab good info man! thanks for digging that all up :)

I think the problem is that when you're building a normal desktop game, having things sized in meters is normal for large games. You rarely deal with small objects other then maybe bullets. The collision shapes are around characters, vehicles, etc. We're usually dealing with an area that is an entire building, or a town, definitely a large amount of space, so we can be efficient if everything is handled at a larger scale.

VR breaks that mold, we're dealing with things close to the user, objects you hold, direct interactions with your hands, etc. So we're working at a completely different scale but seeing floating point math doesn't change noticeably whether your units are meters or centimeters and tracking is already done in meters.... It would also suggest that larger worlds in VR will start to clash because you're dealing with two scales at the same time, needing precise collisions for things close to you while loosing efficiency because you're trying to do physics in a large area.

Anyway, it does sound that the default configuration for bullet does not rhyme well with what we're trying to do and we're going to have to start experimenting with that :)

[aaronfranke]

@pwab Is bullet (or can it be) configured to use double precision? Is this supported by Godot before the PR about double-precision support for physics?

Bullet can use doubles, but not in Godot (yet). It might be supported in 4.0.

However, from a quick glance at this, I doubt the issue is with single-precision float limitations. You need to have a world several kilometers in size to notice such issues.

@pouleyKetchoupp The ideal would be to scale the whole scene up (x20) to make sure it's closer to standard scale, because the physics engine works best when things are around a scale of 1.

For many reasons, it's best to keep objects at as close to a real-world scale as possible. If there are issues with physics objects at very large or very small scales, that's a bug.

[pwab]

Hey @aaronfranke thanks for joining the discussion.

However, from a quick glance at this, I doubt the issue is with single-precision float limitations. You need to have a world several kilometers in size to notice such issues.

Yes I also think that this might not be the case here. We were able to make the demo scene work again with the following adjustments:

• Change FPS to around 300 (at least higher than 120)
• Change the margin of all CollisionShapes programmatically to 0.0001

The last point seems to be a bug in the Godot Inspector where the user can't set values lower than 0.001 for the margin. This is critical in VR environments because we work with objects in a scale that is in this range and therefore the margin must be way smaller.

[aaronfranke]

@pwab The last point seems to be a bug in the Godot Inspector where the user can't set values lower than 0.001 for the margin.

This is an intentional feature, which can be changed here:

[pwab]

This is an intentional feature, which can be changed here:

Interesting. I changed it to 0.0001 and 0.00000001 but I'm still not able to set my margin below 0.001. Just type in 0 for the margin and you see that it jumps to 0.001.

(I'm on Godot 3.2.1 by the way)

[aaronfranke]

Ah yes, that limit comes from elsewhere, from this line of code:

https://github.com/godotengine/godot/blob/ee7bde231c37fa24960b19556645e259c5a18381/scene/resources/shape.cpp#L109

ADD_PROPERTY(PropertyInfo(Variant::REAL, "margin", PROPERTY_HINT_RANGE, "0.001,10,0.001"), "set_margin", "get_margin");

That limit came from... @BastiaanOlij, here lowered from its previous value of 0.04 (?!?)

It does seem surprising that you need a value smaller than 0.001, though. That's 1 millimeter.

I think this solution you've found is masking the real problem of Godot's physics just being buggy.

[pwab]

It does seem surprising that you need a value smaller than 0.001, though. That's 1 millimeter.

Well this comes from two points:

• If I want to create a pen for example (my pen here at the desk is around 1cm in diameter which is 0.5cm in radius which is 0.005m and so it is 0.005units in Godot). The margin is calculated around the CollisionShape which is a simple cylinder here and then the 'recalculated CollisionShape (with margin)' is not only a small fraction of the pen - it actually is 1/5th. And that leads to problems in visuals which are well noticeable in VR if you look close enough at the objects (and you do that in VR). Those visuals could then be adapted to fit this CollisionShape with margin (see Bullet forums and docs) but that would add more work which isn't necessary if the margin could be set smaller. (Other examples of small objects: buttons, switches, cables, bolts and nuts etc.)
• The docs in the bullet engine recommend using 0.04 as the standard value for an object with a size around 1m (talking about a box). So I assume that the margin should be a 1/25 fraction of an objects size and this is 0.0004 for a cube with the size of 1cm³ (a dice for example).

I think this solution you've found is masking the real problem of Godot's physics just being buggy.

Could be true. I don't want to assume that Godots physics is free of bugs 😅

That limit came from... @BastiaanOlij, here lowered from its previous value of 0.04 (?!?)

And exactly here seems to be the comment that talks about that topic:
godotengine/godot#23472 (comment)

Why did @erwincoumans stated the lowest margin size should be 0.001? Was this statement just made for CollisionShapes in the scale of around 1m³? Would it be a problem to set it to a lower value for smaller objects in the scale of around 1cm³ and less (despite of performance)?