shape distance radii bug

[k88936]

bool hit = false;
bool callback( b2ShapeId id, void* context )
{
	printf( "hit" );
	hit = true;
	return false;
}
int testOverlap( void )
{
	{
		float x=3.0f;
		float y=5.0f;
		float boxSize=0.5f;
		float testSize=0.4f;
		b2WorldDef worldDef = b2DefaultWorldDef();
		b2WorldId world_id = b2CreateWorld( &worldDef );
		b2BodyDef body_def = b2DefaultBodyDef();
		body_def.type = b2_staticBody;
		body_def.position = (b2Vec2){ x, y };
		b2BodyId body_id = b2CreateBody( world_id, &body_def );
		b2Polygon polygon = b2MakeSquare( boxSize);
		b2ShapeDef shape_def = b2DefaultShapeDef();
		b2CreatePolygonShape( body_id, &shape_def, &polygon );
		b2World_Step( world_id, 1. / 60., 4 );
		b2Polygon test_polygon = b2MakeSquare( testSize);
		b2Transform tfm = { { x,y}, { 1.0f, 0.0f } };
		b2World_OverlapPolygon( world_id, &test_polygon, tfm, b2DefaultQueryFilter(), callback, NULL );
		ENSURE( hit );
		b2DestroyBody( body_id );
		b2DestroyWorld( world_id );
	}
	return 0;
}

this is the how to reproduce the situation
environment:

Linux 6.13.1-arch1-1 amd64 CPU: 8 

Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-linux-gnu/14.2.1/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --enable-languages=ada,c,c++,d,fortran,go,lto,m2,objc,obj-c++,rust --enable-bootstrap --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://gitlab.archlinux.org/archlinux/packaging/packages/gcc/-/issues --with-build-config=bootstrap-lto --with-linker-hash-style=gnu --with-system-zlib --enable-__cxa_atexit --enable-cet=auto --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-default-ssp --enable-gnu-indirect-function --enable-gnu-unique-object --enable-libstdcxx-backtrace --enable-link-serialization=1 --enable-linker-build-id --enable-lto --enable-multilib --enable-plugin --enable-shared --enable-threads=posix --disable-libssp --disable-libstdcxx-pch --disable-werror
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 14.2.1 20250128 (GCC) 

I have done some check ,found that if x , y exactly to be 3.0,5.0 and testSize to be 0.4, the hit is not detected, but put a little diff to these vars, it occurred . so i guess it is the float error problem:

and i found these:
src/distance.c:519

	// Apply radii if requested
	if ( input->useRadii )
	{
		if ( output.distance < FLT_EPSILON )
		{
			...
		}
		else
		{
			...
		}
	}

here output.distance (4.76837158E-7)is exactly 4 times of FLT_EPSILON(1.1920929E-7)
so i think the minimal distance error the algorithm produce is not simply the float epsilon,maybe 4 times of that ?

really hard to locate this , i went along my code and libgdx binding and finally here

[erincatto]

Thanks for the report. This is a troublesome configuration for the distance algorithm. Nevertheless, I found a way to improve the accuracy. I'm also changing the overlap function to use a tolerance so that small distances are considered overlap.

This change will be in my next PR.