Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

testpopup segfaults with direct3d11 renderer #10494

Closed
madebr opened this issue Aug 6, 2024 · 2 comments · Fixed by #10498
Closed

testpopup segfaults with direct3d11 renderer #10494

madebr opened this issue Aug 6, 2024 · 2 comments · Fixed by #10498

Comments

@madebr
Copy link
Contributor

madebr commented Aug 6, 2024

When randomly clicking with the left and right mouse in the testpop example, a segfault occurs.

When running under sdlprocdump, I see a few bad_alloc and com_error exceptions before segfaulting..

Call stack

d3d11.dll!CContext::TID3D11DeviceContext_UpdateSubresource_<1>()
SDL3.dll!D3D11_SetDrawState(SDL_Renderer * renderer, const SDL_RenderCommand * cmd, D3D11_Shader shader, const PixelShaderConstants * shader_constants, const int numShaderResources, ID3D11ShaderResourceView * * shaderResources, ID3D11SamplerState * sampler, const Float4X4 * matrix) Line 2424
	at C:\Projects\SDL\src\render\direct3d11\SDL_render_d3d11.c(2424)
SDL3.dll!D3D11_SetCopyState(SDL_Renderer * renderer, const SDL_RenderCommand * cmd, const Float4X4 * matrix) Line 2492
	at C:\Projects\SDL\src\render\direct3d11\SDL_render_d3d11.c(2492)
SDL3.dll!D3D11_RunCommandQueue(SDL_Renderer * renderer, SDL_RenderCommand * cmd, void * vertices, unsigned __int64 vertsize) Line 2621
	at C:\Projects\SDL\src\render\direct3d11\SDL_render_d3d11.c(2621)
SDL3.dll!FlushRenderCommands(SDL_Renderer * renderer) Line 324
	at C:\Projects\SDL\src\render\SDL_render.c(324)
SDL3.dll!SDL_RenderPresent_REAL(SDL_Renderer * renderer) Line 4970
	at C:\Projects\SDL\src\render\SDL_render.c(4970)
SDL3.dll!SDL_RenderPresent(SDL_Renderer * a) Line 700
	at C:\Projects\SDL\src\dynapi\SDL_dynapi_procs.h(700)
testpopup.exe!loop() Line 239
	at C:\Projects\SDL\test\testpopup.c(239)
testpopup.exe!SDL_main(int argc, char * * argv) Line 277
	at C:\Projects\SDL\test\testpopup.c(277)
SDL3.dll!SDL_RunApp_REAL(int _argc, char * * _argv, int(*)(int, char * *) mainFunction, void * reserved) Line 88
	at C:\Projects\SDL\src\main\windows\SDL_sysmain_runapp.c(88)
SDL3.dll!SDL_RunApp_DEFAULT(int a, char * * b, int(*)(int, char * *) c, void * d) Line 724
	at C:\Projects\SDL\src\dynapi\SDL_dynapi_procs.h(724)
SDL3.dll!SDL_RunApp(int a, char * * b, int(*)(int, char * *) c, void * d) Line 724
	at C:\Projects\SDL\src\dynapi\SDL_dynapi_procs.h(724)
testpopup.exe!main(int argc, char * * argv) Line 99
	at C:\Projects\SDL\include\SDL3\SDL_main_impl.h(99)
[External Code]

Inside this call:

SDL/src/render/direct3d11/SDL_render_d3d11.c

Lines 2415 to 2421 in fe5dcf7

ID3D11DeviceContext_UpdateSubresource(rendererData->d3dContext,
(ID3D11Resource *)rendererData->vertexShaderConstants,
0,
NULL,
&rendererData->vertexShaderConstantsData,
0,
0);

rendererData->vertexShaderConstants, the first argument of ID3D11DeviceContext::UpdateSubresource is NULL.
@madebr
Copy link
Contributor Author

madebr commented Aug 7, 2024 •
edited
Loading

When building with MSVC's address sanitizer, I get an immediate heap-use-after-free when doing an immediate right mouse button -> left mouse button.

=================================================================
==24944==ERROR: AddressSanitizer: heap-use-after-free on address 0x119bfea39288 at pc 0x7ffc3860ea6a bp 0x001d2756d4d0 sp 0x001d2756d4d8
READ of size 8 at 0x119bfea39288 thread T0
    #0 0x7ffc3860ea69 in SDL_UpdateMouseCapture C:\projects\SDL\src\events\SDL_mouse.c:1433
    #1 0x7ffc38617093 in SDL_PrivateSendMouseButton C:\projects\SDL\src\events\SDL_mouse.c:1043
    #2 0x7ffc3860f362 in SDL_SendMouseButton C:\projects\SDL\src\events\SDL_mouse.c:1057
    #3 0x7ffc389e6818 in WIN_CheckWParamMouseButton C:\projects\SDL\src\video\windows\SDL_windowsevents.c:235
    #4 0x7ffc389e69e7 in WIN_CheckWParamMouseButtons C:\projects\SDL\src\video\windows\SDL_windowsevents.c:253
    #5 0x7ffc389dfbdc in WIN_WindowProc C:\projects\SDL\src\video\windows\SDL_windowsevents.c:1169
    #6 0x7ffca2e6ef5b  (C:\WINDOWS\System32\USER32.dll+0x18000ef5b)
    #7 0x7ffca2e6e683  (C:\WINDOWS\System32\USER32.dll+0x18000e683)
    #8 0x7ffc389e5aad in WIN_PumpEvents C:\projects\SDL\src\video\windows\SDL_windowsevents.c:2262
    #9 0x7ffc38600eb5 in SDL_PumpEventsInternal C:\projects\SDL\src\events\SDL_events.c:1170
    #10 0x7ffc385f8cee in SDL_WaitEventTimeoutNS C:\projects\SDL\src\events\SDL_events.c:1367
    #11 0x7ffc385f7654 in SDL_PollEvent_REAL C:\projects\SDL\src\events\SDL_events.c:1221
    #12 0x7ffc385dfbf3 in SDL_PollEvent C:\projects\SDL\src\dynapi\SDL_dynapi_procs.h:651
    #13 0x7ff7511a1c8d in loop C:\projects\SDL\test\testpopup.c:148
    #14 0x7ff7511a123a in SDL_main C:\projects\SDL\test\testpopup.c:276
    #15 0x7ffc389cf8e2 in SDL_RunApp_REAL C:\projects\SDL\src\main\windows\SDL_sysmain_runapp.c:88
    #16 0x7ffc385ebdf4 in SDL_RunApp_DEFAULT C:\projects\SDL\src\dynapi\SDL_dynapi_procs.h:724
    #17 0x7ffc385e2cef in SDL_RunApp C:\projects\SDL\src\dynapi\SDL_dynapi_procs.h:724
    #18 0x7ff7511a127f in main C:\projects\SDL\include\SDL3\SDL_main_impl.h:98
    #19 0x7ff7511b98a8 in invoke_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
    #20 0x7ff7511b97fd in __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
    #21 0x7ff7511b96bd in __scrt_common_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:330
    #22 0x7ff7511b991d in mainCRTStartup D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp:16
    #23 0x7ffca2b27373  (C:\WINDOWS\System32\KERNEL32.DLL+0x180017373)
    #24 0x7ffca35bcc90  (C:\WINDOWS\SYSTEM32\ntdll.dll+0x18004cc90)

0x119bfea39288 is located 72 bytes inside of 440-byte region [0x119bfea39240,0x119bfea393f8)
freed by thread T0 here:
    #0 0x7ffc22c80062  (C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\clang_rt.asan_dbg_dynamic-x86_64.dll+0x180050062)
    #1 0x7ffc387c0cb3 in real_free C:\projects\SDL\src\stdlib\SDL_malloc.c:5191
    #2 0x7ffc387c074d in SDL_free_REAL C:\projects\SDL\src\stdlib\SDL_malloc.c:5340
    #3 0x7ffc38953a19 in SDL_DestroyWindow_REAL C:\projects\SDL\src\video\SDL_video.c:4157
    #4 0x7ffc385dcf83 in SDL_DestroyWindow C:\projects\SDL\src\dynapi\SDL_dynapi_procs.h:151
    #5 0x7ff7511a1b0b in close_popups C:\projects\SDL\test\testpopup.c:128
    #6 0x7ff7511a2003 in loop C:\projects\SDL\test\testpopup.c:166
    #7 0x7ff7511a123a in SDL_main C:\projects\SDL\test\testpopup.c:276
    #8 0x7ffc389cf8e2 in SDL_RunApp_REAL C:\projects\SDL\src\main\windows\SDL_sysmain_runapp.c:88
    #9 0x7ffc385ebdf4 in SDL_RunApp_DEFAULT C:\projects\SDL\src\dynapi\SDL_dynapi_procs.h:724
    #10 0x7ffc385e2cef in SDL_RunApp C:\projects\SDL\src\dynapi\SDL_dynapi_procs.h:724
    #11 0x7ff7511a127f in main C:\projects\SDL\include\SDL3\SDL_main_impl.h:98
    #12 0x7ff7511b98a8 in invoke_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
    #13 0x7ff7511b97fd in __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
    #14 0x7ff7511b96bd in __scrt_common_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:330
    #15 0x7ff7511b991d in mainCRTStartup D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp:16
    #16 0x7ffca2b27373  (C:\WINDOWS\System32\KERNEL32.DLL+0x180017373)
    #17 0x7ffca35bcc90  (C:\WINDOWS\SYSTEM32\ntdll.dll+0x18004cc90)

previously allocated by thread T0 here:
    #0 0x7ffc22c7fede  (C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64\clang_rt.asan_dbg_dynamic-x86_64.dll+0x18004fede)
    #1 0x7ffc387c0c5d in real_calloc C:\projects\SDL\src\stdlib\SDL_malloc.c:5189
    #2 0x7ffc387c066f in SDL_calloc_REAL C:\projects\SDL\src\stdlib\SDL_malloc.c:5306
    #3 0x7ffc3894879d in SDL_CreateWindowWithProperties_REAL C:\projects\SDL\src\video\SDL_video.c:2384
    #4 0x7ffc38947d15 in SDL_CreatePopupWindow_REAL C:\projects\SDL\src\video\SDL_video.c:2502
    #5 0x7ffc385dc581 in SDL_CreatePopupWindow C:\projects\SDL\src\dynapi\SDL_dynapi_procs.h:113
    #6 0x7ff7511a172f in create_popup C:\projects\SDL\test\testpopup.c:104
    #7 0x7ff7511a20b1 in loop C:\projects\SDL\test\testpopup.c:170
    #8 0x7ff7511a123a in SDL_main C:\projects\SDL\test\testpopup.c:276
    #9 0x7ffc389cf8e2 in SDL_RunApp_REAL C:\projects\SDL\src\main\windows\SDL_sysmain_runapp.c:88
    #10 0x7ffc385ebdf4 in SDL_RunApp_DEFAULT C:\projects\SDL\src\dynapi\SDL_dynapi_procs.h:724
    #11 0x7ffc385e2cef in SDL_RunApp C:\projects\SDL\src\dynapi\SDL_dynapi_procs.h:724
    #12 0x7ff7511a127f in main C:\projects\SDL\include\SDL3\SDL_main_impl.h:98
    #13 0x7ff7511b98a8 in invoke_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
    #14 0x7ff7511b97fd in __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
    #15 0x7ff7511b96bd in __scrt_common_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:330
    #16 0x7ff7511b991d in mainCRTStartup D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp:16
    #17 0x7ffca2b27373  (C:\WINDOWS\System32\KERNEL32.DLL+0x180017373)
    #18 0x7ffca35bcc90  (C:\WINDOWS\SYSTEM32\ntdll.dll+0x18004cc90)

slouken added a commit to slouken/SDL that referenced this issue Aug 7, 2024
@slouken
Fixed crash when the current mouse capture window is destroyed
ab54503 
Fixes libsdl-org#10494
slouken added a commit to slouken/SDL that referenced this issue Aug 7, 2024
@slouken
Validate window pointer before sending events for it
6dedf71 
If there are bugs it's possible that the window pointer is invalid. Double check it before sending the application events and potentially dereferencing it.

This showed up in libsdl-org#10494 as a window getting mouse focus as it was being destroyed and then crashing later when mouse focus was set to a different window.
@slouken slouken mentioned this issue Aug 7, 2024
Merged
slouken added a commit that referenced this issue Aug 7, 2024
@slouken
Fixed crash when the current mouse capture window is destroyed
91d97a3 
Fixes #10494
slouken added a commit that referenced this issue Aug 7, 2024
@slouken
Validate window pointer before sending events for it
9493e69 
If there are bugs it's possible that the window pointer is invalid. Double check it before sending the application events and potentially dereferencing it.

This showed up in #10494 as a window getting mouse focus as it was being destroyed and then crashing later when mouse focus was set to a different window.
slouken added a commit that referenced this issue Aug 7, 2024
@slouken
Fixed crash when the current mouse capture window is destroyed
5ca0639 
Fixes #10494

(cherry picked from commit 91d97a3)
slouken added a commit that referenced this issue Aug 7, 2024
@slouken
Fixed crash when the current mouse capture window is destroyed
403f873 
Fixes #10494

(cherry picked from commit 91d97a3)
(cherry picked from commit 5ca0639)
@slouken
Copy link
Collaborator

slouken commented Aug 7, 2024

Fixed, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixed crash when the current mouse capture window is destroyed slouken/SDL
2 participants
@slouken @madebr