invalid swapInterval behavior of glXSwapIntervalEXT OpenGL extension - it breaks OpenGL specification

[qrp73]

There is a broken behavior of swapInterval behavior in Raspberry Pi OpenGL driver. It breaks OpenGL specification and works in incorrect way. As result it leads to issues with broken video playback, broken animation and other issues in different applications.

The term "swap interval" itself refers to the number of v-blanks that must occur before the front and back frame buffers are swapped. There are different OpenGL extensions available which give access to control swapInterval on different platforms.

For Linux X11 OpenGL there is glXSwapIntervalEXT extension. You can find its specification here:
https://registry.khronos.org/OpenGL/extensions/EXT/EXT_swap_control.txt

I just note that it mention swapInterval=2 as example to explain how it should works:

    glXSwapIntervalEXT specifies the minimum number of video frame
    periods per buffer swap for a particular GLX drawable (e.g. a value
    of two means that the color buffers will be swapped at most every
    other video frame). The interval takes effect when glXSwapBuffers
    is first called on the drawable subsequent to the glXSwapIntervalEXT
    call.

As you can see, OpenGL specification explains that when you set swapInterval=2, this should result two display frames displaying one application frame. swapInterval=3 should result three display frames displaying one application frame. etc.

This is exactly how OpenGL swapInterval extensions working on all other platforms like Windows/Linux/MacOS and for different driver vendors like NVidia/AMD/Intel/etc. But unfortunately not for Raspberry Pi OpenGL driver which breaks OpenGL specification.

According to OpenGL specification, max supported swapInterval value can be obtained with GLX_MAX_SWAP_INTERVAL_EXT. Raspberry Pi OS Bookworm reports that max supported swapInterval value is 2147483647, you can check it with this test code:

test-maxSwapInterval.c

#include <stdio.h>
#include <SDL2/SDL.h>
#include <SDL2/SDL_syswm.h>
#include <GL/glx.h>
#include <X11/Xlib.h>

int main() {
    if (SDL_Init(SDL_INIT_VIDEO) < 0) {
        printf("SDL Init Error: %s\n", SDL_GetError());
        return 1;
    }

    SDL_Window* window = SDL_CreateWindow("SDL2 OpenGL",
                                          SDL_WINDOWPOS_CENTERED, SDL_WINDOWPOS_CENTERED,
                                          800, 600, SDL_WINDOW_OPENGL);
    if (!window) {
        printf("Window Creation Error: %s\n", SDL_GetError());
        SDL_Quit();
        return 1;
    }

    SDL_GLContext context = SDL_GL_CreateContext(window);
    if (!context) {
        printf("OpenGL Context Error: %s\n", SDL_GetError());
        SDL_DestroyWindow(window);
        SDL_Quit();
        return 1;
    }

    // Get X11 Display and Window from SDL2
    SDL_SysWMinfo wmInfo;
    SDL_VERSION(&wmInfo.version);
    if (!SDL_GetWindowWMInfo(window, &wmInfo)) {
        printf("Failed to get X11 window info: %s\n", SDL_GetError());
        SDL_GL_DeleteContext(context);
        SDL_DestroyWindow(window);
        SDL_Quit();
        return 1;
    }

    Display* display = wmInfo.info.x11.display;
    Window sdlWindow = wmInfo.info.x11.window;
    int maxSwapInterval = 0;

    glXQueryDrawable(display, sdlWindow, GLX_MAX_SWAP_INTERVAL_EXT, (unsigned int*)&maxSwapInterval);

    printf("GLX_MAX_SWAP_INTERVAL_EXT: %d\n", maxSwapInterval);

    SDL_GL_DeleteContext(context);
    SDL_DestroyWindow(window);
    SDL_Quit();
    return 0;
}

Result:

GLX_MAX_SWAP_INTERVAL_EXT: 2147483647

But for some unknown reason Raspberry Pi OpenGL driver works incorrectly for swapInterval > 1. It just lead to swap buffers with some random unpredictable and illogical swap rate which value is even hard to explain and it may vary depends on conditions. It does not follow OpenGL specification.

As result it leads to many issues with broken animation and video playback in applications, because they expect that swap buffers works according to OpenGL specification and its timing depends on predictable swap buffer behavior which is explained in OpenGL specification. Unfortunately it leads to broken timing when such apps running with Raspberry Pi OpenGL driver.

You can find test app and steps to reproduce which shows this issue on issue #367. It was mistakenly closed as SDL2 issue. But this is incorrect, it just uses SDL2 to show OpenGL issue. I catch this issue in app which uses direct calls to glXSwapIntervalEXT and don't uses SDL2 at all and it has exactly the same issue. And there is no such issue with this SDL2 test code on NVidia/AMD/Intel drivers.

I can reproduce this issue in my own library, which does not use SDL2 and makes direct OpenGL calls. However, the library is quite large and not suitable as a simple demo. If the SDL2 application from issue #367 is not sufficient to demonstrate the problem, I can rewrite it using direct OpenGL calls. That said, the issue is clearly within the Raspberry Pi OpenGL driver.

[lurch]

As result it leads to issues with broken video playback, broken animation and other issues in different applications.

Just out of curiosity, could you provide some examples of applications where it breaks things? Or are you just referring to your own applications? (so that we can get a better idea of how widespread this problem is)

ping @popcornmix and @6by9

[qrp73]

Just out of curiosity, could you provide some examples of applications where it breaks things? Or are you just referring to your own applications? (so that we can get a better idea of how widespread this problem is)

it affects any app that uses swapInterval extension. Usually this extension is used for low power devices which don't have powerful enough GPU, so increasing swapInterval allows to keep smooth and stable graphics at lower fps.

For example chromium browser using it, so it may affect graphics in chromium browser. I don't know exact details how it can affect browser, but according to source code this feature is used by chromium and since it is broken on Raspberry Pi OpenGL driver it may be the reason for broken video/animation playback on some sites.

Another examples of apps which are affected are OpenGL games, for example Arx Libertatis (open source port of Arx Fatalis) and other uses configurable frame rate setting, but is broken due to broken behavior of Raspberry Pi OpenGL driver.

It's hard to predict results, because Raspberry Pi OpenGL driver behavior for swapInterval is unstable and very unpredictable. For example when you set swapInterval=2 for display with refresh rate 60 Hz, it should lead to a stable 30 fps. But on Raspberry Pi OpenGL driver you will get floating fps around 40 fps which is changed depends on unknown conditions. As result it leads to broken animations and jerky movements in apps which expect stable fps.

Since 40 fps is close to 30 fps, you can confuse unstable animation/fps and jerky movements issues on poor performance of Raspberry Pi, but actual root of cause is broken swapInterval behavior and actually Raspberry Pi can run these apps better.

The main problem here is that swapInterval is intended to keep stable and fixed interval between buffer swap, while Raspberry Pi OpenGL driver produce invalid and varying time interval when you set swapInterval > 1.

Technically, it's very easy to implement the correct swapInterval behavior: just use a counter that is initialized on every buffer swap from the swapInterval setting and decremented on every display frame. Just skip other display frames to display previous buffer content. When it reaches 0, perform a buffer swap. For swapInterval = 0, simply disable v-blank sync and swap the buffer immediately. That's it—pretty easy and doesn't require complex logic or computational resources.

So I don't understand why it works incorrectly on the Raspberry Pi OpenGL driver. Its behavior should just be fixed to comply with the OpenGL specification.