Fixed bug 4819 - Attempting to create an OpenGL ES context with unachievable MSAA parameters under X11 dooms the program
authorSam Lantinga
Sat, 16 Nov 2019 22:35:48 -0800
changeset 132516e17e1a6cf2e
parent 13250 81a6e178a551
child 13252 900d20d46990
Fixed bug 4819 - Attempting to create an OpenGL ES context with unachievable MSAA parameters under X11 dooms the program

Solra Bizna

I have written a program that, in the event that the user requests more MSAA samples than their hardware supports, attempts to gracefully fall back to the best MSAA available. This code works with my conventional OpenGL renderer, but if I change nothing about the code except to make it request an OpenGL ES profile instead, Xlib kills the program with an error that looks like:

X Error of failed request: BadWindow (invalid Window parameter)
Major opcode of failed request: 4 (X_DestroyWindow)
Resource id in failed request: 0x5c00008
Serial number of failed request: 188
Current serial number in output stream: 193

To trigger the bug, attempt to create a window with the SDL_WINDOW_OPENGL flag, with SDL_GL_CONTEXT_PROFILE_MASK set to SDL_GL_CONTEXT_PROFILE_ES, and with SDL_GL_MULTISAMPLESAMPLES set to any unsupported value. SDL_CreateWindow properly returns NULL, but at this point the program is already doomed. Xlib will shortly terminate the program with an error. Calling SDL_CreateWindow again will immediately trigger this termination.

I have attached a skeletal program that reproduces this bug for me. Replacing SDL_GL_CONTEXT_PROFILE_ES with SDL_GL_CONTEXT_PROFILE_COMPATIBILITY avoids the bug (but, obviously, doesn't create an OpenGL ES context).

As I suspected, the problem was with XDestroyWindow being called twice on the same window. The X11_CreateWindow function in src/video/x11/SDL_x11window.c calls SetupWindowData. If initialization fails after that point, XDestroyWindow gets called on the window by a subsequent call to X11_DestroyWindow. But, later in the same function, iff a GLES context is requested and initializing it fails, X11_XDestroyWindow (which wraps XDestroyWindow) is manually called. Shortly after, the intended call to X11_DestroyWindow occurs, which attempts to destroy the same window again. Boom.

(The above confusing summary involves three separate, similarly-named functions: XDestroyWindow, X11_DestroyWindow, X11_XDestroyWindow)

I have attached a simple patch that removes the redundant X11_XDestroyWindow calls. I've tested that XDestroyWindow still gets called for the windows in question, and that it only gets called once.
     1.1 --- a/src/video/x11/SDL_x11window.c	Fri Oct 11 22:08:53 2019 +0100
     1.2 +++ b/src/video/x11/SDL_x11window.c	Sat Nov 16 22:35:48 2019 -0800
     1.3 @@ -637,7 +637,6 @@
     1.4      ) {
     1.5  #if SDL_VIDEO_OPENGL_EGL  
     1.6          if (!_this->egl_data) {
     1.7 -            X11_XDestroyWindow(display, w);
     1.8              return -1;
     1.9          }
    1.11 @@ -645,7 +644,6 @@
    1.12          windowdata->egl_surface = SDL_EGL_CreateSurface(_this, (NativeWindowType) w);
    1.14          if (windowdata->egl_surface == EGL_NO_SURFACE) {
    1.15 -            X11_XDestroyWindow(display, w);
    1.16              return SDL_SetError("Could not create GLES window surface");
    1.17          }
    1.18  #else