Fixed bug 3359 - Software renderer does incorrect blending with SDL_RenderCopyEx
authorSam Lantinga <slouken@libsdl.org>
Tue, 15 Nov 2016 01:12:27 -0800
changeset 106126b2307dbec54
parent 10611 52ba31230c02
child 10613 ea44906e19b8
Fixed bug 3359 - Software renderer does incorrect blending with SDL_RenderCopyEx

Simon Hug

The software renderer produces incorrect results when blending textures at an angle with certain blend modes. It seems that there were some edge cases that weren't considered when the SW_RenderCopyEx function was last changed. Or another bug possibly covered up the problem. (More on that in another bug report.)

Most of the issues come from the fact that the rotating function sets a black colorkey. This is problematic because black is most likely appearing in the surface and the final blit will ignore these pixels. Unless a colorkey is already set (the software renderer currently never sets one), it's very hard to find a free color. Of course it could scan over the whole image until one is found, but that seems inefficient.

The following blend modes have issues when drawn at an angle.

NONE: The black pixels get ignored, making them essentially transparent. This breaks the 'dstRGBA = srcRGBA' definition of the NONE blend mode.

MOD: Again, the black pixels get ignored. This also breaks the 'dstRGB = dstRGB * srcRGB' definition of the MOD blend mode, where black pixels would make the destination black as well. A white colorkey will work though, with some preparations.

BLEND: There are some issues when blending a texture with a translucent RGBA target texture. I - uh - forgot what the problem here exactly is.

This patch fixes the issues mentioned above. It mainly changes the code so it tries to do things without the colorkey and removes the automatic format conversion part from the SDLgfx_rotateSurface function. Getting the format right is something the caller has to do now and the required code has been added to the SW_RenderCopyEx function.

There's a small change to the SW_CreateTexture function. RLE encoding a surface with an alpha mask can be a lossy process. Depending on how the user uses the RGBA channels, this may be undesired. The change that surfaces with an alpha mask don't get encoded makes the software renderer consistent with the other renderers.

The SW_RenderCopyEx function now does these steps: Lock the source surface if necessary. Create a clone of the source by using the pixel buffer directly. Check the format and set a flag if a conversion is necessary. Check if scaling or cropping is necessary and set the flag for that as well. Check if color and alpha modulation has to be done before the rotate. Check if the source is an opaque surface. If not, it creates a mask surface that is necessary for the NONE blend mode. If any of the flags were set, a new surface is created and the source will be converted, scaled, cropped, and modulated. The rest of the function stays somewhat the same. The mask also needs to be rotated of course and then there is the NONE blend mode...

It's surprisingly hard to get the pixel from a rotated surface to the destination buffer without affecting the pixel outside the rotated area. I found a way to do this with three blits which is pretty hard on the performance. Perhaps someone has an idea how to do this faster?

As mentioned above, the SDLgfx_rotateSurface now only takes 8-bit paletted or 32-bit with alpha mask surfaces. It additionally sets the new surfaces up for the MOD blend mode.

I shortly tested the 8-bit path of SDLgfx_rotateSurface and it seemed to work so far. This path is not used by the software renderer anyway.
src/render/software/SDL_render_sw.c
src/render/software/SDL_rotate.c
     1.1 --- a/src/render/software/SDL_render_sw.c	Sun Nov 13 23:09:42 2016 -0800
     1.2 +++ b/src/render/software/SDL_render_sw.c	Tue Nov 15 01:12:27 2016 -0800
     1.3 @@ -239,7 +239,10 @@
     1.4      SDL_SetSurfaceAlphaMod(texture->driverdata, texture->a);
     1.5      SDL_SetSurfaceBlendMode(texture->driverdata, texture->blendMode);
     1.6  
     1.7 -    if (texture->access == SDL_TEXTUREACCESS_STATIC) {
     1.8 +    /* Only RLE encode textures without an alpha channel since the RLE coder
     1.9 +     * discards the color values of pixels with an alpha value of zero.
    1.10 +     */
    1.11 +    if (texture->access == SDL_TEXTUREACCESS_STATIC && !Amask) {
    1.12          SDL_SetSurfaceRLE(texture->driverdata, 1);
    1.13      }
    1.14  
    1.15 @@ -599,9 +602,15 @@
    1.16      SDL_Surface *surface = SW_ActivateRenderer(renderer);
    1.17      SDL_Surface *src = (SDL_Surface *) texture->driverdata;
    1.18      SDL_Rect final_rect, tmp_rect;
    1.19 -    SDL_Surface *surface_rotated, *surface_scaled;
    1.20 -    int retval, dstwidth, dstheight, abscenterx, abscentery;
    1.21 +    SDL_Surface *src_clone, *src_rotated, *src_scaled;
    1.22 +    SDL_Surface *mask = NULL, *mask_rotated = NULL;
    1.23 +    int retval = 0, dstwidth, dstheight, abscenterx, abscentery;
    1.24      double cangle, sangle, px, py, p1x, p1y, p2x, p2y, p3x, p3y, p4x, p4y;
    1.25 +    SDL_BlendMode blendmode;
    1.26 +    Uint8 alphaMod, rMod, gMod, bMod;
    1.27 +    int applyModulation = SDL_FALSE;
    1.28 +    int blitRequired = SDL_FALSE;
    1.29 +    int isOpaque = SDL_FALSE;
    1.30  
    1.31      if (!surface) {
    1.32          return -1;
    1.33 @@ -617,69 +626,104 @@
    1.34      final_rect.w = (int)dstrect->w;
    1.35      final_rect.h = (int)dstrect->h;
    1.36  
    1.37 -    /* SDLgfx_rotateSurface doesn't accept a source rectangle, so crop and scale if we need to */
    1.38      tmp_rect = final_rect;
    1.39      tmp_rect.x = 0;
    1.40      tmp_rect.y = 0;
    1.41 -    if (srcrect->w == final_rect.w && srcrect->h == final_rect.h && srcrect->x == 0 && srcrect->y == 0) {
    1.42 -        surface_scaled = src; /* but if we don't need to, just use the original */
    1.43 -        retval = 0;
    1.44 -    } else {
    1.45 -        SDL_Surface *blit_src = src;
    1.46 -        Uint32 colorkey;
    1.47 -        SDL_BlendMode blendMode;
    1.48 -        Uint8 alphaMod, r, g, b;
    1.49 -        SDL_bool cloneSource = SDL_FALSE;
    1.50  
    1.51 -        surface_scaled = SDL_CreateRGBSurface(SDL_SWSURFACE, final_rect.w, final_rect.h, src->format->BitsPerPixel,
    1.52 -                                              src->format->Rmask, src->format->Gmask,
    1.53 -                                              src->format->Bmask, src->format->Amask );
    1.54 -        if (!surface_scaled) {
    1.55 -            return -1;
    1.56 +    /* It is possible to encounter an RLE encoded surface here and locking it is
    1.57 +     * necessary because this code is going to access the pixel buffer directly.
    1.58 +     */
    1.59 +    if (SDL_MUSTLOCK(src)) {
    1.60 +        SDL_LockSurface(src);
    1.61 +    }
    1.62 +
    1.63 +    /* Clone the source surface but use its pixel buffer directly.
    1.64 +     * The original source surface must be treated as read-only.
    1.65 +     */
    1.66 +    src_clone = SDL_CreateRGBSurfaceFrom(src->pixels, src->w, src->h, src->format->BitsPerPixel, src->pitch,
    1.67 +                                         src->format->Rmask, src->format->Gmask,
    1.68 +                                         src->format->Bmask, src->format->Amask);
    1.69 +    if (src_clone == NULL) {
    1.70 +        if (SDL_MUSTLOCK(src)) {
    1.71 +            SDL_UnlockSurface(src);
    1.72          }
    1.73 +        return -1;
    1.74 +    }
    1.75  
    1.76 -        /* copy the color key, alpha mod, blend mode, and color mod so the scaled surface behaves like the source */
    1.77 -        if (SDL_GetColorKey(src, &colorkey) == 0) {
    1.78 -            SDL_SetColorKey(surface_scaled, SDL_TRUE, colorkey);
    1.79 -            cloneSource = SDL_TRUE;
    1.80 -        }
    1.81 -        SDL_GetSurfaceAlphaMod(src, &alphaMod); /* these will be copied to surface_scaled below if necessary */
    1.82 -        SDL_GetSurfaceBlendMode(src, &blendMode);
    1.83 -        SDL_GetSurfaceColorMod(src, &r, &g, &b);
    1.84 +    SDL_GetSurfaceBlendMode(src, &blendmode);
    1.85 +    SDL_GetSurfaceAlphaMod(src, &alphaMod);
    1.86 +    SDL_GetSurfaceColorMod(src, &rMod, &gMod, &bMod);
    1.87  
    1.88 -        /* now we need to blit the src into surface_scaled. since we want to copy the colors from the source to
    1.89 -         * surface_scaled rather than blend them, etc. we'll need to disable the blend mode, alpha mod, etc.
    1.90 -         * but we don't want to modify src (in case it's being used on other threads), so we'll need to clone it
    1.91 -         * before changing the blend options
    1.92 -         */
    1.93 -        cloneSource |= blendMode != SDL_BLENDMODE_NONE || (alphaMod & r & g & b) != 255;
    1.94 -        if (cloneSource) {
    1.95 -            blit_src = SDL_ConvertSurface(src, src->format, src->flags); /* clone src */
    1.96 -            if (!blit_src) {
    1.97 -                SDL_FreeSurface(surface_scaled);
    1.98 -                return -1;
    1.99 -            }
   1.100 -            SDL_SetSurfaceAlphaMod(blit_src, 255); /* disable all blending options in blit_src */
   1.101 -            SDL_SetSurfaceBlendMode(blit_src, SDL_BLENDMODE_NONE);
   1.102 -            SDL_SetColorKey(blit_src, 0, 0);
   1.103 -            SDL_SetSurfaceColorMod(blit_src, 255, 255, 255);
   1.104 -            SDL_SetSurfaceRLE(blit_src, 0); /* don't RLE encode a surface we'll only use once */
   1.105 +    /* SDLgfx_rotateSurface only accepts 32-bit surfaces with a 8888 layout. Everything else has to be converted. */
   1.106 +    if (src->format->BitsPerPixel != 32 || SDL_PIXELLAYOUT(src->format->format) != SDL_PACKEDLAYOUT_8888 || !src->format->Amask) {
   1.107 +        blitRequired = SDL_TRUE;
   1.108 +    }
   1.109  
   1.110 -            SDL_SetSurfaceAlphaMod(surface_scaled, alphaMod); /* copy blending options to surface_scaled */
   1.111 -            SDL_SetSurfaceBlendMode(surface_scaled, blendMode);
   1.112 -            SDL_SetSurfaceColorMod(surface_scaled, r, g, b);
   1.113 -        }
   1.114 +    /* If scaling and cropping is necessary, it has to be taken care of before the rotation. */
   1.115 +    if (!(srcrect->w == final_rect.w && srcrect->h == final_rect.h && srcrect->x == 0 && srcrect->y == 0)) {
   1.116 +        blitRequired = SDL_TRUE;
   1.117 +    }
   1.118  
   1.119 -        retval = SDL_BlitScaled(blit_src, srcrect, surface_scaled, &tmp_rect);
   1.120 -        if (blit_src != src) {
   1.121 -            SDL_FreeSurface(blit_src);
   1.122 +    /* The color and alpha modulation has to be applied before the rotation when using the NONE and MOD blend modes. */
   1.123 +    if ((blendmode == SDL_BLENDMODE_NONE || blendmode == SDL_BLENDMODE_MOD) && (alphaMod & rMod & gMod & bMod) != 255) {
   1.124 +        applyModulation = SDL_TRUE;
   1.125 +        SDL_SetSurfaceAlphaMod(src_clone, alphaMod);
   1.126 +        SDL_SetSurfaceColorMod(src_clone, rMod, gMod, bMod);
   1.127 +    }
   1.128 +
   1.129 +    /* Opaque surfaces are much easier to handle with the NONE blend mode. */
   1.130 +    if (blendmode == SDL_BLENDMODE_NONE && !src->format->Amask && alphaMod == 255) {
   1.131 +        isOpaque = SDL_TRUE;
   1.132 +    }
   1.133 +
   1.134 +    /* The NONE blend mode requires a mask for non-opaque surfaces. This mask will be used
   1.135 +     * to clear the pixels in the destination surface. The other steps are explained below.
   1.136 +     */
   1.137 +    if (blendmode == SDL_BLENDMODE_NONE && !isOpaque) {
   1.138 +        mask = SDL_CreateRGBSurface(0, final_rect.w, final_rect.h, 32,
   1.139 +                                    0x00ff0000, 0x0000ff00, 0x000000ff, 0xff000000);
   1.140 +        if (mask == NULL) {
   1.141 +            retval = -1;
   1.142 +        } else {
   1.143 +            SDL_SetSurfaceBlendMode(mask, SDL_BLENDMODE_MOD);
   1.144          }
   1.145      }
   1.146  
   1.147 +    /* Create a new surface should there be a format mismatch or if scaling, cropping,
   1.148 +     * or modulation is required. It's possible to use the source surface directly otherwise.
   1.149 +     */
   1.150 +    if (!retval && (blitRequired || applyModulation)) {
   1.151 +        SDL_Rect scale_rect = tmp_rect;
   1.152 +        src_scaled = SDL_CreateRGBSurface(0, final_rect.w, final_rect.h, 32,
   1.153 +                                          0x00ff0000, 0x0000ff00, 0x000000ff, 0xff000000);
   1.154 +        if (src_scaled == NULL) {
   1.155 +            retval = -1;
   1.156 +        } else {
   1.157 +            SDL_SetSurfaceBlendMode(src_clone, SDL_BLENDMODE_NONE);
   1.158 +            retval = SDL_BlitScaled(src_clone, srcrect, src_scaled, &scale_rect);
   1.159 +            SDL_FreeSurface(src_clone);
   1.160 +            src_clone = src_scaled;
   1.161 +            src_scaled = NULL;
   1.162 +        }
   1.163 +    }
   1.164 +
   1.165 +    /* SDLgfx_rotateSurface is going to make decisions depending on the blend mode. */
   1.166 +    SDL_SetSurfaceBlendMode(src_clone, blendmode);
   1.167 +
   1.168      if (!retval) {
   1.169          SDLgfx_rotozoomSurfaceSizeTrig(tmp_rect.w, tmp_rect.h, angle, &dstwidth, &dstheight, &cangle, &sangle);
   1.170 -        surface_rotated = SDLgfx_rotateSurface(surface_scaled, angle, dstwidth/2, dstheight/2, GetScaleQuality(), flip & SDL_FLIP_HORIZONTAL, flip & SDL_FLIP_VERTICAL, dstwidth, dstheight, cangle, sangle);
   1.171 -        if(surface_rotated) {
   1.172 +        src_rotated = SDLgfx_rotateSurface(src_clone, angle, dstwidth/2, dstheight/2, GetScaleQuality(), flip & SDL_FLIP_HORIZONTAL, flip & SDL_FLIP_VERTICAL, dstwidth, dstheight, cangle, sangle);
   1.173 +        if (src_rotated == NULL) {
   1.174 +            retval = -1;
   1.175 +        }
   1.176 +        if (!retval && mask != NULL) {
   1.177 +            /* The mask needed for the NONE blend mode gets rotated with the same parameters. */
   1.178 +            mask_rotated = SDLgfx_rotateSurface(mask, angle, dstwidth/2, dstheight/2, SDL_FALSE, 0, 0, dstwidth, dstheight, cangle, sangle);
   1.179 +            if (mask_rotated == NULL) {
   1.180 +                retval = -1;
   1.181 +            }
   1.182 +        }
   1.183 +        if (!retval) {
   1.184              /* Find out where the new origin is by rotating the four final_rect points around the center and then taking the extremes */
   1.185              abscenterx = final_rect.x + (int)center->x;
   1.186              abscentery = final_rect.y + (int)center->y;
   1.187 @@ -715,13 +759,69 @@
   1.188              tmp_rect.w = dstwidth;
   1.189              tmp_rect.h = dstheight;
   1.190  
   1.191 -            retval = SDL_BlitSurface(surface_rotated, NULL, surface, &tmp_rect);
   1.192 -            SDL_FreeSurface(surface_rotated);
   1.193 +            /* The NONE blend mode needs some special care with non-opaque surfaces.
   1.194 +             * Other blend modes or opaque surfaces can be blitted directly.
   1.195 +             */
   1.196 +            if (blendmode != SDL_BLENDMODE_NONE || isOpaque) {
   1.197 +                if (applyModulation == SDL_FALSE) {
   1.198 +                    /* If the modulation wasn't already applied, make it happen now. */
   1.199 +                    SDL_SetSurfaceAlphaMod(src_rotated, alphaMod);
   1.200 +                    SDL_SetSurfaceColorMod(src_rotated, rMod, gMod, bMod);
   1.201 +                }
   1.202 +                retval = SDL_BlitSurface(src_rotated, NULL, surface, &tmp_rect);
   1.203 +            } else {
   1.204 +                /* The NONE blend mode requires three steps to get the pixels onto the destination surface.
   1.205 +                 * First, the area where the rotated pixels will be blitted to get set to zero.
   1.206 +                 * This is accomplished by simply blitting a mask with the NONE blend mode.
   1.207 +                 * The colorkey set by the rotate function will discard the correct pixels.
   1.208 +                 */
   1.209 +                SDL_Rect mask_rect = tmp_rect;
   1.210 +                SDL_SetSurfaceBlendMode(mask_rotated, SDL_BLENDMODE_NONE);
   1.211 +                retval = SDL_BlitSurface(mask_rotated, NULL, surface, &mask_rect);
   1.212 +                if (!retval) {
   1.213 +                    /* The next step copies the alpha value. This is done with the BLEND blend mode and
   1.214 +                     * by modulating the source colors with 0. Since the destination is all zeros, this
   1.215 +                     * will effectively set the destination alpha to the source alpha.
   1.216 +                     */
   1.217 +                    SDL_SetSurfaceColorMod(src_rotated, 0, 0, 0);
   1.218 +                    mask_rect = tmp_rect;
   1.219 +                    retval = SDL_BlitSurface(src_rotated, NULL, surface, &mask_rect);
   1.220 +                    if (!retval) {
   1.221 +                        /* The last step gets the color values in place. The ADD blend mode simply adds them to
   1.222 +                         * the destination (where the color values are all zero). However, because the ADD blend
   1.223 +                         * mode modulates the colors with the alpha channel, a surface without an alpha mask needs
   1.224 +                         * to be created. This makes all source pixels opaque and the colors get copied correctly.
   1.225 +                         */
   1.226 +                        SDL_Surface *src_rotated_rgb;
   1.227 +                        src_rotated_rgb = SDL_CreateRGBSurfaceFrom(src_rotated->pixels, src_rotated->w, src_rotated->h,
   1.228 +                                                                   src_rotated->format->BitsPerPixel, src_rotated->pitch,
   1.229 +                                                                   src_rotated->format->Rmask, src_rotated->format->Gmask,
   1.230 +                                                                   src_rotated->format->Bmask, 0);
   1.231 +                        if (src_rotated_rgb == NULL) {
   1.232 +                            retval = -1;
   1.233 +                        } else {
   1.234 +                            SDL_SetSurfaceBlendMode(src_rotated_rgb, SDL_BLENDMODE_ADD);
   1.235 +                            retval = SDL_BlitSurface(src_rotated_rgb, NULL, surface, &tmp_rect);
   1.236 +                            SDL_FreeSurface(src_rotated_rgb);
   1.237 +                        }
   1.238 +                    }
   1.239 +                }
   1.240 +                SDL_FreeSurface(mask_rotated);
   1.241 +            }
   1.242 +            if (src_rotated != NULL) {
   1.243 +                SDL_FreeSurface(src_rotated);
   1.244 +            }
   1.245          }
   1.246      }
   1.247  
   1.248 -    if (surface_scaled != src) {
   1.249 -        SDL_FreeSurface(surface_scaled);
   1.250 +    if (SDL_MUSTLOCK(src)) {
   1.251 +        SDL_UnlockSurface(src);
   1.252 +    }
   1.253 +    if (mask != NULL) {
   1.254 +        SDL_FreeSurface(mask);
   1.255 +    }
   1.256 +    if (src_clone != NULL) {
   1.257 +        SDL_FreeSurface(src_clone);
   1.258      }
   1.259      return retval;
   1.260  }
     2.1 --- a/src/render/software/SDL_rotate.c	Sun Nov 13 23:09:42 2016 -0800
     2.2 +++ b/src/render/software/SDL_rotate.c	Tue Nov 15 01:12:27 2016 -0800
     2.3 @@ -142,7 +142,7 @@
     2.4          cy = *cangle * y;
     2.5          sx = *sangle * x;
     2.6          sy = *sangle * y;
     2.7 -        
     2.8 +
     2.9          dstwidthhalf = MAX((int)
    2.10              SDL_ceil(MAX(MAX(MAX(SDL_fabs(cx + sy), SDL_fabs(cx - sy)), SDL_fabs(-cx + sy)), SDL_fabs(-cx - sy))), 1);
    2.11          dstheighthalf = MAX((int)
    2.12 @@ -390,10 +390,14 @@
    2.13  /* !
    2.14  \brief Rotates and zooms a surface with different horizontal and vertival scaling factors and optional anti-aliasing.
    2.15  
    2.16 -Rotates a 32bit or 8bit 'src' surface to newly created 'dst' surface.
    2.17 +Rotates a 32-bit or 8-bit 'src' surface to newly created 'dst' surface.
    2.18  'angle' is the rotation in degrees, 'centerx' and 'centery' the rotation center. If 'smooth' is set
    2.19 -then the destination 32bit surface is anti-aliased. If the surface is not 8bit
    2.20 -or 32bit RGBA/ABGR it will be converted into a 32bit RGBA format on the fly.
    2.21 +then the destination 32-bit surface is anti-aliased. 8-bit surfaces must have a colorkey. 32-bit
    2.22 +surfaces must have a 8888 layout with red, green, blue and alpha masks (any ordering goes).
    2.23 +The blend mode of the 'src' surface has some effects on generation of the 'dst' surface: The NONE
    2.24 +mode will set the BLEND mode on the 'dst' surface. The MOD mode either generates a white 'dst'
    2.25 +surface and sets the colorkey or fills the it with the colorkey before copying the pixels.
    2.26 +When using the NONE and MOD modes, color and alpha modulation must be applied before using this function.
    2.27  
    2.28  \param src The surface to rotozoom.
    2.29  \param angle The angle to rotate in degrees.
    2.30 @@ -413,69 +417,47 @@
    2.31  SDL_Surface *
    2.32  SDLgfx_rotateSurface(SDL_Surface * src, double angle, int centerx, int centery, int smooth, int flipx, int flipy, int dstwidth, int dstheight, double cangle, double sangle)
    2.33  {
    2.34 -    SDL_Surface *rz_src;
    2.35      SDL_Surface *rz_dst;
    2.36 -    int is32bit, angle90;
    2.37 +    int is8bit, angle90;
    2.38      int i;
    2.39 -    Uint8 r = 0, g = 0, b = 0, a = 0;
    2.40 +    SDL_BlendMode blendmode;
    2.41      Uint32 colorkey = 0;
    2.42 -    int colorKeyAvailable = 0;
    2.43 +    int colorKeyAvailable = SDL_FALSE;
    2.44      double sangleinv, cangleinv;
    2.45  
    2.46 -    /*
    2.47 -    * Sanity check
    2.48 -    */
    2.49 +    /* Sanity check */
    2.50      if (src == NULL)
    2.51 -        return (NULL);
    2.52 +        return NULL;
    2.53  
    2.54 -    if (src->flags & SDL_TRUE/* SDL_SRCCOLORKEY */) {
    2.55 -        colorkey = _colorkey(src);
    2.56 -        SDL_GetRGBA(colorkey, src->format, &r, &g, &b, &a);
    2.57 -        colorKeyAvailable = 1;
    2.58 +    if (SDL_GetColorKey(src, &colorkey) == 0) {
    2.59 +        colorKeyAvailable = SDL_TRUE;
    2.60      }
    2.61  
    2.62 -    /*
    2.63 -    * Determine if source surface is 32bit or 8bit
    2.64 -    */
    2.65 -    is32bit = (src->format->BitsPerPixel == 32);
    2.66 -    if ((is32bit) || (src->format->BitsPerPixel == 8)) {
    2.67 -        /*
    2.68 -        * Use source surface 'as is'
    2.69 -        */
    2.70 -        rz_src = src;
    2.71 -    } else {
    2.72 -        rz_src = SDL_ConvertSurfaceFormat(src, SDL_PIXELFORMAT_ARGB32, src->flags);
    2.73 -        if (rz_src == NULL) {
    2.74 -            return NULL;
    2.75 -        }
    2.76 -        is32bit = 1;
    2.77 -    }
    2.78 +    /* This function requires a 32-bit surface or 8-bit surface with a colorkey */
    2.79 +    is8bit = src->format->BitsPerPixel == 8 && colorKeyAvailable;
    2.80 +    if (!(is8bit || (src->format->BitsPerPixel == 32 && src->format->Amask)))
    2.81 +        return NULL;
    2.82  
    2.83 -    /* Determine target size */
    2.84 -    /* _rotozoomSurfaceSizeTrig(rz_src->w, rz_src->h, angle, &dstwidth, &dstheight, &cangle, &sangle); */
    2.85 -
    2.86 -    /*
    2.87 -    * Calculate target factors from sin/cos and zoom
    2.88 -    */
    2.89 +    /* Calculate target factors from sin/cos and zoom */
    2.90      sangleinv = sangle*65536.0;
    2.91      cangleinv = cangle*65536.0;
    2.92  
    2.93 -    /*
    2.94 -    * Alloc space to completely contain the rotated surface
    2.95 -    */
    2.96 -    if (is32bit) {
    2.97 -        /*
    2.98 -        * Target surface is 32bit with source RGBA/ABGR ordering
    2.99 -        */
   2.100 -        rz_dst =
   2.101 -            SDL_CreateRGBSurface(SDL_SWSURFACE, dstwidth, dstheight + GUARD_ROWS, 32,
   2.102 -            rz_src->format->Rmask, rz_src->format->Gmask,
   2.103 -            rz_src->format->Bmask, rz_src->format->Amask);
   2.104 +    /* Alloc space to completely contain the rotated surface */
   2.105 +    rz_dst = NULL;
   2.106 +    if (is8bit) {
   2.107 +        /* Target surface is 8 bit */
   2.108 +        rz_dst = SDL_CreateRGBSurface(0, dstwidth, dstheight + GUARD_ROWS, 8, 0, 0, 0, 0);
   2.109 +        if (rz_dst != NULL) {
   2.110 +            for (i = 0; i < src->format->palette->ncolors; i++) {
   2.111 +                rz_dst->format->palette->colors[i] = src->format->palette->colors[i];
   2.112 +            }
   2.113 +            rz_dst->format->palette->ncolors = src->format->palette->ncolors;
   2.114 +        }
   2.115      } else {
   2.116 -        /*
   2.117 -        * Target surface is 8bit
   2.118 -        */
   2.119 -        rz_dst = SDL_CreateRGBSurface(SDL_SWSURFACE, dstwidth, dstheight + GUARD_ROWS, 8, 0, 0, 0, 0);
   2.120 +        /* Target surface is 32 bit with source RGBA ordering */
   2.121 +        rz_dst = SDL_CreateRGBSurface(0, dstwidth, dstheight + GUARD_ROWS, 32,
   2.122 +                                      src->format->Rmask, src->format->Gmask,
   2.123 +                                      src->format->Bmask, src->format->Amask);
   2.124      }
   2.125  
   2.126      /* Check target */
   2.127 @@ -485,17 +467,32 @@
   2.128      /* Adjust for guard rows */
   2.129      rz_dst->h = dstheight;
   2.130  
   2.131 -    if (colorKeyAvailable == 1) {
   2.132 -        colorkey = SDL_MapRGBA(rz_dst->format, r, g, b, a);
   2.133 +    SDL_GetSurfaceBlendMode(src, &blendmode);
   2.134  
   2.135 +    if (colorKeyAvailable == SDL_TRUE) {
   2.136 +        /* If available, the colorkey will be used to discard the pixels that are outside of the rotated area. */
   2.137 +        SDL_SetColorKey(rz_dst, SDL_TRUE, colorkey);
   2.138          SDL_FillRect(rz_dst, NULL, colorkey);
   2.139 +    } else if (blendmode == SDL_BLENDMODE_NONE) {
   2.140 +        blendmode = SDL_BLENDMODE_BLEND;
   2.141 +    } else if (blendmode == SDL_BLENDMODE_MOD) {
   2.142 +        /* Without a colorkey, the target texture has to be white for the MOD blend mode so
   2.143 +         * that the pixels outside the rotated area don't affect the destination surface.
   2.144 +         */
   2.145 +        colorkey = SDL_MapRGBA(rz_dst->format, 255, 255, 255, 0);
   2.146 +        SDL_FillRect(rz_dst, NULL, colorkey);
   2.147 +        /* Setting a white colorkey for the destination surface makes the final blit discard
   2.148 +         * all pixels outside of the rotated area. This doesn't interfere with anything because
   2.149 +         * white pixels are already a no-op and the MOD blend mode does not interact with alpha.
   2.150 +         */
   2.151 +        SDL_SetColorKey(rz_dst, SDL_TRUE, colorkey);
   2.152      }
   2.153  
   2.154 -    /*
   2.155 -    * Lock source surface
   2.156 -    */
   2.157 -    if (SDL_MUSTLOCK(rz_src)) {
   2.158 -        SDL_LockSurface(rz_src);
   2.159 +    SDL_SetSurfaceBlendMode(rz_dst, blendmode);
   2.160 +
   2.161 +    /* Lock source surface */
   2.162 +    if (SDL_MUSTLOCK(src)) {
   2.163 +        SDL_LockSurface(src);
   2.164      }
   2.165  
   2.166      /* check if the rotation is a multiple of 90 degrees so we can take a fast path and also somewhat reduce
   2.167 @@ -510,74 +507,29 @@
   2.168          angle90 = -1;
   2.169      }
   2.170  
   2.171 -    /*
   2.172 -    * Check which kind of surface we have
   2.173 -    */
   2.174 -    if (is32bit) {
   2.175 -        /*
   2.176 -        * Call the 32bit transformation routine to do the rotation (using alpha)
   2.177 -        */
   2.178 +    if (is8bit) {
   2.179 +        /* Call the 8-bit transformation routine to do the rotation */
   2.180 +        if(angle90 >= 0) {
   2.181 +            transformSurfaceY90(src, rz_dst, angle90, flipx, flipy);
   2.182 +        } else {
   2.183 +            transformSurfaceY(src, rz_dst, centerx, centery, (int)sangleinv, (int)cangleinv,
   2.184 +                              flipx, flipy);
   2.185 +        }
   2.186 +    } else {
   2.187 +        /* Call the 32-bit transformation routine to do the rotation */
   2.188          if (angle90 >= 0) {
   2.189 -            transformSurfaceRGBA90(rz_src, rz_dst, angle90, flipx, flipy);
   2.190 +            transformSurfaceRGBA90(src, rz_dst, angle90, flipx, flipy);
   2.191          } else {
   2.192 -            _transformSurfaceRGBA(rz_src, rz_dst, centerx, centery, (int) (sangleinv), (int) (cangleinv), flipx, flipy, smooth);
   2.193 -        }
   2.194 -        /*
   2.195 -         * Turn on source-alpha support
   2.196 -         */
   2.197 -        /* SDL_SetAlpha(rz_dst, SDL_SRCALPHA, 255); */
   2.198 -    } else {
   2.199 -        /*
   2.200 -        * Copy palette and colorkey info
   2.201 -        */
   2.202 -        for (i = 0; i < rz_src->format->palette->ncolors; i++) {
   2.203 -            rz_dst->format->palette->colors[i] = rz_src->format->palette->colors[i];
   2.204 -        }
   2.205 -        rz_dst->format->palette->ncolors = rz_src->format->palette->ncolors;
   2.206 -        /*
   2.207 -        * Call the 8bit transformation routine to do the rotation
   2.208 -        */
   2.209 -        if(angle90 >= 0) {
   2.210 -            transformSurfaceY90(rz_src, rz_dst, angle90, flipx, flipy);
   2.211 -        } else {
   2.212 -            transformSurfaceY(rz_src, rz_dst, centerx, centery, (int)(sangleinv), (int)(cangleinv), flipx, flipy);
   2.213 +            _transformSurfaceRGBA(src, rz_dst, centerx, centery, (int)sangleinv, (int)cangleinv,
   2.214 +                                  flipx, flipy, smooth);
   2.215          }
   2.216      }
   2.217  
   2.218 -    if (colorKeyAvailable == 1) {
   2.219 -       SDL_SetColorKey(rz_dst, /* SDL_SRCCOLORKEY */ SDL_TRUE | SDL_RLEACCEL, colorkey);
   2.220 -    } else {
   2.221 -       SDL_SetColorKey(rz_dst, SDL_FALSE, 0);
   2.222 +    /* Unlock source surface */
   2.223 +    if (SDL_MUSTLOCK(src)) {
   2.224 +        SDL_UnlockSurface(src);
   2.225      }
   2.226  
   2.227 -    /* copy alpha mod, color mod, and blend mode */
   2.228 -    {
   2.229 -      SDL_BlendMode blendMode;
   2.230 -      Uint8 alphaMod, cr, cg, cb;
   2.231 -      SDL_GetSurfaceAlphaMod(src, &alphaMod);
   2.232 -      SDL_GetSurfaceBlendMode(src, &blendMode);
   2.233 -      SDL_GetSurfaceColorMod(src, &cr, &cg, &cb);
   2.234 -      SDL_SetSurfaceAlphaMod(rz_dst, alphaMod);
   2.235 -      SDL_SetSurfaceBlendMode(rz_dst, blendMode);
   2.236 -      SDL_SetSurfaceColorMod(rz_dst, cr, cg, cb);
   2.237 -    }
   2.238 -
   2.239 -    /*
   2.240 -    * Unlock source surface
   2.241 -    */
   2.242 -    if (SDL_MUSTLOCK(rz_src)) {
   2.243 -        SDL_UnlockSurface(rz_src);
   2.244 -    }
   2.245 -
   2.246 -    /*
   2.247 -    * Cleanup temp surface
   2.248 -    */
   2.249 -    if (rz_src != src) {
   2.250 -        SDL_FreeSurface(rz_src);
   2.251 -    }
   2.252 -
   2.253 -    /*
   2.254 -    * Return destination surface
   2.255 -    */
   2.256 -    return (rz_dst);
   2.257 +    /* Return rotated surface */
   2.258 +    return rz_dst;
   2.259  }