c# - How do I draw anti-aliased holes in a bitmap

I have an artillery game (hobby-learning project) and when the projectile hits it leaves a hole in the ground. I want this hole to have antialiased edges. I'm using System.Drawing for this. I've tried with clipping paths, and drawing with a transparent color using gfx.CompositingMode = CompositingMode.SourceCopy, but it gives me the same result. If I draw a circle with a solid color it works fine, but I need a hole, a circle with 0 alpha values.

I have enabled these but they work only with solid colors:

gfx.CompositingQuality = CompositingQuality.HighQuality;
gfx.InterpolationMode = InterpolationMode.HighQualityBicubic;
gfx.SmoothingMode = SmoothingMode.AntiAlias;

In the two pictures consider black as being transparent.

This is what I have (zoomed in):

And what I need is something like this (made with photoshop):

This will be just a visual effect, in code for collision detection I still treat everything with alpha > 128 as solid.

Edit: I'm usink OpenTK for this game. But for this question I think it doesn't really matter probably it is gdi+ related.

Answer

As you have noticed clipping or regions in GDI+ are not antialiased as they are both pixel based where a given pixel can either be completely included or excluded.

To do what you want in GDI+ you can apply an alpha mask to the image yourself. Basically you fill a rectangle of the size of the area affected with black color and then draw the hole with white using the high quality settings for Antialiasing and smoothing. Then you apply the resulting image as an alpha mask to the original image. (which means that you re-calculate the alpha value of the pixels involved based on the black/white image where completely white pixels will be transparent) To do this quickly you should use the fast pointer access to the pixel data via LockBits()..

Here is a GDI+ example. This will only work if the images are 32bppArgb format and both are the same size - if you only need to apply an alpha mask to a small portion of the original image I would suggest you adjust the code to do that.

public static void ApplyAlphaMask(Bitmap bmp, Bitmap alphaMaskImage)
{    
  int width = bmp.Width;
  int height = bmp.Height;

  BitmapData dataAlphaMask = alphaMaskImage.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
  try
  {
    BitmapData data = bmp.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);
    try

    {
      unsafe //using pointer requires the unsafe keyword
      {
        byte* pData0Mask = (byte*)dataAlphaMask.Scan0;
        byte* pData0 = (byte*)data.Scan0;

        for (int x = 0; x < width; x++)
        {
          for (int y = 0; y < height; y++)
          {

            byte* pData = pData0 + (y * data.Stride) + (x * 4);
            byte* pDataMask = pData0Mask + (y * dataAlphaMask.Stride) + (x * 4);

            byte maskBlue = pDataMask[0];
            byte maskGreen = pDataMask[1];
            byte maskRed = pDataMask[2];

            //the closer the color is to black the more opaque it will be.
            byte alpha = (byte)(255 - (maskRed + maskBlue + maskGreen) / 3);


            //respect the original alpha value
            byte originalAlpha = pData[3];
            pData[3] = (byte)(((float)(alpha * originalAlpha)) / 255f);
          }
        }
      }
    }
    finally
    {
      bmp.UnlockBits(data);

    }
  }
  finally
  {
    alphaMaskImage.UnlockBits(dataAlphaMask);
  }
}