Note: Debaying is required for any output from any colour camera sensor when used in RAW mode. This article applies to deepsky images (.FITS format) and for solar system/planetary videos (.SER format).

Firstly, what is Debayering?

Colour CMOS camera sensors usually have four colour channels (two greens, one red and one blue). The sensor has a sensitive layer of pixels which measure the “intensity” of light only, and converts it to numbers. This sensitive surface can be considered “monochrome” or black and white only. It does not record colour information. However, just above this sensitive layer of pixels is a grid of tiny filters, one for each pixel. These filters selectively allow allow red, green or blue light through. So a “green” pixel is the same as a “red” pixel – but the filter above it is red or green. This is referred to as the Bayer Matrix or Bayer Filter (link to Wikipedia).

The order or sequence in which these tiny filters appear is usually different for each sensor, for example RGGB, GRBG, GBRG, BGGR…

Here’s a picture from the Wiki article:

When the image comes from the sensor into the computer, it is composed of one big monochrome image (it has white, black and all the levels of grey inbetween).

The computer displaying the image (and your image processing software) does not know which pixels have, say, a red filter above them, or which have a blue or green filter above them. It’s just a big mono image.

Therefore, when imaging and creating an RGB image from a .FITS Image File or .SER Video file in RAW mode, we need to tell the computer which of these black, grey and white pixels have red, green and blue filters associated with them. Only then can they form a meaningful “RGB” colour image, with the usual RGB (Red Green Blue) colour channels.

Lucky for us, this information is described as the “Bayer Matrix” for that particular sensor, which just a formula describing the pixel filter sequence, e.g. for example RGGB or GBRG, GBBR.

OK, so at what stage do I Debayer when processing my colour images?

When using a dedicated astronomy cameras we always capture images or video in RAW mode (for best signal purity), and we stack many sub-frames (or “subs”) to get more detail in the image, to reduce noise, and to improve dynamic range so we can use all sorts of nice processing techniques to show amazing detail. The more subs you stack the better.

Various stacking software is used, like Autostakkert or Registax for stacking video, and for deepsky we use software like Pixinsight, Deepsky Stacker, Astro Art and so-on.

However we cannot stack RAW files from a colour camera!

We must first apply a “Debayer Matrix” to the sub-frame images. Only then we can stack it with other sub-frames and finally process it to bring out the details.

Why? Because, when we take sub-frames, there is usually a little drift or variation between each frame. A star or small detail for example, will never land on the exact same pixel every time. The pixels are too small. A breath of wind, or a wobble in the atmosphere (heat haze or Seeing), vibrations from the tracking drive, or minute variations in the tracking of your mount (Periodic error) will cause that star to move a few pixels to one side and a few to the other (sometimes a lot!). Stacking software compensates for these changes and tries to drop a star in frame #1 on top of the same star in frame #2, but there is always unavoidable variation between sub-frames, and no two subs are exactly alike.

So if we did not Debayer the images before stacking them, the star or feature would land on several adjacent pixels, and when stacked, it would be composed of many different (and wrong) colours. You would get a strange rainbow effect across the image, and it would make no sense.

Here are some important things to remember about Debayering:

1. AltairCapture and SharpCap software does not influence the Bayer sequence or pattern. It is always the same for each sensor! You can’t change the position of those tiny RGB filters on your sensor after all – they are hardware not software!

2. Processing software such as Pixinsight, Deepsky Stacker, Astro Art and so-on all have different default Debayering sequences. Therefore you must manually enter and check the Debayering pattern settings BEFORE stacking your images. You can check if it’s correct by trying to Debayer one RAW image and then running an auto colour balance on it (to remove excess green caused by that extra green sensitivity) and perhaps a stretch to show the detail. If the colour in the image appears normal, then you have the right settings. If not, try again – there are only 4 possible combinations: RGGB, GRBG, GBRG, BGGR.

3. If you “calibrate” your RAW files to remove read noise patterns, vignetting, dust shadows, fixed pattern noise and amp-glow, i.e. Subtract Bias frames, Subtract darkframes, and Subtract flat frames (which are beyond the scope of this article) then you need to do all calibration FIRST while the files are still RAW un-Debayered! Otherwise, you will get some very strange results when you stack and combine the resulting images. Why? Because the pixels in the stacked images will all have the same Debayer formula applied to them, and the result would once again, be different colours when the stacking software attempts to align each frame, to correct for guiding errors and so-on.

4. .FITS format is where confusion arises – for instance PixInsight and AstroArt disagree over the bayer format of an image. Take an image with the Hypercam 183C in RAW mode. To debayer it correctly in PixInsight, you must select RGGB, but in AstroArt you have to select GBRG. This is because .FITS files are read starting bottom left instead of top left for most image files. AstroArt and PixInsight have different ways of dealing with this leading to their different take on Bayer pattern.

5. The .SER video format specification includes a way to specify the Bayer pattern and all the software that reads it (and writes it) and usually gets it right, so if you are doing high speed imaging in .SER format you don’t have much to worry about when using the “Autodetect” feature in Autostakkert. But if there’s a problem and your images come out all coloured, you can still manually select the various Debayering options, RGGB, GRBG, GBRG, BGGR.

6. Using flip in an image capture program will change the Debayer pattern!

7.  Deep Sky Stacker does not debayer RAW8 (it works with RAW16). This can be addressed by using the debayer functionality in PIPP.