P13: CCD image reduction

P13: CCD image reduction using AstroImageJ

In the previous lecture we saw how to use bias frames, dark frames and flat fields to correct our raw CCD images and prepare them for accurate scientific work, like photometry. In this short practical we will use astroImageJ to process a series of images of M51 taken using the department's 10-inch robotic telescope, ROSA.

AstroImageJ is already installed on the lab PCs. Should you want a copy for your own machine, it is available for download. A detailed and helpful manual is available. If you encounter difficulty, or do not understand a setting, check the manual.

To open AstroImageJ, double-click the file C:\SharedFolder\astrolab\AstroImageJ\ImageJ\AstroImageJ.

The Data

You will need a copy of the data. The data files are located at C:\SharedFolder\astrolab\M51. There are 18 images of M51 in each filter. Also in this directory are three bias frames, three dark frames and three flat fields in each filter.

You can work on the data in the SharedFolder. However, this folder is local to the PC you are working on now. If you want to keep files from this practical to use later, you will need to copy those files to your U: or M: drives. There is just over 100MB of raw data, so make sure you have enough free space before doing this!

Data Processing with astroImageJ

Before you start any data processing job, you should make sure you have the calibration files you need.

Task 0: Open astroImageJ and open the file named "M51_B_001.fit". Look at the FITS header metadata by choosing Edit > FITS Header. Note the filter used and the exposure time. What calibration files will you need for this image? If you're unsure, try and ask.

Close this image and the FITS header window.

Data processing in astroImageJ is performed the data processing window, shown in figure 79. Your next task is to familiarise yourself with this window.

**Figure 79:** The astroImageJ data processing (DP) window, which is used to process CCD images, and to create the master bias, dark and flat frames.

Task 1: Open the data processing window by choosing Plugins > Astronomy > Data Processor, or by clicking the DP icon in the main astroImageJ window. Two windows will open. Close the one titled "DP Coordinate Converter" - we won't need it.

The data processing (DP) window is shown in figure 79. DP operates like a script; it processes selected calibration and science images in a pre-defined way. The DP window is described in full in the astroImageJ manual. We shall only describe the sections of interest here.

The DP window is organized into five sections horizontally and nine sections vertically. Starting from the left, the horizontal sections are:

Control: The control checkboxes determine if master calibration frames are to be built (the Build checkboxes) and which master frames are to be used (the Enable checkboxes). For example, if the Bias Subtraction Build box is checked, as master bias frame will be made. If the bias subtraction Enable box is checked, the master bias will be subtracted from the science frames.
Options: The options checkboxes change the details of how calibration frames are built and used. For example, the ave and med radio buttons control if the master calibration frame is created using a mean or a median.
Directory: The directory boxes define the directory/folder location of each category of image files. The folder icon to the right of the directory boxes can be used to navigate to the appropriate folders, or the folder name can be typed manually.
Filename/Pattern: The filename/pattern boxes specify which files in the directory are to be processed. Wildcard notation can be used to choose all files which match a certain pattern. For example, the pattern "bias*.fit" will match all filenames which start with "bias", have any number of other characters, and end in ".fit". This is very useful for selecting only the bias frames in a certain directory, for example.
Totals: The totals column indicates the number of files found that match the pattern specified in the chosen directory. You can use this to check that you are finding the number of files you expect.

The DP window is also broken into a number of vertical sections:

The science image processing section determines which frames are considered as science frames - these are the frames we wish to de-bias etc. You can select frames by folder, by pattern and also filter based on frame number.
The Bias Subtraction, Dark Subtraction and Flat Division sections determines whether to build or use (enable) the master calibration frames. The frames which make up the master calibration frame, and the name and location of the master frame are set using the Directory and Filename/Pattern boxes.
The Non-linearity, FITS Header Update and Post-Processing sections are not of interest to us in this practical. Make sure they are made inactive by de-selecting the radio boxes in the Control area of these sections.
The Save Calibrated Images determines where our processed images are saved. We can save to a sub-directory of the main science folder and/or add a suffix to our filenames. Delete any text in the suffix box, and type "calib" in the Sub-dir box to save calibrated images to a sub-directory named calib.

Making a master bias

It is possible to create all the calibration frames, and calibrate all our images in one step. Do not do this the first time round. It's easy to get it wrong and end up with a load of useless files. Instead, let's take things step-by-step.

Task 2: In the Control section, make sure everything is un-checked, apart from Build in the Bias Subtraction area. Make sure the Options are set to use a median to build the master bias.

Set both directory sections to the folder containing the raw data.

Enter "bias*.fit" in the top filename/pattern section. This will choose all files which match this pattern, e.g bias1.fit, bias2.fit etc. In the bottom filename/pattern section, type "masterbias.fit" to save a master bias frame with this name.

Check the totals column to ensure you've got your settings right (you should be combining three bias frames) and hit START. A log window will appear describing the steps taken. Open your masterbias.fit frame to check everything looks OK. It should look like figure 80.

What's your mean bias level?

Figure 80: A master bias frame, displayed in astroImageJ. The mean bias level is displayed at the bottom of the window.

Dark Frame

With our master bias built, let's make a dark frame. As a reminder, we must subtract the bias from the individual dark frames, and combine them using the median.

Task 3: We want to adjust our settings so that we do not re-build our master bias, but we do build a master dark from a set of bias-subtracted dark frames. Ensure you check only the following:

Bias Subtraction -> Enable
Dark Subtraction -> Build
Dark Subtraction -> Med
Dark Subtraction -> deBias

The last item makes astroImageJ produce a master dark which is bias subtracted. If you un-check it, astroImageJ produces a master dark which is not bias subtracted, even if Bias Subtraction Subtraction is enabled

Set both directory sections in the Dark Subtraction area to the folder containing the raw data.

Enter "dark*.fit" in the top filename/pattern section. This will choose all files which match this pattern, e.g dark1.fit, dark2.fit etc. In the bottom filename/pattern section, type "masterdark.fit" to save a master bias frame with this name.

Check the totals column to ensure you've got your settings right (you should be combining three dark frames) and hit START. A log window will appear describing the steps taken. Open your masterdark.fit frame to check everything looks OK. It should look like figure 81, and the average dark current level should be around 50 counts. If it is nearer 370 counts, you have messed up the bias subtraction settings.

Find the exposure time in the FITS Header, and work out the dark current in counts/pixel/sec.

Figure 81: A master dark frame displayed in astroImageJ. The mean dark current is around 50 counts/pixel, but several 'hot pixels' have much higher levels of dark current.

Flat Field

Now we need to prepare master flat fields. Since the flat field depends upon the filter used, we need to create a master flat field for each of the B, V and I filters. To start with we will create a flat for the I-band.

The flat field images are only very short exposures (less than a couple of seconds), so the dark current in the images will be tiny (You can check this with the measured dark current from earlier). Hence we will not need to subtract a dark from the flat fields. Instead, we create a master flat field by subtracting a bias from each image, normalising each frame, and combining the normalised frames using a median.

Task 4: Ensure you check only the following:

Bias Subtraction -> Enable
Flat Division -> Build
Flat Division -> Med

For clarity, do not check Build or Enable in the Dark Subtraction area. We do not wish to re-build our master dark, or subtract darks from the flat fields.

Set both directory sections in the Flat Division area to the folder containing the raw data.

Enter "twiflat_I*.fit" in the top filename/pattern section. This will choose all files which match this pattern, e.g twiflat_I1.fit, twiflat_I2.fit etc. In the bottom filename/pattern section, type "masterflat_I.fit" to save a master flat frame with this name.

Check the totals column to ensure you've got your settings right (you should be combining three flat field frames) and hit START. A log window will appear describing the steps taken. Open your masterflat_I.fit frame to check everything looks OK. It should look like figure 82, and the average level should be 1 count.

Once you're happy with the I-band flat field, make flat fields for the B and V-band filters as well.

Figure 82: The master flat field in the I-band filter, displayed in astroImageJ. The mean level is 1 count. The vignetting pattern, flat field grain and dust donuts are clearly visible.

Calibrating science frames

With our master calibration frames in hand, we can proceed to calibrate our science images. We will start with the I-band data. As a reminder, we will need to subtract a bias and a dark from our science image, and then divide by the master flat field for the appropriate filter. Because the dark frames were taken with a 300 second exposure time, and our science frames were taken with a 200 second exposure time, we will need to scale the master dark frame under the assumption that dark current builds up linearly with time.

Task 5: Ensure you check only the following:

Science Image Processing -> Filename Pattern Matching -> Enable
Bias Subtraction -> Enable
Dark Subtraction -> Enable
Dark Subtraction -> Scale
Flat Division -> Enable
Save Calibrated Images -> Enable
Save Calibrated Images -> 16

The last option will save the pixel values in our calibrated files as 16-bit digital numbers. There is no need to use 32-bit numbers, since the CCD has a 16-bit ADC.

In the Sub-dir box in the Save Calibrated Images area, enter "calib", to save our calibrated files to a sub-directory called "calib". Make sure the suffix box is empty.

In the Filename Pattern Matching area, set the directory to the folder containing the raw images. In the Filename/Pattern box, type "M51_I_*.fit". This will match files with names like "M51_I_001.fit", "M51_I_002.fit" etc.

Check the totals column to make sure you have got everything right. You should have 18 science images, and one of each calibration frame. When you're happy that everything is OK, hit START and check the log for errors. Depending on the settings you have, astroImageJ may display the calibrated images as it goes along.

Open the calibrated and un-calibrated versions of M51_I_018.fit, and compare them. Make sure your calibration has worked as you would expect. Your calibrated version should show roughly the same count levels, but have no visible vignetting, and a marked reduction in hot pixels.

When you are done, close the DP window.

Stacking Images

There will be many occasions when you will want to average several calibrated images of the same target. In this case we want to make a pretty colour image, but more usually you will want to perform photometry on a single image with an improved signal-to-noise ratio.

We can stack our images in astroImageJ by loading them into a virtual stack and using the alignment tools, as we did when we used astroImageJ for relative photometry.

Task 6: Load the calibrated I-band images into astroImageJ by choosing File > Import > Image Sequence. Select the calib folder containing the calibrated B, V and I images. When the Sequence Options window appears, we will need to enter "M51_I" into the File name contains box, so that we only load the I-band images into a sequence. Make sure you tick the Use Virtual Stack checkbox

Scroll through the images in your sequence. You will notice they are not perfectly aligned, due to imperfections in telescope tracking and autoguiding.

Align the images by choosing Process > Align stack using WCS or apertures... This brings up the stack aligner window. Make sure the Use previous N apertures box and the Align only to whole pixels boxes are unchecked. Make sure the Show help panel box is checked.

The radii of the object aperture and sky annuli are not too important, but you might want to make sure the object radius is set to a reasonable value, since if the star lies outside this radius, AstroImageJ will not be able to calculate the image shift. Set this value to somewhere in the range 15-20.

Click OK to begin selecting stars to use for alignment. Left click on a star to add an aperture around it. Add two or three bright stars to use for alignment and press Enter or right-click to begin the alignment procedure.

AstroImageJ will now look for a star within each aperture, and measure the centroid of those stars. For each image, it uses the centroid of the stars to estimate the image shift, and saves an aligned version of the image into a subdirectory called aligned.

When AstroImageJ has finished, close the photometry window, and load the aligned images in as an Image Sequence. Once you've loaded the aligned images into the stack, hit the button and check the alignment. Make sure all 18 images have been successfully aligned!

Finally, if you are happy with your alignment, select the main astroImageJ window and choose Image > Stacks > Z Project... to combine all the images in a stack to a single image. Choose Median as the Projection type and click OK.

Save the aligned I-band image as a fits image (File > Save Image/Slice as FITS) using the name "median_I.fit", and repeat the whole process for the B and I-band images.

Making a colour image

At this point you should have a single image for each filter, which is the result of median combining each of the calibrated images in that filter. For something like an open cluster project, these images would be a great starting point for your absolute photometry. We're going to reward ourselves for all our hard work by making a colour image of M51.

Task 7: Follow the instructions for task 6 to align the three images in each filter.

Open the three aligned images in a virtual stack. Combine them into a colour image by choosing Color > Make Composite color image. A box will appear; use the drop-down menu to select Display Mode: Color. A second box will appear, with the title Channels and the check box for Channel 1 selected.

We have to assign the correct colour to each channel. astroImageJ doesn't know in advance which file corresponds to which colour. When you check the box for a given channel the image will be displayed, and you can tell from the filename in the main window which file has been put in which channel. If you need to change them, use the button labelled More>> in the Channels window to choose the appropriate color for each file. You want to place the I-band image in the red channel, the V-band in the green and the B-band in the blue.

Once you’ve assigned the correct colour to each channel, you can see the combined results of your work by changing the display mode from Color to Composite. Ideally the resulting composite image should show relatively white stars and relatively black sky. You can go back to Color mode, and adjust the grayscale sliders (or even choose a different display colour for a given master frame) and remake your composite image.

To get a nice looking image will take some effort here!

To save the combined channels as a single image use the More>> button and the Convert to RGB option. This will open a new window with a single, combined colour image. Then you can save the combined image in any number of formats using the file menu.

With a bit of work, you can re-produce the image shown at the end of the previous lecture.