Data


I’ve recently returned from a trip to Socorro, NM where I was working with data from the Very Large Array (recently renamed the Jansky Very Large Array or JVLA). Specifically, we had observations of galaxies found in the ALFALFA survey. While walking into the office one morning I saw the following vanity plate on a truck in the parking lot:

ALFALFA: New Mexico's No. 1 Crop

ALFALFA: New Mexico's No. 1 Crop

It was just too perfect because we jokingly refer to the sort of work I was doing in New Mexico as “harvesting ALFALFA”, so of course I whipped out my cell phone to take the picture above. The ALFALFA survey is nearing completion which means we’re starting on the next phase (harvesting ALFALFA) – following up on all the intriguing sources found by ALFALFA. The recent posts from the Undergraduate ALFALFA Team detail part of that effort – confirming detections using the L-Band Wide (LBW) receiver at Arecibo.

This is only part of effort to understand the interesting sources. ALFALFA is great for finding galaxies based on their gas content and figuring out which galaxies are unusual, and hence interesting. Unfortunately, we don’t get any information about the distribution of gas within a galaxy from ALFALFA; the “beam” (or resolving unit) of the Arecibo telescope is larger than the galaxies we observe. This means that if we want to localize and understand the gas within galaxies we need a larger telescope since larger telescopes have smaller beam sizes.

It’s not physically feasible to build a single telescope that is large enough to resolve structure on the scales astronomers are interested in – it would need to be a kilometre or more across! Instead, individual telescopes spread out over those distances work together to function as a single telescope. This is called interferometry and I’ve describe the idea behind it in more detail before. One important aspect of interferometry is that it’s not easy – combining the data from the individual telescopes together to create a single image is tricky (and figuring it out was worth a Nobel prize!).

The main reason I was in Socorro was to get help from experts and figure out the secret tips and tricks for making the best images. That way, as we find more and more sources from ALFALFA that we want to “harvest” with the JVLA we can have a strategy in place for dealing with the data calibration, imaging and analysis, just like we have a method in place for handling ALFALFA data. I’d like to explain those general steps one day, but for now I’m still working on figuring it out!

Advertisements

This post is by Rachel Almodovar, Union College ’15

My name is Rachel Almodovar and I am a freshman astronomy major at Union College. For spring break I went to the Observatory of Arecibo with my astronomy professor. It had been a few years since the last time I had been there during my childhood in Puerto Rico! At that
time I was twelve years old and the observatory fascinated me. I said to my mother that I wanted to work in a place like the observatory someday. And well, to my surprise Prof. Koopmann, my astronomy professor, invited me to go there during spring break and I had the amazing opportunity to do radio astronomy observing. In my first two nights of observing, I learned how to use CIMA, which is the program that allows you to control the observing mode and the setup of the receiver and also the spectrometer; I also learned how to type in the log the information of the sources like the local sidereal time, zenith angle, the scan number and the source number; and I also learned how to use IDL to reduce the data obtained from the on and off position 3 min exposures and how to look at the spectrum and tell if it had been a detection or not.

Rachel on the Arecibo platform

In my third night observing, Prof. Koopmann taught me how to do flagging of data from the ALFALFA survey. Flagging became my favorite thing to do while I was in Arecibo. Flagging is the technique where you have to examine the spectrum recorded in different drifts and polarizations for bad data like GPS and radars and flag (mark) bad data. One also looks in the spectrum for the detection of galaxies, and records them in the information of the drift.

Well, overall it was a amazing and remarkable experience where besides learning a lot of techniques on how to do radio observing and going up to the Gregorian dome and seeing how it all functions, I also got the opportunity to meet and share time and experiences with a lot of wonderful people at Arecibo that inspired me in a unique way to keep learning about what I am passionate about, which is and will always be Astronomy. I will never forget such an extraordinary experience.

(Left to right): Rachel '15, Becky, Halley Darling '13 and Lucas Viani '14

This post is by Martha Haynes, just before the 5th annual Undergraduate ALFALFA Team workshop at Arecibo, Jan 15-18, 2012.

The UAT is gearing up for our annual workshop in Arecibo (no problem getting those of us from upstate New York to travel to Puerto Rico in January!). This is the 5th workshop, so things ought to be pretty routine, right?

Well, not really. First of all, every year we have lots of new undergraduate participants and also a few new faculty ones. And, this year, as the ALFALFA main survey observations come to completion, we are starting to gear up for the 2nd phase of observing for ALFALFA: conducting longer observations at the positions of very interesting sources or ones which are just marginally detected. Among the most exciting (to me, at least) are the HI sources that seem to have no optical counterparts and are not near any galaxies at similar redshift. These are the candidate “dark galaxies”: are they real HI detections or were the ALFALFA observations corrupted by (insidious) man-made radio frequency interference (RFI)? We have identified the more egregious or expected RFI, but it can sometimes fool us. So before we get too excited, we will make some short (but still longer than ALFALFA observes a given target — about 40 seconds in total — that’s where the “fast” in ALFALFA comes from) observations with the L-band wide receiver just to confirm that the ALFALFA detection is real. This program requires a different receiver, observing strategy and reduction software, all of which we get to try out during the workshop. So, some of us may not get much sleep. But, I know I’m ready for a little astro-excitement: “A sleepy astronomer is a happy astronomer.” So, let the hunt for dark galaxies begin!

Stay tuned for more once we get assembled in Puerto Rico this weekend.

Day N

I’ve been done with the first part of the processing for a while, and now it’s time for the final step to become Level 1 data – flagging. Basically, this is the step where I’m “cleaned up” for future work. This step is somewhat time consuming, which is why I’ve had to wait to get here – the people doing the flagging have lots of other work to help keep them busy and haven’t been able to do the flagging as fast as I’m being observed at the telescope.

The main reason for having to go through this step in the reduction process is that people use radio waves for communication. This means that I am full of human-generated radio frequency interference (RFI) in addition to astronomical signals. RFI is not of interest for science and it can cause problems in later reduction steps, so it needs to be ‘flagged’ away. In order to do this, an ALFALFA team member sits down and looks over me and places boxes to mark all the spots that are bad (contain RFI) and should be ignored in future data reduction steps. For each ‘drift’ I contain, there are seven beams with two polarizations, for fourteen total images that need to be examined by an astronomer. Depending on how long a block of time was assigned the night I was observed, I can consist of anywhere from 20-50 drifts, so this becomes a lot of images to be examined. Below is a screenshot of an example image from this process. This is one of the fourteen images for a single drift. (Click for a larger image.)

View of data flagging

View of data flagging

This is a ‘good’ picture of the flagging process – I’m being very well behaved in this data set. (There are plenty of examples where I’m not so nice that I’m sure will be shared in the future.) The horizontal axis is channel number, or frequency. The left end is the lowest frequency observed and the right end the highest. The vertical axis is time, from 0 to 599 seconds, encompassing the length of one drift. (For the ALFALFA drift-scan survey mode, time corresponds to a position in right ascension and the known beam position for the night gives a declination.) The two boxes on the edges (numbers 0 and 1) are the result of the bandpass edges having a lower response (see the first picture in this post), which generates poor data quality, and not of human-generated RFI. In this picture, there’s actually very little RFI present. You can see a skinny box to the left labelled with the number 5. This box is where the FAA radar for the San Juan airport shows up. It is barely present here, which means that the airport has decided to use their radar at a lower frequency, outside of the range observed for ALFALFA. The other prominent feature is the bright signal to the right. This isn’t flagged with a box because it’s not RFI – it’s our Galaxy! This is a real signal, so I don’t want people to remove it from me. While this is a good picture of me, it’s also a little boring; you can see bright galaxies at this step, but there aren’t any in this picture.

Now that I’ve been flagged, I’m Level 1 data. Next up, I start processing to become Level 2 data. These steps will be about creating ‘grids’ – this is the format I’m put into for source extraction, the end goal.

Day 1, continued:

Last time I talked about my initial processing. Part of this was the ‘bpd’ process in which I was bandpass corrected. I have a few pictures that I thought I would share.

The first picture is from before I was bandpass corrected. You can see that the response (vertical axis, measured in temperature) varies with frequency (the channel number). Ideally, I should have the same response everywhere so that it’s easy to tell how strong a signal is, no matter what channel it falls in. You can see that the bandpass edges are particularly bad and my response falls off quickly. In order to have the best data quality possible, the bandpass response needs to be divided out; areas of high response are weighted down and areas of low response are weighted up. Now, I don’t want to lose any of my information which means we have to be a little careful. That strong response you see in the middle is real – it’s hydrogen in our own Galaxy! I don’t want to lose that, so we interpolate around the Galactic hydrogen, fitting a smooth function over the extent of the signal.

Before bandpass correction

Before bandpass correction

This is a picture of me after I’ve been bpd-ed. You can see that my response is now flat across all channels and that the Galactic hydrogen is still visible. This means I’m all ready to go for the next step.

After bandpass correction

After bandpass correction

Day 1:

Last night I was observed as data for the ALFALFA survey.  The observer wrote all about it here.  This morning, before I even had a chance to get used to being data, I started being transferred from Arecibo (warm and sunny) to Cornell University (not so warm and sunny).  It takes a while for me to arrive fully – during the night I grew by about 1 Gb per hour, so there is a lot of me to travel along the internet trunks from Puerto Rico to Ithaca.

Finally, there’s a whole copy of me in Ithaca.  I find out that I’m considered Level 0 data right now. The first thing that is going to happen to me is that I’ll be processed to become Level 1 data.  This process actually takes a while, but the first part will happen today.

The first thing that happens is I’m calibrated.  This is where I go from being in units measured by the telescope (a ‘temperature’) to units used by astronomers (the Janksy). Since a calibrator of known signal was fired every ten minutes last night during the observing, the records of the calibrator can be used to convert from telescope units to the flux units of interest.

After I’m calibrated, it’s time for ‘bpd’. This step is a bandpass correction. The filter on the telescope that allows the frequencies of interest through isn’t exactly flat. During this step, any frequency-dependent response I may have is removed. My fluxes are scaled so that the same signal strength on the sky corresponds to the same flux in the data file.

Now I’ll wait for the next step, flagging, after which I’ll be Level 1 data.