huibintemaspampipeline
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
huibintemaspampipeline [2017/06/27 09:16] – [Combining multiple LTA files of one observation] huibintema | huibintemaspampipeline [2020/10/05 17:46] (current) – [EXPERIMENTAL: Processing uGMRT wideband data] huibintema | ||
---|---|---|---|
Line 23: | Line 23: | ||
Then we derive calibration and flagging information from the primary calibrator(s), | Then we derive calibration and flagging information from the primary calibrator(s), | ||
<code python> | <code python> | ||
+ | uvfits_file_name = " | ||
pre_calibrate_targets( uvfits_file_name, | pre_calibrate_targets( uvfits_file_name, | ||
</ | </ | ||
Line 40: | Line 41: | ||
./ | ./ | ||
- | There' | + | The pipeline run can be summarized by typing: |
+ | <code python> | ||
+ | summarize_spam_log( " | ||
+ | </ | ||
+ | |||
+ | |||
+ | There' | ||
----- | ----- | ||
Line 116: | Line 123: | ||
gv ./ | gv ./ | ||
</ | </ | ||
- | Select a channel range over which the bandpass phases are well-behaved (linear) for most antennas, | + | Select a channel range over which the bandpass phases are well-behaved (approximately |
+ | {{ : | ||
+ | |||
+ | Note down the lower- and upper-limit of the channel range. Then re-run | ||
<code python> | <code python> | ||
- | channel_range = [ 10, 225 ] # example channel range to keep | + | channel_range = [ 150, 425 ] # example channel range to keep |
pre_calibrate_targets( uvfits_file_name, | pre_calibrate_targets( uvfits_file_name, | ||
+ | </ | ||
+ | |||
+ | ----- | ||
+ | |||
+ | ==== Old hardware-correlator observations ==== | ||
+ | |||
+ | If you downloaded data from cycle 17 or earlier, it is likely correlated using the GMRT Hardware Backend (GHB; a.k.a. the hardware correlator). For frequencies of 325 MHz and higher, the 32 MHz bandwidth is typically split over an upper-side band (USB) and lower-side band (LSB), both captured in separate LTA files (typical extensions are .lta and .ltb). In SPAM, both sideband LTA files need to be pre-processed separately: | ||
+ | <code python> | ||
+ | convert_lta_to_uvfits( lta_file_name ) | ||
+ | convert_lta_to_uvfits( ltb_file_name ) | ||
+ | </ | ||
+ | The next step is run using the // | ||
+ | <code python> | ||
+ | pre_calibrate_targets( uvfits_file_name_lta, | ||
+ | pre_calibrate_targets( uvfits_file_name_ltb, | ||
+ | </ | ||
+ | The resulting UVFITS files for USB and LSB per target can be combined: | ||
+ | <code python> | ||
+ | uvfits_file_name_usb = " | ||
+ | uvfits_file_name_lsb = " | ||
+ | uvfits_file_name = " | ||
+ | combine_usb_lsb( uvfits_file_name_usb, | ||
+ | </ | ||
+ | The output UVFITS file can be processed further in the main pipeline. | ||
+ | |||
+ | Regarding the main pipeline, there are two options that may be relevant to get to better results. The first option related to the situation explained above, where two sidebands (USB and LSB) are joined together to cover 32 MHz of bandwidth. In that case, it may help to turn on an image-based flagging option that treats the joined USB and LSB separately. Reason for this is that the USB and LSB have separate signal chains, and thus there can be system problems that relate only to one of the two sidebands. | ||
+ | <code python> | ||
+ | process_target( target_uvfits_file_name, | ||
+ | </ | ||
+ | |||
+ | The second option is to turn on baseline-based calibration, | ||
+ | <code python> | ||
+ | process_target( target_uvfits_file_name, | ||
</ | </ | ||
Line 142: | Line 185: | ||
----- | ----- | ||
+ | |||
+ | ==== EXPERIMENTAL: | ||
+ | |||
+ | SPAM has some options to process uGMRT wideband data. SPAM does not support the processing of large fractional bandwidths (df/ | ||
+ | |||
+ | The first step is to convert LTA to UVFITS format: | ||
+ | <code python> | ||
+ | lta_file_name = " | ||
+ | convert_lta_to_uvfits( lta_file_name ) | ||
+ | </ | ||
+ | Next, we split the UVFITS file into smaller frequency chunks (subbands): | ||
+ | <code python> | ||
+ | uvfits_file_name = " | ||
+ | split_wideband_uvdata( uvfits_file_name ) | ||
+ | </ | ||
+ | The width of the frequency chunks is automatically set to a sensible value. The resulting 4 or 6 UVFITS files are also located in the fits subdirectory and named " | ||
+ | |||
+ | From here, each frequency chunk is processed independently in a similar fashion as a narrow-band GMRT observations, | ||
+ | < | ||
+ | uvfits_file_name = " | ||
+ | reference_frequency = 450.e6 | ||
+ | pre_calibrate_wideband_targets( uvfits_file_name, | ||
+ | </ | ||
+ | Setting a fixed reference frequency ensures that the frequency averaging of all frequency chunks is the same, which is important when jointly imaging the SPAM output visibilities later. Sensible values seem to be: | ||
+ | < | ||
+ | reference_frequency = 450.e6 | ||
+ | reference_frequency = 650.e6 | ||
+ | </ | ||
+ | The pre-calibrated visibilities per target are located in the fits subdirectory and are named per subband. | ||
+ | |||
+ | Next comes the SPAM main pipeline run. This is best done in separate project directories per subband. If possible, use a good, single reference sky model for all runs. For example, this reference model can be obtained from first running SPAM on the narrow-band GMRT (GSB) data that was recorded alongside the uGMRT wideband data, and extracting a sky model from the final SP2B image using PyBDSF. | ||
+ | < | ||
+ | target_uvfits_file_name = " | ||
+ | UVFITS" | ||
+ | catalog_name = "< | ||
+ | catalog = read_pybdsm_ascii_catalog( catalog_name ) | ||
+ | source_list = create_source_list_from_catalog( catalog ) | ||
+ | resolution = 10. # representative resolution of model image in arcsec | ||
+ | process_wideband_target( target_uvfits_file_name, | ||
+ | </ | ||
+ | |||
+ | If all went well, each SPAM pipeline run on a subband yielded a final image and a calibrated visibility data set (.SP2B.CAL.RR.UVFITS). For use in WSClean, the calibrated visibilities all need to be collected in one directory and converted into measurement sets using CASA. Then WSClean can be used to do a final wideband imaging run. Here is an example: | ||
+ | < | ||
+ | wsclean -weight briggs 0 -pol RR -size 5000 5000 -scale 1.5asec -niter 15000 -auto-threshold 0.5 -auto-mask 3 -gain 0.25 -mgain 0.8 -weighting-rank-filter 3 -join-channels -channels-out 6 -j 8 -mem 80 -name SOURCE_UGMRT3 SOURCE_UGMRT3-01.MS SOURCE_UGMRT3-02.MS SOURCE_UGMRT3-03.MS SOURCE_UGMRT3-04.MS SOURCE_UGMRT3-05.MS SOURCE_UGMRT3-06.MS | ||
+ | </ | ||
+ | Here, SOURCE_UGMRT3-0x.MS are the input measurement sets as produced by CASA. Make sure that " | ||
+ | |||
+ | |||
+ | |||
+ | ----- | ||
+ | |||
+ | Feedback: [[intema@strw.leidenuniv.nl|Click here]] | ||
huibintemaspampipeline.1498547763.txt.gz · Last modified: 2017/06/27 09:16 by huibintema