User Tools

Site Tools


huibintemaspampipeline

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
huibintemaspampipeline [2019/02/07 11:14] – [Basic pipeline run] huibintemahuibintemaspampipeline [2020/10/05 17:46] (current) – [EXPERIMENTAL: Processing uGMRT wideband data] huibintema
Line 154: Line 154:
 </code> </code>
 The output UVFITS file can be processed further in the main pipeline. The output UVFITS file can be processed further in the main pipeline.
 +
 +Regarding the main pipeline, there are two options that may be relevant to get to better results. The first option related to the situation explained above, where two sidebands (USB and LSB) are joined together to cover 32 MHz of bandwidth. In that case, it may help to turn on an image-based flagging option that treats the joined USB and LSB separately. Reason for this is that the USB and LSB have separate signal chains, and thus there can be system problems that relate only to one of the two sidebands.
 +<code python>
 +process_target( target_uvfits_file_name, flag_image_usb_lsb = True )
 +</code>
 +
 +The second option is to turn on baseline-based calibration, BUT ONLY AFTER VERY CAREFUL CONSIDERATION. In an ideal world, baseline-based calibration would never be necessary. However, I have witnessed situations using hardware-correlator data in which residual gain errors on baselines did not seem to be solely antenna-based. Baseline-based calibration is something to try as a last resort, and can be switched on in the pipeline as shown below. Please note that this is implemented under strict limitations: baseline-calibration is applied only in the final stages of the processing, and is determined on a per-observation base (meaning only one correction per baseline per observe session).
 +<code python>
 +process_target( target_uvfits_file_name, do_blcal = True )
 +</code>
  
 ----- -----
Line 176: Line 186:
 ----- -----
  
-Feedback: [[huib.intema@curtin.edu.au|Click here]]+==== EXPERIMENTAL: Processing uGMRT wideband data ==== 
 + 
 +SPAM has some options to process uGMRT wideband data. SPAM does not support the processing of large fractional bandwidths (df/f>~0.2) in one run, but instead the bandwidth can be split up into smaller chunks (subbands) that can be processed independently. If done carefully, the calibrated output visibilities of SPAM pipeline runs on multiple subbands can be jointly imaged with a wideband imager (WSClean) as a final step. This approach has produced good results when applied on bands 3 (250-500 MHz) and band 4 (550-850 MHz) data. Processing band 2 data (120-250 MHz) has given mixed results. Since there is not yet a good way to apply the wideband primary beam corrections, this approach only works for observations where the target's angular size is small with respect to the primary beam size (size <~ 0.1 * FWHM). 
 + 
 +The first step is to convert LTA to UVFITS format: 
 +<code python> 
 +lta_file_name = "./<project name>_<observe date>.lta" 
 +convert_lta_to_uvfits( lta_file_name ) 
 +</code> 
 +Next, we split the UVFITS file into smaller frequency chunks (subbands): 
 +<code python> 
 +uvfits_file_name = "./fits/<project name>_<observe date>.UVFITS" 
 +split_wideband_uvdata( uvfits_file_name ) 
 +</code> 
 +The width of the frequency chunks is automatically set to a sensible value. The resulting 4 or 6 UVFITS files are also located in the fits subdirectory and named "./fits/<project name>_<observe date>.BAND<xx>.UVFITS", where <xx> is a 2-digit simple counter starting at 1 for the lowest frequency chunk.  
 + 
 +From here, each frequency chunk is processed independently in a similar fashion as a narrow-band GMRT observations, but using slightly different function calls. The first step is the pre-calibration: 
 +<code> 
 +uvfits_file_name = "./fits/<project name>_<observe date>.lta.BAND<xx>.UVFITS" 
 +reference_frequency = 450.e6 
 +pre_calibrate_wideband_targets( uvfits_file_name, flags_file_name = lta_file_name + "##*.FLAGS*", reference_frequency = reference_frequency ) 
 +</code> 
 +Setting a fixed reference frequency ensures that the frequency averaging of all frequency chunks is the same, which is important when jointly imaging the SPAM output visibilities later. Sensible values seem to be: 
 +<code> 
 +reference_frequency = 450.e6  # for uGMRT band 3 observations 
 +reference_frequency = 650.e6  # for uGMRT band 4 observations 
 +</code> 
 +The pre-calibrated visibilities per target are located in the fits subdirectory and are named per subband. 
 + 
 +Next comes the SPAM main pipeline run. This is best done in separate project directories per subband. If possible, use a good, single reference sky model for all runs. For example, this reference model can be obtained from first running SPAM on the narrow-band GMRT (GSB) data that was recorded alongside the uGMRT wideband data, and extracting a sky model from the final SP2B image using PyBDSF. 
 +<code> 
 +target_uvfits_file_name = "./fits/<source_name>_UGMRT<band>-<subband>_<observation_reference_date>_<polarization(s)>_<sideband>
 +UVFITS" 
 +catalog_name = "<narrowband_project_dir>/fits/<source_name>.SP2B.PBCOR.pybdsm.gaul" 
 +catalog = read_pybdsm_ascii_catalog( catalog_name ) 
 +source_list = create_source_list_from_catalog( catalog )                
 +resolution = 10. # representative resolution of model image in arcsec 
 +process_wideband_target( target_uvfits_file_name, model_source_list = source_list, model_resolution = resolution ) 
 +</code> 
 + 
 +If all went well, each SPAM pipeline run on a subband yielded a final image and a calibrated visibility data set (.SP2B.CAL.RR.UVFITS). For use in WSClean, the calibrated visibilities all need to be collected in one directory and converted into measurement sets using CASA. Then WSClean can be used to do a final wideband imaging run. Here is an example: 
 +<code> 
 +wsclean -weight briggs 0 -pol RR -size 5000 5000 -scale 1.5asec -niter 15000 -auto-threshold 0.5 -auto-mask 3 -gain 0.25 -mgain 0.8 -weighting-rank-filter 3 -join-channels -channels-out 6 -j 8 -mem 80 -name SOURCE_UGMRT3 SOURCE_UGMRT3-01.MS SOURCE_UGMRT3-02.MS SOURCE_UGMRT3-03.MS SOURCE_UGMRT3-04.MS SOURCE_UGMRT3-05.MS SOURCE_UGMRT3-06.MS 
 +</code> 
 +Here, SOURCE_UGMRT3-0x.MS are the input measurement sets as produced by CASA. Make sure that "-scale" is set to the proper pixel scale, and that "-channels-out" matches the number of input measurement sets. WSClean produces a lot of output files, but probably you are interested in the final wideband image "*-MFS-image.fits"
 + 
 + 
 + 
 +----- 
 + 
 +Feedback: [[intema@strw.leidenuniv.nl|Click here]]
  
huibintemaspampipeline.1549534459.txt.gz · Last modified: 2019/02/07 11:14 by huibintema

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki