Using the "new" batch and .info files

Matt Borthwick, Feb. 2, 2001.

Quick-n-dirty rundown

How to use ccdnewfile, ccdsetup, and ccdseries by hand
How to use ccd_batch
An example of how to use ccd_batch
How to get yorick to read .info files

Possibly Asked Questions

(still to come)

The Gory Details

(still to come)

The Brief Introduction to doing things by hand

For those who have used ccdnewfile, ccdseries, etc. before

  • Run ccdsetup as previously.
    • Note that  you are now asked here for some info about the beamline,
    including the sample-to-detector distance, beam-zero location, etc.
    Any time one of these paramters changes, it must be updated by running
    either 'ccdsetup' or 'beamparams'. Running 'beamparams' avoids the
    questions about kinetics mode, slow ADC, etc.
  • Run ccdnewfile as previously.
    • Note that new directories will be made automatically by ccdnewfile.
  • Run ccdseries as previously.
    • ccdseries will  record the first and last frame numbers, the time
    of day, and, if available, the ring current. If ruby is running in remote
    mode, the samx, samz, and bstop positions will be recorded as well.
    These will be put into a .info file
  • The main new feature: Each ccdseries generates its own .info file.
    • The .info file resides in the same directory as the .imm files. The
    name convention is such that if the series produces images
    sample1_0001.imm through sample1_0100.imm, then the .info file
    will be named sample1_0001-0100.info.

    The file contains information like the series' first and last image numbers,
    the exposure time (preset), the time of day of the scan, the ring current
    during the scan, the parameters set in 'ccdsetup' and 'ccdnewfile', and
    the samx, samz, and bstop positions. The last are available only if
    ruby is being run remotely by opal.

    It is a simple ASCII file, so it should make sense when looked at with
    'more', 'vi', 'emacs', or another viewer.

    For the new user

    Once the beamline is set up for your measurement, do the following in spec on opal,
     
  • Run 'ccdsetup'
    • This will ask a series of questions about the CCD in use, as well as
    some physical parameters of the beamline. Answer "yes" when asked if you
    want to set the beamline paramters. These values must be updated any time
    they change. The parameters involving "slit width" refer to the slits which
    define the beam. The beam-zero coordinates refer to the pixel on the CCD
    which is in line with the direct beam (scattering vector = 0).
     
  • Run 'ccdroi'
    • This will ask about what region of the CCD to use.
  • Run 'ccdnewfile'
    • (1) Pick a "parent" directory in which to put your files (e.g. /opal1/feb2001/feb02/)
    (2) The"child" directory should be specific to the current sample (e.g. sample1_n5)
         would correspond to sample #1 at -5 degrees C).
    (3) The filename (e.g. sample1_n5_) is used for each individual image, along with
        a number unique to each image
    (4) The suffix is typically .imm.
    (5) The names of the images in this example would be:
        /opal1/feb2001/feb02/sample1_n5/sample1_n5_0001.imm
        /opal1/feb2001/feb02/sample1_n5/sample1_n5_0002.imm
        /opal1/feb2001/feb02/sample1_n5/sample1_n5_0003.imm
        . . . and so on.
  • If opal is not already remotely controlling ruby, you may wish to start it doing so.
    • This will allow you to control both the CCD and some relevant motors from a
    single instance of spec. Run 'server' on ruby, followed by 'ruby_on' on opal.
    See this page for details.

    The motors are samx and samz (sample position), and bstop (beamstop position).
    The intensity on the beamstop is also available remotely through the pdbs counter.

  • Run 'ccdseries <number of images> <exposure time [sec]>'
    • This will expose the CCD  a certain number of times for a certain duration each.
    Replace the <phrase>'s above with the actual numbers you want to use.

    It is possible to run ccdseries many times in a row. The image numbers will continue
    to increment until 'ccdnewfile' is run again and the names changed. It is typical to run
    one ccdseries without the beam present  (for dark current subtraction) directly before or
    after running one ccdseries with the beam present (for dark current subtraction).

  • Possibly run 'getflux', or 'ccdseries' again, or start the whole process over again with another sample.
    • getflux measures the photons/sec hitting the beamstop in front of the CCD. The bstop motor often
    must be moved to put the beamstop detector into the correct position for getflux.
  • Each ccdseries generates its own .info file.
    • See the above section on the same topic for details.

    The Brief Introduction to ccd_batch

    This section assumes a passing familiarity with the procedures outlined above.

    The ccd_batch macro is intended to automate processes like, {ccdseries (dark); ccdseries (data); getflux}, which are usually repeated many times during a set of measurements.  It also allows the computer to record a larger set of parameters relevant to the scans, so that later analysis is even further simplified.
     

  • Create a .do file on opal.
    • Having spec perform several different ccdseries or getflux commands in sequence requires the
    user to supply beforehandsome additional information. That information, along with one or more
    ccd_batch commands, is entered into a .do file. The .do files are ASCII text, and are typically
    stored in opal:/home1/user1/batches/.

    An example.do file is available, with an explanation of what a .do file should contain. It can be
    used as a template (an ASCII copy is located in opal:/home1/user1/batches/).

  • Run the .do file
    • In spec on opal, 'qdo <filename.do>'

    This will run the sequence of getflux and ccdseries calls. Make sure ruby is in remote mode, if
    necessary, before entering the qdo command.

  • Each instance of 'qdo <filename.do>' generates is own .batchinfo file.
    • This is in addition to the usual .info files that may be generated, in the same directory as the .info and
    .imm files. A .batchinfo file is of the same format as a .info file, and contains all the information present
    in the .info files. It also contains paramters relevant to the particular batch sequence, for example, the
    dark frame numbers and the fluxes recorded.
  • Each instance of 'qdo <filename.do>' generates two log files.
    • These contain records of what went on during the ccd_batch commands run from the .do file. The
    "short" log file (extension .short) contains a brief summary of the most relevant information, and is
    suitable for pasting into a lab book. The "long" file (extension .out) contains the normal screen output
    during the sequence.

    The Example .do File

    How to get yorick to read .info files

    The .info file is a regular ASCII file, and can be read or edited by hand if necessary.

    It is also in a yorick-compatible format, so running #include "sample1_0001-0050.info" in yorick would result in a lot of variables being assigned, such as ndata0, samx, preset (the exposure time), start_time, etc.

    Any given analysis routine may use some or all of these variables. However, the .info file may not contain the exhaustive list of parameters needed by the analysis routine, so some may still need to be entered by hand. For example, using .info files only, one would have to set ndark0 = ndata0 from the dark .info file, followed by reading the data .info file. This particular instance is obviated with a .batchinfo file.
     

    With sumdata (sum_func.i): The parameter naming convention in the .info files is chosen to correspond to the variable naming convention used in sum_func.i. The function,

    read_adata_from_file,"sample1_0001-0050.info",adata
    will fill sum_func's parameter data structure with as much information as can be gleaned from the .info file.
     

    With kinetics (kinetics.i): Typically, one enters all the parameters into a single .i file and then gives the name of that file when the kinetics program asks for it.  Those which are in the .info file can be read directly from the .i file by using the commands,

    require,"kinconvert.i"
    include,"sample1_0001-0050.info",1
    kinconvert
    at the beginning of the .i file. It is important to make sure that the variables set by the .info file will not be inadvertently re-defined somewhere later on in the .i file.

    Between the 'include' command and the 'kinconvert' command, one can re-define variables one wishes to change from the .batchinfo file. For example, if the images have been moved into /data, and the dark frame information needs to be entered (as in the case of a .info file), one could say,

    require,"kinconvert.i"
    include,"/data/sample1_/sample1_0001-0050.info",1
    parent = "/data/"
    ndark0 = 51
    ndark = 20
    kinconvert
    An example .i file to use with the kinetics analysis is available.
     

    NOTE: Everything in this section applies equally to .batchinfo as to .info files except where explicitly noted.