furanku From the Bax Group at the National Institutes of Health ...
NMRPipe Spectral Processing and Analysis System
Tutorial I: Introduction to Processing with NMRPipe
by Gabriel Cornilescu

DESCRIPTION

WORKFLOW

Spectrometer format conversion

Inspecting the time domain data

Trial processing

Inspecting the processed result

Final processing

Processing Tips

Automated peak detection

Hard copy plotting

Strip Display for 3D data

DESCRIPTION

NMRPipe is a collection of UNIX-based programs for multidimensional spectral processing and analysis. The tools of the program are operated by use of graphical interfaces, and also by UNIX shell scripts and TCL scripts, and this reliance on scripting makes the software extremely flexible and customizable. Some examples of things NMRPipe can do:

  • Process, rephase and display multidimensional data, including options for Maximum Entropy Reconstruction and Linear Prediction (nmrPipe nmrDraw).
  • Algebraic combination of spectra and FIDs (addNMR).
  • Efficient Automated Peak Detection for 1D-4D (nmrDraw nmrWish).
  • Extensive Line-Shape fitting functions, including direct fitting of pseudo-3D data such as relaxation series or J-modulated series autoFit.tcl showEvolve.tcl fitXY.tcl relax.tar.Z jmod.tar.Z).
  • General purpose least-squares fitting utility with user-defined functions and Monte Carlo error analysis (fitXY.tcl).
  • Summary statistics of data from spectra or text tables (mstat, plotTab.tcl).
  • Principal Component Analysis of Spectral Data (pcaNMR)
  • Create simulated time or frequency domain data (autoFit.tcl simSpecND simTimeND sim3d.tar.Z), including mock triple resonance from chemical shift tables (cs2pk.tcl).
  • Create and draw strip plots, projections, and overlays (scroll.tcl stripPlot.tcl proj3D.tcl view2D.tcl).
  • Predict protein backbone angles based on backbone chemical shifts (talos.tcl vina.tcl rama.tcl).
  • Calculate J-couplings from Karplus parameters (nmrWish mfr.tar.Z).
  • Simulate or fit and display Dipolar Couplings (nmrWish DC mfr.tar.Z).
  • Estimate protein alignment tensor parameters from measured dipolar couplings without prior knowledge of the structure. (nmrWish mfr.tcl mfr.tar.Z).
  • Visualize tensor parameters with respect to a PDB file. (rotDC.tcl rotPCS.tcl mfr.tar.Z).
  • List or display Protein PDB backbone and sidechain angles, vizualize ramachandran trajectory for one or more proteins or fragments (angles.tcl dynAngles.tcl scrollRama.tcl mfr.tar.Z dyn.tar.Z).
  • Analyze Protein PDB for H-bonds and secondary structure and turn classification (ss.tcl dyn.tar.Z).
  • Find coordinate or torsion RMSD between two or more structures, form overlay (ov.tcl dyn.tar.Z).
  • Simulated annealing structure calculation, incluing NOEs, J-coupling, torsion restraints, radius of gyration, pseudo-contact shifts, and dipolar couplings (dyn.tar.Z).
  • Search the PDB Database for NMR Parameter Homology, as a tool for quantitative structure analysis (mfr.tar.Z).

The NMRPipe software package consists of a series of standalone programs such as NMRPipe (spectrometer data conversion, advanced multidimensional data processing), NMRDraw (graphical interface for most package tools, interactive processing, script editing interface, time domain data inspection, real-time phasing, automated peak detection, spectral analysis), NMRWish (custom version of Tcl/Tk for spectral analysis, spectral database engine, communication between applications), DYNAMO (Cartesian‑coordinate simulated annealing engine, homology search and assemble molecular fragments consistent with NMR observables, with graphs of energy terms during refinement), TALOS (prediction of protein Phi and Psi backbone dihedral angles using a chemical shift database), ACME (measuring coupling constants from 2D COSY cross peaks), DC (dipolar coupling and chemical shift analysis) and a variety of spectral analysis tools.

The NMRPipe processing program is the central part of a system of tools for multi‑dimensional spectral processing and analysis using UNIX pipes. It applies a succession of spectral processing functions, for instance a one-dimensional Fourier Transform, to a series of spectral data vectors. When used in conjunction with data input and output programs such as xyz2pipe and pipe2xyz, nmrPipe can be used to construct complete multi-dimensional processing schemes. In most cases, nmrPipe processing schemes are constructed as UNIX shell‑scripts, which are text files used to execute nmrPipe and related programs. In practice, these shell-scripts just define the input data source, the list of processing functions to apply, and the output data destination. The shell scripts can be constructed with a UNIX text-editor such as vi, or inside a graphical interface such as nmrDraw.

 

NMRPIPE WORKFLOW

A typical NMRPipe processing session involves the following steps:

  • spectrometer raw data conversion

  • time domain data inspection (commonly performed in nmrDraw)

  • low resolution trial processing (initial phasing, solvent signal suppression)

  • inspection of the processed result, to decide if any additional processing is required, for instance re‑phasing or baseline correction (also performed in nmrDraw)

  • full (high resolution) processing

  • automatic peak detection

 

DATA CONVERSION 

  • Conversion facilities specifically for Varian and Bruker binary time‑domain data are provided, as well as general purpose facilities accommodating most other formats. All data is converted to a common format with a uniform organization of real and imaginary points.
  • The conversion commands can read or write data from a command pipeline rather than a file, allowing such capabilities as conversion of compressed data, conversion of input data dispersed over more than one file, and processing schemes which go directly from spectrometer format to processed result.
  • Dedicated interactive Varian conversion interface reads the “procpar” file and automatically extracts many acquisition parameters.
  • Dedicated interactive Bruker conversion interface reads and interprets pulseprogram and “acq” files to deduce many acquisition parameters. Full compensation for Bruker digital filter format is performed during conversion.
  • All conversion facilities support special options for complex acquisition schemes, including sensitivity-enhanced data (Rance-Kay/Echo:Anti-Echo) and accommodation for interleaved data formats.

The programs bruk2pipe, var2pipe, and bin2pipe are used to convert spectral data, usually unprocessed binary data from a spectrometer or imager, to the nmrPipe format. These programs are implemented as UNIX filters; they read a stream of binary data from a file or pipeline, perform a format conversion, and write the converted data to a file, file series, or pipeline. 

The conversion is usually performed by editing a shell-script which runs a conversion program such as bruk2pipe or var2pipe. The most convenient way of editing the conversion scripts is the Tk graphical interface invoked by the commands bruker or varian.

The bruk2pipe program is used to convert Bruker format "ser" files to the nmrPipe data format. The bruk2pipe program can be used to convert data of one to four dimensions, but 3D and 4D data must have a specific acquisition order (alternating real and imaginary hypercomplex planes) to be converted properly. A complementary command bruker is now also provided; this command activates a graphical interface which can interpret the spectrometer "acq" and "pulseprogram" files to assist in the creation of conversion scripts.

The var2pipe program performs conversions for Varian Unity and INOVA "fid" files. Two-byte integer, four-byte integer, and DSP four-byte floating point data formats are supported. In many cases, var2pipe can select the correct data format automatically by interpreting the binary header of the Varian "fid" file. The var2pipe program can be used to convert data of one to four dimensions, but 3D and 4D conversions are restricted to two kinds of acquisition orders. A complementary command varian is also provided; this command activates a graphical interface which can interpret "procpar" parameter files from the spectrometer to assist in the creation of conversion scripts.

The bin2pipe program performs conversions of four-byte integer or floating point binary files which have a sequential data order, and a fixed number of bytes between adjacent 1D vectors. GE Omega export format, JEOL Alpha, Lambda, EX, and GSX data, Chemagnetics data, SPIFF spectral images, and formats of other processing programs may fall into this category. The bin2pipe program can be used to convert data of one to four dimensions, but the data must have a specific acquisition order. Non‑default acquisition orders can be accommodated via use of nmrPipe macro functions.

Example:

The bruker command will open a window like this:

nmr image

With a mouse click on “Read Parameters” the program will try to locate and read automatically the raw data file (e.g. “ser” or “fid”) in the current directory. If the file is elsewhere its location can be specified in the “Spectrometer Input” pull-down selection box. After specifying its path click again the “Read Parameters” button.

Most of the acquisition specific parameters will be read automatically and displayed in grey selection boxes. The parameters that need user inspection and eventual re-selection or manual input are highlighted in yellow:

nmr image

Note 1: The conversion programs use acquisition parameters contained in the spectrometer’s “acq” (Bruker) or “procpar” (Varian) files, therefore these file should be present in the same directory as the raw spectral data file (“ser” or “fid”).

Note 2: The conversion program utilizes a standard referencing for the water signal relative to TSP according to the temperature specified by user in the graphical interface (if different from the value read automatically from the parameter file). If the spectrometer’s transmitter was set on the water signal this ensures a correct referencing of all spectral dimensions. The program initially reads the temperature parameter from the spectrometer acquisition parameter file, if the actual sample temperature differs from this parameter it should be manually corrected by the user and the parameters should be re-read (with another mouse click on “Read Parameters”) to ensure proper chemical shift referencing in all dimensions. User can also specify if an offset relative to the water signal was used in the experimental setup by manually entering it in the 1H dimension. Re-reading the parameters with this offset correction (and the right sample temperature!) will still provide a correct referencing in all other dimensions.

These programs make it possible to convert and process both real and complex data: this includes data acquired in sequential or simultaneous (complex) mode, as well as indirect detection in the States, TPPI, States-TPPI and Magnitude modes. The interface of the conversion program can also read sensitivity enhanced (or echo-antiecho or Rance gradient mode or Rance-Kay) detection for a particular dimension. Often the sensitivity enhanced type detection is encoded in the pulse sequence in ways that the program cannot recognize automatically, therefore user need to select it manually from the drop-down menu:

nmr image

If the “ser” or “fid” file contains two or more interleaved experiments, the total number of points needs to be manually adjusted (i.e. doubled or multiplied with the number of interleaved data). However, the number of complex points refers to a single experiment and need not to be changed (i.e. is read from the parameter file)

The use of the conversion programs is essentially the same in all cases; the conversion program is invoked with the name of the binary input file, the desired name of the converted output, and a list of relevant acquisition parameters. In addition, there may be one or more parameters describing hardware-specific or format details, such as the byte-order used to record intensities on the spectrometer. In most cases, the conversion programs themselves do not read or deduce acquisition parameters on their own; the user must supply each one explicitly as part of the conversion command.

Since there are many parameters that have to be specified or confirmed manually, these conversion commands are more complicated than those used by other systems. However, once the parameters are established, there are many advantages:

  • Complete multidimensional processing schemes can be constructed without the need to anticipate or explicitly specify data sizes.
  • The dimensions can be processed and re‑processed in any order, with the correct combination of real and imaginary data supplied automatically.
  • Spectral axis calibrations are maintained during all stages of processing, so that processing parameters can be specified in terms of spectral units where desired.

The processing modules can identify processing schemes which seem inconsistent with the current data. The conversion commands can read data from a UNIX command pipeline rather than a file, allowing such capabilities as conversion of compressed data (as via the UNIX commands compress and zcat), or conversion of input data dispersed over more than one file (as via the UNIX command dd).

USING THE CONVERSION PROGRAMS

 In order to understand the use of the conversion programs, it might be helpful to review the details of the nmrPipe and the programs xyz2pipe and pipe2xyz program first.  Details are presented in the nmrPipe and xyz2pipe sections. In addition, once spectral data has been converted, the converted result can be inspected visually with the graphical interface NMRDraw.

 A typical 2D conversion command is shown here, in this case for a Bruker AMX 2D HN-correlated FID with States-mode detection; a description of its contents follows:

 bruk2pipe -in ./ser -bad 0.0 -noaswap -DMX -decim 16 -dspfvs 12  \
-xN                              1024                -yN                  256  \
-xT                               512                  -yT                   128  \
-xMODE                     DQD                -yMODE         Complex  \
-xSW                           8992.806         -ySW               2173.913  \
-xOBS                         600.133           -yOBS             60.818  \
-xCAR                         4.773               -yCAR             118.040  \
-xLAB                         1H                   -yLAB             15N  \
-ndim                           2                      -aq2D              States  \
-out ./test.fid -verb -ov

 The conversion program is usually invoked by a UNIX C-shell script, which is a list of commands in a text-file; the shell-script should always begin with the line:

 #!/bin/csh

 The conversion command itself has an input specification which names the input binary FID file, and an output specification which names the output data file or file series. In addition, the conversion command includes a list of axis‑specific parameters, which have a particular value for each dimension of the input data. For example, each dimension has a spectral width (sweep width), so a conversion command has one sweep width specification for each dimension of the data.

The nmrPipe nomenclature uses the terms X-Axis, Y-Axis, Z-Axis, and A-Axis to refer to the dimensions of the data. Correspondingly, the conversion command uses the prefixes -x, -y, -z, and -a for parameters that pertain to these dimensions. For example, the sweep width arguments are specified by the flags -xSW, -ySW, -zSW, and -aSW.

In the current implementation, the conversion command should include specification of all of the following parameters for each dimension:

Acquisition Mode

This parameter defines the data type of the given dimension, which controls how the real and imaginary parts should be stored and processed. The acquisition mode is usually specified as a keyword. While there are several possible keywords, in most cases it is sufficient to specify either "Real" or "Complex"; other options are described in the ACQUISITION MODES section below. The acquisition modes are specified by the options -xMODE, -yMODE, -zMODE, and -aMODE.

Actual Size

This parameter specifies the actual total number of points (real + imaginary) stored in the input file for a given dimension. In most cases (complex detection, States, and States-TPPI modes) this parameter will be twice the value of the corresponding time-domain size below. The actual sizes are specified by the parameters ‑xN, ‑yN, ‑zN, and ‑aN.

Valid Time-Domain Size

This parameter specifies the number of valid points recorded in the input file for a given dimension. NOTE WELL! If the dimension is type Complex, the time-domain size should be given in COMPLEX points; if the dimension is type Real, the time-domain size should be given in TOTAL points; data types are summarized by a table in the ACQUISITION MODES section below.

The time-domain size is used to determine the default length of the window function for a given dimension. In some cases, the spectrometer format FID may be padded so that the number of points saved in the input file is actually larger than the number of points acquired. In other cases, a dimension may contain interleaved data which is reduced in size by sums or differences before Fourier processing. It is for cases like these that separate parameters for actual size and valid size are required, as shown in the DATA CONVERSION EXAMPLES section.

After all parameters are specified, a click on the “Save Script” button will save the “Output Script” with the name specified in the current directory (unless user specifies a different path):

nmr image

Subsequently the conversion program can be started from the same interface by clicking on the “Execute Script” button. A window will pop-up in which the progress of the conversion can be followed:

nmr image

 

INSPECTING THE TIME DOMAIN DATA

The converted, unprocessed, time domain NMR data can be read and inspected in NMRDraw to see whether it was acquired and converted successfully.

Introduction to NMRDraw

NMRDraw is a user‑oriented X11 graphical interface for inspecting and processing multidimensional NMR data. In its current implementation, NMRDraw may be used to prepare data for use with computer‑assisted analysis and assignment programs such as PIPP (Dan Garrett, NIH), ANSIG (Per Kraulis, Karolinska Institute), or NMRView (Bruce Johnson, One Moon Scientific).

NMRDraw facilities include:

  • Display of 2D Planes from either processed or unprocessed multidimensional data
  • Interactive and User‑specified 2D Zoom
  • Adjustable color schemes
  • Viewing of the next 2D plane in a series by a single mouse click
  • Real‑time display of one or more 1D slices superimposed over 2D spectral graphics, with interactive adjustment of zoom region, vertical scale, and vertical offset
  • Interactive processing of 1D slices from any dimension of a spectrum, as well as automatic processing according to pre‑defined schemes
  • Real‑time phase correction of one or more 1D slices simultaneously, with coarse and fine phase adjustment, interactive pivot point selection, and automatic reconstruction of imaginary data when needed
  • An nmrPipe shell‑script editor, for creating, saving, and executing multidimensional conversion and processing schemes
  • Interactive adjustment of chemical shift calibration
  • Color and monochrome PostScript hard copy plot generation
  • Other extraction, display, and analysis facilities, including projections, graphics overlays from one or more spectra, and automated/interactive peak picking, are included or under development

STARTING AND EXITING NMRDRAW

In the usual installation, the NMRDraw program can be run by typing the command nmrDraw. This command executes the UNIX shell‑script file called nmrDraw, which in turn runs the program itself, which is called nmrdraw. You can customize many aspects of NMRDraw by editing a version of the nmrDraw shell‑script, to change the various arguments used to invoke the program. You can also include arguments directly on the NMRDraw command‑line. For instance, as a shortcut, you can specify the default data directory or data name: nmrDraw ‑dir ft ‑name test%03d.ft3

In order to exit the NMRDraw program, you can select the Quit NMRDraw command from the NMRDraw File Menu. You can also exit the program by moving the mouse pointer into the spectral graphics area and typing the character Q (upper‑case Q) at the keyboard.

SHORT DESCRIPTION

By default, the NMRDraw window takes up the entire screen. The window border at the top edge of the screen describes the current functions of the mouse buttons, if any. It is also sometimes used to display information about the current data:

nmr image

The command panel, which extends across the top of the nmrDraw window, contains pull‑down menu buttons for command entry, numerical fields for specifying spectral locations and display parameters, and slider bars for adjusting phase correction values interactively. The command panel usually has a gray background.

Outline of the functions of these items

X: Y: Z: A: Use these to increment, decrement, or select a particular 1D slice when viewing 1D slices from the X, Y, Z or A axis of a 1D, 2D, 3D and 4D data set respectively. Under default settings, the new 1D slice or 2D plane will be automatically read and drawn with the previous zoom region and contour settings.

Plus: Use this to change the number of positive contour levels to use the next time the data is drawn.

Minus: Use this to change the number of negative contour levels to use the next time the data is drawn.

First: Use this to select the height of the first contour used the next time the data is drawn. Negative contours will begin at the negative of this height.

Factor: Use this to select the spacing factor between adjacent contours used the next time the data is drawn.

(+) Use this button to increment the first contour height used the next time the data is drawn.

(-) Use this button to decrement the first contour height used the next time the data is drawn.

P0: (Sliders)

These sliders control the coarse and fine zero‑order phase applied at the selected pivot point of the current 1D spectra. These sliders will only work if a 1D slice is selected, and the Phasing Toggle button is ON.

P1: (Sliders)

These sliders control the coarse and fine first –order phase correction. These sliders will only work if a 1D slice is selected, and the Phasing Toggle button is ON.

P0: (Text)

Displays or changes the current P0 phase value; this value can be used directly with nmrPipe regardless of the pivot setting.

P1: (Text)

Displays or changes the current P1 phase value; this value can be used directly with nmrPipe regardless of the pivot setting.

Pivot:

Displays or changes the position of the phasing pivot.

The pivot position can also be selected via the mouse when viewing 1D slices.

Phasing Toggle

This toggle button must be set to ON in order to apply phasing to the current 1D data. But, once the toggle is turned ON, the current 1D spectra will not be rephased until a slider or phase value is adjusted.

The canvas takes up the remaining portion of the nmrDraw window; it has a purple background by default. The canvas contains a rectangular spectral graphics area with black background, where displays such as contour plots will be drawn.

Note: Many facilities of NMRDraw make use of the mouse; in these cases, the mouse buttons will perform one set of actions while the mouse pointer is inside the spectral graphics area, and a different set of actions when the mouse pointer is to the left or below the graphics area, in the purple borders. As mentioned above, the mouse button functions will be displayed the topmost border of the NMRDraw window.

In addition to NMRDraw’s primary window, one or more pop‑up windows may appear during a session. These windows are used for special‑purpose command entry, such as selection of the current data file. More than one pop‑up may appear on the screen at a given time. As with NMRDraw’s primary window, the top border (grey) of a pop‑up will often display useful information. Most pop‑ups have a Done Button, which should be used to close the pop‑up and remove it from the screen.

Note: if you remove a pop‑up by other means, such as via window manager commands, you may not be able to use the pop‑up again during the session.

ENTERING COMMANDS IN NMRDRAW

The primary commands are selected via Menu Buttons in the command panel.

Example: the command to draw a contour plot on the screen is listed under the Draw pop‑up menu as “d) Contours”, and can be selected via the mouse. As an alternative, the command can be selected from the keyboard without the need to use the mouse. In this case, the command can be selected by typing d (lower‑case d), as indicated at the beginning of the Command Text.

Note: You can only type commands from the keyboard if the keyboard focus is set to the spectral graphics area. This generally means that the mouse pointer will have to be positioned inside the graphics area (black background) before commands are entered.

SOME GRAPHICAL COMMAND ITEMS

Buttons are used to display a single command or command choice. To select a Button’s command, move the pointer on it and click the left mouse button. Menu Buttons are special Buttons which cause a pop‑up menu of additional options to be displayed. Menu Buttons include a triangular caret symbol after the button text; this caret signifies that there is a menu associated with the button. To view the pop‑up menu associated with a Menu Button, press and hold the right mouse button; the menu will appear, and will stay on the screen as long as the mouse button is held. Command options in the pop‑up menu can be selected by dragging the mouse to highlight the desired option, then releasing the mouse.

The first command entry in any given menu is the default command. The menus are often arranged so this default command is a commonly used one. As a shortcut, user can select this default command without the need to pop‑up the menu, by a single left-click  

Slider Bar

Slider bars are used for interactive specification of some continuous value, for instance zero or first order phase. The value associated with a Slider can be adjusted by pointing to the Slider’s Button with the mouse, then dragging the mouse with the left mouse button pressed. In addition, you can increment or decrement the slider value by clicking left mouse button while pointing to either end of the Slider bar groove.

READ AND DISPLAY TIME DOMAIN DATA

The File Menu includes commands to select the spectral data of interest, as well as commands to read or re‑read the selected data. The “Read” command reads a 2D file plane into NMRDraw’s data memory; depending on the 2D Settings, a 2D plot of the plane may be drawn automatically once the data is read. The name of the data to read is specified by the Select File command in the File Menu; 1D and 2D data sets are selected according to their file names, while 3D and 4D data are selected according to their template names. In the case of 3D and 4D data, the Z‑Axis and A‑Axis locations for the plane to read are taken from the Z: and A: fields in the command panel:

nmr image

After reading in the data using the File/Read, the threshold used in displaying the interferograms needs to be adjusted to obtain a reasonable balance between the level of detail above the cutoff and the time needed to display it. This can be done by mouse clicks on the +/‑ buttons of the Factor display parameter in the command panel. The height of the First contour displayed will change accordingly.

Another way to set an acceptable threshold level is by using the “Estimate Noise” command from the “Draw” or “Peak” menus or its “T” (Shift + t) keyboard shortcut. The command estimates the standard deviation of the spectral noise, and resets the first contour height accordingly. Note that the estimate may be crude.

nmr image

Subsequently any FID (indexed by the value of Y parameter) can be visualized as a 1D slice by selecting “1D horizontal” from the “Mouse” menu or by using the “h” keyboard shortcut. Occasional distortions of a particular interferogram can be easily detected in this way. User can display different FIDs by changing the Y value in the command panel.:

nmr image

Note: Once entered in the 1D horizontal display mode (as in most other display modes) the functions of the 3 mouse buttons both when the pointer is inside or outside the graphical display area are conveniently provided in the top title bar (grey).

Any time‑domain slice can be Fourier transformed by selecting the “AutoProc” command from the “Proc” menu. As this is the first (default) command in the menu, a single mouse left‑click on the “Proc” menu button is sufficient to execute it:

nmr image 

User can scale or move in real-time vertically (offset) the yellow 1D trace by middle or right mouse drag with the mouse pointer placed outside the graphical display area (i.e. on the purple border).

The 1D trace of the first increment, which corresponds to the integral of all signals of a multidimensional experiment, can be displayed by setting Y, Z, and A to 1, selecting the horizontal 1D slice corresponding to these parameters and Fourier transforming it using the “AutoProc” command from the “Proc” menu. In most cases peaks in the direct detected dimension will have the same phase as this first increment trace, therefore manually (interactively) phasing it will provide the phase correction of the direct detected dimension (usually 1H) in the final multidimensional processing scheme. The phase corrections in the indirect dimensions of a 3D experiment can be easily found by using the proj3D.tcl script, which will create the projections off all peaks in all 3 possible 2D planes that can be easily phased separately.

Note: User needs to enable phase alteration by clicking the “On” button from the “Phasing” command panel as the “Off” value is selected by default.

The display of  an axis in user-selected units below the 1D slice can be toggled on and off using the “Axis Toggle” command from the “1D” menu:

nmr image

The phase pivot can be set graphically by a left mouse click on the purple border (outside the graphical display area) just below the peak. An arrow will be displayed below the signal, on the purple border, to indicate the position of the phase pivot. The 1D slice can be phased by setting the phase pivot on an intense signal, adjusting the 0‑order phase correction (P0) slider and then adjusting the 1st order phase correction (P1) slider so that the rest of signals would be in phase with the peak at the pivot position:

nmr image

 

TRIAL PROCESSING

Users can use the Macro Edit interface from the File menu to create, save and execute UNIX shell‑scripts for spectral processing. It provides a point‑and‑click text editor augmented with a series of predefined conversion and processing scripts. It also allows edited scripts to be saved on the disk and executed.

Users can then read and display the processed plane; evaluate the need for any additional processing, such as baseline correction. Finally they can create, save, and execute a UNIX shell‑script to process the entire 3D spectrum, incorporating the previous 2D scheme.

nmr image

A typical 3D processing scheme (default) can be loaded in the editor window with a click on the Process 3D button:

nmr image

A description for any nmrPipe function (including all options available) can be obtained by typing in a shell window: “nmrPipe –help –fn FUNCTION”, where FUNCTION should be replaced is the name of the function of interest (e.g. “nmrPipe –help –fn SOL” will display the help file for the solvent filter function “SOL”).

In the 3D processing script the converted data matrix from the “fid” directory is loaded as a template (test%03d.fid) and becomes input for the xyz2pipe function. The data matrix is “piped” sequentially to the following functions:

  • solvent filter (SOL) is applied to the time domain data;
  • a sine square (-pow 2) weighting (window) function (SP) with a first point scaled to ½  (-c 0.5) is multiplied to the 1st dimension (x vectors)
  • the number of data points is zero‑filled to the next power of 2 (ZF)
  • Fourier transform (FT) of the 1st dimension (x  vectors)
  • Phase correction (PS) followed by deletion of imaginary points (-di)
  • Extraction (EXT) of the left half of the spectrum (common for looking only at the spectral area to the left of the transmitter positioned on the solvent signal) with automatic adjustment of the spectral window (-sw)
  • Data matrix is transposed (TP) for the 2nd dimension (y) Fourier transform
  • a sine (-pow 1) weighting (window) function (SP) with a first point unscaled (-c 1.0) is multiplied to the 2nd dimension (y vectors)
  • the number of data points is zero‑filled to the next power of 2 (ZF)
  • Fourier transform (FT) of the 2nd dimension (y vectors)
  • Phase correction (PS) of the 2nd dimension followed by deletion of imaginary points (-di)

The data matrix is written out in the “ft” directory again as a template (test%03d.ft2) with only the 2 out of 3 dimensions processed (i.e. in frequency domain). The resulting planes can be inspected (commonly the first plane is of interest as it contains a sum of all signals as no evolution in the 3rd dimension has taken place).

User can edit the script in this window (e.g. changing the 0 and 1st order phasing parameters):

nmr image

The phase correction can be adjusted  in the first 2 dimensions (x and y axes) and reprocessed using the above fraction of the entire script by inserting an “exit 0” command after the first part of the script.

Launching the execution of the processing script is done by clicking on the “Execute” button:

nmr image

Another window (red background) will pop-up and will show the processing progress:

nmr image

Subsequently the 3rd dimension (axis z) can be processed using only the 2nd part of the script (after removal of the “exit” command). The data matrix from the “ft” directory is loaded as a template (test%03d.ft2) and becomes input for the xyz2pipe function:

  • a sine (-pow 1) weighting (window) function (SP) with a first point unscaled (-c 1.0) is multiplied to the 3rd dimension (z vectors)
  • the number of data points is zero‑filled to the next power of 2 (ZF)
  • Fourier transform (FT) of the 3rd dimension (z vectors)
  • Phase correction (PS) of the 3rd dimension followed by deletion of imaginary points (-di)

Finally the data matrix is written out in the “ft” directory again as template (test%03d.ft3) with all 3 dimensions processed:

nmr image

Reprocessing the 3rd dimension (e.g. after phase readjustments of the z‑axis) can be done using only the 2nd part of the script with the “test%03d.ft2” matrix as the input template by saving the 2nd part of the script as a different macro and executing it.

 

INSPECTING THE PROCESSED RESULT

For easy phase correction of 3D spectra users should run the “proj3D.tcl” script that will create 2D projection planes of the 3D processed (frequency domain) dataset. The script uses the template “ft/test%03d.ft3” as default input (this can be changed as a command line option, give “proj3D.tcl –help” in a shell window for details).

Phasing of each dimension of the processed 2D planes is done as described in detail previously. The steps are reminded below:

Use File/Read to read a 2D plane into NMRDraw. The name of the data is specified by the Select File command in the File Menu; 1D and 2D data sets are selected according to their file names, 3D and 4D data are selected according to their template names. A 2D plot of the plane will be drawn automatically once the data is read:

nmr image

Adjust the threshold used in displaying the interferograms by mouse clicks on the +/‑ buttons of the Factor display parameter in the command panel. The other way to set an acceptable threshold level is by using the “Estimate Noise” command from the “Draw” or “Peak” menus or its “T” (Shift + t) keyboard shortcut.

nmr image

The height of the First contour displayed will change accordingly:

nmr image

To visualize a 1D slice use the “h” keyboard shortcut or, alternatively, by select “1D horizontal” from the “Mouse” menu:

nmr image

User can scale or move in real-time vertically (offset) the yellow 1D trace by middle or right mouse drag with the mouse pointer placed outside the graphical display area (i.e. on the purple border).

Note: Don’t forget to enable phasing by clicking the “On” button from the “Phasing” command panel as the “Off” value is selected by default!

The phase pivot can be set graphically by a left mouse click on the purple border (outside the graphical display area) just below the peak. An arrow will be displayed below the signal, on the purple border, to indicate the position of the phase pivot. The 1D slice can be phased by setting the phase pivot on an intense signal, adjusting the 0‑order phase correction (P0) slider and then adjusting the 1st order phase correction (P1) slider so that the rest of signals would be in phase with the peak at the pivot position:

The same procedure is applied on the vertical axis (keyboard shortcut “v”) to complete the phasing of a 2D plane (or a 2D projection of a 3D dataset):

nmr image

 

FINAL PROCESSING

Once phasing and all other processing parameters result in an acceptable spectrum, a full (high resolution) processing can be accomplished by increasing the zero-filling in all dimensions. Note that more than twice zero-filling in any dimension won't increase the peak picking precision and, even though the lineshapes will become smoother, the data size will also increase resulting in slower processing.

PROCESSING TIPS

Processing Schemes: in some cases, the processing scheme itself may not be appropriate; for example use of the wrong kind of Fourier Transform, or not deleting imaginary data where appropriate. When possible, nmrPipe will try to detect such problems, and issue warning messages.

Conversion Parameters: if the spectrometer format data has not been converted correctly, processing may complete without error messages but still give the wrong result. Pay close attention to the sizes and acquisition modes of all the dimensions when converting the data.

First Order Phase Correction, Folding, and First-point Scaling

A delay in the acquisition introduces a first-order phase distortion.  If there is no delay, no first order phase correction is required.  Each delay of 1 dwell (1 point) introduces an additional 360 first order phase correction.

Many current NMR pulse sequences are designed so that there is no delay in the directly acquired dimension.  So, when phasing a spectrum interactively, it is best to try phasing with P0 only first.

Depending on the delay, the first point of the FID should be adjusted before Fourier Transform.  The first point scaling factor is selected by the window function argument "-c".

If the required first order phase P1 for the given dimension is 0.0, the first point scaling factor should be 0.5.  This is because the discrete Fourier transform does the equivalent of counting the point at t=0 twice.  If the first point is not scaled properly in this case, ridge-line baseline offsets in the spectrum will result.

In all other cases (P1 is not zero), this scale factor should be 1.0. This is because the first point of the FID no longer corresponds to t=0, and so it shouldn't be scaled. If the scale factor is not set correctly, it will introduce a baseline distortion which is either zero-order or sinusoidal, depending on what first-order phase is required. When possible, it is best to set up experiments with either exactly 0, 1/2, or 1-point delay.  There are several reasons:

  1. Phase correction values can be determined easily.
  2. If the delay is not a multiple of 1/2 point, the phase of folded peaks will be distorted.
  3. The Hilbert transform (HT) is used, sometimes automatically, to reconstruct previously deleted imaginary data for interactive rephasing or inverse processing. But, the HT can only reconstruct imaginary data perfectly if the phase is a multiple of 1/2 point.
  4. Data with P1 = 360 have the first point t=0 missing (i.e. 1 point delay). Since the first point of the FID corresponds to the sum of points in the corresponding spectrum, this missing first point can be "restored" by adding a constant to the phased spectrum.  This can be done conveniently by automated zero-order baseline correction, as shown below:

     Summary, First Point Scaling and Phase Correction

      Delay: 0   point    P1 =   0    FID: Scale -c 0.5

      Delay: 1/2 point    P1 = 180    FID: Scale -c 1.0

                                     Spectrum: Folded peaks have opposite sign

      Delay: 1   point    P1 = 360   FID: Scale -c 1.0

                                     Spectrum: Use "POLY -auto -ord 0"

Automated Baseline Correction

The default automated baseline correction (POLY -auto) should usually only be used when at least two dimensions of the data have been transformed, so that there is as much empty baseline as possible for automated detection.  So, it is common practice to create a processing scheme like this, which processes the first two dimensions, then transposes the data to correct the first dimension:

nmrPipe -in fid/test001.fid \
| nmrPipe  -fn POLY \
| nmrPipe  -fn SP -off 0.5 -end 0.98 -pow 2 -c 0.5 \
| nmrPipe  -fn ZF -auto \
| nmrPipe  -fn FT -verb \
| nmrPipe  -fn PS -p0 43 -p1 0.0 -di \
| nmrPipe  -fn EXT -left -sw \
| nmrPipe  -fn TP \
| nmrPipe  -fn SP -off 0.5 -end 0.98 -pow 1 -c 1.0 \
| nmrPipe  -fn ZF -auto \
| nmrPipe  -fn FT -verb \
| nmrPipe  -fn PS -p0 -135 -p1 180 -di \
| nmrPipe  -fn TP \
| nmrPipe  -fn POLY -auto \
-verb -ov -out test.ft2

Zero-order automated baseline correction (POLY -auto -ord 0) is commonly applied to dimensions that have a missing first point (P1 = 360).

Automated baseline correction should only be applied as needed.

In many cases, only the directly-detected dimension, or dimensions with one-dwell delay (P1=360) need a baseline correction.

NOTE WELL that cases for P1=0, P1=180, and P1=360 can all be handled by the proper combination of first point scaling and zero-order baseline correction.  All other cases have the potential to introduce more difficult baseline distortions, as well as phase distortions for folded peaks.  For these reasons, great care should be taken to set the acquisition delays to 0, 1/2, or 1 point whenever possible.  This also eliminates the need to choose phase correction values manually, which can be very difficult with some spectra.

Reversed Spectra, and Left/Right Swapped Spectra

   Use "FT -neg" if a given dimension is reversed.

   Use "FT -alt" if a given dimension is left/right swapped.

   The "-alt" and "-neg" arguments can be used together if needed.

NOTE WELL that "FT -neg" is NOT exactly the same as simply reversing the spectrum with "REV".  "FT -neg" is equivalent to REV followed by a one-point circular shift.  So, if REV is applied instead, the PPM calibration will be off by one point.

Of course, the "-neg" or "-alt" arguments should not be used if the data are not reversed or left/right swapped.

 

HARD COPY PLOTTING

File/Hard Copy Plot

The Hard Copy Plot command (keyboard shortcut “P”) creates simple PostScript plots of the current data, and optionally sends the plots to a printer queue. The plots are generated according to the current display settings such as first contour height, number of contour levels, etc.

Adding Strip Plot Display to a 3D Processing Scheme

There are a few simple utilities which can be combined to automatically generate and display 2D strips from a 3D spectrum.  In order to generate such a display, we first need a table of 2D locations which specifies the positions where strips will be extracted from the 3D data.  A simple way to do this is to create a 2D projection of the 3D data, use peak detection on the 2D projection to generate a list of 2D positions, and then use these 2D positions to extract and display the strips.  Try adding commands like these to the end of a 3D processing scheme:

        proj3D.tcl -in ft/test%03d.ft3 -abs

        peakHN.tcl -proj -in ft/test%03d.ft3 -out hn.proj.tab -hi Full

        scroll.tcl -in ft/test%03d.ft3 -tab hn.proj.tab –hi 1.0e+5 -pair

THE PROC MENU

The Proc Menu provides two ways to process the current 1D slice, as explained here:

NMRPipe Command

Provides a pop‑up window for executing individual nmrPipe processing functions on the current 1D slice. To perform processing, enter the name of an nmrPipe function on the given command line, along with any additional arguments; For instance, “ft ‑inv” would be the command for inverse Fourier transform. As an alternative to typing the function and arguments, a small menu of common functions is provided. Once the function is entered, execute the processing by typing a carriage return, or by clicking the Execute button.

Auto‑Process 1D

This command will execute a pre‑defined macro which is responsible for processing the current 1D slice. The macro used is specified by the command‑line argument ‑proc in the nmrDraw script.

The default version of the macro will apply a window, zero fill, and FT to time‑domain data; if the axis name is “HN”, a solvent filter and extraction of an amide chemical shift range will also be applied.

In the case of frequency‑domain data, inverse processing will be applied, automatically restoring the current 1D slice it to its original time‑domain form as best as possible:

nmr image

The conversion programs for Bruker, Varian and other data formats are also included in the Proc Menu and can be conveniently invoked from NMRDraw.

 

AUTOMATED PEAK DETECTION

The documentation for automatic peak detection is included in the Peak Analysis and Quantification section of this manual.