Offline observation operator

The obs oper maps model variables to observation space. It is possible to apply this mapping without running the model. The software which performs this functionality is known as the offline obs oper. The obs oper is divided into three stages. An initialisation phase, an interpolation phase and an output phase. The implementation of which is outlined in the previous sections. During the interpolation phase the offline obs oper populates the model arrays by reading saved model fields from disk.

There are two ways of exploiting this offline capacity. The first is to mimic the behaviour of the online system by supplying model fields at regular intervals between the start and the end of the run. This approach results in a single model counterpart per observation. This kind of usage produces feedback files the same file format as the online obs oper. The second is to take advantage of the offline setting in which multiple model counterparts can be calculated per observation. In this case it is possible to consider all forecasts verifying at the same time. By forecast, I mean any method which produces an estimate of physical reality which is not an observed value. In the case of class 4 files this means forecasts, analyses, persisted analyses and climatological values verifying at the same time. Although the class 4 file format doesn't account for multiple ensemble members or multiple experiments per observation, it is possible to include these components in the same or multiple files.

Using the offline observation operator

Building

In addition to OPA_SRC the offline obs oper requires the inclusion of the OOO_SRC directory. OOO_SRC contains a replacement nemo.f90 and nemogcm.F90 which overwrites the resultant nemo.exe. This is the approach taken by SAS_SRC and OFF_SRC.

Running

The simplest way to use the executable is to edit and append the ooo.nml namelist to a full NEMO namelist and then to run the executable as if it were nemo.exe.

Quick script

A useful Python utility to control the namelist options can be found in OBSTOOLS/OOO. The functions which locate model fields and observation files can be manually specified. The package can be installed by appropriate use of the included setup.py script.

Documentation can be auto-generated by Sphinx by running make html in the doc directory.

Configuring the offline observation operator

Single field

In offline mode model arrays are populated at appropriate time steps via input files. At present, tsn and sshn are populated by the default read routines. These routines will be expanded upon in future versions to allow the specification of any model variable. As such, input files must be global versions of the model domain with votemper, vosaline and optionally sshn present.

For each field read there must be an entry in the namooo namelist specifying the name of the file to read and the index along the time_counter. For example, to read the second time counter from a single file the namelist would be.

 

!----------------------------------------------------------------------

!       namooo Offline obs_oper namelist

!----------------------------------------------------------------------

!   ooo_files    specifies the files containing the model counterpart

!   nn_ooo_idx   specifies the time_counter index within the model file

&namooo


ooo_files = "foo.nc"


nn_ooo_idx = 2

/

Multiple fields per run

Model field iteration is controlled via nn_ooo_freq which specifies the number of model steps at which the next field gets read. For example, if 12 hourly fields are to be interpolated in a setup where 288 steps equals 24 hours.

 

!----------------------------------------------------------------------

!       namooo Offline obs_oper namelist

!----------------------------------------------------------------------

!   ooo_files    specifies the files containing the model counterpart

!   nn_ooo_idx   specifies the time_counter index within the model file

!   nn_ooo_freq  specifies number of time steps between read operations

&namooo


ooo_files = "foo.nc" "foo.nc"


nn_ooo_idx = 1 2


nn_ooo_freq = 144

/

The above namelist will result in feedback files whose first 12 hours contain the first field of foo.nc and the second 12 hours contain the second field.

It is easy to see how a collection of fields taken fron a number of files at different indices can be combined at a particular frequency in time to generate a pseudo model evolution. As long as all that is needed is a single model counterpart at a regular interval then namooo is all that needs to be edited. However, a far more interesting approach can be taken in which multiple forecasts, analyses, persisted analyses and climatologies are considered against the same set of observations. For this a slightly more complicated approach is needed. It is referred to as Class 4 since it is the fourth metric defined by the GODAE intercomparison project.

Multiple model counterparts per observation a.k.a Class 4

A generalisation of feedback files to allow multiple model components per observation. For a single observation, as well as previous forecasts verifying at the same time there are also analyses, persisted analyses and climatologies.

The above namelist performs two basic functions. It organises the fields given in namooo into groups so that observations can be matched up multiple times. It also controls the metadata and the output variable of the class 4 file when a write routine is called.

Class 4 naming convention

Much of the namelist is devoted to specifying this convention. The following namelist settings control the elements of the output file names. Each should be specified as a single string of character data.

The kind is specified by the observation type internally to the obs oper. The processor number is specified internally in NEMO.

Class 4 file global attributes

Global attributes necessary to fulfill the class 4 file definition. These are also useful pieces of information when collaborating with external partners.

The obs_type, creation date and validity time are specified internally to the obs oper.

Class 4 model counterpart configuration

As seen previously it is possible to perform a single sweep of the obs oper and specify a collection of model fields equally spaced along that sweep. In the class 4 case the single sweep is replaced with multiple sweeps and a certain ammount of book keeping is needed to ensure each model counterpart makes its way to the correct piece of memory in the output files.

In terms of book keeping, the offline obs oper needs to know how many full sweeps need to be performed. This is specified via the cl4_match_len variable and is the total number of model counterparts per observation. For example, a 3 forecasts plus 3 persistence fields plus an analysis field would be 7 counterparts per observation.

Then to correctly allocate a class 4 file the forecast axis must be defined. This is controlled via cl4_fcst_len, which in out above example would be 3.

Then for each model field it is necessary to designate what class 4 variable and index along the forecast dimension the model counterpart should be stored in the output file. As well as a value for that lead time in hours, this will be useful when interpreting the data afterwards.



cl4_vars = "forecast" "forecast" "forecast" "persistence" "persistence"

              "persistence" "best_estimate"


cl4_fcst_idx = 1 2 3 1 2 3 1


cl4_leadtime = 12 36 60

In terms of files and indices of fields inside each file the class 4 approach makes use of the namooo namelist. If our fields are in separate files with a single field per file our example inputs will be specified.



ooo_files = "F.1.nc" "F.2.nc" "F.3.nc" "P.1.nc" "P.2.nc" "P.3.nc" "A.1.nc"


nn_ooo_idx = 1 1 1 1 1 1 1

When we combine all of the naming conventions, global attributes and i/o instructions the class 4 namelist becomes.


!----------------------------------------------------------------------

!       namooo Offline obs_oper namelist

!----------------------------------------------------------------------

!   ooo_files    specifies the files containing the model counterpart

!   nn_ooo_idx   specifies the time_counter index within the model file

!   nn_ooo_freq  specifies number of time steps between read operations

&namooo


ooo_files = "F.1.nc" "F.2.nc" "F.3.nc" "P.1.nc" "P.2.nc" "P.3.nc" "A.1.nc"


nn_ooo_idx = 1 1 1 1 1 1 1

/

!----------------------------------------------------------------------

!       namcl4 Offline obs_oper class 4 namelist

!----------------------------------------------------------------------

!

!  Naming convention

!  -----------------

!  cl4_prefix    specifies the output file prefix

!  cl4_date      specifies the output file validity date

!  cl4_sys       specifies the model counterpart system

!  cl4_cfg       specifies the model counterpart configuration

!  cl4_vn        specifies the model counterpart version

!  cl4_inst      specifies the model counterpart institute

!  cl4_contact   specifies the file producers contact details

!

!  I/O specification

!  -----------------

!  cl4_vars      specifies the names of the output file netcdf variable

!  cl4_fcst_idx  specifies output file forecast index

!  cl4_fcst_len  specifies forecast axis length

!  cl4_match_len specifies number of unique matches per observation

!  cl4_leadtime  specifies the forecast axis lead time 

!

&namcl4


cl4_match_len = 7


cl4_fcst_len = 3


cl4_fcst_idx = 1 2 3 1 2 3 1


cl4_vars = "forecast" "forecast" "forecast" "persistence" "persistence"

              "persistence" "best_estimate"


cl4_leadtime = 12 36 60


cl4_prefix = "class4"


cl4_date = "20130101"


cl4_vn = "12.0"


cl4_sys = "FOAM"


cl4_cfg = "AMM7"


cl4_contact = "example@example.com"


cl4_inst = "UK Met Office"

/

Climatology interpolation

The climatological counterpart is generated at the start of the run by restarting the model from climatology through appropriate use of namtsd. To override the offline observation operator read routine and to take advantage of the restart settings, specify the first entry in cl4_vars as "climatology". This will then pipe the restart from climatology into the output class 4 file. As in every other class 4 matchup the input file, input index and output index must be specified. These can be replaced with dummy data since they are not used but they must be present to cycle through the matchups correctly.

Advanced usage

In certain cases it may be desirable to include both multiple model fields per observation window with multiple match ups per observation. This can be achieved by specifying nn_ooo_freq as well as the class 4 settings. Care must be taken in generating the ooo_files list such that the files are arranged into consecutive blocks of single match ups. For example, 2 forecast fields of 12 hourly data would result in 4 separate read operations but only 2 write operations, 1 per forecast.



ooo_files = "F1.nc" "F1.nc" "F2.nc" "F2.nc"

...


cl4_fcst_idx = 1 2

The above notation reveals the internal split between match up iterators and file iterators. This technique has not been used before so experimentation is needed before results can be trusted.