GENIE:GENIEToolboxFAQ

Andrew Price, University of Southampton ([mailto:a.r.price@soton.ac.uk a.r.price@soton.ac.uk])

GENIE Toolbox: Frequently Asked Questions

 * How do I convert a config file for use in the GENIE Toolbox?
 * How do I create a model archiving function for the GENIE Toolbox?

How do I convert a config file for use in the GENIE Toolbox?
The GENIE Toolbox has been designed to manage GENIE on multiple heterogeneous platforms including Windows. A consequence of this is that genie_example.job cannot be used to configure and execute the simulations - it is too complicated to get a bash script to run reliably on remote Windows platforms. The GENIE Toolbox therefore performs many of the same activities defined in genie_example.job. There are issues with this duplication of functionality and efforts are under way to rigorously define the individual activities so that alternate tooling can be safely employed - but for now we need to explain how config files can be ported to the GENIE Toolbox environment.

To configure a model with the GENIE Toolbox we follow a similar config file mechanism as genie_example.job. The default parameter set is loaded into Matlab and then corrections/amendments are applied to specify the particular configuration of the model to run. A configuration function is written that has the same name as the corresponding config file:


 * genie_{model}.config --> genie_{model}_config.m

To convert a config file from the CVS/SVN repository to a GENIE Toolbox equivalent the following steps are necessary. We present an example of porting the genie_eb_go_gs_ml.config file from the source repository to the GENIE Toolbox equivalent genie_eb_go_gs_ml_config.m:

 Navigate to the GENIEToolbox/configs directory in your installtion of GENIELab</li> <li>Create a copy of the GENIE Toolbox config template with a name corresponding to the model instance. <ul> <li>$ cp genie_model_config.m.template genie_eb_go_gs_ml_config.m</tt></li> </ul> </li> <li>Edit the new config script and replace all instances of the string model</tt> with the model descriptor for this config file. <ul> <li>Search and replace: model</tt> --> eb_go_gs_ml</tt></li> </ul></li> <li>Remove any module metadata that is not required for the particular configuration of the model. For genie_eb_go_gs_ml we just need configuration information for the main, embm, goldstein, land and seaice modules. % Load the default parameters (specify files using relative paths) metadata.genie_main         = metadata_main('../..','.'); metadata.genie_embm         = metadata_embm('../..','.'); metadata.genie_goldstein    = metadata_goldstein('../..','.'); metadata.genie_land         = metadata_land('../..','.'); metadata.genie_seaice       = metadata_seaice('../..','.'); </li> <li>Copy the parameter override entries from the bash config file to the new Matlab config function in the marked section of the template. % Add the adapted entries from the corresponding config file here % =========================================================================

% ========================================================================= For the genie_eb_go_gs_ml_config.m example % Add the adapted entries from the corresponding config file here % =========================================================================
 * 1) C-GOLDSTEIN DEFAULT INTEGRATION OF 5000 YEARS WITH GENIE-land
 * 2) Fixed present-day vegetation

EXPID=genie_eb_go_gs_ml

ma_flag_ebatmos=.TRUE. ma_flag_goldsteinocean=.TRUE. ma_flag_goldsteinseaice=.TRUE. ma_flag_land=.TRUE. ma_flag_igcmatmos=.FALSE. ma_flag_fixedocean=.FALSE. ma_flag_fixedseaice=.FALSE. ma_flag_fixedicesheet=.FALSE. ma_flag_fixedchem=.FALSE.

GENIEDP=TRUE IGCMATMOSDP=TRUE
 * 1) DP flags are important for global water and energy
 * 2) conservation tests

GENIENXOPTS='$(DEFINE)GENIENX=36' GENIENYOPTS='$(DEFINE)GENIENY=36'

ma_write_flag_atm=.false. ma_write_flag_sic=.false.
 * 1) this is to only write ocean-grid data in genie-main

ma_dt_write=50000
 * 1) this is to write genie-main data every 50000 timesteps=100 years

ea_3=50000 ea_4=10000 ea_5=5000 ea_6=50000
 * 1) this is to control embm output periods
 * 2)   npstp='health check' from 1000=10 years to 50000=500 years
 * 3)   iwstp='restarts' from 50000=500 years to 10000=100 years
 * 4)   itstp='time series' from 100=1 year to 5000=50 years
 * 5)   ianav='an average' stays 50000=500 years

go_3=50000 go_4=10000 go_5=5000 go_6=50000
 * 1) this is to control goldstein output periods
 * 2)   npstp='health check' from 1000=10 years to 50000=500 years
 * 3)   iwstp='restarts' from 50000=500 years to 10000=100 years
 * 4)   itstp='time series' from 100=1 year to 5000=50 years
 * 5)   ianav='an average' stays 50000=500 years

gs_3=50000 gs_4=10000 gs_5=5000 gs_6=50000
 * 1) this is to control seaice output periods
 * 2)   npstp='health check' from 1000=10 years to 50000=500 years
 * 3)   iwstp='restarts' from 50000=500 years to 10000=100 years
 * 4)   itstp='time series' from 100=1 year to 5000=50 years
 * 5)   ianav='an average' stays 50000=500 years

ma_lgraphics=.false.
 * 1) this is to turn graphics off

ma_koverall_total=2500000
 * 1) this is to change the model run length
 * 2)   720=1 month of igcm (timestep=1 hour) to 2500000=5000 years of c-goldstein

ma_ksic_loop=5 ma_kocn_loop=5 ma_klnd_loop=5
 * 1) this changes the relative atmos/ocean/seaice calling frequency

ml_idiag_land=5000 ml_irest_land=50000 ml_iacc_land=5000 ml_c_restart=$CODEDIR/genie-land/data/input/land_rs_embm_36x36.nc ml_c_runoff_fl=$CODEDIR/genie-land/data/input/runoff_mask_gold36x36.nc % ========================================================================= </li> <li>Move the EXPID</tt> entry to the bottom of the script and replace the template entry</li> <li>Parse the new section of script <ul> <li>Change the comment lines back into comments by pre-prending these lines with a '%</tt>' % # C-GOLDSTEIN DEFAULT INTEGRATION OF 5000 YEARS WITH GENIE-land % # Fixed present-day vegetation </li> <li>Remove or comment out the compiler directives from the script as these are applied at build time and are not part of the GENIE Toolbox configuration % GENIEDP=TRUE % IGCMATMOSDP=TRUE % GENIENXOPTS='$(DEFINE)GENIENX=36' % GENIENYOPTS='$(DEFINE)GENIENY=36' </li> <li>For each parameter assignment operation assign each parameter to the appropriate field of the metadata</tt> structure metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_ebatmos=.TRUE. metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_goldsteinocean=.TRUE. metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_goldsteinseaice=.TRUE. </li> <li>Change the Fortran boolean constants .TRUE.</tt> and .FALSE.</tt> to Matlab strings 'true'</tt> and 'false'</tt> metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_ebatmos='true' metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_goldsteinocean='true' metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_goldsteinseaice='true' </li> <li>Surround any path definitions with single quotes as these need to be treated as strings in Matlab metadata.genie_land.Parameter.GENIELAND_CONTROL.c_restart='$CODEDIR/genie-land/data/input/land_rs_embm_36x36.nc' metadata.genie_land.Parameter.GENIELAND_CONTROL.c_runoff_fl='$CODEDIR/genie-land/data/input/runoff_mask_gold36x36.nc' </li> <li>Change any instances of $CODEDIR</tt> to the value of runtime_root</tt> that is provided in the first argument to the metadata_module</tt> function calls at the top of the script. For the default builds of GENIE this entry should be ../..</tt> as the code directory lies to directories up the hierarchy from the executable binary metadata.genie_land.Parameter.GENIELAND_CONTROL.c_restart='../../genie-land/data/input/land_rs_embm_36x36.nc' metadata.genie_land.Parameter.GENIELAND_CONTROL.c_runoff_fl='../../genie-land/data/input/runoff_mask_gold36x36.nc' </li> <li>Add a ';</tt>' line terminator to every assignment to avoid echoing output to screen metadata.genie_land.Parameter.GENIELAND_CONTROL.c_restart='../../genie-land/data/input/land_rs_embm_36x36.nc'; metadata.genie_land.Parameter.GENIELAND_CONTROL.c_runoff_fl='../../genie-land/data/input/runoff_mask_gold36x36.nc'; </li> </ul> </li> <li>If the configuration of the model involves modules that are initiated from <tt>goin_</tt> files rather than namelists then the ordering of the fields in the metadata structure is critically important. We strongly recommend passing the data for these module through the '<tt>goin_dictionary</tt>' to ensure correct ordering. Add the following lines as appropriate for any module of the goin variety dict=goin_dictionary; [members indices]=ismember(fieldnames(metadata.genie_embm.Parameter), dict); list=dict(sort(indices)); metadata.genie_embm.Parameter=orderfields(metadata.genie_embm.Parameter,list); </li> <li>The <tt>genie_eb_go_gs_ml_config.m</tt> looks like this function [metadata, EXPID] = genie_eb_go_gs_ml_config % Loads the configuration metadata for the genie_eb_go_gs_ml eb_go_gs_ml % % SYNTAX %   [metadata, EXPID] = genie_eb_go_gs_ml_config % % DESCRIPTION %   Loads the default GENIE metadata data to set up the genie_eb_go_gs_ml %   eb_go_gs_ml. This function is a direct conversion of the file %   configs/genie_eb_go_gs_ml.config % % See also genie_eb_go_gs_ml_archive
 * 1) GENIE-land config changes
 * 1) GENIE-land config changes

% Copyright 2007 GENIE Project, University of Southampton % Andrew Price, $Date: $ % $Revision: $ % GENIE Toolbox for Matlab

% Load the default parameters (specify files using relative paths) metadata.genie_main         = metadata_main('../..','.'); metadata.genie_embm         = metadata_embm('../..','.'); metadata.genie_goldstein    = metadata_goldstein('../..','.'); metadata.genie_land         = metadata_land('../..','.'); metadata.genie_seaice       = metadata_seaice('../..','.');

% Add the adapted entries from the corresponding config file here % ========================================================================= % # C-GOLDSTEIN DEFAULT INTEGRATION OF 5000 YEARS WITH GENIE-land % # Fixed present-day vegetation

metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_ebatmos='true'; metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_goldsteinocean='true'; metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_goldsteinseaice='true'; metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_land='true'; metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_igcmatmos='false'; metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_fixedocean='false'; metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_fixedseaice='false'; metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_fixedicesheet='false'; metadata.genie_main.Parameter.GENIE_CONTROL_NML.flag_fixedchem='false';

% # DP flags are important for global water and energy % # conservation tests % GENIEDP=TRUE % IGCMATMOSDP=TRUE % % GENIENXOPTS='$(DEFINE)GENIENX=36' % GENIENYOPTS='$(DEFINE)GENIENY=36'

% # this is to only write ocean-grid data in genie-main metadata.genie_main.Parameter.GENIE_CONTROL_NML.write_flag_atm='false'; metadata.genie_main.Parameter.GENIE_CONTROL_NML.write_flag_sic='false';

% # this is to write genie-main data every 50000 timesteps=100 years metadata.genie_main.Parameter.GENIE_CONTROL_NML.dt_write=50000;

% # this is to control embm output periods % #  npstp='health check' from 1000=10 years to 50000=500 years % #  iwstp='restarts' from 50000=500 years to 10000=100 years % #  itstp='time series' from 100=1 year to 5000=50 years % #  ianav='an average' stays 50000=500 years metadata.genie_embm.Parameter.npstp_embm=50000; metadata.genie_embm.Parameter.iwstp=10000; metadata.genie_embm.Parameter.itstp=5000; metadata.genie_embm.Parameter.ianav=50000;

% # this is to control goldstein output periods % #  npstp='health check' from 1000=10 years to 50000=500 years % #  iwstp='restarts' from 50000=500 years to 10000=100 years % #  itstp='time series' from 100=1 year to 5000=50 years % #  ianav='an average' stays 50000=500 years metadata.genie_goldstein.Parameter.npstp_goldstein=50000; metadata.genie_goldstein.Parameter.iwstp=10000; metadata.genie_goldstein.Parameter.itstp=5000; metadata.genie_goldstein.Parameter.ianav=50000;

% # this is to control seaice output periods % #  npstp='health check' from 1000=10 years to 50000=500 years % #  iwstp='restarts' from 50000=500 years to 10000=100 years % #  itstp='time series' from 100=1 year to 5000=50 years % #  ianav='an average' stays 50000=500 years metadata.genie_seaice.Parameter.npstp_seaice=50000; metadata.genie_seaice.Parameter.iwstp=10000; metadata.genie_seaice.Parameter.itstp=5000; metadata.genie_seaice.Parameter.ianav=50000;

% # this is to turn graphics off metadata.genie_main.Parameter.GENIE_CONTROL_NML.lgraphics='false';

% # this is to change the model run length % #  720=1 month of igcm (timestep=1 hour) to 2500000=5000 years of c-goldstein metadata.genie_main.Parameter.GENIE_CONTROL_NML.koverall_total=2500000;

% # this changes the relative atmos/ocean/seaice calling frequency metadata.genie_main.Parameter.GENIE_CONTROL_NML.ksic_loop=5; metadata.genie_main.Parameter.GENIE_CONTROL_NML.kocn_loop=5; metadata.genie_main.Parameter.GENIE_CONTROL_NML.klnd_loop=5;

% ######################################### % # GENIE-land config changes % ######################################### metadata.genie_land.Parameter.GENIELAND_CONTROL.idiag_land=5000; metadata.genie_land.Parameter.GENIELAND_CONTROL.irest_land=50000; metadata.genie_land.Parameter.GENIELAND_CONTROL.iacc_land=5000; metadata.genie_land.Parameter.GENIELAND_CONTROL.c_restart='../../genie-land/data/input/land_rs_embm_36x36.nc'; metadata.genie_land.Parameter.GENIELAND_CONTROL.c_runoff_fl='../../genie-land/data/input/runoff_mask_gold36x36.nc'; % =========================================================================

% Sort the fields in the structures according to the goin dictionary dict=goin_dictionary; [members indices]=ismember(fieldnames(metadata.genie_embm.Parameter), dict); list=dict(sort(indices)); metadata.genie_embm.Parameter=orderfields(metadata.genie_embm.Parameter,list); [members indices]=ismember(fieldnames(metadata.genie_goldstein.Parameter), dict); list=dict(sort(indices)); metadata.genie_goldstein.Parameter=orderfields(metadata.genie_goldstein.Parameter,list); [members indices]=ismember(fieldnames(metadata.genie_seaice.Parameter), dict); list=dict(sort(indices)); metadata.genie_seaice.Parameter=orderfields(metadata.genie_seaice.Parameter,list);

% Specify the experiment identifier EXPID = 'genie_eb_go_gs_ml';

</li> </ol>

How do I create a model archiving function for the GENIE Toolbox?
A template is provided to aid construction of a function to archive GENIE models and their runtime files. A genie_model_config.m function should exist before creating this archive function because we want to tie the EXPID parameter to the contents of the code archive that we create. To create an archival script: <ol> <li>Navigate to the GENIEToolbox/configs directory in your installtion of GENIELab</li> <li>Create a copy of the GENIE Toolbox archive template with a name corresponding to the model instance. <ul> <li><tt>$ cp genie_model_archive.m.template genie_eb_go_gs_ml_archive.m</tt></li> </ul> </li> <li>Edit the new archive script and replace all instances of the string <tt>model</tt> with the model descriptor for this archive file. <ul> <li>Search and replace: <tt>model</tt> --> <tt>eb_go_gs_ml</tt></li> </ul></li> <li>Check that the function obtains the <tt>EXPID</tt> value from the appropriate model configuration function % Get the experiment identifier from genie_eb_go_gs_ml.config [md, EXPID] = genie_eb_go_gs_ml_config; </li> <li>Update the <tt>FILELIST</tt> cell array with the locations of the files that need to be included in the archive. The files are specified relative to the CODEDIR which contains genie-main etc. The file definitions should be single quoted strings with the relative paths to the files. The <tt>*</tt> wildcard can be used to specify multiple files. Please consult module documentation to find out which data files are likely to be required. % Files to be included in the archive FILELIST = { ['genie_output/' EXPID '/genie.exe'], ...            'genie-main/inputdata/zjmap.dat', ... 'genie-main/data/input/main_restart_0.nc' 'genie-embm/data/input/tauy_u.interp', ...            'genie-embm/data/input/ta_ncep.silo', ...             'genie-embm/data/input/worbe2.k1', ...             'genie-embm/data/input/taux_v.interp', ...             'genie-embm/data/input/tauy_v.interp', ...             'genie-embm/data/input/uncep.silo', ...             'genie-embm/data/input/qa_ncep.silo', ...             'genie-embm/data/input/taux_u.interp', ...             'genie-embm/data/input/vncep.silo', ...             'genie-goldstein/data/input/tauy_u.interp', ...             'genie-goldstein/data/input/worbe2.k1', ...             'genie-goldstein/data/input/taux_v.interp', ...             'genie-goldstein/data/input/saliann.silo', ...             'genie-goldstein/data/input/worbe2.psiles', ...             'genie-goldstein/data/input/tauy_v.interp', ...             'genie-goldstein/data/input/worbe2.paths', ...             'genie-goldstein/data/input/taux_u.interp', ...             'genie-goldstein/data/input/tempann.silo', ...             'genie-land/data/input/land_rs_embm_36x36.nc', ...             'genie-land/data/input/runoff_mask_gold36x36.nc', ... 'genie-seaice/data/input/worbe2.k1' }; </li> <li>If you are unsure of what files to archive a reasonable fall back position might be to include all content from the appropriate module data/input directories % Files to be included in the archive FILELIST = { ['genie_output/' EXPID '/genie.exe'], ...            'genie-main/inputdata/*', ...             'genie-main/data/input/*', ...             'genie-embm/data/input/*', ...             'genie-goldstein/data/input/*', ...             'genie-land/data/input/*', ... 'genie-seaice/data/input/*' }; </li> </ol>