GENIE:GENIEToolboxUsefulInfo

Andrew Price, University of Southampton, ([mailto:a.r.price@soton.ac.uk a.r.price@soton.ac.uk])

Grid certificate management
To use the GENIE database system and any Grid resource managed by the Globus Toolkit you will need an X.509 cetrtificate. The UK e-Science community has its own Certification Authority and UK based members of the GENIE project can obtain a personal certificate from this authority. Details of how to obtain a UK e-Science personal certificate are avilable here along with instructions of how to prepare a certificate for use with the GENIE Toolbox.

In order to access the GENIE database or utilise Globus managed compute resource a proxy certificate must be instantiated. This is a time-limited self-signed certificate that authenticates you on the Grid. A proxy certificate can be instantiated in GENIELab by calling the gd_createproxy function.


 * >> gd_createproxy


 * [[image:gd_createproxy.png|gd_createproxy]]

The default proxy certificate will be valid for 12 hours after creation. It is important to note that any job that is submitted to a Globus managed resource is only valid for the lifetime of the proxy. If you were to submit a job that required 18 hours of compute time, the job would fail after 12 hours. The GENIE Toolbox insists that you specify a maximum execution time for a model run and will check that your proxy certificate will outlive the job. The default of 12 hours is a little restrictive - to change the lifetime of your proxy certificate click the Options button.


 * [[image:gd_createproxy-options.png|gd_createproxy Options]]

A user should only need to adjust the liftime entry for their certificate to ensure that the proxy will outlive the jobs that they wish to submit. Once a suitable lifetime has been selected click Apply to return to the creation dialog. The certificate password should be entered and the Create clicked.


 * [[image:gd_createproxy-creating.png|gd_createproxy Creating]]

Once the proxy certificate has been successfully created a dialog will inform the user.


 * [[image:gd_createproxy-created.png|gd_createproxy Created]]

Click OK to return to the original dialog. Click Cancel to close the dialog. Unfortunately, the third party library places Matlab in the paused state so a further key press is required to return to the Matlab prompt.

>> gd_createproxy Paused: Press any key...

Available compute resource
Common resource descriptors for platforms available to some / all members of the GENIE project:

 UK National Grid Service  Oxford</li> resource =

type: 'globus' name: 'NGS Oxford' MaxJobs: 16 broker: 'PBS' RemoteTargetOS: 'linux' RemoteHost: 'grid-compute.oesc.ox.ac.uk' RemoteRunDir: '' RemoteFileSep: '/' RemoteJobManager: 'jobmanager-pbs' RemoteMaxWallTime: 2880 jarutil: '/usr/bin/jar' JobsPerNode: 1 Leeds</li> resource =

type: 'globus' name: 'NGS Leeds' MaxJobs: 16 broker: 'PBS' RemoteTargetOS: 'linux' RemoteHost: 'grid-compute.leeds.ac.uk' RemoteRunDir: '' RemoteFileSep: '/' RemoteJobManager: 'jobmanager-pbs' RemoteMaxWallTime: 2880 jarutil: '/usr/bin/jar' JobsPerNode: 1 Rutherford</li> resource =

type: 'globus' name: 'NGS RAL' MaxJobs: 8 broker: 'LSF' RemoteTargetOS: 'linux' RemoteHost: 'grid-data.rl.ac.uk' RemoteRunDir: '' RemoteFileSep: '/' RemoteJobManager: 'jobmanager-lsf' RemoteMaxWallTime: 2880 jarutil: '/usr/bin/jar' JobsPerNode: 1 Manchester</li> resource =

type: 'globus' name: 'NGS Manchester' MaxJobs: 8 broker: 'PBS' RemoteTargetOS: 'linux' RemoteHost: 'grid-data.man.ac.uk' RemoteRunDir: '' RemoteFileSep: '/' RemoteJobManager: 'jobmanager-pbs' RemoteMaxWallTime: 2880 jarutil: '/usr/bin/jar' JobsPerNode: 1 </ul> Institutional Resource</li> <ul> Cluster1, University of East Anglia</li> resource =

type: 'globus' name: 'NGS Cluster1' MaxJobs: 50 broker: 'SGE' RemoteTargetOS: 'linux' RemoteHost: 'cluster1.uea.ac.uk' RemoteRunDir: '' RemoteFileSep: '/' RemoteJobManager: 'jobmanager-sge' RemoteMaxWallTime: 2880 jarutil: '/usr/bin/jar' JobsPerNode: 1 Pacifica2, University of Southampton</li> resource =

type: 'globus' name: 'NGS Pacifica2' MaxJobs: 32 broker: 'PBS' RemoteTargetOS: 'linux' RemoteHost: 'blue05.iridis.soton.ac.uk' RemoteRunDir: '' RemoteFileSep: '/' RemoteJobManager: 'jobmanager-pbs' RemoteMaxWallTime: 2880 jarutil: '/usr/bin/jar' JobsPerNode: 1 </ul> Condor</li> <ul> Linux</li> resource =

type: 'condor' name: 'Condor Pool - Linux Nodes' MaxJobs: 10 broker: 'condor' RemoteTargetOS: 'linux' RemoteFileSep: '/' Win32</li> resource =

type: 'condor' name: 'Condor Pool - Win32 Nodes' MaxJobs: 10 broker: 'condor' RemoteTargetOS: 'win32' RemoteFileSep: '\' </ul> </ul>

Firewall configuration
The GridFTP functionality of the Globus toolkit is built upon the WU-FTPD ftp daemon for unix. This daemon uses Authentication Server Protocol as part of the authentication process and listens on port 113 (IDENT/AUTH) for connections. The system dependent user identifier of the connection of interest is sent out on this connection. This communication can often be blocked by firewalls and the GridFTP process must wait for a 30 second timeout to occur before the file transfer can be performed:

http://www.wu-ftpd.org/wu-ftpd-faq.html#QA86

If you find that GriFTP commands (e.g. gc_findMyHomeDir, gd_putfile, gd_getfile) are taking about 30 seconds to complete then it is likely that incoming communications on port 113 are being blocked.


 * Check your operating system firewall first. It is increasingly common for operating systems to ship with a software firewall and for it to be enabled by default. If a firewall is operational on your computer then please look into allowing incoming TCP communications on port 113.
 * Windows firewall. In Windows XP and Windows XP x64 you can configure an exception in the firewall as follows:
 * Start</tt> --> Control Panel</tt> --> Network Connections</tt>
 * Right click Local Area Connection</tt> (or your active network connection) and select Properties</tt>
 * Select the Advanced</tt> tab and click on <tt>Settings...</tt> in the Windows Firewall section
 * Select the <tt>Exceptions</tt> tab
 * [[image:WindowsFirewall.png|Windows Firewall]]
 * Click <tt>Add port...</tt>
 * Enter a name for the exception and enter <tt>113</tt> as the port number
 * [[image:AddaPort.png|Add a Port]]
 * If you wish increase security you can click <tt>Change scope...</tt> and enter the addresses of the machines that you want to limit communications to
 * Click <tt>OK</tt> on the open dialogs to accept the change
 * If a local firewall does not seem to be the problem the you will need to contact the firewall administrator for your institution and discuss the possibility of opening this port to your machine

Script Hierarchy - Core functionality

 * [[image:CoreFunctionHierarchy.png|800px|Core functionality script hierarchy]]

Script Hierarchy - OptionsMatlab functionality

 * [[image:OptionsFunctionHierarchy.png|800px|OptionsMatlab functionality script hierarchy]]

Script Hierarchy - Collaborative ensemble functionality

 * [[image:DatabaseFunctionHierarchy.png|800px|Collaborative ensemble functionality script hierarchy]]