Overview
A machine in ES-DOC is defined as a computer or system or platform which is used for simulation. A CMIP6 machine is used to indicate a machine used to run one or more CMIP6 simulations.
The machines used to run CMIP6 simulations are documented and published using spreadsheets; one spreadsheet is used to describe every machine used by an institute for CMIP6.
Below are instructions for collecting information about one or more machine(s) using this spreadsheet.
1. Open the spreadsheet using Microsoft Excel, LibreOffice, or another spreadsheet editor.
The spreadsheet is found in the institutional GitHub repository in the cmip6/machines directory (see the layout of institutional GitHub repositories).
The spreadsheet as provided is composed of three pages, however institutes with multiple machines to document are expected to copy the final tab (see the next section) so the final number of tabs may be more than this.
2. Copy the third tab for as many machines as there are to document
The page breakdown of the spreadsheet is as follows:
- The first page is a title page which contains a link to these instructions.
- The second page contains an example of a documented machine to demonstrate how it works.
- The third page is a tab for documenting one CMIP6 machine only. If there is more than one CMIP6 machine to document, first create a copy of this tab for every other machine to be documented.
Therefore an institute with N machines should make N-1 copies of the third tab, resulting in a total of N+2 tabs inclusive of the title and example page.
It may be helpful to rename any copied tabs ‘Machine X’ where X is some number to distinguish arbitrarily between machines if there are multiple to document, but the tab names will not be parsed so they do not matter and only need to be different to the names of other tabs.
The third page and any copies should then be filled out to document each machine as fully as possible, as described in the next sections.
3. Document each machine (sections 1.1-1.7 for all ‘Machine 1’ tabs)
For a given tab to document a specific machine, there are seven sections in which to describe the nature of the machine, as summarised below:
- Identity: the name of the machine and any partitioning information;
- General properties: general information about the machine;
- Vendor information: vendor-related information about the machine;
- Compute pools: layout of compute nodes;
- Storage pools: storage resource(s) available;
- Interconnect: the interconnect used within a machine to join nodes together;
- Benchmark performance: the relative generic performance of a machine.
These sections are outlined in a linear fashion where headings are provided as highlighted in blue or purple and further detail is provided below each heading to indicate the information to provide for each item. Underneath the sectioning number in the leftmost columns is text that describes the type of input required for each item (a string or a float for example, with ‘ENUM’ indicating a drop-down box with a limited number of choices). For floats, units may be provided if required in context.
Grey boxes indicate items to be filled out. Note that items which have an asterisk mark (*) immediately following the sectioning number (for example ‘1.1.1 *’) are compulsory and must be answered. Other sections are optional, but the more information that is provided the more comprehensive and useful the resulting documentation, therefore effort to answer optional sections is greatly appreciated and indeed worthwhile.
As a general guide, here is a breakdown of the answering of two items, namely 1.5.1.1 and 1.5.1.2, as shown in the image below:
Note both of these items are compulsory (due to the asterisk mark). To complete these items, for example:
- In the grey entry box under the Name heading, the (string) name of a storage pool of the machine would be entered.
- Similarly, in the grey box under ‘File system size’ an appropriate floating point number describing the machine’s file system would be entered, based on the units provided. Or, if there were multiple file systems to document, the row with the grey box would first be copied the necessary number of times directly underneath the location of the row of the existing grey box, and then each box would be populated with a floating point covering the size in terabytes of each file system.
In a similar fashion, work through the tab sections 1.1-1.7 for every tab corresponding to every CMIP6 machine there is to document.
For the top-level sections 1.4 (Compute pools) and 1.5 (Storage pools) , note that it is expected that most institutes will only have a single compute pool or storage pool to document, in which case the items for 1.4.1 and 1.5.1 should be completed as much as possible but items 1.4.2 and 1.5.2, designed to cover a potential second compute and storage pool (respectively), can be skipped and left blank.
4. Indicate all models which were run on each machine (section 1.8 for all ‘Machine 1’ tabs)
It is vital to know the models which were run on all machines documented in this stage (stage one) in order to put into context the performances which will be documented for each machine in the next stage (two) of the Machine and Performance documentation process.
Therefore section 1.8 for each tab is to specify all models applicable to the given machine.
To make this as painless as possible, all of the models that each institute is known to run will be pre-processed into the spreadsheet as separate items under this section. Therefore to complete section 1.8 for each machine tab, select from the drop-down boxes whether or not each model listed was run on the machine in question.
5. Indicate all experiments which were run on each machine (section 1.9 for all ‘Machine 1’ tabs)
For the same reason as given at the start of part (4) above, it is also vital to know the experiments which were run on all machines documented in this stage.
Therefore section 1.9 for each tab is to specify all experiments applicable to the given machine.
The most likely case, and the certain case for an institute having only a single CMIP6 machine, is that all experiments the institute is participating in were run on a given machine, in which case ‘ALL’ can be indicated for item 1.9.1 to complete the whole of section 1.9.
Otherwise, all of the MIPs that each institute is known to run have been pre-processed under section 1.9 as separate items, with all corresponding experiments listed in the drop-down choice box for each MIP, so please go through and specify for each MIP if all, some or none of the experiments for the MIP were run on the machine in question. If it is some but not all, the grey entry boxes should be copied as appropriate to list out any applicable experiments. If it is none then leave the box blank.
6. Upload the updated machines information to the institutional GitHub repository
When complete, upload the populated machines spreadsheet to the institutional GitHub repository.
When you have done so, the ES-DOC team will be in touch with further instructions to initiate the second stage for documenting the representative performance of each machine described in the spreadsheet.