MPI Info and PnetCDF API

The PnetCDF routines ncmpi_create, ncmpi_open, nfmpi_create, and nfmpi_open take a hint parameter. That parameter is usually MPI_INFO_NULL, which in most cases is just fine.

If you create an MPI-IO Info, and set it up with some "hints", then you can direct the MPI implementation to perhaps take a different action. Implementations can ignore MPI-IO hints, so if you are running on an implementation that does not understand a particular hint, that's ok.

PnetCDF Hints

Below is the list of hints that recognized by PnetCDF.

• "nc_header_align_size" -- aligns the header size of a newly created file. The default is 512 bytes if file striping size is unknown. If the file striping size is know and the aggregate variable size is larger than 4 times the file striping size, the default is set to the file striping size.
• "nc_var_align_size" -- aligns the starting file offsets of each non-record variable. The default is 512 bytes if file striping size is unknown. If the file striping size is know and the aggregate variable size is larger than 4 times the file striping size, the default is set to the file striping size.
• "nc_header_read_chunk_size" -- PnetCDF reads the file headers in chunks. This hint indicates the chunk size (in bytes). The default is 256 KB.
• "nc_record_align_size" -- aligns the starting file offset of the record variable section. Note this is for the entire section, not individual records.

Using MPI Info objects

At a high-level, here's how PnetCDF uses MPI Info objects Here's how you might be able to use hints in your code.

1. create an info object
2. set hints
3. pass the hints to create/open

Useful Hints

On the Lustre file system, especially on Cray systems, we've seen quite high file locking costs. For some workloads, the optimizations for noncontiguous access might actually prove to be slower.

• set "romio_ds_read" "disable"
• set "romio_ds_write" "disable"

or

• try setting "ind_wr_buffer_size" to "16777216"

Examples

 INTEGER INFO, IERROR 
 CHARACTER*(*) KEY, VALUE

 CALL MPI_INFO_CREATE(INFO, IERROR)
 CALL MPI_INFO_SET(INFO, KEY, VALUE, IERROR)
 IERROR = NFMPI_CREATE(MPI_COMM_WORLD, "myfile.nc", flags, INFO, NCID)
 CALL MPI_INFO_FREE(INFO, IERROR) 

More Information

The MPI-2 standard's chapter on MPI Info objects