Build PnetCDF Library

• INSTALL contains detailed instructions and required software for building PnetCDF library.
• Several README files contain build recipes for specific platforms, including
  • README.CRAY
  • README.Fujitsu
  • README.IBM
  • README.INTEL
  • README.K-Computer
  • README.LINUX
  • README.SGI
  • README.SX
  • README.Ubuntu

PnetCDF API Reference

• PnetCDF C interface guide can be found here.

Tutorial

• A Quick Tutorial demonstrates some simple aspects of the library, with full code examples.
• Additional example programs in C, Fortran 77, Fortran 90, and C++ are provided in all PnetCDF source releases under the directory "examples". Refer examples/README for brief descriptions of individual example programs.

Performance Tuning

• For performance improvement, some general guide lines are available in Questions and Answers.
• I/O hints for PnetCDF describes some low-level tuning parameters
  • File offset alignment documents a way to adjust the layout of file header and variables in a dataset, and why/when you might want to do so
• Nonblocking APIs are introduced as an I/O optimization in version 1.1. They can be used to aggregate multiple I/O requests into a larger one for better performance.
  • Usage: users first post multiple nonblocking I/O requests and then make a call to the wait API to commit the requests. The wait call combines all the pending requests into one bigger MPI I/O request.
  • Buffered write API describes the non-blocking routines that copy data to a temporary buffer. The advantage is the user buffers can freely be modified once the requests are posted.
• Subfiling is a mechanism to internally partition a netCDF file into multiple files (subfiles), making the netCDF data appear as a single file to users. Subfiling can reduce file access contention when a large number of processes make concurrent I/O requests to a shared file.

PnetCDF API design document

• In each release, the file pnetcdf-api.pdf (built by running command 'make' in directory doc/) describes how the PnetCDF APIs are designed. We have designed the APIs to be more friendly to parallel I/O while maintaining file format compatibility with the serial version of NetCDF.

Unix man pages

• Unix man pages for C, Fortran 77 and 90 are also available together with the release.
• Unix commands: 'man pnetcdf', 'man pnetcdf_f77', and 'man pnetcdf_f90'

Publications and presentations

• Jianwei Li, Wei-keng Liao, Alok Choudhary, Robert Ross, Rajeev Thakur, William Gropp, Rob Latham, Andrew Siegel, Brad Gallagher, and Michael Zingale. Parallel netCDF: A Scientific High-Performance I/O Interface. In the Proceedings of Supercomputing Conference, November, 2003. ( PDF, 97k)
• The above paper discusses the design of PnetCDF library and presents some performance results.
• Jianwei Li's presentation from the SC 2003 conference ( PDF, 167k).

Hacking

• start with SourceCodeWalkthrough
• ProjectIdeas?
• TestCoverage
• DesignDiscussions: we discuss the design of PnetCDF over email mostly. We try to collect the highlights here.

Others

• NetCDF-4 can use PnetCDF. We think this is great. More information: PnetCDF and Netcdf-4
• Forrest Hoffman wrote an article about PnetCDF, "High Performance I/O: Parallel netCDF", in the July 2004 issue of Linux Magazine.
• The HDF group at NCSA ported a serial NetCDF code to one using PnetCDF. They posted a writeup of their efforts . It's a little old but does provide some additional information to supplement doc/porting_notes.txt
• Unidata's serial NetCDF documentation sometimes comes in handy for comparison.