Home/ Questions/Q 6812191
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-26T20:25:09+00:00

I have a very simple compound dataset, about 1 million rows of a compound

  • 0

I have a very simple compound dataset, about 1 million rows of a compound datatype:

1 long, 3 doubles.

I would like to read it in parallel with MPI using a collective call, equally distributed on all the processors.

HDF5 low-level interface is very complex to use, and for such a simple dataset I wonder if I can use one of the high level API, like Lite or Table.
What is not clear from the documentation is if these APIs support MPI collective read.

Could somebody write the simplest code snippet that can read this simple dataset into a C structure using HDF5 1.8?

I believe for an expert this should be trivial, but for a beginner the complexity of HDF5 (and its docs) makes it really a daunting task.

thanks.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Have you found the HDF5 parallel I/O tutorial?

    http://www.hdfgroup.org/HDF5/Tutor/parallel.html

    Sounds like you will also need to learn how to construct HDF5 datatypes:

    http://www.hdfgroup.org/HDF5/Tutor/datatypes.html

    In order to do a parallel collective call you’ve got to do a few things. First, you need to decompose your dataset across the processors. A little arithmatic gives you the start and count parameters you’ll need. You will need to enable parallel I/O with an HDF5 property list, which is well documented in the parallel I/O tutorial. Less well documented is the flag for enabling collective I/O:

    xfer_plist = H5Pcreate (H5P_DATASET_XFER);
ret=H5Pset_dxpl_mpio(xfer_plist, H5FD_MPIO_COLLECTIVE);

    There’s a great example at the end of this page:

    http://www.hdfgroup.org/Parallel_HDF/PHDF5/ph5design.html

    • 0