CoSimIO with MPI support (C++ interface)

Main Page of Documentation

Table of Contents

• Building the CoSimIO with MPI support
• Number of processes
• Connecting
• Importing and Exporting

---

This document describes the C++ MPI interface of the CoSimIO for performing MPI-parallel coupled simulations. Most functionalities work in the same way in serial and MPI, the differences are pointed out and explained here.

Building the CoSimIO with MPI support

Please check the serial instructions first.

The CoSimIO has to be built with MPI support in order to perform mpi-parallel distributed coupled simulations. For this an MPI implementation (e.g OpenMPI or IntelMPI) is necessary. Set the CMake option CO_SIM_IO_BUILD_MPI to ON in order to compile the CoSimIO with MPI support.

One can use build_cpp.sh for compiling it. Check here for the available build options.

$ bash scripts/build_cpp.sh

The shared library co_sim_io_mpi will be installed in the bin/ folder (note that co_sim_io_mpi already links against co_sim_io and MPI). After building and linking it to your project, you may use the interface defined in co_sim_io_mpi.hpp (which already includes co_sim_io.hpp):

// CoSimulation includes
#include "co_sim_io_mpi.hpp"

int main()
{
    return 0;
}

With CMake this can be achieved with the following:

include_directories(path/to/co_sim_io)
target_link_libraries(my_executable co_sim_io_mpi)

Number of processes

Before going into the details of the interface, it is important to clarify that both connection partners must use the same number of processes! This means that each rank on one side has its dedicated partner rank on the other side of the interface.

Connecting

Please check the serial instructions first.

For using the CoSimIO in an mpi-parallel context, it is required to establish a connection with ConnectMPI instead of Connect. This is the first step to establish a connection to Kratos CoSimulation. Besides the settings, it takes an MPI_Comm communicator. This communicator should only have the ranks that contain the interface. It is also possible for it to contain all the ranks (e.g. MPI_COMM_WORLD) when some ranks are empty i.e. have no part of the coupling interface, but this is less efficient.

// The connect must be called before any CosimIO method
// mpi_comm_interface should only contain the ranks that have a part of the coupling interface
auto info = CoSimIO::ConnectMPI(settings, mpi_comm_interface);

Note that it is required to initialize MPI before calling ConnectMPI, e.g. with MPI_Init.

Everything else related to connecting and disconnecting works the same as in serial.

Importing and Exporting

The functions for importing and exporting don’t change, neither in terms of interface nor in terms of functionality. As explained above, both connection partners must use the same number of processes, and hence the ranks can directly communicate with each other.

Examples can be found here.

One minor difference is that ModelParts can have ghost nodes in case they are distributed across multiple ranks. See here for more details. The does not change the inteface of Im-/ExportMesh.