Communicators

All of the examples in this chapter use only one communicator, which links all processes. However, it is possible to create more communicators to link subsets of processes. This is especially useful for collective operations that don’t need to be executed by all processes.

#include <boost/mpi.hpp>
#include <boost/serialization/string.hpp>
#include <string>
#include <iostream>

int main(int argc, char *argv[])
{
  boost::mpi::environment env{argc, argv};
  boost::mpi::communicator world;
  boost::mpi::communicator local = world.split(world.rank() < 2 ? 99 : 100);
  std::string s;
  if (world.rank() == 0)
    s = "Hello, world!";
  boost::mpi::broadcast(local, s, 0);
  std::cout << world.rank() << ": " << s << '\n';
}

Example 47.15 uses the function boost::mpi::broadcast(). This function sends the string “Hello, world!” from the process with rank 0 to all processes that are linked to the communicator local. The process with rank 0 has to be linked to that communicator, too.

The communicator local is created by a call to split(). split() is a member function called on the global communicator world. split() expects an integer to link processes together. All processes that pass the same integer to split() are linked to the same communicator. The value of the integer passed to split() doesn’t matter. All that’s important is that all processes that should be linked by a particular communicator pass the same value.

In Example 47.15, the two processes with the ranks 0 and 1 pass 99 to split(). If the program is started with more than two processes, the additional processes pass 100. This means the first two processes have one local communicator and all the other process have another local communicator. Every process is linked to the communicator that is returned by split(). Whether there are other processes linked to the same communicator depends on whether other processes passed the same integer to split().

Please note that ranks are always relative to a communicator. The lowest rank is always 0. In Example 47.15, the process with the rank 0 relative to the global communicator has also the rank 0 relative to its local communicator. The process with the rank 2 relative to the global communicator has the rank 0 relative to its local communicator.

If you start Example 47.15 with two or more processes, Hello, world! will be displayed twice – once each by the processes with the ranks 0 and 1 relative to the global communicator. Because s is set to “Hello, world!” only in the process with the global rank 0, this string is sent through the communicator only to those processes that are linked to the same communicator. This is just the process with global rank 1, which is the only other process that passed 99 to split().

#include <boost/mpi.hpp>
#include <boost/serialization/string.hpp>
#include <boost/range/irange.hpp>
#include <boost/optional.hpp>
#include <string>
#include <iostream>

int main(int argc, char *argv[])
{
  boost::mpi::environment env{argc, argv};
  boost::mpi::communicator world;
  boost::mpi::group local = world.group();
  boost::integer_range<int> r = boost::irange(0, 1);
  boost::mpi::group subgroup = local.exclude(r.begin(), r.end());
  boost::mpi::communicator others{world, subgroup};
  std::string s;
  boost::optional<int> rank = subgroup.rank();
  if (rank)
  {
    if (rank == 0)
      s = "Hello, world!";
    boost::mpi::broadcast(others, s, 0);
  }
  std::cout << world.rank() << ": " << s << '\n';
}

MPI supports grouping processes. This is done with the help of the class boost::mpi::group. If you call the member function group() on a communicator, all processes linked to the communicator are returned in an object of type boost::mpi::group. You can’t use this object for communication. It can only be used to form a new group of processes from which a communicator can then be created.

boost::mpi::group provides member functions like include() and exclude(). You pass iterators to include or exclude processes. include() and exclude() return a new group of type boost::mpi::group.

Example 47.16 passes two iterators to exclude() that refer to an object of type boost::integer_range. This object represents a range of integers. It is created with the help of the function boost::irange(), which expects a lower and upper bound. The upper bound is an integer that doesn’t belong to the range. In this example, this means that r only includes the integer 0.

The call to exclude() results in subgroup being created, which contains all processes except the one with the rank 0. This group is then used to create a new communicator others. This is done by passing the global communicator world and subgroup to the constructor of boost::mpi::communicator.

Please note that others is a communicator that is empty in the process with rank 0. The process with rank 0 isn’t linked to this communicator, but the variable others still exists in this process. You must be careful not to use others in this process. Example 47.16 prevents this by calling rank() on subgroup. The member function returns an empty object of type boost::optional in processes that don’t belong to the group. Other processes receive their rank relative to the group.

If rank() returns a rank and no empty object of type boost::optional, boost::mpi::broadcast() is called. The process with rank 0 sends the string “Hello, world!” to all processes linked to the communicator others. Please note that the rank is relative to that communicator. The process with the rank 0 relative to others has the rank 1 relative to the global communicator world.

If you run Example 47.16 with more than two processes, all processes with a global rank greater than 0 will display Hello, world!.