Getting Started

Manual Build

Toolchain and Build Dependencies

GNU Make > 4.2
C/C++17 compiler (GCC, Clang, Cray, Intel, IBM XL, NVHPC).
Optional: Fortran compiler (GFortran, Cray, Intel, IBM XL, NVHPC).
Optional: MPI implementation supporting mpicc, mpicxx, and mpif90 compiler wrappers.
Optional: NVIDIA CUDA >= 11.8
Optional: AMD HIP (ROCm)

These are likely provided in some form by either your distribution’s package manager or pre-installed and loaded via commands such as with Environment Modules.

Library Dependencies

These dependencies can also come from package managers or modules, but they must be built with a specific configuration and the same compiler toolchain that you will use to build OP2.

Optional: (PT-)Scotch: Used for mesh partitioning. You must build both the sequential Scotch and parallel PT-Scotch with 32-bit indicies (-DIDXSIZE=32) and without threading support (remove -DSCOTCH_PTHREAD).
Optional: ParMETIS: Used for mesh partitioning.
Optional: KaHIP: Used for mesh partitioning.
Optional: HDF5: Used for HDF5 I/O. You may build with and without --enable-parallel depending on whether MPI support is needed, and then specify both builds using the environment variables listed below.

Note

Building the MPI-enabled OP2 libraries require a parallel HDF5 build. A sequential HDF5 build is needed only for HDF5 support in the sequential OP2 libraries.

Building

Clone the repository:

git clone https://github.com/OP-DSL/OP2-Common.git
cd OP2-Common

Select your compiler:

export OP2_COMPILER={gnu, cray, intel, xl, nvhpc}

Alternatively, for a greater level of control:

export OP2_C_COMPILER={gnu, clang, cray, intel, xl, nvhpc}
export OP2_C_CUDA_COMPILER={nvhpc}  # optional
export OP2_C_HIP_COMPILER={hip}  # optional
export OP2_F_COMPILER={gnu, cray, intel, xl, nvhpc}   # optional

Note

In some scenarios you may be able to use a profile rather than specifying an OP2_COMPILER. See Makefile-README for more information.

Set library paths (if needed):

export PTSCOTCH_INSTALL_PATH=<path/to/ptscotch>
export PARMETIS_INSTALL_PATH=<path/to/parmetis>
export KAHIP_INSTALL_PATH=<path/to/kahip>
export HDF5_{SEQ, PAR}_INSTALL_PATH=<path/to/hdf5>

export CUDA_INSTALL_PATH=<path/to/cuda/toolkit>
export HIP_INSTALL_PATH=<path/to/hip/rocm>

Note

You may not need to specify the X_INSTALL_PATH varaibles if the include paths and library search paths are automatically injected by your package manager or module system.

If you are using CUDA or HIP, you may also specify a comma separated list of target architectures for which to generate code for:

export NV_ARCH={Pascal, Volta, ..., Hopper}[,{Pascal, ...}]
export HIP_ARCH={gfx803, gfx90a, ..., gfx908}[,{gfx803, ...}]

Configure the build:

make -C op2 config

Note

Check the terminal log to ensure the compilers, libraries, and flags are as expected.

Build OP2 library and an example app:

make -C op2 -j$(nproc)
make -C apps/c/airfoil/airfoil_plain/dp -j$(nproc)

Note

A new folder generated will be created inside the example app folder containing the generated source files. The compiled executable will be in the example app folder.

Warning

MPI builds require an MPI wrapper (mpicxx) pointing to the compiler defined by OP2_COMPILER. You can manually set the MPI executable path using MPI_INSTALL_PATH.

Application Build Variants

When building an application, the following parallelisation variants are available as Make targets:

Target	Description
`seq`	Single-threaded sequential build using the developer sequential library.
`genseq`	Code-generated sequential build (translator-v2 `seq` target). Recommended over `seq` for performance measurement.
`openmp`	Multi-threaded CPU build using OpenMP.
`cuda`	NVIDIA GPU build using CUDA (translator-v2 `cuda` target, ahead-of-time compiled).
`hip`	AMD GPU build using HIP (translator-v2 `hip` target, ahead-of-time compiled).
`c_cuda`	NVIDIA GPU build using CUDA with JIT compilation (translator-v2 `c_cuda` target). Device kernels are compiled at application start-up using NVRTC, enabling runtime specialisation.
`c_hip`	AMD GPU build using HIP with JIT compilation (translator-v2 `c_hip` target). Device kernels are compiled at application start-up using the HIP RTC library.
`mpi_<variant>`	Distributed-memory MPI variant of any of the above (e.g. `mpi_cuda`, `mpi_c_hip`). Requires an MPI-enabled OP2 library build.

For example, to build the JIT CUDA variant of the Airfoil benchmark:

make -C apps/c/airfoil/airfoil_plain/dp c_cuda

See Code Generation for details on how to generate the required source files for each variant.

Fortran Application Build Variants

Fortran application variants are prefixed with f_:

Target	Description
`f_seq`	Sequential Fortran build.
`f_openmp`	OpenMP multi-threaded Fortran build.
`f_cuda`	Native CUDA Fortran build. Requires a CUDA Fortran-capable compiler (NVHPC).
`f_c_cuda`	Fortran interop with JIT CUDA kernels (recommended GPU target for Fortran).
`f_c_hip`	Fortran interop with JIT HIP kernels.
`f_mpi_<variant>`	Distributed-memory MPI variant of any of the above.

For example, to build the Fortran Airfoil benchmark with JIT CUDA:

make -C apps/fortran/airfoil f_c_cuda

See OP2 Fortran 90 API for the Fortran API reference and Code Generation for Fortran code generation targets.

Spack

A Spack package for OP2 is not yet available. Building from source using the manual steps above is currently the recommended installation method.

If you are using a Spack-managed environment, the required compilers and libraries (MPI, CUDA, HDF5) will generally be available through the Spack-generated environment or compiler wrappers. Once the appropriate modules or environment is activated, follow the manual build steps. You do not need to set X_INSTALL_PATH variables if the include and library paths are already injected by the module system.