# Welcome to DORiE! (**D**UNE-**O**perated **Ri**chards equation solving **E**nvironment) DORiE is a software suite for solving the Richards Equation. The core feature is a C++ PDE-solver powered by [DUNE](https://dune-project.org/) and the [DUNE-PDELab](https://dune-project.org/modules/dune-pdelab/) module. It implements a Discontinous Galerkin (DG) discretization scheme on structured rectangular / cubic and unstructured simplex grids in two and three spatial dimensions, and makes use of advanced features like adaptive grid refinement. The suite encapsulates a documentation and various tools for program setup, program testing, and output analysis, which are mostly written in Python. DORiE is developed and maintained by the [DORiE Developers](mailto:dorieteam@iup.uni-heidelberg.de) of the [TS-CCEES](http://ts.iup.uni-heidelberg.de/) research group at [IUP Heidelberg](http://www.iup.uni-heidelberg.de/), supervised by [Kurt Roth](http://ts.iup.uni-heidelberg.de/people/prof-dr-kurt-roth/), in collaboration with [Ole Klein](https://conan.iwr.uni-heidelberg.de/people/oklein/) and the [Scientific Computing Group](https://conan.iwr.uni-heidelberg.de/) of [IWR Heidelberg](https://typo.iwr.uni-heidelberg.de/home/). # Installation Instructions DORiE is a [DUNE](https://dune-project.org/) module and requires several other DUNE modules as well as third party software packages. Installation can be handled manually on your local machine, but for inexperienced users it is recommended to use the deployment software [Docker](https://www.docker.com/) instead. No installation is necessary if you download DORiE as Docker image from [Docker Hub](https://hub.docker.com/r/dorie/dorie/). You can then proceed directly to the the instructions on [how to execute DORiE](#run-dorie-run). The commands listed there are appended to the usual commands for running a Docker container. See the description on Docker Hub for further details. ## Download Docker image If you want to use any stable version of DORiE, or the most recent unstable version, you can download the appropriate images from Docker Hub. To do so, execute docker pull dorie/dorie[:] Omitting the tag information downloads the image with tag `latest` which refers to the latest stable version. You can download any tag by specifying ``. The list of available tags can be found on Docker Hub and matches the release tags list of the Git repository. The latest unstable version is tagged as `devel`. ## Docker Installation You can also compile any local version of DORiE into a new Docker image. To do so, execute docker build -f docker/dorie.dockerfile -t . from within the top-level DORiE source directory. Docker will use a prepared image from Dockerhub and install DORiE into it. The created image will be called ``. ## Manual Installation Installing all packages manually can be quite an effort, but useful for developers who want to have easy access to the source files or users who prefer to run DORiE without the Docker overhead. Whenever possible, dependencies should be installed using a package manager, like [APT](https://wiki.ubuntuusers.de/APT/) on Ubuntu or [Homebrew](http://brew.sh/) on Mac. Manual installation on a Windows environment is strongly discouraged! DORiE is configured, built, and installed via the [DUNE Buildsystem](https://dune-project.org/doc/installation/), using the `dunecontrol` script to handle DUNE-internal dependencies. ### Dependencies Depending on your system configuration, there will be more packages necessary to install DORiE on your machine. See the step-by-step manual for further details. The specified versions are the _supported_ ones, where compatibility is ensured by CI tests. | Software | Version/Branch | Comments | | ---------| -------------- | -------- | | CMake | 3.10.2 | | GCC | 7.3 | Alternatively: LLVM Clang >=6, or Apple Clang 10 | git | | pkg-config | | HDF5 | 1.10 | with MPI support | FFTW3 | 3.3.7 | with MPI support | Python | 3.6 | | pip | 3.6 | | MPI | | Tested with OpenMPI 2.1.1 | SuperLU | 5.2 | | [yaml-cpp](https://github.com/jbeder/yaml-cpp) | >= 5.2.0 | | [spdlog](https://github.com/gabime/spdlog) | 1.1.0 | Included as Git Submodule | [dune-common](https://gitlab.dune-project.org/core/dune-common) | releases/2.6 | [dune-geometry](https://gitlab.dune-project.org/core/dune-geometry) | releases/2.6 | [dune-grid](https://gitlab.dune-project.org/core/dune-grid) | releases/2.6 | [dune-uggrid](https://gitlab.dune-project.org/staging/dune-uggrid) | releases/2.6 | [dune-istl](https://gitlab.dune-project.org/core/dune-istl) | releases/2.6 | [dune-localfunctions](https://gitlab.dune-project.org/core/dune-localfunctions) | releases/2.6 | [dune-functions](https://gitlab.dune-project.org/staging/dune-functions) | releases/2.6 | [dune-typetree](https://gitlab.dune-project.org/staging/dune-typetree) | releases/2.6 | [dune-pdelab](https://gitlab.dune-project.org/pdelab/dune-pdelab) | releases/2.6 | [dune-randomfield](https://gitlab.dune-project.org/oklein/dune-randomfield) | master ### Optional Packages | Software | Version/Branch | Comments | | ---------| -------------- | -------- | | [dune-testtools](https://gitlab.dune-project.org/quality/dune-testtools) | releases/2.6 | Handles system tests | [doxygen](http://www.stack.nl/~dimitri/doxygen/) | 1.8.13 | Builds documentation | [METIS](http://glaros.dtc.umn.edu/gkhome/views/metis) | 5 | For parallel runs | [ParMETIS](http://glaros.dtc.umn.edu/gkhome/views/metis) | 4 | For parallel runs ### Step-by-step Instructions These instructions are suitable for a clean **Ubuntu** or **macOS** setup. The main difference between the two systems is the package manager. Debian-based systems have the APT manager already built in. On Mac, we recommend installing [Homebrew](http://brew.sh/). If you prefer to use [MacPorts](https://www.macports.org/), you need to check if some of the packages require different installation options than displayed here. If you installed [Anaconda](https://conda.io/docs/user-guide/install/download.html) on your machine, you don't need to install Python or Pip. Simply skip these packages when using the package managers for installing the software. However, notice the warnings when compiling DORiE below! 1. **macOS** users need to start by installing the Apple Command Line Tools by executing xcode-select --install Make sure you have no pending software updates for your respective version of macOS! 2. Install third party packages: **Ubuntu:** apt update apt install cmake doxygen gcc g++ gfortran \ git libatlas-base-dev libfftw3-dev libfftw3-mpi-dev \ libfreetype6-dev libhdf5-mpi-dev libopenmpi-dev libpng-dev \ libsuperlu-dev libyaml-cpp-dev libxft-dev \ python3-dev python3-pip **macOS:** brew update brew install cmake doxygen gcc libpng open-mpi \ pkg-config python3 superlu yaml-cpp brew install hdf5 --with-mpi brew install fftw --with-mpi 3. The parallel linear solver of DORiE can make use of the ParMETIS package. If you want to run DORiE in parallel on multiple processes, additionally install METIS and ParMETIS: **Ubuntu:** apt install libmetis-dev libparmetis-dev **macOS:** _Support is dropped because ParMETIS is currently unavailable from Homebrew._ **Parallel runs without these two packages are possible but not supported!** 4. Clone the DUNE modules into a suitable folder on your machine. Use `git checkout` to switch to the correct branches (see above). 5. Clone DORiE into the same folder. DORiE includes [Git Submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules), which requires you to add the `--recurse-submodules` option to the `git clone` command. Switch to the desired branch or tag. 6. Enter the parent folder, and call CMAKE_FLAGS="-DDUNE_PYTHON_VIRTUALENV_SETUP=True -DDUNE_PYTHON_ALLOW_GET_PIP=True" ./dune-common/bin/dunecontrol all to build all DUNE modules. Additionally, you can add `MAKE_FLAGS="-j X"` before the call to `dunecontrol` to compile on `X` processes in parallel. If you installed software into paths not appended to your `PATH` variable, you will have to add `CMAKE_FLAGS` to the call to make sure that CMake finds all packages. Alternatively, you can add a custom options file. See the [DUNE Installation Instructions](https://dune-project.org/doc/installation/) for details. CMake will throw an error if required packages are not found. **Warning:** Anacoda supplies its own version of HDF5 which is typically found first by CMake. If you have Anaconda installed on your machine, add HDF5_ROOT=/usr/local to the `CMAKE_FLAGS` in the call to `dunecontrol` above. ### Experimental Features The local operator implementing Richards equation's discretization supports multiple scheme settings. Setting these via the config file is disabled by default. You can enable this feature by reconfiguring DORiE with the CMake flag `-DEXPERIMENTAL_DG_FEATURES=On`, and rebuilding it. The configuration settings in the section `[dg.experimental]` will then override the default settings. ## Recommended Third-Party Software The following software packages are cross-platform, so you should be able to find a release that fits your operating system: * [Gmsh](http://gmsh.info/): An open-source CAD that can be used to create the `.msh` files used by DORiE to define triangular meshes. * [ParaView](http://www.paraview.org/): A powerful post-processing tool for VTK files. Offers both visualization and data analysis tools. # Usage ## Documentation The documentation of the latest release branch can be found [online](https://dorie-doc.netlify.com/). The documentation can be built after DORiE has been properly configured (i.e., by calling `dunecontrol`). Note that you might have to re-configure DORiE once after installing it, because some dependencies are installed at configure time (e.g. by `dunecontrol --only=dorie configure`). To build the documentation, just run dunecontrol --only=dorie make doc or navigate to the `dorie/build-cmake` directory and call make doc You will then find the index page of the documentation at `dorie/build-cmake/doc/html/index.html`. ## Run, DORiE, Run! DORiE provides a command line interface (CLI) for all its functions. The required Python modules and all their dependencies are readily installed into a virtual environment (`venv`), which has to be activated within a shell session. You can do so by activating it in your current session (Manual Installation only) or by running the Docker application. ### Run the `venv` using the Docker application If you did not install DORiE locally, you can use the Docker application to boot up the virtual environment in a mounted directory of choice. Start up the Docker application by calling docker run -it -v :/mnt where you replace `` with a local directory for storing input and output data, and `` with `dorie/dorie[:]`. If you built a local Docker image, replace `` appropriately. You can also insert `$PWD` as `` if you want to mount your current working directory. The command boots up a (`bash`) shell inside a Docker container and mounts the directory `` and all its subdirectories into the directory `/mnt` inside the container. Your shell session starts in this directory with the virtual environment activated. Notice, that you can only use **local file paths** in all configuration settings due to the directory mount. ### Activate the `venv` locally To activate the virtual environment within your current shell session, execute source dorie/build-cmake/activate where you replace `` with the path to the appropriate directory. Your shell will now display the prefix `(dune-env)` to indicate that it is configured appropriately. You can exit the environent at any time by simply executing deactivate Notice that any virtual environment only applies to, and lasts for your current terminal session! _With the virtual environment activated,_ you can now navigate to any directory that you would like to contain your simulation input and/or output data. ### Execute the application Any command to the DORiE application has the signature dorie [] [] Using the `-h` or `--help` option, you can find all available commands and further help. To start your first simulation run, create a new directory and enter it. Place some exemplary configuration and boundary condition data files in there by calling dorie create DORiE implements a lightweight wrapper around the `dune-randomfield` generator. You can use it to easily create a heterogeneous soil architecture. This step is optional. Tweak the parameters of `parfield.ini` to your liking and then call dorie pfg parfield.ini The DORiE main routine is executed with the `run` command. Tweak the parameters of `config.ini` to your liking. You will need to reference several additional input files for soil parameters, boundary conditions, GMSH grid files (optional), and grid mappings (optional). Refer to the documentation for further information. Once prepared, call dorie run config.ini to execute the solver. ## Troubleshooting CMake heavily caches the results of its configuration process. In case you encounter errors or strange behavior, especially after an update, you should delete the DORiE build folder (called `build-cmake` by default) and re-install DORiE using `dunecontrol`. If the problem persists, take a look at the [List of Known Bugs](https://ts-gitlab.iup.uni-heidelberg.de/dorie/dorie/issues), or create an issue yourself. For problems related to the installation, refer to the sections below. ### Debugging DORiE can be built with debugging flags via CMake. To do so, run CMAKE_FLAGS="-DCMAKE_BUILD_TYPE=Debug" dunecontrol --only=dorie all After building, a debugger can hook into the executables. **Note:** If no `CMAKE_BUILD_TYPE` is specified during re-configuration, the last configuration build type is used. If no CMake files exist, it defaults to `Release`. You will find the actual value displayed in the final output of CMake. To re-create a release build, configure DORiE with the release build type by executing CMAKE_FLAGS="-DCMAKE_BUILD_TYPE=Release" dunecontrol --only=dorie all #### Debugging inside Docker A debugger needs special security privileges usually not provided by the Docker engine. To enable debugging inside Docker, these privileges have to be granted when calling `docker run` by adding the options --security-opt seccomp=unconfined The debugger `gdb` is pre-installed in the Docker container. ### Running System Tests DORiE includes a testing system for comparing its results the ones of ODE solvers or former versions of itself. This ensures that DORiE is running correctly and producing the expected results. We distinguish _unit tests_ for testing certain features of the code, and _system tests_ for verifying the results of the final application. As system tests require executing the DUNE solvers, it is recommended to build them in a `Release` environment. | Test category | Build tests | Execute tests | Recommended build type | | ------------- | ----------- | ------------- | ---------------------- | | Unit tests | `make build_unit_tests` | `make unit_tests` | `Debug` | System tests | `make build_system_tests` | `make system_tests` | `Release` The `make` commands are to be executed from within the `build-cmake` directory. The unit tests automatically include compiler flags for code coverage reports. After building them with `Debug` flags and executing them, you can retrieve code coverage information using the [`gcovr`](https://gcovr.com/index.html) utility. ### Further Help [Open an issue](https://ts-gitlab.iup.uni-heidelberg.de/dorie/dorie/issues/new), or write to the [DORiE developer mailing list](mailto:dorieteam@iup.uni-heidelberg.de).