README.md 16.7 KB
Newer Older
1
# DORiE
2
(**D**UNE-**O**perated **Ri**chards equation solving **E**nvironment)
Dion Haefner's avatar
Dion Haefner committed
3

Lukas Riedel's avatar
Lukas Riedel committed
4
DORiE is a software suite for solving the Richards Equation. The core feature is a C++ PDE-solver powered by [DUNE](https://dune-project.org/) and the [DUNE-PDELab](https://dune-project.org/modules/dune-pdelab/) module. It implements a Discontinous Galerkin (DG) discretization scheme on structured rectangular / cubic and unstructured simplex grids in two and three spatial dimensions, and makes use of advanced features like adaptive grid refinement.
5 6 7

The suite encapsulates a documentation and various tools for program setup, program testing, and output analysis, which are mostly written in Python.

8
DORiE is developed and maintained by the [DORiE Developers](mailto:dorieteam@iup.uni-heidelberg.de) of the [TS-CCEES](http://ts.iup.uni-heidelberg.de/) research group at [IUP Heidelberg](http://www.iup.uni-heidelberg.de/), supervised by [Kurt Roth](http://ts.iup.uni-heidelberg.de/people/prof-dr-kurt-roth/), in collaboration with [Ole Klein](https://conan.iwr.uni-heidelberg.de/people/oklein/) and the [Scientific Computing Group](https://conan.iwr.uni-heidelberg.de/) of [IWR Heidelberg](https://typo.iwr.uni-heidelberg.de/home/).
9

10 11
DORiE is licensed under the [GNU General Public License v3.0](LICENSE).
For the copyright notice and the list of copyright holders,
Lukas Riedel's avatar
Lukas Riedel committed
12
see [COPYING.md](COPYING.md).
13

Lukas Riedel's avatar
Lukas Riedel committed
14
## Installation Instructions
15

Dion Haefner's avatar
Dion Haefner committed
16
DORiE is a [DUNE](https://dune-project.org/) module and requires several other DUNE modules as well as third party software packages. Installation can be handled manually on your local machine, but for inexperienced users it is recommended to use the deployment software [Docker](https://www.docker.com/) instead.
17

18 19 20 21 22
No installation is necessary if you download DORiE as Docker image from
[Docker Hub](https://hub.docker.com/r/dorie/dorie/). You can then proceed
directly to the the instructions on [how to execute DORiE](#run-dorie-run).
The commands listed there are appended to the usual commands for running a
Docker container. See the description on Docker Hub for further details.
23

Lukas Riedel's avatar
Lukas Riedel committed
24
### Download Docker image
25

Lukas Riedel's avatar
Lukas Riedel committed
26 27 28
If you want to use any stable version of DORiE, or the most recent unstable
version, you can download the appropriate images from Docker Hub. To do so,
execute
29

Lukas Riedel's avatar
Lukas Riedel committed
30
    docker pull dorie/dorie[:<tag>]
31

Lukas Riedel's avatar
Lukas Riedel committed
32 33 34 35 36
Omitting the tag information downloads the image with tag `latest` which
refers to the latest stable version. You can download any tag by specifying
`<tag>`. The list of available tags can be found on Docker Hub and matches
the release tags list of the Git repository. The latest unstable version is
tagged as `devel`.
37

Lukas Riedel's avatar
Lukas Riedel committed
38
### Docker Installation
39

Lukas Riedel's avatar
Lukas Riedel committed
40 41
You can also compile any local version of DORiE into a new Docker image. To do
so, execute
42

Lukas Riedel's avatar
Lukas Riedel committed
43
    docker build -f docker/dorie.dockerfile -t <img> .
44

Lukas Riedel's avatar
Lukas Riedel committed
45 46 47
from within the top-level DORiE source directory. Docker will use a prepared
image from Dockerhub and install DORiE into it. The created image will be called
`<img>`.
Dion Haefner's avatar
Dion Haefner committed
48

Lukas Riedel's avatar
Lukas Riedel committed
49
### Manual Installation
50

Lukas Riedel's avatar
Lukas Riedel committed
51
Installing all packages manually can be quite an effort, but useful for developers who want to have easy access to the source files or users who prefer to run DORiE without the Docker overhead.
Dion Häfner's avatar
Dion Häfner committed
52 53

Whenever possible, dependencies should be installed using a package manager, like [APT](https://wiki.ubuntuusers.de/APT/) on Ubuntu or [Homebrew](http://brew.sh/) on Mac. Manual installation on a Windows environment is strongly discouraged!
54

55 56 57 58
DORiE is configured, built, and installed via the
[DUNE Buildsystem](https://dune-project.org/doc/installation/), using the
`dunecontrol` script to handle DUNE-internal dependencies.

Lukas Riedel's avatar
Lukas Riedel committed
59
#### Dependencies
Lukas Riedel's avatar
Lukas Riedel committed
60 61 62 63 64
Depending on your system configuration, there will be more packages necessary to
install DORiE on your machine. See the step-by-step manual for further details.

The specified versions are the _supported_ ones, where compatibility is ensured
by CI tests.
65 66 67

| Software | Version/Branch | Comments |
| ---------| -------------- | -------- |
Lukas Riedel's avatar
Lukas Riedel committed
68
| CMake | 3.10.2 |
69
| GCC | 7.3 | Alternatively: LLVM Clang >=6, or Apple Clang 10
70 71
| git |
| pkg-config |
Lukas Riedel's avatar
Lukas Riedel committed
72 73 74 75 76 77
| HDF5 | 1.10 | with MPI support
| FFTW3 | 3.3.7 | with MPI support
| Python | 3.6 |
| pip | 3.6 |
| MPI | | Tested with OpenMPI 2.1.1
| SuperLU | 5.2 |
Lukas Riedel's avatar
Lukas Riedel committed
78
| [yaml-cpp](https://github.com/jbeder/yaml-cpp) | >= 5.2.0 |
79
| [spdlog](https://github.com/gabime/spdlog) | 1.1.0 | Included as Git Submodule
80
| [Google Test](https://github.com/google/googletest) | `HEAD` | Included as Git Submodule
81
| [muparser](http://beltoforion.de/article.php?a=muparser) | master |
Lukas Riedel's avatar
Lukas Riedel committed
82
| [VTK](https://vtk.org/) | >= 7.1.1 | For the Python module only
Lukas Riedel's avatar
Lukas Riedel committed
83 84 85 86 87 88 89 90 91
| [dune-common](https://gitlab.dune-project.org/core/dune-common) | releases/2.6
| [dune-geometry](https://gitlab.dune-project.org/core/dune-geometry) | releases/2.6
| [dune-grid](https://gitlab.dune-project.org/core/dune-grid) | releases/2.6
| [dune-uggrid](https://gitlab.dune-project.org/staging/dune-uggrid) | releases/2.6
| [dune-istl](https://gitlab.dune-project.org/core/dune-istl) | releases/2.6
| [dune-localfunctions](https://gitlab.dune-project.org/core/dune-localfunctions) | releases/2.6
| [dune-functions](https://gitlab.dune-project.org/staging/dune-functions) | releases/2.6
| [dune-typetree](https://gitlab.dune-project.org/staging/dune-typetree) | releases/2.6
| [dune-pdelab](https://gitlab.dune-project.org/pdelab/dune-pdelab) | releases/2.6
92
| [dune-randomfield](https://gitlab.dune-project.org/oklein/dune-randomfield) | releases/2.6
Lukas Riedel's avatar
Lukas Riedel committed
93 94


Lukas Riedel's avatar
Lukas Riedel committed
95
#### Optional Packages
Lukas Riedel's avatar
Lukas Riedel committed
96 97
| Software | Version/Branch | Comments |
| ---------| -------------- | -------- |
98
| [dune-testtools](https://gitlab.dune-project.org/quality/dune-testtools) | releases/2.6 | Handles system tests
Lukas Riedel's avatar
Lukas Riedel committed
99
| [doxygen](http://www.stack.nl/~dimitri/doxygen/) | 1.8.13 | Builds documentation
100 101
| [METIS](http://glaros.dtc.umn.edu/gkhome/views/metis) | 5 | For parallel runs
| [ParMETIS](http://glaros.dtc.umn.edu/gkhome/views/metis) | 4 | For parallel runs
102 103


Lukas Riedel's avatar
Lukas Riedel committed
104
#### Step-by-step Instructions
Lukas Riedel's avatar
Lukas Riedel committed
105
These instructions are suitable for a clean **Ubuntu** or **macOS** setup. The main difference between the two systems is the package manager. Debian-based systems have the APT manager already built in. On Mac, we recommend installing [Homebrew](http://brew.sh/). If you prefer to use [MacPorts](https://www.macports.org/), you need to check if some of the packages require different installation options than displayed here.
106

107 108 109
Manual installations on macOS require installing HDF5 from source. This can
be tricky, but the following instructions should work on a clean system.

110 111
If you installed [Anaconda](https://conda.io/docs/user-guide/install/download.html) on your machine, you don't need to install Python or Pip. Simply skip these packages when using the package managers for installing the software. However, notice the warnings when compiling DORiE below!

Lukas Riedel's avatar
Lukas Riedel committed
112
1. **macOS** users need to start by installing the Apple Command Line Tools by executing
113 114

        xcode-select --install
115 116
    
    Make sure you have no pending software updates for your respective version of macOS!
Dion Haefner's avatar
Dion Haefner committed
117

Lukas Riedel's avatar
Lukas Riedel committed
118
2. Install third party packages:
Lukas Riedel's avatar
Lukas Riedel committed
119

120
    **Ubuntu:**
Lukas Riedel's avatar
Lukas Riedel committed
121 122

        apt update
Lukas Riedel's avatar
Lukas Riedel committed
123
        apt install cmake doxygen gcc g++ gfortran \
Dion Haefner's avatar
Dion Haefner committed
124
                    git libatlas-base-dev libfftw3-dev libfftw3-mpi-dev \
125 126
                    libfreetype6-dev libhdf5-mpi-dev libmuparser-dev \
                    libopenmpi-dev libpng-dev libsuperlu-dev libyaml-cpp-dev \
Lukas Riedel's avatar
Lukas Riedel committed
127
                    libxft-dev python3-dev python3-pip python3-vtk7
128

Lukas Riedel's avatar
Lukas Riedel committed
129
    **macOS:**
130 131

        brew update
132
        brew install cmake doxygen fftw gcc libpng open-mpi muparser \
Lukas Riedel's avatar
Lukas Riedel committed
133
                     pkg-config python3 superlu yaml-cpp
134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158

2. **macOS only:** Install HDF5 with MPI support from source.

    1. Download an archive of the
    [HDF5 source code](https://www.hdfgroup.org/downloads/hdf5/source-code/),
    and extract it.

    2. Enter the extracted folder. In there, create a `build` directory, and
    enter it:

            mkdir build && cd build

    3. Configure your build. If you followed the instructions above, the
    OpenMPI C compiler is reachable via the command `mpicc`. If not, you have
    to specify a full path to it. Use the option `prefix` to specify where
    you want the package to be installed. This should *not* be a
    system-reserved path like `/usr/local`, and *not* be located in a
    sub-directory of the source code. Execute the configuration script:

            ./../configure CC=mpicc --prefix=<hdf5-path> --enable-parallel

    4. Build and install the library:

            make && make install

159

Lukas Riedel's avatar
Lukas Riedel committed
160
3. The parallel linear solver of DORiE can make use of the ParMETIS package. If you want to run DORiE in parallel on multiple processes, additionally install METIS and ParMETIS:
Lukas Riedel's avatar
Lukas Riedel committed
161 162 163

    **Ubuntu:**

164
        apt install libmetis-dev libparmetis-dev
Lukas Riedel's avatar
Lukas Riedel committed
165

Lukas Riedel's avatar
Lukas Riedel committed
166
    **macOS:** _Support is dropped because ParMETIS is currently unavailable from Homebrew._
Lukas Riedel's avatar
Lukas Riedel committed
167 168 169

    **Parallel runs without these two packages are possible but not supported!**

170 171 172 173 174 175 176 177 178 179 180
4. Clone the DUNE modules into a suitable folder on your machine.
    Use `git checkout` to switch to the correct branches (see above).

5. Clone DORiE into the same folder.

    DORiE includes
    [Git Submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules),
    which requires you to add the `--recurse-submodules` option to the
    `git clone` command. Switch to the desired branch or tag.

6. Enter the parent folder, and call
181 182

        CMAKE_FLAGS="-DDUNE_PYTHON_VIRTUALENV_SETUP=True -DDUNE_PYTHON_ALLOW_GET_PIP=True" ./dune-common/bin/dunecontrol all
183

184
    to build all DUNE modules. Additionally, you can add `MAKE_FLAGS="-j X"` before the call to `dunecontrol` to compile on `X` processes in parallel.
185

186
    If you installed software into paths not appended to your `PATH` variable, you will have to add `CMAKE_FLAGS` to the call to make sure that CMake finds all packages. Alternatively, you can add a custom options file. See the [DUNE Installation Instructions](https://dune-project.org/doc/installation/) for details. CMake will throw an error if required packages are not found.
187

188 189 190 191 192 193 194
    If you installed HDF5 from source (all macOS users) or use Anaconda,
    specify the path to your HDF5 installation by using the `HDF5_ROOT`
    variable. On Ubuntu, add the path to the APT package,

        -DHDF5_ROOT=/usr/

    and on macOS, add
195

196
        -DHDF5_ROOT=<hdf5-path>
197

198 199
    to the `CMAKE_FLAGS` in the above command, replacing `<hdf5-path>` with the
    path chosen as installation prefix when configuring HDF5.
200

201

Lukas Riedel's avatar
Lukas Riedel committed
202
### Recommended Third-Party Software
203 204

The following software packages are cross-platform, so you should be able to find a release that fits your operating system:
Dion Haefner's avatar
Dion Haefner committed
205 206 207 208

* [Gmsh](http://gmsh.info/): An open-source CAD that can be used to create the `.msh` files used by DORiE to define triangular meshes.
* [ParaView](http://www.paraview.org/): A powerful post-processing tool for VTK files. Offers both visualization and data analysis tools.

Lukas Riedel's avatar
Lukas Riedel committed
209
## Usage
Dion Haefner's avatar
Dion Haefner committed
210

Lukas Riedel's avatar
Lukas Riedel committed
211
### Documentation
Dion Haefner's avatar
Dion Haefner committed
212

213
The documentation of the latest release branch can be found [online](https://dorie-doc.netlify.com/).
Dion Haefner's avatar
Dion Haefner committed
214

215
The documentation can be built after DORiE has been properly configured (i.e., by calling `dunecontrol`). Note that you might have to re-configure DORiE once after installing it, because some dependencies are installed at configure time (e.g. by `dunecontrol --only=dorie configure`). To build the documentation, just run
Dion Haefner's avatar
Dion Haefner committed
216

Dion Haefner's avatar
Dion Haefner committed
217
    dunecontrol --only=dorie make doc
Dion Haefner's avatar
Dion Haefner committed
218

Lukas Riedel's avatar
Lukas Riedel committed
219 220 221 222
or navigate to the `dorie/build-cmake` directory and call

    make doc

Lukas Riedel's avatar
Lukas Riedel committed
223 224
You will then find the index page of the documentation at
`dorie/build-cmake/doc/html/index.html`.
Dion Haefner's avatar
Dion Haefner committed
225

Lukas Riedel's avatar
Lukas Riedel committed
226
### Run, DORiE, Run!
Lukas Riedel's avatar
Lukas Riedel committed
227 228
DORiE provides a command line interface (CLI) for all its functions.
The required Python modules and all their dependencies are readily installed
229
into a virtual environment (`venv`), which has to be activated within a
Lukas Riedel's avatar
Lukas Riedel committed
230 231
shell session. You can do so by activating it in your current session
(Manual Installation only) or by running the Docker application.
232

Lukas Riedel's avatar
Lukas Riedel committed
233
#### Run the `venv` using the Docker application
Lukas Riedel's avatar
Lukas Riedel committed
234 235
If you did not install DORiE locally, you can use the Docker
application to boot up the virtual environment in a mounted directory of choice.
236

Lukas Riedel's avatar
Lukas Riedel committed
237
Start up the Docker application by calling
Lukas Riedel's avatar
Lukas Riedel committed
238

Lukas Riedel's avatar
Lukas Riedel committed
239
    docker run -it -v <dir>:/mnt <img>
240

Lukas Riedel's avatar
Lukas Riedel committed
241 242 243 244 245
where you replace `<dir>` with a local directory for storing input and output
data, and `<img>` with `dorie/dorie[:<tag>]`.
If you built a local Docker image, replace `<img>` appropriately.
You can also insert `$PWD` as `<dir>` if you want to mount your
current working directory.
246

Lukas Riedel's avatar
Lukas Riedel committed
247 248 249 250
The command boots up a (`bash`) shell inside a Docker container and mounts
the directory `<dir>` and all its subdirectories into the directory `/mnt`
inside the container. Your shell session starts in this directory with the
virtual environment activated.
251

Lukas Riedel's avatar
Lukas Riedel committed
252 253
Notice, that you can only use **local file paths** in all configuration settings
due to the directory mount.
254

Lukas Riedel's avatar
Lukas Riedel committed
255
#### Activate the `venv` locally
Lukas Riedel's avatar
Lukas Riedel committed
256
To activate the virtual environment within your current shell session, execute
257

Lukas Riedel's avatar
Lukas Riedel committed
258
    source <path/to/>dorie/build-cmake/activate
259

Lukas Riedel's avatar
Lukas Riedel committed
260
where you replace `<path/to/>` with the path to the appropriate directory.
261

Lukas Riedel's avatar
Lukas Riedel committed
262 263 264
Your shell will now display the prefix `(dune-env)` to indicate that it is
configured appropriately. You can exit the environent at any time by simply
executing
Dion Haefner's avatar
Dion Haefner committed
265

Lukas Riedel's avatar
Lukas Riedel committed
266
    deactivate
Dion Haefner's avatar
Dion Haefner committed
267

Lukas Riedel's avatar
Lukas Riedel committed
268 269
Notice that any virtual environment only applies to, and lasts for your current
terminal session!
Dion Haefner's avatar
Dion Haefner committed
270

Lukas Riedel's avatar
Lukas Riedel committed
271 272
_With the virtual environment activated,_ you can now navigate to any directory
that you would like to contain your simulation input and/or output data.
Dion Haefner's avatar
Dion Haefner committed
273

Lukas Riedel's avatar
Lukas Riedel committed
274
#### Execute the application
Lukas Riedel's avatar
Lukas Riedel committed
275
Any command to the DORiE application has the signature
276

Lukas Riedel's avatar
Lukas Riedel committed
277
    dorie <cmd> [<opts>] [<args>]
278

Lukas Riedel's avatar
Lukas Riedel committed
279 280
Using the `-h` or `--help` option, you can find all available commands and
further help.
Lukas Riedel's avatar
Lukas Riedel committed
281

Lukas Riedel's avatar
Lukas Riedel committed
282 283 284 285 286
To start your first simulation run, create a new directory and enter it.
Place some exemplary configuration and boundary condition data files in there
by calling

    dorie create
Lukas Riedel's avatar
Lukas Riedel committed
287

Lukas Riedel's avatar
Lukas Riedel committed
288
DORiE implements a lightweight wrapper around the `dune-randomfield`
Lukas Riedel's avatar
Lukas Riedel committed
289 290 291
generator. You can use it to easily create a heterogeneous soil architecture.
This step is optional. Tweak the parameters of `parfield.ini` to your liking
and then call
Lukas Riedel's avatar
Lukas Riedel committed
292

Lukas Riedel's avatar
Lukas Riedel committed
293 294
    dorie pfg parfield.ini

Lukas Riedel's avatar
Lukas Riedel committed
295 296 297 298 299 300 301
The DORiE main routine is executed with the `run` command.
Tweak the parameters of `config.ini` to your liking. You will need to
reference several additional input files for soil parameters, boundary
conditions, GMSH grid files (optional), and grid mappings (optional).
Refer to the documentation for further information.

Once prepared, call
Lukas Riedel's avatar
Lukas Riedel committed
302 303 304

    dorie run config.ini

Lukas Riedel's avatar
Lukas Riedel committed
305
to execute the solver.
Lukas Riedel's avatar
Lukas Riedel committed
306

Lukas Riedel's avatar
Lukas Riedel committed
307
### Troubleshooting
Dion Haefner's avatar
Dion Haefner committed
308 309
CMake heavily caches the results of its configuration process. In case you encounter errors or strange behavior, especially after an update, you should delete the DORiE build folder (called `build-cmake` by default) and re-install DORiE using `dunecontrol`.

310
If the problem persists, take a look at the [List of Known Bugs](https://ts-gitlab.iup.uni-heidelberg.de/dorie/dorie/issues), or create an issue yourself. For problems related to the installation, refer to the sections below.
Dion Haefner's avatar
Dion Haefner committed
311

Lukas Riedel's avatar
Lukas Riedel committed
312
#### Debugging
Dion Haefner's avatar
Dion Haefner committed
313
DORiE can be built with debugging flags via CMake. To do so, run
314

Dion Haefner's avatar
Dion Haefner committed
315
    CMAKE_FLAGS="-DCMAKE_BUILD_TYPE=Debug" dunecontrol --only=dorie all
316

Dion Haefner's avatar
Dion Haefner committed
317
After building, a debugger can hook into the executables.
318

Dion Haefner's avatar
Dion Haefner committed
319
**Note:** If no `CMAKE_BUILD_TYPE` is specified during re-configuration, the last configuration build type is used. If no CMake files exist, it defaults to `Release`. You will find the actual value displayed in the final output of CMake.
320

Dion Haefner's avatar
Dion Haefner committed
321
To re-create a release build, configure DORiE with the release build type by executing
322

Dion Haefner's avatar
Dion Haefner committed
323
    CMAKE_FLAGS="-DCMAKE_BUILD_TYPE=Release" dunecontrol --only=dorie all
324

Lukas Riedel's avatar
Lukas Riedel committed
325
#### Running System Tests
Lukas Riedel's avatar
Lukas Riedel committed
326 327 328 329 330
DORiE includes a testing system for comparing its results the ones of ODE solvers
or former versions of itself. This ensures that DORiE is running correctly and
producing the expected results. We distinguish _unit tests_ for testing certain
features of the code, and _system tests_ for verifying the results of the
final application. As system tests require executing the DUNE solvers,
331 332
it is recommended to build them in a `Release` environment. Additionaly, there
is a set of tests for the Python module.
Lukas Riedel's avatar
Lukas Riedel committed
333 334

| Test category | Build tests | Execute tests | Recommended build type |
335
| ------------- | ----------- | ------------- | ---------------------- |
336 337
| Unit tests | `make build_unit_tests` | `make unit_tests` | `Debug`
| System tests | `make build_system_tests` | `make system_tests` | `Release`
338
| Python tests | N/A | `make test_python` | any
Lukas Riedel's avatar
Lukas Riedel committed
339 340 341 342 343 344 345

The `make` commands are to be executed from within the `build-cmake` directory.

The unit tests automatically include compiler flags for code coverage reports.
After building them with `Debug` flags and executing them, you can
retrieve code coverage information using the
[`gcovr`](https://gcovr.com/index.html) utility.
Dion Haefner's avatar
Dion Haefner committed
346

Lukas Riedel's avatar
Lukas Riedel committed
347
#### Further Help
348
[Open an issue](https://ts-gitlab.iup.uni-heidelberg.de/dorie/dorie/issues/new), or write to the [DORiE developer mailing list](mailto:dorieteam@iup.uni-heidelberg.de).