Commit 840bf3a7 authored by Santiago Ospina's avatar Santiago Ospina

Merge branch 'master' into 98-implement-a-simulation-object-for-transport-transportsimulation

parents 73106b85 84c91bf1
......@@ -5,6 +5,7 @@ build-cmake/
python/dorie/wrapper/pf_from_file.py
python/dorie/wrapper/test_dorie.py
python/dorie/cli/cmds.py
test/maps/cell_ids.h5
# Ignore temporary and auto-generated files #
*~
......
......@@ -2,7 +2,7 @@ variables:
DUNE_CONTROL_PATH: /opt/dune:$CI_PROJECT_DIR
DUNECONTROL: ./dune-common/bin/dunecontrol
CPUS_MULTICORE: 4
CPUS_DIND: 3
CPUS_DIND: 2
DOCKER_LOGIN:
docker login -u $DOCKER_HUB_USER -p $DOCKER_HUB_PW
DUNE_ENV_IMAGE:
......@@ -10,6 +10,9 @@ variables:
CMAKE_FLAGS:
-DDUNE_PYTHON_VIRTUALENV_SETUP=True
-DDUNE_PYTHON_ALLOW_GET_PIP=True
MAKE_FLAGS:
-j $CPUS_MULTICORE
RUN_IN_DUNE_ENV: $CI_PROJECT_DIR/build-cmake/run-in-dune-env
image: $DUNE_ENV_IMAGE
......@@ -86,7 +89,7 @@ build:system-tests: &build-tests
script:
- CMAKE_FLAGS="$CMAKE_FLAGS"
$DUNECONTROL --only=dorie configure
- $DUNECONTROL --only=dorie make build_system_tests
- $DUNECONTROL --only=dorie make $MAKE_FLAGS build_system_tests
- $DUNECONTROL --only=dorie make doc
artifacts:
name: "$CI_JOB_NAME"
......@@ -100,7 +103,7 @@ build:unit-tests:
- CMAKE_FLAGS="$CMAKE_FLAGS
-DCMAKE_BUILD_TYPE=Debug"
$DUNECONTROL --only=dorie configure
- $DUNECONTROL --only=dorie make build_unit_tests
- $DUNECONTROL --only=dorie make $MAKE_FLAGS build_unit_tests
build:debug: &debug
stage: build
......@@ -111,7 +114,7 @@ build:debug: &debug
-DCMAKE_BUILD_TYPE=Debug
-DCMAKE_CXX_FLAGS_DEBUG='-Werror'"
$DUNECONTROL --only=dorie configure
- $DUNECONTROL --only=dorie make build_unit_tests
- $DUNECONTROL --only=dorie make $MAKE_FLAGS build_unit_tests
build:debug-clang:
<<: *debug
......@@ -124,56 +127,32 @@ build:debug-clang:
-DCMAKE_CXX_COMPILER=clang++
-DCMAKE_CXX_FLAGS_DEBUG='-Werror'"
$DUNECONTROL --only=dorie configure
- $DUNECONTROL --only=dorie make build_unit_tests
- $DUNECONTROL --only=dorie make $MAKE_FLAGS build_unit_tests
# --- Tests ---
test:st:exec_cmds: &test
test:system-tests: &test
stage: test
dependencies:
- build:system-tests
script:
- $DUNECONTROL --only=dorie configure
- $DUNECONTROL --only=dorie make test_dorie_exec
- $DUNECONTROL --only=dorie make system_tests
artifacts:
name: "$CI_JOB_NAME"
paths:
- $CI_PROJECT_DIR/build-cmake/test
expire_in: 1 day
test:st:ode:
<<: *test
script:
- $DUNECONTROL --only=dorie configure
- $DUNECONTROL --only=dorie make test_run_ode
test:st:parallel:
<<: *test
tags:
- multicore
script:
- $DUNECONTROL --only=dorie configure
- $DUNECONTROL --only=dorie make test_run_parallel
test:st:reference:
<<: *test
script:
- $DUNECONTROL --only=dorie configure
- $DUNECONTROL --only=dorie make test_run_ref
test:st:mass_conserve:
<<: *test
script:
- $DUNECONTROL --only=dorie configure
- $DUNECONTROL --only=dorie make test_mass_conservation
test:unit-tests:
<<: *test
dependencies:
- build:unit-tests
script:
# install coverage reporter
- $DUNECONTROL --only=dorie bexec
"./run-in-dune-env python3 -m pip install gcovr"
# perform the tests
- $DUNECONTROL --only=dorie configure
- $DUNECONTROL --only=dorie make unit_tests
# report coverage
......@@ -181,11 +160,13 @@ test:unit-tests:
- mkdir -p dune/dorie/test/coverage
- ./run-in-dune-env gcovr
--root ../
--exclude ../dune/dorie/test
--html --html-details
-o dune/dorie/test/coverage/coverage.html
# run again for coverage report in GitLab
- ./run-in-dune-env gcovr
--root ../
--exclude ../dune/dorie/test
coverage: '/^TOTAL.*\s+(\d+\%)$/'
artifacts:
name: "$CI_JOB_NAME"
......
......@@ -40,12 +40,7 @@
to call `FlowParameters::bind` with a grid entity to cache the appropriate
data. All classes and functions querying parameter data have been updated.
The old parameterization classes located in
[`param_base.hh`](dune/dorie/solver/param_base.hh),
[`param_factory.hh`](dune/dorie/solver/param_factory.hh), and
[`param_van_genuchten.hh`](dune/dorie/solver/param_van_genuchten.hh)
are still used for reading in data for the new storage scheme.
The respective objects are freed once a simulation commences.
The old parameterization classes have been removed.
* `RichardsSimulation` now uses one vector of coefficients instead of two, which
now is a `shared_ptr` instead a `unique_ptr`.
* Every `VTKAdapter` is now able to be used as `GridFunctionAdapter` of the
......@@ -75,13 +70,38 @@
* The `dorie` CLI has been moved into the DORiE Python package. It can now only
be invoked after activating the DUNE virtual environment. The usage
instructions in docs and `README.md` have been updated accordingly.
* Soil architecture and parameter input has been completely revised.
DORiE now expects a YAML file containing parameterization data and separate
domain mapping data which can be supplied through the input GMSH file, or as
H5 dataset. See the documentation for more information. Several keys in the
main routine config file have been changed.
See [#86](https://ts-gitlab.iup.uni-heidelberg.de/dorie/dorie/issues/86).
* The Parameter Field Generator (PFG) module has been revised and does not
write parameterization data anymore.
It can be used to create mappings for heterogeneous media through the
binary converter, or continuous scaling fields (feature pending).
Several keys in the PFG config file have been changed.
See [#86](https://ts-gitlab.iup.uni-heidelberg.de/dorie/dorie/issues/86).
* Improved and refurbished `Dorie::H5File`.
The source file was renamed to `dune/dorie/solver/util_h5file.hh`. The
`read_dataset` function is now capable of directly opening dataset paths
that contain groups.
### Deprecated
* The configuration file key `[parameters.interpolation]` is deprecated due to
the new scheme for storing parameterization data. DORiE now only supports
nearest-neighbor interpolation of parameters and scaling factors.
### Removed
* The class `H5File::AttributeReader` was completely removed.
## 1.1.1 (2018-08-21)
......
......@@ -145,14 +145,6 @@ If you installed [Anaconda](https://conda.io/docs/user-guide/install/download.ht
HDF5_ROOT=/usr/local
to the `CMAKE_FLAGS` in the call to `dunecontrol` above.
5. After a successful build you can call the `dorie` wrapper located in `dorie/build-cmake/bin` from anywhere on your machine, but it is more handy to add this location to your search path. You can add the directory to your `PATH` variable by calling
PATH=<path/to/>dorie/build-cmake/bin:$PATH
However, this will only last for your current terminal session. Another option is to add this path permanently in your shell configuration file.
**Installing DUNE and DORiE via `make install` into your usual work environment is strongly discouraged!**
### Experimental Features
The local operator implementing Richards equation's discretization supports
......@@ -250,20 +242,24 @@ by calling
dorie create
DORiE encapsulates two main routines, the Parameter Field Generator (PFG) and
the main program routine. Each takes a single `.ini` configuration file as
argument.
Tweak the parameters of `parfield.ini` to your liking and then call
DORiE implements a lightweight wrapper around the `dune-randomfield`
generator. You can use it to easily create a heterogeneous soil architecture.
This step is optional. Tweak the parameters of `parfield.ini` to your liking
and then call
dorie pfg parfield.ini
to create a parameter field file. Do the same with the `config.ini`, while
ensuring that you use the parameter field file you just created. Then call
The DORiE main routine is executed with the `run` command.
Tweak the parameters of `config.ini` to your liking. You will need to
reference several additional input files for soil parameters, boundary
conditions, GMSH grid files (optional), and grid mappings (optional).
Refer to the documentation for further information.
Once prepared, call
dorie run config.ini
to execute the main program.
to execute the solver.
## Troubleshooting
CMake heavily caches the results of its configuration process. In case you encounter errors or strange behavior, especially after an update, you should delete the DORiE build folder (called `build-cmake` by default) and re-install DORiE using `dunecontrol`.
......
......@@ -10,6 +10,13 @@ add_custom_target(system_tests
COMMAND ctest --output-on-failure --exclude-regex ^ut.+$
)
# create the mapping datafile as preparation for all tests
add_custom_target(prepare_testing
COMMAND ${DUNE_PYTHON_VIRTUALENV_EXECUTABLE} ${PROJECT_SOURCE_DIR}/test/maps/create_param_maps.py ${PROJECT_SOURCE_DIR}/test/maps/cell_ids.h5
)
add_dependencies(system_tests prepare_testing)
add_dependencies(unit_tests prepare_testing)
#
# .. cmake_function:: add_coverage_links
#
......@@ -134,7 +141,7 @@ function(dorie_add_metaini_test)
# configure meta ini file or just copy.
get_filename_component(metaini-name ${SYSTEM_TEST_METAINI} NAME_WE)
get_filename_component(metaini-extension ${SYSTEM_TEST_METAINI} EXT)
if(metaini-extension EQUAL ".mini.in")
if(metaini-extension STREQUAL ".mini.in")
configure_file(${SYSTEM_TEST_METAINI} ${metaini-name}.mini)
set(SYSTEM_TEST_METAINI "${metaini-name}.mini")
else()
......
......@@ -32,215 +32,108 @@ adding an empty line, make text **bold** or ``monospaced``.
<dorie>
<category name="general" hidden="true">
<parameter name="generator">
<definition>
Specifies which parameter field generator module should be used.
</definition>
<values> image, csv, fft, hdf5 </values>
<suggestion> csv </suggestion>
<comment> One of: image, csv, fft, hdf5 </comment>
</parameter>
<parameter name="inputFile">
<definition>
Path to file holding the parameter distribution data (i.e., image, CSV file,
or HDF5 file). Has no effect when using FFT generator.
</definition>
<values> path </values>
</parameter>
<parameter name="outputFile">
<definition>
Output file name of the parameter field array.
Output file path of the parameter field array.
</definition>
<values> path </values>
<suggestion> field.h5 </suggestion>
</parameter>
<parameter name="parameterization">
<parameter name="dataset">
<definition>
Parameterization to be used when mapping field value to parameter.
Essentially sets which ``parameters.&lt;parameterization&gt;.&lt;x&gt;``
values should be used.
The dataset name inside the ``outputFile``.
</definition>
<values> vanGenuchten </values>
<suggestion> vanGenuchten </suggestion>
<values> path </values>
</parameter>
<parameter name="overwrite">
<definition>
Whether an existing parameter field array should be overwritten.
<parameter name="converter">
<definition> Identifier of the random field converter. The converter is
applied after the random field is created and modifies it.
</definition>
<values> bool </values>
<suggestion> false </suggestion>
<values> none, binary </values>
<suggestion> binary </suggestion>
<comment> none, binary </comment>
</parameter>
</category>
<category name="parameters.vanGenuchten.1" hidden="true">
<parameter name="theta_r">
<parameter name="tempDir">
<definition>
Residual water content of medium 1, used in the Mualem-van Genuchten
parameterization. Default values for sand, according to Soil Physics, p. 92.
Temporary output directory of the dune-randomfield module.
Note that this only accepts a folder (file names are fixed).
</definition>
<values> float </values>
<suggestion> 0.03 </suggestion>
</parameter>
<parameter name="theta_s">
<definition> Saturated water content. </definition>
<values> float </values>
<suggestion> 0.32 </suggestion>
</parameter>
<parameter name="alpha">
<definition> log10(&alpha;) in the van Genuchten parameterization. Units: [1/m] </definition>
<values> float </values>
<suggestion> -2.3 </suggestion>
</parameter>
<parameter name="tau">
<definition> &tau; in the Mualem parameterization of the hydraulic permeability. </definition>
<values> float </values>
<suggestion> -1.1 </suggestion>
</parameter>
<parameter name="n">
<definition> n in the van Genuchten parameterization. </definition>
<values> float </values>
<suggestion> 4.17 </suggestion>
</parameter>
<parameter name="k0">
<definition> Saturated hydraulic conductivity. Units: [m/s] </definition>
<values> float </values>
<suggestion> 2.2E-5 </suggestion>
<values> path </values>
<suggestion> /tmp/fft_generator/ </suggestion>
</parameter>
</category>
<category name="parameters.vanGenuchten.2" hidden="true">
<parameter name="theta_r">
<definition>
Residual water content of medium 2, used in the Mualem-van Genuchten
parameterization. Default values for silt, according to Soil Physics, p. 92.
</definition>
<values> float </values>
<suggestion> 0.01 </suggestion>
</parameter>
<parameter name="theta_s">
<definition> Saturated water content. </definition>
<values> float </values>
<suggestion> 0.41 </suggestion>
</parameter>
<parameter name="alpha">
<definition> log10(&alpha;) in the van Genuchten parameterization. Units: [1/m] </definition>
<values> float </values>
<suggestion> -0.7 </suggestion>
</parameter>
<parameter name="tau">
<definition> &tau; in the Mualem parameterization of the hydraulic permeability. </definition>
<values> float </values>
<suggestion> .0 </suggestion>
</parameter>
<parameter name="n">
<definition> n in the van Genuchten parameterization. </definition>
<values> float </values>
<suggestion> 1.3 </suggestion>
<category name="grid">
<parameter name="dimensions">
<definition> Physical dimensions of the field. </definition>
<suggestion> 2 </suggestion>
<values> 2, 3 </values>
</parameter>
<parameter name="k0">
<definition> Saturated hydraulic conductivity. Units: [m/s] </definition>
<values> float </values>
<suggestion> 1E-5 </suggestion>
<parameter name="extensions">
<definition> Physical extensions of the created random field.
This information is not used by DORiE but required for meaningful
correlation lengths (see ``grid.corrLength`` below).
</definition>
<values> int &times; int (&times; int) </values>
<suggestion> 1 1 </suggestion>
</parameter>
</category>
<category name="generator">
<parameter name="millerSimilarity" hidden="true">
<definition>
Whether to use Miller similarity or not. If set to true, only
the first specified parameter set is used. Also, make sure to set a proper
variance.
<parameter name="cells">
<definition> Resolution (in grid cells) of the created random field.
</definition>
<values> bool </values>
<suggestion> false </suggestion>
<values> int &times; int (&times; int) </values>
<suggestion> 100 100 </suggestion>
</parameter>
</category>
<parameter name="dimensions">
<definition>
Spatial dimension of the created field. 3-dimensional fields are only supported
by the fft and hdf5 generators.
<category name="stochastic">
<parameter name="seed">
<definition> Seed of all random number generators involved.
The same seed will lead to the same random field, if the application
is used with the same settings
(compiler, number of parallel processes, ...)
</definition>
<values> 2, 3 </values>
<suggestion> 2 </suggestion>
<values> int </values>
<suggestion> 1 </suggestion>
</parameter>
<parameter name="variance">
<definition>
Variance of the resulting field, if ``millerSimilarity`` is used.
Variance of the resulting field.
</definition>
<values> float </values>
<suggestion> 0.2 </suggestion>
</parameter>
<parameter name="extensions">
<definition>
Physical extensions of the created field.
<parameter name="corrLength">
<definition> Physical correlation lengths for axiparallel anisotropy.
Specify one value for each dimension of the field.
</definition>
<values> float &times; float (&times; float) </values>
<suggestion> 1 1 </suggestion>
</parameter>
</category>
<category name="generator.fft">
<parameter name="outputPath">
<definition>
Path to the output folder of the generated field. Note that this only accepts
a folder, not a file name (file names are fixed).
</definition>
<values> path </values>
<suggestion> /tmp/fft_generator/ </suggestion>
</parameter>
<parameter name="N">
<definition> Suggested size (in cells) of the created random field. </definition>
<values> int &times; int (&times; int) </values>
<suggestion> 1000 1000 </suggestion>
</parameter>
<parameter name="correlationLengths">
<definition> Correlation lengths in each dimension of the random field, in
meters. </definition>
<values> float &times; float (&times; float) </values>
<suggestion> .1 .05 </suggestion>
</parameter>
<parameter name="seed">
<definition> Seed of all random number generators involved. Ensures reproducibility
(the same seed will always lead to the same random field). </definition>
<values> int </values>
<suggestion> 123456789 </suggestion>
</parameter>
<parameter name="covariance">
<definition> Defines which variogram function should be used to create the
random field. </definition>
<values> exponential, gaussian, spherical </values>
<values> exponential, gaussian, spherical, whiteNoise </values>
<suggestion> gaussian </suggestion>
<comment> exponential, gaussian, spherical </comment>
<comment> exponential, gaussian, spherical, whiteNoise </comment>
</parameter>
</category>
<category name="generator.image" hidden="true">
</category>
<category name="generator.csv" hidden="true">
</category>
<category name="generator.h5" hidden="true">
<category name="converter.binary">
<parameter name="indices">
<definition> Binary values A and B to transform the field into.
The field will be set to A where its value is less equal 0, and B where
its value is greater 0.
</definition>
<values> int </values>
<suggestion> 0 1 </suggestion>
</parameter>
</category>
</dorie>
......@@ -135,6 +135,29 @@ adding an empty line, make text **bold** or ``monospaced``.
<values> int </values>
<suggestion> 0 </suggestion>
</parameter>
<parameter name="mappingFile">
<definition> The H5 file containing the mapping from cell to medium index.
Specify the dataset inside the file with the next key. Leave empty or
set to ``None`` for a global homogeneous medium. In this case,
``grid.globalIndex`` has to be specified.
</definition>
<values> path </values>
</parameter>
<parameter name="mappingFileDataset">
<definition> The H5 dataset inside ``grid.mappingFile`` containing the
mapping from cell to medium index.
</definition>
<values> path </values>
</parameter>
<parameter name="globalIndex">
<definition> The medium index to use for all grid cells, if
``mappingFile`` is unset or ``None``.
</definition>
<values> int </values>
</parameter>
</category>
<category name="boundary">
......@@ -248,33 +271,10 @@ adding an empty line, make text **bold** or ``monospaced``.
</category>
<category name="parameters">
<parameter name="arrayFile">
<definition> Path to an HDF5 array file containing all parameters used in the chosen
parameterization. Can be created using the ``dorie pfg`` command. </definition>
<values> path </values>
<comment> Create with 'dorie pfg' </comment>
</parameter>
<parameter name="scale">
<definition> Scaling factor of the parameter field. A value &gt; 1 zooms into
the field; a value &lt; 1 zooms out. This may of course change the statistical
properties of the field (e.g. correlation lengths).
<parameter name="file">
<definition> YAML file containing the parameter definitions.
</definition>
<values> float &times; float (&times; float) </values>
<suggestion> 1 1 </suggestion>
</parameter>
<parameter name="offset">
<definition> The parameter field is shifted by this value (in physical units) </definition>
<values> float &times; float (&times; float) </values>
<suggestion> 0 0 </suggestion>
</parameter>
<parameter name="interpolation">
<definition> DEPRECATED: Sets the interpolation behavior when querying parameter field values
at a certain grid cell. </definition>
<values> nearest </values>
<suggestion> nearest </suggestion>
<values> path </values>
</parameter>
</category>
......
......@@ -50,6 +50,7 @@ Manual
man-config-file
man-bcfile
man-cookbook
man-parameter-file
public-api
.. _c-reference:
......
***********************************
Parameter Field File Specifications
***********************************
Soil Parameters and Architecture
================================
The parameter field file is used to supply soil parameter information to the
simulation. It is supplied as
`H5 File <https://support.hdfgroup.org/HDF5/doc/H5.intro.html>`_.
Specifying the domain properties requires input of the soil architecture
(in relation to the grid) and of the soil parameters. The latter incorporate
the *subscale physics*, the representation of the soil hydraulic properties
below the REV scale. DORiE requires several input files for retrieving this
information, depending on the type of grid used for the computation.
The Parameter Field Generator module always generates files compliant to this
specification.
A YAML_ parameter file is always required. It specifies a set of soil
parameterizations, along with a medium index. This index is used to reference
the medium specified in the architecture files. There are two types of these:
If you insert an unstructured grid with GMSH, you have to incorporate the
medium information. If you create a rectangular grid, you need to add a mapping
dataset, specifying the medium index for each (initial) grid cell.
The structure of this file is as follows:
YAML Parameter File
-------------------
This file is used to specify the parameterization for each medium inside the
simulated domain. It follows a simple hierarchical syntax. The file path and
name must be specified via the ``parameters.file`` key of the
:doc:`config file <man-config-file>`.
* H5Group: `parameters`
The to