Commit e7b70443 authored by Santiago Ospina De Los Ríos's avatar Santiago Ospina De Los Ríos

Merge branch 'master' into 111-add-dg-local-operator-to-the-simulationtransport-class

parents 029efbdd b0290153
......@@ -10,7 +10,7 @@ variables:
BASE_IMAGE: dorie/dune-env
# Use semantic versioning (not the version of DUNE) and bump according to
# to whether changes are backwards-compatible or not.
IMAGE_VERSION: "1.0"
IMAGE_VERSION: "1.1"
DUNE_ENV_IMAGE: ${BASE_IMAGE}:img-v${IMAGE_VERSION}
CMAKE_FLAGS:
......@@ -184,6 +184,13 @@ test:unit-tests:
- $CI_PROJECT_DIR/build-cmake/dune/dorie/test
expire_in: 1 day
test:python-tests:
<<: *test
dependencies: []
script:
- $DUNECONTROL --only=dorie configure
- $DUNECONTROL --only=dorie make test_python
# --- Deploy jobs ---
deploy:dockerhub-devel: &deploy
stage: deploy
......
......@@ -38,6 +38,8 @@
* Initial conditions expressed as analytic functions using
[muparser](http://beltoforion.de/article.php?a=muparser) !131
* Coupling between transient models for water flow and solute transport !96
* Initial conditions generated from H5 input data !130
* Generic Python VTK file reader !143
### Changed
* `Simulation` is renamed `RichardsSimulation` and moved to
......
......@@ -78,6 +78,7 @@ by CI tests.
| [yaml-cpp](https://github.com/jbeder/yaml-cpp) | >= 5.2.0 |
| [spdlog](https://github.com/gabime/spdlog) | 1.1.0 | Included as Git Submodule
| [muparser](http://beltoforion.de/article.php?a=muparser) | master |
| [VTK](https://vtk.org/) | >= 7.1.1 | For the Python module only
| [dune-common](https://gitlab.dune-project.org/core/dune-common) | releases/2.6
| [dune-geometry](https://gitlab.dune-project.org/core/dune-geometry) | releases/2.6
| [dune-grid](https://gitlab.dune-project.org/core/dune-grid) | releases/2.6
......@@ -95,8 +96,8 @@ by CI tests.
| ---------| -------------- | -------- |
| [dune-testtools](https://gitlab.dune-project.org/quality/dune-testtools) | releases/2.6 | Handles system tests
| [doxygen](http://www.stack.nl/~dimitri/doxygen/) | 1.8.13 | Builds documentation
| [METIS](http://glaros.dtc.umn.edu/gkhome/views/metis) | 5 | For parallel runs
| [ParMETIS](http://glaros.dtc.umn.edu/gkhome/views/metis) | 4 | For parallel runs
| [METIS](http://glaros.dtc.umn.edu/gkhome/views/metis) | 5 | For parallel runs
| [ParMETIS](http://glaros.dtc.umn.edu/gkhome/views/metis) | 4 | For parallel runs
#### Step-by-step Instructions
......@@ -119,7 +120,7 @@ If you installed [Anaconda](https://conda.io/docs/user-guide/install/download.ht
git libatlas-base-dev libfftw3-dev libfftw3-mpi-dev \
libfreetype6-dev libhdf5-mpi-dev libmuparser-dev \
libopenmpi-dev libpng-dev libsuperlu-dev libyaml-cpp-dev \
libxft-dev python3-dev python3-pip
libxft-dev python3-dev python3-pip python3-vtk7
**macOS:**
......@@ -330,12 +331,14 @@ or former versions of itself. This ensures that DORiE is running correctly and
producing the expected results. We distinguish _unit tests_ for testing certain
features of the code, and _system tests_ for verifying the results of the
final application. As system tests require executing the DUNE solvers,
it is recommended to build them in a `Release` environment.
it is recommended to build them in a `Release` environment. Additionaly, there
is a set of tests for the Python module.
| Test category | Build tests | Execute tests | Recommended build type |
| ------------- | ----------- | ------------- | ---------------------- |
| Unit tests | `make build_unit_tests` | `make unit_tests` | `Debug`
| System tests | `make build_system_tests` | `make system_tests` | `Release`
| Python tests | N/A | `make test_python` | any
The `make` commands are to be executed from within the `build-cmake` directory.
......
......@@ -92,6 +92,9 @@ exclude_patterns = []
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = None
# Include todo-directives into the output
todo_include_todos = True
# -- Options for HTML output ----------------------------------------------
......
Restart
=======
.. warning:: This document is work in progress (WIP).
Currently, DORiE does not have an option to restart simulations. However, it
is possible to make a pseudo-restart from a past simulation using
:doc:`initial conditions<initial>` from data files.
Pseudo-restart
--------------
Since this is not a proper restart, it is necessary to setup the simulation so
that it will start a new one from the data of the last simulation. This
includes to modify the starting time and the initial condition.
* ``time.start``:
Set the last time written by the restarted simulation.
* Initial condition:
It is not possible to use VTK files directly as initial condition, however
it is possible to transfer such data into HDF5 file which are accepted by
DORiE. A possible script to make such transfer is the following one:
.. todo:: create python script to write vtk data into h5 files.
* ``output.outputPath`` and ``output.fileName``:
Output path and filename must be changed from last simulation, otherwise,
data will be overwritten.
* Merge pvd files:
The old simulation and the pseudo-reset simulation will output different
pdv files because they are independent. However, merging pdv files so that
they are shown in paraview as one simulation is an easy task.
1. Open both pdv files and identify the data sets corresponding to each
time-step.
2. Copy the data sets into a new pdv file.
Output file from the first simulation:
.. code-block:: xml
<?xml version="1.0"?>
<VTKFile type="Collection" version="0.1" byte_order="LittleEndian">
<Collection>
<DataSet timestep="0" group="" part="0" name="" file="old_simulation/old_simulation-00000.vtu"/>
</Collection>
</VTKFile>
Output file from the second simulation:
.. code-block:: xml
<?xml version="1.0"?>
<VTKFile type="Collection" version="0.1" byte_order="LittleEndian">
<Collection>
<DataSet timestep="100" group="" part="0" name="" file="restarted/restarted_simulation-00000.vtu"/>
</Collection>
</VTKFile>
Merged file:
.. code-block:: xml
<?xml version="1.0"?>
<VTKFile type="Collection" version="0.1" byte_order="LittleEndian">
<Collection>
<DataSet timestep="0" group="" part="0" name="" file="old_simulation/old_simulation-00000.vtu"/>
<DataSet timestep="100" group="" part="0" name="" file="restarted/restarted_simulation-00000.vtu"/>
</Collection>
</VTKFile>
Notice that this qualifies as a pseudo-restart because the degrees of freedom
of the last simulation are not completely recovered. Indeed, they are
degenerated! Hence, strictly speaking, a pseudo-restart will lead to different
results compared with respect to a one-run simulation.
......@@ -113,7 +113,9 @@ adding an empty line, make text **bold** or ``monospaced``.
<category name="initial">
<parameter name="type">
<definition> There are currently two possible initial conditions to choose from: A HDF datafile (``data``), or analytic equations (``analytic``).</definition>
<definition> The data type representing the initial condition. Either an
HDF datafile (``data``), or analytic equations (``analytic``).
</definition>
<values> data, analytic </values>
<suggestion> analytic </suggestion>
<comment> Choose initial condition type: data, analytic </comment>
......@@ -134,18 +136,26 @@ adding an empty line, make text **bold** or ``monospaced``.
<suggestion> -h </suggestion>
</parameter>
<parameter name="datafile">
<definition> Path to the initial condition data file.
(``data`` type only)
<parameter name="file">
<definition> Path to the initial condition data file
(``data`` type only). DORiE currently only supports H5 files with
file extension ``.h5``.
</definition>
<values> path </values>
</parameter>
<parameter name="dataset">
<definition> Dataset to use as initial condition. (``data`` type only)
<definition> Dataset to use as initial condition (``data`` type only).
</definition>
<values> string </values>
</parameter>
<parameter name="interpolation">
<definition> Interpolation type used for the data (``data`` type only).
</definition>
<values> nearest </values>
<suggestion> nearest </suggestion>
</parameter>
</category>
<category name="time">
......
......@@ -120,7 +120,9 @@ adding an empty line, make text **bold** or ``monospaced``.
<category name="initial">
<parameter name="type">
<definition> There are currently two possible initial conditions to choose from: A HDF datafile (``data``), or analytic equations (``analytic``).</definition>
<definition> The data type representing the initial condition. Either an
HDF datafile (``data``), or analytic equations (``analytic``).
</definition>
<values> data, analytic </values>
<suggestion> analytic </suggestion>
<comment> Choose initial condition type: data, analytic </comment>
......@@ -141,18 +143,26 @@ adding an empty line, make text **bold** or ``monospaced``.
<suggestion> 0 </suggestion>
</parameter>
<parameter name="datafile">
<definition> Path to the initial condition data file.
(``data`` type only)
<parameter name="file">
<definition> Path to the initial condition data file
(``data`` type only). DORiE currently only supports H5 files with
file extension ``.h5``.
</definition>
<values> path </values>
</parameter>
<parameter name="dataset">
<definition> Dataset to use as initial condition. (``data`` type only)
<definition> Dataset to use as initial condition (``data`` type only).
</definition>
<values> string </values>
</parameter>
<parameter name="interpolation">
<definition> Interpolation type used for the data (``data`` type only).
</definition>
<values> nearest </values>
<suggestion> nearest </suggestion>
</parameter>
</category>
<category name="time">
......
......@@ -40,6 +40,7 @@ assignment and increment are based on the :doc:`public-api`.
:caption: Cook Book
cookbook/index
cookbook/restart
.. toctree::
:maxdepth: 1
......
......@@ -16,6 +16,12 @@ solution quantity.
Initial condition input is controlled entirely via the
:doc:`Configuration File <man-config-file>`.
.. note::
The initial condition is projected onto the actual solution function space.
Depending on grid shape and resolution, function space (order), and
interpolator (if applicable), the resulting solution may vary greatly from
the actual input data.
Input Types
-----------
This is an overview of all input types for initial conditions.
......@@ -57,6 +63,22 @@ They are controlled by the ``initial.type`` key and available for every model.
initial.equation = exp(-sqrt((x-0.5)^2+(y-0.5)^2)/(4.*0.002))/(4*pi*0.002)^(2/dim).
.. object:: Dataset
* ``type = data``
Load the initial condition from a data file ``initial.file`` by opening the
dataset ``initial.dataset`` inside this file. The data is interpreted as
function :math:`f(\mathbf{p})` of the physical position :math:`\mathbf{p}`
using one of the available :ref:`interpolators <man-interpolators>`, which
can be chosen using the setting ``initial.interpolation``. The input data
is automatically streched to match the grid extensions.
Supported file extensions:
* ``.h5``: H5_ data file. ``initial.dataset`` may be a file-internal path
to the target dataset.
.. _initial-transformation:
Transformation Types
......@@ -72,11 +94,11 @@ Initial condition tranformations for the Richards solver.
* ``quantity = matricHead``
The input data is directly interpreted as matric head,
The input data is directly interpreted as matric head
:math:`f = h_m [\mathrm{m}]`.
Transport
^^^^^^^^
^^^^^^^^^
Initial condition tranformations for the Transport solver.
.. object:: No Transformation
......@@ -86,5 +108,5 @@ Initial condition tranformations for the Transport solver.
The input data is directly interpreted as solute concentration,
:math:`f = c_w [\mathrm{kg}/\mathrm{m}^3]`.
.. _H5: https://www.h5py.org/
.. _muparser: http://beltoforion.de/article.php?a=muparser&p=features
......@@ -203,6 +203,8 @@ coordinates in the respective spatial dimensions.
scale_miller:
# ...
.. _man-interpolators:
Interpolators
^^^^^^^^^^^^^
......
......@@ -49,6 +49,17 @@ Package reference
:members:
:undoc-members:
Generic VTK Reader
------------------
This is a flexible VTK XML file reader which can evaluate all data sets written
by DORiE.
.. autoclass:: dorie.utilities.vtktools.vtkreader.VTKReader
:members:
.. automethod:: __init__
dorie.utilities.check_path
--------------------------
......
......@@ -6,30 +6,35 @@ ARG PROCNUM=1
ARG CC=gcc
ARG CXX=g++
RUN apt-get clean && apt-get update && apt-get install -y \
clang \
cmake \
doxygen \
gcc \
g++ \
gfortran \
git \
libatlas-base-dev \
libfftw3-dev \
libfftw3-mpi-dev \
libfreetype6-dev \
libhdf5-mpi-dev \
libmetis-dev \
libmuparser-dev \
libopenmpi-dev \
libpng-dev \
libparmetis-dev \
libsuperlu-dev \
libxft-dev \
libyaml-cpp-dev \
locales \
python3-dev \
python3-pip \
# disable any prompts while installing packages
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get clean \
&& apt-get update \
&& apt-get install -y \
clang \
cmake \
doxygen \
gcc \
g++ \
gfortran \
git \
libatlas-base-dev \
libfftw3-dev \
libfftw3-mpi-dev \
libfreetype6-dev \
libhdf5-mpi-dev \
libmetis-dev \
libmuparser-dev \
libopenmpi-dev \
libpng-dev \
libparmetis-dev \
libsuperlu-dev \
libxft-dev \
libyaml-cpp-dev \
locales \
python3-dev \
python3-pip \
python3-vtk7 \
&& apt-get clean
RUN rm -rf /var/lib/apt/lists/* \
......
......@@ -8,6 +8,7 @@
#include <dune/dorie/common/initial_condition/analytic.hh>
#include <dune/dorie/common/logging.hh>
#include <dune/dorie/common/initial_condition/h5.hh>
namespace Dune{
namespace Dorie{
......@@ -30,7 +31,7 @@ protected:
static auto create(
const Dune::ParameterTree& ini_file,
const typename T::GV& grid_view,
const std::shared_ptr<spdlog::logger> log=get_logger(log_base)
std::shared_ptr<spdlog::logger> log=get_logger(log_base)
)
{
std::unique_ptr<InitialCondition<T>> ic;
......@@ -40,14 +41,13 @@ protected:
if (ic_type == "data")
{
const auto ic_datafile = ini_file.get<std::string>("initial.datafile");
const auto ic_dataset = ini_file.get<std::string>("initial.dataset");
const auto ic_datafile = ini_file.get<std::string>("initial.file");
// determine data type
const auto ext_start = ic_datafile.find_last_of(".");
if (ext_start == std::string::npos) {
log->error("Cannot determine extension of initial condition "
"datafile: {}",
"data file: {}",
ic_datafile);
DUNE_THROW(IOError, "Initial condition datafile has no apparent "
"extension");
......@@ -55,8 +55,9 @@ protected:
const auto file_type = ic_datafile.substr(ext_start + 1);
if (file_type == "h5") {
DUNE_THROW(NotImplemented,
"Initial condition from HDF data files is not implemented yet!");
using ICH5 = InitialConditionH5<T>;
ic = std::make_unique<ICH5>(grid_view, ini_file, log);
} else if (file_type == "vtu") {
DUNE_THROW(NotImplemented,
"Initial condition from VTK data files is not implemented yet!");
......@@ -64,7 +65,7 @@ protected:
log->error("Unsupported initial condition datafile type: .{}",
file_type);
DUNE_THROW(NotImplemented,
"Unsupported initial condition datafile type");
"Unsupported initial condition data file type");
}
} // ic_type == "data"
else if (ic_type == "analytic")
......
// -*- tab-width: 4; indent-tabs-mode: nil -*-
#ifndef DUNE_DORIE_INITIAL_CONDITION_H5_HH
#define DUNE_DORIE_INITIAL_CONDITION_H5_HH
#include <string>
#include <dune/common/exceptions.hh>
#include <dune/common/parametertree.hh>
#include <dune/dorie/common/interpolator.hh>
#include <dune/dorie/common/initial_condition/initial_condition.hh>
namespace Dune{
namespace Dorie{
/*-------------------------------------------------------------------------*//**
* @brief Class for initial conditions from hdf5 data files.
*
* @ingroup InitialConditions
* @author Santiago Ospina
* @date 2019
*
* @tparam T BaseTraits
*/
template<class T>
class InitialConditionH5
: public InitialCondition<T>
{
using Base = InitialCondition<T>;
using RF = typename T::RF;
using Domain = typename T::Domain;
using GV = typename T::GV;
static constexpr int dim = T::dim;
public:
using Traits = typename Base::Traits;
InitialConditionH5( const GV& grid_view,
const Dune::ParameterTree& config,
std::shared_ptr<spdlog::logger> log)
: Base(grid_view),
_interpolator(InterpolatorFactory<T>::create(config.sub("initial"),
grid_view,
log))
{ }
/// Evaluate the initial condition at certain position
/** \param y
* \param x Position in local element coordinates
* \param e Element enitity
*/
void evaluate ( const typename Traits::ElementType& e,
const typename Traits::DomainType& x,
typename Traits::RangeType& y) const override
{
auto x_global = e.geometry().global(x);
y = _interpolator->evaluate(x_global);
}
private:
std::shared_ptr<Interpolator<RF, T>> _interpolator;
};
} // namespace Dorie
} // namespace Dune
#endif // DUNE_DORIE_INITIAL_CONDITION_H5_HH
......@@ -10,9 +10,13 @@
#include <dune/common/fvector.hh>
#include <dune/common/exceptions.hh>
#include <dune/common/parametertree.hh>
#include <dune/dorie/common/logging.hh>
#include <dune/dorie/common/utility.hh>
#include <dune/dorie/common/h5file.hh>
#include <yaml-cpp/yaml.h>
namespace Dune {
namespace Dorie {
......@@ -33,7 +37,7 @@ protected:
const std::vector<T> _data; //!< contiguously stored data of the array
const std::vector<size_t> _shape; //!< inverted shape of the original array
//! phsical extensions of the dataset
//! physical extensions of the dataset
const Domain _extensions;
//! origin offset of the dataset
const Domain _offset;
......@@ -175,6 +179,9 @@ template<typename Traits>
struct InterpolatorFactory
{
using RF = typename Traits::RF;
using Domain = typename Traits::Domain;
/// Create the interpolator.
/** Use perfect forwarding for the data.
* \param type The type of interpolator to use
......@@ -254,6 +261,135 @@ static auto create (
log);
}
/// Create an interpolator from a ini file config
/** \param config Dune config file tree
* \param grid_view The grid view to use for determining extensions
* \param log The logger for the created object
*/
template<typename GridView>
static auto create (
const Dune::ParameterTree& config,
const GridView& grid_view,
const std::shared_ptr<spdlog::logger> log=get_logger(log_base)
)
-> std::shared_ptr<Interpolator<RF, Traits>>
{
const auto filename = config.get<std::string>("file");
const auto dataset = config.get<std::string>("dataset");
const auto [data, shape] = read_h5_dataset(filename, dataset, log);
const auto interpolation = config.get<std::string>("interpolation");
return create(interpolation,
data,
shape,
grid_view,
log);
}
/// Create an interpolator from a YAML config node
/// Create an interpolator from a ini file config
/** \param cfg YAML config tree node
* \param node_name Name of the YAML config node for better error messages
* \param grid_view The grid view to use for determining extensions
* \param log The logger for the created object
*/
template<typename GridView>
static auto create (
const YAML::Node& cfg,
const std::string& node_name,
const GridView& grid_view,
const std::shared_ptr<spdlog::logger> log=get_logger(log_base)
)
-> std::shared_ptr<Interpolator<RF, Traits>>
{
log->trace("Creating interpolator from YAML cfg node: {}", node_name);
const auto dim = GridView::Grid::dimension;
// open the H5 file
const auto filename = cfg["file"].as<std::string>();
const auto dataset = cfg["dataset"].as<std::string>();
const auto [data, shape] = read_h5_dataset(filename, dataset, log);
// read extensions and offset
const auto cfg_ext = cfg["extensions"];
const auto cfg_off = cfg["offset"];
if (cfg_ext and cfg_off)
{
const auto ext = cfg_ext.as<std::vector<RF>>();
const auto off = cfg_off.as<std::vector<RF>>();
if (shape.size() != dim) {
log->error("Expected {}-dimensional dataset. File: {}, "
"Dataset: {}, Dimensions: {}",
dim, filename, dataset, shape.size());
DUNE_THROW(Dune::IOError, "Invalid dataset for interpolator");
}
if (ext.size() != dim) {
log->error("Expected {}-dimensional sequence as "
"'extensions' at scaling section node: {}",
dim, node_name);
DUNE_THROW(Dune::IOError, "Invalid interpolator input");
}
if (off.size() != dim) {
log->error("Expected {}-dimensional sequence as 'offset' "
"at scaling section node: {}",
dim, node_name);
DUNE_THROW(Dune::IOError, "Invalid interpolator input");
}
// copy into correct data structure
Domain extensions;
std::copy(begin(ext), end(ext), extensions.begin());
Domain offset;
std::copy(begin(off), end(off), offset.begin());
// call other factory function
return create(
cfg["interpolation"].template as<std::string>(),
data,
shape,
extensions,
offset,
log
);
}
// extensions or offset not given