Commit d5b3df7d authored by Lukas Riedel's avatar Lukas Riedel 📝

Merge remote-tracking branch 'origin/update/dune-v2.5.1' into development/dorie-v1.0

parents 2b3163c9 0a6cab3f
image: dorie/dune-env:2.5
image: dorie/dune-env:2.5.1
variables:
DUNE_CONTROL_PATH: /opt/dune:$CI_PROJECT_DIR
......@@ -14,7 +14,7 @@ build:main:
tags:
- docker
script:
- MAKE_FLAGS="-j 2" ./dune-common/bin/dunecontrol --only=dorie all
- CMAKE_FLAGS="-DDUNE_PYTHON_VIRTUALENV_SETUP=True -DDUNE_PYTHON_ALLOW_GET_PIP=True" MAKE_FLAGS="-j 2" ./dune-common/bin/dunecontrol --only=dorie all
- export PATH=/opt/dune/dorie/build-cmake/bin:$PATH
- ARGS="--output-on-failure -j 2" ./dune-common/bin/dunecontrol --only=dorie make test
- ./dune-common/bin/dunecontrol --only=dorie make doc
......@@ -25,25 +25,13 @@ build:main:
- $CI_PROJECT_DIR/build-cmake/doc/html/
expire_in: 1 week
build:python3:
tags:
- docker
script:
- ./dune-common/bin/dunecontrol --only=dune-python exec "rm -rf build-cmake"
- ./dune-common/bin/dunecontrol --only=dune-testtools exec "rm -rf build-cmake"
- ./dune-common/bin/dunecontrol --only=dorie exec "rm -rf build-cmake"
- CMAKE_FLAGS="-DDUNE_FORCE_PYTHON3=True" MAKE_FLAGS="-j 2" ./dune-common/bin/dunecontrol --module=dorie all
- export PATH=/opt/dune/dorie/build-cmake/bin:$PATH
- ARGS="--output-on-failure -j 2" ./dune-common/bin/dunecontrol --only=dorie make test
stage: build
build:update_dune:
tags:
- docker
script:
- ./dune-common/bin/dunecontrol update || true
- ./dune-common/bin/dunecontrol exec "rm -rf build-cmake"
- MAKE_FLAGS="-j 2" ./dune-common/bin/dunecontrol all
- CMAKE_FLAGS="-DDUNE_PYTHON_VIRTUALENV_SETUP=True -DDUNE_PYTHON_ALLOW_GET_PIP=True" MAKE_FLAGS="-j 2" ./dune-common/bin/dunecontrol all
- export PATH=/opt/dune/dorie/build-cmake/bin:$PATH
- ARGS="--output-on-failure -j 2" ./dune-common/bin/dunecontrol --only=dorie make test
stage: build
......@@ -52,7 +40,7 @@ build:debug:
tags:
- docker
script:
- CMAKE_FLAGS="-DCMAKE_BUILD_TYPE=Debug" ./dune-common/bin/dunecontrol --only=dorie all
- CMAKE_FLAGS="-DCMAKE_BUILD_TYPE=Debug -DDUNE_PYTHON_VIRTUALENV_SETUP=True -DDUNE_PYTHON_ALLOW_GET_PIP=True" ./dune-common/bin/dunecontrol --only=dorie all
stage: build
deploy:docs:
......
......@@ -34,18 +34,13 @@ In the section 'Usage' you will find a list of possible commands. Note that inpu
## Docker Installation - Interactive Setup
This setup is intended for advanced users. You will gain access to the DORiE module outside the container, and be able to make changes to the source code.
We have prepared a [DORiE DUNE Environment Image on Dockerhub](https://hub.docker.com/r/dorie/dune-env/), which is a modified image of the Ubuntu OS that has all dependencies readily installed. The current version is 2.5 (referencing the DUNE module version 2.5).
Run a new container from this image by calling
We have prepared a [DORiE DUNE Environment Image on Dockerhub](https://hub.docker.com/r/dorie/dune-env/), which is a modified image of the Ubuntu OS that has all dependencies readily installed. The current version is 2.5 (referencing the DUNE module version 2.5). We will run a new container from this image and mount some local directory `<hostdir>/dorie` into it by calling
docker run -i -t dorie/dune-env:2.5 /bin/bash
docker run -v <hostdir>/dorie:/opt/dune/dorie -i -t dorie/dune-env:2.5 /bin/bash
Docker will automatically download the image if necessary. Use the `-v` option of the `run` command to mount a local directory into the container:
Docker will automatically download the image if necessary. Now you can access the content of the (still empty) folder of the virtual machine from your local `<hostdir>/dorie` directory. *Mounting directories is not possible after your container has been started!*
-v <hostdir/dorie>:/opt/dune/dorie
This way, you can access the content of the (still empty) folder of the virtual machine from your local `<hostdir>/dorie` directory. *Mounting directories is not possible after your container has been started!*
Next, use `git clone` to download the DORiE repository into the shared folder. Enter the container, and execute
Next, use `git clone` to download the DORiE repository into the shared (mounted) folder. Enter the container, and execute
dunecontrol --only=dorie all
......@@ -58,7 +53,7 @@ to build in parallel on `N` processes. Lastly, install the DORiE binaries by cal
dunecontrol --only=dorie make install
### Update DUNE
The DUNE modules in the repository might be outdated. You can update to the newest versions by calling
The DUNE modules (as well as DORiE itself) in the repository might be outdated. You can update to the newest versions by calling
dunecontrol update
......@@ -87,7 +82,7 @@ Depending on your system configuration, there will be more packages necessary to
| pip | 2.7 or 3.x |
| MPI | | Tested with OpenMPI
| SuperLU | 4.3 or 5.x |
| VirtualEnv | | Install via `pip` |
| VirtualEnv | | For Python 2 only. Install via `pip` |
| [dune-common](https://gitlab.dune-project.org/core/dune-common) | releases/2.5
| [dune-geometry](https://gitlab.dune-project.org/core/dune-geometry) | releases/2.5
| [dune-grid](https://gitlab.dune-project.org/core/dune-grid) | releases/2.5
......@@ -97,10 +92,18 @@ Depending on your system configuration, there will be more packages necessary to
| [dune-functions](https://gitlab.dune-project.org/staging/dune-functions) | releases/2.5
| [dune-typetree](https://gitlab.dune-project.org/staging/dune-typetree) | releases/2.5
| [dune-pdelab](https://gitlab.dune-project.org/pdelab/dune-pdelab) | releases/2.5
| [dune-python](https://gitlab.dune-project.org/quality/dune-python) | releases/2.5
| [dune-testtools](https://gitlab.dune-project.org/quality/dune-testtools) | releases/2.5
| [dune-testtools](https://gitlab.dune-project.org/quality/dune-testtools) | master
### Optional Packages
| Software | Version/Branch | Comments |
| ---------| -------------- | -------- |
| [doxygen](http://www.stack.nl/~dimitri/doxygen/) | | Builds documentation
| [Sphinx](http://www.sphinx-doc.org/en/stable/) | | Builds documentation
| [Breathe](https://github.com/michaeljones/breathe) | | Builds documentation
| [METIS](http://glaros.dtc.umn.edu/gkhome/views/metis) | 5 | For parallel runs
| [ParMETIS](http://glaros.dtc.umn.edu/gkhome/views/metis) | 4 | For parallel runs
If you also want to build the documentation, you will additionally need to install Doxygen, Sphinx and Breathe.
### Step-by-step Instructions
These instructions are suitable for a clean **Ubuntu** or **Mac OS X** setup. The main difference between the two systems is the package manager. Debian-based systems have the APT manager already built in. On Mac, we recommend installing [Homebrew](http://brew.sh/). If you prefer to use [MacPorts](https://www.macports.org/), you need to check if some of the packages require different installation options than displayed here.
......@@ -110,55 +113,57 @@ These instructions are suitable for a clean **Ubuntu** or **Mac OS X** setup. Th
**Ubuntu:**
apt update
apt install autoconf bison cmake flex gcc-5 g++-5 gfortran-5 \
apt install cmake doxygen gcc-5 g++-5 gfortran-5 \
git libatlas-base-dev libfftw3-dev libfftw3-mpi-dev \
libfreetype6-dev libhdf5-mpi-dev libopenmpi-dev libpng-dev \
libsuperlu-dev libxft-dev python-dev python-pip python-sphinx \
python-breathe doxygen
libsuperlu-dev libxft-dev python3-dev python3-pip
**Mac OS X:**
brew update
brew install automake cmake doxygen gcc libtool libpng open-mpi \
brew install cmake doxygen gcc libpng open-mpi \
pkg-config python3 superlu
brew install hdf5 --with-mpi
brew install fftw --with-mpi
2. Install the necessary Python packages using `pip`:
1. The parallel linear solver of DORiE can make use of the ParMETIS package. If you want to run DORiE in parallel on multiple processes, additionally install METIS and ParMETIS:
**Ubuntu:**
python -m pip install virtualenv
apt install libmetis-dev libparmetis-dev
**Mac OS X:**
sudo -H python3 -m pip install virtualenv sphinx breathe
brew install metis parmetis
**Parallel runs without these two packages are possible but not supported!**
2. Install the necessary Python packages using `pip`:
pip3 install --upgrade pip
pip3 install sphinx breathe
3. **Mac OS X** only:
The Apple Clang compiler shipped with CMake is not suitable for compiling DUNE and UG. Before proceeding, call
The Apple Clang compiler shipped with CMake is not suitable for compiling DORiE. Before proceeding, call
export CC=gcc-6
export CXX=g++-6
export CC=gcc-7
export CXX=g++-7
to enfore the Homebrew'd GCC compiler. Note that this variable export only lasts for your current terminal session. Always make sure that the configuration tool actually finds GCC instead of the Apple Clang.
4. Clone the DUNE modules and DORiE into a suitable folder on your machine. Make sure that you check out the correct branches (see above). Enter the parent folder, and call
**Ubuntu:**
MAKE_FLAGS="-j <N>" ./dune-common/bin/dunecontrol all
**Mac OS X:**
CMAKE_FLAGS="-DDUNE_FORCE_PYTHON3=True" MAKE_FLAGS="-j <N>" ./dune-common/bin/dunecontrol all
CMAKE_FLAGS="-DDUNE_PYTHON_VIRTUALENV_SETUP=True -DDUNE_PYTHON_ALLOW_GET_PIP=True" MAKE_FLAGS="-j <N>" ./dune-common/bin/dunecontrol all
to build all DUNE modules in parallel on `N` processes. If you installed software into paths not appended to your `PATH` variable, you will have to add `CMAKE_FLAGS` to the call to make sure that CMake finds all packages. Alternatively, you can add a custom options file. See the [DUNE Installation Instructions](https://dune-project.org/doc/installation/) for details. CMake will throw an error if required packages are not found.
5. To install all DUNE packages into system locations (so you can call `dunecontrol` and `dorie` from anywhere), you can run
5. After a successful build you can call the `dorie` wrapper located in `dorie/build-cmake/bin` from anywhere on your machine, but it is more handy to add this location to your search path. You can add the directory to your `PATH` variable by calling
PATH=<path/to/>dorie/build-cmake/bin:$PATH
./dune-common/bin/dunecontrol make install
However, this will only last for your current terminal session. Another option is to add this path permanently in your shell configuration file.
which may require `sudo` rights on your machine. This action is **optional and not required to run DORiE**. If you choose not to install DORiE, make sure to append the location of `dunecontrol` (`dune-common/bin`) and `dorie` (`dorie/build-cmake/bin`) to your search path.
**Installing DUNE and DORiE via `make install` into your usual work environment is strongly discouraged!**
## Recommended Third-Party Software
......@@ -186,14 +191,20 @@ The documentation can be built after DORiE has been properly configured (i.e., b
dunecontrol --only=dorie make doc
or navigate to the `dorie/build-cmake` directory and call
make doc
The documentation files can now be found in the subfolder `dorie/build-cmake/doc`.
## Run, DORiE, Run!
During `make install`, DORiE installs a wrapper to access the functions and executables of DORiE from any location on your device. After a successful installation, you can call the wrapper by simply executing
If you made the DORiE wrapper directory available through your `PATH` variable, you can call it by simply executing
dorie <command>
If you did not run `dunecontrol make install`, you can find the wrapper in the `dorie/build-cmake/bin` folder of your installation.
You can always find the wrapper in the `dorie/build-cmake/bin` folder of your installation and call it directly with
<path/to/>dorie/build-cmake/bin/dorie <command>
To start your first simulation run, create a new directory and enter it. Place some exemplary configuration and boundary condition data files in there by calling
......
......@@ -24,7 +24,7 @@ MPIEXEC_POSTFLAGS = "@MPIEXEC_POSTFLAGS@"
# set some paths
DORIE_EXECUTABLE = os.path.join(DORIEDIR, "dune/dorie/dorie")
PARAMETERDIR = os.path.join(DORIEDIR, "doc/default_files")
DORIE_PYTHON = os.path.join(DORIEDIR, "dune-env")
DORIE_PYTHON = os.path.join(DORIEDIR, "run-in-dune-env")
def MPIRUN(nproc,exe,*args,**kwargs):
mpi_flags = kwargs.get("mpi_flags") or []
return [k for k in [MPIEXEC,MPIEXEC_NUMPROC_FLAG,str(nproc)] \
......
......@@ -11,11 +11,6 @@ FIND_PACKAGE (FFTW REQUIRED)
FIND_PACKAGE (SuperLU REQUIRED)
FIND_PACKAGE (MPI REQUIRED)
if(NOT dune-uggrid_FOUND)
MESSAGE(STATUS "Module dune-uggrid was not found. Falling back to external UG grid manager")
FIND_PACKAGE(UG REQUIRED)
endif()
FIND_PACKAGE (METIS)
FIND_PACKAGE (ParMETIS)
......
function(scrape_parameters SOURCE_DIR XML_FILE CSS OUTPUT RESULT_NAME)
message(STATUS "Running parameter scraper on sources ${SOURCE_DIR}")
if(DEPLOY_SPHINX_SOURCE_URL)
execute_process(COMMAND ${CMAKE_BINARY_DIR}/dune-env scrape_folder.py --source ${SOURCE_DIR}
execute_process(COMMAND ${CMAKE_BINARY_DIR}/run-in-dune-env scrape_folder.py --source ${SOURCE_DIR}
--xml ${XML_FILE} --out ${OUTPUT} --css ${CSS}
--source_url ${DEPLOY_SPHINX_SOURCE_URL}
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
RESULT_VARIABLE RETURN_CODE)
else()
execute_process(COMMAND ${CMAKE_BINARY_DIR}/dune-env scrape_folder.py --source ${SOURCE_DIR}
execute_process(COMMAND ${CMAKE_BINARY_DIR}/run-in-dune-env scrape_folder.py --source ${SOURCE_DIR}
--xml ${XML_FILE} --out ${OUTPUT} --css ${CSS}
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
RESULT_VARIABLE RETURN_CODE)
......@@ -18,11 +18,11 @@ endfunction()
file(COPY ${CMAKE_CURRENT_LIST_DIR} DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/..)
scrape_parameters(${PROJECT_SOURCE_DIR}/dune/dorie-rfg ${CMAKE_CURRENT_SOURCE_DIR}/field-parameters.xml ${CMAKE_CURRENT_SOURCE_DIR}/parameters.css "parfield.ini;field-parameters.html;field-parameters.rst" FIELDPARSCRAPE_RETURN)
if (${FIELDPARSCRAPE_RETURN} GREATER 0)
if (NOT ${FIELDPARSCRAPE_RETURN} EQUAL 0)
message(FATAL_ERROR "Parameter scraper failed. DORiE can not be built.")
endif()
scrape_parameters(${PROJECT_SOURCE_DIR}/dune/dorie ${CMAKE_CURRENT_SOURCE_DIR}/parameters.xml ${CMAKE_CURRENT_SOURCE_DIR}/parameters.css "config.ini;parameters.html;parameters.rst" PARSCRAPE_RETURN)
if (${PARSCRAPE_RETURN} GREATER 0)
if (NOT ${PARSCRAPE_RETURN} EQUAL 0)
message(FATAL_ERROR "Parameter scraper failed. DORiE can not be built.")
endif()
......@@ -7,5 +7,5 @@ Module: dorie
Version: 0.9
Maintainer: dorieteam@iup.uni-heidelberg.de
#depending on
Depends: dune-python dune-pdelab
Suggests: dune-testtools dune-uggrid
Depends: dune-pdelab dune-uggrid
Suggests: dune-testtools
\ No newline at end of file
function(install_python_package curdir packagelist)
message(STATUS "Installing python package dorie.${curdir}")
dune_install_python_package(PATH ${curdir} ADDITIONAL_PIP_PARAMS -r ${CMAKE_CURRENT_SOURCE_DIR}/${curdir}/requirements.txt -q)
set(${packagelist} ${${packagelist}} ${CMAKE_CURRENT_SOURCE_DIR}/${curdir} PARENT_SCOPE)
endfunction()
set(python_paths "")
install_python_package("utilities" python_paths)
install_python_package("parscraper" python_paths)
# install required python modules
dune_python_install_package(PATH utilities)
dune_python_install_package(PATH parscraper)
add_subdirectory(parfield/wrapper)
install_python_package("parfield" python_paths)
dune_python_install_package(PATH parfield)
# install testing module
if(dune-testtools_FOUND)
add_subdirectory(testtools/wrapper)
install_python_package("testtools" python_paths)
add_subdirectory(testtools/wrapper)
dune_python_install_package(PATH testtools)
endif()
set(DORIE_PYTHON_MODULES ${python_paths} CACHE PATH "Paths to the python modules")
......@@ -13,7 +13,12 @@ setup(name='dorie.parfield',
author='Dion Häfner <mail@dionhaefner.de>',
author_email='dorieteam@iup.uni-heidelberg.de',
url='',
install_requires=['dorie.utilities'],
packages=['dorie.parfield'],
install_requires=['dorie.utilities',
'h5py',
'Pillow',
'configparser'],
packages = ['dorie.parfield',
'dorie.parfield.converter',
'dorie.parfield.parameterization'],
scripts=randomfield_scripts()
)
cycler
matplotlib
numpy
pyparsing
python-dateutil
pytz
scipy
six
wheel
......@@ -13,6 +13,19 @@ setup(name='dorie.parscraper',
author='Dion Häfner <mail@dionhaefner.de>',
author_email='dorieteam@iup.uni-heidelberg.de',
url='',
packages=['dorie.parscraper'],
packages= ['dorie.parscraper',
'dorie.parscraper.wrapper',
'dorie.parscraper.writers',
'dorie.parscraper.readers'],
install_requires= ['cycler',
'matplotlib',
'numpy',
'pyparsing',
'python-dateutil',
'pytz',
'scipy',
'six',
'wheel'
],
scripts=dorie_parscraper_scripts()
)
cycler
matplotlib
numpy
pyparsing
python-dateutil
pytz
scipy
six
wheel
......@@ -14,6 +14,22 @@ setup(name='dorie.testtools',
author='Dion Häfner <mail@dionhaefner.de>',
author_email='dorieteam@iup.uni-heidelberg.de',
url='',
packages=['dorie.testtools'],
install_requires=['dune.testtools'],
packages=['dorie.testtools',
'dorie.testtools.dorie_create',
'dorie.testtools.dorie_pfg',
'dorie.testtools.dorie_plot',
'dorie.testtools.dorie_run',
'dorie.testtools.utilities',
'dorie.testtools.wrapper'
],
install_requires=['dune.testtools',
'cycler',
'matplotlib',
'numpy',
'pyparsing',
'python-dateutil',
'pytz',
'scipy',
'six',
'wheel'],
scripts=dorie_testtools_scripts())
cycler
matplotlib
numpy
pyparsing
python-dateutil
pytz
scipy
six
wheel
......@@ -14,6 +14,16 @@ setup(name='dorie.utilities',
author='Dion Häfner <mail@dionhaefner.de>',
author_email='dorieteam@iup.uni-heidelberg.de',
url='',
packages=['dorie.utilities'],
packages=['dorie.utilities',
'dorie.utilities.vtktools'],
install_requires=['cycler',
'matplotlib',
'numpy',
'pyparsing',
'python-dateutil',
'pytz',
'scipy',
'six',
'wheel'],
scripts=dorie_utilities_scripts()
)
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment