giosystem-ecflow¶
Giosystem-ecflow is a python package that manages the creation of ecflow servers and suites.
ecflow is a software that allows defining complex workflows, grouping them in suites, families and tasks and scheduling them for execution according to predefined rules.
The giosystem-ecflow library hooks up with the online giosystem-settings-django app and uses the information on the defined servers and suites to build the ecflow structure for the processing lines
Installation¶
Detailed installation instructions are available at the projects source code repository
Defining ecflow servers and suites¶
Ecflow servers and suites are defined in the giosystem-settings-django app. The settings are available at:
http://<operational_server>/giosystem/admin
There are three sections of the settings administration area that are used to configure suites and ecfflow servers:
- packages - Packages define some settings that are used inside suites:
- ecflow_servers
- suites
Custom suite variables¶
Giosystem generated suites have some specific variables that are automatically generated by the giosystemecflow code.
- COMPRESS_OUTPUTS
- COLLABORATORS_TO_EMAIL
- COPY_OUTPUTS_TO_ARCHIVE
- COPY_OUTPUTS_TO_IO_BUFFER
- ECF_MICRO
- EXECUTION_HOST
- EXTRA_HOSTS_TO_COPY_OUTPUTS
- OPS_DATABASE_API_KEY
- OPS_DATABASE_URL
- OPS_DATABASE_USER
- PYTHON_COMMAND
- PYTHON_PATH
- REMOVE_PACKAGE_WORKING_DIR
- SERVER_NAME
- SETTINGS_URL
- URLBASE
- USE_ARCHIVE_FOR_SEARCHING
- USE_IO_BUFFER_FOR_SEARCHING
- PAKAGE
Custom suite families and tasks¶
Giosystemecflow generated servers and suites automatically add some families and tasks to each server or suite.
- /server_maintenance
- /<suite_name>/maintenance
- /<suite_name>/archive
- /<suite_name>/clean
Starting suites¶
Execute the start_ecflow_servers.py script that is available after installation of the giosystemecflow package. This script will take care of starting the predefined servers and suites. When using the script you will have to provide the URL to the settings that you wish to use.
~/dev/venv/bin/python ~/dev/venv/bin/start_giosystem_servers.py <settings_url>
The script starts the server and loads the suites but it does not start them. Use the ecflowview GUI app to access the server and manually start the suite.
Editing suites¶
Edit the online settings, save them and then run the reload_suite task in the affected suite. It will be reloaded and fetch the new settings.
Monitoring suites¶
Use the ecflowview GUI app to monitor suites execution in realtime.