Module 6 – Workflow & Job Management Sample Clauses

Module 6 – Workflow & Job Management. The Workflow and Job management module purpose is to provide an easy interface for develop, orchestrate, execute, and register secure workflow submissions and pipeline executions. The Workflow and Job management module is composed mainly by the Ryax [17] engine tool that provides a complete data engineering platform enabling functionalities for end users to deploy, run, and scale their models on production infrastructures. The Workflow and Job management module could execute HEROES architecture internal processes and scientifical pipelines deployed by the end-users. The needs inside the HEROES platform for a Workflows and Job Management module are: • Automate, orchestrate, and execute scientific and engineering workflows customized and parametrized by the end-users. • Automate, orchestrate, and execute workflows for HEROES architecture processes. • Different levels of Workflows will be designed for the different stages of the pipeline. • Export and import workflows from a Workflow Repository to reuse previously developed code. • Monitor the workflows execution. • Trigger the Deployment Module to generate and start the remote infrastructure, conducting every step of the workflow through HEROES technology to the optimal hardware needed. • Obtain workflow placement recommendations based on the Decision Module to select the best environment to deploy the models. • Save time on engineering workflows, automating, and optimizing the methodology to manage resulting files. • Deploy containers with state-of-the-art tools to analyse data and to monitor the models and predictions.