Embedded Files
DATAVIEW
DATAVIEW
A Scientific Workflow Management System for Cloud Computing and Deep Learning
Design, manage, and execute scientific workflows across local machines, cloud environments, and GPU-enabled computing infrastructures.
DATAVIEW is an open-source scientific workflow management system designed to support data-intensive scientific computing, workflow automation, and deep learning applications.
Explore on GitHub · Download DATAVIEW · Watch Tutorials
About DATAVIEW
About DATAVIEW
DATAVIEW provides a unified environment for developing, executing, and managing scientific workflows. It supports both programmatic workflow development through a Java API and visual workflow creation through a web-based interface called Webbench.
DATAVIEW integrates Dropbox for cloud-based data storage and Amazon EC2 for cloud-based computation, while also supporting local workflow execution.
Its research and development areas include workflow performance optimization, cloud computing cost optimization, and infrastructure-level support for GPU-enabled deep learning workflows.
Key Features
Key Features
Scientific Workflow Management
Scientific Workflow Management
Create and execute complex scientific and data-processing workflows using reusable tasks and configurable execution environments.
Visual Workflow Design
Visual Workflow Design
Use Webbench, a web-based visual programming interface, to create workflows through drag-and-drop task composition and dependency connections.
Java Programming API
Java Programming API
Develop customized workflow tasks and applications using DATAVIEW's Java libraries.
Cloud and Local Execution
Cloud and Local Execution
Execute workflows on local computers or Amazon EC2 cloud infrastructure, with Dropbox integration for workflow data management.
GPU-Enabled Deep Learning
GPU-Enabled Deep Learning
Develop and execute deep learning workflows using supported NVIDIA GPU environments, including local GPUs, NVIDIA Jetson Xavier and Nano modules, and heterogeneous GPU clusters.
Workflow Optimization
Workflow Optimization
Explore research-driven approaches to improving workflow execution performance, resource utilization, and computing costs.
Programming and Execution
Programming and Execution
DATAVIEW provides two primary workflow development interfaces.
Java API
Develop customized workflows using Java classes and the DATAVIEW libraries. The repository provides example applications demonstrating workflow creation and execution on local and cloud infrastructure.
Visual Programming with Webbench
Design workflows visually by connecting tasks in a browser-based interface. Webbench provides a graphical environment for workflow composition and execution.
DATAVIEW also includes support for neural network workflows (NNWorkflow), with execution options targeting different GPU infrastructures.
Getting Started
Getting Started
The DATAVIEW source code, example workflows, and setup instructions are available through the project's GitHub repository.
Source Code and Documentation: GitHub Repository
Downloads: GitHub Releases
Getting Started with Java API: Video Tutorial
Website Configuration and Deployment: Video Tutorial
For installation requirements, dependencies, and configuration instructions, please consult the repository README.
Tutorials and Learning Resources
Tutorials and Learning Resources
Explore introductory videos and demonstrations covering DATAVIEW's workflow development and execution capabilities.
Tutorial
Link
Introduction to DATAVIEW
Java API Setup and Local Workflow Execution
Creating a Relational Algebra Workflow
Deploying DATAVIEW as a Website
Deep Learning as a Workflow (DlaaW)
Research and Development
Research and Development
DATAVIEW is also a research platform for investigating scientific workflow execution in heterogeneous computing environments.
Research areas described in the project include:
Performance and cost optimization for cloud-based scientific workflows.
GPU-enabled deep learning workflow management.
Workflow execution across heterogeneous computing infrastructures.
Security and privacy in scientific workflow management.
The project brings together workflow systems, cloud computing, and deep learning infrastructure to support research into efficient data-intensive computing.
Open Source and Community
Open Source and Community
DATAVIEW is publicly available on GitHub. Researchers, developers, and students can explore the source code, review existing issues, and contribute improvements.
Repository: shiyonglu/DATAVIEW
Latest listed release: DATAVIEW 3.1
Primary programming language: Java
Additional technologies: JavaScript, CSS, HTML
Project links:
Page updated
Google Sites
Report abuse