A Gentle Introduction to Scientific Workflow Languages and Systems

Scientific workflows are an approach to implement automated, scalable, portable, and reproducible data analyses and in-silico experiments with low development costs.

Development Costs

What sets this approach apart from other distributed computing paradigms is its focus on the composition of programs. As a bioinformatics example, the output of a program that aligns reads to a reference genome is often processed by another program that analyzes genomic variants. Each of the programs is assumed to be readily available and is treated as a black box.

Continue reading “A Gentle Introduction to Scientific Workflow Languages and Systems”

Randomized Scheduling Paper Accepted at WORKS

Last week, my paper on randomized task graph scheduling was accepted at the Workshop on Workflows in Support of Large-Scale Science collocated with the SC18 in Dallas.

My idea behind this paper was to improve on the extremely good performance of the HEFT [1] scheduler I had observed in various experiments. My attack on the problem was to allocate a larger time budget to allow exploring variations of HEFT’s usually already good schedules.

Continue reading “Randomized Scheduling Paper Accepted at WORKS”