Technology Category
Search and Optimization: Algorithm ConfigurationMachine Learning: Hardware-aware MLConstraint Satisfaction and Optimization: Distributed CSP/Optimization
Accelerating Cloud-Based Transcriptomics: Performance Analysis and Optimization of the STAR Aligner Workflow
🤖 AI Summary
To address the poor performance and high cost of the STAR aligner when processing hundred-terabyte-scale RNA-seq data in cloud environments, this paper proposes a cloud-native optimization framework. Methodologically, it introduces an early-termination strategy tailored for STAR—reducing total alignment time by 23% for the first time—and develops a performance-modeling–driven EC2 instance selection and Spot Instance dynamic scheduling mechanism, integrated with large-scale workflow orchestration. The key contribution lies in synergistically combining computation-characteristic–aware, workflow-level optimization with elastic cloud resource scheduling. Component-wise gains are quantified in medium-scale experiments, and end-to-end validation on hundred-terabyte datasets demonstrates a 31% reduction in total execution time and a 47% decrease in per-alignment cost versus the baseline, while maintaining high throughput and strong scalability.
Technology Category
Application Category
📝 Abstract
In this work, we explore the Transcriptomics Atlas pipeline adapted for cost-efficient and high-throughput computing in the cloud. We propose a scalable, cloud-native architecture designed for running a resource-intensive aligner -- STAR -- and processing tens or hundreds of terabytes of RNA-sequencing data. We implement multiple optimization techniques that give significant execution time and cost reduction. The impact of particular optimizations is measured in medium-scale experiments followed by a large-scale experiment that leverages all of them and validates the current design. Early stopping optimization allows a reduction in total alignment time by 23%. We analyze the scalability and efficiency of one of the most widely used sequence aligners. For the cloud environment, we identify one of the most suitable EC2 instance types and verify the applicability of spot instances usage.
Problem
Research questions and friction points this paper is trying to address.
Optimize STAR aligner workflow for cloud-based transcriptomics
Reduce execution time and cost for RNA-seq data processing
Identify suitable EC2 instance types for scalable alignment
Innovation
Methods, ideas, or system contributions that make the work stand out.
Cloud-native scalable architecture for STAR aligner
Multiple optimizations reduce time and cost
Optimal EC2 instance and spot instances usage
🔎 Similar Papers
2023-10-25arXiv.orgCitations: 2
