BScParallelBamIOScheduleProject

Project Schedule

Tasks

1) Extract the compression and decompression into a BgzfCompressor and a BgzfDecompressor class

2) Create Initial (Sequential) =Stream<ParallelBgzf>

3) Parallelize Stream<ParallelBgzf> in a Fork/Join Fashion

4) Write Parallel BGZF/BAM-Handling Tools

4a) Optional: Write Parallel BAM Sorter

4b) Optional: Asynchronous Parallelism

Schedule

Week	Planned Activities	Status
5	optional Task
6	optional Task
8-10	writing process
1	Read up on parallelization theory and algorithmic strategies, setup build system, start with Task 1	completed
2	Finish Task 1 and Task 2, create a concept for Task 3	completed
3	Task 3	completed
4	Finish Task 3, Task 4	completed
7	performance analysis and comparison to other solutions	in progress

Work Log

Week 1 Activities

• Research Parallel Software Engineering (concepts and usage scenarios)
• setup seqan environment
• create ParallelBgzf Stream from Bgzf Stream
• isolate decompression from ParallelBgzf Stream into Decompressor

Week 2 Activities

Mo (Nov 4, 2013)

• Consultation Manuel
• cleanup Decompressor
• introduce exceptions for IO-error handling

Di (Nov 5, 2013)

• isolate Compressor

Mi (Nov 6, 2013)

• test isolated Compressor

Do (Nov 7, 2013)

• test Compressor with integration to streamFlush

Fr (Nov 8, 2013)

• test/debug Compressor with integration to streamFlush and streamWrite

Week 3 Activities

Week 3 TODO's

• extend compressor so that it splits long inputs into blocks
• finish testing Stream with compressor/decompressor
• refactor Stream so that the sequential compression/decompression can be parallelized in a join/fork fashion using new classes BgzfCompressionThread and BgzfDecompressionThread

Mo (Nov 11, 2013)

• Consultation Manuel
• extend compressor so that it splits long inputs into blocks
• continue testing Stream

Di (Nov 11, 2013)

• continue testing Stream

Week 4 Activities

Week 4 TODO's

• debug decompressor
• extend tests for decompression
• create parallel decompressor
• create apps that use parallel compressor/decompressor

Fr (Dec 12, 2013)

• Debug Decompressor/Compressor
• Extend Tests

Mo (Dec 22, 2013)

• debug decompressor

Di (Dec 22, 2013)

• debug decompressor
• refactor parallel compressor
• implement parallel decompressor

Mi (Dec 22, 2013)

• add tests to parallel decompressor
• create BGZF App
• create BAM<->SAM tools

Do (Dec 22, 2013)

• found more problems in decompressor streamReadBlock Bookkeeping
• more decompressor debugging

Mo (Jan 6, 2014) - Fr (Jan 10)

• more streamReadBlock Bookkeeping

Mo (Jan 13, 2014) - Fr (Jan 17, 2014)

• more streamReadBlock Bookkeeping

Mo (Jan 20, 2014) - Fr (Jan 24, 2014)

• more work on BGZF App
• Meeting
• completed Multithreaded Decompressor integration and all associated Tests
• fixed multithreaded compressor book-keeping bug
• work on sam2bam

Mo (Jan 27, 2014) - Tue (Feb 5, 2014)

• more sam2bam, bam2sam
• profiling
• found and fixed performance bug
• running analysis of multithreading on compression and decompression in bgzf app
• thesis outline

Comments