{"id":81,"date":"2025-01-29T18:23:14","date_gmt":"2025-01-30T01:23:14","guid":{"rendered":"https:\/\/blogs.ubc.ca\/evobiodriresources\/?page_id=81"},"modified":"2025-01-29T18:45:32","modified_gmt":"2025-01-30T01:45:32","slug":"module-3","status":"publish","type":"page","link":"https:\/\/blogs.ubc.ca\/evobiodriresources\/module-3\/","title":{"rendered":"High-Throughput Sequencing Data Preprocessing"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">This tutorial provides a step-by-step guide for preprocessing high-throughput sequencing data using tools such as Trimmomatic, BWA-MEM2, Samtools, and Picard. The workflow includes quality trimming of sequencing reads, aligning them to the reference genome, sorting the resulting BAM files, and removing duplicate reads to ensure accurate variant calling.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Required Tools and Files:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Trimmomatic (for quality trimming)<\/li>\n\n\n\n<li>BWA-MEM2 (for read alignment)<\/li>\n\n\n\n<li>Samtools (for BAM file sorting and statistics)<\/li>\n\n\n\n<li>Picard (for duplicate removal)<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Required Files:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Reference Genome File: genome.fasta (FASTA format containing the reference genome sequences)<\/li>\n\n\n\n<li>Sequencing Data Files: Paired-end FASTQ files, such as S1_1.fq.gz and S1_2.fq.gz.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Step 1: Quality Trimming with Trimmomatic<\/strong><\/p>\n\n\n\n<p class=\"has-subtle-background-background-color has-background wp-block-paragraph\" style=\"font-size:15px\"><code>java -jar trimmomatic-0.36.jar PE S1_1.fq.gz S1_2.fq.gz --baseout S1_out.fq.gz LEADING:20 TRAILING:20 SLIDINGWINDOW:4:20 MINLEN:36 -threads 10<\/code><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">#PE: Paired-end sequencing data.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">#LEADING:20: Trim bases with quality below 20 from the start of the read.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">#TRAILING:20: Trim bases with quality below 20 from the end of the read.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">#SLIDINGWINDOW:4:20: Trim when the average quality within a sliding window of 4 bases falls below 20.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">#MINLEN:36: Discard reads shorter than 36 bases after trimming.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">#-threads 10: Use 10 threads to speed up the processing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Step 2: Build Genome Index<\/strong><\/p>\n\n\n\n<p class=\"has-subtle-background-background-color has-background wp-block-paragraph\" style=\"font-size:15px\"><code>samtools faidx genome.fasta<\/code><\/p>\n\n\n\n<p class=\"has-subtle-background-background-color has-background wp-block-paragraph\" style=\"font-size:15px\"><code>bwa-mem2 index genome.fasta<\/code><\/p>\n\n\n\n<p class=\"has-subtle-background-background-color has-background wp-block-paragraph\" style=\"font-size:15px\"><code>java -jar picard.jar CreateSequenceDictionary R=genome.fasta<\/code>java -jar picard.jar CreateSequenceDictionary R=genome.fasta<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Step 3: Align Reads to the Reference Genome<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Once the genome index is built, the next step is to align the sequencing reads to the reference genome using BWA-MEM2. After alignment, we will use Samtools to convert the resulting SAM file to BAM format and sort it.<\/p>\n\n\n\n<p class=\"has-subtle-background-background-color has-background wp-block-paragraph\" style=\"font-size:15px\"><code>bwa-mem2 mem -t 2 -R '@RG\\tID:S1\\tSM:S1\\tPL:illumina' genome.fasta S1_out_1P.fq.gz S1_out_2P.fq.gz &gt; S1.sam<\/code><\/p>\n\n\n\n<p class=\"has-subtle-background-background-color has-background wp-block-paragraph\" style=\"font-size:15px\"><code>samtools view -bS S1.sam &gt; S1.bam<\/code><\/p>\n\n\n\n<p class=\"has-subtle-background-background-color has-background wp-block-paragraph\" style=\"font-size:15px\"><code>samtools sort -@ 2 -m 1G -o S1.sorted.bam S1.bam<\/code><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Step 4: Remove Duplicates<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Duplicate reads may arise due to PCR amplification and should be removed to avoid bias in variant calling.<\/p>\n\n\n\n<p class=\"has-subtle-background-background-color has-background wp-block-paragraph\" style=\"font-size:15px\"><code>java -jar picard.jar MarkDuplicates I=S1.sorted.bam O=S1.sorted.markdup.bam CREATE_INDEX=true REMOVE_DUPLICATES=true M=S1.marked_dup_metrics.txt<\/code><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Step 5: Quality Control and Statistics<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">After removing duplicates, it&#8217;s important to check the quality of the BAM files. This includes alignment statistics and coverage information.<\/p>\n\n\n\n<p class=\"has-subtle-background-background-color has-background wp-block-paragraph\" style=\"font-size:15px\"><code>samtools flagstat S1.sorted.bam &gt; S1.sorted.bam.flagstat<\/code><\/p>\n\n\n\n<p class=\"has-subtle-background-background-color has-background wp-block-paragraph\" style=\"font-size:15px\"><code>samtools depth S1.sorted.bam &gt; S1.sorted.bam.depth<\/code><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These steps prepare your data for downstream variant calling and ensure that only high-quality data is used for further analysis.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>License and Copyright<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Copyright (C) 2024 Xingwan Yi, Zhiqin Long.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This tutorial provides a step-by-step guide for preprocessing high-throughput sequencing data using tools such as Trimmomatic, BWA-MEM2, Samtools, and Picard. The workflow includes quality trimming of sequencing reads, aligning them to the reference genome, sorting the resulting BAM files, and removing duplicate reads to ensure accurate variant calling. Required Tools and Files: Required Files: Step [&hellip;]<\/p>\n","protected":false},"author":103178,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-81","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/blogs.ubc.ca\/evobiodriresources\/wp-json\/wp\/v2\/pages\/81","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.ubc.ca\/evobiodriresources\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/blogs.ubc.ca\/evobiodriresources\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.ubc.ca\/evobiodriresources\/wp-json\/wp\/v2\/users\/103178"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.ubc.ca\/evobiodriresources\/wp-json\/wp\/v2\/comments?post=81"}],"version-history":[{"count":5,"href":"https:\/\/blogs.ubc.ca\/evobiodriresources\/wp-json\/wp\/v2\/pages\/81\/revisions"}],"predecessor-version":[{"id":86,"href":"https:\/\/blogs.ubc.ca\/evobiodriresources\/wp-json\/wp\/v2\/pages\/81\/revisions\/86"}],"wp:attachment":[{"href":"https:\/\/blogs.ubc.ca\/evobiodriresources\/wp-json\/wp\/v2\/media?parent=81"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}