source bioinf_intro_config.sh
mkdir -p $TRIMMED $STAR_OUT $IGV_DIR

Prepare Data

This notebook depends on BAM files generated in the A globy pipeline section of the Looping with Globs notebook. If you have not already generated the BAM files, please run that notebook now

IGV needs indices for the BAM files. The index allows it to quickly load reads from different parts of the genome.

for BAM in ${STAR_OUT}/*_Aligned.sortedByCoord.out.bam
    do
        echo $BAM
        samtools index $BAM
    done

/home/jovyan/work/scratch/bioinf_intro/star_out/21_2019_P_M1_S21_L001_R1_Aligned.sortedByCoord.out.bam
/home/jovyan/work/scratch/bioinf_intro/star_out/21_2019_P_M1_S21_L001_R1_short_introns_Aligned.sortedByCoord.out.bam
/home/jovyan/work/scratch/bioinf_intro/star_out/21_2019_P_M1_S21_L002_R1_Aligned.sortedByCoord.out.bam
/home/jovyan/work/scratch/bioinf_intro/star_out/21_2019_P_M1_S21_L002_R1_short_introns_Aligned.sortedByCoord.out.bam

ls -ltr ${STAR_OUT}

total 577416
-rw-r--r-- 1 jovyan users    166944 Jun 24 09:50 21_2019_P_M1_S21_L001_R1_short_introns_ReadsPerGene.out.tab
-rw-r--r-- 1 jovyan users 104774284 Jun 24 09:50 21_2019_P_M1_S21_L001_R1_short_introns_Aligned.sortedByCoord.out.bam
-rw-r--r-- 1 jovyan users    918773 Jun 24 09:50 21_2019_P_M1_S21_L001_R1_short_introns_SJ.out.tab
-rw-r--r-- 1 jovyan users       246 Jun 24 09:50 21_2019_P_M1_S21_L001_R1_short_introns_Log.progress.out
-rw-r--r-- 1 jovyan users     18641 Jun 24 09:50 21_2019_P_M1_S21_L001_R1_short_introns_Log.out
-rw-r--r-- 1 jovyan users      1849 Jun 24 09:50 21_2019_P_M1_S21_L001_R1_short_introns_Log.final.out
-rw-r--r-- 1 jovyan users    166808 Jun 24 09:51 21_2019_P_M1_S21_L002_R1_short_introns_ReadsPerGene.out.tab
-rw-r--r-- 1 jovyan users 102828697 Jun 24 09:51 21_2019_P_M1_S21_L002_R1_short_introns_Aligned.sortedByCoord.out.bam
-rw-r--r-- 1 jovyan users    913218 Jun 24 09:51 21_2019_P_M1_S21_L002_R1_short_introns_SJ.out.tab
-rw-r--r-- 1 jovyan users       246 Jun 24 09:51 21_2019_P_M1_S21_L002_R1_short_introns_Log.progress.out
-rw-r--r-- 1 jovyan users     18643 Jun 24 09:51 21_2019_P_M1_S21_L002_R1_short_introns_Log.out
-rw-r--r-- 1 jovyan users      1849 Jun 24 09:51 21_2019_P_M1_S21_L002_R1_short_introns_Log.final.out
-rw-r--r-- 1 jovyan users 168781979 Jun 26 15:11 21_2019_P_M1_S21_L002_R1_Aligned.out.bam
-rw-r--r-- 1 jovyan users   1068750 Jun 26 15:11 multiqc_report.html
drwxr-xr-x 2 jovyan users      4096 Jun 26 15:11 multiqc_data
-rw-r--r-- 1 jovyan users     19079 Jun 26 16:06 genome_Log.out
-rw-r--r-- 1 jovyan users    167049 Jun 26 16:29 21_2019_P_M1_S21_L003_R1_ReadsPerGene.out.tab
-rw-r--r-- 1 jovyan users    921520 Jun 26 16:29 21_2019_P_M1_S21_L003_R1_SJ.out.tab
-rw-r--r-- 1 jovyan users       364 Jun 26 16:29 21_2019_P_M1_S21_L003_R1_Log.progress.out
-rw-r--r-- 1 jovyan users     17731 Jun 26 16:29 21_2019_P_M1_S21_L003_R1_Log.out
-rw-r--r-- 1 jovyan users      1849 Jun 26 16:29 21_2019_P_M1_S21_L003_R1_Log.final.out
-rw-r--r-- 1 jovyan users    166942 Jun 26 16:32 21_2019_P_M1_S21_L001_R1_ReadsPerGene.out.tab
-rw-r--r-- 1 jovyan users 104982330 Jun 26 16:32 21_2019_P_M1_S21_L001_R1_Aligned.sortedByCoord.out.bam
-rw-r--r-- 1 jovyan users    923575 Jun 26 16:32 21_2019_P_M1_S21_L001_R1_SJ.out.tab
-rw-r--r-- 1 jovyan users       364 Jun 26 16:32 21_2019_P_M1_S21_L001_R1_Log.progress.out
-rw-r--r-- 1 jovyan users     18155 Jun 26 16:32 21_2019_P_M1_S21_L001_R1_Log.out
-rw-r--r-- 1 jovyan users      1848 Jun 26 16:32 21_2019_P_M1_S21_L001_R1_Log.final.out
-rw-r--r-- 1 jovyan users    166811 Jun 26 16:34 21_2019_P_M1_S21_L002_R1_ReadsPerGene.out.tab
-rw-r--r-- 1 jovyan users 103015847 Jun 26 16:34 21_2019_P_M1_S21_L002_R1_Aligned.sortedByCoord.out.bam
-rw-r--r-- 1 jovyan users    918360 Jun 26 16:34 21_2019_P_M1_S21_L002_R1_SJ.out.tab
-rw-r--r-- 1 jovyan users       364 Jun 26 16:34 21_2019_P_M1_S21_L002_R1_Log.progress.out
-rw-r--r-- 1 jovyan users     18155 Jun 26 16:34 21_2019_P_M1_S21_L002_R1_Log.out
-rw-r--r-- 1 jovyan users      1848 Jun 26 16:34 21_2019_P_M1_S21_L002_R1_Log.final.out
-rw-r--r-- 1 jovyan users     65344 Jun 26 16:40 21_2019_P_M1_S21_L001_R1_Aligned.sortedByCoord.out.bam.bai
-rw-r--r-- 1 jovyan users     32536 Jun 26 16:40 21_2019_P_M1_S21_L001_R1_short_introns_Aligned.sortedByCoord.out.bam.bai
-rw-r--r-- 1 jovyan users     63664 Jun 26 16:40 21_2019_P_M1_S21_L002_R1_Aligned.sortedByCoord.out.bam.bai
-rw-r--r-- 1 jovyan users     32248 Jun 26 16:40 21_2019_P_M1_S21_L002_R1_short_introns_Aligned.sortedByCoord.out.bam.bai

Downloading Everything

Downloading files through Jupyter

We need to download the following files: 1. BAM file(s) 2. Index for each BAM file 3. Genome sequence (FASTA) 4. Genome annotation (GTF)

Jupyter File Browser

For each file: 1. Select checkbox next to filename 2. Click download

Packaging up files

Because we want to download several files, it might be easier to package them up using a program called tar, then download the resulting package file (commonly called a tarball) containing all of the files we need.

Link Directory

First we will do a little hack - we will create a directory of links to all the files we need to download. Since the original files are in two different directories, we are essentially makig a single “virtual directory” containing all the files.

ln -s ${STAR_OUT}/*.bam* $GENOME_DIR/$GTF $GENOME_DIR/$FA $IGV_DIR

Taring

Here are the command line options we will use with tar

• --dereference treat the soft-links in our virtual directory as if they were the files that are linked to

• --create we are creating a tarball, not unpackaging it

• --gzip tells tar to also gzip (compress) the file

• --verbose tell us what is happening while running

• --file TARBALL_NAME tells tar what to name the tarball it is creating

• --directory PATH the base directory for the files to be tarred

• FILE[S]_TO_PACKAGE

tar --dereference \
    --create \
    --gzip \
    --verbose \
    --file $CUROUT/stuff_for_igv.tgz \
    --directory $CUROUT \
    $(basename $IGV_DIR)

igv/
igv/21_2019_P_M1_S21_L001_R1_Aligned.sortedByCoord.out.bam
igv/21_2019_P_M1_S21_L002_R1_Aligned.out.bam
igv/Cryptococcus_neoformans_var_grubii_h99.CNA3.39.gtf
igv/21_2019_P_M1_S21_L001_R1_Aligned.sortedByCoord.out.bam.bai
igv/21_2019_P_M1_S21_L002_R1_Aligned.sortedByCoord.out.bam.bai
igv/21_2019_P_M1_S21_L002_R1_short_introns_Aligned.sortedByCoord.out.bam
igv/21_2019_P_M1_S21_L001_R1_short_introns_Aligned.sortedByCoord.out.bam.bai
igv/21_2019_P_M1_S21_L002_R1_Aligned.sortedByCoord.out.bam
igv/Cryptococcus_neoformans_var_grubii_h99.CNA3.dna.toplevel.fa
igv/21_2019_P_M1_S21_L002_R1_short_introns_Aligned.sortedByCoord.out.bam.bai
igv/21_2019_P_M1_S21_L001_R1_short_introns_Aligned.sortedByCoord.out.bam

Let’s check that it worked …

echo $CUROUT
ls $CUROUT

/home/jovyan/work/scratch/bioinf_intro
count_out  igv     qc_output  stuff_for_igv_shorter_intron.tgz  trimmed_fastqs
genome     myinfo  star_out   stuff_for_igv.tgz

Download the tarball

Now you can do one of the following to download the tarball to your laptop:

1. Click on the “Jupyter” logo above to open the Jupyter file browser

2. Naviagte your way to the directory where we saved the tarball (see echo $CUROUT above)

3. Click the checkbox next to stuff_for_igv.tgz, then click on the Download button near the top of the JUpyter windown to download it.

Unpacking our tarball

On a Mac you can “untar” by double clicking on the file in finder, or at the terminal with the command tar -zxf my_notebooks.tgz.

On Windows, you can download software that will do it, such as 7-Zip

Download IGV

It is often helpful to use visualization software to interact with an assembly. We will be using Integrative Genomics Viewer (IGV) because it is pretty good, somewhat user friendly, and cross-platform. We need to download Integrative Genomics Viewer (IGV) for visualizing reads on our laptops. See instructions below for the type of computer you are using.

OS X (Macs)

1. a Mac only version

2. the cross-platform version ### Windows

3. a Windows only version

4. the cross-platform version ### Linux

5. the cross-platform version

Using IGV

Run IGV

If you downloaded the Mac specific version, just double click. If you have the cross-platform version: unzip the binary distribution archive in a folder of your choosing. IGV is launched from a command prompt: follow instructions in the “readme” file. To launch igv on Mac or Linux platforms use the shell script “igv.sh”. On Windows use “igv.bat”.

Load Files

Once IGV is running do the following within IGV: 1. Genome Sequence: Genomes->Load Genome From File: (select the FASTA file we just downloaded from Jupyter) 2. Annotation: File->Load From File: (select the GTF file we just downloaded from Jupyter) 3. Bamfile: File->Load From File: (select the BAM files we just downloaded from Jupyter)

Configurations to explore

1. Zoom in until reads are visible

2. Right click -> Color alignments by -> first-of-pair strand

3. Right click->Collapsed

Look around

A few things to look for: - read strand relative to annotated gene strand - intron-spanning reads - SNPs - Areas with no reads - Coverage depth plot - antisense reads - non-protein-coding RNAs