{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Working with Loops \n",
    "\n",
    "## Shell Variables\n",
    "Assign the variables in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "source bioinf_intro_config.sh\n",
    "mkdir -p $TRIMMED $STAR_OUT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Brief journey into `for` loops\n",
    "`for` loops take our use of the `$FASTQ` variable to the next level! It is analogous to how you would teach a child to set the table: \"FOR each place at the table, put a plate . . .,\n",
    "At the shell you phrase it like this:\n",
    "\n",
    "    for PERSON in Alice Bob Carol Dave Eve\n",
    "    do\n",
    "    put plate at PERSON's place\n",
    "    put napkin at PERSON's place\n",
    "    put fork at PERSON's place\n",
    "    put spoon at PERSON's place\n",
    "    put knife at PERSON's place\n",
    "    done\n",
    "\n",
    "Here is a real example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "______A________\n",
      "______B________\n",
      "______C________\n",
      "______D________\n",
      "______E________\n",
      "______F________\n"
     ]
    }
   ],
   "source": [
    "for FASTQ in A B C D E F\n",
    "    do\n",
    "       echo \"______${FASTQ}________\"\n",
    "    done"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `for` loop in Bash is conceptually the same as in any other programming language, although the syntax may be different.  The `do` and `done` are essential - `do` needs to be before the \"loop body\" (what is going to be repeated) and `done` needs to be after it.\n",
    "\n",
    "So let's try something almost useful:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RUNNING FASTQ: 21_2019_P_M1_S21_L002_R1\n"
     ]
    }
   ],
   "source": [
    "for FASTQ in 21_2019_P_M1_S21_L002_R1\n",
    "    do\n",
    "        echo \"RUNNING FASTQ: ${FASTQ}\"\n",
    "    done"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Now for the real thing . . .\n",
    "### Let's run the pipeline in a loop:\n",
    "Notice that we are now assigning to the `$FASTQ` variable in the `for` statement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------- TRIMMING: 21_2019_P_M1_S21_L002_R1 ----------------\n",
      "Command Line: /home/jovyan/work/scratch/bioinf_intro/myinfo/neb_e7600_adapters.fasta /data/hts_2019_data/hts2019_pilot_rawdata/21_2019_P_M1_S21_L002_R1_001.fastq.gz -q 20 -x 0.5 -o /home/jovyan/work/scratch/bioinf_intro/trimmed_fastqs/21_2019_P_M1_S21_L002_R1_001.trim.fastq.gz\n",
      "Scale used: 2.2\n",
      "Phred: 33\n",
      "Threshold used: 751 out of 300000\n",
      "Adapter Adapter (AGATCGGAAGAGCACACGTCTGAACTCCAGTCA): counted 2515 at the 'end' of '/data/hts_2019_data/hts2019_pilot_rawdata/21_2019_P_M1_S21_L002_R1_001.fastq.gz', clip set to 6\n",
      "Files: 1\n",
      "Total reads: 2437108\n",
      "Too short after clip: 1347\n",
      "Clipped 'end' reads: Count: 44977, Mean: 15.55, Sd: 8.27\n",
      "Trimmed 288960 reads by an average of 1.70 bases on quality < 20\n",
      "---------------- MAPPING: 21_2019_P_M1_S21_L002_R1 ----------------\n",
      "Jun 26 15:35:08 ..... started STAR run\n",
      "Jun 26 15:35:08 ..... loading genome\n",
      "Jun 26 15:35:09 ..... started mapping\n",
      "Jun 26 15:36:32 ..... finished successfully\n"
     ]
    }
   ],
   "source": [
    "for FASTQ in 21_2019_P_M1_S21_L002_R1\n",
    "    do\n",
    "        echo \"---------------- TRIMMING: $FASTQ ----------------\"\n",
    "        fastq-mcf \\\n",
    "            $MYINFO/neb_e7600_adapters.fasta \\\n",
    "            $RAW_FASTQS/${FASTQ}_001.fastq.gz \\\n",
    "            -q 20 -x 0.5 \\\n",
    "            -o $TRIMMED/${FASTQ}_001.trim.fastq.gz\n",
    "        \n",
    "        echo \"---------------- MAPPING: $FASTQ ----------------\"\n",
    "        STAR \\\n",
    "            --runMode alignReads \\\n",
    "            --twopassMode None \\\n",
    "            --genomeDir $GENOME_DIR \\\n",
    "            --readFilesIn $TRIMMED/${FASTQ}_001.trim.fastq.gz \\\n",
    "            --readFilesCommand gunzip -c \\\n",
    "            --outFileNamePrefix ${STAR_OUT}/${FASTQ}_ \\\n",
    "            --quantMode GeneCounts \\\n",
    "            --outSAMtype None \\\n",
    "            --runThreadN 2\n",
    "    done"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### And let's check the result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "21_2019_P_M1_S21_L001_R1_short_introns_Aligned.sortedByCoord.out.bam\n",
      "21_2019_P_M1_S21_L001_R1_short_introns_Aligned.sortedByCoord.out.bam.bai\n",
      "21_2019_P_M1_S21_L001_R1_short_introns_Log.final.out\n",
      "21_2019_P_M1_S21_L001_R1_short_introns_Log.out\n",
      "21_2019_P_M1_S21_L001_R1_short_introns_Log.progress.out\n",
      "21_2019_P_M1_S21_L001_R1_short_introns_ReadsPerGene.out.tab\n",
      "21_2019_P_M1_S21_L001_R1_short_introns_SJ.out.tab\n",
      "21_2019_P_M1_S21_L002_R1_Aligned.out.bam\n",
      "21_2019_P_M1_S21_L002_R1_Log.final.out\n",
      "21_2019_P_M1_S21_L002_R1_Log.out\n",
      "21_2019_P_M1_S21_L002_R1_Log.progress.out\n",
      "21_2019_P_M1_S21_L002_R1_ReadsPerGene.out.tab\n",
      "21_2019_P_M1_S21_L002_R1_short_introns_Aligned.sortedByCoord.out.bam\n",
      "21_2019_P_M1_S21_L002_R1_short_introns_Aligned.sortedByCoord.out.bam.bai\n",
      "21_2019_P_M1_S21_L002_R1_short_introns_Log.final.out\n",
      "21_2019_P_M1_S21_L002_R1_short_introns_Log.out\n",
      "21_2019_P_M1_S21_L002_R1_short_introns_Log.progress.out\n",
      "21_2019_P_M1_S21_L002_R1_short_introns_ReadsPerGene.out.tab\n",
      "21_2019_P_M1_S21_L002_R1_short_introns_SJ.out.tab\n",
      "21_2019_P_M1_S21_L002_R1_SJ.out.tab\n",
      "genome_Log.out\n",
      "\u001b[0m\u001b[01;34mmultiqc_data\u001b[0m\n",
      "multiqc_report.html\n"
     ]
    }
   ],
   "source": [
    "ls ${STAR_OUT}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "N_unmapped\t46060\t46060\t46060\n",
      "N_multimapping\t35006\t35006\t35006\n",
      "N_noFeature\t12466\t2145291\t18783\n",
      "N_ambiguous\t203327\t820\t316\n",
      "CNAG_04548\t0\t0\t0\n",
      "CNAG_07303\t0\t0\t0\n",
      "CNAG_07304\t6\t0\t6\n",
      "CNAG_00001\t0\t0\t0\n",
      "CNAG_07305\t1\t0\t1\n",
      "CNAG_00002\t51\t0\t51\n"
     ]
    }
   ],
   "source": [
    "head ${STAR_OUT}/21_2019_P_M1_S21_L002_R1_ReadsPerGene.out.tab"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}