May 23, 2019

Start Docker Demo

Overview

What is Reproducible Analysis?

"Reproducible Analysis is an important part of reproducible research. Reproducible analysis requires that all components of the analysis be archived so that anyone can independently repeat the analysis and arrive at exactly the same results" - Josh Granek

Who is "anyone"

  • Labmates
  • Collaborators
  • Competitors
  • Everyone else interested in your work
  • Someone who wants to apply your analysis to their own data
  • You, 6 months from now

Not Covered Here

  • Reproducibility in the Wet Lab
  • Lots of Other Stuff
  • Many Other Valid Approaches
  • Important Details

Soapbox

  1. Setup
  2. Stand on

Reproducible Analysis

. . . is like eating your vegetables

Three Pillars of Reproducible Analysis

  • Raw Data
  • Compute Environment
  • Analysis Process

Executive Summary

  • Archive the Necessary Components!

TLDR

Raw Data

Care and Handling of Raw Data

  1. Download
  2. Provenance
    • Verify checksum
    • Generate checksum if it didn't come with the data
  3. Protection
    • Set READ-ONLY permissions on data files and data directory
    • Archive ASAP

Switch to Jupyter

Checksums

cd $RAWDATA_DIR
head *.csv
## ==> data1.csv <==
## value
## 1
## 2
## 3
## 4
## 
## ==> data2.csv <==
## value
## 5
## 6
## 7
## 8

Checksums

cd $RAWDATA_DIR
md5sum *.csv
## 25ef3a11e49406d5f4be5163c0d1ce06  data1.csv
## 7e9e9bf71d2aaa2486a6536f3043311e  data2.csv

Checksums File

cd $RAWDATA_DIR
md5sum *.csv > mydata_md5.txt
md5sum -c mydata_md5.txt
## data1.csv: OK
## data2.csv: OK

Catching Changes

sed -i s/7/3/ $DATA2
cat $DATA2
## value
## 5
## 6
## 3
## 8
cd $RAWDATA_DIR
md5sum -c mydata_md5.txt
## data1.csv: OK
## data2.csv: FAILED
## md5sum: WARNING: 1 computed checksum did NOT match

READ-ONLY

ls -ld $RAWDATA_DIR
ls -ltr $RAWDATA_DIR
## drwxr-xr-x 2 guest users 4096 May 23 05:29 /tmp/RtmpzzX7pj/rr_raw_data
## total 12
## -rw-r--r-- 1 guest users 14 May 23 05:29 data1.csv
## -rw-r--r-- 1 guest users 88 May 23 05:29 mydata_md5.txt
## -rw-r--r-- 1 guest users 14 May 23 05:29 data2.csv
chmod -R a-w $RAWDATA_DIR
ls -ld $RAWDATA_DIR
ls -ltr $RAWDATA_DIR
## dr-xr-xr-x 2 guest users 4096 May 23 05:29 /tmp/RtmpzzX7pj/rr_raw_data
## total 12
## -r--r--r-- 1 guest users 14 May 23 05:29 data1.csv
## -r--r--r-- 1 guest users 88 May 23 05:29 mydata_md5.txt
## -r--r--r-- 1 guest users 14 May 23 05:29 data2.csv

Preventing Modification

echo "This is not the data you are looking for" > $DATA2
## bash: /tmp/RtmpzzX7pj/rr_raw_data/data2.csv: Permission denied
sed s/7/3/ -i $DATA2
## sed: couldn't open temporary file /tmp/RtmpzzX7pj/rr_raw_data/sedpXBHr5: Permission denied

Preventing Deletion

rm -rf $RAWDATA_DIR
## rm: cannot remove '/tmp/RtmpzzX7pj/rr_raw_data/mydata_md5.txt': Permission denied
## rm: cannot remove '/tmp/RtmpzzX7pj/rr_raw_data/data1.csv': Permission denied
## rm: cannot remove '/tmp/RtmpzzX7pj/rr_raw_data/data2.csv': Permission denied

Archiving Raw Sequence Data @ NCBI

"Archive First" Movement

  • Free Backup
  • Organize metadata

Alternatives to NCBI

Other Data Types

???????????????

Compute Environment

Reproducible Computing Environment

Containerization for Reproducible Research

  • Versioning: Lock down the specific computing environment used for an analysis
  • Portability: Runs on Linux, Mac, and Windows
  • Sharebility: Docker Hub/Singularity Hub
  • Scalability: Runs on a laptop, massive server, and everything in between

Container Platforms

Docker Demo

Analysis Process

Analysis Process

  • Script Everything
  • Share Everything
  • Version Control

Script Everything

  • R, Python, Shell Scripts, Rmarkdown, Makefiles, Jupyter, etc
  • No Excel

Share Everything

  • Scripts
    • run parameters are embedded
  • Metadata
  • Documentation
  • Manuscript (optional)

Version Control

Heard of Version Control?

Heard of Github?

What is Version Control?

What is Version Control?

  • Track
  • Backup
  • Rewind
  • Branch
  • Collaborate
  • Publish

What is Version Control?

Version Control Software

Git-repository Hosts

Organization

My strategy

  1. Raw Data directory: must be read-only
  2. Output directory: everything generated by a script
  3. Git repository: Code and metadata

Alternatives

Resources

Version Control

Scientific Computing Advice