# -*- org-export-babel-evaluate: (quote inline-only);  -*-
#+TITLE: Conducting Genomic Symphonies with Bioconductor
#+TAGS: borrowed(b)

#+OPTIONS: toc:t H:2
#+PROPERTY: exports both
#+PROPERTY: results output
#+PROPERTY: eval no-export
#+PROPERTY: session *R*

#+BEGIN_LaTeX
\AtBeginSection[]
{
  \begin{frame}<beamer>{Outline}
    \tableofcontents[currentsection]
  \end{frame}
}
#+END_LaTeX

# First I will talk a bit about Genentech, in part to tell you where I
# am coming from, in part to motivate the talk. And also so that I
# have at least three pretty slides in an hour long talk. So enjoy
# them while they last.
* Genentech
** 
   :PROPERTIES:
   :BEAMER_OPT: plain
   :END:
   #+ATTR_LATEX: :width 11cm
[[file:gne-1.pdf]]

   # Genentech was founded about 40 years ago, in South San
   # Francisco. About 8 years ago it was merged into Roche. Roche, a
   # Swiss company, is the largest biotech in the world with almost
   # 90,000 employees.
** 
   :PROPERTIES:
   :BEAMER_OPT: plain
   :END:
   #+ATTR_LATEX: :width 11cm
[[file:gne-2.pdf]]

   # gRED, Genentech Research and Early Development is a
   # research-focused organization with XXX researchers occupying the
   # largest amount of research lab space in the world.
** 
   :PROPERTIES:
   :BEAMER_OPT: plain
   :END:
   #+ATTR_LATEX: :width 11cm
[[file:gne-3.pdf]]

   # Traditionally we have researched and discovered targets and
   # delivered those targets to the development side, with the hope
   # that they eventually become targets. Recently, thanks to the
   # reduced cost of genomic sequencing, we have started sequencing
   # all of our clinical trials, so we have genomic data on a massive
   # scale, and we want to feed that data back into research to test
   # focused hypotheses and hopefully discover new targets and
   # biomarkers.
   
* Genomic workflows
  # This renewed emphasis on genomics calls attention to how we carry
  # out genomic workflows.
** Genomic workflows are symphonies of different tools
*** 
  :PROPERTIES:
  :BEAMER_act: <only@1>
  :END:
   #+ATTR_LATEX: :width 11cm
  [[file:symphony.pdf]]

   # The typical workflow begins with some raw data, maybe sequences
   # or alignments, reduces those data through alignment or variant
   # calling, etc, to data that we can explore, model, and
   # visualize. Eventually we report our findings in dynamic reports.
*** 
  :PROPERTIES:
  :BEAMER_act: <only@2>
  :END:
   #+ATTR_LATEX: :width 11cm
   [[file:symphony-tools.pdf]]

   # To implement these workflows, we have to use a myriad of
   # different packages....
** Tweet-size example from bedtools tutorial

*** 
    :PROPERTIES:
    :BEAMER_act: <only@1>
    :END:
 #+ATTR_LATEX: :width 9cm
 [[file:tweet-question.jpg]]

 # Probably the simplest example you can find would be the size of a
 # tweet, so I took this example that was cited in the bedtools
 # tutorial.
 
*** 
    :PROPERTIES:
    :BEAMER_act: <only@2-3>
    :END:
   #+ATTR_LATEX: :width 9cm
 [[file:tweet-question-answer.jpg]]

*** 
    :PROPERTIES:
    :BEAMER_act: <only@4>
    :END:
 #+ATTR_LATEX: :width 9cm

# But if you dig a bit deeper in to the twitter thread, you find this
# response. It points out that the awk is cryptic and undesirable, but
# there is no feasible way for bedtools to solve it. Unfortunately,
# once you go outside of the scope of bedtools, there is no safety
# net, and you fall down to the level of unix tools.
 [[file:tweet-lose-the-awk.jpg]]

*** 
  :PROPERTIES:
  :BEAMER_act: <only@3->
  :END:
  #+ATTR_LATEX: :width 11cm

   [[file:tweet-pipeline.pdf]]
   
** Typical real-world example from bedtools tutorial
   # Here is a real-world example from the bedtools tutorial...
   Compute the pairwise similarity between samples of DNAse
   hypersensitivity regions, according to the =bedtools= Jaccard
   statistic.

   [[file:jaccard-pipeline.pdf]]

** bedtools solution

*** 								      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.3
    :END:

**** Languages used
    * @@beamer:<2->@@ shell
    * @@beamer:<2->@@ GNU parallel
    * @@beamer:<2->@@ awk
    * @@beamer:<3->@@ sed
    * @@beamer:<3->@@ perl
    * @@beamer:<4->@@ python
    * @@beamer:<5->@@ R

**** Side-effects
    * @@beamer:<2->@@ 400 =.jaccard=
    * @@beamer:<3->@@ =pairwise.txt=
    * @@beamer:<4->@@ =pairwise.mat=

*** 								      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.7
    :END:

**** Compute pairwise distances in parallel
     :PROPERTIES:
     :BEAMER_act: <only@2>
     :END:
  #+begin_src shell
  parallel "bedtools jaccard -a {1} -b {2} \
        | awk 'NR>1' \
        | cut -f 3 \
        > {1}.{2}.jaccard" \
        ::: `ls *.merge.bed`
	::: `ls *.merge.bed`
  #+end_src

**** Combine jaccard files
     :PROPERTIES:
     :BEAMER_act: <only@3>
     :END:
  #+begin_src shell
  find . \
    | grep jaccard \
    | xargs grep "" \
    | sed -e s"/\.\///" \
    | perl -pi -e "s/.bed./.bed\t/" \
    | perl -pi -e "s/.jaccard:/\t/" \
    > pairwise.txt
  #+end_src
  
**** Reshape into matrix
     :PROPERTIES:
     :BEAMER_act: <only@4>
     :END:
  #+begin_src shell
  awk 'NF==3' pairwise.txt \
  | awk '$1 ~ /^f/ && $2 ~ /^f/' \
  | python make-matrix.py \
  > pairwise.mat
  #+end_src

**** Plot the matrix
     :PROPERTIES:
     :BEAMER_act: <only@5>
     :END:
  #+begin_src shell
  R
  #+end_src
  #+begin_src R
    library(gplots)
    library(RColorBrewer)
    jaccard_df <-
        read.table('pairwise.dnase.mat')
    jaccard_matrix <-
        as.matrix(jaccard_df[,-1])
    heatmap.2(jaccard_matrix,
              col = brewer.pal(9, "Blues"),
              margins = c(14, 14),
              density.info = "none",
              lhei = c(2, 8),
              trace = "none")
  #+end_src

** Typical obstacles in implementing genomic data analyses
   * Tools are difficult to build, install and run
   * Limitations require mixing languages and semi-compatible,
     inconsistently documented toolsets
   * Interoperability depends on inefficient, complex file formats
   * Analyst has to directly manipulate and manage files, instead of
     focusing on the analysis
   * Reproducibility is hard

#+BEAMER: \centering{
#+ATTR_LATEX: :width 4cm
[[file:obstacle-course.jpg]]
#+BEAMER: }

* R
** R is a platform and language for statistical computing
*** Text							      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.8
    :END:
    * Core principes according to John Chambers in "Extending R":
      * Everything is an \alert{object}
      * Everything that happens is a \alert{function} call
      * \alert{Interfaces} to other software are core to R
    * Addendum: every published extension is a \alert{package}
      * Primary mechanism for distributing statistical computing
	research

*** Logo							      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.2
    :END:
   [[file:Rlogo.png]]
   
** R packages are easy to install
   * CRAN, Bioconductor distribute vetted \alert{packages}
     * Tested as a cohort
     * Standardized through =R CMD check=
   * Package installation usually just works
     * =install.packages("gplots")=

** R has consistent, function-level documentation
   Standalone programs provide documentation in different ways:
   * =man bedtools=?
   * =bedtools intersect --help=?
   * Google?

   Every R \alert{package} provides a man page of each
   \alert{function}:
   #+begin_src R
   ?brewer.pal
   #+end_src

   #+LATEX: \tiny{
   #+begin_example
ColorBrewer palettes

Description:

     Creates nice looking color palettes especially for thematic maps

Usage:

     brewer.pal(n, name)
     
Arguments:

       n: Number of different colors in the palette, minimum 3, maximum
          depending on palette

    name: A palette name from the lists below
   #+end_example
   #+LATEX: }
   
** R enables reproducibility
   * Dependencies trackable through versioned \alert{packages}
   * Packages like switchr and packrat make it easy to record and
     restore sets of package versions
   * =sessionInfo()=

   #+BEAMER: \tiny{
   #+begin_example
R Under development (unstable) (2017-08-02 r73018)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: OS X El Capitan 10.11.6

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] gplots_3.0.1       RColorBrewer_1.1-2

loaded via a namespace (and not attached):
[1] compiler_3.5.0     tools_3.5.0        KernSmooth_2.23-15 gdata_2.18.0      
[5] caTools_1.17.1     bitops_1.0-6       gtools_3.5.0      
   #+end_example
   #+BEAMER: }
   
** R unifies workflows through object-oriented interfaces
   An \alert{object} affords interoperability and abstraction:
  #+begin_src R
    library(gplots)
    library(RColorBrewer)
    jaccard_df <- utils::read.table('pairwise.mat')
    jaccard_matrix <- as.matrix(jaccard_df[,-1])
    heatmap.2(jaccard_matrix, col = brewer.pal(9, "Blues"))
  #+end_src

  #+ATTR_latex: :width 3cm
  [[file:heatmap.jpg]]

** R is improving
#+begin_latex
\framesubtitle{Pushing object orientation down to the C level}
#+end_latex

*** Text							      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.7
    :END:
    
   R 3.5 will add:
   * Object-oriented mechanism for custom implementations of R vectors
     * Compact representations :: Run-length encodings, =1:10= sequences
     * External storage :: Spark, databases, HDF5, Arrow, etc
   * Notions of sortedness and any missingness to the vector API
   * Heuristics that construct compact vectors when it makes sense

   Luke Tierney, Gabe Becker, Tomas Kalibera

*** Picture							      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.3
    :END:
    [[file:wolf-as-sheep.jpg]]
    
* Bioconductor
** Bioconductor
*** 								      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.7
    :END:
  A unified platform for the analysis and comprehension of high-throughput
  genomic data.
  * Started 2002
  * Led by Martin Morgan
  * Core infrastructure maintained by about 8 people, based in
    Roswell Park CRC in Buffalo, NY
  * 1476 \R{} software packages that form a unified platform
  * Well-used and respected.
    * 53k unique IP downloads / month.
    * 21,700 PubMedCentral citations.
  * Embraces the R principles of \alert{object}, \alert{function},
    \alert{interface} and \alert{package}

*** 								      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.3
    :END:

 \begin{center}
 \includegraphics[width=.8\textwidth]{bioconductor_logo_rgb.pdf}
 \bigskip\par
 \includegraphics[width=.3\textwidth]{nid61380-a1-rpci-cmyk-black.pdf}
 \bigskip\par
 \includegraphics[width=!, height=2em]{logo_nhgri.png}
 \bigskip\par
 \includegraphics[width=.8\textwidth]{nci-logo-full.png}
 \bigskip\par
 \includegraphics[width=.6\textwidth]{sound_wordmark.png}
 \end{center}
    
** Bioconductor is growing
   [[file:packages-per-release.png]]
 
** Bioconductor qualities
*** Qualities 							      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.3
    :END:

   #+ATTR_BEAMER: :overlay <alert@+>
   * Discoverable
   * Installable
   * Reliable
   * Documented
   * Supported
   * Integrated
   * Scalable
   * State of the art
   * Community-driven

*** Illustrations						      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.7
    :END:

**** 
     :PROPERTIES:
     :BEAMER_act: <only@1>
     :END:

     [[file:biocviews.jpg]]
     \bigskip
     \par
     [[file:badges.jpg]]

**** 
     :PROPERTIES:
     :BEAMER_act: <only@2>
     :END:

     #+BEAMER: \small{
     #+begin_src R
     source("https://bioconductor.org/biocLite.R")
     biocLite()
     biocLite("Gviz")
     #+end_src
     #+BEAMER: }

**** 
     :PROPERTIES:
     :BEAMER_act: <only@3>
     :END:
     [[file:build-report.jpg]]

**** 
     :PROPERTIES:
     :BEAMER_act: <only@4>
     :END:
     [[file:documentation.jpg]]

**** 
     :PROPERTIES:
     :BEAMER_act: <only@5>
     :END:
     [[file:support.png]]

**** 
     :PROPERTIES:
     :BEAMER_act: <only@6>
     :END:
     [[file:bioc-ecosystem.png]]

**** 
     :PROPERTIES:
     :BEAMER_act: <only@7>
     :END:
#+BEAMER: \tiny{
     #+begin_src R
     se <- TENxBrainData()
     se
     #+end_src

     #+begin_example
## class: SingleCellExperiment 
## dim: 27998 1306127 
## metadata(0):
## assays(1): counts
## rownames: NULL
## rowData names(2): Ensembl Symbol
## colnames(1306127): AAACCTGAGATAGGAG-1 AAACCTGAGCGGCTTC-1 ...
##   TTTGTCAGTTAAAGTG-133 TTTGTCATCTGAAAGA-133
## colData names(4): Barcode Sequence Library Mouse
## reducedDimNames(0):
## spikeNames(0):
     #+end_example

  #+begin_src R
  libSize <- colSums(assay(se)[, 1:1000])
  range(libSize)
  #+end_src

  #+begin_example
  ## [1]  1453 34233
  #+end_example
#+BEAMER: }

**** 
     :PROPERTIES:
     :BEAMER_act: <only@8>
     :END:
     #+ATTR_LATEX: :width 2cm
     [[file:limma.png]]
     #+ATTR_LATEX: :width 2cm
     [[file:DESeq2.png]]
     \bigskip
     \par
     #+ATTR_LATEX: :width 2cm
     [[file:edgeR.png]]
     #+ATTR_LATEX: :width 2cm
     [[file:scater.png]]

**** 
     :PROPERTIES:
     :BEAMER_act: <only@9>
     :END:

      * 1064 unique package maintainers
      * Web users by country:
	
   #+ATTR_LATEX: :width .6\textwidth
   [[file:web-users-country.png]]

    
** Bioconductor is built on shared infrastructure
   #+ATTR_latex: :width 8cm
   [[file:sticker-integration.pdf]]
   
** Central data structures of Bioconductor
*** Data on genomic ranges					      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.5
    :BEAMER_env: block
    :END:
    [[file:karyogram.png]]
    
*** Summarized data						      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.5
    :BEAMER_env: block
    :END:
    [[file:summarized.jpg]]

** GRanges: data on genomic ranges
      [[file:granges.pdf]]
   

   | seqnames | start | end | strand | ... |
   |----------+-------+-----+--------+-----|
   | chr1     |     1 |  10 | +      |     |
   | chr1     |    15 |  24 | -      |     |

   * Plus, sequence information (lengths, genome, etc)

** SummarizedExperiment: the central data model
   [[file:se.jpg]]   

** Bioconducting the tweeted workflow
*** 
    :PROPERTIES:
    :BEAMER_act: <only@1>
    :END:
    #+ATTR_LATEX: :width 11cm
    [[file:tweet-pipeline.pdf]]

*** 
    :PROPERTIES:
    :BEAMER_act: <only@2>
    :END:
    #+ATTR_LATEX: :width 11cm   
    [[file:tweet-pipeline-bioc.pdf]]

** Bioconducting the pairwise Jaccard workflow
*** Define a function for the Jaccard statistic
    :PROPERTIES:
    :BEAMER_act: <only@1>
    :END:
    
   #+begin_src R
     jaccard <- function(x, y) {
         gr_x <- import(x)
         gr_y <- import(y)
         intersects <- intersect(gr_x, gr_y, ignore.strand=TRUE)
         unions <- union(gr_x, gr_y, ignore.strand=TRUE)
         sum(width(intersects)) / sum(width(unions))
     }
   #+end_src

*** Compute the statistics in parallel
    :PROPERTIES:
    :BEAMER_act: <only@2>
    :END:

   #+begin_src R
     files <- Sys.glob("*.merge.bed")
     jaccard_matrix <- outer(files, files, 
           function(a, b) mcmapply(jaccard, a, b))
   #+end_src

*** Make the plot
    :PROPERTIES:
    :BEAMER_act: <only@3>
    :END:

   #+begin_src R
     library(gplots)
     library(RColorBrewer)
     heatmap.2(jaccard_matrix, col = brewer.pal(9, "Blues"))
   #+end_src

  #+ATTR_latex: :width 3cm
  [[file:heatmap.jpg]]

** GenomicWidgets: interactive genomic plots for Shiny/RMD 
#+begin_latex
\framesubtitle{by Alicia Schep, Sarah Kummerfeld at Genentech}
#+end_latex
   [[file:genomic-widgets.jpg]]

* Usability
** The Ranges infrastructure is an incubator
   [[file:insights.pdf]]
   * Should be accessible to the average Bioconductor user
   
** Is the transition happening?
   * From a typical package submission:
     #+begin_quote
     =Imports: checkmate, dplyr, ggplot2, tidyr=
     #+end_quote
   * A typical initial response:
     #+attr_LATEX: :width 10cm
     [[file:martin-comment.jpg]]

** Aspects of software quality: the ilities
  #+begin_latex
  \includegraphics<1>[width=8cm]{ilities.png}
  \includegraphics<2>[width=8cm]{ilities-usability.png}
  #+end_latex

** Cognitive Dimensions of Notations
   * Thomas Green and Marian Petre (1996) proposed 14 dimensions of
     usability in the context of visual programming
   * Many are interrelated and in balance with each other
   * Guide for evaluating usability and as a framework for discussing
     interface design trade-offs
     
** Green's cognitive dimensions 				  :B_columns:
   :PROPERTIES:
   :BEAMER_env: columns
   :BEAMER_env: block
   :END:
    
*** Column 1
    :PROPERTIES:
    :BEAMER_col: 0.5
    :END:
 * *Abstraction gradient*
 * *Closeness of mapping*
   # relates to quality of abstraction, linked to learnability
 * Consistency
 * *Diffuseness*
   # really about expressiveness
 * Error-proneness 
 * *Hard mental operations*
 * Hidden dependencies 

*** Column 2
    :PROPERTIES:
    :BEAMER_col: 0.5
    :END:
 * Provisionality
   # Helps the user predict outcome of actions, avoids premature
   # commitment, like a "dry run"
 * Premature commitment
   # Potential for over-tightening, but same is true of other cleanings
 * Progressive evaluation
   # May frame some objections to the pipe
 * Role-expressiveness
   # Folded into diffuseness/terseness, but relates a bit more to
   # discoverability, e.g., is the purpose of a function clear from
   # its name?
 * Secondary notation
 * Viscosity (robustness)
   # How easy is it to adapt code to new problems? Abstraction is key...
 * *Visibility*

** Abstraction
*** Procedural abstraction
     A compound operation that enables the *user* tell the computer
     what to do without telling it how to do it.
    
*** Data abstraction
    #+begin_quote
     "A methodology that enables us to isolate how a compound data
     object is *used* from the details of how it is constructed from
     more primitive data objects"
    #+end_quote
    Structure and Interpretation of Computer Programs (1979)
     
** In the absence of abstraction
   * We often start with a BED file:
     #+begin_src bash
     bash-3.2$  ls *.bed
     #+end_src
     #+begin_example
     my.bed
     #+end_example
     
   * And we turn to R to analyze the data
     #+begin_src R
       df <- read.table("my.bed", sep="\t")
       colnames(df) <- c("chrom", "start", "end")
     #+end_src     
     #+begin_example
  chrom     start       end
1  chr7 127471196 127472363
2  chr7 127472363 127473530
3  chr7 127473530 127474697
4  chr9 127474697 127475864
5  chr9 127475864 127477031
     #+end_example
     
** But file formats differ in important ways
   Now for a GFF file:
     #+begin_src R
       df <- read.table("my.bed", sep="\t")
       colnames(df) <- c("chr", "start", "end")
     #+end_src
     
*** Columns							  :B_columns:
    :PROPERTIES:
    :BEAMER_env: columns
    :END:
    
*** GFF 							      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.5
    :BEAMER_env: block
    :END:

    #+begin_example
   chr     start       end
1 chr7 127471197 127472363
2 chr7 127472364 127473530
3 chr7 127473531 127474697
4 chr9 127474698 127475864
5 chr9 127475865 127477031
    #+end_example
    
*** BED 							      :BMCOL:
    :PROPERTIES:
    :BEAMER_col: 0.5
    :BEAMER_env: block
    :END:

    #+begin_example
  chrom     start       end
1  chr7 127471196 127472363
2  chr7 127472363 127473530
3  chr7 127473530 127474697
4  chr9 127474697 127475864
5  chr9 127475864 127477031
     #+end_example

** Abstraction lets us focus on the important
   [[file:evolution.pdf]]
   
   * Abstraction is semantic enrichment
     * Enables the user to think of data in terms of the problem domain
     * Hides implementation details
     * Unifies frameworks

** Semantic slack with adjectives
   [[file:mcols-bridge.pdf]]
   
   * Science defies rigidity: we define flexible objects that combine
     strongly typed fields with arbitrary user-level metadata

** Diffuseness (vs expressiveness)
   * Relates to the information density of the code and how well it
     communicates the /intent/ of the programmer
   * Enable the user to convey more meaning with less code
   * Terseness for its own sake makes code obscure, difficult to
     unpack
   * For genomic data, we want the user to express computations in
     terms of the biology

** Our workflow could be more expressive
*** 
    :PROPERTIES:
    :BEAMER_act: <only@1>
    :END:

    #+ATTR_latex: :width 11cm
    [[file:tweet-pipeline-just-bioc.pdf]]

*** 
    :PROPERTIES:
    :BEAMER_act: <only@2>
    :END:
    #+ATTR_latex: :width 11cm   
    [[file:tweet-pipeline-expr.pdf]]

** Hard mental operations
   How hard the user has to think about things other than the
   motivating task

** Bioconductor is intrinsically complex
*** 
    :PROPERTIES:
    :BEAMER_act: <only@1>
    :END:
    #+ATTR_latex: :width 11cm
    [[file:tweet-pipeline-just-bioc.pdf]]

*** 
    :PROPERTIES:
    :BEAMER_act: <only@2>
    :END:
     #+ATTR_latex: :width 11cm   
    [[file:tweet-pipeline-complex.pdf]]

** Language complexity
   * Bioconductor has large, complex APIs
     #+begin_src R
     library(VariantAnnotation)
     length(methods(class="GRanges"))
     #+end_src

     #+RESULTS:
     : [1] 278
     
   * Bioconductor has large, complex class hierarchies
     #+begin_src R
       pkgs <- package_dependencies("rtracklayer",
                                    installed.packages())[[1L]]
       pkgs <- setdiff(pkgs, c("methods", "XML", "RCurl"))
       cl <- unlist(lapply(pkgs,
                    function(p) getClasses(getNamespace(p))))
       length(cl)
     #+end_src

     #+RESULTS:
     : [1] 243

   * In total, 2239 methods on 422 generics

** What needs to improve?
   * Education?
   * Documentation?
   * The software?
   * All of the above?

* HelloRanges
** HelloRanges: an onramp to Bioconductor
   * =bedtools= has a low barrier to entry but lacks the supporting
     ecosystem to cleanly handle realistic workflows
   * We want to teach new users how to perform =bedtools=-style
     operations within R/Bioconductor
   * HelloRanges compiles R code from =bedtools= invocations, so the
     student can learn by:
     * studying the output,
     * integrating it into the workflow,
     * and potentially customizing it
   * Output prompts the user to fill in details like the genome build
   * Supports all =bedtools= operations and arguments
   * Research goal: comparative analysis of =bedtools= and
     Bioconductor

** HelloRanges exposes the complexity of Bioconductor
*** 
    :PROPERTIES:
    :BEAMER_act: <only@1>
    :END:
    #+ATTR_latex: :width 11cm
    [[file:tweet-pipeline-hello-ranges-zero.pdf]]

*** 
    :PROPERTIES:
    :BEAMER_act: <only@2->
    :END:
    #+ATTR_latex: :width 11cm   
    [[file:tweet-pipeline-hello-ranges.pdf]]

*** 
    :PROPERTIES:
    :BEAMER_act: <3>
    :END:

    Data structures required:
    * /Seqinfo/
    * /GAlignments/
    * /GRanges/
    * /RleList/
    * /Pairs/
    
** Lesson learned
   * Better onramps only help to a point
   * Simplifying the software would make everything easier
   * The =bedtools= approach of "everything is a BED file" motivates
     the axiom:

*** Everything is a GRanges (or SummarizedExperiment)
    Consolidating to a small number of data structures enables:
    * comprehension,
    * endomorphism, and thus
    * fluency and chainability
      
* plyranges
** Simplify, but keep the semantics
   # TODO: Einstein picture
   #+begin_quote
   It can scarcely be denied that the supreme goal of
   all theory is to make the irreducible basic elements as simple and
   as few as possible without having to surrender the adequate
   representation of a single datum of experience. -- Albert Einstein
   #+end_quote

   #+begin_quote
   Everything Should Be Made as Simple as Possible, But Not Simpler \\
   -- Apocryphal Einstein quote, paraphasing above
   #+end_quote

** Taking cues from the dplyr package
   * dplyr is a API for tabular data manipulation
   * Inspired by relational algebra, SQL
   * Unified about a single, data model: the tibble
   * Operations are:
     * Cohesive (do a single thing)
     * Endomorphic (return the same type as their input)
     * Verb-oriented in syntax
   * Fluency emerges from chaining of verbs
     
   #+begin_src R
     genes %>%
         group_by(seqnames) %>%
         summarize(count_per_chr=n())
   #+end_src
   
** Goal
   Extend dplyr to genomics, a more complex problem domain, to achieve
   the accessibility of bedtools
   
** plyranges
#+Beamer: \framesubtitle{https://github.com/sa-lee/plyranges}
   * A dplyr-based API for computing on genomic ranges
   * Extending the relational algebra with genomic notions
   * Large set of visible verbs acting only on the core data structures:
     * GRanges :: represents annotated genomic ranges
     * SummarizedExperiment :: coordinates experimental assay data
       with sample and feature annotations
   * Collaboration with \alert{Stuart Lee} and Di Cook @ Monash

** plyranges is simple and expressive
*** 
    :PROPERTIES:
    :BEAMER_act: <only@1>
    :END:
    #+ATTR_latex: :width 11cm
    [[file:tweet-pipeline-plyranges-0.pdf]]

*** 
    :PROPERTIES:
    :BEAMER_act: <only@2>
    :END:
    #+ATTR_latex: :width 11cm
    [[file:tweet-pipeline-plyranges.pdf]]

*** 
    :PROPERTIES:
    :BEAMER_act: <only@3>
    :END:
    #+ATTR_latex: :width 11cm
    [[file:tweet-pipeline-hello-ranges-simple.pdf]]

*** 
    :PROPERTIES:
    :BEAMER_act: <only@4>
    :END:
    #+ATTR_latex: :width 11cm
    [[file:tweet-pipeline-hello-ranges-plyranges.pdf]]

** The ever evolving Bioconductor
*** 
  :PROPERTIES:
  :BEAMER_act: <only@1>
  :END:
  #+ATTR_latex: :width 11cm
   [[file:symphony.pdf]]

*** 
  :PROPERTIES:
  :BEAMER_act: <only@2>
  :END:
  #+ATTR_latex: :width 11cm
   [[file:symphony-emerging.pdf]]
   
       
