%\VignetteIndexEntry{02. Sequences and Ranges}
%\VignettePackage{useR2013}

\section{Sequences and Ranges}
\label{sec:seqandrange}


Many \Bioconductor{} packages are available for analysis of
high-throughput sequence data. This section introduces two essential
ways in which sequence data are manipulated. Sets of DNA strings
represent the reads themselves and the nucleotide sequence of
reference genomes. Ranges describe both aligned reads and features of
interest on the genome. Key packages are summarized in
Table \ref{tab:ranges-and-strings-pkgs}.

\begin{table}
  \centering
  \caption{Selected \Bioconductor{} packages for representing and
    manipulating ranges, strings, and other data structures.}
  \label{tab:ranges-and-strings-pkgs}
  \begin{tabular}{lp{.7\textwidth}}
    Package & Description\\
    \hline

    \Biocpkg{IRanges} & Defines important classes (e.g.,
    \Rclass{IRanges}, \Rclass{Rle}) and methods (e.g.,
    \Rfunction{findOverlaps}, \Rfunction{countOverlaps}) for
    representing and manipulating ranges of consecutive values. Also
    introduces \Rclass{DataFrame}, \Rclass{SimpleList} and other
    classes tailored to representing very large data.\\

    \Biocpkg{GenomicRanges} & Range-based classes tailored to sequence
    representation (e.g., \Rclass{GRanges}, \Rclass{GRangesList}),
    with information about strand and sequence name.\\

    \Biocpkg{GenomicFeatures} & Foundation for manipulating data bases
    of genomic ranges, e.g., representing coordinates and organization
    of exons and transcripts of known genes.\\

    \Biocpkg{Biostrings} & Classes (e.g., \Rclass{DNAStringSet}) and
    methods (e.g., \Rfunction{alphabetFrequency},
    \Rfunction{pairwiseAlignment}) for representing and manipulating
    DNA and other biological sequences.\\

    \Biocpkg{BSgenome} & Representation and manipulation of large
    (e.g., whole-genome) sequences.\\
    \hline
  \end{tabular}
\end{table}

\subsection{\Biocpkg{Biostrings}}

The \Biocpkg{Biostrings} package provides tools for working with DNA
(and other biological) sequence data. The essential data structures
are \Rclass{DNAString} and \Rclass{DNAStringSet}, for working with one
or multiple DNA sequences. The \Biocpkg{Biostrings} package contains
additional classes for representing amino acid and general biological
strings.  The \Biocpkg{BSgenome} and related packages (e.g.,
\Biocannopkg{BSgenome.Dmelanogaster.UCSC.dm3}) are used to represent
whole-genome sequences. Subsequent exercises explore this
packages.

%% \fixme{Exercise: trimLRPattern}

\subsection{\Biocpkg{GenomicRanges}}

Next-generation sequencing data consists of a large number of short
reads. These are, typically, aligned to a reference genome. Basic
operations are performed on the alignment, asking e.g., how many reads
are aligned in a genomic range defined by nucleotide coordinates
(e.g., in the exons of a gene), or how many nucleotides from all the
aligned reads cover a set of genomic coordinates. How is this type of
data, the aligned reads and the reference genome, to be represented in
\R{} in a way that allows for effective computation?

The \Biocpkg{IRanges}, \Biocpkg{GenomicRanges}, and
\Biocpkg{GenomicFeatures} \Bioconductor{} packages provide the
essential infrastructure for these operations; we start with the
\Rclass{GRanges} class, defined in \Biocpkg{GenomicRanges}.

\paragraph{\Rclass{GRanges}}

Instances of \Rclass{GRanges} are used to specify genomic
coordinates. Suppose we wish to represent two \Dmel{} genes. The
first is located on the positive strand of chromosome 3R, from
position 19967117 to 19973212. The second is on the minus strand of
the X chromosome, with `left-most' base at 18962306, and right-most
base at 18962925. The coordinates are \emph{1-based} (i.e., the first
nucleotide on a chromosome is numbered 1, rather than 0),
\emph{left-most} (i.e., reads on the minus strand are defined to
`start' at the left-most coordinate, rather than the 5' coordinate),
and \emph{closed} (the start and end coordinates are included in the
range; a range with identical start and end coordinates has width 1, a
0-width range is represented by the special construct where the end
coordinate is one less than the start coordinate).

A complete definition of these genes as \Rclass{GRanges} is:
\begin{Schunk}
\begin{Sinput}
> genes <- GRanges(seqnames=c("3R", "X"),
+                  ranges=IRanges(
+                      start=c(19967117, 18962306),
+                      end=c(19973212, 18962925)),
+                  strand=c("+", "-"),
+                  seqlengths=c(`3R`=27905053L, `X`=22422827L))
\end{Sinput}
\end{Schunk}
The components of a \Rclass{GRanges} object are defined as vectors,
e.g., of \Rcode{seqnames}, much as one would define a
\Rclass{data.frame}. The start and end coordinates are grouped into an
\Rclass{IRanges} instance. The optional \Rcode{seqlengths} argument
specifies the maximum size of each sequence, in this case the lengths
of chromosomes 3R and X in the `dm2' build of the \Dmel{} genome. This
data is displayed as
\begin{Schunk}
\begin{Sinput}
> genes
\end{Sinput}
\begin{Soutput}
GRanges with 2 ranges and 0 metadata columns:
      seqnames               ranges strand
         <Rle>            <IRanges>  <Rle>
  [1]       3R [19967117, 19973212]      +
  [2]        X [18962306, 18962925]      -
  ---
  seqlengths:
         3R        X
   27905053 22422827
\end{Soutput}
\end{Schunk}
For the curious, the gene coordinates and sequence lengths are derived
from the \Biocannopkg{org.Dm.eg.db} package for genes with Flybase
identifiers FBgn0039155 and FBgn0085359, using the annotation
facilities described in section \ref{sec:annotation}.

The \Rclass{GRanges} class has many useful methods defined on
it. Consult the help page
\begin{Schunk}
\begin{Sinput}
> ?GRanges
\end{Sinput}
\end{Schunk}
and package vignettes (especially `An Introduction to
\Biocpkg{GenomicRanges}')
\begin{Schunk}
\begin{Sinput}
> vignette(package="GenomicRanges")
\end{Sinput}
\end{Schunk}
for a comprehensive introduction.

\paragraph{Operations on ranges}

The \Rclass{GRanges} class has many useful methods from the
\Rclass{IRanges} class; some of these methods are illustrated here. We
use \Rclass{IRanges} to illustrate these operations to avoid
complexities associated with strand and seqname, but the operations
are comparable on \Rclass{GRanges}. We begin with a simple set of
ranges:
\begin{Schunk}
\begin{Sinput}
> ir <- IRanges(start=c(7, 9, 12, 14, 22:24),
+               end=c(15, 11, 12, 18, 26, 27, 28))
\end{Sinput}
\end{Schunk}
%%
%
These and some common operations are illustrated in the upper panel of
Figure \ref{fig:ranges} and summarized in Table \ref{tab:range-ops}.

\begin{figure}
\centering
\includegraphics[width=.5\textwidth,height=!]{ranges-ir-plot}\\
\includegraphics[width=.5\textwidth,height=!]{ranges-shift-ir-plot}\\
\includegraphics[width=.5\textwidth,height=!]{ranges-reduce-ir-plot}
\caption{Ranges}
\label{fig:ranges}
\end{figure}

\begin{table}
  \centering
  \caption{Common operations on \Rclass{IRanges}, \Rclass{GRanges} and
    \Rclass{GRangesList}.}
  \label{tab:range-ops}
  \begin{tabular}{lll}
    Category & Function & Description \\
    \hline

    Accessors & \Rfunction{start}, \Rfunction{end}, \Rfunction{width} & Get or s et the starts, ends and widths \\
    & \Rfunction{names} & Get or set the names \\
    & \Rfunction{mcols}, \Rfunction{metadata} & Get or set metadata on elements or object \\
    & \Rfunction{length} & Number of ranges in the vector \\
    & \Rfunction{range} & Range formed from min start and max end \\

    Ordering & \Rfunction{<}, \Rfunction{<=}, \Rfunction{>}, \Rfunction{>=}, \Rfunction{==}, \Rfunction{!=} & Compare ranges, ordering by start then width \\
    & \Rfunction{sort}, \Rfunction{order}, \Rfunction{rank} & Sort by the ordering \\
    & \Rfunction{duplicated} & Find ranges with multiple instances \\
    & \Rfunction{unique} & Find unique instances, removing duplicates \\

    Arithmetic & \Rcode{r + x}, \Rcode{r - x}, \Rcode{r * x} & Shrink or expand ranges \Robject{r} by number \Robject{x} \\
    & \Rfunction{shift} & Move the ranges by specified amount \\
    & \Rfunction{resize} & Change width, ancoring on start, end or mid \\
    & \Rfunction{distance} & Separation between ranges (closest endpoints) \\
    & \Rfunction{restrict} & Clamp ranges to within some start and end \\
    & \Rfunction{flank} & Generate adjacent regions on start or end \\

    Set operations & \Rfunction{reduce} & Merge overlapping and adjacent ranges \\
    & \Rfunction{intersect}, \Rfunction{union}, \Rfunction{setdiff} & Set operations on reduced ranges \\
    & \Rfunction{pintersect}, \Rfunction{punion}, \Rfunction{psetdiff} & Parallel set operations, on each \Rcode{x[i]}, \Rcode{y[i]} \\
    & \Rfunction{gaps}, \Rfunction{pgap} & Find regions not covered by reduced ranges \\
    & \Rfunction{disjoin} & Ranges formed from union of endpoints \\

    Overlaps & \Rfunction{findOverlaps} & Find all overlaps for each \Robject{x} in \Robject{y} \\
    & \Rfunction{countOverlaps} & Count overlaps of each \Robject{x} range in \Robject{y} \\
    & \Rfunction{nearest} & Find nearest neighbors (closest endpoints) \\
    & \Rfunction{precede}, \Rfunction{follow} & Find nearest \Robject{y} that \Robject{x} precedes or follows \\
    & \Rcode{x \%in\% y} & Find ranges in \Robject{x} that overlap range in \Robject{y} \\

    Coverage & \Rfunction{coverage} & Count ranges covering each position \\

    Extraction & \Rcode{r[i]} & Get or set by logical or numeric index \\
    & \Rcode{r[[i]]} & Get integer sequence from \Rcode{start[i]} to \Rcode{end[i]} \\
    & \Rfunction{subsetByOverlaps} & Subset \Robject{x} for those that overlap in \Robject{y} \\
    & \Rfunction{head}, \Rfunction{tail}, \Rfunction{rev}, \Rfunction{rep} & Conventional R semantics \\

    Split, combine & \Rfunction{split} & Split ranges by a factor into a \Rclass{RangesList} \\
    & \Rfunction{c} & Concatenate two or more range objects \\
    \hline
  \end{tabular}
\end{table}

\paragraph{\Rfunction{mcols}}

The \Rclass{GRanges} class (actually, most of the data structures
defined or extending those in the \Biocpkg{IRanges} package) has two
additional very useful data components. The \Rfunction{mcols} function
allows information on each range to be stored and manipulated (e.g.,
subset) along with the \Rclass{GRanges} instance. The element metadata
is represented as a \Rclass{DataFrame}, defined in \Biocpkg{IRanges}
and acting like a standard \R{} \Rclass{data.frame} but with the
ability to hold more complicated data structures as columns (and with
element metadata of its own, providing an enhanced alternative to the
\Biocpkg{Biobase} class \Rclass{AnnotatedDataFrame}).
\begin{Schunk}
\begin{Sinput}
> mcols(genes) <- DataFrame(EntrezId=c("42865", "2768869"),
+                           Symbol=c("kal-1", "CG34330"))
\end{Sinput}
\end{Schunk}
\Rfunction{metadata} allows addition of information to the entire
object. The information is in the form of a list; any data can be
provided.
\begin{Schunk}
\begin{Sinput}
> metadata(genes) <-
+     list(CreatedBy="A. User", Date=date())
\end{Sinput}
\end{Schunk}

\paragraph{\Rclass{GRangesList}}

The \Rfunction{GRanges} class is extremely useful for representing
simple ranges. Some next-generation sequence data and genomic features
are more hierarchically structured. A gene may be represented by several
exons within it. An aligned read may be represented by discontinuous
ranges of alignment to a reference.  The \Rclass{GRangesList} class
represents this type of information. It is a list-like data structure,
which each element of the list itself a \Rclass{GRanges} instance. The
gene FBgn0039155 contains several exons, and can be represented as a
list of length 1, where the element of the list contains a
\Rclass{GRanges} object with 7 elements:
\begin{Schunk}
\begin{Soutput}
GRangesList of length 1:
$FBgn0039155 
GRanges with 7 ranges and 2 metadata columns:
      seqnames               ranges strand |   exon_id   exon_name
         <Rle>            <IRanges>  <Rle> | <integer> <character>
  [1]    chr3R [19967117, 19967382]      + |     45515        <NA>
  [2]    chr3R [19970915, 19971592]      + |     45516        <NA>
  [3]    chr3R [19971652, 19971770]      + |     45517        <NA>
  [4]    chr3R [19971831, 19972024]      + |     45518        <NA>
  [5]    chr3R [19972088, 19972461]      + |     45519        <NA>
  [6]    chr3R [19972523, 19972589]      + |     45520        <NA>
  [7]    chr3R [19972918, 19973212]      + |     45521        <NA>

---
seqlengths:
    chr3R
 27905053
\end{Soutput}
\end{Schunk}

\paragraph{The \Biocpkg{GenomicFeatures} package}

Many public resources provide annotations about genomic features. For
instance, the UCSC genome browser maintains the `knownGene' track of
established exons, transcripts, and coding sequences of many model
organisms. The \Biocpkg{GenomicFeatures} package provides a way to
retrieve, save, and query these resources. The underlying
representation is as sqlite data bases, but the data are available in
\R{} as \Rclass{GRangesList} objects. The following exercise explores
the \Biocpkg{GenomicFeatures} package and some of the functionality
for the \Rclass{IRanges} family introduced above.

\begin{Exercise}
  Load the \Biocannopkg{TxDb.Dmelanogaster.UCSC.dm3.ensGene}
  annotation package, and create an alias \Robject{txdb} pointing to
  the \Rclass{TranscriptDb} object this class defines.

  Extract all exon coordinates, organized by gene, using
  \Rfunction{exonsBy}. What is the class of this object? How many
  elements are in the object? What does each element correspond to?
  And the elements of each element? Use \Rfunction{elementLengths} and
  \Rfunction{table} to summarize the number of exons in each gene, for
  instance, how many single-exon genes are there?

  Select just those elements corresponding to flybase gene ids
  FBgn0002183, FBgn0003360, FBgn0025111, and FBgn0036449. Use
  \Rfunction{reduce} to simplify gene models, so that exons that
  overlap are considered `the same'.
\end{Exercise}

\begin{Solution}
\begin{Schunk}
\begin{Sinput}
> library(TxDb.Dmelanogaster.UCSC.dm3.ensGene)
> txdb <- TxDb.Dmelanogaster.UCSC.dm3.ensGene # alias
> ex0 <- exonsBy(txdb, "gene")
> head(table(elementLengths(ex0)))
\end{Sinput}
\begin{Soutput}
   1    2    3    4    5    6 
3182 2608 2070 1628 1133  886 
\end{Soutput}
\begin{Sinput}
> ids <- c("FBgn0002183", "FBgn0003360", "FBgn0025111", "FBgn0036449")
> ex <- reduce(ex0[ids])
\end{Sinput}
\end{Schunk}
\end{Solution}

\begin{Exercise}
  The objective of this exercise is to calculate the GC content of the
  exons of a single gene, whose coordinates are specified by the
  \Robject{ex} object of the previous exercise.

  Load the \Biocannopkg{BSgenome.Dmelanogaster.UCSC.dm3} data package,
  containing the UCSC representation of \Dmel{} genome assembly dm3.

  Extract the sequence name of the first gene of \Robject{ex}. Use
  this to load the appropriate \Dmel{} chromosome.

  Use \Rfunction{Views} to create views on to the chromosome that span
  the start and end coordinates of all exons.

  The \Rpackage{useR2013} package defines a helper function
  \Rfunction{gcFunction} to calculate GC content. Use this to
  calculate the GC content in each of the exons.

  Look at the helper function, and describe what it does.
\end{Exercise}

\begin{Solution}
  Here we load the \Dmel{} genome, select a single chromosome, and
  create \Rfunction{Views} that reflect the ranges of the
  \texttt{FBgn0002183}.
\begin{Schunk}
\begin{Sinput}
> library(BSgenome.Dmelanogaster.UCSC.dm3)
> nm <- as.character(unique(seqnames(ex[[1]])))
> chr <- Dmelanogaster[[nm]]
> v <- Views(chr, start=start(ex[[1]]), end=end(ex[[1]]))
\end{Sinput}
\end{Schunk}
%% 
Using the \Rfunction{gcFunction} helper function, the subject GC
content is
\begin{Schunk}
\begin{Sinput}
> gcFunction(v)
\end{Sinput}
\begin{Soutput}
[1] 0.4767442 0.5555556 0.5389610 0.5513308 0.5351788 0.5441426 0.4933333 0.5189394
[9] 0.5110132
\end{Soutput}
\end{Schunk}
%
The \Rfunction{gcFunction} is really straight-forward: it invokes the
function \Rfunction{alphabetFrequency} from the \Biocpkg{Biostrings}
package. This returns a simple matrix of exon $\times$ nuclotiede
probabilities. The row sums of the \texttt{G} and \texttt{C} columns
of this matrix are the GC contents of each exon.
\begin{Schunk}
\begin{Sinput}
> gcFunction
\end{Sinput}
\begin{Soutput}
function (x) 
{
    alf <- alphabetFrequency(x, as.prob = TRUE)
    rowSums(alf[, c("G", "C")])
}
<environment: namespace:useR2013>
\end{Soutput}
\end{Schunk}
\end{Solution}

\subsection{Resources}

There are extensive vignettes for \Biocpkg{Biostrings} and
\Biocpkg{GenomicRanges} packages. A useful online resource is from
\href{http://manuals.bioinformatics.ucr.edu/home/ht-seq}{Thomas
  Grike}'s group.

