$Id: README,v 1.17 2009/04/05 13:25:43 jt Exp $

Abstract:
===================================================================
The Cygwin rebase distribution contains four utilities, rebase, rebaseall,
peflags, and peflagsall.  The first utility is modeled after Microsoft's SDK
rebase.  However, instead of linking rebase against Microsoft's imagehlp
library, it is linked against Ralf Habacker's imagehelper library.

The following are the advantages of using Ralf's library:

    1. rebase can be a Cygwin application.  If rebase is linked against
       Microsoft's imagehlp library, then it must be a Mingw application.
       This is because imagehlp is dependent on Microsoft's C runtime and
       therefore cannot be used in an application that is dependent on
       another C runtime (i.e., Cygwin).

    2. rebase can be used on Windows Me.  For some reason, Microsoft's
       imagehlp.dll does not function properly on this platform.

    3. Ralf's imagehelper library also provides FixImage() which fixes
       bad relocations in DLLs that can be caused by stripping.

I would like to thank Ralf Habacker for providing the imagehelper
library.  This library has enabled me to create a rebase utility that
is usable by all Cygwin user (regardless of platform) on all DLLs
(regardless of stripping).

I would like to thank Chuck Wilson for providing peflags and peflagsall.  This
enables Cygwin to take advantage of the Address Space Layout Randomization
facility on Windows Vista and later.


Background
===================================================================
The rebaseall utility is a convenient way for users that suffer from the
Cygwin rebase problem to rebase their entire system (i.e., all of their
DLLs).  The following is a list of known Cygwin applications that are
affected by the rebase problem:

    Apache
    Perl
    Python

The rebase problem is due to fork() failing when it is unable to load
DLLs in the child at the same address as in the parent.  This is caused
by DLLs which have conflicting base addresses.  An error message like
the following will be displayed when the problem is triggered:

    C:\cygwin\bin\python.exe: *** unable to remap C:\cygwin\bin\cygssl.dll to same address as parent(0xDF0000) != 0xE00000

Note that rebaseall is only a stop-gap measure.  Eventually the rebase
functionality will be added to Cygwin's setup.exe, so that rebasing will
happen automatically.

On Vista and newer, it is possible to employ the Address Space Layout
Randomization facility to help solve the image load address clashing
problem:

    Since Windows relies on relocations instead of position
    independent code, a DLL must be loaded at the same address
    in each process that uses it to allow the physical memory
    used by the DLL to be shared. To facilitate this behaviour,
    a global bitmap called _MiImageBitMap is used to represent
    the address space from 0x50000000 to 0x78000000. The bitmap
    is 0x2800 bits in length with each bit representing 64KB
    of memory. As each DLL is loaded, its position is recorded
    by setting the appropriate bits in the bitmap to mark the
    memory where the DLL is being mapped. When the same DLL
    is loaded in another process, its section object is reused
    and it is mapped at the same virtual addresses.
    http://taossa.com.nyud.net:8080/archive/bh08sotirovdowd.pdf

Thus, IF all cygwin dlls (except cygwin1.dll) -- or, at least the
ones for which you often get the *** unable to remap *** error --
are marked ASLR-compatible, then the Windows runtime loader can be
coerced into ensuring that the DLLs are loaded at the same memory
location for all concurrent processes. After using the peflagsall
utility to mark the dlls appropriately, you may need to reboot to
cause Windows to take notice.


Requirements:
===================================================================
The following packages or later are required to build and/or execute
Cygwin rebase and/or rebaseall:

    ash 20040127-3
    binutils 20060709-1
    cygwin 1.5.20-1
    fileutils 4.1-3
    gcc 3.4.4-1
    grep 2.5.1a-2
    gzip 1.3.5-2
    sed 4.1.5-1
    w32api 3.13-1


Install:
===================================================================
Cygwin rebase does not require any special installation procedures.


Usage:
===================================================================
Use the following procedure to rebase your entire system:

    1. shutdown all Cygwin processes and services
    2. start ash (do not use bash or rxvt)
    3. execute /bin/rebaseall (in the ash window)

If you get any errors due to DLLs being in-use or read-only, then take the
appropriate action and rerun rebaseall.  Otherwise, you run the risk of fork()
failing.

Use a similar procedure to set the peflags for all cygwin executables and
dlls on your entire system:

    1. shutdown all Cygwin processes and services
    2. start ash (do not use bash or rxvt)
    3. execute /bin/peflagsall (in the ash window)

Both procedures may be combined into a single, 4-step procedure if desired.

Note that by default, peflagsall will set the tsaware flag on all executables
(ash.exe and peflags.exe themselves excluded), and will set the dynamicbase
flag on all 'dll' and 'so' files (cygwin1.dll and cyglsa64.dll excluded).

If you are used to using the -s option with rebaseall, to add additional
suffixes to the rebase list (such as .oct), be warned that peflags behaves
differently. If you specify custom suffixes, you must restate the default
ones, AND explicitly specify what action to perform; peflagsall disables
all default behavior when custom settings are used.  For instance:

    peflagsall -s oct -s dll -s so -s exe   -d 1 -t 1
               ^^^^^^ ^^^^^^^^^^^^^^^^^^^   ^^^^^^^^^
               custom   restate default      restate
               suffix      suffixes      default actions

Whereas rebaseall only has a single action, and new suffixes are added
to the existing default set of suffixes. See 'Command line syntax', in 
the next section.

Command line syntax
===================================================================

rebaseall
-------------------------------------------------------------------
The following is the rebaseall command line syntax:

    rebaseall [-b BaseAddress] [-o Offset] [-T FileList | -] [-v]

where:

    -b => base address used by rebase (default: 0x70000000)
    -o => offset between each DLL rebased (default: 0x10000)
    -s => specify DLL suffix, use multiple if necessary (default: dll, so)
    -T => specify filelist (or stdin) to list additional files
    -v => verbose (default: off)

peflagsall
-------------------------------------------------------------------
The following is the peflagsall command line syntax:

    peflagsall [-p extra_args] [-d bool] [-t bool] [-s suffix] [-T FileList | -] [-vnh]

    When invoked with no arguments, peflagsall modifies every cygwin exe|dll|so
    on the system: .exe files have their tsaware flag set, while .dll and .so files
    have their dynamicbase flag set. However, if any of [-d|-t|-s] are specified
    then ONLY the actions so specified will occur.
       -p extra_args   pass extra_args to peflags.exe
       -d bool         set the dynamicbase flag to 'bool' on all specified files
       -t bool         set the tsaware flag to 'bool' on all specified files
       -s suffix       search for all files with the specified suffix(es)
                         default: 'exe|dll|so'
       -T FileList     in addition to files located via the normal search and
                       suffix list, also operate on files listed in FileList
                       May be '-' which indicates stdin
       -n              do not modify any files, but display the peflags commands
       -k              keep all temporary files
       -v              verbose mode
       -h              show this help
    bool may be '0', '1', 'true', 'false', 'yes', or 'no'
    Note: peflagsall will NOT set the dynamicbase flag on executables, nor will
          it set the tsaware flag on dlls. If you must do this, use peflags itself

rebase
-------------------------------------------------------------------
The following is the rebase command line syntax:

    rebase -b BaseAddress [-Vdv] [-o Offset] [-T FileList | -] Files...

where:

    -b => base address used by rebase
    -d => rebase down memory (default: up)
    -o => offset between each DLL rebased
    -T => specify filelist (or stdin) to list additional files
    -V => display version and exit
    -v => verbose (default: off)

peflags
-------------------------------------------------------------------
The following is the peflags command line syntax:

    peflags [OPTIONS] file(s)...
    Sets, clears, or displays various flags in PE files (that is,
    exes and dlls).  For each flag, if an argument is given, then
    the specified flag will be set or cleared; if no argument is
    given, then the current value of that flag will be displayed.

      -d, --dynamicbase  [BOOL]   Image base address may be relocated using
                                  address space layout randomization (ASLR)
          --forceinteg   [BOOL]   Code integrity checks are enforced
      -n, --nxcompat     [BOOL]   Image is compatible with data execution
                                  prevention
          --no-isolation [BOOL]   Image understands isolation but do not
                                  isolate the image
          --no-seh       [BOOL]   Image does not use SEH. No SE handler may
                                  be called in this image
          --no-bind      [BOOL]   Do not bind this image
          --wdmdriver    [BOOL]   Driver uses the WDM model
      -t, --tsaware      [BOOL]   Image is Terminal Server aware
          --wstrim       [BOOL]   Aggressively trim the working set.
          --bigaddr      [BOOL]   The application can handle addresses
                                  larger than 2 GB
          --sepdbg       [BOOL]   Debugging information was removed and
                                  stored separately in another file.
      -T, --filelist FILE         Indicate that FILE contains a list
                                  of PE files to process
      -v, --verbose               Display diagnostic information
      -V, --version               Display version information
      -h, --help                  Display this help
    
    BOOL: may be 1, true, or yes - indicates that the flag should be set
              if 0, false, or no - indicates that the flag should be cleared
              if not present, then display symbolicly the value of the flag
    Valid forms for short options: -d, -d0, -d1, -dfalse, etc
    Valid forms for long options :  --tsaware, --tsaware=true, etc
    To set a value, and display the results symbolicly, repeat the option:
      --tsaware=true --tsaware -d0 -d

    
Source:
===================================================================
Cygwin rebase builds OOTB under Cygwin.  With minor Makefile tweaking,
it builds cleanly under Mingw too.


Build:
===================================================================
See build.sh in the source archive for my exact build recipe for configuring,
making, and packaging this distribution.


Test:
===================================================================
rebase does not contain any regression tests.

peflagsall may be invoked with the -n, -k, and -v options to 
allow inspection of what it WOULD do, without actually doing it.

peflags will make no changes to any files unless it is given an
flagname argument *with* a boolean value for that flag (0 or 1).
If no boolean argument is provided, then peflags will merely
display the current value of the flag in the specified files.


Issues:
===================================================================
The following are the known Cygwin rebase issues:

1. rebase should be integrated with Cygwin's setup.exe.

2. rebase does not handle in-use DLLs.

3. rebase does not currently store the DLL base addresses in a database
which would speed up and/or obviate subsequent rebases.

4. rebase skips read-only DLLs.

Issues #1, #2, and #4 also apply to the peflags utility.


Homepage:
===================================================================
The primary rebase web site is:

    http://www.tishler.net/jason/software/rebase/


Download:
===================================================================
The primary rebase download site is:

    http://www.tishler.net/jason/software/rebase/


Mailing Lists:
===================================================================
Please report problems, suggestions, etc. to <cygwin@cygwin.com>.


Maintainer:
===================================================================
    Jason Tishler <jason@tishler.net>
