The configure script chooses to install bali-phy in the directory /usr/local/ by default. You can install executables to another directory dir by passing --prefix=dir. For example, in order to install BAli-Phy under ~/local, you can enter:

% ../bali-phy-2.1.0/configure --prefix=$HOME/local

This is recommended if you do not have permission to install to /usr/local/.

You can instruct the compiler to look for include files in directory dir by passing --with-extra-includes=dir to the configure script.

You can instruct the compiler to look for libraries files in directory dir by passing --with-extra-libs=dir to the configure script.

For example, if your system has GSL installed in /usr/local/, then you might need to add "--with-extra-includes=/usr/local/include --with-extra-libs=/usr/local/lib" to the configure script arguments so that the compiler can find the GSL include files and libraries.

The default C++ compiler is g++. On some systems, g++ invokes GCC version 3.3, and the correct compiler is called something else, such as g++-4.2. To use g++-4.2 as the C++ compiler when compiling BAli-Phy, you would set the CXX environment variable as follows:

% ../bali-phy-2.1.0/configure CXX=g++-4.2

You can specify optimizing for a specific brand of CPU, by specifying the CHIP variable to configure, as follows:

% ../bali-phy-2.1.0/configure CHIP=cpu

You can set CHIP to any of pentium3, pentium4, nocona, core2, G3, G4, or G5. (On recent versions of GCC, you can set CHIP=native to auto-detect the type of CPU you have.) This may produce faster executables, but at the cost of producing executables that may not run on a different kind of chip.

Call configure with the flag --enable-static to build static executables. Static executables will be able to run on other computers with the with the same type of CPU but slightly different versions of the operating system.

All these options to configure can be combined, as follows:

% ../bali-phy-2.1.0/configure --prefix=$HOME/local --enable-static CXX=g++-4.5 CHIP=pentium4

This example uses g++-4.5 to build a pentium4-optimized version of bali-phy with static linkage.

If you installed BAli-Phy to the directory /usr/local/, then you can run bali-phy by typing /usr/local/bin/bali-phy. However, it would be much nicer to simply type bali-phy and let the computer find the executable for you. This can be achieved by putting the directory that contains the bali-phy executables into your "path". The "path" is a colon-separated list of directories that is searched to find program names that you type. It is stored in a variable called PATH.

Setting your PATH is also a pre-requisite for running the bp-analyze.pl script to summarize your runs.

You can examine the current value of this variable by typing:

% echo $PATH

We will assume that you extracted the bali-phy archive in /usr/local and so you want to add /usr/local/bin to your PATH. (If you installed to another directory, replace /usr/local/ with that directory.)

The commands for doing this depend on what "shell" you are using. Type echo $SHELL to find out. If your shell is sh or bash then the command looks like this:

% PATH=/usr/local/bin:$PATH

If your shell is csh or tcsh, then the command looks like this:

% setenv PATH /usr/local/bin:$PATH

Note that these commands will only the window you are typing in, and will vanish when you reboot.

To make this change survive reboots, you can copy the command into a configuration file using a text editor. To find the right configuration file, look in your $HOME directory for .profile (for the Bourne shell sh), .bash_profile (for BASH), or .login (for tcsh). You may have to create the file if it is not present.

If you do not know which directory is your home directory, you can find its full name by typing:

% echo $HOME

Option files use the same option names as the command line. However, the syntax is different: each option is given on its own line using the syntax "option = value" instead of the syntax "--option value". If the option has no value then it is given using the syntax "option = option".

For example, consider the following option file:

#select a data set to analyze
align = examples/EF-Tu/5d.fasta

#select an substitution model
smodel = log-normal+INV

#fix the alignment and do not model indels
traditional = traditional
	  

The first option, align, is the name of the sequence file, which has no name on the command line. Lines that begin with # are comments, and blank lines are ignored. The option --traditional uses the option name as the value, because it does not take a value. Thus, this configuration file corresponds to the command line

%bali-phy examples/EF-Tu/5d.fasta --smodel log-normal+INV --traditional

The file ~/.bali-phy is a special option file called the configuration file. If it exists, it is always loaded. Options given on the command line or an option file override values given in ~/.bali-phy.

BAli-Phy is quite CPU intensive, and so we recommend using 50 or fewer taxa in order to limit the time required to accumulate enough MCMC samples. (Despite this recommendation, data sets with more than 100 taxa have occasionally been known to converge.) We recommend initially pruning as many taxa as possible from your data set, then adding some back if the MCMC is not too slow.

Aligning just a pair of sequences takes O(L 2 ) time and memory, where L represents the sequence length. Therefore sequences longer than (say) 1000 letters become increasingly impractical. However, you might try to see how long you can make your sequences before you run out of memory, or the program becomes too slow.

For multi-gene analyses, two separate data partitions (i.e. genes) of 500 letters will be twice as fast to align as one data partition of 1000 letters. So, it may be possible to analyze several genes as long as each gene individually is not too long.

You can speed up alignment for long genes by specifying alignment constraints (See Section 7.4, “Alignment constraints”). Ideally, 10 evenly spaced constraints should reduce the cost of re-aligning a sequence by a factor of 10.

Also, note that you can sometimes speed up the analysis of protein sequences by coding them as amino acids or codons, rather than nucleotides. This is because it decreases the sequence length.

This section explains the meaning of the various field names in the file C1.p.

To compute the majority consensus tree, do the following. (The program FigTree allows you to view the resulting tree file graphically.)

% trees-consensus C1.trees > c50.PP.tree

You can (and should) pool results from different MCMC runs by adding multiple tree sample files on the command line. The different MCMC runs should have the same input files and parameters.

% trees-consensus dir1/C1.trees dir2/C1.trees > c50.PP.tree

By default, the first 10% of tree samples are skipped as burn-in. You can specify the number of samples (e.g. 1000) to skip by adding the options -s1000 or --skip=1000. You may also specify a percentage of all samples:

% trees-consensus -s20% C1.trees > c50.PP.tree

To discard some samples, keeping (say) every 10th sample, you may add the options -x10 or --sub-sample=10. This can make the program a lot faster, at the possible expense of some loss in accuracy.

% trees-consensus -s20% -x10 C1.trees > c50.PP.tree

By default, splits are included in the consensus tree if they have a PP greater than 0.5. You can specify a more stringent level (e.g. 0.66) by adding the option --consensus-PP=0.66 as follows:

% trees-consensus -s20% -x10 --consensus-PP=0.66 C1.trees > c66.PP.tree

You may also make the program write directly to the output file (e.g. c66.PP.tree) by using the more general form --consensus-PP=0.66:c66.PP.tree. Leaving off the ":c66.PP.tree" part (as we did above) or specifying ":-" sends the output to the standard output (e.g. the terminal, if not redirected).

% trees-consensus -s20% -x10 C1.trees --consensus-PP=0.66:c66.PP.tree

You can supply multiple levels and filenames separated by commas. This is faster than running the program multiple times with different consensus levels.

% trees-consensus -s20% -x10 C1.trees --consensus-PP=0.5:c50.PP.tree,0.66:c66.PP.tree

Finally, you may use the option --consensus= instead of the option --consensus-PP= if you do not wish the resulting tree to contain embedded posterior probabilities on branches, as well as branch lengths.

% trees-consensus -s20% -x10 C1.trees --consensus=0.5:c50.PP.tree,0.66:c66.PP.tree

Both the --consensus= and --consensus-PP= options may be given simultaneously.

See trees-consensus --help for a complete list of options.

To compute the maximum a posteriori tree topology do:

% trees-consensus --skip=burnin C1.trees --map-tree=MAP.tree

The MAP topology may be used instead of a consensus tree when a fully resolved (e.g. bifurcating) tree is required. However, when the topology has many tips, each topology may be sampled only once, leading to low quality estimates of the MAP topology.

The program FigTree allows you to view the consensus tree graphically.

% trees-bootstrap dir1/C1.trees dir2/C1.trees

This command computes the effective sample size for the posterior probability of each split. It also computes the Average Standard Deviation of Split Frequencies (ASDSF) between two or more independent runs.

See Section 8, “Convergence and Mixing: Is it done yet?” for more information.

This command gives a median and confidence interval, ESS, and a stabilization time:

% statreport C1.p > Report 

This command compares multiple runs to give PSRF and joint ESS values as well:

% statreport C1.p C2.p > Report 

The program Tracer allows you to view the same summaries graphically.

See Section 8, “Convergence and Mixing: Is it done yet?” for more information.

% cut-range --skip=burn-in < C1.Pp.fastas | alignment-max > Pp-max.fasta

You can use the program seaview to view the alignment graphically.

% alignment-find < C1.MAP > P1-MAP.fasta

To annotate a specific alignment alignment.fasta, choose a fully resolved tree estimate tree:

% cut-range --skip=burn-in < C1.Pp.fastas | alignment-gild alignment.fasta tree  > alignment-AU.prob
% alignment-draw alignment.fasta --AU alignment-AU.prob > alignment-AU.html

The majority consensus tree is usually not fully resolved, so we recommend using the MAP topology instead.

The Results/ directory will contain the following useful files:

The following files will be generated to summarize alignment uncertainty, unless the analysis uses a fixed alignment.

The following files describe convergence and mixing:

The SRQ plots can be viewed by typing "plot 'file' with lines" in gnuplot.

This file reports the quality of estimates of support for each partition in terms of the posterior probability (PP) and log-10 odds (LOD). It also reports the auto-correlation time (ACT), the effective sample size (Ne), the number of samples that support (1) or do not support (0) the partition, and the number of regenerations. Only partitions with PP > 0.1 are shown by default.

If the substitution model is not specified, then the default model for the alphabet is used. For DNA or RNA, the default model is TN. For Triplets, the default is TNx3. For Codons, the default model is M0. For Amino-Acids, the default model is LG.

The basic substitution models in BAli-Phy are continuous-time Markov chains (CTMC). CTMC models can be characterized by transition rates Q ij from letter i to letter j. After a given time t the probability for transition from state i to state j is given by

using a matrix exponential. Becase the CTMC models used in BAli-Phy are all reversible, the rate matrix for these reversible models can be decomposed into a symmetric matrix S and equilibrium frequencies π as follows:

The matrix S is called the exchangability matrix, and represents how exchangeable letters i and j are, independent of their frequencies.

The basic CTMC models are EQU, HKY, TN, GTR, HKYx3, TNx3, GTRx3, JTT, WAG, LG, and M0. Each of these models is a way of specifying the exchangeability matrix S ij.

The rate matrix Q ij can be more generally expressed as

where f ranges from 0 to 1 . Here the parameter f specifies the relative importance of unequal conservation (f=0 ) and unequal replacement (f=1 ) in maintaining the equilibrium frequencies π.

In fact, this can be generalized even further to

where

These models can therefore be expressed as a combination of an "exchange model" (for S) and a "frequency model" (for R).

Complex substitution models in BAli-Phy are constructed as mixtures of reversible CTMC models (see Section 7.1, “Substitution models”) that run at different rates (e.g. Γ 4 +INV) or have different parameters (e.g. an M2 codon model).

Model modifiers are gamma, log-normal, INV, M2, M3, and M7.

Example: --smodel WAG+F+log-normal+INV

Example: --smodel WAG+log-normal+INV (same as above)

Example: --smodel EQU --alphabet Triplets

Example: --smodel HKY

Example: --smodel TN+F=constant

Example: --smodel M0 --alphabet Codons

Example: --smodel M0+F=nucleotides --alphabet Codons

Example: --smodel M2 --alphabet Codons

Example: --smodel M0[HKY]+M2 --alphabet Codons (same as above)

Example: --smodel M0[TN]+M2 --alphabet Codons

The first line of the file is a header consisting of an ordered list of sequence names separated by spaces. Each subsequent line consists of a space-separated list of sequence positions, with the first position corresponding to the first leaf sequence, the second position corresponding to the second leaf sequence, etc. Thus, if there are n leaf taxa, then each line corresponds to a space-separated list of n integers.

For example, the file

A B C
1 2 2

implies that position 1 of leaf sequence A is aligned to position 2 of leaf sequences B and C. Note that the first position in a sequence is position 0.

Optionally, one may use a '-' instead of an integer, which denotes a lack of constraint for that sequence. This can be useful as follows:

A B C D
2 2 - - 
- - 2 2
	

The above constraints force alignment between position 2 of sequences A and B, and between position 2 of sequence C and D.

The program alignment-indices may be used to aid in computing a constraint file from an input alignment. See Q: 11.6.3.

To calculate the ASDSF and MSDSF run:

% trees-bootstrap C1.trees C2.trees ... Cn.trees > partitions.bs

For each split, the SDSF value is just the standard deviation across runs of the Posterior Probabilities for that split. By averaging the resulting SDSF values across splits, we may obtain the ASDSF value (Huelsenbeck and Ronquist 2001). This is commonly considered acceptable if it is < 0.01.

However, it is also useful to consider the maximum of the SDSF values (MSDSF). This represents the range of variation in PP across the runs for the split with the most variation.

To generate the split-frequency comparison plot, you must have R installed. Locate the script compare-runs.R. Then run:

% trees-bootstrap C1.trees C2.trees ... Cn.trees --LOD-table=LOD-table > partitions.bs 
% R --slave --vanilla --args LOD-table compare-SF.pdf < compare-runs.R

Following Beiko et al (2006), this displays the variation in estimates of split frequencies across runs. Splits are arranged on the x-axis in increasing order of Posterior Probability (PP), which is obtained by averaging over runs. We then plot a vertical bar from the minimum PP to the maximum PP.

To calculate the split ESS values, run:

% statreport C1.p C2.p ... Cn.p > Report 

We calculate effective sample sizes based on integrated autocorrelation times. This method has the nice property that simply duplicating every sample does not increase the ESS.

The program Tracer also computes ESS values.

To calculate the split ESS values, run:

% trees-bootstrap C1.trees C2.trees ... Cn.trees > partitions.bs

To compute the ESS for a split, we consider the presence or absence of a split in each iteration as a series of binary values. We compute the integrated autocorrelation time for this binary sequence, which leads to an ESS. This approach is similar to dividing the iterations into blocks and computing the ESS on the PP estimates in the blocks. It is also similar to estimating the variance reduction under a block bootstrap.

To obtain estimates of the stabilization time for each numerical parameter, you may run:

% statreport C1.p > Report 

Each series of values is counted as having stabilized after the series crosses its upper and then lower 95% confidence bounds twice (if the initial value is below the median) or crosses its lower and then upper confidence bounds twice (if the initial value is above the median). The confidence bounds are those based on its equilibrium distribution as calculated from the last third of the values in the sequence.

In addition to examining convergence diagnostics for continuous parameters, it is important to examine convergence diagnostics for the topology as well (Beiko et al 2006). In theory, we recommend the web tool Are We There Yet (AWTY) (Wilgenbush et al, 2004). However, AWTY gives incorrect results if you upload plain NEWICK tree samples -- which is what BAli-Phy outputs. Therefore, if you wish to use AWTY, you must convert the tree samples files to NEXUS before you upload them to AWTY in order to get correct results.

It is also be possible to assess stabilization of tree topologies using tools distributed with bali-phy by using commands like the following. Here, sub-sampling and burnin does not apply to the equilibrium tree files. Also, note that you need to manually construct the equilibrium samples, which we recommend to contain at least 500 trees; you might do this by sub-sampling using the BAli-Phy tool sub-sample.

Note that the running time is the product of the number of trees in the two files. Therefore, comparing two complete tree samples without sub-sampling will take too long.