My new article about ancestral character estimation under the threshold model is now available online 'Accepted' (i.e., final submitted pre-proof ms.) at the journal Evolution. In this manuscript, I develop a new approach based on Bayesian MCMC to reconstruct the states for an ordered, discretely valued character at internal nodes of the tree based on the threshold model from evolutionary quantitative genetics. Under the threshold model, the state for a discrete character is determined by an underlying, unobservable continuous trait called liability. Whenever liability crosses a 'threshold' value, the discrete character changes state. In the figure above (which is from my article), I have simulated the evolution of liability on the branches of a tree using Brownian motion, and the plotted color changes track changes to the discrete character.

The threshold model is originally due to Wright (1934); however, it was first used for phylogenetic comparative analysis by Felsenstein (2005; 2012). My contribution is relatively minor. I just show how this model can also be used in the endeavor of ancestral state reconstruction for discrete characters using Bayesian MCMC; and that when the evolutionary process is 'threshold-like', ancestral states are more accurate that the states reconstructed using existing alternative models.

Unfortunately, the appearance (not content, so far as I know) of the equations in this pre-release article version are messed up. Hopefully that is fixed in the proofs! Please feel free to contact me if your institution does not have access to Evolution and you would like a copy of this article. I would be happy to send you the PDF.

A colleague and friend emailed me the other day to ask me if I had any R code for the so-called "random skewers" matrix comparison method of Cheverud (1996). According to this method, we 'skewer' our target pair of covariance matrices with a set of random selection vectors and then measure the (vector) correlation between the response. The strength of this correlation (compared to the correlation of random vectors) is a measure of the similarity of our matrices. The method is perhaps most clearly described in Cheverud & Marroig (2007).

Well, I don't have code - but it was relatively easy to come up with. The biggest challenge is trying to figure out where the null distribution for the correlation comes from. In Cheverud & Marroig (2007) they suggest that we can just generate random vectors where the elements come from a uniform distribution & then compute the correlations between these random vectors. In my limited trials, though, this seems to result in elevated type I error. Alternatively, perhaps we should generate pairs of random covariance matrices using some model, and then compute the mean correlation between random vectors used to skewer our random covariance matrices. If the model for our covariance matrices is the same one used to generate our null distribution, then we seem to end up with type I error at or around the nominal level; as well as p-values more or less uniform on the interval [0, 1], which is another good sign. Unfortunately, this method is much more computationally intensive.

I've posted code for this function on the phytools page here. It depends on clusterGeneration, so this should be installed before attempting to run the code!

Let's try it using genPositiveDefMat(...,covMehod="unifcorrmat") for our covariance matrix & null hypothesis test:

Now let's try it for simulated empirical VCV matrices that are derived from data simulated using the same underlying covariance structure:

This is offered without any guarantees. Please let me know if you find any errors.

Recently an R-sig-phylo subscriber made the following request:

"I need to add a list of 230 species in a phylogenetic tree. Is there a logical way to add the species to the root of the genera to which it belongs, in a systematic way, that is, to make a function recognizes the name of the genera to which the species belongs, and so it adds the species to that root?"

I posted a solution to this, but I also thought that other phytools users might face the same problem so I have now automated this in the function add.species.to.genus. This, along with the matrix comparison function skewers, is also in a new minor phytools build (phytools 0.3-73).

The function works by first peeling the genus name out of the species name. It does this by looking for either the underscore character, "_", or a space character, "". It then proceeds to identify the clade containing con-generics in the tree by matching the genus name to the tip labels. Finally, it can either attach the new species to the root node of the most inclusive clade containing congenerics; or it can attach the new species randomly within that clade. In general, the function works best if the input tree is ultrametric. Otherwise, it may return a tree without edge lengths!

Here's a quick demo:

If the user supplies a species name for a genus with only one representative in the tree; a species with no congeners in the tree; or a genus that is non-monophyletic, in each case the function will try to do something rational and return a warning.

That's it.

I'm working on a short comment (perhaps for publication, perhaps not) on ancestral state reconstruction methods when data for some tips are missing or uncertain. The reason for this is twofold: (1) enough people have asked me about this that it seems some kind of literature reference might be handy; and (2) little to no modification of existing approaches is required to reconstruct ancestral states when some tip states are uncertain or unknown. For instance, this is a routine matter in the reconstruction of ancestral DNA sequences when some tip states are ambiguous.

Possibly the least interesting case for reconstructing ancestral states when some of the data for tips are unknown is ML estimation of ancestral values of a continuous character under constant-rate Brownian motion. This is (1) because ML ancestral states for the parent nodes of lineages leading to tip taxa with missing are exactly the same as the states we'd obtain by linearly interpolating on the basis of the states at subtending nodes, and (2) because ML states for the missing tips in the tree are (under BM which has an expected change of zero with variance σ²t along any edge of length t, remember) identical to the reconstructed state at the parent node.

Uninteresting as it might be in most cases, we can nonetheless do this. I just modified the phytools function anc.ML to attempt this if some tips in the tree are missing from the data vector, x. Here's a simple demo to show what I mean:

If you look closely at the list of missing taxa returned by setdiff and the plotted traitgram you'll see (just as promised) the reconstructed phenotypes of missing tips are connected to their ancestors (interpolated perfectly along the subtending branch) by precisely horizontal lines.

That's it.

This morning Joan Maspons kindly supplied code for several S3 methods that can be used with the phytools function phyl.pca to return results via print, summary, and biplot that are similar to the equivalent S3 methods for objects of class prcomp. After a few minor tweaks (source code here) I have added these methods to the phytools package. The object returned by phyl.pca is in no way changed (it's merely been assigned the class attribute "phyl.pca"), so any prior functions designed to work with phyl.pca should still function. The updates are in a new minor phytools release phytools 0.3-74.

Here's a quick demo:

The phytools function ancThresh, which conducts ancestral state reconstruction of a discrete character under the threshold model using Bayesian MCMC (Revell In press), also allows uncertain tip nodes in the tree. Uncertainty in the tip states are treated as a set of prior probability distributions on the states of terminal species in the tree. In this case, we can also compute posterior probabilities that each tip is in each state. These are automatically plotted by ancThresh if ancThresh(...,control=list(tipcol="estimated"). The burn-in that is used to compute these posterior probabilities is the user supplied burn-in in ancThresh(...,control=list(burnin)).

Unfortunately, and unlike the posterior probabilities for internal nodes, these probabilities are not returned to the user nor are they especially easy to compute. This is because the list component $mcmc only contains the implied states at internal nodes; and thus to get the posterior probabilities for tips, we need to use $liab and $par (which contains the sampled positions of the thresholds for each generation of the MCMC) to get the states.

Here is some code that can be used to compute these probabilities. In the example, mcmc is our object returned by ancThresh:

That's it.

I just wrote some code for my phylogeny methods class (class materials here) to simulate evolution on the phylogeny by the Ornstein-Uhlenbeck process using a constant θ and α without transforming the tree. I think that what I did is correct. I have posted the code (it is an internally called function of fastBM) here. It is also in a minor phytools version (phytools 0.3-77) which can be downloaded & installed from source.

One of the main reasons it could be interesting to simulate OU using my function - rather than by first transforming the tree using transform.phylo (formerly ouTree) in the geiger package & then simulating on the transformed tree using BM - is because it can be used to simulate θ (the position of the optimum) that is different from the x₀, the state at the root node. As many of you probably know - it is not possible to fit this model to a tree with contemporaneous tips (the model is non-identifiable); however simulating under the model presents no problem theoretically.

Here's a quick demo:

To create the plot above also requires a new version of phenogram, which handles label spreading slightly different (using the full range on the vertical axis, rather than simply the range of tip values for extant species). This is also in the latest phytools version.

A phytools user reported that phyl.cca breaks if either X or Y contain only one column. Even though we would not normally use CCA when one or the other of our xs or ys contains only one variable (why not use multivariable regression instead?) there is no theoretic reason why it should not work. The main reason seems to be R's different behavior towards vectors & matrices with one column.

The updated code is here. Please let me know if you run into any problems.

I just learned of this recent article by Ryan Ellingson et al. which makes very neat use of the phytools functions densityMap and contMap for plotting a posterior density from stochastic mapping on a tree, and for mapping a continuous character on the tree (respectively). See Revell (2013) for more details. Below is a reproduction of their figure. On the left is a posterior density from stochastic mapping of 'benthic' vs. 'infaunal' habit; whereas on the right is a projection of the observed or reconstructed values of a continuous character, relative eye size. For more details see the original article, Ellingson et al. (2013).

Click here for larger version.

**Disclaimer: neither densityMap nor contMap can be used (directly) to add tiny line images of fishes to your tree!

phytools has a function, anc.Bayes, for ancestral state estimation of continuous traits using Bayesian MCMC. In preparing a tutorial on ancestral character estimation for my phylogeny methods graduate class, I uncovered a serious error in how the prior probability was calculated. I have now fixed it and posted the code, along with a new phytools build (phytools 0.3-79) that contains this fix.

If you used anc.Bayes with an uninformative prior, this bug probably did not affect you. However, if you tried a strong informative prior in your analysis, then prior versions of this function will not have worked properly.

I also (re-) discovered that the way this function handles specification of the prior and other control parameters of the MCMC is pretty annoying. The function is probably due an overhaul! Hopefully, I can get to that at some point.

I just discovered a small bug in phenogram (the phytools function for plotting traitgrams in various ways) which causes phenogram(...,spread.labels=TRUE) to fail when labels actually don't overlap. So, for instance:

The bug is because if the labels do not overlap at all when we start our optimization to minimize overlap, the objective function becomes undefined. The fix just checks to ensure that there is some overlap, and if not spits back the original vertical positions of the tip labels.

I also changed the default spread.cost, which controls how much to penalize overlap & deviation from the vertical position, x, respectively; although this is still totally under user control.

A few days ago, I received an email with the following text:

"What I have been trying to do is to color the branches of a tree according to the number of steps under parsimony that are required if certain species is attached to the corresponding branch of a backbone tree. As I am paleontologist, I usually need to place certain morphological-defined species onto a backbone constraint tree. But I could not find yet the function to do that. Any help is appreciated!"

Like many such emails (unfortunately) I flagged it to (hopefully) return to later, and then I rediscovered it today. Upon closer look, my first reaction was "why do people think I know how to do these things?" As I started to respond as such, it struck me that in fact this shouldn't be that hard (thanks, in large part, to Klaus Schliep's great package phangorn).

First, let's simulate the empirical case in which the 'backbone' tree for our N taxa is 'known', but we have data for N+1 taxa and we want to know where that extra tip belongs:

OK, now we have a data set (X) containing 26 taxa; but an unrooted tree (tree) containing only 25 tips, and we want to know (& plot) the change in parsimony score that results from attaching the one remaining tip to all of the 48 places in the tree to which it could be attached!

Let's see what ps (the parsimony cost of attaching the tip to all edges of the tree) looks like in this one case:

names(ps) here gives the starting & ending node numbers for each edge in the tree &ps its corresponding parsimony cost.

The neatest thing is that we can easily plot these on the tree using the phytools function plotBranchbyTrait (which, unlike other plotting functions in phytools, calls the ape function plot.phylo internally):

It's pretty clear that the cost of attaching tip "Z" is indeed lowest along the edge (the internode connecting the terminal edges leading to tips "V" and "Y") where tip "Z" actually belongs. Cool.

Note that if we had a model for the evolution of our characters (in this case, our data are DNA sequences - but the query pertained to morphological data from fossils so this is not guaranteed) we could easily construct a similar plot showing differences in the likelihood across the tree.

That's it.

A phytools user reported a small, but measurable, difference in Pybus & Harvey's γ computed by gammaStat in ape and ltt in the phytools package. Since I know that ltt and gammaStat return the same value of γ for trees that are genuinely ultrametric, my guess was that the tree might not be precisely ultrametric. In fact, this turns out to be the case. Even though the tree passed the check is.ultrametric, trees can only be ultrametric to the degree of numerical precision that can be represented in the computer, and trees that are read in from file will generally be still considerably more non-ultrametric than that (due to rounding in the input file).

The user was kind enough to send me her tree so I was able to verify that this was indeed the case. The tree passes the check of is.ultrametric, but as we decrease tol, it eventually fails. Furthermore, it turns out to be the case that γ reported by gammaStat for a slightly non-ultrametric tree is exactly the same as a precisely ultrametric tree where all the tips have the same height as the highest tip; and γ reported by ltt for a slightly non-ultrametric tree is (nearly) exactly the same as for a precisely ultrametric tree where all the tips have the same height as the lowest tip. Here's how I found out (using the problematic tree):

I just posted a new version of ltt that first checks that the tree is ultrametric & then if it passes is.ultrametric then it corrects the tree to be precisely ultrametric so that the computed γ lines up with gammaStat.

Just towards the end of this week I started working on a new project, Rphylip, which is designed to provide an R interface for the longstanding, popular phylogenetics package PHYLIP. PHYLIP does a really remarkable number of different analysis - from a wide range of phylogeny inference methods, to a variety of comparative methods (including several important new approaches that are implemented nowhere else), to tree drawing and other things. A complete list of the programs in the PHYLIP package is here.

My goal is to create functions that interface with the programs of PHYLIP, but allow the user to operate completely within R. That is, all inputs for the PHYLIP programs are created by R; and all outputs are parsed back into R or printed to the R console. Finally, Rphylip will even clean up the input & output files it has created for the PHYLIP programs.

So far, I have created test functions to interface with only a few of the programs of PHYLIP: Rdnaml (for dnaml); Rcontml (for contml; and Rcontrast (for contrast).

To use Rphylip one has to, of course, first install PHYLIP on your computer. Having done this, it is straightforward to run any of the programs of PHYLIP through R. Below is a simple demo using Rdnaml under the default conditions (which are slightly different then the defaults for dnaml). Some output excluded:

The source code for the functions I've written so far in Rphylip are here and I posted a little package build with some minimal documentation for the three interface functions here: http://www.phytools.org/Rphylip/. Feedback welcome!

I've spent a little more time working on Rphylip. Here are a few updates since my last post: I have now added the functions Rdnapars (for dnapars), Rneighbor (for neighbor). I have also cleaned up the code in various ways by pulling out (for instance) repeated functions like writing a PHYLIP formatted DNA input file, etc.

Perhaps the most interesting addition (in my opinion) - at least for Windows users who work in R - is the addition of an internal function findPath. (See code here.) This function means that users of Rphylip may no longer be required to specify the path to the directory containing the PHYLIP executable files. Instead, if no path is supplied, R will check various places on the computer where it is likely to be found (for instance the current working directory, C:\Program Files\", etc.). To my surprise, this option works surprisingly well - and also does not bomb R if the directory containing PHYLIP cannot be found (instead returning a sensible error message).

For instance, witness the following example in which PHYLIP is hidden:

OK, now let's put the PHYLIP folder back someplace sensible, in this case in C:/Program Files:

That worked.

Rphylip is also now on GitHub - so you can follow updates & changes to the code there.

I just tried Rphylip in R on Mac OS X. It appears to work fine, although the internal function findPath will not - so the full path to the PHYLIP executables needs to be supplied by the user. However I also discovered that is is very important that the installation protocol for Mac OS X described here on the PHYLIP installation page needs to be followed or else it will not work. Since this protocol involves commands that can be executed from the terminal, I may write a simple R script for Rphylip that will run these commands for the user to make it even easier.

More soon.

I've been gradually plugging away at Rphylip (my R wrapper for PHYLIP), which now includes working interfaces for contml, contrast, dnaml, dnamlk, dnapars, neighbor, threshml, and treedist. Obviously, I have a long way to go; however some of the new additions cover functionality that can't be found in any other software. For example, Rcontrast (interface for contrast) does regular phylogenetic contrasts - but also allows for within-species sampling (Felsenstein 2008). Rthreshml (interface for threshml, which is not actually yet in PHYLIP but will be in the next release) implements the threshold model which can be used for estimating the correlation between discrete and continuous characters, among other things (Felsenstein 2012).

Here's a quick demo of Rthreshml:

We can see that the estimated covariance matrix is actually very close to our generating matrix. Cool.

Unlike the threshml standalone, we can provide our discrete & continuous characters in any order, and our discrete character can be coded anyway we want - so long as it only has two states. Rthreshml creates the input file, reads back in the output, and sorts the estimated covariance matrix back into the order of the columns of X.

Since my last post on Rphylip I've added R interfaces for a few more of PHYLIP programs: consense (Rconsense), proml (proml), and promlk (promlk). The latter two do ML phylogeny estimation (without & with a molecular clock, respectively) from protein sequences.

In addition to these new interfaces, I also added a few different functions for handling protein sequences in R. I should say that I first attempted to establish whether any such methods already existed in other phylogeny packages by running help.search("protein") on all of my installed packages. What I didn't think to do was search for "amino acid". Had I done so, I would have found the function read.aa in the phangorn package, with which at least my Rphylip function read.protein is basically redundant.

The protein handling functions in Rphylip are as follows:

read.protein, an amino acid sequence reading function that can accept data files in formats "fasta" or "sequential". If format="sequential" or format="fasta" with all sequences of the same length then read.protein will store the input data in a matrix; whereas if format="fasta" and sequences have different lengths, then the data will be stored as a list. In both cases the object is assigned the class attribute "proseq" (protein sequence, in case it wasn't obvious enough).

Finally, as.proseq converts protein sequences from other formats. At the moment, it only recognizes & converts from objects of class phyDat (with attr(x,"type")="AA") from the phangorn package.

OK - here is a brief illustration of the Rproml proml interface using an amino acid dataset for chloroplasts from phangorn written to file:

Rphylip can be obtained from its webpage or from github.

I keep plugging away at the Rphylip project. I have now written R interfaces covering 15* of the 35* programs in the PHYLIP package (*including threshml, which is technically not yet in PHYLIP), as well as a number of helper functions. The latest two added to that list are protpars (for MP tree inference from protein sequences) and protdist (for evolutionary distance estimation from protein sequences). The following is a quick demo of the latter:

Unfortunately - as I don't know where these data came from, I can't tell whether this is a good tree, a bad tree, or an average tree; however I believe that it is quite similar to the tree obtained using Rproml.

Rphylip is available here and on github.

Here are three different ways to compute the among-species variance-covariance matrix for multiple characters on the tree. If we assume that the characters are evolving by Brownian motion, this matrix is an unbiased estimate of the instantaneous diffusion matrix of the evolutionary process.

The three options are (1) the GLS estimating equation (used in Revell 2009); (2) the method of equation 5 in Butler et al. (2000); and (3) we can use PICs (Felsenstein 1985).

Here is code to do each of these in R:

OK - now let's try it with some simulated data:

Obviously, although we can see that these calculations seem different, they produce the same results. The PIC method is a bit trickier, but at least in methods (1) & (2) it is fairly easy to see why this is the case. In case (1) we do X^TC^-1X; whereas in case (2) we do Z = C^-1/2X (as Cholesky decomposition is a kind of matrix square root), followed by Z^TZ which is the same as X^TC^-1/2C^-1/2X (in which each of C^-1/2 are the inverse of the lower & upper Cholesky matrices, respectively), which is then equivalent to X^TC^-1X. Obviously, we need to center X on the vector of ancestral values & divide by n - 1 in either case.