ManyPrimates 1: Modeling Challenge

ManyPrimates is a collaborative network of researchers interested in primate cognition. We collect large datasets on primate cognition in order to investigate how cognitive abilities vary across species. These datasets are the basis for studying cognitive evolution in a quantitative way.

Our first project - ManyPrimates 1 (MP1) - tests short-term memory. The final dataset will include data from 400+ primates from 30+ species. As part of MP1 we want to test evolutionary hypotheses about short-term memory. That is, we want to examine which external (social, ecological) species level variables account for differences in short-term memory abilities between species.

The number of potential predictors and models to consider are endless and there is no a priori way of determining which models and predictors make sense and which do not. The only way is to think through each model and its implications. The outcome of this process depends on theoretical views about what drives cognitive evolution.

To remain theoretically neutral, we want the community to select the models for consideration. We therefore announce the first ManyPrimates modeling challenge. We encourage everyone to spell out the process they think drives the evolution of short-term memory abilities and submit it to the challenge as a model.1 All submitted models will enter into a model comparison.2 The winning model(s) will be highlighted in the corresponding paper and the author(s) will be honored at the ManyPrimates symposium.

There are different ways you can submit a model:

  1. As a formula (e.g. y ~ x + z). This kind of submission makes it easy for us to translate the model to a statistical model. When submitting a formula, please make sure that you specify all variables in it. Example: smt ~ tree_density. This model assumes that short-term memory abilities (smt) in/decrease with tree density in the species natural habitat (tree_density).
  2. As graphical model (DAG or structural causal model). In addition to the formula, a causal model allows us to test additional causal implications.
  3. As a verbal description: Here you simply spell out your model verbally. Example: “Primates whose natural habitat has a lot of trees have better short-term memory”. If you submit a verbal description, we will work together with you to translate it into a formula and/or a causal model.

If you submit a formula or a graphical model, please also include a short verbal description of the causal structure3 and relevant references (if applicable). Models can include many variables and also interaction terms. In any case, it’s important that you specify how your predictors will be measured and how the necessary data can be obtained. In the example above, this could be “number of trees per square mile” with a link to a database that lists this information for all primates.

Please submit your model to manyprimates@gmail.com.

1 Only specify the species level external variables and their relations; all models will include a standard set of control predictors and will account for phylogeny.
2 Models will be fit in a Bayesian framework and compared based on WAIC scores and weights - whenever a causal model is submitted, we will also compare models based on their testable implications (see e.g. Peters, Janzing & Schölkopf, 2017; Pearl, Glymour & Jewell, 2016).
3 Please describe causal influences between predictors and outcome variables as well as causal relations between predictors (if there are multiple).