evidence-accumulation-selector

Evidence Accumulation Model Selector

Purpose

This skill encodes expert knowledge for selecting among evidence accumulation models (EAMs) when analyzing choice response-time (RT) data. A competent programmer without cognitive science training would typically analyze only mean RT and accuracy separately, missing the critical insight that RT distributions and speed-accuracy tradeoffs carry rich information about latent cognitive processes. Selecting the wrong EAM -- or applying one when the data violate its assumptions -- leads to uninterpretable or misleading parameter estimates.

When to Use

Use this skill when:

• You have choice-time data (both accuracy and full RT distributions, not just means)
• You want to decompose observed performance into latent cognitive processes (evidence quality, response caution, non-decision time)
• You need to distinguish speed-accuracy tradeoff effects from genuine sensitivity changes
• You are deciding which model class (DDM, LBA, EZ-diffusion, race model) is appropriate for your experimental design

Do not use this skill when:

• You only have accuracy data without RTs (use signal detection theory instead)
• RTs are from simple detection (single response option) rather than choice tasks
• The task involves continuous tracking or free response without discrete choice points

Research Planning Protocol

Before executing the domain-specific steps below, you MUST:

1. State the research question -- What specific question is this analysis/paradigm addressing?
2. Justify the method choice -- Why is this approach appropriate? What alternatives were considered?
3. Declare expected outcomes -- What results would support vs. refute the hypothesis?
4. Note assumptions and limitations -- What does this method assume? Where could it mislead?
5. Present the plan to the user and WAIT for confirmation before proceeding.

For detailed methodology guidance, see the research-literacy skill.

⚠️ Verification Notice

This skill was generated by AI from academic literature. All parameters, thresholds, and citations require independent verification before use in research. If you find errors, please open an issue.

Core Concepts: What EAMs Do

All evidence accumulation models share a common framework: on each trial, noisy evidence is accumulated over time until a decision boundary is reached, triggering a response. The models differ in their assumptions about accumulation architecture.

Key Parameters Across Models

Decision Tree: Selecting a Model

How many response alternatives does the task have?
|
+-- TWO alternatives
| |
| +-- Do you need full distributional analysis?
| | |
| | +-- YES --> Do you have sufficient trial counts (>50/condition)?
| | | |
| | | +-- YES --> Use the FULL DIFFUSION MODEL (DDM)
| | | | (Ratcliff, 1978; Ratcliff & McKoon, 2008)
| | | |
| | | +-- NO (fewer trials) --> Use EZ-DIFFUSION
| | | (Wagenmakers et al., 2007)
| | |
| | +-- NO (means/summaries sufficient)
| | --> Use EZ-DIFFUSION for simplicity
| | (Wagenmakers et al., 2007)
| |
| +-- Is response bias (starting point) a key research question?
| |
| +-- YES --> Use FULL DDM with z parameter free
| | (Ratcliff, 1985; White & Poldrack, 2014)
| |
| +-- NO --> DDM with z fixed at a/2 (unbiased)
|
+-- MORE THAN TWO alternatives
| |
| +-- Use the LINEAR BALLISTIC ACCUMULATOR (LBA)
| | (Brown & Heathcote, 2008)
| | or RACING DIFFUSION MODEL
| | (Tillman et al., 2020)
| |
| +-- Do accumulators need to be independent?
| |
| +-- YES --> LBA (independent accumulators by design)
| |
| +-- NO (competition matters) --> Racing diffusion
| or leaky competing accumulator (LCA; Usher & McClelland, 2001)
|
+-- SPECIAL CASES
 |
 +-- Extremely fast RTs (<200 ms median)?
 | --> EAMs are likely inappropriate; these may be anticipatory
 | responses (Luce, 1986)
 |
 +-- No speed pressure at all (untimed)?
 | --> EAMs are inappropriate; use accuracy-based models
 |
 +-- Go/no-go task?
 --> Use the DDM with absorbing boundary modifications
 or the SSRT framework (Verbruggen & Logan, 2008)

Model Descriptions

Drift Diffusion Model (DDM)

The canonical EAM for two-choice tasks (Ratcliff, 1978; Ratcliff & McKoon, 2008).

Architecture: A single accumulator drifts between two absorbing boundaries. Evidence for option A moves the process toward the upper boundary; evidence for option B moves it toward the lower boundary.

Full DDM parameters (7 parameters; Ratcliff & Tuerlinckx, 2002):

When to use DDM:

• Two-choice tasks with speed-accuracy tradeoff
• At least 40-50 trials per condition for the full model (Ratcliff & Childers, 2015), though 200+ recommended for stable individual parameter estimates (Lerche et al., 2017)
• RTs in the typical range: 200 ms to 2000 ms (Ratcliff & McKoon, 2008)

Key assumption: Only two response options. The DDM cannot natively handle >2 choices.

EZ-Diffusion

A simplified closed-form estimator for three DDM parameters (Wagenmakers et al., 2007).

Estimated parameters: v (drift rate), a (boundary separation), Ter (non-decision time).

Input: Only three summary statistics per condition -- mean RT for correct responses (MRT), variance of RT for correct responses (VRT), and accuracy (Pc).

Closed-form equations (Wagenmakers et al., 2007, Eq. 1-3; see references/ez-diffusion-formulas.md):

When to use EZ-diffusion:

• Quick exploration before committing to full DDM fitting
• Low trial counts where full DDM is unstable (as few as ~10 trials per condition; Wagenmakers et al., 2007)
• When only summary-level data are available (e.g., published means and variances)
• When the research question does not require starting point bias or cross-trial variability parameters

Limitations:

• Assumes no starting point variability (sz = 0) and no cross-trial drift variability (sv = 0)
• Cannot estimate response bias
• The "edge correction" is needed when accuracy is 0.5 or 1.0 (Wagenmakers et al., 2007)

Linear Ballistic Accumulator (LBA)

A multi-alternative accumulator model (Brown & Heathcote, 2008).

Architecture: N independent linear accumulators (one per response option) race to a common threshold. The first accumulator to reach threshold triggers the corresponding response. Accumulation is ballistic (no within-trial noise) -- all variability comes from across-trial variation in drift rates and starting points.

Parameters per accumulator (Brown & Heathcote, 2008):

When to use LBA:

• Tasks with 2 or more response alternatives (Brown & Heathcote, 2008)
• When you need a mathematically tractable multi-choice model
• When accumulators can be assumed independent (no lateral inhibition)
• Minimum ~100 trials per condition recommended (Donkin et al., 2011)

Race Models

Classical race model (Pike, 1966; Townsend & Ashby, 1983): Multiple accumulators race independently; first to finish wins. Unlike DDM, there is no competition between accumulators.

When to use:

• As a baseline/null model to test against more complex models
• When inhibitory competition between responses is not theoretically expected

Limitation: The standard race model cannot account for speed-accuracy tradeoff without additional assumptions (Ratcliff & McKoon, 2008).

Model Comparison Methods

When comparing model fits, use information criteria that penalize complexity:

Preferred approach: Fit competing models and compare using WAIC or Bayes factors in a Bayesian framework (Annis et al., 2017). Lower WAIC = better fit.

Parameter Recovery Check

Before interpreting fitted parameters, always conduct a parameter recovery study (Heathcote et al., 2015):

1. Simulate data from known parameter values matching your design
2. Fit the model to simulated data
3. Check that recovered parameters correlate highly (r > 0.90) with generating parameters
4. If recovery fails, the model is too complex for your data or trial counts are insufficient

Software Recommendations

Common Pitfalls

1. Analyzing mean RT only: Mean RT conflates drift rate, boundary separation, and non-decision time. Two conditions with identical mean RTs can have very different latent processes (Ratcliff & McKoon, 2008).

2. Applying DDM to >2-choice tasks: The standard DDM is defined for two-choice tasks only. For 3+ alternatives, use LBA, racing diffusion, or the multi-alternative DDM extension (Ratcliff & Starns, 2013).

3. Insufficient trial counts: The full DDM requires at least 40-50 trials per condition for group-level estimates and 200+ for stable individual estimates (Ratcliff & Childers, 2015; Lerche et al., 2017). With fewer trials, use EZ-diffusion or hierarchical Bayesian fitting.

4. Ignoring RT distribution shape: EAMs predict specific distributional forms (right-skewed). If your RT distribution is bimodal or has a long left tail, check for contaminant processes (e.g., fast guesses) before fitting (Ratcliff & Tuerlinckx, 2002).

5. Not trimming outlier RTs: Extremely fast (<200 ms) or slow (>3000 ms for speeded tasks) RTs likely reflect processes outside the model. Standard practice: trim RTs below 200 ms and above a task-appropriate upper bound (Ratcliff & McKoon, 2008).

6. Fitting too many free parameters: The full 7-parameter DDM is often overparameterized. Fix parameters that are not theoretically relevant (e.g., fix sz = 0 and st0 = 0 as a starting point; Ratcliff & Childers, 2015).

7. Confusing EZ-diffusion limitations: EZ-diffusion assumes no across-trial variability in drift or starting point. If your design manipulates prior probability (affecting starting point bias), EZ cannot capture this (Wagenmakers et al., 2007).

8. Skipping parameter recovery: Without recovery checks, you cannot know whether your data are informative for the parameters you want to interpret (Heathcote et al., 2015).

Minimum Reporting Checklist

Based on Dutilh et al. (2019) and current best practices:

• Model selected and justification (why DDM vs. LBA vs. EZ)
• Number of trials per condition per participant
• RT trimming criteria and percentage of data excluded
• Complete list of free vs. fixed parameters with rationale
• Fitting method (MLE, chi-square, Bayesian) and software (with version)
• Model fit assessment (quantile probability plots, AIC/BIC/WAIC)
• Parameter recovery results (simulated data check)
• All parameter estimates with uncertainty (SE or credible intervals)
• Model comparison results if multiple models were fit
• Diagnostic plots: observed vs. predicted RT quantiles (0.1, 0.3, 0.5, 0.7, 0.9) for correct and error responses

References

• Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716-723.
• Annis, J., Miller, B. J., & Palmeri, T. J. (2017). Bayesian inference with Stan: A tutorial on adding custom distributions. Behavior Research Methods, 49, 863-886.
• Brown, S. D., & Heathcote, A. (2008). The simplest complete model of choice response time: Linear ballistic accumulation. Cognitive Psychology, 57, 153-178.
• Donkin, C., Averell, L., Brown, S., & Heathcote, A. (2011). Getting more from accuracy and response time data: Methods for fitting the linear ballistic accumulator. Behavior Research Methods, 43, 332-343.
• Dutilh, G., et al. (2019). The quality of response time data inference: A blinded, collaborative assessment of the validity of cognitive models. Psychonomic Bulletin & Review, 26, 1051-1069.
• Heathcote, A., Brown, S. D., & Wagenmakers, E.-J. (2015). An introduction to good practices in cognitive modeling. In B. U. Forstmann & E.-J. Wagenmakers (Eds.), An introduction to model-based cognitive neuroscience. New York: Springer.
• Heathcote, A., Lin, Y.-S., Reynolds, A., Strickland, L., Gretton, M., & Matzke, D. (2019). Dynamic models of choice. Behavior Research Methods, 51, 961-985.
• Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773-795.
• Lerche, V., Voss, A., & Nagler, M. (2017). How many trials are required for parameter estimation in diffusion modeling? Behavior Research Methods, 49, 513-537.
• Luce, R. D. (1986). Response times: Their role in inferring elementary mental organization. New York: Oxford University Press.
• Pike, R. (1966). Stochastic models of choice behaviour: Response probabilities and latencies of finite Markov chain systems. British Journal of Mathematical and Statistical Psychology, 19, 15-32.
• Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59-108.
• Ratcliff, R. (1985). Theoretical interpretations of the speed and accuracy of positive and negative responses. Psychological Review, 92, 212-225.
• Ratcliff, R., & Childers, R. (2015). Individual differences and fitting methods for the two-choice diffusion model of decision making. Decision, 2, 237-279.
• Ratcliff, R., & McKoon, G. (2008). The diffusion decision model: Theory and data for two-choice decision tasks. Neural Computation, 20, 873-922.
• Ratcliff, R., & Rouder, J. N. (1998). Modeling response times for two-choice decisions. Psychological Science, 9, 347-356.
• Ratcliff, R., & Starns, J. J. (2013). Modeling response times, accuracy, and confidence in two-choice tasks. Psychological Review, 120, 510-560.
• Ratcliff, R., & Tuerlinckx, F. (2002). Estimating parameters of the diffusion model. Psychonomic Bulletin & Review, 9, 438-481.
• Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
• Shinn, M., Lam, N. H., & Murray, J. D. (2020). A flexible framework for simulating and fitting generalized drift-diffusion models. eLife, 9, e56938.
• Singmann, H., Brown, S., Gretton, M., & Heathcote, A. (2016). rtdists: Response time distributions. R package.
• Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & van der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society B, 64, 583-639.
• Tillman, G., Van Zandt, T., & Logan, G. D. (2020). Sequential sampling models without random between-trial variability: The racing diffusion model with its competing risks. Psychonomic Bulletin & Review, 27, 1170-1190.
• Townsend, J. T., & Ashby, F. G. (1983). Stochastic modeling of elementary psychological processes. Cambridge University Press.
• Usher, M., & McClelland, J. L. (2001). The time course of perceptual choice: The leaky, competing accumulator model. Psychological Review, 108, 550-592.
• Verbruggen, F., & Logan, G. D. (2008). Response inhibition in the stop-signal paradigm. Trends in Cognitive Sciences, 12, 418-424.
• Voss, A., & Voss, J. (2007). Fast-dm: A free program for efficient diffusion model analysis. Behavior Research Methods, 39, 767-775.
• Wagenmakers, E.-J., van der Maas, H. L. J., & Grasman, R. P. P. P. (2007). An EZ-diffusion model for response time and accuracy. Psychonomic Bulletin & Review, 14, 3-22.
• Watanabe, S. (2010). Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research, 11, 3571-3594.
• White, C. N., & Poldrack, R. A. (2014). Decomposing bias in different types of simple decisions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40, 385-398.
• Wiecki, T. V., Sofer, I., & Frank, M. J. (2013). HDDM: Hierarchical Bayesian estimation of the drift-diffusion model in Python. Frontiers in Neuroinformatics, 7, 14.

See references/ez-diffusion-formulas.md for EZ-diffusion closed-form equations and worked examples.