neuroimaging-sample-size-calculator

Neuroimaging Sample Size Calculator

Purpose

Traditional power analysis (e.g., using G*Power for a t-test) fails for neuroimaging because it cannot account for the massive multiple comparisons problem, spatial correlation structure, or the multi-level nature of neuroimaging inference. Neuroimaging requires simulation-based approaches that generate synthetic datasets, apply the full analysis pipeline including multiple comparison correction, and estimate power as the proportion of simulations detecting the effect.

A competent programmer without neuroimaging training would use standard power formulas and dramatically overestimate the power of a whole-brain analysis. They would not know that cluster-extent thresholds, random field theory corrections, and spatial smoothness all affect the effective number of tests, nor that pilot-data-based simulation is the gold standard for neuroimaging power analysis. This skill encodes the domain-specific methodology for simulation-based sample size planning.

When to Use This Skill

• Planning sample size for a new fMRI, EEG, or MEG study
• Conducting power analysis for a grant application or registered report
• Estimating required N when pilot data or published effect size maps are available
• Choosing between whole-brain and ROI-based analysis based on power constraints
• Evaluating the statistical adequacy of a proposed or completed study

Research Planning Protocol

Before executing the domain-specific steps below, you MUST:

1. State the research question — What specific question is this analysis/paradigm addressing?
2. Justify the method choice — Why is this approach appropriate? What alternatives were considered?
3. Declare expected outcomes — What results would support vs. refute the hypothesis?
4. Note assumptions and limitations — What does this method assume? Where could it mislead?
5. Present the plan to the user and WAIT for confirmation before proceeding.

For detailed methodology guidance, see the research-literacy skill.

⚠️ Verification Notice

This skill was generated by AI from academic literature. All parameters, thresholds, and citations require independent verification before use in research. If you find errors, please open an issue.

Why Traditional Power Analysis Fails for Neuroimaging

The Fundamental Problem

Standard power analysis computes the sample size for a single statistical test at a given effect size, alpha, and power. Neuroimaging violates every assumption of this framework:

Source: Mumford & Nichols, 2008; Poldrack et al., 2017.

The Pilot-Data-Based Simulation Approach

The gold standard for neuroimaging power analysis uses pilot data to simulate full datasets at varying sample sizes (Mumford & Nichols, 2008).

Step-by-Step Procedure

Step 1: Obtain pilot data or published effect-size maps
 |
Step 2: Estimate expected effect sizes at regions of interest
 |
Step 3: Simulate datasets with varying N
 |
Step 4: Apply full analysis pipeline (including multiple comparison correction)
 |
Step 5: Compute power = proportion of simulations detecting the effect
 |
Step 6: Find the N that achieves target power (typically 80% or 90%)

Step 1: Obtain Pilot Data

Source: Mumford & Nichols, 2008; Poldrack et al., 2017.

Critical warning: Effect sizes from small pilot studies (N < 20) are inflated due to the winner's curse. Assume the true effect is 50-75% of the pilot estimate (Button et al., 2013).

Step 2: Estimate Effect Sizes

For ROI-based analysis:

1. Define the ROI a priori (from atlas, meta-analysis, or independent data)
2. Extract the mean effect size (Cohen's d or percent signal change) from the pilot data within the ROI
3. Apply the deflation correction (multiply by 0.5-0.75) for conservative estimation

For whole-brain analysis:

1. Use the full unthresholded statistical map as the effect-size map
2. The map captures spatial variation in effect size across the brain
3. Power will vary by region -- focus on the primary region of interest for sample size determination

Step 3: Simulate Datasets

For each candidate sample size N:

1. Generate 1,000-5,000 simulated group maps by: a. Sampling N subjects from a population with the estimated effect size and variance b. Adding realistic noise (estimated from pilot residuals or assumed Gaussian with spatial smoothness matching the pilot data) c. Creating a group-level statistical map

2. Apply the smoothness estimate from the pilot data (or the planned smoothing kernel) to each simulated map

Step 4: Apply Full Analysis Pipeline

For each simulated dataset:

1. Compute the group-level statistical map (e.g., one-sample t-test)
2. Apply the planned multiple comparison correction method:

• Cluster-based inference: apply cluster-defining threshold (CDT) of p < 0.001 (Eklund et al., 2016) and identify significant clusters
• Voxelwise FWE: apply random field theory correction at p < 0.05 FWE
• TFCE: compute TFCE image and apply permutation-based correction
• FDR: apply Benjamini-Hochberg at q < 0.05

Step 5: Compute Power

• Voxel-level power: For each voxel, power = proportion of simulations in which that voxel is significant
• ROI-level power: Power = proportion of simulations in which at least one voxel in the target ROI is significant
• Cluster-level power: Power = proportion of simulations in which a significant cluster overlaps with the target region

Report the power metric most relevant to your planned analysis (Mumford & Nichols, 2008).

Tools and Implementations

fMRIpower (Mumford & Nichols, 2008)

NeuroPowerTools (Durnez et al., 2016)

PowerMap (Joyce & Hayasaka, 2012)

AFNI 3dClustSim

ROI-Based Power Shortcuts

When full simulation is impractical, ROI-based power analysis provides a reasonable alternative:

Procedure

1. Define the target ROI a priori (from atlas, meta-analysis, or independent data)
2. Extract the expected effect size (Cohen's d) from pilot data or literature:

• Mean activation within ROI / standard deviation of activation across subjects

3. Use standard power formulas (G*Power or similar) with the ROI-level effect size
4. No multiple comparison correction is needed for a single a priori ROI

Effect Size Extraction from Published Results

Meta-Analytic Effect Sizes

Use coordinate-based meta-analysis tools to estimate effect sizes at specific brain locations:

Caveat: Meta-analytic effect sizes aggregate across many studies with different designs, populations, and analysis pipelines. They provide a reasonable lower bound but may not match your specific paradigm (Yarkoni et al., 2011).

Current Sample Size Recommendations

Landmark Findings

Minimum Sample Size Table

Registered Report Considerations

Registered reports require pre-specification of sample size with a formal power analysis. For neuroimaging registered reports:

1. Specify the primary analysis (whole-brain vs. ROI) and the corresponding power analysis method
2. Use simulation-based power when possible; if not, use ROI-based power with conservative effect size estimates
3. Pre-specify the multiple comparison correction method and document its impact on required N
4. Include sensitivity analysis: What is the minimum detectable effect size at the planned N?
5. State stopping rules: Pre-register the exact N and analysis plan; sequential analysis requires adjustment (Lakens, 2014)
6. Account for attrition: Specify expected exclusion rate (typically 10-20% for fMRI) and over-recruit

Domain insight: Reviewers will be suspicious of power analyses based on large effect sizes from small pilot studies. Use conservative (deflated) effect size estimates and show power curves across a range of plausible effect sizes.

Practical Workflow for Grant Applications

When Pilot Data Are Available

1. Run fMRIpower or NeuroPowerTools with pilot maps
2. Generate power curves showing power vs. N for the primary contrast and ROI
3. Select N that achieves 80-90% power for the primary analysis
4. Add 15-20% for expected participant exclusions
5. Report: pilot study details, effect size estimates, power tool used, correction method, target power, final N

When No Pilot Data Are Available

1. Search NeuroVault for comparable paradigms; download unthresholded maps
2. Use NeuroPowerTools with the published map
3. Alternatively, estimate ROI-level effect sizes from published papers:

• Extract t-values and convert to Cohen's d
• Apply deflation (multiply by 0.5-0.75; Button et al., 2013)
• Use G*Power for ROI-based power

4. As a last resort, use the benchmark table above with the analysis type closest to your planned study
5. Document all assumptions and state that the power analysis is based on estimated (not measured) effect sizes

Common Pitfalls

1. Using G*Power for whole-brain analyses: Standard power tools compute power for a single test and do not account for multiple comparison correction. This overestimates power by an order of magnitude (Mumford & Nichols, 2008)
2. Trusting pilot study effect sizes: Small pilot studies (N < 20) produce inflated effect sizes. Always deflate by 25-50% (Button et al., 2013)
3. Ignoring the correction method: Power depends critically on whether you use voxelwise FWE, cluster-based, FDR, or permutation-based correction. Power at FDR q < 0.05 can be 2-3x higher than voxelwise FWE p < 0.05 for the same N
4. Conflating within-subject and between-subject power: Within-subject designs (one-sample t-test on contrast maps) are much more powerful than between-subject designs (two-sample t-test) because they eliminate between-subject variance (Mumford & Nichols, 2008)
5. Not accounting for attrition: In fMRI, 10-20% of data may be unusable due to motion, scanner artifacts, or task non-compliance. Over-recruit accordingly
6. Treating all regions equally: Power varies across the brain because effect sizes and noise vary spatially. Power at your primary ROI may be adequate even if whole-brain power is low
7. Assuming published N is adequate: Most published fMRI studies are underpowered (Button et al., 2013). Matching a published study's N does not guarantee adequate power
8. Not reporting sensitivity analysis: Always report the minimum detectable effect size at your planned N, in addition to the power estimate for the expected effect

Minimum Reporting Checklist

• Source of effect size estimate (pilot data, published study, meta-analysis)
• Effect size metric (Cohen's d, r, percent signal change) and value used
• Whether effect size deflation was applied and the correction factor
• Power analysis method (simulation-based, ROI-based analytical, benchmark-based)
• Power analysis tool and version (fMRIpower, NeuroPowerTools, G*Power, custom simulation)
• Number of simulations (for simulation-based approaches)
• Multiple comparison correction method assumed in power analysis
• Statistical threshold used (e.g., CDT p < 0.001, cluster p < 0.05 FWE)
• Target power level (80% or 90%)
• Planned total N and N per group (if applicable)
• Expected attrition rate and over-recruitment plan
• Sensitivity analysis (minimum detectable effect at planned N)

References

• Button, K. S., Ioannidis, J. P. A., Mokrysz, C., et al. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365-376.
• Cox, R. W., Chen, G., Glen, D. R., Reynolds, R. C., & Taylor, P. A. (2017). FMRI clustering in AFNI: False-positive rates redux. Brain Connectivity, 7(3), 152-171.
• Desmond, J. E., & Glover, G. H. (2002). Estimating sample size in functional MRI (fMRI) neuroimaging studies. Journal of Neuroscience Methods, 118(2), 115-128.
• Durnez, J., Degryse, J., Moerkerke, B., et al. (2016). Power and sample size calculations for fMRI studies based on the prevalence of active peaks. bioRxiv, 049429.
• Eklund, A., Nichols, T. E., & Knutsson, H. (2016). Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates. PNAS, 113(28), 7900-7905.
• Joyce, K. E., & Hayasaka, S. (2012). Development of PowerMap: A software package for statistical power calculation in neuroimaging studies. Neuroinformatics, 10(4), 351-365.
• Laird, A. R., Fox, P. M., Price, C. J., et al. (2005). ALE meta-analysis: Controlling the false discovery rate and performing statistical contrasts. Human Brain Mapping, 25(1), 155-164.
• Lakens, D. (2014). Performing high-powered studies efficiently with sequential analyses. European Journal of Social Psychology, 44(7), 701-710.
• Marek, S., Tervo-Clemmens, B., Calabro, F. J., et al. (2022). Reproducible brain-wide association studies require thousands of individuals. Nature, 603(7902), 654-660.
• Mumford, J. A., & Nichols, T. E. (2008). Power calculation for group fMRI studies accounting for arbitrary design and temporal autocorrelation. NeuroImage, 39(1), 261-268.
• Poldrack, R. A., Baker, C. I., Durnez, J., et al. (2017). Scanning the horizon: Towards transparent and reproducible neuroimaging research. Nature Reviews Neuroscience, 18(2), 115-126.
• Salo, T., Yarkoni, T., Nichols, T. E., et al. (2023). NiMARE: Neuroimaging Meta-Analysis Research Environment. NeuroImage, 268, 119862.
• Smith, S. M., Miller, K. L., Salimi-Khorshidi, G., et al. (2011). Network modelling methods for FMRI. NeuroImage, 54(2), 875-891.
• Thirion, B., Pinel, P., Meriaux, S., et al. (2007). Analysis of a large fMRI cohort: Statistical and methodological issues for group analyses. NeuroImage, 35(1), 105-120.
• Turner, B. O., Paul, E. J., Miller, M. B., & Barbey, A. K. (2018). Small sample sizes reduce the replicability of task-based fMRI studies. Communications Biology, 1, 62.
• Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011). Large-scale automated synthesis of human functional neuroimaging data. Nature Methods, 8(8), 665-670.

See references/ for worked examples and simulation code templates.