Methods & open-science toolkit

Methods & open-science toolkit#

Last reviewed: October 2026. Many of these resources were originally curated by Victoria Guazzelli Williamson during the 2021 ABCD Workshop. Suggest additions via an issue.

General methods and open-science resources, organized by the decisions you’ll face in an ABCD project. Want to code along? Jump to the hands-on tutorials. For ABCD-specific guidance, see Best-practice guides; for ABCD tools and packages, see Working with the data and ABCD tools & resources.

1. Plan & preregister#

ABCD data are already collected, so preregistration is about committing to your analysis plan before you look at your outcomes, and being honest about what you’ve already seen. The DUC strongly encourages it.

2. Reproducible workflow#

Need

Resource

Learn R for data science

R for Data Science, 2nd edition · Coding Club tutorials

Version control with Git/GitHub

Happy Git and GitHub for the useR

Reproducible documents (R, Python)

Quarto

Lock package versions

renv (R) · conda/requirements.txt (Python)

Containers (HPC / cloud)

Apptainer (most clusters) · Docker

Version-control large data

DataLad

Broad guide to reproducible research

The Turing Way · ReproNim

ABCD workshop tutorials (NDA-era code, methods still useful)

2021 ABCD Workshop bookdown

Warning

Never commit ABCD data, or outputs that contain individual-level rows, to Git. Keep data directories in .gitignore. See Disseminating your work.

Why bother? Reasons to work reproducibly (talk), plus the ReproducibiliTea journal clubs (podcast) and the RIOT Science Club talks.

3. Choosing covariates & framing causal questions#

Most ABCD questions are causal at heart, even when the analysis is observational. Covariates should be chosen by reasoning about causal structure, not by habit.

  • Wysocki, A. C., Lawson, K. M., & Rhemtulla, M. (2022). Statistical control requires causal justification. Advances in Methods and Practices in Psychological Science. doi:10.1177/25152459221095823

  • Draw your assumptions as a DAG (directed acyclic graph) with DAGitty (browser or R) or ggdag (R) to identify which variables to adjust for and which to leave alone (mediators, colliders).

  • For race/ethnicity and SES specifically, see the equity section of Best-practice guides.

4. Nesting: family, site, and repeated measures#

ABCD observations are not independent. Siblings and twins share families, families share sites (and scanners), and participants are measured repeatedly. Ignoring this understates uncertainty.

  • Typical structure: random intercepts for family nested in site, plus participant for longitudinal data, e.g. lme4::lmer(y ~ x + age + (1 | site/family/participant)). Many analysts simplify (e.g., one sibling per family, or site as a fixed effect) when models don’t converge. Whatever you choose, report and justify it.

  • Site vs. scanner: ABCD’s DAIRC has advised using site for non-imaging analyses and scanner (device) for imaging analyses, rather than both (May 2020 ABCD newsletter).

  • Worked examples: Thompson et al. (2019) and Smith et al. (2023) in Best-practice guides. Packages (lme4, glmmTMB, gamm4, FEMA) are on Working with the data.

5. Longitudinal models#

Start with the ABCD-specific guides (Hawes et al., 2025; McCormick et al., 2023; King et al., 2018) in Best-practice guides. For specific model families:

Cross-lagged panel models (CLPM) and the random-intercept CLPM (RI-CLPM)

Latent change score models

  • Kievit, R. A., et al. (2018). Developmental cognitive neuroscience using latent change score models: A tutorial and applications. Developmental Cognitive Neuroscience. doi:10.1016/j.dcn.2017.11.007

6. Measurement & validity#

ABCD has many brief or adapted measures. Check that a scale measures what you think it does before modeling it.

  • Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review. doi:10.1037/0033-295X.111.4.1061

  • Flake, J. K., Pek, J., & Hehman, E. (2017). Construct validation in social and personality research: Current practice and recommendations. Social Psychological and Personality Science. doi:10.1177/1948550617693063

  • Rhemtulla, M., van Bork, R., & Borsboom, D. (2020). Worse than measurement error: Consequences of inappropriate latent variable measurement models. Psychological Methods. doi:10.1037/met0000220

7. Effect sizes & inference#

With ~11,000 participants nearly everything is “significant”. The question is whether an effect is meaningful.

8. Prediction & machine learning#

The most common mistake with ABCD is leakage: siblings, or participants from the same site, ending up in both training and test sets, or the same participant’s later visits being used to “predict” their earlier ones.

  • Split by group, not by row. Keep families together, and consider leave-one-site-out validation to test generalization across sites: scikit-learn GroupKFold (Python) · rsample group_vfold_cv() (R)

  • Stratified and other splitting strategies: caret data-splitting guide

  • Do all preprocessing inside the cross-validation loop (scaling, imputation, feature selection, and harmonization such as ComBat).

  • Expect small effects: Marek et al. (2022) in Best-practice guides.

9. Imaging preprocessing#

10. Hands-on tutorials#

Code-along tutorials written by this community, with side-by-side R and Python versions:

Tutorial

What you’ll learn

Visualization tips

Scatterplot matrices for exploring many correlations (GGally / seaborn), and plotting ROI-wise statistics on the brain (ggseg / nilearn)

Tip

Have a tutorial to share, e.g. building a dataset with NBDCtools, fitting a family/site mixed model, or site-aware cross-validation? See How to contribute. Tutorials should use simulated data (like the ones above), never real ABCD data.

11. Podcasts & communities#

Contents