Working with the data: tools & packages

Working with the data: tools & packages#

Last reviewed: October 2026 (ABCD Release 7.0). Official source: ABCD docs: Tools and Packages.

This page collects the official and community tools for finding variables, building datasets, computing scores, and modeling ABCD data. Package versions noted here were current at last review, so check each link for updates.

1. Finding variables#

DEAP Data Dictionary & Ontology Explorer (official)#

DEAP lets you browse every variable by domain (ontology tree), with descriptions, value levels, and per-variable data-consideration and responsible-use flags. No DUC is needed to browse the dictionary. Search is keyword-based (space-separated terms are combined with AND). Docs

ABCD Data Dictionary Semantic Search (community)#

The ABCD Data Dictionary Semantic Search app, by Biplabendu Das, lets you describe what you’re looking for in plain language (e.g., “sleep problems reported by parents” or “neighborhood safety”) and returns the most related variables, even when they don’t share your exact words.

  • How it works: it compares a sentence embedding of your query with embeddings of each variable’s label (all-MiniLM-L6-v2) and ranks results by cosine similarity. You can filter by similarity threshold, source, domain, and variable type.

  • Why it’s useful: ABCD has tens of thousands of variables with cryptic names, and keyword search misses synonyms. Semantic search helps with that.

  • Output: export the matching variables to CSV, and copy a ready-made code snippet that feeds them into NBDCtools::create_dataset().

  • Coverage: 6.0 and 7.0 data dictionaries. No DUC needed, since it searches only metadata.

  • Links: App (beepboopstats.com) · mirror (shinyapps.io) · 7.0 version on Hugging Face · GitHub · Docs · Demo video

Caution

Semantic search is a discovery aid, not a substitute for reading the variable documentation. Always confirm a variable’s wording, respondent (youth vs. parent), sessions collected, and data-consideration flags in DEAP or the ABCD docs. The search tool works on the data dictionary only. Never paste ABCD participant data into any chatbot or AI tool, because the DUC prohibits it.

2. Building analysis datasets#

NBDCtools (R, official)#

NBDCtools (DAIRC) builds an analysis-ready data frame from the locally downloaded tabulated release (rawdata/phenotype/), given a list of variables and/or tables. It handles joining, labels, missing codes, and event filtering.

install.packages("NBDCtools")
# metadata package (data dictionary + levels), updated each release:
install.packages("NBDCtoolsData", repos = "https://nbdc-datahub.r-universe.dev")

Key functions:

Task

Functions

Build a dataset

create_dataset(), create_dataset_abcd()

Join / read tables

join_tabulated(), read_dsv_formatted()

Missing codes → NA

transf_value_to_na()

Labels / factors

transf_factor(), transf_label(), transf_value_to_label()

Filter sessions/rows

filter_events_abcd(), filter_id_events(), filter_empty_rows(), filter_empty_cols()

Missingness (“shadow”) matrices

shadow_bind_data(), shadow_replace_binding_missing()

Metadata

get_dd(), get_levels(), get_sessions(), get_releases()

Old ↔ new variable names

convert_names_data(), convert_names_file()

Vignettes: Get started · Joining · Transformations · Filtering · Metadata · Shadow matrices

Tip

ABCD tables are long (one row per participant × session). NBDCtools doesn’t reshape to wide, so use tidyr::pivot_wider() if you need wide format. Many models (mixed models, GAMMs) want long data anyway.

nbdctools (Python, official)#

nbdctools is the Python counterpart (requires Python ≥ 3.11).

pip install nbdctools

  • create_dataset_py(): pure-Python dataset builder.

  • create_dataset(): wraps the R package via rpy2 (requires R + NBDCtools installed).

  • download_metadata() / load_metadata(): load the data dictionary and levels as Polars DataFrames.

GitHub

Reading Parquet directly#

# R
library(arrow)
df <- read_parquet("rawdata/phenotype/<table>.parquet")

# Python
import polars as pl            # or: import pandas as pd; pd.read_parquet(...)
df = pl.read_parquet("rawdata/phenotype/<table>.parquet")

3. Computing summary scores#

ABCDscores (R, official)#

ABCDscores reproduces the official non-proprietary summary scores (release 6.0+), and lets you change the missingness threshold (default ≈ 80% item completion) or compute alternative scores. Its major version tracks the data release (7.x for release 7).

install.packages("ABCDscores")
# pin a version to match your release, e.g.:
# remotes::install_github("nbdc-datahub/ABCDscores@v6.0.0")

Functions are named compute_<domain>_<respondent>_<instrument>__<subscale>_<stat>() (e.g., compute_mh_p_ders__attun_mean()), with compute_<prefix>_all() helpers per instrument. Preprint: bioRxiv 2025.09.04.674066.

4. Analysis in DEAP#

Since May 2026, DEAP includes:

  • Explorations: interactive plots of any variable, with PNG export.

  • Analyses: server-side linear mixed models and GAMMs powered by FEMA, on tabulated variables or imaging outcomes (ROI, vertexwise, voxelwise, connectome), with family and subject random effects by default, optional genetic relatedness and PCs, splines and transforms, and built-in brain viewers.

Docs: FEMA in DEAP · Model specification · Splines · Brain viewers · What’s new

DEAP is great for exploration and quick checks. For publications, make sure you can document and reproduce every modeling choice.

5. Modeling and analysis packages#

ABCD’s design creates dependencies you need to model: siblings/twins within families, families within sites, repeated measures within participants, and scanner differences for imaging. See Best-practice guides for the reasoning.

Need

R

Python / other

Mixed models (family, site, subject nesting)

lme4, glmmTMB

statsmodels MixedLM

Nonlinear age trajectories

gamm4, mgcv

—

Whole-brain mixed models with family structure

—

FEMA / cmig_tools (MATLAB; ABCD docs)

Survey/population weights

survey

—

Site/scanner harmonization (ComBat)

longCombat

neuroHarmonize

Missing data

mice, naniar

—

SEM / longitudinal SEM

lavaan

semopy

Brain plots

ggseg

nilearn

See also this site’s Visualization tips.

6. Imaging pipelines#

Tool

What it is

Status

ABCC (ABCD-BIDS Community Collection)

Processed derivatives (fMRIPrep, XCP-D, QSIPrep/QSIRecon) distributed in the official release

Active (v4.0.0)

DAIRC MMPS pipeline

Source of the tabulated imaging measures; Hagler et al. 2019

Active

abcd-hcp-pipeline

DCAN fMRI BIDS app (Docker)

Active; derivatives no longer in ABCC as of 7.0

abcd-dicom2bids

DICOM → BIDS conversion

Built for NDA era. Raw BIDS is now provided directly

7. Training & learning resources#

Contents