CLI Usage

Purpose

Define the supported command-line interface for scripts/dsambayes.R, including required flags, optional flags, and execution semantics.

Prerequisites

Before using the CLI:

• Complete Install and Setup.
• Run commands from repository root.
• Ensure DSAMbayes is installed in the role-specific R_LIBS_USER selected by dsambayes_set_r_library host.

Entry point

Rscript scripts/dsambayes.R <command> [flags]

Operational commands construct the package-owned DSAMbayes::runner_cli_adapter() before reading configs, fitting models, or writing artefacts. The startup panel records both the loaded package version and the checkout version from DESCRIPTION. runme.R applies the same gate and records the versions as package_version and checkout_version.

The versions must match exactly. A mismatch exits with status 2 before runner work begins. Reinstall the current checkout into the active role-specific library, then retry:

source scripts/r-library-path.sh
dsambayes_set_r_library host
R -q -e 'remotes::install_local(".", dependencies = NA, upgrade = "never")'
R -q -e 'library(DSAMbayes); packageVersion("DSAMbayes")'

This gate proves declared-version equality. It cannot prove Git-revision equality when the installed package has no revision metadata. Reinstall after changing branches or pulling changes that keep the same package version. For a CLI run, session_info.txt records the commit of the checkout that supplied the entry-point script. This identifies the checkout; it does not independently prove that a locally installed package with the same version contains identical source bytes.

The script supports these commands:

• init
• validate
• run
• help (or -h / --help)

Command summary

Flag reference

init

• --out <path> (required): output path for the generated YAML file.
• --template <name> (optional): template name. Default is blm.
  • Supported values in script: master, blm, re, cre, pooled, hierarchical.
  • hierarchical maps to the same template file as re.
• --overwrite (optional flag): allow overwrite of an existing --out file.

validate

• --config <path> (required): YAML config path.
• --run-dir <path> (optional): explicit run directory path.

run

• --config <path> (required): YAML config path.
• --run-dir <path> (optional): explicit run directory path.

Usage examples

Show help

Rscript scripts/dsambayes.R --help

Expected outcome: usage panel is printed with command syntax and notes.

Create a new config from template

Rscript scripts/dsambayes.R init --template blm --out config/local_quickstart.yaml

Expected outcome: config/local_quickstart.yaml is created.

Validate only (dry-run behaviour)

Rscript scripts/dsambayes.R validate --config config/blm_timeseries.yaml

Expected outcome: validation completes without fitting Stan models.

Validate with explicit run directory

Rscript scripts/dsambayes.R validate \
    --config config/blm_timeseries.yaml \
    --run-dir results/quickstart_validate

Expected outcome: validation uses the provided run directory path when writing run metadata.

Execute full run

Rscript scripts/dsambayes.R run --config config/cre_geo_panel.yaml

Expected outcome: full modelling pipeline executes and artefacts are written under results/.

Execute full run with explicit run directory

Rscript scripts/dsambayes.R run \
    --config config/cre_geo_panel.yaml \
    --run-dir results/quickstart_run

Expected outcome: artefacts are written to results/quickstart_run (subject to overwrite rules in config).

Exit and error behaviour

• Exit 0: command completed successfully. For run, this means the pipeline completed and diagnostics did not end in overall_status: fail.
• Exit 1: run completed far enough to preserve the fitted result, but the outcome is non-publishable. This includes diagnostics overall_status: fail, requested scenario-analysis failures, diagnostics publish-gate failures, and post-fit artefact-write failures.
• Exit 2: CLI argument, config, or runtime error before a completed run result could be returned.
• Typical hard failures include:
  • DSAMbayes not installed.
  • Loaded DSAMbayes and checkout versions do not match.
  • Missing required flags (--out or --config).
  • Unknown command.
  • Unknown argument format.

Operational notes

• validate is the recommended pre-run gate. Use it before run whenever you change config or data.
• run prints a run summary and suggested next-step artefacts at completion.
• The CLI itself does not define model semantics. Its package adapter delegates execution to DSAMbayes::run_from_yaml().

Config Schema

Purpose

This page documents the authored YAML contract used by:

• scripts/dsambayes.R
• DSAMbayes::run_from_yaml()
• runme.R

The authored schema is schema_version: 2 only. Older formula-driven YAML files are intentionally rejected.

Processing order

The runner processes configs in this order:

1. Parse YAML.
2. Coerce YAML infinity tokens (.Inf, -.Inf).
3. Apply v2 defaults.
4. Resolve relative paths against the config file directory.
5. Validate the authored v2 contract.
6. Compile the authored config into the internal runner config.
7. Apply managed holiday terms, then build the model and run.

Root sections

Unknown keys fail validation.

Minimal valid config

schema_version: 2

data:
  path: ../data/timeseries/demo_data_synthetic.csv
  format: csv
  date_var: date

target:
  column: revenue
  type: revenue
  transform: identity

media:
  - channel0_signal
  - channel1_signal

controls:
  - t_scaled
  - sin52_1
  - cos52_1

model:
  type: blm

Key differences from the retired schema

• model.formula is no longer authored directly.
• schema_version: 1 configs are rejected.
• Trend and seasonality stay user-authored as ordinary columns under controls.
• Managed time effects are limited to holidays under effects.holidays.
• re and cre models use hierarchy, not cre.enabled flags.
• pooled models use pooling, not pooling.enabled.
• fe models use fixed_effects.unit; hierarchy.group is not an FE alias.

Section reference

schema_version

data

target

media and controls

• media is a required list of modeled media terms.
• controls is a required list, but it may be empty ([]).
• A term may not appear in both lists.
• Manual trend and seasonality terms belong in controls.

All v2 names that become formula terms must be syntactic R names whose value is preserved by make.names(). This rule applies to target.column, target.offset_column, media, controls, fixed_effects.unit, hierarchy.group, and generated term prefixes. ASCII names such as sales_total and media.spend are the portable baseline. Non-ASCII names such as média are supported only when the active R locale preserves the exact name through make.names(). Rename columns containing spaces, operators, backticks, or backslashes before using them in a v2 config; examples such as sales value, paid-search, and sales`net are rejected during config validation. The formula sentinels ., ..., and ..1-style pronouns are also rejected because they do not evaluate as ordinary data columns.

Compiled formula order is:

1. generated holiday terms
2. controls
3. media
4. generated CRE mean terms
5. optional offset
6. hierarchical random-effects term

effects.holidays

Managed holidays are optional and are the only managed effect in M1.

effects:
  holidays:
    enabled: true
    path: ../data/holidays.csv
    label_col: holiday
    country: gb
    country_col: country
    week_start: monday
    prefix: holiday_

Notes:

• The data date column must be aligned to the configured weekly anchor.
• Country filtering materializes a filtered calendar artifact before the compiled config is written.

model

fixed_effects

fixed_effects is required exactly when model.type: fe. The approved shape has one unit key:

model:
  type: fe
  scale: true

fixed_effects:
  unit: geo

fixed_effects.unit must name one syntactic source column. It must differ from the target, date, media, and control columns. For long-format data it must equal data.long_id_col. FE configs cannot also contain hierarchy or pooling.

FE runner support includes validation, dry-run, and bounded non-dry MCMC fitting. Validation and dry-run construct and pre-flight the same unfitted fixed_effects model used by the direct API without Stan compilation or sampling. A materialised dry-run writes only 00_run_metadata/. A non-dry run uses the configured MCMC settings, the existing fit.fixed_effects() path, and the dedicated FE artefact writer.

The FE-safe resolved defaults are:

FE grouped priors support media_beta, control_beta, holiday_beta, and noise_sd. Explicit prior overrides may target an authored slope or noise_sd. Grouped boundaries support media_beta, control_beta, and holiday_beta; explicit boundary overrides may target authored slopes only. Intercept, CRE, pooling, random-effect, and residual-noise boundary requests are rejected.

FE v1 also rejects offsets, log targets, MAP, non-centred positive priors, internal media transformations, implicit row omission, scenario analysis, allocation, forecasting, model selection, time-series selection, generic identifiability output, publish-gate enforcement, flat output layout, level fitted or residual output, deployment, decomposition, and optimisation.

The resolved config exposes save_within_design_csv, save_contrast_residuals_csv, and save_contrast_ppc_csv for the dedicated FE artefact stage. Contrast-space fitted values are diagnostics, not level-scale fitted values or predictions. A completed FE run reports qualification_status: not_assessed and does not set a publishability result. No generic level-scale fitted, observed, residual, diagnostics, decomposition, scenario, optimisation, model-selection, time-series-selection, forecast, or deployment artefacts are written.

hierarchy

Required for model.type: re and model.type: cre.

pooling

Required for model.type: pooled.

priors

Grouped families are available when applicable:

• intercept
• media_beta
• control_beta
• holiday_beta
• cre_beta
• pooling_beta
• random_effect_sd
• noise_sd

Each grouped family accepts either the legacy DSAMbayes style:

family: normal    # or lognormal_ms where supported
mean: 0
sd: 0.5

or the more explicit alias:

distribution: Normal   # or HalfNormal / LogNormalMS where supported
mu: 0
sigma: 0.5

HalfNormal compiles to a zero-centered Normal prior plus an implied lower bound of 0 for unconstrained targeted parameter(s). Parameters that are already positive by construction, such as noise_sd and hierarchical sd_*[...], do not receive an extra boundary row.

The residual-noise prior also accepts this alias:

priors:
  likelihood:
    sigma:
      distribution: HalfNormal
      sigma: 2

boundaries

Boundary families mirror the grouped prior families and may also use explicit boundaries.overrides.

Each grouped or explicit boundary row uses:

lower: -Inf
upper: Inf

For FE, grouped boundaries support media_beta, control_beta, and holiday_beta. A boundary on noise_sd is not supported.

fit

diagnostics

Retains the current runner surface for:

• model_selection
• time_series_selection
• identifiability
• publish-gate controls

Important M1 rule:

• diagnostics.time_series_selection.enabled: true is not supported for pooled runs.
• time-series selection is advisory only in the current release contract; it is not part of publish-gate enforcement.
• lower-level runner paths with adstock/Hill media_transforms are not supported by time-series selection.
• diagnostics.time_series_selection.gap_weeks is optional, defaults to 0, and inserts an embargo between the training window and the scored holdout window.
• FE requires model selection, time-series selection, generic identifiability, and publish-gate enforcement to remain disabled.

scenario_analysis

Opt-in posterior scenario/reference evaluation after a successful MCMC fit:

scenario_analysis:
  enabled: true
  scenario_path: ../data/scenarios/planned.csv
  reference_path: ../data/scenarios/reference.csv
  scale: kpi
  log_response: mean
  interval: 0.9
  aggregate_by: []
  include_percent_lift: true
  save_draws: false

• scenario_path and reference_path must identify CSV or RDS data frames. Relative paths resolve from the config file directory.
• Both inputs must contain aligned rows and all predictors, offsets, date fields, and grouping fields required by the fitted model. They need not contain the response column.
• fit.method must be mcmc. MAP output does not provide posterior contrasts.
• scale is response or kpi. For log-response models, log_response chooses the lognormal conditional mean or the conditional median on the KPI scale.
• aggregate_by names retained label columns. An empty list aggregates the full supplied path within each draw.
• save_draws: false writes summaries and metadata only. Set it to true to add scenario_aggregate_draws.csv.
• For fitted adstock/Hill media, the first supplied observation resets carry-over. If decision-horizon values depend on known earlier exposure, prepend the same observed history to both inputs. The runner reports every supplied row, so retain a field that distinguishes warm-up rows from decision rows if needed.

The output is a model-implied fitted-response contrast, not automatic causal attribution. See Counterfactual response.

allocation

Retains the current runner surface for budget optimisation, with channel targeting based on authored media terms.

• allocation.n_candidates defaults to 2000 and must be a finite integer from 10 through .Machine$integer.max.
• allocation.posterior.draws defaults to 500 and must be a finite integer from 1 through .Machine$integer.max.
• Work scales approximately with candidates multiplied by retained posterior draws and, for efficient frontiers, by the number of feasible budget levels. See Budget Optimisation for measured review thresholds and the benchmark command.

outputs

outputs.root_dir and outputs.run_dir behave as before, but the metadata contract now includes:

• config.original.yaml
• config.resolved.yaml
• config.compiled.yaml
• outputs.save_model_rds controls the full fitted analysis artifact 20_model_fit/model.rds
• outputs.save_deployment_model_rds controls the compact deployment artifact 20_model_fit/deployment_model.rds

When outputs.overwrite: true targets an existing run directory, the runner first validates the complete directory tree. It deletes only recognised flat or staged artifact paths. Unknown files, nested directories, and symbolic links cause an error before any existing artifact is deleted. Empty recognised stage directories may remain and are reused. Concurrent mutation of a run directory during overwrite is not supported.

Current first-slice limit:

• outputs.save_deployment_model_rds: true is supported for model.type: blm, model.type: pooled with fit.method: mcmc, or hierarchical model.type: re / cre with fit.method: mcmc.
• Pooled deployment artifacts score on authored terms and keep the normalized pooling map, but deployment-time newdata / data = ... does not need the pooling columns unless they are also ordinary formula terms.
• Hierarchical deployment artifacts are seen-groups-only; explicit scoring/decomposition data must include the raw grouping columns, and decomposition also requires the response source column(s).
• FE dry-runs write metadata only. Non-dry FE runs use bounded MCMC fitting and may write the dedicated posterior summary, sampler text, within-design, contrast-residual, and contrast posterior-predictive artefacts. They do not enter generic post-fit or decision-layer paths and remain unqualified.

forecast

Reserved placeholder only. In v1.3.3, enabling forecast can materialise 70_forecast/, but the runner does not emit forecast files or plots.

Examples in this repository

• config/blm_timeseries.yaml, weekly time-series BLM example
• config/fe_panel.yaml, weekly geo-panel FE validation, dry-run, and bounded MCMC example
• config/re_geo_panel.yaml, weekly geo-panel RE example
• config/cre_geo_panel.yaml, weekly geo-panel CRE example

Related pages

• CLI Usage
• Run from YAML
• Time Components
• CRE / Mundlak

Output Artefacts

Purpose

This page defines what the YAML runner writes, where files are written, and which config flags control each artefact.

Related pages:

• CLI Usage
• Config Schema

Run directory and layout semantics

Run directory precedence:

1. CLI --run-dir
2. outputs.run_dir
3. Timestamped folder under outputs.root_dir

Layout behaviour:

• outputs.layout: staged (default) writes files under numbered stage folders.
• outputs.layout: flat writes all files directly under the run directory.

Stage folders used by the runner:

• 00_run_metadata
• 10_pre_run
• 20_model_fit
• 30_post_run
• 40_diagnostics
• 50_model_selection
• 60_scenario_analysis (only when scenario_analysis.enabled: true)
• 70_forecast (reserved; directory only when forecast.enabled: true)
• 80_optimisation (only when optimisation or allocation output is written)

artifact_schema_version: 2 identifies this layout. Schema-v1 run directories retain 60_optimisation/; the runner does not rename historical results.

Command behaviour

validate

• validate uses dry_run = TRUE.
• If no run directory is resolved, no artefacts are written.
• If a run directory is resolved (--run-dir or outputs.run_dir), config.original.yaml is written.
• If a run directory is resolved (--run-dir or outputs.run_dir), config.resolved.yaml is written.
• If a run directory is resolved (--run-dir or outputs.run_dir), config.compiled.yaml is written.
• If a managed holiday country filter is active and a run directory is resolved, holiday_calendar.filtered.csv is materialised under 10_pre_run/.
• If a run directory is resolved and outputs.save_session_info_txt: true, session_info.txt is written.
• If forecast is enabled and a run directory is materialised, the 70_forecast/ directory is created.

run

• run writes the full artefact set subject to config toggles and runtime conditions.

Fixed-effects run boundary

Fixed-effects (model.type: fe) runs use a dedicated artefact contract. They fit the existing coefficient-only MCMC estimator, then return before the generic post-fit and decision-layer paths.

With the tracked config/fe_panel.yaml defaults, a completed staged run writes:

• 00_run_metadata/artifact_schema.yaml
• 00_run_metadata/config.original.yaml
• 00_run_metadata/config.resolved.yaml
• 00_run_metadata/config.compiled.yaml
• 00_run_metadata/run_status.yaml
• 00_run_metadata/session_info.txt
• 20_model_fit/model.rds
• 30_post_run/posterior_summary.csv
• 40_diagnostics/chain_diagnostics.txt
• 40_diagnostics/diagnostics_summary.txt
• 40_diagnostics/within_design.csv
• 40_diagnostics/contrast_residuals.csv
• 40_diagnostics/contrast_ppc.csv

The relevant outputs.* flags may remove optional files from this inventory. Setting outputs.save_posterior_rds: true adds 20_model_fit/posterior.rds. Managed holidays may add the existing holiday provenance files under 10_pre_run/.

The two RDS files have different purposes. model.rds is the full fitted same-environment analysis object; it preserves the fitted RStan state and retained FE metadata for reload under the compatible R, RStan, and package environment. posterior.rds is the optional compact extracted posterior table; it is not an executable fitted model and does not replace model.rds.

FE CSV column contracts are:

posterior_summary.csv contains slopes and noise_sd; it does not contain a unit intercept, RMSE, or SMAPE. The three contrast tables use the deterministic orthonormal Helmert basis retained by the fitted model. Their values and labels are basis-dependent diagnostics, not row-level observations, level-scale fitted values, or predictions. Preserve contrast_label and contrast_basis when comparing or joining these files.

diagnostics_summary.txt records factual design metadata and includes:

generic_diagnostics_report: not_run
publish_gate: disabled
qualification_status: not_assessed

Therefore, a completed FE run makes no diagnostic pass, publishability, or production-qualification claim. No generic level-scale fitted, observed, residual, diagnostics, decomposition, scenario, optimisation, model-selection, time-series-selection, forecast, or deployment artefacts are written.

Artefact contract by stage

00_run_metadata

10_pre_run

20_model_fit

30_post_run

Active v1.3 pipeline note:

• 30_post_run/ emits observed and fitted summaries, posterior summaries, and runner linear term-contribution artefacts when their toggles are enabled.
• Runner contribution artefacts fail closed for hierarchical models, models with offsets, and models fitted with probabilistic media transforms. Use the native interactive decomp() contract where that model class is supported.
• When decomposition is unavailable, the runner records deterministic skip rows in 40_diagnostics/artifact_status.csv rather than silently dropping the contract entries.

40_diagnostics

50_model_selection

60_scenario_analysis

The runner emits model-implied fitted-response contrasts. These are not causal effects unless the model design and external assumptions justify that claim.

70_forecast

80_optimisation

Deployment artifact note:

• deployment_model.rds lives under 20_model_fit/, not 70_forecast/.
• It is a compact packaging artifact for deployment consumers, not a signal that the runner now generates future-data forecasts or scenarios.

Response scale semantics (*_kpi.csv vs base files)

Base files (observed.csv, fitted.csv) are always on the model response scale:

• identity response: KPI units
• log response: log(KPI)

KPI-scale files are written only for log-response models:

• observed_kpi.csv
• fitted_kpi.csv

Conversion metadata:

• observed_kpi.csv uses conversion_method = point_exp.
• fitted_kpi.csv uses conversion_method = lognormal_mean by default for log-response fitted values.
• fitted_kpi.csv uses conversion_method = point_exp only when the median back-transform is explicitly requested.

Diagnostics status semantics

diagnostics_report.csv status values:

• pass: check passed configured thresholds
• warn: check breached warning threshold
• fail: check breached fail threshold
• skipped: check not applicable or intentionally skipped

Overall status logic:

• fail if any check is fail
• warn if no fails and at least one warn
• pass otherwise

diagnostics_summary.txt reports:

• overall_status
• counts for pass, warn, fail, skipped

Quick verification commands

List produced files for a run:

latest_run="$(ls -td results/* | head -n 1)"
find "$latest_run" -type f | sort

Inspect key diagnostics files:

latest_run="$(ls -td results/* | head -n 1)"
head -n 20 "$latest_run/40_diagnostics/diagnostics_report.csv"
head -n 20 "$latest_run/40_diagnostics/diagnostics_summary.txt"