Define manual quality gates and release-readiness checks for DSAMbayes. The
normal cadence is quarterly; these pages deliberately do not prescribe CI/CD.
Define the canonical release-quality gates for the current DSAMbayes release (v1.3.5), including commands, pass/fail criteria, and evidence requirements.
Audience
Maintainers preparing a release candidate
Reviewers signing off release readiness
Engineers running local pre-merge quality checks
Gate Matrix
Gate ID
Gate
Command
Pass Criteria
Evidence
QG-1
Lint
Rscript scripts/check.R --lint
Exit code 0, no lint failures, no SKIP: output
Terminal log and exit code
QG-2
Style
Rscript scripts/check.R --style
Exit code 0, no style failures, no SKIP: output
Terminal log and exit code
QG-3
Unit tests
Rscript scripts/check.R --test
Exit code 0, no test failures
Terminal log and exit code
QG-4
Stan release evidence
Rscript scripts/check.R --stan-release-evidence
Exit code 0, high-budget pooled and hierarchical checks pass diagnostic thresholds
Terminal log and exit code
QG-5
Package check
_R_CHECK_FORCE_SUGGESTS_=false R -q -e 'rcmdcheck::rcmdcheck(args = c("--no-manual"), error_on = "warning")'
No ERROR. No unresolved WARNING for release sign-off. Any NOTE requires explicit reviewer acceptance
CLI log and results/quality_gate_validate/00_run_metadata/config.compiled.yaml
QG-7
Runner smoke: run
Rscript scripts/dsambayes.R run --config config/blm_timeseries.yaml --run-dir results/quality_gate_run
Exit code 0 and core artefacts exist
CLI log and selected artefacts under results/quality_gate_run/
QG-8
Docs build check
python3 docs-site/build_content.py && (cd docs-site && hugo --cleanDestinationDir)
Exit code 0
Build log and generated site output under docs-site/public/
Gate Definitions
QG-1 Lint
Command:
Rscript scripts/check.R --lint
Fail conditions:
Non-zero exit code
Any lint issue reported
Any SKIP: output
QG-2 Style
Command:
Rscript scripts/check.R --style
Fail conditions:
Non-zero exit code
Any file reported as requiring reformat
Any SKIP: output
QG-3 Unit Tests
Command:
Rscript scripts/check.R --test
Fail conditions:
Non-zero exit code
Any test failure or error
QG-4 Stan Release Evidence
Command:
Rscript scripts/check.R --stan-release-evidence
Fail conditions:
Non-zero exit code
Any high-budget Stan evidence test failure
Any unresolved diagnostic-threshold failure
QG-5 Package Check
Command:
_R_CHECK_FORCE_SUGGESTS_=false\
R -q -e 'rcmdcheck::rcmdcheck(args = c("--no-manual"), error_on = "warning")'
Fail conditions:
Any ERROR
Any WARNING for release sign-off
Escalation condition:
Any NOTE must be reviewed and explicitly accepted with rationale.
Operational note:
Local package checks do not force Suggests. Native decomposition is covered
by the normal DSAMbayes test suite and has no separately provisioned runtime
dependency.
python3 docs-site/build_content.py
(cd docs-site && hugo --cleanDestinationDir)
Fail conditions:
Non-zero exit code
docs-site/build_content.py fails before content mirroring completes
Hugo build aborts before site generation
Missing docs-site/public/index.html
Command Reference
Recommended environment setup before running gates:
# Navigate to your local DSAMbayes checkout and select the host librarycd /path/to/DSAMbayes
source scripts/r-library-path.sh
dsambayes_set_r_library host
mkdir -p "$R_LIBS_USER" .cache
exportXDG_CACHE_HOME="$PWD/.cache"export_R_CHECK_FORCE_SUGGESTS_=false
Optional consolidated local gate (does not replace all release gates):
--docs runs the local docs/link sanity helper against README and the tracked docs surfaces.
--smoke runs the fast unit path plus the minimal Stan-backed smoke subset.
--stan-recovery runs the full-gated DGP coefficient-recovery file only, intended for targeted numerical evidence refreshes.
--stan-release-evidence runs high-budget pooled and hierarchical Stan checks with diagnostic thresholds for release-candidate evidence.
--release runs lint, style, unit tests, minimal Stan smoke, docs sanity, and coverage. Coverage below the current 20% hard floor fails the lane; raise that floor only from measured, retained evidence.
Do not proceed to sign-off with unresolved ERROR or WARNING.
NOTEs require written rationale and reviewer acceptance.
If a gate fails due to environment setup, fix the environment and re-run the full affected gate.
Missing retired decomposition packages are not a blocker for QG-4 because
native decomposition has no external runtime dependency.
If a gate fails due to product code, raise a remediation change and re-run from QG-1.
Sign-off Criteria
Release sign-off requires all of the following:
QG-1 to QG-8 passed.
No SKIP outcomes across mandatory gates.
Evidence bundle completed and reviewed.
Final decision recorded in sign-off-template.md.
Testing and Validation
Purpose
Define the canonical testing and validation workflow for the current DSAMbayes release (v1.3.5), from local pre-merge checks through release-quality gates.
Audience
Engineers running local checks before merge
Maintainers preparing release candidates
Reviewers validating release evidence
Validation layers
Layer
Objective
Primary command(s)
Output proof
Lint
Catch style and static issues early
Rscript scripts/check.R --lint
Exit code 0, no lint failures
Style
Enforce formatting compliance on changed files
Rscript scripts/check.R --style
Exit code 0, no reformat-required files
Unit tests
Catch behavioural regressions in package logic
Rscript scripts/check.R --test
Exit code 0, no test failures
Minimal smoke
Keep a cheap Stan-backed safety check in the routine local loop
Rscript scripts/check.R --smoke
Exit code 0, fast unit tests plus minimal Stan smoke pass
Stan smoke
Exercise the broader compiled Stan suite beyond the minimal smoke tier
Rscript scripts/check.R --stan-smoke
Exit code 0, Stan smoke tests enabled
Stan recovery evidence
Re-run the full coefficient-recovery file under the nightly Stan gate
Rscript scripts/check.R --stan-recovery
Exit code 0, full-Stan DGP recovery tests pass
Stan release evidence
Re-run high-budget pooled and hierarchical evidence checks for release candidates
Rscript scripts/check.R --stan-release-evidence
Exit code 0, targeted high-budget Stan evidence passes diagnostic thresholds
Docs sanity
Catch local docs-link breakage and high-value contract drift
Rscript scripts/check.R --docs
Exit code 0, docs sanity checks pass
Package check
Validate package-level install and check behaviour
R -q -e 'rcmdcheck::rcmdcheck(...)'
No ERROR; no unresolved WARNING
Runner validate
Validate config and data contracts without fitting
Rscript scripts/dsambayes.R validate ...
Exit code 0, metadata artefacts
Runner run
Validate end-to-end runner execution and artefacts
Rscript scripts/dsambayes.R run ...
Exit code 0, core run artefacts
Docs build
Validate docs-site/Hugo buildability
python3 docs-site/build_content.py && (cd docs-site && hugo --cleanDestinationDir)
Exit code 0, successful site build
Environment setup
Run all commands from repository root:
# Navigate to your local DSAMbayes checkoutcd /path/to/DSAMbayes
source scripts/r-library-path.sh
dsambayes_set_r_library host
mkdir -p "$R_LIBS_USER" .cache
exportXDG_CACHE_HOME="$PWD/.cache"
Expected outcome: checks run in a repo-scoped environment with reproducible library and cache paths.
Development dependency profile
The canonical development tools are declared in DESCRIPTION under
Suggests and recorded in renv.lock: testthat, pkgload, lintr,
styler, covr, and rcmdcheck. Restore this profile into the selected
ABI-safe library without forcing optional modelling integrations:
This targeted restore does not replace the release dependency-source check for
private Git packages or a clean restore under the R version recorded in
renv.lock.
Dependency source portability
Before a release candidate is signed off, verify all non-CRAN dependency
sources in renv.lock and DESCRIPTION.
DSAMbayes owns the decomposition implementation. DSAMdecomp and teller are
not package or lockfile dependencies. Release evidence for decomposition must
record the DSAMbayes candidate commit and the native decomposition test results;
do not record credentials in release evidence.
--smoke runs the unit path plus a minimal Stan-backed subset: one cheap BLM MCMC smoke, one pooled deployment-artifact smoke on a real pooled fit, and tiny run_from_yaml() runner smokes including pooled deployment_model.rds roundtrip coverage.
--docs runs the local docs/link sanity helper against README and the tracked docs surfaces.
--release runs lint, style, unit tests, minimal Stan smoke, docs sanity, and coverage as a broader local code gate. Coverage below the current 20% hard floor fails the lane.
Implementation note:
scripts/check.R --all remains a legacy convenience gate for lint, style, tests, and coverage only.
scripts/check.R --docs is the explicit docs/link drift check.
scripts/check.R --release is the clearer code-focused local release profile.
Neither profile replaces rcmdcheck, runner smoke checks, or docs build.
Stan-specific note:
use Rscript scripts/check.R --stan-smoke for the broader opt-in compiled Stan suite
use --stan-smoke-full for the fuller nightly variant
use --stan-recovery when you need a targeted rerun of the DGP recovery evidence without invoking the rest of the full Stan suite
use --stan-release-evidence for the high-budget release-candidate lane covering the warning-prone pooled and hierarchical paths
TSCV operational benchmark
Use the source-only benchmark to measure the current sequential, full-MCMC
time-series cross-validation (TSCV) path before proposing checkpointing or
parallel fold execution.
Inspect the fixed workload without loading the package, writing files, or
running Stan:
The named profiles fix the dataset sizes, folds, chains, iterations and cores.
Only the profile, repetition count from 1 to 10, deterministic base seed, and
output directory are configurable. The driver first performs one unmeasured
one-fold cache warm-up per model class, then runs every measured case in a
fresh R process. It writes raw worker rows, logs, GNU time output,
warmup_results.csv, and benchmark_results.csv below an ignored
results/benchmark_tscv_* directory. GNU time writes stable, locale-neutral
elapsed-time and peak-RSS markers; missing or malformed timing evidence fails
the worker.
For review-profile comparisons, one- and four-fold cases of the same model
class and repetition use identical synthetic data but distinct deterministic
fit seeds. The output records both seeds, the Git commit and dirty state, the
benchmark script hash, R/rstan/StanHeaders and platform details, the CPU model,
the cache root, and whether the worker compiled or restored a cached model.
Warm-up provenance is retained separately in warmup_results.csv.
Interpret the timing fields separately:
process_elapsed_sec and peak_rss_kb cover the whole fresh worker process;
runner_elapsed_sec covers the public runner call inside that process;
fold_fit_sec and fold_score_sec sum only the TSCV fold-level operations.
The benchmark disables unrelated runner diagnostics but keeps TSCV enabled.
Its sampler budgets are intentionally too small for inference or convergence
assessment. A successful worker still requires runner exit status zero, an
overall TSCV status of ok, and every scheduled fold to succeed.
Treat the results as host-specific operational evidence. Cache state, host
load, R and Stan versions, and filesystem behaviour affect them. They do not
establish convergence, predictive superiority, statistical validity or causal
validity. Use representative workload measurements and interruption evidence
to decide whether TSCV optimisation is needed. If that work proceeds, specify
and test checkpoint identity, deterministic seed recovery, cache isolation and
atomic fold writes before considering bounded process-level parallelism.
The fixed review geometry uses the final 100-week training window for the
one-fold case. The four-fold cases use 88-, 92-, 96- and 100-week expanding
training windows with non-overlapping four-week holdouts. The 52-week minimum
is a feasibility threshold, not the realised training size. Cross-class timing
also reflects different workloads: review BLM has 104 input rows, whereas
review CRE has 624 rows and hierarchical structure. Do not interpret their
difference as an intrinsic model-class multiplier.
Expected outcome: all gates complete with exit code 0, with no unresolved release blockers.
Local package-check note:
Canonical local rcmdcheck runs set _R_CHECK_FORCE_SUGGESTS_=false.
Native decomposition needs no external decomposition or telemetry package.
Runner smoke-test expectations
Minimum release smoke expectations:
scripts/check.R --smoke succeeds, proving at least one cheap Stan MCMC path, pooled deployment-artifact coverage on a real pooled fit, and tiny runner fit paths.
validate command succeeds and writes metadata artefacts.
run command succeeds and writes model, fitted/observed output, and diagnostics artefacts.
Required runner artefact paths exist under results/quality_gate_validate/ and results/quality_gate_run/.
If validate fails, run the same command again with a clean run directory path and inspect CLI error output.
If run fails before fitting, inspect both 00_run_metadata/config.resolved.yaml and 00_run_metadata/config.compiled.yaml to confirm the authored and compiled values.
If run fails during fitting, verify local Stan toolchain and cache path from Install and Setup.
If artefacts are missing after success exit code, inspect outputs.* flags in the resolved config and confirm any compile-time artifacts in config.compiled.yaml.
Evidence capture
For release evidence, capture:
Full terminal logs and exit codes for SMK-VAL-01 to SMK-RUN-01.
repository history size does not materially shrink yet
Phase 2: Shrink existing history
This phase is disruptive and should only happen after coordination with every collaborator and any CI jobs or deployment hooks that clone this repository.
Safety rules
Freeze merges to main during the rewrite window.
Ask every collaborator to stop pushing until the rewrite is complete.
Create a backup mirror before modifying history.
Use a temporary clone or mirror for the rewrite, not an active working copy.
Suggested backup
cd ..
git clone --mirror DSAMbayes-Charles-Dev DSAMbayes-Charles-Dev.git-backup
Suggested rewrite target set
Remove historical content for paths that are local caches or generated outputs:
If branch protections block force-pushes, temporarily adjust them before the rewrite window and restore them immediately after.
Collaborator recovery after rewrite
Every collaborator should re-clone. If someone must salvage local work, they should:
git fetch origin
git switch main
git reset --hard origin/main
Re-cloning is still safer than trying to reuse an old clone after a large rewrite.
Phase 3: Optional future hardening
If large binary artifacts must remain versioned in future, move them to Git LFS. Do not use Git LFS for ephemeral caches, run outputs, or package libraries that should stay untracked.
Recommended order
Merge the .gitignore change.
Remove tracked benchmark cache files from the index with git rm --cached.
Confirm CI and docs are unaffected.
Decide whether the current 1.4G.git size justifies a history rewrite.
If yes, schedule a short maintenance window and perform the rewrite from a mirror clone.
Release Evidence Pack
Purpose
Define the exact evidence bundle required before manual DSAMbayes release
sign-off. Substitute the actual release version and candidate SHA; this page is
not version-specific.
Audience
Release owner preparing sign-off materials
Reviewers validating release readiness
Maintainers reproducing release gate outcomes
Evidence root and naming
Use one evidence root per candidate release.
Recommended path:
release_evidence/vX.Y.Z/<YYYYMMDD>_<short_sha>/
Example:
release_evidence/vX.Y.Z/20260714_ab12cd3/
Expected outcome: all sign-off evidence is stored in one deterministic location.
Candidate identity rule:
The candidate hash in 00_release_identity/release_identity.txt,
40_signoff/sign_off_record.md, and the evidence-root short SHA must agree.
If administrative docs or sign-off text change after gate execution, either
update the existing bundle without changing candidate identity or rerun the
gates into a new evidence root for a new candidate.
Mandatory evidence bundle
All items below are mandatory.
ID
Evidence item
Required content
Source
Required path in evidence root
EVD-01
Release identity
Candidate commit hash, branch, intended tag, package version
Expected outcome: dependency sources are visible to reviewers. Any file://
source must be accepted as a local-release prerequisite or replaced with a
reachable pinned remote before external release.
Capture gate logs and exit codes:
Run these commands inside a container whose R version matches renv.lock
(currently 4.5.1). Set the container-specific library before collecting logs.
Provide the mandatory go or no-go checklist before manually creating a
DSAMbayes release tag. It supports the normal quarterly release cycle; it does
not require CI/CD or a rapid release cadence.
How to use this checklist
Complete this checklist after running all release-quality gates.
GO only if every checklist item RL-01 to RL-15 passes.
NO-GO if any mandatory item fails or evidence is incomplete.
HOLD if no hard failure exists but final approval is pending.
Tag creation is allowed only after GO decision is recorded.
A NOTE or expected low-budget smoke-test sampler warning is not a blanket
exception: its source and reviewer treatment must be recorded. QG-4
high-budget diagnostic-threshold failures always block release pending an
explicit human decision.
Completion record template
Use this section when running the checklist.
Field
Value
Release version
<fill>
Candidate commit hash
<fill>
Checklist executor
<fill>
Checklist completion date (UTC)
<YYYY-MM-DD>
Checklist result (GO/NO-GO/HOLD)
<fill>
Evidence root path
<fill>
Sign-off record path
<fill>
Audit continuity reference
For programme-level historical traceability, also review:
Define the deliberate, evidence-led release process for DSAMbayes. It supports
the normal quarterly release cycle and material ad-hoc releases. It is not a
CI/CD process and does not imply continuous deployment or a high release
cadence.
The point is to make the infrequent release decision reproducible and
reviewable, not to automate it for its own sake.
Roles and release boundary
Release owner: fixes the candidate commit, runs or coordinates the gates,
and prepares the evidence bundle.
Reviewer: independently checks the evidence and records the GO, NO-GO, or
HOLD decision. The reviewer should be different from the release owner when
practical.
Maintainers: resolve failures or explicitly approve a documented exception.
One release has one candidate commit. Do not make product, dependency, or
documentation changes after gates begin. If a change is required, create a new
candidate and rerun the affected gates.
Preconditions
Before starting:
The intended version is consistent in DESCRIPTION and CHANGELOG.md.
The candidate commit is identified and the working tree is clean apart from
deliberately excluded local evidence and run artefacts.
The local R runtime and package library are recorded. The current tested
baseline is R 4.5.1; DESCRIPTION declares the supported runtime floor.
The floor is a compatibility policy, not a claim that every R version has
been exercised for every release.
If the release evidence includes decomposition, record the candidate commit
and native decomposition test results in the evidence bundle. Do not record
credentials. DSAMdecomp and teller are not approved dependencies.
No Stan template, prior, boundary-default, or fit-semantics change is
released without the required human model review.
Manual release flow
1. Freeze and identify the candidate
git status --short
git rev-parse HEAD
git rev-parse --abbrev-ref HEAD
Record the full SHA, branch, intended tag, package version, R version, and
platform in 00_release_identity/release_identity.txt of the evidence bundle.
Use the current package version rather than editing this playbook for each
release.
Use a local container only when it makes the environment easier to reproduce;
it is evidence capture, not a CI service. Its R version must match renv.lock
(currently 4.5.1); record the image and digest if one is used. The helper
selects .Rlib-container-r<active-R-version>/; never mount a host library as
the container package library.
3. Create the evidence root
Use the layout defined in Release Evidence Pack.
The evidence root is local working material until the release owner decides
how it will be retained. It must not contain .env, credentials, private data,
or unredacted tokens.
4. Run the mandatory gates once, against the frozen candidate
Run QG-1 to QG-8 from Quality Gates, capturing
both complete logs and exit codes. The canonical commands are:
scripts/check.R --release is a useful local consolidation check, but it does
not replace rcmdcheck, the two runner checks, the docs-site build, or the
high-budget --stan-release-evidence gate.
5. Interpret warnings correctly
The release decision is based on the mandatory gates, not on a search for a
silent terminal.
A non-zero exit code, missing required tool, test failure, unresolved package
WARNING, failed diagnostic threshold, or missing required runner artefact
is a blocker.
rcmdcheckNOTEs require written review and acceptance; they are not
silently waived.
Low-budget Stan smoke tests can emit deliberately exercised sampler warnings.
They are not release evidence and must be recorded as expected test-fixture
behaviour if observed. The high-budget QG-4 diagnostics are the release
evidence and may not be waived without an explicit human decision.
Any new, unexplained warning is a HOLD until it is understood or removed.
6. Review and sign off
Complete the Release Readiness Checklist
and copy Release Sign-off Template into the
evidence bundle. A GO requires all mandatory checks to pass, the candidate SHA
to match throughout, and the reviewer to sign the decision.
7. Tag and publish manually
Only after GO:
git tag -a vX.Y.Z -m "DSAMbayes vX.Y.Z"git push origin vX.Y.Z
Create any GitHub release and publish any docs mirror as separate manual acts.
Record what was published, by whom, and when. No automatic deployment follows
from tagging.
8. Retain the decision record
Retain the signed evidence bundle for approved and rejected candidates. Record
post-release defects as a new hotfix candidate; do not retag or rewrite the
published release history.
Go/no-go rules
GO only when QG-1 to QG-8 pass and the sign-off record is complete.
HOLD when evidence is incomplete, a warning is unexplained, or approval is
pending.
NO-GO for any release blocker. Fix the issue, select a new candidate SHA,
and rerun the affected gates.
Hotfixes
For a post-release defect, branch from the affected tag, make the smallest
safe change, update the version and changelog, and run the same manual process.
The smaller scope does not remove the evidence or human-review requirement.
Provide the final approval record for a DSAMbayes release candidate after all mandatory evidence has been reviewed.
Instructions
Copy this template into the candidate evidence bundle as 40_signoff/sign_off_record.md.
Complete every field.
Use GO, NO-GO, or HOLD for the decision.
If any exception is accepted, record explicit rationale and owner.
Copy the candidate commit hash and evidence-root path directly from
00_release_identity/release_identity.txt; do not introduce a second
candidate hash during sign-off.