CI gate¶

tingle check is stat --diff with an opinion. It measures the same branch impact, then exits 1 if the branch made things worse — so a pull request that takes on debt fails the build.

Output is trimmed to what CI needs: the metrics that grew, and only the lines the branch added.

$ tingle check
noqa-comment (regex_count): +2
  + src/api/views.py:23
  + src/api/views.py:41

ruff-ignores (toml_list_length): +1
  + pyproject.toml: D203

$ echo $?
1

A branch that worsens nothing prints nothing and exits 0.

Policies¶

Two policies decide what "worse" means:

Policy	Fails when
`sum` (default)	the metrics grow in total — paying off debt in one metric offsets taking it on in another
`any`	any single metric grows, whatever else improved

[check]
policy = "sum"        # or "any"
ignore = ["loc"]      # metrics that never fail the check

Only the config decides what the gate judges and how: check takes no option that changes the policy or narrows the metrics, so loosening the gate is a config change a review can see.

Under sum, a metric can grow and the branch still pass. Its added lines are still printed, because the trade is worth seeing even when it is allowed.

Ignoring metrics¶

ignore names metrics that are expected to grow — lines of code, say. They take no part in the check: they neither move the total nor fail the build, and they stay out of the output.

A name that matches no configured metric is a config error, so a typo cannot silently ignore nothing.

Strict mode¶

A false positive is fixed in the config, and that fix can ride along on the branch it turns green, where a review sees a feature rather than a looser gate. strict puts loosening on a branch of its own:

[check]
strict = true

With it on, a branch fails when it loosens the config and changes anything else. The same config change alone passes.

$ tingle check
config loosened alongside other changes:
  noqa-comment: 12 to 11
  ~ src/api/views.py

check failed: noqa-comment loosened alongside 1 other changed file

The branch's tree is measured under the base's config and under its own, so only the config can move the numbers. Loosening is any of these:

Change	Reported as
a judged metric scores lower	`noqa-comment: 12 to 11`
a judged metric is removed, or renamed	`noqa-comment: removed`
a judged metric moves to `[check] ignore`	`noqa-comment: moved to [check] ignore`
`policy` goes from `any` to `sum`	`[check] policy: any to sum`
`strict` is turned off	`[check] strict: turned off`
a template module in the project changes	`pack/rules.py: template module changed`

A config change that only raises a score may ride along. Lowered metrics are judged one by one, so raising one never pays for lowering another.

Everything the branch changed counts as "anything else" except the config file and those template modules. With config in pyproject.toml, a change outside [tool.tingle] counts too.

What decides is the base's config: strict is read from the merge-base, so a branch cannot drop it to get past it, and --config cannot point around it. The branch that turns it on is not held to it.

With no base config to compare with, because there is none, it cannot be parsed or it no longer validates, strict mode cannot judge the branch. check says so and carries on. The exception is a base that asked for strict and stopped validating while the branch changed or deleted a template module it names: that counts as loosening. The module is recognised by its dotted path, so a package whose directory does not match its name is not seen.

Only the config is compared. A newer tingle whose bundled patterns differ is not seen, because only one version is installed. Under advisory CI a strict failure is a red job like any other, and does not block the merge.

Gating on spread rather than volume¶

ignore is the blunt instrument. Where a metric is not so much expected to grow as expected to churn, the sharper answer is to gate on how far the thing has spread instead of how often it is written: regex_spread and symbol_spread count the files a thing appears in, so a branch that reworks legacy code passes while a branch that carries it into a new file does not.

It is the difference between "stop using this" and "stop spreading this", and only the second can be gated on without failing every bug fix.

Exit codes¶

Code	Meaning
`0`	the branch worsened nothing
`1`	the branch is a regression under the active policy (or a metric function failed)
`2`	config or usage error

GitHub Actions¶

A complete workflow that fails a pull request which takes on debt:

name: Metrics

on: [pull_request]

jobs:
  tingle:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0        # check needs history for the merge-base
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install tingle
      - run: tingle check

fetch-depth: 0 is not optional

actions/checkout clones shallow by default, and check needs history to find the merge-base with the base branch. Without it the job fails on every run.

Advisory: report without blocking the merge¶

tingle has false positives, so a red check is not always a real problem. To show the branch's debt on every pull request without blocking the merge, put check in its own job with continue-on-error: true. The job goes red when the branch worsens the metrics, so the failure shows in the checks list, but the workflow still passes. tingle's own CI runs it this way.

  tingle:
    runs-on: ubuntu-latest
    continue-on-error: true     # report, but never block the merge
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - run: pip install tingle
      - run: tingle check

Branch protection overrides this

continue-on-error keeps the workflow from failing, but it does not make a check non-required. If the job is listed as a required status check in a branch protection rule, it will still block the merge. Leave it out of that list.

Either way, fix each false positive in the config with a narrower pattern, an ignore_lines entry or a range exclude, and name in [check] ignore the metrics that are expected to grow.

Feedback on the pull request¶

A failing job says the branch took on debt; the reviewer still has to open the log to see where. The pr-feedback action puts it where review happens. It runs check, then:

annotates each line the branch added debt on, under the metrics that grew: an error when the check fails, a warning when the policy let it pass;
comments once on the pull request with the per-metric delta and every addition, and edits that comment on each push rather than adding another.

It fails the step exactly when check does, so it replaces the check step:

  tingle:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write    # the comment
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - run: pip install tingle
      - uses: fancysnake/tingle/actions/pr-feedback@main

Input	Default	Meaning
`config`	auto-discovered	path to `tingle.toml`; annotations follow it into a subdirectory
`comment`	`true`	`false` keeps the annotations and posts no comment
`key`	the `config` input, else `default`	names the comment a run owns, together with the user or App behind the token; give each config its own when one pull request runs several
`github-token`	`github.token`	token used to read and write the comments; a run edits only a comment the same user or App wrote, so a renewed token keeps editing the old comment, and a token for a different user or App starts a new comment and leaves the old one as it was

What GitHub will not show

GitHub shows at most ten annotations of each level per step, so a branch adding more is annotated in part; the comment lists every addition. A pull request from a fork gets a read-only token: the annotations still land, the comment is skipped with a warning.

Annotations sit on lines; an addition with no line — a list entry such as a new ruff ignore, or a whole new file — annotates the file instead.

Publishing the numbers¶

check only judges. To also record the values, add stat --json — it is a separate command and does not affect the exit code:

      - run: tingle stat --json > metrics.json
      - run: jq -r '.metrics[] | "\(.name)\t\(.value)"' metrics.json
      - uses: actions/upload-artifact@v4
        with:
          name: metrics
          path: metrics.json

An artifact is a file, not a history — it expires and nothing plots it. To keep the values and chart them over time, see History.

GitLab¶

Set GIT_DEPTH: 0 so the merge-base is reachable. tingle report --cobertura emits Cobertura XML in which each occurrence line is an uncovered line, which the MR widget consumes directly.

tingle:
  variables:
    GIT_DEPTH: 0
  script:
    - tingle check
    - tingle report --cobertura > tingle.xml
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: tingle.xml

Git hooks¶

CI judges the branch; a hook only reports, before the commit leaves your machine. tingle report --diff --base HEAD measures your uncommitted changes against the last commit and lists what they add and remove; it fails only on a config bug: a metric that errors, or a config or diff it cannot load. A hook that failed on new debt would leave --no-verify as the way past a false positive, and that is a suppression; the fix belongs in the config, and the gate in CI.

The hook measures the working tree. pre-commit stashes unstaged edits but not untracked files, so a stray untracked file with a # noqa in it shows up in the report even though the commit leaves it out.

pre-commit¶

repos:
  - repo: https://github.com/fancysnake/tingle
    rev: v0.10.0
    hooks:
      - id: tingle

pre-commit installs tingle from the tag rev names, so the hook needs no pip install. The hook is verbose, so pre-commit prints its report even though it passes.

To report on the whole branch at push instead, run the hook at the push stage and drop its --base HEAD, so it falls back to [diff] base, then main:

      - id: tingle
        stages: [pre-push]
        args: []

The push hook measures your checkout, not the commits you push: the working tree and its untracked files against the merge-base, with nothing stashed. Push from a clean tree and the two agree.

hk¶

tingle publishes no hk package; the step is one line in your hk.pkl, and it runs the tingle on your PATH:

amends "package://github.com/jdx/hk/releases/download/v2.8.1/hk@2.8.1#/Config.pkl"

hooks {
  ["pre-commit"] {
    steps {
      ["tingle"] { check = "tingle report --diff --base HEAD" }
    }
  }
}