* ci: fall back to develop when frappe has no matching branch
The frappe branch to install is taken from the pull request's base branch. A
stacked pull request targets another erpnext branch, so the clone fails with
"couldn't find remote ref", no bench is installed, and every job that needs one
fails with it.
Fall back to develop when the base branch does not exist in frappe. An explicit
FRAPPE_BRANCH is left alone, since it can be a commit sha rather than a branch.
* ci: only fall back when frappe is known to lack the branch
git ls-remote --exit-code reports 2 for a branch that is not there and 128 for a
remote it could not reach. Treating both as absence let a transient network or
DNS failure install develop over the branch the pull request was built against.
Fall back on 2 alone and log anything else, so a flaky probe leaves the branch
as it was.
The "Stop DB and stage datadir" step swallowed a failed `pg_ctl -m fast -w
stop` with `|| true`, then moved and tarred the PGDATA regardless. A stop
that times out or errors would bake a still-running, crash-inconsistent
cluster into the artifact every test shard consumes — and with
full_page_writes off, crash recovery can't repair torn pages. Drop the
`|| true` so a failed stop fails the job, mirroring the MariaDB sister's
"don't bake a dirty datadir" guard.
Also drop the redundant `ALTER SYSTEM SET fsync/synchronous_commit/
full_page_writes = off` block from install.sh. Its comment claimed the
postgres workflow "runs a service-container DB and never calls start-db.sh",
but it does call start-db.sh, which already applies those flags via `-o` on
every postgres start (setup job and each shard). The block was a no-op and
its justification was factually wrong.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Wire up warm-bench: set BENCH_CACHE_DIR on the setup job so the bench-base
cache actually activates (was inert with no dir set, so every run did a
full bench init). Lives on the node-local bench-staging hostPath; any
miss/failure still falls back to a full init.
- run_ci_step: capture the timeout exit code under `set -e` (the previous
`timeout ...` + `ec=$?` aborted at the timeout line on failure, skipping
::endgroup:: and the exit-code return).
- Raise the per-step timeout 600s -> 1800s so a contended reinstall isn't
killed before the 40-min job timeout.
- Propagate DB through the su re-exec in start-db.sh / hydrate.sh so a
DB=postgres invocation can't silently fall back to the mariadb branch.
- Simplify the coverage job `if` to the equivalent plain non-PR gate.
The fan-out moved fsync/synchronous_commit/full_page_writes=off into
start-db.sh startup flags, but the Postgres workflow runs a postgres:13.3
service container and calls install.sh directly — it never runs start-db.sh.
So those flags never reached the Postgres CI, regressing it to full durability
on a commit-heavy suite. Re-apply them via ALTER SYSTEM (reloadable) in the
DB == "postgres" path, where the service-container workflow executes. MariaDB
is unaffected (DB != postgres).
Adds the division-by-zero divergence class to the PG-compat review tooling:
on a divisor that the data can drive to 0 (e.g. Sum(a)/Sum(b)), MariaDB
returns NULL for division by zero while PostgreSQL raises `division by zero`
and aborts the query. The portable fix is to wrap the divisor in
NullIf(divisor, 0), which yields NULL on both engines (matching MariaDB).
- .greptile/config.json: add it to the "would ERROR on PostgreSQL" list.
- .github/POSTGRES_COMPATIBILITY.md: document it under §1 (hard breaks).
- .github/helper/postgres_compat.py: note it in the docstring as a
deliberately-not-statically-checked semantic divergence (data-dependent,
like integer-division intent), so it stays a reviewer/Greptile concern.
Tooling-only; no source query changes. The instance fix shipped in #56361.
The [^)]* span stopped at the first inner ')', so CAST(ABS(col) AS CHAR) slipped through. Use a non-greedy .+? with re.S; still zero production false positives (verified). Addresses review feedback.
The whole-repo MariaDB<->PostgreSQL audits surfaced classes the checker and
review guide did not yet cover. Add them:
Static checker (.github/helper/postgres_compat.py) - new mechanical breaks:
- .rlike() / raw RLIKE: frappe rewrites REGEXP->~* on Postgres but NOT RLIKE.
- Cast(x, "char") / raw CAST AS CHAR: bare CHAR is character(1) on Postgres
and truncates multi-digit values; use "varchar".
(Both flag zero production code; the only repo hit is in patches/, which the
hook already excludes.)
Greptile config + POSTGRES_COMPATIBILITY.md - new semantic/hard classes:
- aggregate (Sum/Count) selected next to bare columns with no GROUP BY at all.
- .like()/LIKE on a non-text column (bigint ILIKE) -> Cast_ to varchar.
- get_all(fields=["CapitalCase"]) identifier-case (extends the get_value case).
- bool into a Check column via qb.update().set() (extends set_value/db_set).
- int/int division: float a literal (col/1440 -> col/1440.0).
- Concat over a nullable column leaking a bare prefix on Postgres.
- clarify REGEXP/.regexp() is translated but RLIKE/.rlike() is not.
frappe's db_query SILENTLY drops ORDER BY for distinct queries on Postgres (the ORDER BY
column must appear in the SELECT-DISTINCT list), so `get_all/get_list(distinct=True,
order_by="<col>")` is a no-op there and the result comes back unordered — the root cause of
the Sales Register, Purchase Register and Sales Analytics ordering fixes. Add an AST rule to
.github/helper/postgres_compat.py that flags this (literal order_by only; an empty order_by=""
suppression and a dynamic/variable order_by are not flagged). `# pg-ok` escape hatch as usual.
Grandfather the three pre-existing low-impact sites the rule surfaces (paging/iteration order
only, not data): job_card operation autocomplete, inventory_dimension config list, and a
work_order test loop.
- semgrep: annotate the source-reading open() with # nosemgrep for the
frappe-security-file-traversal rule (dev-only lint tool; path comes from pre-commit,
not user input).
- bool-scan: only inspect the field *value* arg (db_set args[1]/dict args[0];
set_value args[3]/dict args[2]) so a positional update_modified=False
(e.g. db_set('f', 0, False)) no longer false-positives.
- # pg-ok: also honour the annotation on a multi-line call's closing paren line
(scan one line past the node's end).
The Postgres test job is label-gated, so it does not run on every PR. This adds an
always-on pre-commit hook that statically flags the *mechanical* breaks: MySQL-only
functions (timestamp(date,time), timediff, str_to_date, date_format/add/sub,
group_concat, period_diff, SQL IF()), SHOW INDEX/TABLES/COLUMNS, single-quoted
aliases, UPDATE..JOIN, interpolated/f-string SQL carrying MySQL-isms,
set_value/db_set(<Check>, bool), and MySQL SHOW INDEX result keys.
It deliberately does NOT flag the framework auto-translations (ifnull->coalesce,
backtick/locate/REGEXP, .like()->ILIKE) nor the *semantic* divergences (loose GROUP
BY, case-sensitive ==/IN, NULL ordering, tiebreakers) — those need the test suite,
which remains the backstop. AST + structure-gated regex keep false positives near
zero (docstrings and prose skipped); '# pg-ok' exempts intentional MariaDB-only
branches. Scoped to erpnext/ excluding patches/. Includes a unit test of the checker.
The Postgres CI site only listed erpnext in install_apps, so the payments app
(fetched and built by install.sh via 'bench get-app payments') was never
installed on the site — leaving 'tabPayment Gateway' absent. test_payment_request
(and other payment-gateway-dependent tests) then errored on Postgres with
'relation "tabPayment Gateway" does not exist', while MariaDB passed because its
site_config already lists ["payments", "erpnext"]. Match that ordering for parity.
Postgres fsyncs on every commit by default, which dominates a commit-heavy test suite.
Turn off synchronous_commit/fsync/full_page_writes on the throwaway CI database (reload-
time settings, no restart). MariaDB CI is unaffected (DB != postgres).
* ci: re-fetch before push to avoid force-push on translations_hotfix
If upstream/translations_hotfix moved forward while the script was
running (e.g., a concurrent run or manual push), git push would fail
with "behind remote". Re-fetch right before pushing and merge any new
remote commits with -X ours so our .po changes and the main.pot from
${HOTFIX_BRANCH} (set by the initial -X theirs merge) are preserved
without needing a force-push.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: define HOTFIX_BRANCH once as job env; pass it to sync script
version-16-hotfix is now declared as env.HOTFIX_BRANCH at the job level
so the checkout ref and the script argument both derive from the same
value. Quoting the GITHUB_WORKSPACE path guards against spaces on
self-hosted runners.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: define HOTFIX_BRANCH as job env and pass to sync script via env
Declares version-16-hotfix once as env.HOTFIX_BRANCH at the job level
so the checkout ref and the script both derive from the same value.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: read HOTFIX_BRANCH from env instead of hardcoding in script
Removes the hardcoded branch name from the script; reads it from the
HOTFIX_BRANCH env var set by the workflow. Fails immediately with a
clear message if the var is not set.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: use ls-remote to check branch existence instead of swallowing fetch errors
Replace `git fetch ... 2>/dev/null || true` + `git rev-parse --verify`
with `git ls-remote --exit-code --heads upstream translations_hotfix`.
ls-remote queries the remote directly so the check is never based on
stale local state, and real failures (auth, network) propagate through
set -e instead of being silently ignored. Applied to both the initial
branch setup and the pre-push re-fetch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: rewrite orchestrator to dispatch runner per hotfix branch via matrix
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: add per-branch runner workflow for hotfix translation sync
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: generalise sync script to use APP_NAME and GITHUB_REPOSITORY
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: remove push trigger from runner workflow
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: rename working branch to sync_translations_{hotfix_branch}
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: fix patch test
* ci: use v13 DB as starting point for patch
* ci: remove payments from patch test
* ci: print bench logs after tests
* ci: restart bench on each update
* ci: patch test v13db > v14 > develop
and when v15 is out v13db > v14 > v15 > develop
ci: use version specific payments repo (#34468)
ci: use version-14 branch of payments repo for v14 erpnext
(cherry picked from commit befd1a0f91)
Co-authored-by: Ritwik Puri <ritwikpuri5678@gmail.com>
