Compare commits

..

2 Commits

Author SHA1 Message Date
copilot-swe-agent[bot]
69075c7db0 fix(accounts): add account to GROUP BY for PostgreSQL compatibility, fix ruff formatting 2026-06-10 05:34:47 +00:00
copilot-swe-agent[bot]
ffd0dbdfd9 Initial plan 2026-06-10 05:29:00 +00:00
1152 changed files with 165361 additions and 445968 deletions

View File

@@ -1,248 +0,0 @@
# PostgreSQL compatibility — review guide
ERPNext targets **both MariaDB and PostgreSQL from a single codebase**. The full server
test suite passes on both, but the PostgreSQL CI job is **label-gated** (it does not run on
every PR), so until it is required this guide is the always-on guard. Greptile loads it as
review context (`.greptile/config.json`).
When reviewing a PR, flag any **new or changed query** (raw `frappe.db.sql`, `frappe.qb`,
`frappe.get_all/get_list/get_value`, report SQL) that would **error on PostgreSQL** or
**return different results on the two engines**.
## The one rule that governs everything
**MariaDB behaviour must not change; PostgreSQL is brought into line with MariaDB — never the
reverse.** A "fix" that changes the value, row count, or ordering MariaDB produced is a
regression, even if the new behaviour looks more correct. The only accepted MariaDB-output
change is replacing a genuinely *undefined/arbitrary* result with a deterministic one (row
count preserved) — and that should be called out explicitly.
There are two failure modes to watch for:
1. **Hard breaks** — PostgreSQL raises an exception; MariaDB is green. Easy to catch in CI,
but the gated job may not run.
2. **Silent divergences** — both engines succeed but return *different* results. CI on one
engine stays green; the bug only shows on a PostgreSQL site. These are the dangerous ones.
---
## 1. Hard breaks — would error on PostgreSQL
Flag a changed query that uses any of these:
- **Loose `GROUP BY`** — selecting/ordering a column that is neither in `GROUP BY` nor wrapped
in an aggregate. MariaDB tolerates it; PostgreSQL errors (`must appear in the GROUP BY
clause or be used in an aggregate function`). This **also covers an aggregate (`Sum`/`Count`/…)
selected alongside bare columns with NO `.groupby()` at all** — MariaDB silently collapses
every row into one arbitrary-valued row (often a *wrong-output* bug there too), PostgreSQL
errors. Fix: add the bare column to `GROUP BY` **if it is functionally dependent on the group
key**, otherwise wrap it in `Max()`/`Min()`. **See §3 — the row-count trap — before suggesting
"add it to GROUP BY".**
- **MySQL-only functions** — `TIMESTAMP(date,time)`, `TIMEDIFF`, `STR_TO_DATE`, `DATE_FORMAT`,
`DATE_ADD/SUB`, `GROUP_CONCAT`, `PERIOD_DIFF`, SQL `IF(cond,a,b)`. Use the portable
`frappe.query_builder.functions` equivalents (`CombineDatetime`, `DateDiff`, `Case`,
`GroupConcat`, …) or a precomputed column (e.g. `posting_datetime`).
- **`UPDATE … JOIN`** — not valid on PostgreSQL. Rewrite as `UPDATE … WHERE name IN (subquery)`.
- **`HAVING` referencing a `SELECT` alias** — PostgreSQL rejects output-column aliases in
`HAVING` (regardless of whether the query has a `GROUP BY`; MariaDB allows them). Repeat the
underlying expression in `HAVING`, or move a non-aggregate predicate into `WHERE`.
- **`SELECT DISTINCT … ORDER BY <expr not in the select list>`** — add the expr to the select
**only if it is single-valued per distinct row**; otherwise it grows the `DISTINCT` key and the
MariaDB row count (see §3) — drop the SQL `ORDER BY` and sort in Python instead.
- **Single-quoted column alias** `AS 'x'` — PostgreSQL reads `'x'` as a string literal. Use an
unquoted (or double-quoted) alias.
- **`varchar | varchar`** (bitwise OR misused as a coalesce) — errors on PostgreSQL. Use
`Coalesce(...)`.
- **Capital-cased identifiers** used as column/field names in `get_value(dt, dn, "Status")`,
`get_all(dt, fields=["Account"])`, and similar — PostgreSQL quotes the identifier and matches
it case-sensitively; a stored column named `status`/`account` won't match `"Status"`/`"Account"`
(`column "Account" does not exist`). Use the exact stored (lower-case) fieldname.
- **Boolean passed where an integer column is expected** — `frappe.db.set_value(dt, dn,
check_field, True)`, `doc.db_set(field, False)`, or `frappe.qb.update(dt).set(check_field, True)`
emit `SET col = true`, which PostgreSQL rejects on a `smallint`/`Check` column
(`column is of type smallint but expression is of type boolean`). Pass `1`/`0`.
- **`.like()`/`.ilike()` (or raw `LIKE`) on a NON-text column** — `idx`, `docstatus`, a date, etc.
frappe maps `.like()` → `ILIKE`, and PostgreSQL has no `bigint ILIKE text` operator (`operator
does not exist: bigint ~~* unknown`). Cast the column to text first — **`Cast_(col, "varchar")`**,
not `Cast(col, "char")` (see below). MariaDB coerces the int implicitly, so the cast is a no-op there.
- **`CAST(… AS CHAR)` / `Cast(x, "char")`** — on PostgreSQL bare `CHAR` is `character(1)`, so
`CAST(12 AS CHAR)` → `'1'` (silently truncates multi-digit values); MariaDB gives the full string.
Use `VARCHAR` / `Cast_(x, "varchar")`.
- **`.rlike()` / raw `RLIKE`** — frappe rewrites `REGEXP` → `~*` on PostgreSQL but does **not**
translate `RLIKE` (no such PostgreSQL operator). Use `.regexp()` (or `.like()` for a simple prefix).
- **`IfNull`/`Coalesce` of a typed column with a different-typed literal** — `IfNull(asset.disposal_date, 0)`
renders `COALESCE("disposal_date", 0)`, coalescing a **DATE** with an **integer**. PostgreSQL requires
`COALESCE` args to share a type (`DatatypeMismatch: COALESCE types date and integer cannot be matched`);
MariaDB's `IFNULL` is permissive. The common shape is `IfNull(date_col, 0) != 0 / == 0` as a presence test —
replace with `date_col.isnotnull()` / `date_col.isnull()` (identical, and valid on both). Otherwise coalesce
to a **same-type** default (`Coalesce(date_col, '1900-01-01')`, `Coalesce(text_col, '')`).
- **Division by a possibly-zero divisor** — `Sum(a) / Sum(b)`, `x / col`, etc. where the
divisor can be `0`/empty. MariaDB returns `NULL` for division by zero; PostgreSQL raises
`division by zero` and aborts the query. Wrap the divisor in `NullIf(divisor, 0)` — that
yields `NULL` on both engines, matching MariaDB's value. (Only the *literal* `/ 0` is a parse
constant; the trap is a divisor that is an aggregate or column the data can drive to zero.)
---
## 2. Silent divergences — succeeds on both, returns different results
These don't error, so a one-engine CI stays green. Flag them:
- **Case sensitivity on text equality** — `==`, `.isin()`, `Strpos`/`Locate` on free-text
columns are case-**sensitive** on PostgreSQL but case-**insensitive** under MariaDB's default
collation. `Lower()` both sides. *(Not `.like()`/`["like", …]` — those already render as
`ILIKE` on PostgreSQL; see §4.)*
- **Case sensitivity in a doc-`name` lookup** — lower-casing a value then using it as a
document name in `get_value`/`get_doc`/`exists` misses on PostgreSQL (names are
case-sensitive). Keep original case for the identifier; lower-case only comparison operands.
- **Empty string vs NULL** — PostgreSQL stores a blank link/data field as `NULL` on some paths
while MariaDB keeps `''`; `Concat`/`Concat_ws` then diverge. Prefer the stored full value, or
`Coalesce(col, '')` per argument.
- **NULL ordering** — MariaDB sorts `NULL` first, PostgreSQL sorts it last. For
`ORDER BY … LIMIT 1`/`[0]` on a nullable column, guard with `Coalesce`/`isnotnull()`.
- **`ORDER BY … LIMIT 1` with no unique tiebreaker** — when rows tie on the ordered column the
two engines may pick different rows. Add a `creation`/`name` tiebreaker **only if it does not
change MariaDB's current pick** (see §4).
- **Integer division** — `int / int` truncates on PostgreSQL but is decimal on MariaDB, e.g.
`COUNT(...) / COUNT(...) * 100` → `0`, or `manufacturing_time_in_mins / 1440` flooring a
lead-time to whole days. Force float: multiply by `100.0`, or make a literal a float
(`/ 1440` → `/ 1440.0`), or cast an operand. (Only SQL-level `/` on integer **columns/literals**
— Python `/` is already float.)
- **`DISTINCT` list ordering** — `frappe.get_all(distinct=True, order_by=…)` /
`SELECT DISTINCT … ORDER BY`: frappe's `db_query` **silently drops `ORDER BY` for distinct
queries on PostgreSQL**, so the result is unordered there. Sort in Python instead — and use
`key=str.casefold`, because bare `sorted()` is case-sensitive (ASCII) while MariaDB's
collation is case-insensitive, so a plain sort reorders MariaDB's output.
- **Engine-specific function rewrites** — e.g. a PostgreSQL `regexp_replace` branch
reimplementing MariaDB's `CAST(SUBSTRING_INDEX(name,' ',-1) AS UNSIGNED)` (leading digits of
the last whitespace token). Verify the rewrite matches MariaDB on edge cases (`"X - 3a"→3`,
`"X - 1.5"→1`) by diffing both engines on literal rows.
- **`UnixTimestamp(date)` / date→epoch** is timezone-dependent (midnight in the DB session TZ),
so a strict `epoch <= now` bound is flaky on PostgreSQL.
---
## 3. The row-count trap — `GROUP BY` **and** `DISTINCT` (the single most important rule)
When making a loose `GROUP BY` PostgreSQL-valid, **do not add a non-functionally-dependent
column to the `GROUP BY` just to satisfy PostgreSQL** — that turns one group row into N and
**changes the MariaDB row count** (a regression). The classic traps are adding the **child/row
primary key** or an **editable per-row field**. Instead **`Max()`/`Min()`-wrap** the offending
column: the row count is preserved and the value goes from arbitrary (MariaDB's old loose pick)
to deterministic.
**Judge functional dependence by the source table, not the column name:**
- A column from a **master joined on the group key** (`t3.x` where `t1.key = t3.name`) is FD →
safe to keep in `GROUP BY`.
- A descriptive field on the **transaction** table (`t1.supplier_name`, `t1.territory`,
`t1.item_name` — fetched/editable, can differ across historical rows for the same key) is
**not** FD even though it looks master-derived → `Max()`-wrap it.
Conversely, do **not** suggest changing a `Max()`/`Min()`-wrapped column to `Sum()` (or vice
versa) to make a number "more correct" — that changes the MariaDB value. The wrap reproduces
MariaDB's prior one-value-per-group output; a different aggregate is a product change, out of
scope for a portability fix.
**The same trap applies to `SELECT DISTINCT`.** To satisfy PostgreSQL's "an `ORDER BY` expr must
appear in the select list under `DISTINCT`" rule, **do not blindly add the ordered column to the
select** — if it is not single-valued per existing distinct row, the `DISTINCT` key grows and
MariaDB returns **more rows** (a regression), exactly as adding a non-FD column to `GROUP BY` does.
Add it only when it is functionally dependent on the existing select columns; otherwise drop the
SQL `ORDER BY` and **sort in Python** (`key=str.casefold`, per §2) so the distinct row set is
unchanged.
### 3.1 Second-order traps — when the `Max()`/`Min()` wrap itself is the bug
The wrap is only a no-op when the column is provably single-valued per group (**"`Max()` means
provably constant"**). When the column can genuinely vary, the wrap is a decision, and a full
audit of these fixes found four recurring mistakes:
- **Incoherent pair** — two semantically-coupled columns (a flag + a link:
`is_phantom_item` + `bom_no`; a discriminator + its value) aggregated with *independent*
`Max()`/`Min()` can pair values from **different rows** — a chimera row that never existed.
MariaDB's loose pick was at least row-coherent. Fix: group by the pair (when consumers
tolerate the extra rows), or select one **representative row** (`Min(child.name)` subquery +
join-back) so every column comes from the same line.
- **NULL-skipping** — `MAX`/`MIN` ignore NULLs, so `Max()` over a mostly-NULL discriminator
(an `original_item`-style column) *deterministically* returns the non-NULL value where
MariaDB could return NULL — deterministically wrong where the old behavior was only
intermittently wrong. Flag it wherever "no value" is a meaningful state (fallback gates,
dict keys).
- **Fabricated arithmetic** — `Sum(x) * Max(y)` where `y` varies within the group invents a
number no row ever had (and `Max` biases it upward) — poisonous when it feeds validation,
budgets, valuation, or GL/stock values. Fix per-row: `Sum(x * y)`.
- **Wrong bound** — where the value has a semantic, pick the bound deliberately:
`Min(schedule_date)` for a "required by", `Min(idx)` for first-line ordering, a qty-weighted
average for a rate. A blind `Max` can understate urgency or overstate a figure.
Review heuristic: **if choosing between `Max` and `Min` would change the answer, the column is
not functionally dependent** — wrapping either is the wrong fix. Group by it, restructure, or
pick a bound for a stated reason, and cover the varying-group case with a test.
---
## 4. False positives — do NOT flag these
These are auto-handled by the framework and are **not** breaks:
- **`.like()` / `["like", …]`** already renders as `ILIKE` on PostgreSQL — not a
case-sensitivity bug. *(Exception: `.like()` on a **non-text** column — `idx`, `docstatus` —
is a hard break, `bigint ILIKE`; see §1.)*
- **Raw `ifnull(...)`** inside `frappe.db.sql()` is rewritten to `coalesce(...)` on all engines.
- **Backticks**, **`LOCATE`**, **`REGEXP`** / **`.regexp()`** in raw SQL are auto-translated on
PostgreSQL (`REGEXP` → `~*`). **But `RLIKE` / `.rlike()` is NOT translated** — that one is a
hard break (see §1).
- **An `ORDER BY … LIMIT 1` tie where the two engines already agree**, or where adding a
tiebreaker would *change* MariaDB's current pick — leave it; "fixing" it would either change
MariaDB or has no observable effect.
---
## 5. Transaction / runtime (not query-shape, still PostgreSQL-only)
- **Catch-and-continue inserts** — on PostgreSQL a failed `insert()` aborts the **whole
transaction**, so code that swallows a duplicate and keeps going dies on the next statement
with `InFailedSqlTransaction` (frappe dropped its blanket per-statement savepoint in
frappe#40075). Such a handler must wrap the fallible insert in `frappe.db.savepoint(name)` +
`rollback(save_point=name)` — unless it re-`throw`s with no DB call before the throw, or the
insert uses `ignore_if_duplicate=True` / `autoname="hash"` (→ `ON CONFLICT DO NOTHING`).
- **Recover the txn with a *scoped* savepoint, not a full `frappe.db.rollback()`, if any prior work
must survive.** A full rollback un-poisons the txn but also discards every row the handler committed
*before* the failure — which MariaDB kept (it has no statement-abort), so it's a **silent MariaDB
regression**. **"The background job / whitelist entrypoint owns the txn" does NOT make a full rollback
safe** if it did multiple inserts in a loop first — it drops the partial results MariaDB retained. A
full rollback is safe only when it (a) immediately re-`throw`s/`raise`s (MariaDB rolls back anyway),
(b) has nothing successful before it (a single op), or (c) the batch is genuinely meant to be
**atomic** (a partial result is an invalid state → rollback + mark *Failed* is correct). Otherwise use
a **per-iteration / per-record savepoint** — and keep the function's success/`None` return contract:
do **not** return the doc when the savepoint was rolled back.
---
## 6. Refactors and raw-SQL→ORM conversions are not automatically 1:1
A commit labeled a **refactor** or a **raw-`frappe.db.sql` → `frappe.qb`/ORM conversion** is meant
to preserve behaviour — but it easily doesn't, and the change passes the static checker and a
one-engine green run. **Diff the `WHERE`/predicate, the `JOIN`/`ON` conditions, and the resulting
row set — not just the `SELECT` shape.** A conversion that silently widens or narrows the filter
changes the rows touched on **both** engines and is a regression hiding under a "refactor" label.
Real example: an `UPDATE` whose bound was `posting_datetime > X` gained an
`OR (posting_datetime == X AND creation > args.creation)` branch during a "`sql` → `qb` refactor",
widening the rows updated on both engines. Even when such a change is a deliberate bug-fix it must
be called out and tested — it is **not** the no-op the refactor label implies. Confirm the
converted query touches exactly the same rows with the same values MariaDB produced before.
---
## How to review
For every changed query: does it (a) use a construct from §1 (would error on PostgreSQL),
(b) match a divergence in §2/§3 (different result across engines), or (c) change the row set under
a refactor/conversion label (§6)? If so, comment with the
portable fix and confirm it leaves **MariaDB output unchanged**. Skip the §4 false positives.
Prefer a comment that names the rule (e.g. "loose GROUP BY — Max()-wrap, don't add to GROUP BY:
splits the row count") so the fix is unambiguous.
The static pre-commit checker (`.github/helper/postgres_compat.py`) catches the *mechanical*
§1 breaks; the **semantic** §2/§3 divergences and the §6 refactor/conversion row-set changes are
exactly what a reviewer (and this guide) must cover, because no static check can see them.

View File

@@ -1,72 +0,0 @@
#!/bin/bash
#
# Hydrate a test shard from the setup job's artifact.
#
# The bench (apps, venv, node_modules, sites) is already on disk at ~/frappe-bench — the
# workflow untar'd it from the artifact the setup job built. So there is NO bench init, no
# asset build, and no reinstall here: just bring the DB up on the baked datadir and start redis
# so tests can run. The whole point is that the expensive work happened ONCE in the setup job.
#
set -e
ci_user="${ERPNEXT_CI_USER:-frappe}"
db_host="${DB_HOST:-127.0.0.1}"
# Re-exec as the ci user (uid 1001) so bench/cache ownership matches the artifact, same as
# install.sh. The workflow untar'd as root with -p, so the files are already owned by ci.
if [ "$(id -u)" = "0" ] && [ "${SKIP_SYSTEM_SETUP:-0}" = "1" ] && [ "$ci_user" != "root" ]; then
exec su -m "$ci_user" -s /bin/bash -c \
"ERPNEXT_CI_USER='$ci_user' DB_HOST='$db_host' DB='${DB:-}' bash '$0'"
fi
cd ~/frappe-bench
# Start the DB on the datadir baked into the artifact. It's already populated (the setup job
# reinstalled into this very datadir), so there is NO restore — the server comes up on the
# existing files. This is what replaces the per-shard SQL replay.
bash ~/frappe-bench/start-db.sh
# Bring up redis (lightmode unit tests need cache + queue). In the self-hosted container we use the
# full `bench start` (web/workers too, like install.sh). On the bare GitHub Postgres shard
# `bench start` (honcho) lagged — it blocks the redis procs behind web/worker procs the lightmode
# suite never uses, so the wait below burned its full timeout (~4m). There, start the two redis
# instances directly: fast and deterministic.
if [ "${DB:-mariadb}" = "postgres" ]; then
# Start redis directly as daemons — reliable and persists across steps. Do NOT route it through
# `bench start`: honcho tears the whole process group down if any one Procfile proc dies on the
# bare shard, which took redis with it (redis @ 13000 refused in Run Tests). Keeping redis
# independent is what makes it survive. The web server (for PDF tests) is NOT started here — a
# backgrounded server doesn't survive into the next step; it's started inside the Run Tests step.
for conf in redis_cache redis_queue; do
[ -f ~/frappe-bench/config/$conf.conf ] && redis-server ~/frappe-bench/config/$conf.conf --daemonize yes
done
else
bench start >> ~/frappe-bench/bench_start.log 2>&1 &
fi
# Wait for redis, failing fast instead of silently burning minutes if it never comes up.
cfg=~/frappe-bench/sites/common_site_config.json
if [ -f "$cfg" ]; then
ports=$(python - "$cfg" <<'PY'
import json, re, sys
try:
cfg = json.load(open(sys.argv[1]))
except Exception:
sys.exit(0)
for key in ("redis_cache", "redis_queue"):
m = re.search(r":(\d+)", str(cfg.get(key, "")))
if m:
print(m.group(1))
PY
)
for port in $ports; do
up=0
for _ in $(seq 1 60); do
if (exec 3<>"/dev/tcp/127.0.0.1/$port") 2>/dev/null; then exec 3>&- 3<&-; up=1; break; fi
sleep 1
done
[ "$up" = "1" ] || { echo "redis did not come up on port $port"; exit 1; }
done
fi
echo "Hydrated: DB up on baked datadir, redis up — ready for tests."

View File

@@ -4,356 +4,75 @@ set -e
cd ~ || exit
sudo apt update
sudo apt remove mysql-server mysql-client
sudo apt install libcups2-dev redis-server mariadb-client libmariadb-dev
pip install frappe-bench
githubbranch=${GITHUB_BASE_REF:-${GITHUB_REF##*/}}
frappeuser=${FRAPPE_USER:-"frappe"}
frappecommitish=${FRAPPE_BRANCH:-$githubbranch}
db_host=${DB_HOST:-"127.0.0.1"}
db_user_host=${DB_USER_HOST:-"localhost"}
wkhtmltox_deb=${WKHTMLTOX_DEB:-"/tmp/wkhtmltox.deb"}
bench_cache_dir=${BENCH_CACHE_DIR:-}
run_as_ci_user_if_needed() {
if [ "$(id -u)" != "0" ] || [ "${SKIP_SYSTEM_SETUP:-0}" != "1" ] || [ "${ERPNEXT_CI_NON_ROOT:-0}" = "1" ]; then
return
fi
local missing_packages=()
if ! command -v pkg-config >/dev/null 2>&1; then
missing_packages+=("pkg-config")
fi
if ! command -v mariadb_config >/dev/null 2>&1 && ! command -v mysql_config >/dev/null 2>&1; then
missing_packages+=("libmariadb-dev")
fi
if ! command -v crontab >/dev/null 2>&1; then
missing_packages+=("cron")
fi
if [ "${#missing_packages[@]}" -gt 0 ]; then
apt-get update
apt-get install -y --no-install-recommends "${missing_packages[@]}"
fi
local ci_user="${ERPNEXT_CI_USER:-frappe}"
if ! id "$ci_user" >/dev/null 2>&1; then
useradd --home-dir "$HOME" --no-create-home --shell /bin/bash "$ci_user"
fi
rm -rf ~/frappe ~/frappe-bench
local ci_dirs=(
"$HOME"
"$GITHUB_WORKSPACE"
"$HOME/.cache"
"${PIP_CACHE_DIR:-$HOME/.cache/pip}"
"${npm_config_cache:-$HOME/.npm}"
"${YARN_CACHE_FOLDER:-$HOME/.cache/yarn}"
"$HOME/.yarn"
"${UV_CACHE_DIR:-$HOME/.cache/uv}"
"$(dirname "$wkhtmltox_deb")"
)
if [ -n "$bench_cache_dir" ]; then
ci_dirs+=("$bench_cache_dir")
fi
# Create + own (non-recursively) the home/cache/workspace dirs before dropping to
# the ci user. We deliberately do NOT wipe the yarn/uv caches here so a persistent
# cache (mounted volume or baked image layer) stays warm across runs.
mkdir -p "${ci_dirs[@]}" "$HOME/.yarn"
chown "$ci_user:$ci_user" "${ci_dirs[@]}" "$HOME/.yarn"
export ERPNEXT_CI_NON_ROOT=1
exec su -m "$ci_user" -s /bin/bash -c "cd '$HOME' && bash '$GITHUB_WORKSPACE/.github/helper/install.sh'"
}
run_as_ci_user_if_needed
run_ci_step() {
local label=$1
shift
echo "::group::${label}"
date -u
local exit_code=0
timeout --foreground "${CI_INSTALL_STEP_TIMEOUT:-1800}" "$@" || exit_code=$?
date -u
echo "::endgroup::"
return "$exit_code"
}
if [ -n "${GITHUB_WORKSPACE:-}" ]; then
git config --global --add safe.directory "$GITHUB_WORKSPACE" || true
git config --global --add safe.directory "$GITHUB_WORKSPACE/.git" || true
fi
rm -rf ~/frappe ~/frappe-bench
# ---------------------------------------------------------------------------
# Phase 1 — parallelise the three slow, independent setup steps:
# a) system packages b) frappe-bench pip install c) frappe git fetch
# ---------------------------------------------------------------------------
if [ "${SKIP_SYSTEM_SETUP:-0}" != "1" ]; then
sudo apt-get update
# apt remove/install must run sequentially but can overlap with pip and git.
sudo apt-get remove -y mysql-server mysql-client
sudo apt-get install -y libcups2-dev redis-server mariadb-client libmariadb-dev &
apt_pid=$!
pip install frappe-bench &
pip_pid=$!
else
apt_pid=
pip_pid=
fi
mkdir frappe
(
cd frappe
git init
git remote add origin "https://github.com/${frappeuser}/frappe"
git fetch origin "${frappecommitish}" --depth 1
) &
clone_pid=$!
if [ -n "$apt_pid" ]; then wait $apt_pid; fi
if [ -n "$pip_pid" ]; then wait $pip_pid; fi
wait $clone_pid
pushd frappe
git init
git remote add origin "https://github.com/${frappeuser}/frappe"
git fetch origin "${frappecommitish}" --depth 1
git checkout FETCH_HEAD
popd
frappe_sha=$(git -C frappe rev-parse HEAD)
get_bench_cache_archive() {
if [ -z "$bench_cache_dir" ]; then
return
fi
bench init --skip-assets --frappe-path ~/frappe --python "$(which python)" frappe-bench
mkdir -p "$bench_cache_dir"
# Keyed on tool versions only (NOT the frappe SHA): any recent base bench works, because
# restore_warm_bench fast-forwards it to the exact live develop SHA. This is what lets a
# constantly-moving develop still hit the cache.
local cache_key
cache_key=$(
{
uname -m
python --version
node --version
bench --version
} | sha256sum | awk '{print $1}'
)
echo "${bench_cache_dir}/frappe-bench-base-${cache_key}.tar.zst"
}
restore_warm_bench() {
bench_cache_archive=$(get_bench_cache_archive)
[ -n "$bench_cache_archive" ] && [ -f "$bench_cache_archive" ] || return 1
echo "Restoring base bench from ${bench_cache_archive}"
tar --use-compress-program=unzstd -xf "$bench_cache_archive" -C ~ || return 1
[ -d ~/frappe-bench/apps/frappe/.git ] || return 1
mkdir -p ~/frappe-bench/sites ~/frappe-bench/logs
[ -f ~/frappe-bench/sites/apps.txt ] || printf "frappe\n" > ~/frappe-bench/sites/apps.txt
[ -f ~/frappe-bench/sites/common_site_config.json ] || printf "{}\n" > ~/frappe-bench/sites/common_site_config.json
# Fast-forward the restored frappe to the EXACT live develop SHA fetched in phase 1, then
# rebuild only what changed. The editable install means the venv tracks the new code with
# no reinstall. Any failure returns non-zero so the caller falls back to a full bench init.
if ! (
cd ~/frappe-bench/apps/frappe || exit 1
# Phase 1 already fetched ~/frappe to the exact live develop SHA. Fetch that commit
# straight from it (bench init names the remote 'upstream', not 'origin', and points
# it at this local clone — so a plain `git fetch origin` does not work).
git fetch --no-tags "$HOME/frappe" HEAD || exit 1
git checkout --force FETCH_HEAD || exit 1
); then
echo "Fast-forward to ${frappe_sha} failed; falling back to full init"
rm -rf ~/frappe-bench
return 1
fi
# Pick up any frappe dependency changes since the base was built (cached → fast if none),
# so a develop commit that bumped requirements doesn't leave a stale venv.
if ! ~/frappe-bench/env/bin/python -m pip install -q -e ~/frappe-bench/apps/frappe; then
echo "frappe dependency refresh failed; falling back to full init"
rm -rf ~/frappe-bench
return 1
fi
( cd ~/frappe-bench && CI=Yes bench build --app frappe ) || { rm -rf ~/frappe-bench; return 1; }
return 0
}
save_warm_bench() {
if [ -z "${bench_cache_archive:-}" ] || [ -f "$bench_cache_archive" ]; then
return
fi
if [ -n "$bench_cache_dir" ] && [ ! -w "$bench_cache_dir" ]; then
echo "Skipping warm bench save because ${bench_cache_dir} is not writable"
return
fi
local tmp_archive
tmp_archive="${bench_cache_archive}.${$}.tmp"
echo "Saving warm bench to ${bench_cache_archive}"
# Keep sites/common_site_config.json (the redis ports live there — dropping it makes the
# restore path fall back to a default redis port that bench start never bound, so reinstall
# fails with "redis ... connection refused"). Only the rebuildable sites/assets is excluded;
# restore_warm_bench runs `bench build` to regenerate it.
tar \
--use-compress-program="zstd -T0 -3" \
--exclude="frappe-bench/logs" \
--exclude="frappe-bench/sites/assets" \
-cf "$tmp_archive" \
-C ~ frappe-bench
mv "$tmp_archive" "$bench_cache_archive"
}
# ---------------------------------------------------------------------------
# Phase 2 — bench init and site setup
# ---------------------------------------------------------------------------
install_whktml() {
# Re-use the .deb if the wkhtmltopdf cache step already restored it.
if [ ! -f "$wkhtmltox_deb" ]; then
wget -O "$wkhtmltox_deb" https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6.1-2/wkhtmltox_0.12.6.1-2.jammy_amd64.deb
fi
sudo apt-get install -y "$wkhtmltox_deb"
}
if [ "${SKIP_WKHTMLTOX_SETUP:-0}" != "1" ]; then
install_whktml &
wkpid=$!
else
wkpid=
fi
if ! restore_warm_bench; then
bench init --skip-assets --frappe-path ~/frappe --python "$(which python)" frappe-bench
cd ~/frappe-bench || exit
sed -i 's/watch:/# watch:/g' Procfile
sed -i 's/schedule:/# schedule:/g' Procfile
sed -i 's/socketio:/# socketio:/g' Procfile
sed -i 's/redis_socketio:/# redis_socketio:/g' Procfile
CI=Yes bench build --app frappe
save_warm_bench
fi
if [ -n "$wkpid" ]; then wait $wkpid; fi
mkdir -p ~/frappe-bench/sites/test_site
mkdir ~/frappe-bench/sites/test_site
if [ "$DB" == "mariadb" ];then
cp -r "${GITHUB_WORKSPACE}/.github/helper/site_config_mariadb.json" ~/frappe-bench/sites/test_site/site_config.json
if [ "$db_host" != "127.0.0.1" ]; then
sed -i "s/\"db_host\": \"127.0.0.1\"/\"db_host\": \"${db_host}\"/" ~/frappe-bench/sites/test_site/site_config.json
fi
else
cp -r "${GITHUB_WORKSPACE}/.github/helper/site_config_postgres.json" ~/frappe-bench/sites/test_site/site_config.json
fi
if [ "$DB" == "mariadb" ];then
for _ in {1..60}; do
if mariadb-admin ping --host "$db_host" --port 3306 -u root -proot --silent; then
break
fi
sleep 1
done
mariadb-admin ping --host "$db_host" --port 3306 -u root -proot --silent
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "SET GLOBAL character_set_server = 'utf8mb4'"
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "SET GLOBAL collation_server = 'utf8mb4_unicode_ci'"
mariadb --host "$db_host" --port 3306 -u root -proot -e "SET GLOBAL character_set_server = 'utf8mb4'"
mariadb --host "$db_host" --port 3306 -u root -proot -e "SET GLOBAL collation_server = 'utf8mb4_unicode_ci'"
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "CREATE USER 'test_frappe'@'localhost' IDENTIFIED BY 'test_frappe'"
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "CREATE DATABASE test_frappe"
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "GRANT ALL PRIVILEGES ON \`test_frappe\`.* TO 'test_frappe'@'localhost'"
# Throwaway-DB durability tuning at runtime. (innodb_doublewrite is read-only on MariaDB
# 10.6, so it can't be disabled here — would need a server startup flag.)
mariadb --host "$db_host" --port 3306 -u root -proot \
-e "SET GLOBAL innodb_flush_log_at_trx_commit=0; SET GLOBAL sync_binlog=0;"
# Opt-in DDL speedup: a shared tablespace avoids a create+fsync per DocType table during
# reinstall — a big win under disk contention. But ROW_FORMAT=DYNAMIC must be accepted in
# the system tablespace on this MariaDB. Enable with CI_INNODB_SHARED_TABLESPACE=1; if
# reinstall then errors on table creation, unset it (off by default — zero risk).
if [ "${CI_INNODB_SHARED_TABLESPACE:-0}" = "1" ]; then
mariadb --host "$db_host" --port 3306 -u root -proot -e "SET GLOBAL innodb_file_per_table=0;"
fi
mariadb --host "$db_host" --port 3306 -u root -proot -e "CREATE USER 'test_frappe'@'${db_user_host}' IDENTIFIED BY 'test_frappe'"
mariadb --host "$db_host" --port 3306 -u root -proot -e "CREATE DATABASE test_frappe"
mariadb --host "$db_host" --port 3306 -u root -proot -e "GRANT ALL PRIVILEGES ON \`test_frappe\`.* TO 'test_frappe'@'${db_user_host}'"
mariadb --host "$db_host" --port 3306 -u root -proot -e "FLUSH PRIVILEGES"
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "FLUSH PRIVILEGES"
fi
if [ "$DB" == "postgres" ];then
echo "travis" | psql -h 127.0.0.1 -p 5432 -c "CREATE DATABASE test_frappe" -U postgres;
echo "travis" | psql -h 127.0.0.1 -p 5432 -c "CREATE USER test_frappe WITH PASSWORD 'test_frappe'" -U postgres;
# Durability-off for speed (no fsync/synchronous_commit/full_page_writes) is applied by
# start-db.sh's postgres `-o` flags on every start — setup job AND each test shard — so it is
# NOT repeated here. The postgres workflow runs in-runner via start-db.sh, not a service
# container.
fi
install_whktml() {
wget -O /tmp/wkhtmltox.deb https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6.1-2/wkhtmltox_0.12.6.1-2.jammy_amd64.deb
sudo apt install /tmp/wkhtmltox.deb
}
install_whktml &
wkpid=$!
cd ~/frappe-bench || exit
run_ci_step "Get payments app" bench get-app payments --branch develop
sed -i 's/watch:/# watch:/g' Procfile
sed -i 's/schedule:/# schedule:/g' Procfile
sed -i 's/socketio:/# socketio:/g' Procfile
sed -i 's/redis_socketio:/# redis_socketio:/g' Procfile
# Opt-in: skip building erpnext's frontend assets. Server tests don't need them, but PDF
# tests (print formats) do — they pass only if the PDF renderer ignores missing assets.
# Enable with CI_SKIP_ERPNEXT_ASSETS=1 to test; if PDF tests fail, unset it.
erpnext_get_app_args=()
if [ "${CI_SKIP_ERPNEXT_ASSETS:-0}" = "1" ]; then erpnext_get_app_args=(--skip-assets); fi
run_ci_step "Get erpnext app" bench get-app erpnext "${GITHUB_WORKSPACE}" "${erpnext_get_app_args[@]}"
bench get-app payments --branch develop
bench get-app erpnext "${GITHUB_WORKSPACE}"
if [ "$TYPE" == "server" ]; then run_ci_step "Setup dev requirements" bench setup requirements --dev; fi
if [ "$TYPE" == "server" ]; then bench setup requirements --dev; fi
bench start >> ~/frappe-bench/bench_start.log 2>&1 &
wait $wkpid
# Under heavy concurrency, gunicorn's startup can delay redis coming up. reinstall and the
# tests need redis, so wait for it (best-effort, bounded) instead of racing — contention
# then slows the job rather than failing it.
wait_for_redis() {
local cfg=~/frappe-bench/sites/common_site_config.json
[ -f "$cfg" ] || return 0
local ports port
ports=$(python - "$cfg" <<'PY'
import json, re, sys
try:
cfg = json.load(open(sys.argv[1]))
except Exception:
sys.exit(0)
for key in ("redis_cache", "redis_queue"):
match = re.search(r":(\d+)", str(cfg.get(key, "")))
if match:
print(match.group(1))
PY
)
for port in $ports; do
local up=0
for _ in $(seq 1 120); do
if (exec 3<>"/dev/tcp/127.0.0.1/$port") 2>/dev/null; then
exec 3>&- 3<&-; up=1
break
fi
sleep 1
done
# Fail clearly instead of letting reinstall die later on a vague socket-connection error
# when redis never bound.
[ "$up" = "1" ] || { echo "redis did not come up on port $port"; return 1; }
done
}
wait_for_redis
# Site setup: build the schema (~1000 DocTypes) into the DB. This is the single-threaded-Python
# bottleneck, but the fan-out amortises it — it runs once here in the setup job, and the test
# shards start the DB on the baked datadir instead of repeating the reinstall.
run_ci_step "Reinstall test site" bench --site test_site reinstall --yes
bench start &>> ~/frappe-bench/bench_start.log &
CI=Yes bench build --app frappe &
bench --site test_site reinstall --yes

View File

@@ -1,241 +0,0 @@
#!/usr/bin/env python3
"""Static guard against MySQL-only SQL that breaks on PostgreSQL.
The Postgres test job is label-gated, so it does not run on every PR. This pre-commit
hook is the always-on first line of defence: it flags the *mechanical* Postgres breaks
that static analysis can catch reliably with a low false-positive rate.
It deliberately does NOT try to catch the *semantic* divergences (loose GROUP BY,
case-sensitive ==/IN, NULL ordering, ORDER BY ... LIMIT 1 tiebreakers, integer-division
intent, division by a possibly-zero divisor, savepoint discipline) — those genuinely need
the test suite or a human/Greptile reviewer. Run the full suite on a Postgres site for those.
Escape hatch: put `# pg-ok` anywhere on the offending statement's line span (e.g. on a
`SHOW INDEX` query that lives inside an `if frappe.db.db_type == "mariadb":` branch).
Usage: postgres_compat.py <file.py> [<file.py> ...] (pre-commit passes staged files)
"""
from __future__ import annotations
import ast
import re
import sys
IGNORE = "pg-ok"
# Strings are only scanned for the patterns below when they have real SQL *structure*
# (not just an English word like "select" or "from"), to keep false positives near zero.
SQL_HINT = re.compile(
r"\bselect\b[\s\S]{0,800}\bfrom\b" # SELECT ... FROM
r"|\bupdate\b[\s\S]{0,400}\bset\b" # UPDATE ... SET
r"|\bdelete\s+from\b"
r"|\binsert\s+into\b"
r"|\bshow\s+(?:index|tables|columns)\b"
r"|\bfrom\s+[\"'`]?tab", # FROM `tabDocType`
re.I,
)
# MySQL-only constructs with NO frappe auto-translation. (frappe.db.sql already rewrites
# ifnull->coalesce on all engines and backtick/locate/REGEXP on Postgres, and .like()
# renders ILIKE — so those are NOT listed here; flagging them would be false positives.)
SQL_PATTERNS: list[tuple[re.Pattern, str]] = [
(re.compile(r"\btimestamp\s*\(\s*[^,()]+,", re.I),
"timestamp(date, time) is MySQL-only -> use CombineDatetime() or a precomputed datetime column"),
(re.compile(r"\btimediff\s*\(", re.I),
"timediff() is MySQL-only -> compute the delta in Python"),
(re.compile(r"\bstr_to_date\s*\(", re.I),
"str_to_date() is MySQL-only -> parse in Python and pass a real date"),
(re.compile(r"\bdate_format\s*\(", re.I),
"date_format() is MySQL-only -> filter on a date range instead"),
(re.compile(r"\bdate_(add|sub)\s*\(", re.I),
"date_add()/date_sub() are MySQL-only -> use Python date math or interval arithmetic"),
(re.compile(r"\bgroup_concat\s*\(", re.I),
"group_concat() is MySQL-only -> use GroupConcat (string_agg) or aggregate in Python"),
(re.compile(r"\bperiod_diff\s*\(", re.I),
"period_diff() is MySQL-only -> compute in Python"),
(re.compile(r"\bshow\s+index\b", re.I),
"SHOW INDEX is MySQL-only -> use frappe.db.has_index() / get_column_index()"),
(re.compile(r"\bshow\s+(tables|columns)\b", re.I),
"SHOW TABLES/COLUMNS is MySQL-only -> use frappe.db.get_tables()/table_columns / information-schema helpers"),
(re.compile(r"\bas\s+'[^']+'", re.I),
"single-quoted column alias breaks on Postgres -> use a bare or double-quoted alias"),
(re.compile(r"\bif\s*\(", re.I),
"SQL IF() is MySQL-only -> use CASE WHEN ... THEN ... ELSE ... END (frappe.qb.Case())"),
(re.compile(r"\brlike\b", re.I),
"RLIKE is MySQL-only -> frappe rewrites REGEXP->~* on Postgres but NOT RLIKE; use REGEXP / .regexp() / ~"),
(re.compile(r"\bcast\s*\(.+?\bas\s+char\b", re.I | re.S), # .+? spans nested parens, e.g. CAST(ABS(x) AS CHAR)
"CAST(... AS CHAR) is character(1) on Postgres and truncates -> CAST AS VARCHAR (frappe Cast_(x, 'varchar'))"),
]
# UPDATE ... JOIN: both keywords in the same SQL string.
UPDATE_JOIN = (re.compile(r"\bupdate\b", re.I), re.compile(r"\bjoin\b", re.I))
MYSQL_RESULT_KEYS = {"Column_name", "Key_name", "Seq_in_index", "Non_unique", "Index_type"}
SET_BOOL_FUNCS = {"set_value", "db_set"}
# query-builder cast helpers: pypika Cast / frappe Cast_. A "char" target type is character(1)
# on Postgres (truncates); "varchar" is the full-length cast.
CAST_FUNCS = {"Cast", "Cast_"}
# frappe.get_all / get_list: frappe's db_query SILENTLY drops ORDER BY for `distinct` queries on
# Postgres (the ORDER BY column must appear in the SELECT-DISTINCT list), so `distinct=True` together
# with a literal `order_by` is a no-op on PG and the result comes back unordered.
DISTINCT_ORDER_FUNCS = {"get_all", "get_list"}
def _docstring_ids(tree: ast.AST) -> set[int]:
"""ids of Constant nodes that are docstrings (so prose describing the rules isn't flagged)."""
ids: set[int] = set()
for node in ast.walk(tree):
if isinstance(node, (ast.Module, ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
body = getattr(node, "body", None)
if body and isinstance(body[0], ast.Expr) and isinstance(body[0].value, ast.Constant) and isinstance(body[0].value.value, str):
ids.add(id(body[0].value))
return ids
class Visitor(ast.NodeVisitor):
def __init__(self, lines: list[str], docstrings: set[int]):
self.lines = lines
self.docstrings = docstrings
self.violations: list[tuple[int, str]] = []
def _ignored(self, node: ast.AST) -> bool:
start = getattr(node, "lineno", 1)
end = getattr(node, "end_lineno", start) or start
# honour `# pg-ok` anywhere on the node's line span, the line just above (the enclosing
# call, e.g. `frappe.db.sql( # pg-ok`), or the line just below (a multi-line call's `) # pg-ok`).
lo = max(0, start - 2)
return any(IGNORE in self.lines[i] for i in range(lo, min(end + 1, len(self.lines))))
def _flag(self, node: ast.AST, msg: str) -> None:
if not self._ignored(node):
self.violations.append((getattr(node, "lineno", 1), msg))
def _scan_sql(self, text: str, node: ast.AST) -> None:
if not SQL_HINT.search(text):
return
for pattern, msg in SQL_PATTERNS:
if pattern.search(text):
self._flag(node, msg)
if UPDATE_JOIN[0].search(text) and UPDATE_JOIN[1].search(text):
self._flag(node, "UPDATE ... JOIN is MySQL-only -> use a correlated subquery (WHERE ... IN/EXISTS)")
def visit_Constant(self, node: ast.Constant) -> None:
# plain string literals, incl. `"...".format()` and `"..." % (...)` templates
if isinstance(node.value, str) and id(node) not in self.docstrings:
self._scan_sql(node.value, node)
self.generic_visit(node)
def visit_JoinedStr(self, node: ast.JoinedStr) -> None:
# f-string: scan its STATIC text (interpolated values become a placeholder) so MySQL-isms
# in dynamic SQL are caught, without flagging safe interpolation of identifiers.
text = "".join(
v.value if isinstance(v, ast.Constant) and isinstance(v.value, str) else " ? "
for v in node.values
)
self._scan_sql(text, node)
# don't recurse: child literal chunks would otherwise be re-scanned individually
def visit_Call(self, node: ast.Call) -> None:
fn = node.func
name = fn.attr if isinstance(fn, ast.Attribute) else (fn.id if isinstance(fn, ast.Name) else "")
# row.get("Column_name") — MySQL SHOW INDEX result key
if name == "get" and node.args and isinstance(node.args[0], ast.Constant) and node.args[0].value in MYSQL_RESULT_KEYS:
self._flag(node, f'"{node.args[0].value}" is a MySQL SHOW INDEX result key -> use frappe.db.has_index()/get_column_index()')
# set_value(..., True) / db_set("field", True) on a Check (int) column.
# Only the field *value* arg carries bool->smallint risk — NOT trailing flags like
# update_modified. db_set(field, value, update_modified, ...) -> value at args[1] (or a dict
# at args[0]); set_value(dt, dn, field, value, ...) -> value at args[3] (or a dict at args[2]).
if name in SET_BOOL_FUNCS:
value_idx, dict_idx = (1, 0) if name == "db_set" else (3, 2)
dict_arg = (
node.args[dict_idx]
if len(node.args) > dict_idx and isinstance(node.args[dict_idx], ast.Dict)
else None
)
if dict_arg is not None:
for v in dict_arg.values:
if isinstance(v, ast.Constant) and isinstance(v.value, bool):
self._flag(node, f"{name}(...) sets an int/Check column with a bool in a dict -> pass 1/0 (Postgres rejects bool->smallint)")
elif len(node.args) > value_idx:
a = node.args[value_idx]
if isinstance(a, ast.Constant) and isinstance(a.value, bool):
self._flag(node, f"{name}(..., {a.value}) sets an int/Check column with a bool -> pass 1/0 (Postgres rejects bool->smallint)")
# frappe.get_all/get_list(..., distinct=True, order_by="<col>") -> ORDER BY is silently dropped
# for distinct queries on Postgres, so the result is unordered there. Sort in python instead
# (e.g. sorted(frappe.get_all(..., distinct=True), key=str.casefold)). An empty order_by="" (the
# explicit "suppress the injected default" idiom) and a dynamic/variable order_by are not flagged.
if name in DISTINCT_ORDER_FUNCS:
has_distinct = any(
kw.arg == "distinct" and isinstance(kw.value, ast.Constant) and kw.value.value
for kw in node.keywords
)
order_kw = next((kw for kw in node.keywords if kw.arg == "order_by"), None)
has_literal_order = (
order_kw is not None
and isinstance(order_kw.value, ast.Constant)
and isinstance(order_kw.value.value, str)
and order_kw.value.value.strip()
)
if has_distinct and has_literal_order:
self._flag(node, f"{name}(distinct=True, order_by=...) -> frappe drops ORDER BY for distinct queries on Postgres; sort in python instead, e.g. sorted(..., key=str.casefold)")
# query-builder .rlike(...): pypika emits the MySQL-only RLIKE operator, which frappe does
# NOT translate for Postgres (it rewrites only REGEXP -> ~*).
if name == "rlike":
self._flag(node, ".rlike() emits MySQL-only RLIKE (not translated on Postgres) -> use .regexp() (rewritten to ~*) or .like()")
# Cast(col, "char") / Cast_(col, "char"): on Postgres a bare CHAR is character(1) and truncates
# (e.g. CAST(12 AS CHAR) -> '1'); use "varchar" for a full-length string cast.
if name in CAST_FUNCS:
for arg in (*node.args, *(kw.value for kw in node.keywords)):
if isinstance(arg, ast.Constant) and isinstance(arg.value, str) and arg.value.strip().lower() == "char":
self._flag(node, f"{name}(..., 'char') is character(1) on Postgres and truncates -> use 'varchar'")
self.generic_visit(node)
def visit_Subscript(self, node: ast.Subscript) -> None:
key = node.slice
if isinstance(key, ast.Constant) and key.value in MYSQL_RESULT_KEYS:
self._flag(node, f'"{key.value}" is a MySQL SHOW INDEX result key -> use frappe.db.has_index()/get_column_index()')
self.generic_visit(node)
def check_file(path: str) -> list[str]:
try:
# nosemgrep: frappe-semgrep-rules.rules.security.frappe-security-file-traversal -- dev-only lint tool; `path` is a source file supplied by pre-commit, not user input
src = open(path, encoding="utf-8").read()
except (OSError, UnicodeDecodeError):
return []
try:
tree = ast.parse(src, filename=path)
except SyntaxError:
return [] # check-ast hook reports real syntax errors
v = Visitor(src.splitlines(), _docstring_ids(tree))
v.visit(tree)
return [f"{path}:{line}: [pg-compat] {msg}" for line, msg in sorted(set(v.violations))]
def main(argv: list[str]) -> int:
out: list[str] = []
for path in argv:
if path.endswith(".py"):
out.extend(check_file(path))
if out:
print("\n".join(out))
print(
f"\n{len(out)} PostgreSQL-incompatibility issue(s). Fix them, or add `# pg-ok` to a "
"line that is intentionally MariaDB-only (e.g. inside an `if frappe.db.db_type == 'mariadb':` branch)."
)
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))

View File

@@ -13,6 +13,6 @@
"root_login": "postgres",
"root_password": "travis",
"host_name": "http://test_site:8000",
"install_apps": ["payments", "erpnext"],
"install_apps": ["erpnext"],
"throttle_user_limit": 100
}

View File

@@ -1,79 +0,0 @@
#!/bin/bash
#
# Run MariaDB INSIDE the runner container, on a datadir we control. Because the datadir can be
# packaged into the bench artifact, test shards start an already-loaded server instead of
# replaying a SQL dump (the ~60s hydrate restore). Each shard gets its own copy → isolation kept.
#
# CI_DB_DATADIR picks the path:
# - setup job: /home/ci/db-data (OUTSIDE the bench, so install.sh's `rm -rf ~/frappe-bench`
# doesn't wipe it; it's moved into the bench just before packaging)
# - test shard: ~/frappe-bench/mariadb-data (where the artifact untar'd it)
#
# Idempotent: inits a fresh datadir if absent (setup), else starts on the existing one (shards).
#
set -e
ci_user="${ERPNEXT_CI_USER:-frappe}"
# Re-exec as the ci user so mariadbd and the datadir are owned consistently (root mariadbd is
# refused anyway). Mirrors install.sh's user switch.
if [ "$(id -u)" = "0" ] && [ "${SKIP_SYSTEM_SETUP:-0}" = "1" ] && [ "$ci_user" != "root" ]; then
exec su -m "$ci_user" -s /bin/bash -c \
"ERPNEXT_CI_USER='$ci_user' CI_DB_DATADIR='${CI_DB_DATADIR:-}' DB='${DB:-}' bash '$0'"
fi
# --- PostgreSQL (GitHub-hosted CI): run in-runner on a PGDATA so it bakes into the artifact,
# same idea as the mariadb datadir. Trust auth (throwaway CI) skips password setup; durability
# off for speed. Postgres is preinstalled on ubuntu-latest under /usr/lib/postgresql/<ver>/bin.
if [ "${DB:-mariadb}" = "postgres" ]; then
PG_BIN=$(ls -d /usr/lib/postgresql/*/bin 2>/dev/null | sort -V | tail -1)
[ -n "$PG_BIN" ] && export PATH="$PG_BIN:$PATH"
PGDATA="${CI_DB_DATADIR:-$HOME/frappe-bench/pgdata}"
if [ ! -d "$PGDATA/base" ]; then
initdb -D "$PGDATA" -U postgres --auth-local=trust --auth-host=trust >/dev/null
echo "host all all 127.0.0.1/32 trust" >> "$PGDATA/pg_hba.conf"
fi
pg_ctl -D "$PGDATA" -w -o "-p 5432 -c listen_addresses=127.0.0.1 -c unix_socket_directories=$PGDATA -c fsync=off -c synchronous_commit=off -c full_page_writes=off" start
echo "PostgreSQL up in-runner (pgdata=$PGDATA)"
exit 0
fi
# --- MariaDB ---
DATADIR="${CI_DB_DATADIR:-$HOME/frappe-bench/mariadb-data}"
SOCK="$DATADIR/mysqld.sock"
fresh=0
if [ ! -d "$DATADIR/mysql" ]; then
mkdir -p "$DATADIR"
mariadb-install-db --no-defaults --datadir="$DATADIR" \
--auth-root-authentication-method=normal --skip-test-db >/dev/null 2>&1
fresh=1
fi
# Throwaway-CI durability off; bind TCP 127.0.0.1:3306 so bench/install.sh connect as usual.
mariadbd --no-defaults --datadir="$DATADIR" --socket="$SOCK" --pid-file="$DATADIR/mysqld.pid" \
--port=3306 --bind-address=127.0.0.1 \
--innodb-flush-log-at-trx-commit=0 --sync-binlog=0 --skip-log-bin \
> "$HOME/mariadb.log" 2>&1 &
up=0
for _ in $(seq 1 60); do
if mariadb-admin --socket="$SOCK" ping --silent 2>/dev/null; then up=1; break; fi
sleep 1
done
# Fail loudly instead of letting the loop fall through (exit 0 of the last `sleep`) into SQL that
# would error with a vague socket-connection failure.
[ "$up" = "1" ] || { echo "mariadbd did not come up on $SOCK"; cat "$HOME/mariadb.log" 2>/dev/null; exit 1; }
if [ "$fresh" = "1" ]; then
# A fresh datadir has only a password-less root@localhost. Give it the password install.sh
# uses, plus a TCP-reachable root@127.0.0.1, so the rest of install.sh works unchanged.
mariadb --no-defaults --socket="$SOCK" -u root <<'SQL'
ALTER USER 'root'@'localhost' IDENTIFIED BY 'root';
CREATE USER IF NOT EXISTS 'root'@'127.0.0.1' IDENTIFIED BY 'root';
GRANT ALL PRIVILEGES ON *.* TO 'root'@'127.0.0.1' WITH GRANT OPTION;
FLUSH PRIVILEGES;
SQL
fi
echo "MariaDB up in-container (datadir=$DATADIR, fresh=$fresh)"

View File

@@ -21,7 +21,7 @@ jobs:
cache: pip
- name: Install and Run Pre-commit
uses: pre-commit/action@v3.0.1
uses: pre-commit/action@v3.0.0
semgrep:
name: semgrep

View File

@@ -65,22 +65,6 @@ jobs:
- name: Add to Hosts
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
# The v14 baseline backup is a fixed published file — cache it instead of re-downloading
# it from the GitHub release every run.
- name: Cache erpnext v14 backup
id: cache-v14
uses: actions/cache@v4
with:
path: ~/erpnext-v14.sql.gz
key: erpnext-v14-sql-gz
- name: Download erpnext v14 backup
if: steps.cache-v14.outputs.cache-hit != 'true'
run: |
curl -fSL --retry 5 --retry-all-errors --retry-delay 5 \
-o ~/erpnext-v14.sql.gz \
https://github.com/frappe/erpnext/releases/download/v14-baseline/erpnext-v14.sql.gz
- name: Cache pip
uses: actions/cache@v4
with:
@@ -129,20 +113,12 @@ jobs:
jq 'del(.install_apps)' ~/frappe-bench/sites/test_site/site_config.json > tmp.json
mv tmp.json ~/frappe-bench/sites/test_site/site_config.json
bench --site test_site --force restore ~/erpnext-v14.sql.gz
wget https://frappe.io/files/erpnext-v14.sql.gz
bench --site test_site --force restore ~/frappe-bench/erpnext-v14.sql.gz
git -C "apps/frappe" remote set-url upstream https://github.com/frappe/frappe.git
git -C "apps/erpnext" remote set-url upstream https://github.com/frappe/erpnext.git
# Start every bench process except the background workers. If workers run during a
# migrate, they pick up the orphan-link cleanup jobs it enqueues and race its schema
# changes, which fails with MySQL 1412 "Table definition has changed". Redis and the
# other services stay up; the queued jobs simply wait and are harmless here.
function start_bench_without_workers() {
local procs
procs=$(awk -F: '/^[a-z_]+:/ && $1 !~ /worker/ {print $1}' ~/frappe-bench/Procfile)
honcho start -f ~/frappe-bench/Procfile $procs &>> ~/frappe-bench/bench_start.log &
}
function update_to_version() {
version=$1
@@ -158,11 +134,10 @@ jobs:
# Resetup env and install apps
pgrep honcho | xargs kill
sleep 10
rm -rf ~/frappe-bench/env
bench -v setup env --python python$2
bench pip install -e ./apps/erpnext
start_bench_without_workers
bench start &>> ~/frappe-bench/bench_start.log &
bench --site test_site migrate
}
@@ -179,7 +154,7 @@ jobs:
rm -rf ~/frappe-bench/env
bench -v setup env
bench pip install -e ./apps/erpnext
start_bench_without_workers
bench start &>> ~/frappe-bench/bench_start.log &
bench --site test_site migrate

View File

@@ -22,4 +22,4 @@ jobs:
pull-requests: write
steps:
- uses: alyf-de/po-review-action@v1.1.0
- uses: alyf-de/po-review-action@v1.0.0

View File

@@ -31,49 +31,47 @@ on:
permissions:
contents: read
packages: read
concurrency:
group: server-mariadb-develop-${{ github.event_name }}-${{ github.event.number || github.event_name == 'workflow_dispatch' && github.run_id || '' }}
cancel-in-progress: true
# Shared across both jobs. Both run in the SAME CI image so the bench lives at the identical
# path (/home/ci/frappe-bench) on the setup runner and the test shards — that's what makes the
# packaged Python venv portable between them.
env:
TZ: 'Asia/Kolkata'
DEBIAN_FRONTEND: noninteractive
NODE_ENV: "production"
WITH_COVERAGE: ${{ github.event_name != 'pull_request' }}
ERPNEXT_CI_USER: ci
PIP_CACHE_DIR: /home/ci/.cache/pip
npm_config_cache: /home/ci/.cache/npm
YARN_CACHE_FOLDER: /home/ci/.cache/yarn
UV_CACHE_DIR: /home/ci/.cache/uv
jobs:
# Build the bench (clone + pip + yarn + assets) and reinstall test_site ONCE, on a free
# GitHub-hosted runner, then publish the whole bench (with a DB dump baked in) as an artifact.
# The expensive, non-parallelisable work happens here exactly once instead of on every shard.
setup:
name: Build & reinstall (setup)
# Dedicated scale set (fat cpu request) so the build+reinstall runs at full speed, uncontended
# by the many thin test shards. Same CI image + /home/ci path + 127.0.0.1 DB as the shards,
# so the packaged bench (and its venv) transplants cleanly.
runs-on: erpnext-arc-setup
timeout-minutes: 40
container:
image: ghcr.io/frappe/erpnext-ci-mariadb:py3.14-node24
credentials:
username: ${{ secrets.GHCR_USERNAME || github.actor }}
password: ${{ secrets.GHCR_TOKEN || github.token }}
defaults:
run:
shell: bash
test:
runs-on: ubuntu-latest
timeout-minutes: 60
env:
TZ: 'Asia/Kolkata'
NODE_ENV: "production"
WITH_COVERAGE: ${{ github.event_name != 'pull_request' }}
strategy:
fail-fast: false
matrix:
container: [1, 2, 3, 4]
name: Python Unit Tests
services:
mysql:
image: mariadb:10.6
env:
TZ: 'Asia/Kolkata'
MARIADB_ROOT_PASSWORD: 'root'
ports:
- 3306:3306
options: --health-cmd="mariadb-admin ping" --health-interval=5s --health-timeout=2s --health-retries=3
steps:
- name: Clone
uses: actions/checkout@v6
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: '3.14'
- name: Check for valid Python & Merge Conflicts
run: |
python -m compileall -fq "${GITHUB_WORKSPACE}"
@@ -82,17 +80,47 @@ jobs:
exit 1
fi
- name: Setup Node
uses: actions/setup-node@v6
with:
node-version: 24
check-latest: true
- name: Add to Hosts
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
# MariaDB runs in-container on a datadir OUTSIDE the bench, because install.sh's next step
# does `rm -rf ~/frappe-bench`. After the reinstall, the datadir is moved into the bench so
# it ships in the artifact — test shards then start an already-loaded server (no restore).
- name: Start DB
run: bash ${GITHUB_WORKSPACE}/.github/helper/start-db.sh
- name: Cache pip
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/*requirements.txt', '**/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-
${{ runner.os }}-
- name: Cache node modules
uses: actions/cache@v4
env:
SKIP_SYSTEM_SETUP: "1"
CI_DB_DATADIR: /home/ci/db-data
cache-name: cache-node-modules
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
- name: Get yarn cache directory path
id: yarn-cache-dir-path
run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
- uses: actions/cache@v4
id: yarn-cache
with:
path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install
run: bash ${GITHUB_WORKSPACE}/.github/helper/install.sh
@@ -101,114 +129,26 @@ jobs:
TYPE: server
FRAPPE_USER: ${{ github.event.inputs.user }}
FRAPPE_BRANCH: ${{ github.event.client_payload.sha || github.event.inputs.branch }}
DB_HOST: 127.0.0.1
DB_USER_HOST: '%'
WKHTMLTOX_DEB: /tmp/wkhtmltox.deb
SKIP_SYSTEM_SETUP: "1"
SKIP_WKHTMLTOX_SETUP: "1"
- name: Warm up test data
run: |
su -m "${ERPNEXT_CI_USER:-frappe}" -s /bin/bash <<'EOF'
cd ~/frappe-bench/
bench --site test_site run-tests --lightmode --module erpnext.tests.bootstrap_test_data
EOF
# Clean shutdown (consistent InnoDB datadir), then stage it inside the bench for packaging.
- name: Stop DB and stage datadir
run: |
mariadb-admin -h 127.0.0.1 -P 3306 -u root -proot shutdown || true
for _ in $(seq 1 30); do [ -f /home/ci/db-data/mysqld.pid ] || break; sleep 1; done
# Don't bake a dirty datadir — fail if mariadbd didn't finish stopping, rather than ship
# an inconsistent datadir the shards would have to crash-recover.
[ -f /home/ci/db-data/mysqld.pid ] && { echo "mariadbd did not shut down cleanly"; exit 1; }
mv /home/ci/db-data /home/ci/frappe-bench/mariadb-data
# Package the whole bench (apps, venv, node_modules, sites, the DB dump, and hydrate.sh)
# into one artifact for the test shards to consume.
# Single-node hand-off: stage the bench on a node-local hostPath instead of round-tripping
# through GitHub artifact storage (~60s/shard). Setup and shards share the same disk, so
# the shards just untar it locally. NOTE: this assumes one node — a shard on a different
# node could not read this path (then you'd need GitHub artifacts or an NFS/RWX volume).
- name: Stage bench on node (hostPath)
run: |
cp "${GITHUB_WORKSPACE}/.github/helper/hydrate.sh" /home/ci/frappe-bench/hydrate.sh
cp "${GITHUB_WORKSPACE}/.github/helper/start-db.sh" /home/ci/frappe-bench/start-db.sh
mkdir -p /opt/ci-bench-staging
# self-clean: drop bench tars from runs older than 2h
find /opt/ci-bench-staging -maxdepth 1 -name '*.tar.gz' -mmin +120 -delete 2>/dev/null || true
# Exclude .git/node_modules; the mariadb-data datadir IS included (the pre-loaded DB).
tar czpf "/opt/ci-bench-staging/${GITHUB_RUN_ID}.tar.gz" -C /home/ci \
--exclude='.git' --exclude='node_modules' frappe-bench
ls -lh "/opt/ci-bench-staging/${GITHUB_RUN_ID}.tar.gz"
# Fan-out: each shard downloads the bench, untars it, starts MariaDB on the baked datadir, and
# runs its slice of the suite. No clone, no build, no reinstall, no DB dump restore on the shards.
test:
name: Python Unit Tests
needs: setup
runs-on: erpnext-arc
timeout-minutes: 60
container:
image: ghcr.io/frappe/erpnext-ci-mariadb:py3.14-node24
credentials:
username: ${{ secrets.GHCR_USERNAME || github.actor }}
password: ${{ secrets.GHCR_TOKEN || github.token }}
defaults:
run:
shell: bash
strategy:
fail-fast: false
matrix:
container: [1, 2, 3, 4]
steps:
- name: Add to Hosts
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
# Read the bench straight from the node-local hostPath the setup job staged it on — no
# GitHub download. -p preserves the ci (uid 1001) ownership so bench runs as ci cleanly.
- name: Untar bench from node (hostPath)
run: |
tar xzpf "/opt/ci-bench-staging/${GITHUB_RUN_ID}.tar.gz" -C /home/ci
ls -ld /home/ci/frappe-bench
- name: Hydrate (start DB on baked datadir + bench start)
run: bash /home/ci/frappe-bench/hydrate.sh
env:
DB_HOST: 127.0.0.1
SKIP_SYSTEM_SETUP: "1"
- name: Run Tests
run: |
su -m "${ERPNEXT_CI_USER:-frappe}" -s /bin/bash <<'EOF'
cd ~/frappe-bench/
coverage_flag=""
if [ "$WITH_COVERAGE" = "true" ]; then coverage_flag="--with-coverage"; fi
bench --site test_site run-parallel-tests --lightmode --app erpnext \
--total-builds ${{ strategy.job-total }} \
--build-number ${{ matrix.container }} \
$coverage_flag
EOF
run: 'cd ~/frappe-bench/ && bench --site test_site run-parallel-tests --lightmode --app erpnext --total-builds ${{ strategy.job-total }} --build-number ${{ matrix.container }} --with-coverage'
env:
TYPE: server
- name: Show bench output
if: ${{ always() }}
run: cat ~/frappe-bench/bench_start.log || true
- name: Upload coverage data
if: ${{ env.WITH_COVERAGE == 'true' }}
uses: actions/upload-artifact@v4
with:
name: coverage-${{ matrix.container }}
path: /home/ci/frappe-bench/sites/coverage.xml
path: /home/runner/frappe-bench/sites/coverage.xml
coverage:
name: Coverage Wrap Up
needs: [test]
if: ${{ github.event_name != 'pull_request' }}
needs: test
runs-on: ubuntu-latest
steps:
- name: Clone

View File

@@ -1,12 +1,7 @@
name: Server (Postgres)
on:
schedule:
# 03:00 AM IST daily (21:30 UTC the previous day)
- cron: "30 21 * * *"
pull_request:
# 'labeled' so adding the 'postgres' label to an already-open PR re-triggers the run.
types: [opened, reopened, synchronize, labeled]
paths-ignore:
- '**.js'
- '**.md'
@@ -14,7 +9,7 @@ on:
- 'crowdin.yml'
- '.coderabbit.yml'
- '.mergify.yml'
workflow_dispatch:
types: [opened, labelled, synchronize, reopened]
concurrency:
group: server-postgres-develop-${{ github.event_name }}-${{ github.event.number || github.event_name == 'workflow_dispatch' && github.run_id || '' }}
@@ -23,31 +18,41 @@ concurrency:
permissions:
contents: read
# Postgres CI stays on GitHub-hosted (free, full-speed VM per shard) but follows the same fan-out
# we built for MariaDB: build the bench + reinstall ONCE in the setup job, bake the PostgreSQL
# PGDATA into the artifact, and have 4 test shards start Postgres on that datadir — no per-shard
# clone/build/reinstall/restore. Python is pinned so the venv transplants between VMs.
env:
TZ: 'Asia/Kolkata'
NODE_ENV: "production"
PYTHON_VERSION: '3.14'
jobs:
setup:
name: Build & reinstall (setup)
test:
if: ${{ contains(github.event.pull_request.labels.*.name, 'postgres') }}
runs-on: ubuntu-latest
# Runs on the daily schedule (and workflow_dispatch). On PRs it runs ONLY when the PR carries
# the 'postgres' label — the test job needs setup, so it's skipped too when this is.
if: github.event_name != 'pull_request' || contains(github.event.pull_request.labels.*.name, 'postgres')
timeout-minutes: 40
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
container: [1]
name: Python Unit Tests
services:
postgres:
image: postgres:13.3
env:
POSTGRES_PASSWORD: travis
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
steps:
- name: Clone
uses: actions/checkout@v6
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: ${{ env.PYTHON_VERSION }}
python-version: '3.14'
- name: Check for valid Python & Merge Conflicts
run: |
@@ -66,133 +71,48 @@ jobs:
- name: Add to Hosts
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
- name: Cache deps (uv/pip/npm/yarn)
- name: Cache pip
uses: actions/cache@v4
with:
path: |
~/.cache/uv
~/.cache/pip
~/.npm
~/.cache/yarn
key: ${{ runner.os }}-deps-${{ hashFiles('**/*requirements.txt', '**/pyproject.toml', '**/yarn.lock') }}
restore-keys: ${{ runner.os }}-deps-
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/*requirements.txt', '**/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-
${{ runner.os }}-
# Warm-bench cache (the big one): install.sh saves the built base bench — frappe + env +
# node_modules + assets — here as frappe-bench-base-*.tar.zst. Later runs restore it and only
# fast-forward to the live develop SHA + rebuild the delta, so the bench BUILD is near-free and
# only the test_site reinstall (per-run DB, uncacheable) stays slow — matching the self-hosted
# box. The first run after a deps change populates it; every run after that is fast.
- name: Cache warm bench (base build)
- name: Cache node modules
uses: actions/cache@v4
with:
path: ~/bench-cache
key: ${{ runner.os }}-warmbench-v2-${{ hashFiles('**/*requirements.txt', '**/pyproject.toml', '**/yarn.lock') }}
restore-keys: ${{ runner.os }}-warmbench-v2-
# Postgres runs in-runner on a PGDATA OUTSIDE the bench (install.sh wipes ~/frappe-bench);
# after the reinstall it's moved into the bench so it ships in the artifact.
- name: Start DB
run: bash ${GITHUB_WORKSPACE}/.github/helper/start-db.sh
env:
DB: postgres
CI_DB_DATADIR: /home/runner/pgdata
cache-name: cache-node-modules
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
- name: Get yarn cache directory path
id: yarn-cache-dir-path
run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
- uses: actions/cache@v4
id: yarn-cache
with:
path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install
run: bash ${GITHUB_WORKSPACE}/.github/helper/install.sh
env:
DB: postgres
TYPE: server
FRAPPE_BRANCH: develop
BENCH_CACHE_DIR: /home/runner/bench-cache
- name: Warm up test data
run: |
cd ~/frappe-bench/
bench --site test_site run-tests --lightmode --module erpnext.tests.bootstrap_test_data
- name: Stop DB and stage datadir
run: |
PG_BIN=$(ls -d /usr/lib/postgresql/*/bin | sort -V | tail -1)
# Clean shutdown so the baked datadir is consistent. Do NOT swallow a failed stop with
# `|| true`: moving and tarring a still-running cluster ships a torn datadir the shards
# cannot crash-recover (full_page_writes is off). Fail the job instead — mirrors the
# MariaDB sister's "don't bake a dirty datadir" guard.
"$PG_BIN/pg_ctl" -D /home/runner/pgdata -m fast -w stop
mv /home/runner/pgdata /home/runner/frappe-bench/pgdata
- name: Package bench for test shards
run: |
cp "${GITHUB_WORKSPACE}/.github/helper/hydrate.sh" /home/runner/frappe-bench/hydrate.sh
cp "${GITHUB_WORKSPACE}/.github/helper/start-db.sh" /home/runner/frappe-bench/start-db.sh
tar czpf "${GITHUB_WORKSPACE}/bench.tar.gz" -C /home/runner \
--exclude='.git' --exclude='node_modules' frappe-bench
ls -lh "${GITHUB_WORKSPACE}/bench.tar.gz"
- name: Upload bench artifact
uses: actions/upload-artifact@v4
with:
name: bench-pg
path: bench.tar.gz
retention-days: 1
compression-level: 0
test:
name: Python Unit Tests
needs: setup
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
container: [1, 2, 3, 4]
steps:
- name: Download bench artifact
uses: actions/download-artifact@v4
with:
name: bench-pg
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Add to Hosts
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
# The bench CLI (frappe-bench) and redis are global/system tools — not in the bench tarball.
# The setup runner got them via install.sh; the MariaDB shards get them from the arc5 image.
# GitHub-hosted PG shards install them here (cheap vs the build+reinstall that setup did once).
- name: Install shard runtime (bench CLI + redis + wkhtmltopdf)
run: |
pip install frappe-bench
command -v redis-server >/dev/null || { sudo apt-get update -qq && sudo apt-get install -y -qq redis-server; }
# wkhtmltopdf (patched-qt build) for print-format / PDF tests — same .deb install.sh uses.
if ! command -v wkhtmltopdf >/dev/null; then
wget -qO /tmp/wkhtmltox.deb https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6.1-2/wkhtmltox_0.12.6.1-2.jammy_amd64.deb
sudo apt-get install -y -qq /tmp/wkhtmltox.deb
fi
- name: Untar bench
run: |
tar xzpf "${GITHUB_WORKSPACE}/bench.tar.gz" -C /home/runner
ls -ld /home/runner/frappe-bench
- name: Hydrate (start Postgres on the baked datadir)
run: bash /home/runner/frappe-bench/hydrate.sh
env:
DB: postgres
DB_HOST: 127.0.0.1
- name: Run Tests
run: |
cd ~/frappe-bench/
# print-format / PDF tests are engine-independent (they exercise wkhtmltopdf rendering,
# not postgres SQL — the MariaDB CI already covers them). They only fetch the static asset
# bundles from http://test_site:8000/assets/..., so a plain static file server over sites/
# satisfies wkhtmltopdf without the frappe web server (which never bound on a bare runner).
( cd ~/frappe-bench/sites && nohup python3 -m http.server 8000 --bind 127.0.0.1 > ~/frappe-bench/web.log 2>&1 & )
for _ in $(seq 1 15); do (exec 3<>/dev/tcp/127.0.0.1/8000) 2>/dev/null && { exec 3>&- 3<&-; break; }; sleep 1; done
bench --site test_site run-parallel-tests --lightmode --app erpnext \
--total-builds ${{ strategy.job-total }} --build-number ${{ matrix.container }}
run: cd ~/frappe-bench/ && bench --site test_site run-parallel-tests --app erpnext --use-orchestrator
env:
TYPE: server
CI_BUILD_ID: ${{ github.run_id }}
ORCHESTRATOR_URL: http://test-orchestrator.frappe.io

File diff suppressed because one or more lines are too long

View File

@@ -88,6 +88,7 @@ pull_request_rules:
actions:
merge:
method: squash
commit_message_format:
title: pr-title
body: pr-body
commit_message_template: |
{{ title }} (#{{ number }})
{{ body }}

View File

@@ -48,6 +48,7 @@ repos:
cypress/.*|
.*node_modules.*|
.*boilerplate.*|
erpnext/public/js/controllers/.*|
erpnext/templates/pages/order.js|
erpnext/templates/includes/.*
)$
@@ -65,18 +66,6 @@ repos:
- id: ruff-format
name: "Run ruff formatter"
- repo: local
hooks:
- id: postgres-compat
name: "PostgreSQL compatibility (static check)"
description: "Flags MySQL-only SQL that breaks on Postgres; the label-gated PG test job is the backstop for semantic divergences."
entry: .github/helper/postgres_compat.py
language: script
files: ^erpnext/.*\.py$
# patches/ are historical, version-gated migrations (skipped on fresh Postgres installs);
# out of scope for the always-on gate.
exclude: ^erpnext/patches/
ci:
autoupdate_schedule: weekly
skip: []

View File

@@ -14,52 +14,52 @@
"@dnd-kit/core": "^6.3.1",
"@dnd-kit/sortable": "^10.0.0",
"@dnd-kit/utilities": "^3.2.2",
"@tailwindcss/vite": "^4.3.2",
"@tailwindcss/vite": "^4.3.0",
"@tanstack/react-table": "^8.21.3",
"@tanstack/react-virtual": "^3.13.24",
"@vitejs/plugin-react": "^6.0.3",
"@vitejs/plugin-react": "^6.0.1",
"chrono-node": "^2.9.1",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"cmdk": "^1.1.1",
"date-fns": "^4.1.0",
"dayjs": "^1.11.20",
"frappe-react-sdk": "^1.17.0",
"frappe-react-sdk": "^1.15.0",
"fuse.js": "^7.3.0",
"jotai": "^2.20.1",
"jotai-family": "^1.0.2",
"jotai": "^2.20.0",
"jotai-family": "^1.0.1",
"lodash.isplainobject": "^4.0.6",
"lucide-react": "^1.14.0",
"radix-ui": "^1.6.1",
"react": "^19.2.7",
"radix-ui": "^1.4.3",
"react": "^19.2.6",
"react-currency-input-field": "^4.0.5",
"react-day-picker": "9.14.0",
"react-dom": "^19.2.7",
"react-dom": "^19.2.6",
"react-dropzone": "^15.0.0",
"react-hook-form": "^7.75.0",
"react-hotkeys-hook": "^5.3.2",
"react-markdown": "^10.1.0",
"react-router": "^8.1.0",
"react-router": "^7.15.0",
"react-router-dom": "^7.15.0",
"rehype-raw": "^7.0.0",
"remark-gfm": "^4.0.1",
"safe-expr-eval": "^1.0.4",
"sonner": "^2.0.7",
"tailwind-merge": "^3.5.0",
"tailwindcss": "^4.3.0",
"tw-animate-css": "^1.4.0",
"usehooks-ts": "^3.1.1",
"vite": "^8.0.16"
"vite": "^8.0.11"
},
"devDependencies": {
"@eslint/js": "^9.39.4",
"@eslint/js": "^9.39.1",
"@types/node": "^25.3.0",
"@types/react": "^19.2.7",
"@types/react-dom": "^19.2.3",
"eslint": "^9.39.1",
"eslint-plugin-react-hooks": "^7.1.1",
"eslint-plugin-react-refresh": "^0.5.3",
"eslint-plugin-react-refresh": "^0.4.24",
"globals": "^16.5.0",
"typescript": "~5.9.3",
"typescript-eslint": "^8.62.1"
"typescript-eslint": "^8.48.0"
}
}

View File

@@ -1,5 +1,5 @@
import { lazy, useEffect } from 'react'
import { BrowserRouter, Navigate, Route, Routes } from 'react-router'
import { BrowserRouter, Navigate, Route, Routes } from 'react-router-dom'
import { FrappeProvider } from 'frappe-react-sdk'
import { Toaster } from '@/components/ui/sonner'
import BankReconciliation from '@/pages/BankReconciliation'

View File

@@ -250,7 +250,7 @@ const ClosingBalanceForm = ({ defaultBalance, date, bankAccount, onClose }: { de
{_("Enter the closing balance you see in your bank statement for {0} as of the {1}", [bankAccount?.account_name ?? bankAccount?.name ?? '', formatDate(date, 'Do MMM YYYY')])}
</DialogDescription>
</DialogHeader>
{error && <div className="py-2"><ErrorBanner error={error} /></div>}
{error && <ErrorBanner error={error} />}
<div className="py-4">
<CurrencyFormField
name="balance"

View File

@@ -2,6 +2,7 @@ import { useAtomValue } from "jotai"
import { MissingFiltersBanner } from "./MissingFiltersBanner"
import { bankRecDateAtom, SelectedBank, selectedBankAccountAtom } from "./bankRecAtoms"
import { useCurrentCompany } from "@/hooks/useCurrentCompany"
import { Paragraph } from "@/components/ui/typography"
import type { ColumnDef } from "@tanstack/react-table"
import { useCallback, useMemo, useState } from "react"
import { useFrappeGetCall, useFrappePostCall, useSWRConfig } from "frappe-react-sdk"
@@ -25,7 +26,6 @@ import { Form } from "@/components/ui/form"
import { useForm } from "react-hook-form"
import { DateField } from "@/components/ui/form-elements"
import { Empty, EmptyMedia, EmptyHeader, EmptyTitle, EmptyDescription } from "@/components/ui/empty"
import MarkdownRenderer from "@/components/ui/markdown"
const BankClearanceSummary = () => {
const bankAccount = useAtomValue(selectedBankAccountAtom)
@@ -203,14 +203,14 @@ const BankClearanceSummaryView = () => {
[accountCurrency, bankAccount, companyID, mutate, onCopy],
)
const content = _("Below is a list of all accounting entries posted against the bank account {0} between {1} and {2}.", [`<strong>${bankAccount?.account}</strong>`, `<strong>${formattedFromDate}</strong>`, `<strong>${formattedToDate}</strong>`])
return <div className="space-y-4 py-2">
<div>
<span className="text-p-sm">
<MarkdownRenderer content={content} />
</span>
<Paragraph className="text-sm">
<span dangerouslySetInnerHTML={{
__html: _("Below is a list of all accounting entries posted against the bank account {0} between {1} and {2}.", [`<strong>${bankAccount?.account}</strong>`, `<strong>${formattedFromDate}</strong>`, `<strong>${formattedToDate}</strong>`])
}} />
</Paragraph>
</div>
{error && <ErrorBanner error={error} />}

View File

@@ -18,7 +18,6 @@ import { useMultiFileUploadProgress } from "@/hooks/useMultiFileUploadProgress"
import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "@/components/ui/table"
import { Checkbox } from "@/components/ui/checkbox"
import { ArrowDownRight, ArrowUpRight, Plus, Trash2 } from "lucide-react"
import { evaluateAmountFormula } from "@/lib/amountFormula"
import { flt, formatCurrency } from "@/lib/numbers"
import { cn } from "@/lib/utils"
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip"
@@ -216,13 +215,38 @@ const BankEntryForm = ({ selectedTransaction }: { selectedTransaction: Unreconci
})
} else {
const transactionAmount = selectedTransaction.unallocated_amount ?? 0
/**
* The debit and credit amounts can also be expressions - like "transaction_amount * 0.5"
* So we need to compute the value of the expression
* We can use the eval function to do this. But we need to expose certain variables to the expression.
* One of them is transaction_amount which is the unallocated amount of the selected transaction
* @param expression - The expression to compute
* @returns The computed value
*/
const computeExpression = (expression: string) => {
const script = `
const transaction_amount = ${selectedTransaction.unallocated_amount ?? 0}
${expression};
`
let value = 0;
try {
value = window.eval(script);
} catch (error: unknown) {
console.error(error);
value = 0;
}
return value;
}
if (!acc?.debit && !acc?.credit) {
hasTotallyEmptyRowEarlier = true;
}
const computedDebit = acc?.debit ? flt(evaluateAmountFormula(acc.debit, transactionAmount), 2) : 0
const computedCredit = acc?.credit ? flt(evaluateAmountFormula(acc.credit, transactionAmount), 2) : 0
const computedDebit = acc?.debit ? flt(computeExpression(acc.debit), 2) : 0
const computedCredit = acc?.credit ? flt(computeExpression(acc.credit), 2) : 0
totalDebits = flt(totalDebits + computedDebit, 2)
totalCredits = flt(totalCredits + computedCredit, 2)

View File

@@ -2,6 +2,7 @@ import { useAtomValue } from "jotai"
import { MissingFiltersBanner } from "./MissingFiltersBanner"
import { bankRecDateAtom, selectedBankAccountAtom } from "./bankRecAtoms"
import { useCurrentCompany } from "@/hooks/useCurrentCompany"
import { Paragraph } from "@/components/ui/typography"
import { useCallback, useMemo } from "react"
import type { ColumnDef } from "@tanstack/react-table"
import { useFrappeGetCall } from "frappe-react-sdk"
@@ -18,7 +19,6 @@ import _ from "@/lib/translate"
import { toast } from "sonner"
import { useCopyToClipboard } from "usehooks-ts"
import { Empty, EmptyDescription, EmptyHeader, EmptyMedia, EmptyTitle } from "@/components/ui/empty"
import MarkdownRenderer from "@/components/ui/markdown"
const BankReconciliationStatement = () => {
const bankAccount = useAtomValue(selectedBankAccountAtom)
@@ -189,14 +189,14 @@ const BankReconciliationStatementView = () => {
return data.message.result.filter((row: BankClearanceSummaryEntry) => Boolean(row.payment_entry))
}, [data])
const content = _("Below is a list of all entries posted against the bank account {0} which have not been cleared till {1}.", [`<strong>${bankAccount?.account}</strong>`, `<strong>${formatDate(dates.toDate)}</strong>`])
return <div className="space-y-4 py-2">
<div>
<span className="text-p-sm">
<MarkdownRenderer content={content} />
</span>
<Paragraph className="text-sm">
<span dangerouslySetInnerHTML={{
__html: _("Below is a list of all entries posted against the bank account {0} which have not been cleared till {1}.", [`<strong>${bankAccount?.account}</strong>`, `<strong>${formatDate(dates.toDate)}</strong>`])
}} />
</Paragraph>
</div>
{error && <ErrorBanner error={error} />}

View File

@@ -1,6 +1,7 @@
import { useAtomValue, useSetAtom } from "jotai"
import { MissingFiltersBanner } from "./MissingFiltersBanner"
import { bankRecDateAtom, bankRecUnreconcileModalAtom, selectedBankAccountAtom } from "./bankRecAtoms"
import { Paragraph } from "@/components/ui/typography"
import { formatDate } from "@/lib/date"
import { ListView, type ListViewColumnMeta } from "@/components/ui/list-view"
import { formatCurrency, getCurrencyFormatInfo } from "@/lib/numbers"
@@ -22,7 +23,6 @@ import { useCallback, useMemo, useState } from "react"
import { Link } from "react-router"
import { Empty, EmptyTitle, EmptyHeader, EmptyMedia, EmptyDescription, EmptyContent } from "@/components/ui/empty"
import { InputGroup, InputGroupAddon } from "@/components/ui/input-group"
import MarkdownRenderer from "@/components/ui/markdown"
const BankTransactions = () => {
const selectedBank = useAtomValue(selectedBankAccountAtom)
@@ -243,14 +243,14 @@ const BankTransactionListView = () => {
}, [data, search, amountFilter, typeFilter, status])
const content = _("Below is a list of all bank transactions imported in the system for the bank account {0} between {1} and {2}.", [`<strong>${bankAccount?.account_name}</strong>`, `<strong>${formattedFromDate}</strong>`, `<strong>${formattedToDate}</strong>`])
return <div className="space-y-2 py-2">
<div className="flex gap-2 justify-between items-center">
<span className="text-p-sm">
<MarkdownRenderer content={content} />
</span>
<Paragraph className="text-sm">
<span dangerouslySetInnerHTML={{
__html: _("Below is a list of all bank transactions imported in the system for the bank account {0} between {1} and {2}.", [`<strong>${bankAccount?.account_name}</strong>`, `<strong>${formattedFromDate}</strong>`, `<strong>${formattedToDate}</strong>`])
}} />
</Paragraph>
<Button size='md' variant='subtle' asChild>
<Link to="/statement-importer">

View File

@@ -2,6 +2,7 @@ import { useAtomValue } from "jotai"
import { MissingFiltersBanner } from "./MissingFiltersBanner"
import { bankRecDateAtom, selectedBankAccountAtom } from "./bankRecAtoms"
import { useCurrentCompany } from "@/hooks/useCurrentCompany"
import { Paragraph } from "@/components/ui/typography"
import type { ColumnDef } from "@tanstack/react-table"
import { useCallback, useMemo } from "react"
import { useFrappeGetCall, useFrappePostCall } from "frappe-react-sdk"
@@ -17,7 +18,6 @@ import { PartyPopper } from "lucide-react"
import ErrorBanner from "@/components/ui/error-banner"
import _ from "@/lib/translate"
import { Empty, EmptyTitle, EmptyDescription, EmptyMedia, EmptyHeader } from "@/components/ui/empty"
import MarkdownRenderer from "@/components/ui/markdown"
const IncorrectlyClearedEntries = () => {
const companyID = useCurrentCompany()
@@ -177,22 +177,22 @@ const IncorrectlyClearedEntriesView = () => {
[accountCurrency, onClearClick],
)
const content = _("This report shows all entries in the system where the <strong>clearance date is before the posting date</strong> which is incorrect.")
const entriesContent = _("Entries below have a posting date after {0} but the clearance date is before {1}.", [`<strong>${formattedToDate}</strong>`, `<strong>${formattedToDate}</strong>`])
return <div className="space-y-4 py-2">
<div>
<span className="text-p-sm">
<MarkdownRenderer content={content} />
<Paragraph className="text-sm">
<span dangerouslySetInnerHTML={{
__html: _("This report shows all entries in the system where the <strong>clearance date is before the posting date</strong> which is incorrect.")
}} />
<br />
{data && data.message.result.length > 0 && <span>
<MarkdownRenderer content={entriesContent} />
<span dangerouslySetInnerHTML={{
__html: _("Entries below have a posting date after {0} but the clearance date is before {1}.", [`<strong>${formattedToDate}</strong>`, `<strong>${formattedToDate}</strong>`])
}} />
<br />
{_("You can reset the clearing dates of these entries here.")}
</span>}
</span>
</Paragraph>
</div>
{error && <ErrorBanner error={error} />}

View File

@@ -11,7 +11,6 @@ import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "@
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip"
import { H4, Paragraph } from "@/components/ui/typography"
import { today } from "@/lib/date"
import { evaluateAmountFormula } from "@/lib/amountFormula"
import _ from "@/lib/translate"
import { cn } from "@/lib/utils"
import { BankTransactionRule } from "@/types/Accounts/BankTransactionRule"
@@ -446,10 +445,11 @@ const AmountFormulaRenderer = ({ value }: { value?: string }) => {
// If it's a string and cannot be a number, then show it as a formula
if (isNaN(Number(value))) {
let calculatedValue = "";
try {
calculatedValue = String(evaluateAmountFormula(value ?? "", 200));
calculatedValue = window.eval(`const transaction_amount = 200; ${value}`);
} catch (error: unknown) {
console.error(error);
calculatedValue = "Error";

View File

@@ -14,7 +14,7 @@ import { Tooltip, TooltipContent, TooltipTrigger } from '@/components/ui/tooltip
import { useFrappeEventListener, useFrappePostCall } from 'frappe-react-sdk'
import { toast } from 'sonner'
import ErrorBanner from '@/components/ui/error-banner'
import { Link, useNavigate } from 'react-router'
import { Link, useNavigate } from 'react-router-dom'
import { useMemo, useState } from 'react'
import { Progress } from '@/components/ui/progress'
import { useSetAtom } from 'jotai'

View File

@@ -1,26 +0,0 @@
import { Parser } from 'safe-expr-eval'
const parser = new Parser()
const PLAIN_NUMBER_PATTERN = /^-?\d+(\.\d+)?$/
export function evaluateAmountFormula(expression: string, transactionAmount: number): number {
const trimmed = expression.trim()
if (!trimmed) {
return 0
}
if (PLAIN_NUMBER_PATTERN.test(trimmed)) {
return Number(trimmed)
}
try {
const result = parser.parse(trimmed).evaluate({ transaction_amount: transactionAmount })
if (typeof result !== 'number' || !Number.isFinite(result)) {
return 0
}
return result
} catch {
return 0
}
}

View File

@@ -33,16 +33,6 @@ export const getErrorMessages = (error?: FrappeError | null): ParsedErrorMessage
}
})
// @ts-expect-error - some errors have _error_message
if (error?._error_message) {
eMessages.push({
// @ts-expect-error - some errors have _error_message
message: error?._error_message,
title: "Error",
indicator: "red"
})
}
if (eMessages.length === 0) {
// Get the message from the exception by removing the exc_type
const indexOfFirstColon = error?.exception?.indexOf(':')

File diff suppressed because it is too large Load Diff

View File

@@ -12,5 +12,3 @@ append_commit_message: false
languages_mapping:
two_letters_code:
pt-BR: pt_BR
zh-CN: zh
zh-TW: zh_TW

View File

@@ -3,6 +3,8 @@ import inspect
from typing import TypeVar
import frappe
from frappe.model.document import Document
from frappe.utils.user import is_website_user
__version__ = "17.0.0-dev"
@@ -153,8 +155,6 @@ def allow_regional(fn):
def check_app_permission():
from frappe.utils.user import is_website_user
if frappe.session.user == "Administrator":
return True
@@ -175,16 +175,9 @@ def normalize_ctx_input(T: type) -> callable:
- Casting the result to the specified type T
"""
from frappe.model.document import Document
def decorator(func: callable):
# conserve annotations for frappe.utils.typing_validations
@functools.wraps(
func,
assigned=(
a for a in functools.WRAPPER_ASSIGNMENTS if a not in ("__annotations__", "__annotate__")
),
)
@functools.wraps(func, assigned=(a for a in functools.WRAPPER_ASSIGNMENTS if a != "__annotations__"))
def wrapper(ctx: T | Document | dict | str, *args, **kwargs):
if isinstance(ctx, Document):
ctx = T(**ctx.as_dict())

View File

@@ -1,129 +0,0 @@
import frappe
from frappe import _
from frappe.utils import flt
from erpnext.accounts.doctype.payment_entry.payment_entry import (
get_outstanding_reference_documents,
get_payment_entry,
)
from erpnext.utilities.bulk_transaction import transaction_processing
@frappe.whitelist(methods=["POST"])
def create_payment_entries(
grouped_invoices: str | list | None = None,
ungrouped_invoices: str | list | None = None,
):
"""Create draft Payment Entries from AP report invoice selection."""
frappe.has_permission("Payment Entry", "create", throw=True)
grouped_invoices = [d for d in frappe.parse_json(grouped_invoices or "[]") if d.get("voucher_no")]
ungrouped_invoices = [d for d in frappe.parse_json(ungrouped_invoices or "[]") if d.get("voucher_no")]
if not grouped_invoices and not ungrouped_invoices:
frappe.throw(_("No Purchase Invoices selected"))
if ungrouped_invoices:
data = [{"name": d["voucher_no"]} for d in ungrouped_invoices]
transaction_processing(data, "Purchase Invoice", "Payment Entry")
if grouped_invoices:
groups = {}
for d in grouped_invoices:
key = (d["supplier"], d["party_account"])
groups.setdefault(
key, {"supplier": d["supplier"], "party_account": d["party_account"], "vouchers": []}
)["vouchers"].append(d["voucher_no"])
frappe.msgprint(
_("Started a background job to create {0} Grouped Payment Entries").format(len(groups))
)
frappe.enqueue(
make_grouped_payment_entries,
queue="long",
timeout=1500,
groups=list(groups.values()),
)
def make_grouped_payment_entries(groups):
created, failed = 0, 0
for group in groups:
supplier = group["supplier"]
try:
frappe.db.savepoint("bulk_pe")
pe = _build_grouped_payment_entry(supplier, group["party_account"], group["vouchers"])
if not pe:
frappe.db.rollback(save_point="bulk_pe")
failed += 1
frappe.log_error(
title=_("Bulk Payment Entry skipped for {0}").format(supplier),
message=_(
"No outstanding invoices found for the selected vouchers in account {0}"
).format(group["party_account"]),
)
continue
pe.flags.ignore_validate = True
pe.set_title_field()
pe.insert(ignore_mandatory=True)
created += 1
except Exception:
frappe.db.rollback(save_point="bulk_pe")
failed += 1
frappe.log_error(title=_("Bulk Payment Entry creation failed for {0}").format(supplier))
message = _("Created {0} draft Grouped Payment Entries").format(created)
if failed:
message += "" + _("{0} skipped (see Error Log)").format(failed)
frappe.publish_realtime(
"msgprint",
{"message": message, "title": _("Bulk Payment Entries"), "indicator": "green"},
user=frappe.session.user,
after_commit=True,
)
def _build_grouped_payment_entry(supplier, party_account, names):
pe = get_payment_entry("Purchase Invoice", names[0])
pe.set("references", [])
refs = get_outstanding_reference_documents(
{
"party_type": "Supplier",
"party": supplier,
"party_account": party_account,
"company": pe.company,
"vouchers": [frappe._dict(voucher_type="Purchase Invoice", voucher_no=n) for n in names],
}
)
for r in refs:
if r.voucher_type != "Purchase Invoice":
continue
pe.append(
"references",
{
"reference_doctype": r.voucher_type,
"reference_name": r.voucher_no,
"bill_no": r.get("bill_no"),
"due_date": r.get("due_date"),
"payment_term": r.get("payment_term"),
"total_amount": r.invoice_amount,
"outstanding_amount": r.outstanding_amount,
"allocated_amount": r.outstanding_amount,
"exchange_rate": r.get("exchange_rate") or 1,
},
)
if not pe.references:
return None
# received_amount is in paid_to account currency; convert to paid_from account currency for paid_amount
pe.received_amount = sum(r.allocated_amount for r in pe.references)
pe.paid_amount = flt(pe.received_amount * pe.target_exchange_rate, pe.precision("paid_amount"))
pe.set_amounts()
return pe

View File

@@ -17,7 +17,7 @@ class ERPNextAddress(Address):
def link_address(self):
"""Link address based on owner"""
if self.get("is_your_company_address"):
if self.is_your_company_address:
return
return super().link_address()
@@ -28,9 +28,7 @@ class ERPNextAddress(Address):
self.is_your_company_address = 1
def validate_reference(self):
if self.get("is_your_company_address") and not [
row for row in self.links if row.link_doctype == "Company"
]:
if self.is_your_company_address and not [row for row in self.links if row.link_doctype == "Company"]:
frappe.throw(
_(
"Address needs to be linked to a Company. Please add a row for Company in the Links table."

View File

@@ -1,7 +1,6 @@
import frappe
from frappe import _
from frappe.email import sendmail_to_system_managers
from frappe.query_builder.functions import IfNull, Sum
from frappe.utils import (
add_days,
add_months,
@@ -54,24 +53,20 @@ def validate_service_stop_date(doc):
def build_conditions(process_type, account, company):
if process_type == "Income":
item = frappe.qb.DocType("Sales Invoice Item")
parent = frappe.qb.DocType("Sales Invoice")
deferred_account = item.deferred_revenue_account
else:
item = frappe.qb.DocType("Purchase Invoice Item")
parent = frappe.qb.DocType("Purchase Invoice")
deferred_account = item.deferred_expense_account
conditions = ""
deferred_account = (
"item.deferred_revenue_account" if process_type == "Income" else "item.deferred_expense_account"
)
if account:
return deferred_account == account
conditions += f"AND {deferred_account}={frappe.db.escape(account)}"
elif company:
return parent.company == company
conditions += f"AND p.company = {frappe.db.escape(company)}"
return None
return conditions
def convert_deferred_expense_to_expense(deferred_process, start_date=None, end_date=None, conditions=None):
def convert_deferred_expense_to_expense(deferred_process, start_date=None, end_date=None, conditions=""):
# book the expense/income on the last day, but it will be trigger on the 1st of month at 12:00 AM
if not start_date:
@@ -80,25 +75,17 @@ def convert_deferred_expense_to_expense(deferred_process, start_date=None, end_d
end_date = add_days(today(), -1)
# check for the purchase invoice for which GL entries has to be done
item = frappe.qb.DocType("Purchase Invoice Item")
parent = frappe.qb.DocType("Purchase Invoice")
query = (
frappe.qb.from_(item)
.inner_join(parent)
.on(item.parent == parent.name)
.select(item.parent)
.distinct()
.where(
(item.service_start_date <= end_date)
& (item.service_end_date >= start_date)
& (item.enable_deferred_expense == 1)
& (item.docstatus == 1)
& (IfNull(item.amount, 0) > 0)
)
)
if conditions is not None:
query = query.where(conditions)
invoices = query.run(pluck=True)
invoices = frappe.db.sql_list(
f"""
select distinct item.parent
from `tabPurchase Invoice Item` item, `tabPurchase Invoice` p
where item.service_start_date<=%s and item.service_end_date>=%s
and item.enable_deferred_expense = 1 and item.parent=p.name
and item.docstatus = 1 and ifnull(item.amount, 0) > 0
{conditions}
""",
(end_date, start_date),
) # nosec
# For each invoice, book deferred expense
for invoice in invoices:
@@ -109,7 +96,7 @@ def convert_deferred_expense_to_expense(deferred_process, start_date=None, end_d
send_mail(deferred_process)
def convert_deferred_revenue_to_income(deferred_process, start_date=None, end_date=None, conditions=None):
def convert_deferred_revenue_to_income(deferred_process, start_date=None, end_date=None, conditions=""):
# book the expense/income on the last day, but it will be trigger on the 1st of month at 12:00 AM
if not start_date:
@@ -118,25 +105,17 @@ def convert_deferred_revenue_to_income(deferred_process, start_date=None, end_da
end_date = add_days(today(), -1)
# check for the sales invoice for which GL entries has to be done
item = frappe.qb.DocType("Sales Invoice Item")
parent = frappe.qb.DocType("Sales Invoice")
query = (
frappe.qb.from_(item)
.inner_join(parent)
.on(item.parent == parent.name)
.select(item.parent)
.distinct()
.where(
(item.service_start_date <= end_date)
& (item.service_end_date >= start_date)
& (item.enable_deferred_revenue == 1)
& (item.docstatus == 1)
& (IfNull(item.amount, 0) > 0)
)
)
if conditions is not None:
query = query.where(conditions)
invoices = query.run(pluck=True)
invoices = frappe.db.sql_list(
f"""
select distinct item.parent
from `tabSales Invoice Item` item, `tabSales Invoice` p
where item.service_start_date<=%s and item.service_end_date>=%s
and item.enable_deferred_revenue = 1 and item.parent=p.name
and item.docstatus = 1 and ifnull(item.amount, 0) > 0
{conditions}
""",
(end_date, start_date),
) # nosec
for invoice in invoices:
doc = frappe.get_doc("Sales Invoice", invoice)
@@ -157,39 +136,26 @@ def get_booking_dates(doc, item, posting_date=None, prev_posting_date=None):
)
if not prev_posting_date:
prev_gl_entry = frappe.get_all(
"GL Entry",
filters={
"company": doc.company,
"account": item.get(deferred_account),
"voucher_type": doc.doctype,
"voucher_no": doc.name,
"voucher_detail_no": item.name,
"is_cancelled": 0,
},
fields=["name", "posting_date"],
order_by="posting_date desc",
limit=1,
prev_gl_entry = frappe.db.sql(
"""
select name, posting_date from `tabGL Entry` where company=%s and account=%s and
voucher_type=%s and voucher_no=%s and voucher_detail_no=%s
and is_cancelled = 0
order by posting_date desc limit 1
""",
(doc.company, item.get(deferred_account), doc.doctype, doc.name, item.name),
as_dict=True,
)
je = frappe.qb.DocType("Journal Entry")
jea = frappe.qb.DocType("Journal Entry Account")
prev_gl_via_je = (
frappe.qb.from_(je)
.inner_join(jea)
.on(je.name == jea.parent)
.select(je.name, je.posting_date)
.where(
(je.company == doc.company)
& (jea.account == item.get(deferred_account))
& (jea.reference_type == doc.doctype)
& (jea.reference_name == doc.name)
& (jea.reference_detail_no == item.name)
& (jea.docstatus < 2)
)
.orderby(je.posting_date, order=frappe.qb.desc)
.limit(1)
.run(as_dict=True)
prev_gl_via_je = frappe.db.sql(
"""
SELECT p.name, p.posting_date FROM `tabJournal Entry` p, `tabJournal Entry Account` c
WHERE p.name = c.parent and p.company=%s and c.account=%s
and c.reference_type=%s and c.reference_name=%s
and c.reference_detail_no=%s and c.docstatus < 2 order by posting_date desc limit 1
""",
(doc.company, item.get(deferred_account), doc.doctype, doc.name, item.name),
as_dict=True,
)
if prev_gl_via_je:
@@ -311,47 +277,26 @@ def get_already_booked_amount(doc, item):
total_credit_debit, total_credit_debit_currency = "credit", "credit_in_account_currency"
deferred_account = "deferred_expense_account"
gle = frappe.qb.DocType("GL Entry")
gl_entries_details = (
frappe.qb.from_(gle)
.select(
Sum(gle[total_credit_debit]).as_("total_credit"),
Sum(gle[total_credit_debit_currency]).as_("total_credit_in_account_currency"),
gle.voucher_detail_no,
)
.where(
(gle.company == doc.company)
& (gle.account == item.get(deferred_account))
& (gle.voucher_type == doc.doctype)
& (gle.voucher_no == doc.name)
& (gle.voucher_detail_no == item.name)
& (gle.is_cancelled == 0)
)
.groupby(gle.voucher_detail_no)
.run(as_dict=True)
gl_entries_details = frappe.db.sql(
"""
select sum({}) as total_credit, sum({}) as total_credit_in_account_currency, voucher_detail_no
from `tabGL Entry` where company=%s and account=%s and voucher_type=%s and voucher_no=%s and voucher_detail_no=%s
and is_cancelled = 0
group by voucher_detail_no
""".format(total_credit_debit, total_credit_debit_currency),
(doc.company, item.get(deferred_account), doc.doctype, doc.name, item.name),
as_dict=True,
)
je = frappe.qb.DocType("Journal Entry")
jea = frappe.qb.DocType("Journal Entry Account")
journal_entry_details = (
frappe.qb.from_(je)
.inner_join(jea)
.on(je.name == jea.parent)
.select(
Sum(jea[total_credit_debit]).as_("total_credit"),
Sum(jea[total_credit_debit_currency]).as_("total_credit_in_account_currency"),
jea.reference_detail_no,
)
.where(
(je.company == doc.company)
& (jea.account == item.get(deferred_account))
& (jea.reference_type == doc.doctype)
& (jea.reference_name == doc.name)
& (jea.reference_detail_no == item.name)
& (je.docstatus < 2)
)
.groupby(jea.reference_detail_no)
.run(as_dict=True)
journal_entry_details = frappe.db.sql(
"""
SELECT sum(c.{}) as total_credit, sum(c.{}) as total_credit_in_account_currency, reference_detail_no
FROM `tabJournal Entry` p , `tabJournal Entry Account` c WHERE p.name = c.parent and
p.company = %s and c.account=%s and c.reference_type=%s and c.reference_name=%s and c.reference_detail_no=%s
and p.docstatus < 2 group by reference_detail_no
""".format(total_credit_debit, total_credit_debit_currency),
(doc.company, item.get(deferred_account), doc.doctype, doc.name, item.name),
as_dict=True,
)
already_booked_amount = gl_entries_details[0].total_credit if gl_entries_details else 0
@@ -582,7 +527,6 @@ def make_gl_entries(
frappe.db.commit()
except Exception as e:
if frappe.in_test:
frappe.db.rollback()
doc.log_error(f"Error while processing deferred accounting for Invoice {doc.name}")
raise e
else:

View File

@@ -121,7 +121,6 @@ class Account(NestedSet):
self.validate_account_currency()
self.validate_root_company_and_sync_account_to_children()
self.validate_receivable_payable_account_type()
self.validate_stock_account_type_change()
def validate_parent_child_account_type(self):
if self.parent_account:
@@ -213,36 +212,6 @@ class Account(NestedSet):
frappe.msgprint(msg)
self.add_comment("Comment", msg)
def validate_stock_account_type_change(self):
doc_before_save = self.get_doc_before_save()
if not (doc_before_save and doc_before_save.account_type == "Stock"):
return
if self.account_type == "Stock":
return
if self.stock_ledger_entry_exists():
frappe.throw(
_(
"The account type of {0} cannot be changed from {1} because stock ledger entries exist against it."
).format(frappe.bold(self.name), frappe.bold(_("Stock")))
)
def stock_ledger_entry_exists(self):
from erpnext.stock import get_warehouse_account_map
warehouse_account = get_warehouse_account_map(self.company)
warehouses = [wh for wh, details in warehouse_account.items() if details.account == self.name]
if not warehouses:
return False
return bool(
frappe.db.count(
"Stock Ledger Entry",
filters={"warehouse": ("in", warehouses), "is_cancelled": 0},
)
)
def validate_root_details(self):
doc_before_save = self.get_doc_before_save()
@@ -265,7 +234,7 @@ class Account(NestedSet):
if not frappe.db.get_value(
"Account", {"account_name": self.account_name, "company": ancestors[0]}, "name"
):
frappe.throw(_("Please add the account to root level Company - {0}").format(ancestors[0]))
frappe.throw(_("Please add the account to root level Company - {}").format(ancestors[0]))
elif self.parent_account:
descendants = get_descendants_of("Company", self.company)
if not descendants:
@@ -690,15 +659,8 @@ def _ensure_idle_system():
last_gl_update = None
try:
if frappe.db.db_type == "postgres":
# The MariaDB branch blocks new GL inserts via the gap lock its for_update read takes;
# a postgres row lock never blocks inserts, so take an EXCLUSIVE table lock instead --
# writers block until the rename commits, readers don't. NOWAIT mirrors wait=False.
frappe.db.sql("LOCK TABLE `tabGL Entry` IN EXCLUSIVE MODE NOWAIT")
last_gl_update = frappe.db.get_value("GL Entry", {}, "modified")
else:
# We also lock inserts to GL entry table with for_update here.
last_gl_update = frappe.db.get_value("GL Entry", {}, "modified", for_update=True, wait=False)
# We also lock inserts to GL entry table with for_update here.
last_gl_update = frappe.db.get_value("GL Entry", {}, "modified", for_update=True, wait=False)
except frappe.QueryTimeoutError:
# wait=False fails immediately if there's an active transaction.
last_gl_update = add_to_date(None, seconds=-1)
@@ -709,7 +671,7 @@ def _ensure_idle_system():
if last_gl_update > add_to_date(None, minutes=-5):
frappe.throw(
_(
"Last GL Entry update was done {0}. This operation is not allowed while system is actively being used. Please wait for 5 minutes before retrying."
"Last GL Entry update was done {}. This operation is not allowed while system is actively being used. Please wait for 5 minutes before retrying."
).format(pretty_date(last_gl_update)),
title=_("System In Use"),
)

View File

@@ -236,9 +236,9 @@ frappe.treeview_settings["Account"] = {
function () {
let root_company = treeview.page.fields_dict.root_company.get_value();
if (root_company) {
frappe.throw(
__("Please add the account to root level Company - {0}", [root_company])
);
frappe.throw(__("Please add the account to root level Company - {0}"), [
root_company,
]);
} else {
treeview.new_node();
}

View File

@@ -137,7 +137,7 @@ def get_charts_for_country(country: str, with_standard: bool = False):
def _get_chart_name(content):
if content:
content = frappe.parse_json(content)
content = json.loads(content)
if (
content and content.get("disabled", "No") == "No"
) or frappe.local.flags.allow_unverified_charts:

View File

@@ -24,8 +24,7 @@
"account_number": "11530"
},
"account_number": "115",
"is_group": 1,
"account_type": "Bank"
"is_group": 1
},
"Trade Receivables": {
"Trade Debtors": {
@@ -530,13 +529,6 @@
"account_number": "630",
"is_group": 1
},
"Accrued Manufacturing Expenses": {
"Accrued Expenses - Manufacturing": {
"account_number": "63510"
},
"account_number": "635",
"is_group": 1
},
"account_number": "63",
"is_group": 1
},
@@ -822,4 +814,4 @@
"root_type": "Expense"
}
}
}
}

View File

@@ -37,10 +37,6 @@
"account_type": "Stock",
"account_category": "Stock Assets"
},
"Stock Delivered But Not Billed": {
"account_type": "Stock Delivered But Not Billed",
"account_category": "Stock Assets"
},
"account_type": "Stock",
"account_category": "Stock Assets"
},
@@ -227,6 +223,10 @@
"Stock Received But Not Billed": {
"account_type": "Stock Received But Not Billed",
"account_category": "Trade Payables"
},
"Stock Delivered But Not Billed": {
"account_type": "Stock Delivered But Not Billed",
"account_category": "Trade Payables"
}
},
"Duties and Taxes": {

View File

@@ -22,12 +22,12 @@
"account_type": "Cash"
},
"Petty Cash Fund": {
"account_number": "1110",
"account_number": "1200",
"is_group": 1,
"root_type": "Asset",
"account_type": "Cash",
"Petty Cash Fund": {
"account_number": "1111",
"account_number": "1201",
"is_group": 0,
"root_type": "Asset",
"account_type": "Cash"
@@ -35,16 +35,10 @@
}
},
"Bank Accounts": {
"account_number": "1200",
"account_number": "1102",
"is_group": 1,
"root_type": "Asset",
"account_type": "Bank",
"Cash in Bank - Checking Account": {
"account_number": "1201",
"is_group": 0,
"root_type": "Asset",
"account_type": "Bank"
}
"account_type": "Bank"
},
"Advances to Officers & Employees": {
"account_number": "1290",
@@ -110,20 +104,25 @@
"account_number": "1511",
"is_group": 0,
"root_type": "Asset"
},
"Factory Overhead Variance": {
"account_number": "1512",
"is_group": 0,
"root_type": "Asset"
}
},
"Finished Goods": {
"account_number": "1540",
"account_number": "1520",
"is_group": 1,
"root_type": "Asset",
"Finished Goods Inventory": {
"account_number": "1541",
"account_number": "1531",
"is_group": 0,
"root_type": "Asset",
"account_type": "Stock"
},
"Inventory in Transit": {
"account_number": "1542",
"account_number": "1532",
"is_group": 0,
"root_type": "Asset",
"account_type": "Stock Adjustment"
@@ -269,7 +268,7 @@
"root_type": "Asset"
}
},
"Intangible Assets": {
"System Development": {
"account_number": "1940",
"is_group": 1,
"root_type": "Asset",
@@ -278,17 +277,6 @@
"is_group": 0,
"root_type": "Asset"
}
},
"Accumulated Amortization - Intangible Assets": {
"account_number": "1950",
"is_group": 1,
"root_type": "Asset",
"Accum Amortization - System Development": {
"account_number": "1951",
"is_group": 0,
"root_type": "Asset",
"account_type": "Accumulated Depreciation"
}
}
}
},
@@ -418,7 +406,8 @@
"Customer Deposits": {
"account_number": "2500",
"is_group": 0,
"root_type": "Liability"
"root_type": "Liability",
"account_type": "Payable"
}
},
"Non Current Liabilities": {
@@ -574,28 +563,6 @@
"is_group": 0,
"root_type": "Income"
}
},
"Exchange Gain": {
"account_number": "6030",
"is_group": 1,
"root_type": "Income",
"Exchange Gain - Detail": {
"account_number": "6031",
"is_group": 0,
"root_type": "Income",
"account_type": "Indirect Income"
}
},
"Gain on Asset Disposal": {
"account_number": "6040",
"is_group": 1,
"root_type": "Income",
"Gain on Asset Disposal - Detail": {
"account_number": "6041",
"is_group": 0,
"root_type": "Income",
"account_type": "Indirect Income"
}
}
}
},
@@ -608,7 +575,7 @@
"is_group": 1,
"root_type": "Expense",
"Cost of Goods Sold": {
"account_number": "5002",
"account_number": "5010",
"is_group": 0,
"root_type": "Expense",
"account_type": "Cost of Goods Sold"
@@ -861,61 +828,20 @@
"root_type": "Expense"
}
},
"Other Expenses": {
"account_number": "5200",
"is_group": 1,
"root_type": "Expense",
"Bank Charges": {
"account_number": "5201",
"is_group": 0,
"root_type": "Expense",
"account_type": "Indirect Expense"
},
"Interest Expenses Bank": {
"account_number": "5202",
"is_group": 0,
"root_type": "Expense",
"account_type": "Indirect Expense"
},
"Write Off": {
"account_number": "5203",
"is_group": 0,
"root_type": "Expense",
"account_type": "Indirect Expense"
},
"Exchange Loss": {
"account_number": "5204",
"is_group": 0,
"root_type": "Expense",
"account_type": "Indirect Expense"
},
"Loss on Asset Disposal": {
"account_number": "5205",
"is_group": 0,
"root_type": "Expense",
"account_type": "Indirect Expense"
}
},
"Provision For Income Tax": {
"account_number": "5300",
"is_group": 0,
"root_type": "Expense",
"account_type": "Tax"
},
"Stock Adjustment": {
"account_number": "5400",
"account_number": "5200",
"is_group": 0,
"root_type": "Expense",
"account_type": "Stock Adjustment"
},
"Round Off": {
"account_number": "5500",
"account_number": "5300",
"is_group": 0,
"root_type": "Expense",
"account_type": "Round Off"
},
"Expenses Included In Valuation": {
"account_number": "5600",
"account_number": "5400",
"is_group": 0,
"root_type": "Expense",
"account_type": "Expenses Included In Valuation"

View File

@@ -306,31 +306,6 @@ class TestAccount(ERPNextTestSuite):
acc.account_currency = "USD"
self.assertRaises(frappe.ValidationError, acc.save)
def test_stock_account_type_change_with_ledger_entries(self):
from erpnext.stock.doctype.stock_entry.stock_entry_utils import make_stock_entry
company = "_Test Company with perpetual inventory"
warehouse = "Stores - TCP1"
stock_account = get_warehouse_account(frappe.get_doc("Warehouse", warehouse))
make_stock_entry(
item_code="_Test Item",
target=warehouse,
company=company,
qty=5,
basic_rate=100,
)
account = frappe.get_doc("Account", stock_account)
self.assertEqual(account.account_type, "Stock")
account.account_type = ""
self.assertRaises(frappe.ValidationError, account.save)
account.reload()
account.account_name = f"{account.account_name} Updated"
account.save() # non-type change stays allowed
def test_account_balance(self):
from erpnext.accounts.utils import get_balance_on

View File

@@ -1,59 +1,10 @@
# Copyright (c) 2026, Frappe Technologies Pvt. Ltd. and Contributors
# Copyright (c) 2023, Frappe Technologies Pvt. Ltd. and Contributors
# See license.txt
from erpnext.accounts.doctype.account_closing_balance.account_closing_balance import (
aggregate_with_last_account_closing_balance,
generate_key,
)
# import frappe
from erpnext.tests.utils import ERPNextTestSuite
def entry(**overrides):
row = {"debit": 0, "credit": 0, "debit_in_account_currency": 0, "credit_in_account_currency": 0}
row.update(overrides)
return row
class TestAccountClosingBalance(ERPNextTestSuite):
"""The closing-balance snapshot is built by merging this period's entries with the
previous period's. These lock the merge/key logic that drives that carry-forward."""
def test_matching_entries_are_summed(self):
# this is how a prior-period balance carries forward into the current one
merged = aggregate_with_last_account_closing_balance(
[
entry(account="Cash - _TC", debit=100, debit_in_account_currency=100),
entry(
account="Cash - _TC",
debit=50,
credit=20,
debit_in_account_currency=50,
credit_in_account_currency=20,
),
],
[],
)
self.assertEqual(len(merged), 1)
row = next(iter(merged.values()))
self.assertEqual(row["debit"], 150)
self.assertEqual(row["credit"], 20)
# the account-currency columns are accumulated in the same pass
self.assertEqual(row["debit_in_account_currency"], 150)
self.assertEqual(row["credit_in_account_currency"], 20)
def test_entries_are_kept_separate_per_dimension(self):
merged = aggregate_with_last_account_closing_balance(
[
entry(account="Cash - _TC", cost_center="CC1", debit=100, debit_in_account_currency=100),
entry(account="Cash - _TC", cost_center="CC2", debit=40, debit_in_account_currency=40),
],
[],
)
self.assertEqual(len(merged), 2)
def test_period_closing_flag_is_part_of_the_key(self):
# a P&L reversal (flag 0) and a closing-account entry (flag 1) for the same
# account must not merge, so the flag has to distinguish their keys
key_reversal, _ = generate_key(entry(account="Sales - _TC", is_period_closing_voucher_entry=0), [])
key_closing, _ = generate_key(entry(account="Sales - _TC", is_period_closing_voucher_entry=1), [])
self.assertNotEqual(key_reversal, key_closing)
pass

View File

@@ -224,7 +224,7 @@ def disable_dimension(doc: str):
def toggle_disabling(doc):
doc = frappe.parse_json(doc)
doc = json.loads(doc)
if doc.get("disabled"):
df = {"read_only": 1}
@@ -359,13 +359,3 @@ def create_accounting_dimensions_for_doctype(doctype):
create_custom_field(doctype, df, ignore_validate=True)
frappe.clear_cache(doctype=doctype)
def get_dimension_fieldname(dim_doctype: str) -> str:
"""
Return the `GL Entry` fieldname for a given dimension.
"""
if dim_doctype in ("Cost Center", "Project"):
return frappe.scrub(dim_doctype)
return frappe.db.get_value("Accounting Dimension", {"document_type": dim_doctype}, "fieldname")

View File

@@ -6,7 +6,7 @@ frappe.ui.form.on("Accounting Dimension Filter", {
let help_content = `<table class="table table-bordered" style="background-color: var(--scrollbar-track-color);">
<tr><td>
<p>
<svg class="icon icon-sm"><use href="#icon-info"></use></svg>
<i class="fa fa-hand-right"></i>
{{__('Note: On checking Is Mandatory the accounting dimension will become mandatory against that specific account for all accounting transactions')}}
</p>
</td></tr>

View File

@@ -5,7 +5,6 @@
import frappe
from frappe import _
from frappe.model.document import Document
from frappe.utils import getdate, nowdate
class OverlapError(frappe.ValidationError):
@@ -37,20 +36,8 @@ class AccountingPeriod(Document):
# end: auto-generated types
def validate(self):
self.validate_dates()
self.validate_overlap()
def validate_dates(self):
if getdate(self.start_date) > getdate(self.end_date):
frappe.throw(_("Start Date cannot be after End Date"))
if getdate(self.end_date) > getdate(nowdate()):
frappe.throw(
_(
"Accounting Period cannot be created for a future date. End Date {0} is after today."
).format(frappe.bold(frappe.format(self.end_date, "Date")))
)
def before_insert(self):
self.bootstrap_doctypes_for_closing()

View File

@@ -2,7 +2,7 @@
# See license.txt
import frappe
from frappe.utils import nowdate
from frappe.utils import add_months, nowdate
from erpnext.accounts.doctype.accounting_period.accounting_period import (
ClosedAccountingPeriod,
@@ -93,7 +93,7 @@ def create_accounting_period(**args):
accounting_period = frappe.new_doc("Accounting Period")
accounting_period.start_date = args.start_date or nowdate()
accounting_period.end_date = args.end_date or nowdate()
accounting_period.end_date = args.end_date or add_months(nowdate(), 1)
accounting_period.company = args.company or "_Test Company"
accounting_period.period_name = args.period_name or "_Test_Period_Name_1"
accounting_period.append("closed_documents", {"document_type": "Sales Invoice", "closed": 1})

View File

@@ -10,7 +10,7 @@ frappe.ui.form.on("Accounts Settings", {
},
};
});
if (!frm.naming_controller) frm.naming_controller = new frappe.ui.NamingSeriesController(frm);
if (!frm.naming_controller) frm.naming_controller = new erpnext.NamingSeriesController(frm);
frm.naming_controller.render_table("transaction_naming_html", get_transactions(frm));
},

View File

@@ -22,8 +22,6 @@
"allow_multi_currency_invoices_against_single_party_account",
"confirm_before_resetting_posting_date",
"preview_mode",
"stock_expense_section",
"book_stock_expense_gl_entries",
"analytics_section",
"enable_discounts_and_margin",
"enable_accounting_dimensions",
@@ -78,8 +76,6 @@
"over_billing_allowance",
"credit_controller",
"role_allowed_to_over_bill",
"enable_overdue_billing_threshold",
"role_allowed_to_bypass_overdue_billing",
"column_break_11",
"assets_tab",
"asset_settings_section",
@@ -91,7 +87,6 @@
"period_closing_settings_section",
"ignore_account_closing_balance",
"use_legacy_controller_for_pcv",
"pcv_job_timeout",
"column_break_25",
"reports_tab",
"remarks_section",
@@ -276,21 +271,6 @@
"label": "Role Allowed to over bill ",
"options": "Role"
},
{
"default": "0",
"description": "Block a new Sales Invoice when the customer's overdue amount exceeds the Overdue Limit set on the customer.",
"fieldname": "enable_overdue_billing_threshold",
"fieldtype": "Check",
"label": "Restrict Customer Over Billing"
},
{
"depends_on": "eval:doc.enable_overdue_billing_threshold",
"description": "Users with this role can still submit invoices for customers who have crossed their Overdue Limit.",
"fieldname": "role_allowed_to_bypass_overdue_billing",
"fieldtype": "Link",
"label": "Role Allowed to Bypass Over Billing Restriction",
"options": "Role"
},
{
"fieldname": "period_closing_settings_section",
"fieldtype": "Section Break"
@@ -632,14 +612,6 @@
"fieldtype": "Check",
"label": "Use legacy controller for Period Closing Voucher"
},
{
"default": "3600",
"depends_on": "eval: !doc.use_legacy_controller_for_pcv",
"description": "Timeout (in seconds) for each background job enqueued by Process Period Closing Voucher",
"fieldname": "pcv_job_timeout",
"fieldtype": "Int",
"label": "PCV Job Timeout (seconds)"
},
{
"description": "Users with this role will be notified if the asset depreciation gets failed",
"fieldname": "role_to_notify_on_depreciation_failure",
@@ -776,18 +748,6 @@
"description": "Changing the account in any transaction of the DocTypes listed below will trigger a repost. To prevent reposting, remove the relevant DocType from the list.",
"fieldname": "column_break_mfor",
"fieldtype": "Column Break"
},
{
"fieldname": "stock_expense_section",
"fieldtype": "Section Break",
"label": "Stock Expense Accounting"
},
{
"default": "0",
"description": "Books Purchase Expense and Expenses Added To Stock account pairs against stock value. On enabling this, the accounts become mandatory in Company or Item Defaults for Purchase Receipt, Purchase Invoice, Stock Entry, Stock Reconciliation and Landed Cost Voucher",
"fieldname": "book_stock_expense_gl_entries",
"fieldtype": "Check",
"label": "Book Stock Expense GL Entries"
}
],
"grid_page_length": 50,
@@ -796,7 +756,7 @@
"index_web_pages_for_search": 1,
"issingle": 1,
"links": [],
"modified": "2026-07-15 17:00:00.000000",
"modified": "2026-06-03 13:11:54.721495",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Accounts Settings",

View File

@@ -62,7 +62,6 @@ class AccountsSettings(Document):
book_asset_depreciation_entry_automatically: DF.Check
book_deferred_entries_based_on: DF.Literal["Days", "Months"]
book_deferred_entries_via_journal_entry: DF.Check
book_stock_expense_gl_entries: DF.Check
book_tax_discount_loss: DF.Check
calculate_depr_using_total_days: DF.Check
check_supplier_invoice_uniqueness: DF.Check
@@ -78,7 +77,6 @@ class AccountsSettings(Document):
enable_fuzzy_matching: DF.Check
enable_immutable_ledger: DF.Check
enable_loyalty_point_program: DF.Check
enable_overdue_billing_threshold: DF.Check
enable_party_matching: DF.Check
enable_subscription: DF.Check
exchange_gain_loss_posting_date: DF.Literal["Invoice", "Payment", "Reconciliation Date"]
@@ -92,13 +90,11 @@ class AccountsSettings(Document):
make_payment_via_journal_entry: DF.Check
merge_similar_account_heads: DF.Check
over_billing_allowance: DF.Currency
pcv_job_timeout: DF.Int
preview_mode: DF.Check
receivable_payable_fetch_method: DF.Literal["Buffered Cursor", "UnBuffered Cursor"]
receivable_payable_remarks_length: DF.Int
reconciliation_queue_size: DF.Int
repost_allowed_types: DF.Table[RepostAllowedTypes]
role_allowed_to_bypass_overdue_billing: DF.Link | None
role_allowed_to_over_bill: DF.Link | None
role_to_notify_on_depreciation_failure: DF.Link | None
role_to_override_stop_action: DF.Link | None
@@ -154,10 +150,6 @@ class AccountsSettings(Document):
toggle_subscription_sections(not self.enable_subscription)
clear_cache = True
if old_doc.enable_overdue_billing_threshold != self.enable_overdue_billing_threshold:
toggle_overdue_billing_threshold_field(not self.enable_overdue_billing_threshold)
clear_cache = True
if clear_cache:
frappe.clear_cache()
@@ -249,10 +241,6 @@ def toggle_subscription_sections(hide):
create_property_setter_for_hiding_field(doctype, "subscription_section", hide)
def toggle_overdue_billing_threshold_field(hide):
create_property_setter_for_hiding_field("Customer Credit Limit", "overdue_billing_threshold", hide)
def create_property_setter_for_hiding_field(doctype, field_name, hide):
make_property_setter(
doctype,

View File

@@ -22,13 +22,11 @@ class TestAdvancePaymentLedgerEntry(ERPNextTestSuite, AccountsTestMixin):
"""
def setUp(self):
self.company = "_Test Company"
self.customer = "_Test Customer"
self.supplier = "_Test Supplier"
self.item = "_Test Item"
self.cash = "Cash - _TC"
self.debtors_usd = "_Test Receivable USD - _TC"
self.creditors_usd = "_Test Payable USD - _TC"
self.create_company()
self.create_usd_receivable_account()
self.create_usd_payable_account()
self.create_item()
self.clear_old_entries()
def create_sales_order(self, qty=1, rate=100, currency="INR", do_not_submit=False):
"""

View File

@@ -107,7 +107,7 @@ def get_party_bank_account(party_type, party):
)
def get_default_company_bank_account(company, party_type, party, ignore_permissions=True):
def get_default_company_bank_account(company, party_type, party):
default_company_bank_account = frappe.db.get_value(party_type, party, "default_bank_account")
if default_company_bank_account:
if company != frappe.get_cached_value("Bank Account", default_company_bank_account, "company"):
@@ -118,14 +118,6 @@ def get_default_company_bank_account(company, party_type, party, ignore_permissi
"Bank Account", {"company": company, "is_company_account": 1, "is_default": 1}
)
if not ignore_permissions:
default_company_bank_account = (
default_company_bank_account
if default_company_bank_account
and frappe.get_cached_doc("Bank Account", default_company_bank_account).has_permission("select")
else None
)
return default_company_bank_account
@@ -196,7 +188,7 @@ def get_closing_balance_as_per_statement(bank_account: str, date: str):
return {"balance": 0, "date": None}
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def set_closing_balance_as_per_statement(bank_account: str, date: str | datetime.date, balance: float):
"""
Set the closing balance as per statement for a bank account and date

View File

@@ -1,6 +1,5 @@
{
"actions": [],
"allow_bulk_edit": 1,
"allow_rename": 1,
"creation": "2026-04-11 19:48:13.622253",
"doctype": "DocType",
@@ -8,8 +7,7 @@
"field_order": [
"bank_account",
"date",
"balance",
"company"
"balance"
],
"fields": [
{
@@ -33,20 +31,12 @@
"in_list_view": 1,
"label": "Balance",
"reqd": 1
},
{
"fetch_from": "bank_account.company",
"fieldname": "company",
"fieldtype": "Link",
"label": "Company",
"options": "Company",
"read_only": 1
}
],
"grid_page_length": 50,
"index_web_pages_for_search": 1,
"links": [],
"modified": "2026-06-16 22:17:48.007982",
"modified": "2026-04-11 19:49:45.374695",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Bank Account Balance",

View File

@@ -16,7 +16,6 @@ class BankAccountBalance(Document):
balance: DF.Currency
bank_account: DF.Link
company: DF.Link | None
date: DF.Date
# end: auto-generated types

View File

@@ -7,7 +7,7 @@ from frappe import _, msgprint
from frappe.model.document import Document
from frappe.query_builder import Case
from frappe.query_builder.custom import ConstantColumn
from frappe.query_builder.functions import Coalesce, Max, Sum
from frappe.query_builder.functions import Coalesce, Sum
from frappe.utils import cint, flt, fmt_money, getdate
from pypika import Order
@@ -94,7 +94,6 @@ class BankClearance(Document):
invalid_document = []
invalid_cheque_date = []
entries_to_update = []
self.check_permission("write")
def validate_entry(d):
is_valid = True
@@ -195,17 +194,14 @@ def get_payment_entries_for_bank_clearance(
.select(
ConstantColumn("Journal Entry").as_("payment_document"),
journal_entry.name.as_("payment_entry"),
# non-grouped columns are constant per grouped JE name / account (against_account is
# arbitrary per group on MySQL) -> Max() keeps the GROUP BY valid on postgres with the
# same value MySQL picked.
Max(journal_entry.cheque_no).as_("cheque_number"),
Max(journal_entry.cheque_date).as_("cheque_date"),
journal_entry.cheque_no.as_("cheque_number"),
journal_entry.cheque_date,
Sum(journal_entry_account.debit_in_account_currency).as_("debit"),
Sum(journal_entry_account.credit_in_account_currency).as_("credit"),
Max(journal_entry.posting_date).as_("posting_date"),
Max(journal_entry_account.against_account).as_("against_account"),
Max(journal_entry.clearance_date).as_("clearance_date"),
Max(journal_entry_account.account_currency).as_("account_currency"),
journal_entry.posting_date,
journal_entry_account.against_account,
journal_entry.clearance_date,
journal_entry_account.account_currency,
)
.where(
(journal_entry_account.account == account)
@@ -218,13 +214,12 @@ def get_payment_entries_for_bank_clearance(
if not include_reconciled_entries:
journal_entry_query = journal_entry_query.where(
(journal_entry.clearance_date.isnull())
| (journal_entry.clearance_date == ("0000-00-00" if frappe.db.db_type != "postgres" else None))
(journal_entry.clearance_date.isnull()) | (journal_entry.clearance_date == "0000-00-00")
)
journal_entries = (
journal_entry_query.groupby(journal_entry_account.account, journal_entry.name)
.orderby(Max(journal_entry.posting_date))
.orderby(journal_entry.posting_date)
.orderby(journal_entry.name, order=Order.desc)
).run(as_dict=True)
@@ -294,8 +289,7 @@ def get_payment_entries_for_bank_clearance(
if not include_reconciled_entries:
payment_entry_query = payment_entry_query.where(
(pe.clearance_date.isnull())
| (pe.clearance_date == ("0000-00-00" if frappe.db.db_type != "postgres" else None))
(pe.clearance_date.isnull()) | (pe.clearance_date == "0000-00-00")
)
payment_entries = (payment_entry_query.orderby(pe.posting_date).orderby(pe.name, order=Order.desc)).run(
@@ -332,8 +326,7 @@ def get_payment_entries_for_bank_clearance(
if not include_reconciled_entries:
paid_purchase_invoices_query = paid_purchase_invoices_query.where(
(pi.clearance_date.isnull())
| (pi.clearance_date == ("0000-00-00" if frappe.db.db_type != "postgres" else None))
(pi.clearance_date.isnull()) | (pi.clearance_date == "0000-00-00")
)
paid_purchase_invoices = (
@@ -373,8 +366,7 @@ def get_payment_entries_for_bank_clearance(
if not include_reconciled_entries:
pos_sales_invoices_query = pos_sales_invoices_query.where(
(si_payment.clearance_date.isnull())
| (si_payment.clearance_date == ("0000-00-00" if frappe.db.db_type != "postgres" else None))
(si_payment.clearance_date.isnull()) | (si_payment.clearance_date == "0000-00-00")
)
pos_sales_invoices = (

View File

@@ -9,13 +9,6 @@ cur_frm.add_fetch("bank", "swift_number", "swift_number");
frappe.ui.form.on("Bank Guarantee", {
setup: function (frm) {
frm.set_query("reference_doctype", function () {
return {
filters: {
name: ["in", ["Sales Order", "Purchase Order"]],
},
};
});
frm.set_query("bank_account", function () {
return {
filters: {

View File

@@ -1,6 +1,5 @@
{
"actions": [],
"allow_bulk_edit": 1,
"autoname": "ACC-BG-.YYYY.-.#####",
"creation": "2016-12-17 10:43:35.731631",
"doctype": "DocType",
@@ -51,7 +50,8 @@
"fieldname": "reference_doctype",
"fieldtype": "Link",
"label": "Reference Document Type",
"options": "DocType"
"options": "DocType",
"read_only": 1
},
{
"fieldname": "reference_docname",
@@ -60,14 +60,14 @@
"options": "reference_doctype"
},
{
"depends_on": "eval: doc.reference_doctype == \"Sales Order\"",
"depends_on": "eval: doc.bg_type == \"Receiving\"",
"fieldname": "customer",
"fieldtype": "Link",
"label": "Customer",
"options": "Customer"
},
{
"depends_on": "eval: doc.reference_doctype == \"Purchase Order\"",
"depends_on": "eval: doc.bg_type == \"Providing\"",
"fieldname": "supplier",
"fieldtype": "Link",
"label": "Supplier",
@@ -218,11 +218,10 @@
"grid_page_length": 50,
"is_submittable": 1,
"links": [],
"modified": "2026-05-25 18:12:10.768835",
"modified": "2025-08-29 11:52:33.550847",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Bank Guarantee",
"naming_rule": "Expression",
"owner": "Administrator",
"permissions": [
{

View File

@@ -1,76 +1,8 @@
# Copyright (c) 2026, Frappe Technologies Pvt. Ltd. and Contributors
# Copyright (c) 2018, Frappe Technologies Pvt. Ltd. and Contributors
# See license.txt
import frappe
from frappe.utils import flt
from erpnext.accounts.doctype.bank_guarantee.bank_guarantee import get_voucher_details
from erpnext.selling.doctype.sales_order.test_sales_order import make_sales_order
from erpnext.tests.utils import ERPNextTestSuite
BANK = "_Test BG Bank"
class TestBankGuarantee(ERPNextTestSuite):
"""Bank Guarantee records a guarantee issued/received against a customer or
supplier. validate() needs a party; on_submit() needs the bank details filled in."""
def setUp(self):
frappe.set_user("Administrator")
if not frappe.db.exists("Bank", BANK):
frappe.get_doc({"doctype": "Bank", "bank_name": BANK}).insert()
def make_bg(self, **args):
args = frappe._dict(args)
doc = frappe.new_doc("Bank Guarantee")
doc.bg_type = args.bg_type or "Receiving"
doc.amount = args.amount if args.amount is not None else 1000
doc.start_date = args.start_date or "2026-06-01"
if args.end_date:
doc.end_date = args.end_date
doc.customer = args.get("customer", "_Test Customer")
doc.supplier = args.get("supplier")
# fields on_submit requires — present by default, cleared per-test to assert the guard
doc.bank_guarantee_number = args.get("bank_guarantee_number", "BG-001")
doc.name_of_beneficiary = args.get("name_of_beneficiary", "Test Beneficiary")
doc.bank = args.get("bank", BANK)
return doc
def test_validate_requires_customer_or_supplier(self):
doc = self.make_bg(customer=None)
self.assertRaises(frappe.ValidationError, doc.insert)
def test_submit_requires_guarantee_number(self):
doc = self.make_bg(bank_guarantee_number="")
doc.insert()
self.assertRaises(frappe.ValidationError, doc.submit)
def test_submit_requires_beneficiary_name(self):
doc = self.make_bg(name_of_beneficiary="")
doc.insert()
self.assertRaises(frappe.ValidationError, doc.submit)
def test_submit_requires_bank(self):
doc = self.make_bg(bank="")
doc.insert()
self.assertRaises(frappe.ValidationError, doc.submit)
def test_valid_guarantee_submits(self):
doc = self.make_bg()
doc.insert()
doc.submit()
self.assertEqual(frappe.db.get_value("Bank Guarantee", doc.name, "docstatus"), 1)
def test_get_voucher_details_for_receiving(self):
so = make_sales_order()
details = get_voucher_details("Receiving", so.name)
self.assertEqual(details.customer, so.customer)
self.assertEqual(flt(details.grand_total), flt(so.grand_total))
def test_end_date_before_start_date_is_not_validated(self):
# SUSPECTED BUG: validate() never checks that end_date >= start_date, so a
# guarantee that expires before it starts saves cleanly. Locking the current
# (wrong) behaviour so a future fix that adds the check trips this test.
doc = self.make_bg(start_date="2026-06-30", end_date="2026-06-01")
doc.insert()
self.assertTrue(frappe.db.exists("Bank Guarantee", doc.name))
pass

View File

@@ -8,7 +8,7 @@ import frappe
from frappe import _
from frappe.model.document import Document
from frappe.query_builder.custom import ConstantColumn
from frappe.query_builder.functions import Max, Sum
from frappe.query_builder.functions import Sum
from frappe.utils import cint, create_batch, flt
from erpnext import get_default_cost_center
@@ -116,7 +116,7 @@ def get_account_balance(bank_account: str, till_date: str | date, company: str):
return flt(balance_as_per_system) - flt(total_debit) + flt(total_credit) + amounts_not_reflected_in_system
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def update_bank_transaction(
bank_transaction_name: str, reference_number: str, party_type: str | None = None, party: str | None = None
):
@@ -146,7 +146,7 @@ def update_bank_transaction(
)[0]
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def create_journal_entry_bts(
bank_transaction_name: str,
reference_number: str | None = None,
@@ -305,7 +305,7 @@ def create_journal_entry_bts(
return reconcile_vouchers(bank_transaction_name, vouchers, is_new_voucher=True)
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def create_payment_entry_bts(
bank_transaction_name: str,
reference_number: str | None = None,
@@ -500,7 +500,7 @@ def create_bulk_internal_transfer(bank_transaction_names: list[str | int], bank_
return output
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def create_internal_transfer(
bank_transaction_name: str | int,
posting_date: str | date,
@@ -518,7 +518,6 @@ def create_internal_transfer(
"""
bank_transaction = frappe.get_doc("Bank Transaction", bank_transaction_name)
bank_transaction.check_permission("write")
bank_account = frappe.get_cached_value("Bank Account", bank_transaction.bank_account, "account")
company = frappe.get_cached_value("Account", bank_account, "company")
@@ -779,6 +778,7 @@ def create_bulk_payment_entry_and_reconcile(
"""
Create a payment entry and reconcile it with the bank transaction
"""
output = []
for bank_transaction_name in bank_transaction_names:
@@ -1057,10 +1057,10 @@ def get_auto_reconcile_message(partially_reconciled, reconciled):
return alert_message, indicator
@frappe.whitelist(methods=["POST"])
def reconcile_vouchers(bank_transaction_name: str | int, vouchers: str | list, is_new_voucher: bool = False):
@frappe.whitelist()
def reconcile_vouchers(bank_transaction_name: str | int, vouchers: str, is_new_voucher: bool = False):
# updated clear date of all the vouchers based on the bank transaction
vouchers = frappe.parse_json(vouchers)
vouchers = json.loads(vouchers)
transaction = frappe.get_doc("Bank Transaction", bank_transaction_name)
transaction.add_payment_entries(vouchers, is_new_voucher)
transaction.validate_duplicate_references()
@@ -1410,14 +1410,12 @@ def get_je_matching_query(
Sum(getattr(jea, amount_field)).as_("paid_amount"),
ConstantColumn("Journal Entry").as_("doctype"),
je.name,
# non-grouped columns are constant per grouped JE name (party_type/currency come from the
# single bank-account line) -> Max() keeps the GROUP BY valid on postgres with the same value
Max(je.cheque_no).as_("reference_no"),
Max(je.cheque_date).as_("reference_date"),
Max(je.pay_to_recd_from).as_("party"),
Max(jea.party_type).as_("party_type"),
Max(je.posting_date).as_("posting_date"),
Max(jea.account_currency).as_("currency"),
je.cheque_no.as_("reference_no"),
je.cheque_date.as_("reference_date"),
je.pay_to_recd_from.as_("party"),
jea.party_type,
je.posting_date,
jea.account_currency.as_("currency"),
)
.where(je.docstatus == 1)
.where(je.voucher_type != "Opening Entry")
@@ -1425,7 +1423,7 @@ def get_je_matching_query(
.where(jea.account == common_filters.bank_account)
.where(filter_by_date)
.groupby(je.name)
.orderby(Max(je.cheque_date) if cint(filter_by_reference_date) else Max(je.posting_date))
.orderby(je.cheque_date if cint(filter_by_reference_date) else je.posting_date)
)
if frappe.flags.auto_reconcile_vouchers is True:

View File

@@ -8,7 +8,6 @@ from frappe.utils import add_days, today
from erpnext.accounts.doctype.bank_reconciliation_tool.bank_reconciliation_tool import (
auto_reconcile_vouchers,
get_auto_reconcile_message,
get_bank_transactions,
)
from erpnext.accounts.doctype.payment_entry.test_payment_entry import create_payment_entry
@@ -18,10 +17,9 @@ from erpnext.tests.utils import ERPNextTestSuite
class TestBankReconciliationTool(ERPNextTestSuite, AccountsTestMixin):
def setUp(self):
self.company = "_Test Company"
self.customer = "_Test Customer"
self.bank = "HDFC - _TC"
self.debit_to = "Debtors - _TC"
self.create_company()
self.create_customer()
self.clear_old_entries()
bank_dt = qb.DocType("Bank")
qb.from_(bank_dt).delete().where(bank_dt.name == "HDFC").run()
self.create_bank_account()
@@ -98,40 +96,3 @@ class TestBankReconciliationTool(ERPNextTestSuite, AccountsTestMixin):
# assert API output post reconciliation
transactions = get_bank_transactions(self.bank_account, from_date, to_date)
self.assertEqual(len(transactions), 0)
def make_bank_transaction(self, date, deposit=100):
return (
frappe.get_doc(
{
"doctype": "Bank Transaction",
"date": date,
"deposit": deposit,
"bank_account": self.bank_account,
"currency": "INR",
}
)
.save()
.submit()
)
def test_get_bank_transactions_excludes_dates_after_to_date(self):
self.make_bank_transaction(date=today())
names = [t.name for t in get_bank_transactions(self.bank_account, to_date=add_days(today(), -1))]
self.assertEqual(names, [])
def test_auto_reconcile_message_for_no_matches(self):
message, indicator = get_auto_reconcile_message([], [])
self.assertEqual(indicator, "blue")
self.assertIn("No matches", message)
def test_auto_reconcile_message_counts_and_pluralizes(self):
# reconciled count is reported and the indicator turns green
message, indicator = get_auto_reconcile_message([], ["t1", "t2"])
self.assertEqual(indicator, "green")
self.assertIn("2 Transaction(s) Reconciled", message)
# partially-reconciled label is singular for one, plural for many
singular, _ = get_auto_reconcile_message(["p1"], [])
self.assertIn("1 Transaction Partially Reconciled", singular)
plural, _ = get_auto_reconcile_message(["p1", "p2"], [])
self.assertIn("2 Transactions Partially Reconciled", plural)

View File

@@ -221,12 +221,12 @@
"default": "0",
"fieldname": "import_mt940_fromat",
"fieldtype": "Check",
"label": "Import MT940 Format"
"label": "Import MT940 Fromat"
}
],
"hide_toolbar": 1,
"links": [],
"modified": "2026-06-19 14:18:00.000000",
"modified": "2026-05-31 00:41:11.251215",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Bank Statement Import",

View File

@@ -142,30 +142,6 @@ def preprocess_mt940_content(content: str) -> str:
return processed_content
MT940_CUSTOMER_REFERENCE_MAX_LEN = 16
def get_transaction_reference(txn_data: dict) -> str:
"""Extract the per-transaction reference from an MT940 :61: tag.
The mt940 library exposes ``transaction_reference`` from the :20: tag, which is the
statement-level reference and identical for every transaction in a statement. The
real per-transaction reference is ``customer_reference`` (with any overflow captured
into ``extra_details`` when a bank emits a single-line :61: longer than 16 chars).
"""
customer_reference = (txn_data.get("customer_reference") or "").strip()
if len(customer_reference) == MT940_CUSTOMER_REFERENCE_MAX_LEN:
customer_reference += (txn_data.get("extra_details") or "").strip()
if customer_reference and customer_reference.upper() != "NONREF":
return customer_reference
return (txn_data.get("bank_reference") or "").strip() or (
txn_data.get("transaction_reference") or ""
).strip()
@frappe.whitelist()
def convert_mt940_to_csv(data_import: str, mt940_file_path: str):
doc = frappe.get_doc("Bank Statement Import", data_import)
@@ -213,8 +189,8 @@ def convert_mt940_to_csv(data_import: str, mt940_file_path: str):
deposit = amount_value if amount_value > 0 else ""
withdrawal = abs(amount_value) if amount_value < 0 else ""
description = txn.data.get("transaction_details") or txn.data.get("extra_details") or ""
reference = get_transaction_reference(txn.data)
description = txn.data.get("extra_details") or ""
reference = txn.data.get("transaction_reference") or ""
currency = txn.data.get("currency", "")
writer.writerow([date_str, deposit, withdrawal, description, reference, doc.bank_account, currency])
@@ -314,7 +290,7 @@ def update_mapping_db(bank, template_options):
for d in bank.bank_transaction_mapping:
d.delete()
for d in frappe.parse_json(template_options)["column_to_field_map"].items():
for d in json.loads(template_options)["column_to_field_map"].items():
bank.append("bank_transaction_mapping", {"bank_transaction_field": d[1], "file_field": d[0]})
bank.save()

View File

@@ -1,10 +1,7 @@
# Copyright (c) 2020, Frappe Technologies and Contributors
# See license.txt
import mt940
from erpnext.accounts.doctype.bank_statement_import.bank_statement_import import (
get_transaction_reference,
is_mt940_format,
preprocess_mt940_content,
)
@@ -191,135 +188,6 @@ class TestBankStatementImport(ERPNextTestSuite):
self.assertIn(":20:STMTREF167619", result) # Reference should remain unchanged
self.assertIn("UPI/TEST USER/123456789/PaidViaTestApp", result)
def test_get_transaction_reference_uses_customer_reference(self):
"""Per-transaction reference must come from :61: customer_reference, not :20:."""
self.assertEqual(
get_transaction_reference(
{"customer_reference": "UPI-100000000001", "transaction_reference": "STMTREF12345"}
),
"UPI-100000000001",
)
def test_get_transaction_reference_rejoins_overflow(self):
"""When a bank emits a single-line :61: with >16-char reference, the regex
splits the tail into extra_details. We must rejoin them."""
self.assertEqual(
get_transaction_reference(
{
"customer_reference": "NEFTINW-12345678",
"extra_details": "90",
"transaction_reference": "STMTREF12345",
}
),
"NEFTINW-1234567890",
)
def test_get_transaction_reference_falls_back_to_bank_reference_on_nonref(self):
"""NONREF is the MT940 'no customer reference' sentinel; prefer bank_reference."""
self.assertEqual(
get_transaction_reference(
{
"customer_reference": "NONREF",
"bank_reference": "1234567890123456",
"transaction_reference": "STMTREF12345",
}
),
"1234567890123456",
)
def test_get_transaction_reference_falls_back_to_bank_reference_on_nonref_with_extra_details(self):
"""NONREF sentinel must trigger the bank_reference fallback even when
extra_details is populated. Without the 16-char gate, the old naive concat
would produce a junk reference like 'NONREFsome info' and bypass the check."""
self.assertEqual(
get_transaction_reference(
{
"customer_reference": "NONREF",
"extra_details": "some info",
"bank_reference": "1234567890123456",
"transaction_reference": "STMTREF12345",
}
),
"1234567890123456",
)
def test_get_transaction_reference_does_not_append_extra_details_below_16_chars(self):
"""When customer_reference is below the 16-char cap, extra_details is a
genuine supplementary-info field from :61: — not overflow — and must not
be appended to the reference."""
self.assertEqual(
get_transaction_reference(
{
"customer_reference": "TBMS-123456789",
"extra_details": "note field",
"transaction_reference": "STMTREF12345",
}
),
"TBMS-123456789",
)
def test_get_transaction_reference_keeps_noref_literal(self):
"""Bare 'NOREF' (without bank_reference) stays as-is; still better than the
statement-level reference which is identical across all transactions."""
self.assertEqual(
get_transaction_reference(
{
"customer_reference": "NOREF",
"bank_reference": None,
"transaction_reference": "STMTREF12345",
}
),
"NOREF",
)
def test_mt940_parse_per_transaction_reference_mapping(self):
"""End-to-end: every transaction in a statement must get its own distinct
reference from :61: customer_reference, never the statement-level :20: reference."""
mt940_content = """{1:F0112345678901X0000000000}{2:I94012345678901XN}{4:
:20:STMTREF12345
:25:1234567890
:28C:12345/1
:60F:C250716INR88123,38
:61:2509280928D5000,00NMSCUPI-100000000001
:86:UPI/TEST PAYEE ONE/111111111111/TestApp
:61:2509190919D2606,00NMSCUPI-100000000002
:86:UPI/TEST PAYEE TWO/222222222222/TestApp
:61:2509190919D900,00NMSCUPI-100000000003
:86:UPI/TEST PAYEE THREE/333333333333/TestApp
:61:2508140814D5000,00NMSCUPI-100000000004
:86:UPI/TEST PAYEE FOUR/444444444444/TestApp
:61:2508060806D2000,00NMSCUPI-100000000005
:86:UPI/TEST PAYEE FIVE/555555555555/TestApp
:61:2508030803D1066,00NMSC123456789012
:86:PCD/1234/TEST MERCHANT/01234567890123/12:00
:61:2507310731D305,62NMSCTBMS-123456789
:86:Chrg: Debit Card Annual Fee 1234 for 2025
:61:2507240724C1,00NMSCNEFTINW-1234567890
:86:NEFT TEST123456789 TEST SERVICES
:61:2507170717C100000,00NMSCNOREF
:86:BY CLG INST 123456/01-01-25/TESTBANK/TESTCITY
:62F:C250930INR100000,00
-}"""
transactions = list(mt940.parse(preprocess_mt940_content(mt940_content)))
references = [get_transaction_reference(t.data) for t in transactions]
self.assertEqual(
references,
[
"UPI-100000000001",
"UPI-100000000002",
"UPI-100000000003",
"UPI-100000000004",
"UPI-100000000005",
"123456789012",
"TBMS-123456789",
"NEFTINW-1234567890",
"NOREF",
],
)
# No transaction should carry the statement-level reference from :20:
self.assertNotIn("STMTREF12345", references)
def test_preprocess_mt940_content_whitespace_variants(self):
"""Test handling of whitespace and different line endings"""
# Test with trailing spaces

View File

@@ -54,6 +54,7 @@
"fieldtype": "Currency",
"in_list_view": 1,
"label": "Closing Balance",
"non_negative": 1,
"options": "currency"
},
{
@@ -190,7 +191,7 @@
"grid_page_length": 50,
"index_web_pages_for_search": 1,
"links": [],
"modified": "2026-07-09 17:55:25.615942",
"modified": "2026-05-08 17:55:25.615942",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Bank Statement Import Log",

View File

@@ -557,7 +557,7 @@ class BankStatementImportLog(Document):
docname=self.name,
)
if self.closing_balance is not None and self.end_date:
if self.closing_balance and self.closing_balance > 0 and self.end_date:
set_closing_balance_as_per_statement(
self.bank_account, frappe.utils.getdate(self.end_date), self.closing_balance
)
@@ -829,9 +829,7 @@ def compute_final_transactions(transaction_rows: list, date_format: str, amount_
if amount_format == 'Amount column has "CR"/"DR" values':
amount = transaction_row.get("amount")
# If the amount column has CR/DR in it - we should remove any signs (negative or positive) from the amount
float_amount = abs(get_float_amount(amount) or 0)
float_amount = get_float_amount(amount)
if "cr" in amount.lower():
return 0, float_amount
else:
@@ -934,18 +932,14 @@ def extract_pdf_tables(content: bytes, password: str | None = None) -> list[dict
from pypdf import PdfReader
reader = PdfReader(io.BytesIO(content))
if reader.is_encrypted:
# Try opening the PDF with a password - if no password is provided, try with a blank password
if not password:
password = ""
if not reader.decrypt(password):
frappe.throw(
_(
"This PDF is password protected. Please set the correct statement password on the"
" Bank Account and try again."
),
title=_("Password Required"),
)
if reader.is_encrypted and (not password or not reader.decrypt(password)):
frappe.throw(
_(
"This PDF is password protected. Please set the correct statement password on the"
" Bank Account and try again."
),
title=_("Password Required"),
)
text_settings = {"vertical_strategy": "text", "horizontal_strategy": "text"}
tables = []
@@ -1189,7 +1183,8 @@ def update_pdf_tables(statement_import_id: str, tables: list | str):
if doc.status == "Completed":
frappe.throw(_("This statement has already been imported."), title=_("Already Imported"))
tables = frappe.parse_json(tables)
if isinstance(tables, str):
tables = json.loads(tables)
doc.apply_pdf_tables(tables)
@@ -1209,7 +1204,8 @@ def reextract_pdf_table(statement_import_id: str, page: int, table_index: int, b
if doc.status == "Completed":
frappe.throw(_("This statement has already been imported."), title=_("Already Imported"))
bbox = frappe.parse_json(bbox)
if isinstance(bbox, str):
bbox = json.loads(bbox)
page = int(page)
table_index = int(table_index)
@@ -1294,7 +1290,8 @@ def update_column_mapping(statement_import_id: str, column_mapping: list | str):
if doc.status == "Completed":
frappe.throw(_("This statement has already been imported."), title=_("Already Imported"))
column_mapping = frappe.parse_json(column_mapping)
if isinstance(column_mapping, str):
column_mapping = json.loads(column_mapping)
doc.apply_column_mapping(column_mapping)
doc.save()

View File

@@ -26,9 +26,9 @@ from erpnext.tests.utils import ERPNextTestSuite
class TestBankStatementImportLog(ERPNextTestSuite, AccountsTestMixin):
def setUp(self):
self.company = "_Test Company"
self.customer = "_Test Customer"
self.bank = "HDFC - _TC"
self.create_company()
self.create_customer()
self.clear_old_entries()
bank_dt = qb.DocType("Bank")
qb.from_(bank_dt).delete().where(bank_dt.name == "HDFC").run()
self.create_bank_account()

View File

@@ -5,8 +5,6 @@ import frappe
from frappe import _
from frappe.model.docstatus import DocStatus
from frappe.model.document import Document
from frappe.query_builder import Tuple
from frappe.query_builder.functions import Abs, Max, Sum
from frappe.utils import flt, getdate
@@ -376,7 +374,6 @@ def unreconcile_transaction(transaction_name: str | int):
Else, cancel the individual entries
"""
transaction = frappe.get_doc("Bank Transaction", transaction_name)
transaction.check_permission("write")
vouchers_to_cancel = []
@@ -397,14 +394,13 @@ def unreconcile_transaction(transaction_name: str | int):
frappe.get_doc(voucher["doctype"], voucher["name"]).cancel()
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def unreconcile_transaction_entry(bank_transaction_id: str | int, voucher_type: str, voucher_id: str | int):
"""
Removes a single payment entry from a bank transaction - for example only undoing one voucher instead of undoing the entire transaction
"""
bank_transaction = frappe.get_doc("Bank Transaction", bank_transaction_id)
bank_transaction.check_permission("write")
# Find the voucher in the bank transaction and depending on the action, either remove it or cancel the voucher
for entry in bank_transaction.payment_entries:
@@ -440,7 +436,7 @@ def get_clearance_details(transaction, payment_entry, bt_allocations, gl_entries
if bt_bank_account != gl_bank_account:
frappe.throw(
_("Bank Account {0} in Bank Transaction {1} is not matching with Bank Account {2}").format(
_("Bank Account {} in Bank Transaction {} is not matching with Bank Account {}").format(
bt_bank_account, payment_entry.payment_entry, gl_bank_account
)
)
@@ -449,7 +445,7 @@ def get_clearance_details(transaction, payment_entry, bt_allocations, gl_entries
if gl_bank_account not in gl_entries:
frappe.throw(
_("{0} {1} is not affecting bank account {2}").format(
_("{} {} is not affecting bank account {}").format(
payment_entry.payment_document, payment_entry.payment_entry, gl_bank_account
)
)
@@ -457,7 +453,7 @@ def get_clearance_details(transaction, payment_entry, bt_allocations, gl_entries
allocable_amount = gl_entries.pop(gl_bank_account) or 0
if allocable_amount <= 0.0:
frappe.throw(
_("Invalid amount in accounting entries of {0} {1} for Account {2}: {3}").format(
_("Invalid amount in accounting entries of {} {} for Account {}: {}").format(
payment_entry.payment_document, payment_entry.payment_entry, gl_bank_account, allocable_amount
)
)
@@ -480,28 +476,30 @@ def get_clearance_details(transaction, payment_entry, bt_allocations, gl_entries
def get_related_bank_gl_entries(docs):
# nosemgrep: frappe-semgrep-rules.rules.frappe-using-db-sql
if not docs:
return {}
gle = frappe.qb.DocType("GL Entry")
ac = frappe.qb.DocType("Account")
result = (
frappe.qb.from_(gle)
.left_join(ac)
.on(ac.name == gle.account)
.select(
gle.voucher_type.as_("doctype"),
gle.voucher_no.as_("docname"),
gle.account.as_("gl_account"),
Sum(Abs(gle.credit_in_account_currency - gle.debit_in_account_currency)).as_("amount"),
)
.where(
(ac.account_type == "Bank")
& Tuple(gle.voucher_type, gle.voucher_no).isin([Tuple(vt, vn) for vt, vn in docs])
& (gle.is_cancelled == 0)
)
.groupby(gle.voucher_type, gle.voucher_no, gle.account)
.run(as_dict=True)
result = frappe.db.sql(
"""
SELECT
gle.voucher_type AS doctype,
gle.voucher_no AS docname,
gle.account AS gl_account,
SUM(ABS(gle.credit_in_account_currency - gle.debit_in_account_currency)) AS amount
FROM
`tabGL Entry` gle
LEFT JOIN
`tabAccount` ac ON ac.name = gle.account
WHERE
ac.account_type = 'Bank'
AND (gle.voucher_type, gle.voucher_no) IN %(docs)s
AND gle.is_cancelled = 0
GROUP BY
gle.voucher_type, gle.voucher_no, gle.account
""",
{"docs": docs},
as_dict=True,
)
entries = {}
@@ -523,32 +521,31 @@ def get_total_allocated_amount(docs):
if not docs:
return {}
# The original window query (ROW_NUMBER/FIRST_VALUE + rownum = 1) just collapses to one
# row per (account, payment_document, payment_entry) with the partition's allocation total
# and most recent transaction date — i.e. a plain GROUP BY with SUM and MAX.
btp = frappe.qb.DocType("Bank Transaction Payments")
bt = frappe.qb.DocType("Bank Transaction")
ba = frappe.qb.DocType("Bank Account")
result = (
frappe.qb.from_(btp)
.left_join(bt)
.on(bt.name == btp.parent)
.left_join(ba)
.on(ba.name == bt.bank_account)
.select(
Sum(btp.allocated_amount).as_("total"),
Max(bt.date).as_("latest_date"),
ba.account.as_("gl_account"),
btp.payment_document,
btp.payment_entry,
)
.where(
Tuple(btp.payment_document, btp.payment_entry).isin([Tuple(pd, pe) for pd, pe in docs])
& (bt.docstatus == 1)
)
.groupby(ba.account, btp.payment_document, btp.payment_entry)
.run(as_dict=True)
# nosemgrep: frappe-semgrep-rules.rules.frappe-using-db-sql
result = frappe.db.sql(
"""
SELECT total, latest_date, gl_account, payment_document, payment_entry FROM (
SELECT
ROW_NUMBER() OVER w AS rownum,
SUM(btp.allocated_amount) OVER(PARTITION BY ba.account, btp.payment_document, btp.payment_entry) AS total,
FIRST_VALUE(bt.date) OVER w AS latest_date,
ba.account AS gl_account,
btp.payment_document,
btp.payment_entry
FROM
`tabBank Transaction Payments` btp
LEFT JOIN `tabBank Transaction` bt ON bt.name=btp.parent
LEFT JOIN `tabBank Account` ba ON ba.name=bt.bank_account
WHERE
(btp.payment_document, btp.payment_entry) IN %(docs)s
AND bt.docstatus = 1
WINDOW w AS (PARTITION BY ba.account, btp.payment_document, btp.payment_entry ORDER BY bt.date DESC)
) temp
WHERE
rownum = 1
""",
dict(docs=docs),
as_dict=True,
)
payment_allocation_details = {}

View File

@@ -34,20 +34,19 @@ def upload_bank_statement():
return {"columns": columns, "data": data}
@frappe.whitelist(methods=["POST"])
def create_bank_entries(columns: str, data: str | list, bank_account: str):
@frappe.whitelist()
def create_bank_entries(columns: str, data: str, bank_account: str):
header_map = get_header_mapping(columns, bank_account)
success = 0
errors = 0
for d in frappe.parse_json(data):
for d in json.loads(data):
if all(item is None for item in d) is True:
continue
fields = {}
for key, value in header_map.items():
fields.update({key: d[int(value) - 1]})
frappe.db.savepoint("bank_entry")
try:
bank_transaction = frappe.get_doc({"doctype": "Bank Transaction"})
bank_transaction.update(fields)
@@ -57,8 +56,7 @@ def create_bank_entries(columns: str, data: str | list, bank_account: str):
bank_transaction.submit()
success += 1
except Exception:
frappe.db.rollback(save_point="bank_entry")
frappe.log_error(title="Bank entry creation failed")
bank_transaction.log_error("Bank entry creation failed")
errors += 1
return {"success": success, "errors": errors}
@@ -68,7 +66,7 @@ def get_header_mapping(columns, bank_account):
mapping = get_bank_mapping(bank_account)
header_map = {}
for column in frappe.parse_json(columns):
for column in json.loads(columns):
if column["content"] in mapping:
header_map.update({mapping[column["content"]]: column["colIndex"]})

View File

@@ -104,36 +104,6 @@ class TestBankTransaction(ERPNextTestSuite):
self.assertEqual(bank_transaction.unallocated_amount, 1700)
self.assertEqual(bank_transaction.payment_entries, [])
# Amending a reconciled payment entry must not carry over its clearance date
def test_clearance_date_cleared_on_amend(self):
bank_transaction = frappe.get_doc(
"Bank Transaction",
dict(description="1512567 BG/000003025 OPSKATTUZWXXX AT776000000098709849 Herr G"),
)
payment = frappe.get_doc("Payment Entry", dict(party="Mr G", paid_amount=1700))
vouchers = json.dumps(
[
{
"payment_doctype": "Payment Entry",
"payment_name": payment.name,
"amount": bank_transaction.unallocated_amount,
}
]
)
reconcile_vouchers(bank_transaction.name, vouchers)
self.assertTrue(frappe.db.get_value("Payment Entry", payment.name, "clearance_date"))
payment.reload()
payment.cancel()
amended = frappe.copy_doc(payment)
amended.amended_from = payment.name
amended.docstatus = 0
amended.insert()
self.assertFalse(amended.clearance_date)
# Check if ERPNext can correctly filter a linked payments based on the debit/credit amount
def test_debit_credit_output(self):
bank_transaction = frappe.get_doc(

View File

@@ -9,48 +9,6 @@ from frappe.model.document import Document
from erpnext.accounts.doctype.bank_transaction.bank_transaction import BankTransaction
PLAIN_NUMBER_PATTERN = re.compile(r"^-?\d+(\.\d+)?$")
# Tokens accepted by safe-expr-eval on the frontend (must stay in sync).
ALLOWED_FORMULA_TOKEN = re.compile(r"\s+|transaction_amount|\d+(?:\.\d+)?|[+\-*/%^()]")
PYTHON_ONLY_OPERATORS = ("**", "//")
def _is_expr_eval_formula(formula: str) -> bool:
position = 0
while position < len(formula):
match = ALLOWED_FORMULA_TOKEN.match(formula, position)
if not match:
return False
position = match.end()
return formula.count("(") == formula.count(")")
def validate_amount_formula(formula: str) -> None:
if not formula:
return
stripped = formula.strip()
if PLAIN_NUMBER_PATTERN.match(stripped):
return
if any(operator in stripped for operator in PYTHON_ONLY_OPERATORS):
frappe.throw(_("Invalid debit/credit formula: {0}").format(formula))
if not _is_expr_eval_formula(stripped):
frappe.throw(_("Invalid debit/credit formula: {0}").format(formula))
# expr-eval uses ^ for exponentiation; translate for a smoke-test evaluation only.
python_formula = stripped.replace("^", "**")
try:
result = frappe.safe_eval(python_formula, eval_globals=None, eval_locals={"transaction_amount": 1})
except Exception:
frappe.throw(_("Invalid debit/credit formula: {0}").format(formula))
if not isinstance(result, (int | float)):
frappe.throw(_("Invalid debit/credit formula: {0}").format(formula))
class BankTransactionRule(Document):
# begin: auto-generated types
@@ -108,7 +66,7 @@ class BankTransactionRule(Document):
frappe.throw(_("Party type is required to create a payment entry."))
if not self.party:
frappe.throw(_("Party is required to create a payment entry."))
frappe.throw(_("Party is required create a payment entry."))
if not self.account:
frappe.throw(_("Party account is required to create a payment entry."))
@@ -128,11 +86,6 @@ class BankTransactionRule(Document):
frappe.throw(
_("The last account row must not have any debit or credit amounts set.")
)
else:
if account.debit:
validate_amount_formula(account.debit)
if account.credit:
validate_amount_formula(account.credit)
# Validate regex
for rule in self.description_rules:

View File

@@ -11,11 +11,9 @@ from erpnext.tests.utils import ERPNextTestSuite
class TestBankTransactionRule(ERPNextTestSuite, AccountsTestMixin):
def setUp(self):
self.company = "_Test Company"
self.customer = "_Test Customer"
self.bank = "HDFC - _TC"
self.debit_to = "Debtors - _TC"
self.cash = "Cash - _TC"
self.create_company()
self.create_customer()
self.clear_old_entries()
bank_dt = qb.DocType("Bank")
qb.from_(bank_dt).delete().where(bank_dt.name == "HDFC").run()
self.create_bank_account()
@@ -231,45 +229,3 @@ class TestBankTransactionRule(ERPNextTestSuite, AccountsTestMixin):
doc = self._rule("bad_rx", [{"check": "Regex", "value": "["}])
with self.assertRaises(ValidationError):
doc.insert()
def _multiple_accounts_rule(self, prefix: str, accounts, **fields):
return self._rule(
prefix,
[{"check": "Contains", "value": "x"}],
classify_as="Bank Entry",
bank_entry_type="Multiple Accounts",
accounts=accounts,
**fields,
)
def test_validate_bank_entry_multiple_valid_amount_formulas(self):
doc = self._multiple_accounts_rule(
"be_formula",
accounts=[
{"account": self.bank, "debit": "200", "credit": ""},
{"account": self.cash, "debit": "", "credit": "transaction_amount * 0.25"},
{"account": self.cash, "debit": "", "credit": ""},
],
)
doc.insert()
self.assertTrue(doc.name)
def test_validate_bank_entry_multiple_invalid_amount_formulas(self):
malicious_formulas = [
"__import__('os')",
"eval('1+1')",
"open('/etc/passwd')",
"transaction_amount ** 2",
"transaction_amount // 2",
]
for formula in malicious_formulas:
with self.subTest(formula=formula):
doc = self._multiple_accounts_rule(
"be_bad_formula",
accounts=[
{"account": self.bank, "debit": formula, "credit": ""},
{"account": self.cash, "debit": "", "credit": ""},
],
)
with self.assertRaises(ValidationError):
doc.insert()

View File

@@ -184,7 +184,7 @@ class BisectAccountingStatements(Document):
self.get_report_summary()
self.update_node()
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def bisect_left(self):
if self.current_node is not None:
cur_node = frappe.get_doc("Bisect Nodes", self.current_node)
@@ -198,7 +198,7 @@ class BisectAccountingStatements(Document):
else:
frappe.msgprint(_("No more children on Left"))
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def bisect_right(self):
if self.current_node is not None:
cur_node = frappe.get_doc("Bisect Nodes", self.current_node)
@@ -212,7 +212,7 @@ class BisectAccountingStatements(Document):
else:
frappe.msgprint(_("No more children on Right"))
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def move_up(self):
if self.current_node is not None:
cur_node = frappe.get_doc("Bisect Nodes", self.current_node)

View File

@@ -1,47 +1,11 @@
# Copyright (c) 2026, Frappe Technologies Pvt. Ltd. and Contributors
# Copyright (c) 2023, Frappe Technologies Pvt. Ltd. and Contributors
# See license.txt
import datetime
# import frappe
import frappe
from frappe.utils import getdate
from erpnext.tests.utils import ERPNextTestSuite
class TestBisectAccountingStatements(ERPNextTestSuite):
"""The tool bisects a date range into a tree of Bisect Nodes down to single days.
These cover the date validation and that the bisection cleanly partitions the range."""
def setUp(self):
frappe.set_user("Administrator")
frappe.db.delete("Bisect Nodes")
def _leaf_days(self):
leaves = frappe.get_all(
"Bisect Nodes",
filters={"left_child": ["is", "not set"]},
fields=["period_from_date", "period_to_date"],
)
# every leaf spans a single day
for leaf in leaves:
self.assertEqual(getdate(leaf.period_from_date), getdate(leaf.period_to_date))
return sorted(getdate(leaf.period_from_date) for leaf in leaves)
def test_validate_dates_rejects_reversed_range(self):
doc = frappe.new_doc("Bisect Accounting Statements")
doc.from_date = "2026-01-08"
doc.to_date = "2026-01-01"
self.assertRaises(frappe.ValidationError, doc.validate)
def test_bfs_partitions_range_into_single_days(self):
doc = frappe.new_doc("Bisect Accounting Statements")
doc.bfs(datetime.datetime(2026, 1, 1), datetime.datetime(2026, 1, 8))
# the 8-day span Jan 1..Jan 8 becomes exactly 8 contiguous single-day leaves
self.assertEqual(self._leaf_days(), [getdate(f"2026-01-0{n}") for n in range(1, 9)])
def test_dfs_produces_the_same_partition_as_bfs(self):
doc = frappe.new_doc("Bisect Accounting Statements")
doc.dfs(datetime.datetime(2026, 1, 1), datetime.datetime(2026, 1, 8))
self.assertEqual(self._leaf_days(), [getdate(f"2026-01-0{n}") for n in range(1, 9)])
pass

View File

@@ -17,7 +17,6 @@ frappe.ui.form.on("Budget", {
filters: {
is_group: 0,
company: frm.doc.company,
root_type: ["in", ["Income", "Expense"]],
},
};
});
@@ -136,9 +135,6 @@ function set_total_budget_amount(frm) {
function toggle_distribution_fields(frm) {
const grid = frm.fields_dict.budget_distribution.grid;
frm.set_df_property("budget_distribution", "cannot_add_rows", true);
frm.set_df_property("budget_distribution", "cannot_delete_rows", true);
["amount", "percent"].forEach((field) => {
grid.update_docfield_property(field, "read_only", frm.doc.distribute_equally);
});

View File

@@ -5,11 +5,9 @@
import frappe
from frappe import _
from frappe.model.document import Document
from frappe.query_builder import Criterion
from frappe.query_builder.functions import Coalesce, Sum
from frappe.query_builder.functions import Sum
from frappe.utils import add_months, flt, fmt_money, get_last_day, getdate
from frappe.utils.data import get_first_day
from pypika.terms import ExistsCriterion
from erpnext.accounts.doctype.accounting_dimension.accounting_dimension import (
get_accounting_dimensions,
@@ -117,26 +115,23 @@ class Budget(Document):
if not account:
return
budget = frappe.qb.DocType("Budget")
fy_from = frappe.qb.DocType("Fiscal Year").as_("fy_from")
fy_to = frappe.qb.DocType("Fiscal Year").as_("fy_to")
existing_budget = (
frappe.qb.from_(budget)
.inner_join(fy_from)
.on(fy_from.name == budget.from_fiscal_year)
.inner_join(fy_to)
.on(fy_to.name == budget.to_fiscal_year)
.select(budget.name, budget.account)
.where(
(budget.docstatus < 2)
& (budget.company == self.company)
& (budget[budget_against_field] == budget_against)
& (budget.account == account)
& (budget.name != self.name)
& (fy_from.year_start_date <= self.budget_end_date)
& (fy_to.year_end_date >= self.budget_start_date)
)
.run(as_dict=True)
existing_budget = frappe.db.sql(
f"""
SELECT name, account
FROM `tabBudget`
WHERE
docstatus < 2
AND company = %s
AND {budget_against_field} = %s
AND account = %s
AND name != %s
AND (
(SELECT year_start_date FROM `tabFiscal Year` WHERE name = from_fiscal_year) <= %s
AND (SELECT year_end_date FROM `tabFiscal Year` WHERE name = to_fiscal_year) >= %s
)
""",
(self.company, budget_against, account, self.name, self.budget_end_date, self.budget_start_date),
as_dict=True,
)
if existing_budget:
@@ -162,9 +157,9 @@ class Budget(Document):
frappe.throw(_("Account {0} does not belong to company {1}").format(self.account, self.company))
elif account_details.report_type != "Profit and Loss":
frappe.throw(
_(
"Budget cannot be assigned against {0}, as its Root Type is not of Income or Expense"
).format(self.account)
_("Budget cannot be assigned against {0}, as it's not an Income or Expense account").format(
self.account
)
)
def set_null_value(self):
@@ -358,8 +353,8 @@ class Budget(Document):
if self.should_regenerate_budget_distribution():
return
total_amount = sum(flt(d.amount) for d in self.budget_distribution)
total_percent = sum(flt(d.percent) for d in self.budget_distribution)
total_amount = sum(d.amount for d in self.budget_distribution)
total_percent = sum(d.percent for d in self.budget_distribution)
if flt(abs(total_amount - self.budget_amount), 2) > 0.10:
frappe.throw(
@@ -386,24 +381,17 @@ def validate_expense_against_budget(params, expense_amount=0):
posting_fiscal_year = get_fiscal_year(posting_date, company=params.get("company"))[0]
year_start_date, year_end_date = get_fiscal_year_date_range(posting_fiscal_year, posting_fiscal_year)
budget = frappe.qb.DocType("Budget")
fy_from = frappe.qb.DocType("Fiscal Year").as_("fy_from")
fy_to = frappe.qb.DocType("Fiscal Year").as_("fy_to")
budget_exists = (
frappe.qb.from_(budget)
.inner_join(fy_from)
.on(fy_from.name == budget.from_fiscal_year)
.inner_join(fy_to)
.on(fy_to.name == budget.to_fiscal_year)
.select(budget.name)
.where(
(budget.company == params.company)
& (budget.docstatus == 1)
& (fy_from.year_start_date <= year_end_date)
& (fy_to.year_end_date >= year_start_date)
)
.limit(1)
.run()
budget_exists = frappe.db.sql(
"""
select name
from `tabBudget`
where company = %s
and docstatus = 1
and (SELECT year_start_date FROM `tabFiscal Year` WHERE name = from_fiscal_year) <= %s
and (SELECT year_end_date FROM `tabFiscal Year` WHERE name = to_fiscal_year) >= %s
limit 1
""",
(params.company, year_end_date, year_start_date),
)
if not budget_exists:
@@ -446,52 +434,50 @@ def validate_expense_against_budget(params, expense_amount=0):
and (frappe.get_cached_value("Account", params.account, "root_type") == "Expense")
):
doctype = dimension.get("document_type")
params.is_tree = bool(frappe.get_cached_value("DocType", doctype, "is_tree"))
if frappe.get_cached_value("DocType", doctype, "is_tree"):
lft, rgt = frappe.get_cached_value(doctype, params.get(budget_against), ["lft", "rgt"])
condition = f"""and exists(select name from `tab{doctype}`
where lft<={lft} and rgt>={rgt} and name=b.{budget_against})""" # nosec
params.is_tree = True
else:
condition = f"and b.{budget_against}={frappe.db.escape(params.get(budget_against))}"
params.is_tree = False
params.budget_against_field = budget_against
params.budget_against_doctype = doctype
b = frappe.qb.DocType("Budget")
query = (
frappe.qb.from_(b)
.select(
budget_records = frappe.db.sql(
f"""
SELECT
b.name,
getattr(b, budget_against).as_("budget_against"),
b.{budget_against} AS budget_against,
b.budget_amount,
b.from_fiscal_year,
b.to_fiscal_year,
b.budget_start_date,
b.budget_end_date,
Coalesce(b.applicable_on_material_request, 0).as_("for_material_request"),
Coalesce(b.applicable_on_purchase_order, 0).as_("for_purchase_order"),
Coalesce(b.applicable_on_booking_actual_expenses, 0).as_("for_actual_expenses"),
IFNULL(b.applicable_on_material_request, 0) AS for_material_request,
IFNULL(b.applicable_on_purchase_order, 0) AS for_purchase_order,
IFNULL(b.applicable_on_booking_actual_expenses, 0) AS for_actual_expenses,
b.action_if_annual_budget_exceeded,
b.action_if_accumulated_monthly_budget_exceeded,
b.action_if_annual_budget_exceeded_on_mr,
b.action_if_accumulated_monthly_budget_exceeded_on_mr,
b.action_if_annual_budget_exceeded_on_po,
b.action_if_accumulated_monthly_budget_exceeded_on_po,
)
.where(b.company == params.company)
.where(b.docstatus == 1)
.where(b.budget_start_date <= params.posting_date)
.where(b.budget_end_date >= params.posting_date)
.where(b.account == params.account)
)
if params.is_tree:
lft, rgt = frappe.get_cached_value(doctype, params.get(budget_against), ["lft", "rgt"])
dim = frappe.qb.DocType(doctype)
query = query.where(
ExistsCriterion(
frappe.qb.from_(dim)
.select(dim.name)
.where((dim.lft <= lft) & (dim.rgt >= rgt) & (dim.name == getattr(b, budget_against)))
)
)
else:
query = query.where(getattr(b, budget_against) == params.get(budget_against))
budget_records = query.run(as_dict=True)
b.action_if_accumulated_monthly_budget_exceeded_on_po
FROM
`tabBudget` b
WHERE
b.company = %s
AND b.docstatus = 1
AND %s BETWEEN b.budget_start_date AND b.budget_end_date
AND b.account = %s
{condition}
""",
(params.company, params.posting_date, params.account),
as_dict=True,
) # nosec
if budget_records:
validate_budget_records(params, budget_records, expense_amount)
@@ -688,27 +674,15 @@ def get_actions(params, budget):
def get_requested_amount(params):
item_code = params.get("item_code")
condition = get_other_condition(params, "Material Request")
child = frappe.qb.DocType("Material Request Item")
parent = frappe.qb.DocType("Material Request")
data = (
frappe.qb.from_(child)
.join(parent)
.on(parent.name == child.parent)
.select(
# rate inside the aggregate: Sum(qty * rate) is the correct requested amount and is PG-valid
Coalesce(Sum((child.stock_qty - child.ordered_qty) * child.rate), 0).as_("amount")
)
.where(
(child.item_code == item_code)
& (parent.docstatus == 1)
& (child.stock_qty > child.ordered_qty)
& Criterion.all(get_other_condition(params, child, parent, "Material Request"))
& (parent.material_request_type == "Purchase")
& (parent.status != "Stopped")
)
.run(as_list=1)
data = frappe.db.sql(
""" select ifnull((sum(child.stock_qty - child.ordered_qty) * rate), 0) as amount
from `tabMaterial Request Item` child, `tabMaterial Request` parent where parent.name = child.parent and
child.item_code = %s and parent.docstatus = 1 and child.stock_qty > child.ordered_qty and {} and
parent.material_request_type = 'Purchase' and parent.status != 'Stopped'""".format(condition),
item_code,
as_list=1,
)
return data[0][0] if data else 0
@@ -716,43 +690,37 @@ def get_requested_amount(params):
def get_ordered_amount(params):
item_code = params.get("item_code")
condition = get_other_condition(params, "Purchase Order")
child = frappe.qb.DocType("Purchase Order Item")
parent = frappe.qb.DocType("Purchase Order")
data = (
frappe.qb.from_(child)
.join(parent)
.on(parent.name == child.parent)
.select(Coalesce(Sum(child.amount - child.billed_amt), 0).as_("amount"))
.where(
(child.item_code == item_code)
& (parent.docstatus == 1)
& (child.amount > child.billed_amt)
& (parent.status != "Closed")
& Criterion.all(get_other_condition(params, child, parent, "Purchase Order"))
)
.run(as_list=1)
data = frappe.db.sql(
f""" select ifnull(sum(child.amount - child.billed_amt), 0) as amount
from `tabPurchase Order Item` child, `tabPurchase Order` parent where
parent.name = child.parent and child.item_code = %s and parent.docstatus = 1 and child.amount > child.billed_amt
and parent.status != 'Closed' and {condition}""",
item_code,
as_list=1,
)
return data[0][0] if data else 0
def get_other_condition(params, child, parent, for_doc):
conditions = [child.expense_account == params.expense_account]
def get_other_condition(params, for_doc):
condition = f"expense_account = {frappe.db.escape(params.expense_account)}"
budget_against_field = params.get("budget_against_field")
if budget_against_field and params.get(budget_against_field):
conditions.append(child[budget_against_field] == params.get(budget_against_field))
condition += (
f" and child.{budget_against_field} = {frappe.db.escape(params.get(budget_against_field))}"
)
date_field = "schedule_date" if for_doc == "Material Request" else "transaction_date"
start_date = frappe.get_cached_value("Fiscal Year", params.from_fiscal_year, "year_start_date")
end_date = frappe.get_cached_value("Fiscal Year", params.to_fiscal_year, "year_end_date")
conditions.append(parent[date_field][str(start_date) : str(end_date)])
condition += f" and parent.{date_field} between {frappe.db.escape(str(start_date))} and {frappe.db.escape(str(end_date))}"
return conditions
return condition
def get_actual_expense(params):
@@ -760,19 +728,11 @@ def get_actual_expense(params):
params.budget_against_doctype = frappe.unscrub(params.budget_against_field)
budget_against_field = params.get("budget_against_field")
condition1 = " and gle.posting_date <= %(month_end_date)s" if params.get("month_end_date") else ""
gle = frappe.qb.DocType("GL Entry")
conditions = [
gle.is_cancelled == 0,
gle.account == params.get("account"),
gle.posting_date[str(params.budget_start_date) : str(params.budget_end_date)],
gle.company == params.get("company"),
gle.docstatus == 1,
]
if params.get("month_end_date"):
conditions.append(gle.posting_date <= params.get("month_end_date"))
date_condition = (
f"and gle.posting_date between '{params.budget_start_date}' and '{params.budget_end_date}'"
)
if params.is_tree:
lft_rgt = frappe.db.get_value(
@@ -780,27 +740,35 @@ def get_actual_expense(params):
)
params.update(lft_rgt)
tree = frappe.qb.DocType(params.budget_against_doctype)
conditions.append(
ExistsCriterion(
frappe.qb.from_(tree)
.select(tree.name)
.where(
(tree.lft >= params.get("lft"))
& (tree.rgt <= params.get("rgt"))
& (tree.name == gle[budget_against_field])
)
condition2 = f"""
and exists(
select name from `tab{params.budget_against_doctype}`
where lft >= %(lft)s and rgt <= %(rgt)s
and name = gle.{budget_against_field}
)
)
"""
else:
conditions.append(gle[budget_against_field] == params.get(budget_against_field))
condition2 = f"""
and gle.{budget_against_field} = %({budget_against_field})s
"""
amount = flt(
frappe.qb.from_(gle)
.select(Sum(gle.debit) - Sum(gle.credit))
.where(Criterion.all(conditions))
.run()[0][0]
)
frappe.db.sql(
f"""
select sum(gle.debit) - sum(gle.credit)
from `tabGL Entry` gle
where
is_cancelled = 0
and gle.account = %(account)s
{condition1}
{date_condition}
and gle.company = %(company)s
and gle.docstatus = 1
{condition2}
""",
params,
)[0][0]
) # nosec
return amount
@@ -878,7 +846,7 @@ def get_fiscal_year_date_range(from_fiscal_year, to_fiscal_year):
return from_year.year_start_date, to_year.year_end_date
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def revise_budget(budget_name: str):
old_budget = frappe.get_doc("Budget", budget_name)

View File

@@ -18,7 +18,6 @@
"in_list_view": 1,
"label": "Start Date",
"read_only": 1,
"reqd": 1,
"search_index": 1
},
{
@@ -26,29 +25,26 @@
"fieldtype": "Date",
"in_list_view": 1,
"label": "End Date",
"read_only": 1,
"reqd": 1
"read_only": 1
},
{
"fieldname": "amount",
"fieldtype": "Currency",
"in_list_view": 1,
"label": "Amount",
"reqd": 1
"label": "Amount"
},
{
"fieldname": "percent",
"fieldtype": "Percent",
"in_list_view": 1,
"label": "Percent",
"reqd": 1
"label": "Percent"
}
],
"grid_page_length": 50,
"index_web_pages_for_search": 1,
"istable": 1,
"links": [],
"modified": "2026-06-18 11:23:17.669733",
"modified": "2025-11-03 13:18:28.398198",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Budget Distribution",

View File

@@ -15,12 +15,12 @@ class BudgetDistribution(Document):
from frappe.types import DF
amount: DF.Currency
end_date: DF.Date
end_date: DF.Date | None
parent: DF.Data
parentfield: DF.Data
parenttype: DF.Data
percent: DF.Percent
start_date: DF.Date
start_date: DF.Date | None
# end: auto-generated types
pass

View File

@@ -5,7 +5,6 @@
import frappe
from frappe import _
from frappe.model.document import Document
from frappe.query_builder.functions import Sum
from frappe.utils import flt
@@ -44,17 +43,13 @@ class CashierClosing(Document):
self.make_calculations()
def get_outstanding(self):
si = frappe.qb.DocType("Sales Invoice")
values = (
frappe.qb.from_(si)
.select(Sum(si.outstanding_amount))
.where(
(si.posting_date == self.date)
& (si.posting_time >= self.from_time)
& (si.posting_time <= self.time)
& (si.owner == self.user)
)
.run()
values = frappe.db.sql(
"""
select sum(outstanding_amount)
from `tabSales Invoice`
where posting_date=%s and posting_time>=%s and posting_time<=%s and owner=%s
""",
(self.date, self.from_time, self.time, self.user),
)
self.outstanding_amount = flt(values[0][0] if values else 0)

View File

@@ -1,67 +1,8 @@
# Copyright (c) 2026, Frappe Technologies Pvt. Ltd. and Contributors
# Copyright (c) 2015, Frappe Technologies Pvt. Ltd. and Contributors
# See license.txt
import frappe
from erpnext.accounts.doctype.sales_invoice.test_sales_invoice import create_sales_invoice
from erpnext.tests.utils import ERPNextTestSuite
DATE = "2026-06-15"
class TestCashierClosing(ERPNextTestSuite):
"""Cashier Closing reconciles a shift: it pulls outstanding invoices in a
date/time window and rolls payments, expense, custody and returns into net_amount."""
def setUp(self):
frappe.set_user("Administrator")
def make_invoice_in_window(self, rate=100):
si = create_sales_invoice(rate=rate, qty=1, posting_date=DATE, do_not_submit=True)
si.posting_time = "10:30:00"
si.submit()
si.reload() # read outstanding_amount as persisted after submit
return si
def make_closing(self, user="Administrator", payments=None, **args):
doc = frappe.new_doc("Cashier Closing")
doc.user = user
doc.date = args.get("date", DATE)
doc.from_time = args.get("from_time", "09:00:00")
doc.time = args.get("time", "18:00:00")
for amount in payments or []:
doc.append("payments", {"mode_of_payment": "Cash", "amount": amount})
doc.expense = args.get("expense", 0)
doc.custody = args.get("custody", 0)
doc.returns = args.get("returns", 0)
return doc
def test_from_time_must_be_before_to_time(self):
doc = self.make_closing(from_time="18:00:00", time="09:00:00")
self.assertRaises(frappe.ValidationError, doc.save)
def test_equal_from_and_to_time_is_rejected(self):
# validate_time uses >=, so a zero-length window is also blocked
doc = self.make_closing(from_time="09:00:00", time="09:00:00")
self.assertRaises(frappe.ValidationError, doc.save)
def test_net_amount_rolls_up_outstanding_and_adjustments(self):
si = self.make_invoice_in_window(rate=100)
doc = self.make_closing(payments=[500], expense=50, custody=30, returns=20)
doc.save()
# the in-window invoice is picked up as outstanding
self.assertEqual(doc.outstanding_amount, si.outstanding_amount)
# net = payments + outstanding + expense - custody + returns
self.assertEqual(doc.net_amount, 500 + si.outstanding_amount + 50 - 30 + 20)
def test_outstanding_is_scoped_to_the_invoice_owner(self):
# The invoice is created by Administrator; a closing for a different user does
# not see it. NOTE: get_outstanding keys on Sales Invoice.owner (the document
# creator) rather than an explicit cashier/POS-user field, which is fragile when
# invoices are created by a shared or system user.
self.make_invoice_in_window(rate=100)
doc = self.make_closing(user="Guest", payments=[500])
doc.save()
self.assertEqual(doc.outstanding_amount, 0)
self.assertEqual(doc.net_amount, 500)
pass

View File

@@ -63,8 +63,8 @@ def validate_company(company: str):
)
if parent_company and (not allow_account_creation_against_child_company):
msg = _("{0} is a child company.").format(frappe.bold(company)) + " "
msg += _("Please import accounts against parent company or enable {0} in company master.").format(
msg = _("{} is a child company.").format(frappe.bold(company)) + " "
msg += _("Please import accounts against parent company or enable {} in company master.").format(
frappe.bold(_("Allow Account Creation Against Child Company"))
)
frappe.throw(msg, title=_("Wrong Company"))
@@ -75,10 +75,7 @@ def validate_company(company: str):
@frappe.whitelist()
def import_coa(file_name: str, company: str):
frappe.only_for("Accounts Manager")
# delete existing data for accounts
frappe.has_permission("Company", "write", company, throw=True)
unset_existing_data(company)
# create accounts
@@ -220,7 +217,6 @@ def build_forest(data):
for row in data:
account_name, parent_account, account_number, parent_account_number = row[0:4]
if account_number:
account_number = cstr(account_number).strip()
account_name = f"{account_number} - {account_name}"
if parent_account_number:
parent_account_number = cstr(parent_account_number).strip()
@@ -457,7 +453,6 @@ def unset_existing_data(company):
fieldnames = get_linked_fields("Account").get("Company", {}).get("fieldname", [])
linked = [{"fieldname": name} for name in fieldnames]
update_values = {d.get("fieldname"): "" for d in linked}
frappe.db.set_value("Company", company, update_values, update_values)
# remove accounts data from various doctypes
@@ -469,7 +464,8 @@ def unset_existing_data(company):
"Sales Taxes and Charges Template",
"Purchase Taxes and Charges Template",
]:
frappe.get_query(doctype, delete=True, filters={"company": company}, ignore_permissions=False).run()
dt = frappe.qb.DocType(doctype)
frappe.qb.from_(dt).where(dt.company == company).delete().run()
def set_default_accounts(company):

View File

@@ -1,54 +1,8 @@
# Copyright (c) 2026, Frappe Technologies Pvt. Ltd. and Contributors
# Copyright (c) 2019, Frappe Technologies Pvt. Ltd. and Contributors
# See license.txt
import frappe
from erpnext.accounts.doctype.chart_of_accounts_importer.chart_of_accounts_importer import (
build_forest,
validate_columns,
validate_missing_roots,
)
from erpnext.tests.utils import ERPNextTestSuite
# columns: account_name, parent_account, account_number, parent_account_number,
# is_group, account_type, root_type, account_currency
ROOT = ["Assets", "Assets", "", "", 1, "", "Asset", "INR"]
CHILD = ["Cash", "Assets", "", "", 0, "Cash", "Asset", "INR"]
class TestChartofAccountsImporter(ERPNextTestSuite):
"""The importer parses an uploaded CoA into a nested tree and validates its
shape. These cover the parsing/validation helpers without a file upload."""
def test_validate_columns_rejects_blank_file(self):
self.assertRaises(frappe.ValidationError, validate_columns, [])
def test_validate_columns_requires_eight_columns(self):
self.assertRaises(frappe.ValidationError, validate_columns, [["a", "b", "c"]])
# the standard template width passes
validate_columns([ROOT])
def test_build_forest_nests_child_under_parent(self):
forest = build_forest([ROOT, CHILD])
self.assertIn("Assets", forest)
self.assertIn("Cash", forest["Assets"])
def test_build_forest_rejects_unknown_parent(self):
orphan = ["Cash", "Missing Parent", "", "", 0, "Cash", "Asset", "INR"]
self.assertRaises(frappe.ValidationError, build_forest, [orphan])
def test_build_forest_requires_account_name(self):
nameless = ["", "Assets", "", "", 0, "Cash", "Asset", "INR"]
self.assertRaises(frappe.ValidationError, build_forest, [ROOT, nameless])
def test_validate_missing_roots_requires_all_root_types(self):
present = ("Asset", "Liability", "Expense", "Income") # Equity missing
self.assertRaises(
frappe.ValidationError,
validate_missing_roots,
[{"root_type": rt} for rt in present],
)
# all five root types present -> no error
validate_missing_roots(
[{"root_type": rt} for rt in ("Asset", "Liability", "Expense", "Income", "Equity")]
)
pass

View File

@@ -46,7 +46,7 @@ class ChequePrintTemplate(Document):
pass
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def create_or_update_cheque_print_format(template_name: str):
frappe.only_for("System Manager")

View File

@@ -84,10 +84,10 @@ class CostCenter(NestedSet):
return frappe.db.get_value("GL Entry", {"cost_center": self.name})
def check_if_child_exists(self):
return frappe.get_all(
"Cost Center",
filters={"parent_cost_center": self.name, "docstatus": ["!=", 2]},
pluck="name",
return frappe.db.sql(
"select name from `tabCost Center` where \
parent_cost_center = %s and docstatus != 2",
self.name,
)
def if_allocation_exists_against_cost_center(self):

View File

@@ -173,66 +173,3 @@ class TestCouponCode(ERPNextTestSuite):
# Clean up
coupon.delete()
def test_validate_coupon_code_rejections(self):
from frappe.utils import add_days, nowdate
from erpnext.accounts.doctype.pricing_rule.utils import validate_coupon_code
pricing_rule = frappe.db.get_value("Pricing Rule", {"title": "_Test Pricing Rule for _Test Item"})
def make_coupon(name, **kwargs):
frappe.delete_doc_if_exists("Coupon Code", name)
return frappe.get_doc(
{
"doctype": "Coupon Code",
"coupon_name": name,
"coupon_code": name,
"coupon_type": "Promotional",
"pricing_rule": pricing_rule,
**kwargs,
}
).insert(ignore_permissions=True)
with self.subTest("validity not yet started"):
make_coupon("_Test Coupon Future", valid_from=add_days(nowdate(), 5))
self.assertRaises(frappe.ValidationError, validate_coupon_code, "_Test Coupon Future")
with self.subTest("validity expired"):
make_coupon("_Test Coupon Expired", valid_upto=add_days(nowdate(), -5))
self.assertRaises(frappe.ValidationError, validate_coupon_code, "_Test Coupon Expired")
with self.subTest("maximum use exhausted"):
make_coupon("_Test Coupon Exhausted", maximum_use=2, used=2)
self.assertRaises(frappe.ValidationError, validate_coupon_code, "_Test Coupon Exhausted")
with self.subTest("valid coupon passes"):
make_coupon("_Test Coupon Valid", maximum_use=5, used=1, valid_upto=add_days(nowdate(), 5))
validate_coupon_code("_Test Coupon Valid") # no raise
def test_update_coupon_code_count_cancel_and_exhaust(self):
from erpnext.accounts.doctype.pricing_rule.utils import update_coupon_code_count
pricing_rule = frappe.db.get_value("Pricing Rule", {"title": "_Test Pricing Rule for _Test Item"})
frappe.delete_doc_if_exists("Coupon Code", "_Test Coupon Count")
frappe.get_doc(
{
"doctype": "Coupon Code",
"coupon_name": "_Test Coupon Count",
"coupon_code": "_Test Coupon Count",
"coupon_type": "Promotional",
"pricing_rule": pricing_rule,
"maximum_use": 2,
"used": 1,
}
).insert(ignore_permissions=True)
# cancelling a transaction releases one use
update_coupon_code_count("_Test Coupon Count", "cancelled")
self.assertEqual(frappe.db.get_value("Coupon Code", "_Test Coupon Count", "used"), 0)
# using up to the maximum is allowed, beyond it is rejected
update_coupon_code_count("_Test Coupon Count", "used")
update_coupon_code_count("_Test Coupon Count", "used")
self.assertEqual(frappe.db.get_value("Coupon Code", "_Test Coupon Count", "used"), 2)
self.assertRaises(frappe.ValidationError, update_coupon_code_count, "_Test Coupon Count", "used")

View File

@@ -11,28 +11,22 @@ frappe.ui.form.on("Currency Exchange Settings", {
},
callback: function (r) {
if (r && r.message) {
let result = [],
params = {};
if (frm.doc.service_provider == "exchangerate.host") {
result = ["result"];
params = {
let result = ["result"];
let params = {
date: "{transaction_date}",
from: "{from_currency}",
to: "{to_currency}",
};
add_param(frm, r.message, params, result);
} else if (["frankfurter.app", "frankfurter.dev"].includes(frm.doc.service_provider)) {
result = ["rates", "{to_currency}"];
params = {
let result = ["rates", "{to_currency}"];
let params = {
base: "{from_currency}",
symbols: "{to_currency}",
};
} else if (frm.doc.service_provider == "frankfurter.dev - v2") {
result = ["rate"];
params = {
date: "{transaction_date}",
};
add_param(frm, r.message, params, result);
}
add_param(frm, r.message, params, result);
}
},
});

View File

@@ -78,7 +78,7 @@
"fieldname": "service_provider",
"fieldtype": "Select",
"label": "Service Provider",
"options": "frankfurter.dev\nexchangerate.host\nfrankfurter.dev - v2\nCustom",
"options": "frankfurter.dev\nexchangerate.host\nCustom",
"reqd": 1
},
{
@@ -101,10 +101,11 @@
"label": "Use HTTP Protocol"
}
],
"hide_toolbar": 0,
"index_web_pages_for_search": 1,
"issingle": 1,
"links": [],
"modified": "2026-06-15 11:25:55.873110",
"modified": "2026-03-16 13:28:21.075743",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Currency Exchange Settings",
@@ -121,11 +122,24 @@
"write": 1
},
{
"create": 1,
"delete": 1,
"email": 1,
"print": 1,
"read": 1,
"role": "Accounts Manager",
"share": 1,
"write": 1
},
{
"create": 1,
"delete": 1,
"email": 1,
"print": 1,
"read": 1,
"role": "Accounts User",
"share": 1
"share": 1,
"write": 1
}
],
"row_format": "Dynamic",

View File

@@ -29,7 +29,7 @@ class CurrencyExchangeSettings(Document):
disabled: DF.Check
req_params: DF.Table[CurrencyExchangeSettingsDetails]
result_key: DF.Table[CurrencyExchangeSettingsResult]
service_provider: DF.Literal["frankfurter.dev", "exchangerate.host", "frankfurter.dev - v2", "Custom"]
service_provider: DF.Literal["frankfurter.dev", "exchangerate.host", "Custom"]
url: DF.Data | None
use_http: DF.Check
# end: auto-generated types
@@ -70,14 +70,6 @@ class CurrencyExchangeSettings(Document):
self.append("req_params", {"key": "base", "value": "{from_currency}"})
self.append("req_params", {"key": "symbols", "value": "{to_currency}"})
elif self.service_provider == "frankfurter.dev - v2":
self.set("result_key", [])
self.set("req_params", [])
self.api_endpoint = get_api_endpoint(self.service_provider, self.use_http)
self.append("result_key", {"key": "rate"})
self.append("req_params", {"key": "date", "value": "{transaction_date}"})
def validate_parameters(self):
params = {}
for row in self.req_params:
@@ -90,7 +82,7 @@ class CurrencyExchangeSettings(Document):
try:
response = requests.get(api_url, params=params)
except requests.exceptions.RequestException as e:
frappe.throw(_("Error: {0}").format(str(e)))
frappe.throw("Error: " + str(e))
response.raise_for_status()
value = response.json()
@@ -113,20 +105,13 @@ class CurrencyExchangeSettings(Document):
@frappe.whitelist()
def get_api_endpoint(service_provider: str | None = None, use_http: bool = False):
if service_provider and service_provider in [
"exchangerate.host",
"frankfurter.dev",
"frankfurter.app",
"frankfurter.dev - v2",
]:
if service_provider and service_provider in ["exchangerate.host", "frankfurter.dev", "frankfurter.app"]:
if service_provider == "exchangerate.host":
api = "api.exchangerate.host/convert"
elif service_provider == "frankfurter.app":
api = "api.frankfurter.app/{transaction_date}"
elif service_provider == "frankfurter.dev":
api = "api.frankfurter.dev/v1/{transaction_date}"
elif service_provider == "frankfurter.dev - v2":
api = "api.frankfurter.dev/v2/rate/{from_currency}/{to_currency}"
protocol = "https://"
if use_http:

View File

@@ -169,10 +169,23 @@ frappe.ui.form.on("Dunning", {
},
get_dunning_letter_text: function (frm) {
if (frm.doc.dunning_type) {
frm.call("get_dunning_letter_text").then((r) => {
if (!r.exc) {
frm.refresh_fields();
}
frappe.call({
method: "erpnext.accounts.doctype.dunning.dunning.get_dunning_letter_text",
args: {
dunning_type: frm.doc.dunning_type,
language: frm.doc.language,
doc: frm.doc,
},
callback: function (r) {
if (r.message) {
frm.set_value("body_text", r.message.body_text);
frm.set_value("closing_text", r.message.closing_text);
frm.set_value("language", r.message.language);
} else {
frm.set_value("body_text", "");
frm.set_value("closing_text", "");
}
},
});
}
},

View File

@@ -85,7 +85,7 @@ class Dunning(AccountsController):
if invoice_currency != self.currency:
frappe.throw(
_(
"The currency of invoice {0} ({1}) is different from the currency of this dunning ({2})."
"The currency of invoice {} ({}) is different from the currency of this dunning ({})."
).format(
frappe.get_desk_link(
"Sales Invoice",
@@ -163,46 +163,6 @@ class Dunning(AccountsController):
"Serial and Batch Bundle",
]
@frappe.whitelist()
def get_dunning_letter_text(self):
DOCTYPE = "Dunning Letter Text"
FIELDS = ["body_text", "closing_text", "language"]
if not self.dunning_type:
return
filters = {"parent": self.dunning_type, "is_default_language": 1}
if self.language:
filters.pop("is_default_language")
filters["language"] = self.language
letter_text = frappe.db.get_value(DOCTYPE, filters, FIELDS, as_dict=True)
if not letter_text:
msg = (
_("Dunning Letter for Dunning Type {0} in language '{1}' not found.").format(
frappe.bold(self.dunning_type), frappe.bold(self.language)
)
if self.language
else _("Dunning Letter for Dunning Type {0} not found.").format(
frappe.bold(self.dunning_type)
)
)
frappe.msgprint(msg, alert=True, indicator="yellow")
self.body_text = (
frappe.render_template(letter_text.body_text, self.as_dict(), restrict_globals=True)
if letter_text
else None
)
self.closing_text = (
frappe.render_template(letter_text.closing_text, self.as_dict(), restrict_globals=True)
if letter_text
else None
)
self.language = letter_text.language if letter_text else self.language
def update_linked_dunnings(doc, previous_outstanding_amount):
if (
@@ -281,3 +241,35 @@ def get_linked_dunnings_as_per_state(sales_invoice, state):
& (overdue_payment.sales_invoice == sales_invoice)
)
).run(as_dict=True)
@frappe.whitelist()
def get_dunning_letter_text(dunning_type: str, doc: str | dict, language: str | None = None) -> dict:
DOCTYPE = "Dunning Letter Text"
FIELDS = ["body_text", "closing_text", "language"]
if isinstance(doc, str):
doc = json.loads(doc)
if not language:
language = doc.get("language")
letter_text = None
if language:
letter_text = frappe.db.get_value(
DOCTYPE, {"parent": dunning_type, "language": language}, FIELDS, as_dict=1
)
if not letter_text:
letter_text = frappe.db.get_value(
DOCTYPE, {"parent": dunning_type, "is_default_language": 1}, FIELDS, as_dict=1
)
if not letter_text:
return {}
return {
"body_text": frappe.render_template(letter_text.body_text, doc),
"closing_text": frappe.render_template(letter_text.closing_text, doc),
"language": letter_text.language,
}

View File

@@ -12,7 +12,6 @@ from erpnext.accounts.doctype.sales_invoice.mapper import (
create_dunning as create_dunning_from_sales_invoice,
)
from erpnext.accounts.doctype.sales_invoice.test_sales_invoice import (
create_sales_invoice,
create_sales_invoice_against_cost_center,
)
from erpnext.tests.utils import ERPNextTestSuite
@@ -153,37 +152,6 @@ class TestDunning(ERPNextTestSuite):
dunning.reload()
self.assertEqual(dunning.status, "Unresolved")
@ERPNextTestSuite.change_settings(
"Accounts Settings", {"allow_multi_currency_invoices_against_single_party_account": 1}
)
def test_dunning_outstanding_uses_transaction_currency(self):
"""
Regression for #56006: dunning outstanding must be in the invoice transaction
currency, not in the party account currency.
A USD invoice posted against an INR receivable account stores
outstanding_amount in INR (party account currency). The overdue payment
row on the resulting Dunning must carry the USD amount, not the INR amount.
"""
si = create_sales_invoice(
posting_date=add_days(today(), -10),
currency="USD",
conversion_rate=50,
rate=100,
debit_to="Debtors - _TC",
)
# Sanity-check the invoice state before creating the dunning
self.assertEqual(si.currency, "USD")
self.assertEqual(si.outstanding_amount, 5000.0) # INR (party account currency)
self.assertEqual(si.payment_schedule[0].outstanding, 100.0) # USD (transaction currency)
dunning = create_dunning_from_sales_invoice(si.name)
self.assertEqual(len(dunning.overdue_payments), 1)
# Must reflect 100 USD, not 5000 INR mislabelled as USD
self.assertEqual(dunning.overdue_payments[0].outstanding, 100.0)
def test_dunning_not_affected_by_standalone_credit_note(self):
"""
Test that dunning is NOT resolved when a credit note has update_outstanding_for_self checked.

View File

@@ -3,10 +3,7 @@
import frappe
from frappe import _
from frappe.model.document import Document
from frappe.utils import comma_and
from frappe.utils.jinja import validate_template
class DunningType(Document):
@@ -33,134 +30,3 @@ class DunningType(Document):
def autoname(self):
company_abbr = frappe.get_value("Company", self.company, "abbr")
self.name = f"{self.dunning_type} - {company_abbr}"
def validate(self):
self.validate_dunning_letter_text()
self.validate_income_account()
self.validate_cost_center()
self.set_default_dunning_type()
def validate_dunning_letter_text(self):
self.validate_languages()
self.validate_is_default_language()
self.validate_dunning_letter_text_templates()
def validate_income_account(self):
if not self.income_account:
return
account = frappe.get_cached_doc("Account", self.income_account)
msg = []
if account.company != self.company:
msg.append(
_(
"{0} doesn't belong to Company {1}. Please select an Income Account that belongs to Company {1}."
).format(frappe.bold(self.income_account), frappe.bold(self.company))
)
if account.disabled:
msg.append(
_("{0} is disabled. Please select a valid Income Account.").format(
frappe.bold(self.income_account)
)
)
if account.root_type != "Income":
msg.append(
_("{0} is not an Income Account. Please select a valid Income Account.").format(
frappe.bold(self.income_account)
)
)
if account.is_group:
msg.append(
_("{0} is a group account. Please select a non-group Income Account.").format(
frappe.bold(self.income_account)
)
)
if msg:
frappe.msgprint(
msg,
title=_("Income Account Validation Error"),
as_list=True,
raise_exception=frappe.ValidationError,
)
def validate_cost_center(self):
if not self.cost_center:
return
cost_center = frappe.get_cached_doc("Cost Center", self.cost_center)
msg = []
if cost_center.company != self.company:
msg.append(
_(
"{0} doesn't belong to Company {1}. Please select a Cost Center that belongs to Company {1}."
).format(frappe.bold(self.cost_center), frappe.bold(self.company))
)
if cost_center.disabled:
msg.append(
_("{0} is disabled. Please select an enabled Cost Center.").format(
frappe.bold(self.cost_center)
)
)
if cost_center.is_group:
msg.append(
_("{0} is a group Cost Center. Please select a non-group Cost Center.").format(
frappe.bold(self.cost_center)
)
)
if msg:
frappe.msgprint(
msg,
title=_("Cost Center Validation Error"),
as_list=True,
raise_exception=frappe.ValidationError,
)
def validate_languages(self):
languages = [d.language for d in self.dunning_letter_text]
if len(languages) == len(set(languages)):
return
frappe.throw(_("Duplicate languages found on Dunning Letter Text. Keep only one of them."))
def validate_is_default_language(self):
is_default_language_list = [
d.language for d in self.dunning_letter_text if d.is_default_language == 1
]
if len(is_default_language_list) <= 1:
return
frappe.throw(
_("{0} languages are marked as default languages. Please select only one of them.").format(
comma_and(is_default_language_list, add_quotes=True)
)
)
def validate_dunning_letter_text_templates(self):
for d in self.dunning_letter_text:
if d.body_text:
validate_template(d.body_text, restrict_globals=True)
if d.closing_text:
validate_template(d.closing_text, restrict_globals=True)
def set_default_dunning_type(self):
if self.is_default != 1:
return
frappe.db.set_value(
"Dunning Type",
{"company": self.company, "is_default": 1, "name": ["!=", self.name]},
"is_default",
0,
)

View File

@@ -1,200 +1,9 @@
# Copyright (c) 2020, Frappe Technologies Pvt. Ltd. and Contributors
# See license.txt
import frappe
# import frappe
from erpnext.tests.utils import ERPNextTestSuite
def make_dunning_type(dunning_type, company="_Test Company", **kwargs):
doc = frappe.new_doc("Dunning Type")
doc.dunning_type = dunning_type
doc.company = company
doc.dunning_fee = kwargs.get("dunning_fee", 100)
doc.rate_of_interest = kwargs.get("rate_of_interest", 5)
doc.is_default = kwargs.get("is_default", 0)
if "income_account" in kwargs:
doc.income_account = kwargs["income_account"]
elif kwargs.get("income_account") is not False:
doc.income_account = "Sales - _TC" if company == "_Test Company" else "Sales - _TC1"
if "cost_center" in kwargs:
doc.cost_center = kwargs["cost_center"]
elif kwargs.get("cost_center") is not False:
doc.cost_center = "Main - _TC" if company == "_Test Company" else "Main - _TC1"
for row in kwargs.get("dunning_letter_text", [{"language": "en", "body_text": "Test body"}]):
doc.append("dunning_letter_text", row)
return doc
class TestDunningType(ERPNextTestSuite):
def test_income_account_must_belong_to_company(self):
doc = make_dunning_type("_Test Dunning Wrong Company Account", income_account="Sales - _TC1")
self.assertRaisesRegex(frappe.ValidationError, "doesn't belong to Company", doc.insert)
def test_income_account_must_not_be_disabled(self):
disabled_account = frappe.get_doc(
{
"doctype": "Account",
"account_name": "_Test Disabled Income Account",
"parent_account": "Direct Income - _TC",
"company": "_Test Company",
"account_type": "Income Account",
"disabled": 1,
}
).insert()
doc = make_dunning_type("_Test Dunning Disabled Account", income_account=disabled_account.name)
self.assertRaisesRegex(frappe.ValidationError, "is disabled", doc.insert)
def test_income_account_must_be_income_type(self):
doc = make_dunning_type("_Test Dunning Non Income Account", income_account="Debtors - _TC")
self.assertRaisesRegex(frappe.ValidationError, "is not an Income Account", doc.insert)
def test_income_account_must_not_be_group(self):
doc = make_dunning_type("_Test Dunning Group Account", income_account="Income - _TC")
self.assertRaisesRegex(frappe.ValidationError, "is a group account", doc.insert)
def test_income_account_is_optional(self):
doc = make_dunning_type("_Test Dunning No Income Account", income_account=False)
doc.insert()
self.assertFalse(doc.income_account)
def test_valid_income_account_passes(self):
doc = make_dunning_type("_Test Dunning Valid Income Account", income_account="Sales - _TC")
doc.insert()
self.assertEqual(doc.income_account, "Sales - _TC")
def test_cost_center_must_belong_to_company(self):
doc = make_dunning_type("_Test Dunning Wrong Company CC", cost_center="Main - _TC1")
self.assertRaisesRegex(frappe.ValidationError, "doesn't belong to Company", doc.insert)
def test_cost_center_must_not_be_disabled(self):
disabled_cc = frappe.get_doc(
{
"doctype": "Cost Center",
"cost_center_name": "_Test Disabled Cost Center",
"parent_cost_center": "_Test Company - _TC",
"company": "_Test Company",
"disabled": 1,
}
).insert()
doc = make_dunning_type("_Test Dunning Disabled CC", cost_center=disabled_cc.name)
self.assertRaisesRegex(frappe.ValidationError, "is disabled", doc.insert)
def test_cost_center_must_not_be_group(self):
doc = make_dunning_type("_Test Dunning Group CC", cost_center="_Test Company - _TC")
self.assertRaisesRegex(frappe.ValidationError, "is a group Cost Center", doc.insert)
def test_cost_center_is_optional(self):
doc = make_dunning_type("_Test Dunning No CC", cost_center=False)
doc.insert()
self.assertFalse(doc.cost_center)
def test_valid_cost_center_passes(self):
doc = make_dunning_type("_Test Dunning Valid CC", cost_center="Main - _TC")
doc.insert()
self.assertEqual(doc.cost_center, "Main - _TC")
def test_duplicate_languages_not_allowed(self):
doc = make_dunning_type(
"_Test Dunning Duplicate Language",
dunning_letter_text=[
{"language": "en", "body_text": "Body one"},
{"language": "en", "body_text": "Body two"},
],
)
self.assertRaisesRegex(frappe.ValidationError, "Duplicate languages found", doc.insert)
def test_unique_languages_allowed(self):
doc = make_dunning_type(
"_Test Dunning Unique Languages",
dunning_letter_text=[
{"language": "en", "body_text": "Body one"},
{"language": "de", "body_text": "Body two"},
],
)
doc.insert()
self.assertEqual(len(doc.dunning_letter_text), 2)
def test_only_one_default_language_allowed(self):
doc = make_dunning_type(
"_Test Dunning Multiple Default Language",
dunning_letter_text=[
{"language": "en", "body_text": "Body one", "is_default_language": 1},
{"language": "de", "body_text": "Body two", "is_default_language": 1},
],
)
self.assertRaisesRegex(
frappe.ValidationError, "languages are marked as default languages", doc.insert
)
def test_single_default_language_allowed(self):
doc = make_dunning_type(
"_Test Dunning Single Default Language",
dunning_letter_text=[
{"language": "en", "body_text": "Body one", "is_default_language": 1},
{"language": "de", "body_text": "Body two", "is_default_language": 0},
],
)
doc.insert()
self.assertEqual(doc.dunning_letter_text[0].is_default_language, 1)
def test_invalid_jinja_template_in_body_text_raises(self):
doc = make_dunning_type(
"_Test Dunning Invalid Body Template",
dunning_letter_text=[{"language": "en", "body_text": "{{ unclosed"}],
)
self.assertRaisesRegex(frappe.ValidationError, "Syntax error in template", doc.insert)
def test_invalid_jinja_template_in_closing_text_raises(self):
doc = make_dunning_type(
"_Test Dunning Invalid Closing Template",
dunning_letter_text=[
{"language": "en", "body_text": "Valid body", "closing_text": "{{ unclosed"}
],
)
self.assertRaisesRegex(frappe.ValidationError, "Syntax error in template", doc.insert)
def test_valid_jinja_template_passes(self):
doc = make_dunning_type(
"_Test Dunning Valid Template",
dunning_letter_text=[
{
"language": "en",
"body_text": "Outstanding amount is {{ outstanding_amount }}",
"closing_text": "Regards, {{ company }}",
}
],
)
doc.insert()
self.assertTrue(doc.name)
def test_set_default_dunning_type_unsets_previous_default(self):
first = make_dunning_type("_Test Dunning Default One", is_default=1)
first.insert()
self.assertEqual(frappe.db.get_value("Dunning Type", first.name, "is_default"), 1)
second = make_dunning_type("_Test Dunning Default Two", is_default=1)
second.insert()
self.assertEqual(frappe.db.get_value("Dunning Type", first.name, "is_default"), 0)
self.assertEqual(frappe.db.get_value("Dunning Type", second.name, "is_default"), 1)
def test_set_default_dunning_type_scoped_per_company(self):
company_1 = make_dunning_type("_Test Dunning Default Co1", is_default=1)
company_1.insert()
company_2 = make_dunning_type(
"_Test Dunning Default Co2",
company="_Test Company 1",
is_default=1,
)
company_2.insert()
self.assertEqual(frappe.db.get_value("Dunning Type", company_1.name, "is_default"), 1)
self.assertEqual(frappe.db.get_value("Dunning Type", company_2.name, "is_default"), 1)
pass

View File

@@ -22,27 +22,17 @@ frappe.ui.form.on("Exchange Rate Revaluation", {
refresh: function (frm) {
if (frm.doc.docstatus == 1) {
frappe.call({
method: "check_journal_and_reversal",
method: "check_journal_entry_condition",
doc: frm.doc,
callback: function (r) {
if (r.message) {
if (!r.message.journals_posted) {
frm.add_custom_button(
__("Journal Entries"),
function () {
return frm.events.make_jv(frm);
},
__("Create")
);
} else if (!r.message.reversals_posted) {
frm.add_custom_button(
__("Reversal Journal Entries"),
function () {
return frm.events.make_reverse_journal(frm);
},
__("Create")
);
}
frm.add_custom_button(
__("Journal Entries"),
function () {
return frm.events.make_jv(frm);
},
__("Create")
);
}
},
});
@@ -110,14 +100,6 @@ frappe.ui.form.on("Exchange Rate Revaluation", {
},
});
},
make_reverse_journal: function (frm) {
frappe.call({
method: "make_reverse_journal",
doc: frm.doc,
freeze: true,
freeze_message: __("Reversing Journals..."),
});
},
});
frappe.ui.form.on("Exchange Rate Revaluation Account", {
@@ -154,7 +136,7 @@ frappe.ui.form.on("Exchange Rate Revaluation Account", {
var get_account_details = function (frm, cdt, cdn) {
var row = frappe.get_doc(cdt, cdn);
if (!frm.doc.company || !frm.doc.posting_date) {
frappe.throw(__("Please select Company and Posting Date to get entries"));
frappe.throw(__("Please select Company and Posting Date to getting entries"));
}
frappe.call({
method: "erpnext.accounts.doctype.exchange_rate_revaluation.exchange_rate_revaluation.get_account_details",

View File

@@ -8,8 +8,8 @@ from frappe import _, qb
from frappe.model.document import Document
from frappe.model.meta import get_field_precision
from frappe.query_builder import Criterion, Order
from frappe.query_builder.functions import Max, NullIf, Sum
from frappe.utils import flt, get_link_to_form, nowdate
from frappe.query_builder.functions import NullIf, Sum
from frappe.utils import flt, get_link_to_form
import erpnext
from erpnext.accounts.doctype.journal_entry.journal_entry import get_balance_on
@@ -73,7 +73,7 @@ class ExchangeRateRevaluation(Document):
def validate_mandatory(self):
if not (self.company and self.posting_date):
frappe.throw(_("Please select Company and Posting Date to get entries"))
frappe.throw(_("Please select Company and Posting Date to getting entries"))
def before_submit(self):
self.remove_accounts_without_gain_loss()
@@ -91,31 +91,25 @@ class ExchangeRateRevaluation(Document):
)
def on_cancel(self):
self.ignore_linked_doctypes = ["GL Entry", "Payment Ledger Entry"]
self.ignore_linked_doctypes = "GL Entry"
@frappe.whitelist()
def check_journal_and_reversal(self):
def check_journal_entry_condition(self):
exchange_gain_loss_account = self.get_for_unrealized_gain_loss_account()
journals_posted = False
reversals_posted = False
je = qb.DocType("Journal Entry")
jea = qb.DocType("Journal Entry Account")
journals = (
qb.from_(je)
.join(jea)
.on(je.name == jea.parent)
.select(je.name)
qb.from_(jea)
.select(jea.parent)
.distinct()
.where(
(jea.reference_type == "Exchange Rate Revaluation")
& (jea.reference_name == self.name)
& (jea.docstatus == 1)
& (je.reversal_of.isnull()) # omit journals that have reversals
)
.run(pluck="name")
.run()
)
if journals:
gle = qb.DocType("GL Entry")
total_amt = (
@@ -130,31 +124,12 @@ class ExchangeRateRevaluation(Document):
.run()
)
if total_amt and total_amt[0][0] == self.total_gain_loss:
journals_posted = True
if total_amt and total_amt[0][0] != self.total_gain_loss:
return True
else:
journals_posted = False
return False
# reverse journals
reverse_journals = (
qb.from_(je)
.join(jea)
.on(je.name == jea.parent)
.select(je.name)
.where(
(jea.reference_type == "Exchange Rate Revaluation")
& (jea.reference_name == self.name)
& (jea.docstatus == 1)
& (je.reversal_of.notnull())
)
.run(pluck="name")
)
if reverse_journals:
reversals_posted = True
else:
reversals_posted = False
return {"journals_posted": journals_posted, "reversals_posted": reversals_posted}
return True
def fetch_and_calculate_accounts_data(self):
accounts = self.get_accounts_data()
@@ -213,17 +188,11 @@ class ExchangeRateRevaluation(Document):
accounts = [x[0] for x in res]
if accounts:
gle = qb.DocType("GL Entry")
having_clause = (qb.Field("balance") != qb.Field("balance_in_account_currency")) & (
(qb.Field("balance_in_account_currency") != 0) | (qb.Field("balance") != 0)
)
# balance expressions reused in both SELECT and HAVING; postgres can't reference a
# SELECT alias inside HAVING, so the aggregate expression must be repeated there.
balance = Sum(gle.debit) - Sum(gle.credit)
balance_in_account_currency = Sum(gle.debit_in_account_currency) - Sum(
gle.credit_in_account_currency
)
having_clause = (balance != balance_in_account_currency) & (
(balance_in_account_currency != 0) | (balance != 0)
)
gle = qb.DocType("GL Entry")
# conditions
conditions = []
@@ -240,15 +209,17 @@ class ExchangeRateRevaluation(Document):
qb.from_(gle)
.select(
gle.account,
# grouped by NullIf(party_type/party, ""); the bare columns + account_currency are
# constant per group -> Max() keeps the GROUP BY valid on postgres with the same value.
Max(gle.party_type).as_("party_type"),
Max(gle.party).as_("party"),
Max(gle.account_currency).as_("account_currency"),
balance_in_account_currency.as_("balance_in_account_currency"),
balance.as_("balance"),
# zero_balance is recomputed in Python below (after rounding), so the SQL value is
# unused -- dropped (it used MySQL's XOR operator, which postgres lacks).
gle.party_type,
gle.party,
gle.account_currency,
(Sum(gle.debit_in_account_currency) - Sum(gle.credit_in_account_currency)).as_(
"balance_in_account_currency"
),
(Sum(gle.debit) - Sum(gle.credit)).as_("balance"),
(Sum(gle.debit) - Sum(gle.credit) == 0)
^ (Sum(gle.debit_in_account_currency) - Sum(gle.credit_in_account_currency) == 0).as_(
"zero_balance"
),
)
.where(Criterion.all(conditions))
.groupby(gle.account, NullIf(gle.party_type, ""), NullIf(gle.party, ""))
@@ -372,18 +343,15 @@ class ExchangeRateRevaluation(Document):
@frappe.whitelist()
def make_jv_entries(self):
frappe.has_permission("Journal Entry", "write", throw=True)
zero_balance_jv = self.make_jv_for_zero_balance()
if zero_balance_jv:
frappe.msgprint(
_("Zero Balance Journal: {0}").format(get_link_to_form("Journal Entry", zero_balance_jv.name))
f"Zero Balance Journal: {get_link_to_form('Journal Entry', zero_balance_jv.name)}"
)
revaluation_jv = self.make_jv_for_revaluation()
if revaluation_jv:
frappe.msgprint(
_("Revaluation Journal: {0}").format(get_link_to_form("Journal Entry", revaluation_jv.name))
)
frappe.msgprint(f"Revaluation Journal: {get_link_to_form('Journal Entry', revaluation_jv.name)}")
return {
"revaluation_jv": revaluation_jv.name if revaluation_jv else None,
@@ -601,50 +569,6 @@ class ExchangeRateRevaluation(Document):
journal_entry.save()
return journal_entry
@frappe.whitelist()
def make_reverse_journal(self):
frappe.has_permission("Journal Entry", "write", throw=True)
je = qb.DocType("Journal Entry")
jea = qb.DocType("Journal Entry Account")
journals = (
qb.from_(je)
.join(jea)
.on(je.name == jea.parent)
.select(je.name)
.distinct()
.where(
(jea.reference_type == "Exchange Rate Revaluation")
& (jea.reference_name == self.name)
& (jea.docstatus == 1)
& (je.reversal_of.isnull()) # omit journals that have reversals
)
.run(pluck="name")
)
if journals:
from erpnext.accounts.doctype.journal_entry.mapper import make_reverse_journal_entry
if drafts := frappe.db.get_all(
"Journal Entry",
filters={"docstatus": 0, "reversal_of": ["in", journals]},
pluck="name",
as_list=1,
):
part = "journals are" if len(drafts) > 1 else "journal is"
doc_links = ", ".join(["{}".format(get_link_to_form("Journal Entry", x)) for x in drafts])
frappe.throw(
msg=_("Reverse {0} already available in draft status: {1}").format(part, doc_links),
)
else:
for x in journals:
reversal = make_reverse_journal_entry(x)
reversal.posting_date = nowdate()
reversal.save()
frappe.msgprint(
_("A draft reverse journal for {0} has been created: {1}").format(
frappe.bold(x), get_link_to_form("Journal Entry", reversal.name)
)
)
def calculate_exchange_rate_using_last_gle(company, account, party_type, party):
"""
@@ -671,22 +595,17 @@ def calculate_exchange_rate_using_last_gle(company, account, party_type, party):
.select(gl.voucher_type, gl.voucher_no)
.where(Criterion.all(conditions))
.orderby(gl.posting_date, order=Order.desc)
.orderby(gl.name, order=Order.desc)
.limit(1)
.run()[0]
)
last_exchange_rate = (
qb.from_(gl)
.select(
(gl.debit - gl.credit)
/ NullIf(gl.debit_in_account_currency - gl.credit_in_account_currency, 0)
)
.select((gl.debit - gl.credit) / (gl.debit_in_account_currency - gl.credit_in_account_currency))
.where(
(gl.voucher_type == voucher_type) & (gl.voucher_no == voucher_no) & (gl.account == account)
)
.orderby(gl.posting_date, order=Order.desc)
.orderby(gl.name, order=Order.desc)
.limit(1)
.run()[0][0]
)
@@ -703,8 +622,6 @@ def get_account_details(
party: str | None = None,
rounding_loss_allowance: float = 0.0,
):
frappe.has_permission("Account", doc=account, throw=True)
if not (company and posting_date):
frappe.throw(_("Company and Posting Date is mandatory"))

View File

@@ -9,19 +9,28 @@ from frappe.utils import add_days, flt, today
from erpnext.accounts.doctype.payment_entry.payment_entry import get_payment_entry
from erpnext.accounts.doctype.sales_invoice.test_sales_invoice import create_sales_invoice
from erpnext.accounts.test.accounts_mixin import AccountsTestMixin
from erpnext.tests.utils import ERPNextTestSuite
class TestExchangeRateRevaluation(ERPNextTestSuite):
class TestExchangeRateRevaluation(ERPNextTestSuite, AccountsTestMixin):
def setUp(self):
self.company = "_Test Company"
self.item = "_Test Item"
self.customer = "_Test Customer"
self.cost_center = "Main - _TC"
self.debtors_usd = "_Test Receivable USD - _TC"
self.create_company()
self.create_usd_receivable_account()
self.create_item()
self.create_customer()
self.clear_old_entries()
self.set_system_and_company_settings()
def set_system_and_company_settings(self):
# set number and currency precision
system_settings = frappe.get_doc("System Settings")
system_settings.float_precision = 2
system_settings.currency_precision = 2
system_settings.language = "en"
system_settings.time_zone = "Asia/Kolkata"
system_settings.save()
# Using Exchange Gain/Loss account for unrealized as well.
company_doc = frappe.get_doc("Company", self.company)
company_doc.unrealized_exchange_gain_loss_account = company_doc.exchange_gain_loss_account
@@ -123,8 +132,7 @@ class TestExchangeRateRevaluation(ERPNextTestSuite):
err = err.save().submit()
# Create JV for ERR
ret = err.check_journal_and_reversal()
self.assertFalse(ret.get("journals_posted"))
self.assertTrue(err.check_journal_entry_condition())
err_journals = err.make_jv_entries()
je = frappe.get_doc("Journal Entry", err_journals.get("zero_balance_jv"))
je = je.submit()
@@ -213,8 +221,7 @@ class TestExchangeRateRevaluation(ERPNextTestSuite):
err = err.save().submit()
# Create JV for ERR
ret = err.check_journal_and_reversal()
self.assertFalse(ret.get("journals_posted"))
self.assertTrue(err.check_journal_entry_condition())
err_journals = err.make_jv_entries()
je = frappe.get_doc("Journal Entry", err_journals.get("zero_balance_jv"))
je = je.submit()
@@ -291,159 +298,3 @@ class TestExchangeRateRevaluation(ERPNextTestSuite):
for key, _val in expected_data.items():
self.assertEqual(expected_data.get(key), account_details.get(key))
@ERPNextTestSuite.change_settings(
"Accounts Settings",
{"allow_multi_currency_invoices_against_single_party_account": 1, "allow_stale": 0},
)
def test_05_revaluation_journal_reversal(self):
"""
Test reversing of revaluation journals
"""
si = create_sales_invoice(
item=self.item,
company=self.company,
customer="_Test Customer 1",
debit_to=self.debtors_usd,
posting_date=today(),
parent_cost_center=self.cost_center,
cost_center=self.cost_center,
rate=100,
price_list_rate=100,
do_not_submit=1,
)
si.currency = "USD"
si.conversion_rate = 80
si.save().submit()
err = frappe.new_doc("Exchange Rate Revaluation")
err.company = self.company
err.posting_date = today()
err.fetch_and_calculate_accounts_data()
self.assertEqual(len(err.accounts), 1)
err.save().submit()
gain_loss_account = err.get_for_unrealized_gain_loss_account()
usd_account = err.accounts[0].account
old_balance = err.accounts[0].balance_in_base_currency
new_balance = err.accounts[0].new_balance_in_base_currency
total_gain_loss = err.total_gain_loss
# Create JV for ERR
ret = err.check_journal_and_reversal()
self.assertFalse(ret.get("journals_posted"))
err_journals = err.make_jv_entries()
je = frappe.get_doc("Journal Entry", err_journals.get("revaluation_jv"))
je = je.submit()
je.reload()
self.assertEqual(je.voucher_type, "Exchange Rate Revaluation")
self.assertEqual(len(je.accounts), 3)
# A gain is credited to the gain/loss account, a loss is debited. The current
# exchange rate (from master data) may sit either side of the booked rate, so
# derive the column from the sign instead of assuming a gain.
gain_loss_debit = abs(total_gain_loss) if total_gain_loss < 0 else 0.0
gain_loss_credit = total_gain_loss if total_gain_loss > 0 else 0.0
expected = [
(usd_account, new_balance, 0.0, 100.0, 0.0),
(usd_account, 0.0, old_balance, 0.0, 100.0),
(gain_loss_account, gain_loss_debit, gain_loss_credit, gain_loss_debit, gain_loss_credit),
]
actual = []
for acc in je.accounts:
actual.append(
(
acc.account,
acc.debit,
acc.credit,
acc.debit_in_account_currency,
acc.credit_in_account_currency,
)
)
self.assertEqual(expected, actual)
# Assert reversals are not posted
ret = err.check_journal_and_reversal()
self.assertTrue(ret.get("journals_posted"))
self.assertFalse(ret.get("reversals_posted"))
err.make_reverse_journal()
# submit
draft = frappe.db.get_all(
"Journal Entry",
filters={"docstatus": 0, "reversal_of": je.name, "voucher_type": "Exchange Rate Revaluation"},
pluck="name",
as_list=1,
)
self.assertIsNotNone(draft)
frappe.get_doc("Journal Entry", draft[0]).submit()
ret = err.check_journal_and_reversal()
self.assertTrue(ret.get("journals_posted"))
self.assertTrue(ret.get("reversals_posted"))
reverse_jv = frappe.db.get_all(
"Journal Entry", filters={"reversal_of": err_journals.get("revaluation_jv")}, pluck="name"
)
self.assertIsNotNone(reverse_jv)
class TestExchangeRateRevaluationValidation(ERPNextTestSuite):
"""Validation and gain/loss calculation paths, exercised on the document directly
so they don't need the multi-currency GL setup the integration tests above build."""
def setUp(self):
frappe.set_user("Administrator")
self.company = "_Test Company"
def _revaluation_with_rows(self, rows, rounding_loss_allowance=0.05):
doc = frappe.new_doc("Exchange Rate Revaluation")
doc.company = self.company
doc.posting_date = today()
doc.rounding_loss_allowance = rounding_loss_allowance
for row in rows:
doc.append("accounts", row)
return doc
def test_rounding_loss_allowance_must_be_between_0_and_1(self):
for bad in (-0.1, 1, 1.5):
doc = self._revaluation_with_rows([], rounding_loss_allowance=bad)
self.assertRaises(frappe.ValidationError, doc.validate)
# values inside [0, 1) are accepted, at the lower bound and mid-range
for good in (0.0, 0.5):
self._revaluation_with_rows([], rounding_loss_allowance=good).validate()
def test_gain_loss_computed_and_split_by_zero_balance(self):
doc = self._revaluation_with_rows(
[
# open (unbooked) row: base balance moved 1000 -> 1100, a 100 gain
{"zero_balance": 0, "balance_in_base_currency": 1000, "new_balance_in_base_currency": 1100},
# already-settled (zero_balance) row carries a booked loss of 40
{"zero_balance": 1, "gain_loss": -40},
]
)
doc.validate()
# gain_loss is derived only for open rows; the zero-balance row keeps its value
self.assertEqual(doc.accounts[0].gain_loss, 100)
self.assertEqual(doc.gain_loss_unbooked, 100)
self.assertEqual(doc.gain_loss_booked, -40)
self.assertEqual(doc.total_gain_loss, 60)
def test_before_submit_drops_rows_without_gain_loss(self):
doc = self._revaluation_with_rows(
[
{"zero_balance": 0, "balance_in_base_currency": 1000, "new_balance_in_base_currency": 1100},
{"zero_balance": 0, "balance_in_base_currency": 500, "new_balance_in_base_currency": 500},
]
)
doc.validate() # second row nets to a 0 gain_loss
doc.remove_accounts_without_gain_loss()
self.assertEqual(len(doc.accounts), 1)
self.assertEqual(doc.accounts[0].gain_loss, 100)
def test_before_submit_requires_at_least_one_gain_loss_row(self):
doc = self._revaluation_with_rows(
[{"zero_balance": 0, "balance_in_base_currency": 500, "new_balance_in_base_currency": 500}]
)
doc.validate()
self.assertRaises(frappe.ValidationError, doc.remove_accounts_without_gain_loss)

View File

@@ -12,9 +12,8 @@ from typing import Any, Union
import frappe
from frappe import _
from frappe.database.operator_map import OPERATOR_MAP
from frappe.model import numeric_fieldtypes
from frappe.query_builder import Case
from frappe.query_builder.functions import Cast_, Sum
from frappe.query_builder.functions import Sum
from frappe.utils import cstr, date_diff, flt, getdate
from frappe.utils.xlsxutils import XLSXMetadata, XLSXStyleBuilder
from pypika.terms import Bracket, LiteralValue
@@ -255,27 +254,16 @@ class FinancialReportEngine:
if filters.get("presentation_currency"):
frappe.msgprint(
title=_("Unsupported Feature"),
msg=_("Currency filters are currently unsupported in Custom Financial Report."),
indicator="orange",
title=_("Not Supported"),
msg=_("Currency filters are currently unsupported in Custom Financial Report"),
)
# Margin view is dependent on first row being an income account. Hence not supported.
# Way to implement this would be using calculated rows with formulas.
supported_views = ("Report", "Growth")
if (view := filters.get("selected_view")) and view not in supported_views:
frappe.msgprint(
indicator="orange",
title=_("Not Supported"),
msg=_("{0} view is currently unsupported in Custom Financial Report").format(view),
)
if filters.get("group_by_dimension"):
frappe.msgprint(
indicator="orange",
title=_("Not Supported"),
msg=_("Dimension-based grouping is currently unsupported in Custom Financial Report"),
)
frappe.msgprint(_("{0} view is currently unsupported in Custom Financial Report.").format(view))
def _initialize_context(self, filters: dict[str, Any]) -> ReportContext:
template_name = filters.get("report_template")
@@ -876,15 +864,8 @@ class FilterExpressionParser:
field = getattr(table, field_name, None)
operator_fn = OPERATOR_MAP.get(operator.casefold())
if "like" in operator.casefold():
if "%" not in value:
value = f"%{value}%"
# Postgres has no LIKE/ILIKE operator for non-text columns; MariaDB implicitly casts
# the numeric column to text. Cast a numeric/Check Account field to varchar so the
# match runs on both engines and reproduces MariaDB's result.
meta_field = frappe.get_meta("Account").get_field(field_name)
if meta_field and meta_field.fieldtype in numeric_fieldtypes:
field = Cast_(field, "varchar")
if "like" in operator.casefold() and "%" not in value:
value = f"%{value}%"
return operator_fn(field, value)
@@ -1043,7 +1024,8 @@ class FormulaFieldUpdater:
def get_filtered_accounts(company: str, account_rows: str | list):
frappe.has_permission("Financial Report Template", ptype="read", throw=True)
account_rows = [frappe._dict(row) for row in frappe.parse_json(account_rows)]
if isinstance(account_rows, str):
account_rows = json.loads(account_rows, object_hook=frappe._dict)
return DataCollector.get_filtered_accounts(company, account_rows)
@@ -1871,51 +1853,28 @@ class GrowthViewTransformer:
self.formatted_rows = context.raw_data.get("formatted_data", [])
self.period_list = context.period_list
def transform(self):
def transform(self) -> None:
for row_data in self.formatted_rows:
if row_data.get("is_blank_line"):
continue
if row_data.get("segment_values"):
self._transform_segmented_row(row_data)
else:
self._transform_single_row(row_data)
transformed_values = {}
for i in range(len(self.period_list)):
current_period = self.period_list[i]["key"]
def _compute_growth_values(self, source: dict) -> dict:
transformed = {}
current_value = row_data[current_period]
previous_value = row_data[self.period_list[i - 1]["key"]] if i != 0 else 0
for i, period in enumerate(self.period_list):
current_period = period["key"]
current_value = source.get(current_period)
if i == 0:
transformed_values[current_period] = current_value
else:
growth_percent = self._calculate_growth(previous_value, current_value)
transformed_values[current_period] = growth_percent
if current_value in (None, ""):
continue
if i == 0:
transformed[current_period] = current_value
else:
previous_period = self.period_list[i - 1]["key"]
previous_value = source.get(previous_period) or 0
transformed[current_period] = self._calculate_growth(previous_value, current_value)
return transformed
def _transform_single_row(self, row_data: dict):
row_data.update(self._compute_growth_values(row_data))
def _transform_segmented_row(self, row_data: dict):
for seg_id, seg_data in row_data.get("segment_values", {}).items():
if seg_data.get("is_blank_line"):
continue
transformed = self._compute_growth_values(seg_data)
seg_data.update(transformed)
for period_key, value in transformed.items():
row_data[f"{seg_id}_{period_key}"] = value
row_data.update(transformed_values)
def _calculate_growth(self, previous_value: float, current_value: float) -> float | None:
if current_value in (None, ""):
if current_value is None:
return None
if previous_value == 0 and current_value > 0:

View File

@@ -72,8 +72,10 @@ class FiscalYear(Document):
if existing_fiscal_years:
for existing in existing_fiscal_years:
company_for_existing = frappe.get_all(
"Fiscal Year Company", filters={"parent": existing.name}, pluck="company"
company_for_existing = frappe.db.sql_list(
"""select company from `tabFiscal Year Company`
where parent=%s""",
existing.name,
)
overlap = False
@@ -107,9 +109,6 @@ def auto_create_fiscal_year():
)
for d in fiscal_year:
# savepoint so a duplicate-year INSERT (Fiscal Year autoname=field:year) that aborts the
# statement doesn't poison the whole scheduler transaction on Postgres and kill the next iteration
frappe.db.savepoint("auto_create_fiscal_year")
try:
current_fy = frappe.get_doc("Fiscal Year", d[0])
@@ -130,7 +129,7 @@ def auto_create_fiscal_year():
new_fy.insert(ignore_permissions=True)
except frappe.NameError:
frappe.db.rollback(save_point="auto_create_fiscal_year")
pass
def get_from_and_to_date(fiscal_year):

View File

@@ -7,7 +7,6 @@ from frappe import _
from frappe.model.document import Document
from frappe.model.meta import get_field_precision
from frappe.model.naming import set_name_from_naming_options
from frappe.query_builder.functions import Sum
from frappe.utils import create_batch, flt, fmt_money, now
import erpnext
@@ -332,12 +331,10 @@ def validate_balance_type(account, adv_adj=False):
if not adv_adj and account:
balance_must_be = frappe.get_cached_value("Account", account, "balance_must_be")
if balance_must_be:
gle = frappe.qb.DocType("GL Entry")
balance = (
frappe.qb.from_(gle)
.select(Sum(gle.debit) - Sum(gle.credit))
.where((gle.is_cancelled == 0) & (gle.account == account))
.run()
balance = frappe.db.sql(
"""select sum(debit) - sum(credit)
from `tabGL Entry` where is_cancelled = 0 and account = %s""",
account,
)[0][0]
if (balance_must_be == "Debit" and flt(balance) < 0) or (
@@ -351,48 +348,44 @@ def validate_balance_type(account, adv_adj=False):
def update_outstanding_amt(
account, party_type, party, against_voucher_type, against_voucher, on_cancel=False
):
gle = frappe.qb.DocType("GL Entry")
conditions = (
(gle.against_voucher_type == against_voucher_type)
& (gle.against_voucher == against_voucher)
& (gle.voucher_type != "Invoice Discounting")
)
if party_type and party:
conditions &= (gle.party_type == party_type) & (gle.party == party)
party_condition = " and party_type={} and party={}".format(
frappe.db.escape(party_type), frappe.db.escape(party)
)
else:
party_condition = ""
if against_voucher_type == "Sales Invoice":
party_account = frappe.get_cached_value(against_voucher_type, against_voucher, "debit_to")
conditions &= gle.account.isin([account, party_account])
account_condition = f"and account in ({frappe.db.escape(account)}, {frappe.db.escape(party_account)})"
else:
conditions &= gle.account == account
account_condition = f" and account = {frappe.db.escape(account)}"
# get final outstanding amt
bal = flt(
frappe.qb.from_(gle)
.select(Sum(gle.debit_in_account_currency) - Sum(gle.credit_in_account_currency))
.where(conditions)
.run()[0][0]
frappe.db.sql(
f"""
select sum(debit_in_account_currency) - sum(credit_in_account_currency)
from `tabGL Entry`
where against_voucher_type=%s and against_voucher=%s
and voucher_type != 'Invoice Discounting'
{party_condition} {account_condition}""",
(against_voucher_type, against_voucher),
)[0][0]
or 0.0
)
if against_voucher_type == "Purchase Invoice":
bal = -bal
elif against_voucher_type == "Journal Entry":
je_conditions = (
(gle.voucher_type == "Journal Entry")
& (gle.voucher_no == against_voucher)
& (gle.account == account)
& (gle.against_voucher.isnull() | (gle.against_voucher == ""))
)
if party_type and party:
je_conditions &= (gle.party_type == party_type) & (gle.party == party)
against_voucher_amount = flt(
frappe.qb.from_(gle)
.select(Sum(gle.debit_in_account_currency) - Sum(gle.credit_in_account_currency))
.where(je_conditions)
.run()[0][0]
frappe.db.sql(
f"""
select sum(debit_in_account_currency) - sum(credit_in_account_currency)
from `tabGL Entry` where voucher_type = 'Journal Entry' and voucher_no = %s
and account = %s and (against_voucher is null or against_voucher='') {party_condition}""",
(against_voucher, account),
)[0][0]
)
if not against_voucher_amount:
@@ -471,25 +464,6 @@ def on_doctype_update():
frappe.db.add_index("GL Entry", ["posting_date", "company"])
frappe.db.add_index("GL Entry", ["party_type", "party"])
if frappe.db.db_type == "postgres":
# Postgres-only partial/covering indexes for the financial reports (General Ledger, Trial
# Balance, Balance Sheet, P&L), which always filter `is_cancelled = 0` and scope by company.
# `where`/`include` are no-ops on MariaDB and its optimizer ignores these anyway, so they are
# added only on postgres to avoid dead write overhead on this insert-hot table.
frappe.db.add_index(
"GL Entry",
["company", "posting_date", "account"],
index_name="gle_active_detail",
where="is_cancelled = 0",
)
frappe.db.add_index(
"GL Entry",
["company", "account", "posting_date"],
index_name="gle_active_cover",
where="is_cancelled = 0",
include=["debit", "credit"],
)
def rename_gle_sle_docs():
for doctype in ["GL Entry", "Stock Ledger Entry"]:
@@ -506,14 +480,10 @@ def rename_temporarily_named_docs(doctype):
oldname = doc.name
set_name_from_naming_options(autoname, doc)
newname = doc.name
dt = frappe.qb.DocType(doctype)
(
frappe.qb.update(dt)
.set(dt.name, newname)
.set(dt.to_rename, 0)
.set(dt.modified, now())
.where(dt.name == oldname)
).run()
frappe.db.sql(
f"UPDATE `tab{doctype}` SET name = %s, to_rename = 0, modified = %s where name = %s",
(newname, now(), oldname),
)
for hook_type in ("on_gle_rename", "on_sle_rename"):
for hook in frappe.get_hooks(hook_type):

View File

@@ -26,17 +26,12 @@ class TestGLEntry(ERPNextTestSuite):
jv.flags.ignore_validate = True
jv.submit()
round_off_entry = frappe.get_all(
"GL Entry",
filters={
"voucher_type": "Journal Entry",
"voucher_no": jv.name,
"account": "_Test Write Off - _TC",
"cost_center": "_Test Cost Center - _TC",
"debit": 0,
"credit": 0.01,
},
pluck="name",
round_off_entry = frappe.db.sql(
"""select name from `tabGL Entry`
where voucher_type='Journal Entry' and voucher_no = %s
and account='_Test Write Off - _TC' and cost_center='_Test Cost Center - _TC'
and debit = 0 and credit = '.01'""",
jv.name,
)
self.assertTrue(round_off_entry)
@@ -60,9 +55,8 @@ class TestGLEntry(ERPNextTestSuite):
)
self.assertTrue(all(entry.to_rename == 1 for entry in gl_entries))
series = frappe.qb.DocType("Series")
old_naming_series_current_value = (
frappe.qb.from_(series).select(series["current"]).where(series.name == naming_series).run()
old_naming_series_current_value = frappe.db.sql(
"SELECT current from tabSeries where name = %s", naming_series
)[0][0]
rename_gle_sle_docs()
@@ -79,8 +73,8 @@ class TestGLEntry(ERPNextTestSuite):
all(new.name != old.name for new, old in zip(gl_entries, new_gl_entries, strict=False))
)
new_naming_series_current_value = (
frappe.qb.from_(series).select(series["current"]).where(series.name == naming_series).run()
new_naming_series_current_value = frappe.db.sql(
"SELECT current from tabSeries where name = %s", naming_series
)[0][0]
self.assertEqual(old_naming_series_current_value + 2, new_naming_series_current_value)

View File

@@ -317,50 +317,58 @@ class InvoiceDiscounting(AccountsController):
@frappe.whitelist()
def get_invoices(filters: str | dict):
filters = frappe._dict(frappe.parse_json(filters))
si = frappe.qb.DocType("Sales Invoice")
di = frappe.qb.DocType("Discounted Invoice")
discounted = frappe.qb.from_(di).select(di.sales_invoice).where(di.docstatus == 1)
query = (
frappe.qb.from_(si)
.select(
si.name.as_("sales_invoice"),
si.customer,
si.posting_date,
si.outstanding_amount,
si.debit_to,
)
.where((si.docstatus == 1) & (si.outstanding_amount > 0) & si.name.notin(discounted))
)
def get_invoices(filters: str):
filters = frappe._dict(json.loads(filters))
cond = []
if filters.customer:
query = query.where(si.customer == filters.customer)
cond.append("customer=%(customer)s")
if filters.from_date:
query = query.where(si.posting_date >= filters.from_date)
cond.append("posting_date >= %(from_date)s")
if filters.to_date:
query = query.where(si.posting_date <= filters.to_date)
cond.append("posting_date <= %(to_date)s")
if filters.min_amount:
query = query.where(si.base_grand_total >= filters.min_amount)
cond.append("base_grand_total >= %(min_amount)s")
if filters.max_amount:
query = query.where(si.base_grand_total <= filters.max_amount)
cond.append("base_grand_total <= %(max_amount)s")
return query.run(as_dict=1)
where_condition = ""
if cond:
where_condition += " and " + " and ".join(cond)
return frappe.db.sql(
"""
select
name as sales_invoice,
customer,
posting_date,
outstanding_amount,
debit_to
from `tabSales Invoice` si
where
docstatus = 1
and outstanding_amount > 0
%s
and not exists(select di.name from `tabDiscounted Invoice` di
where di.docstatus=1 and di.sales_invoice=si.name)
"""
% where_condition,
filters,
as_dict=1,
)
def get_party_account_based_on_invoice_discounting(sales_invoice):
party_account = None
par = frappe.qb.DocType("Invoice Discounting")
ch = frappe.qb.DocType("Discounted Invoice")
invoice_discounting = (
frappe.qb.from_(par)
.inner_join(ch)
.on(par.name == ch.parent)
.select(par.accounts_receivable_discounted, par.accounts_receivable_unpaid, par.status)
.where((par.docstatus == 1) & (ch.sales_invoice == sales_invoice))
.run(as_dict=1)
invoice_discounting = frappe.db.sql(
"""
select par.accounts_receivable_discounted, par.accounts_receivable_unpaid, par.status
from `tabInvoice Discounting` par, `tabDiscounted Invoice` ch
where par.name=ch.parent
and par.docstatus=1
and ch.sales_invoice = %s
""",
(sales_invoice),
as_dict=1,
)
if invoice_discounting:
if invoice_discounting[0].status == "Disbursed":

View File

@@ -1,62 +1,8 @@
# Copyright (c) 2026, Frappe Technologies Pvt. Ltd. and Contributors
# Copyright (c) 2018, Frappe Technologies Pvt. Ltd. and Contributors
# See license.txt
import frappe
from erpnext.tests.utils import ERPNextTestSuite
COMPANY = "_Test Company"
TAX_ACCOUNT = "_Test Account VAT - _TC"
RECEIVABLE_ACCOUNT = "Debtors - _TC"
class TestItemTaxTemplate(ERPNextTestSuite):
"""Item Tax Template validates its tax rows: each account must belong to the
company, be a tax-like account type, and appear only once."""
def setUp(self):
frappe.set_user("Administrator")
def make_template(self, rows, title="_Test ITT"):
doc = frappe.new_doc("Item Tax Template")
doc.title = f"{title} {frappe.generate_hash(length=6)}"
doc.company = COMPANY
for account, rate, not_applicable in rows:
doc.append(
"taxes",
{"tax_type": account, "tax_rate": rate, "not_applicable": not_applicable},
)
return doc
def test_valid_template_saves_and_is_named_with_abbr(self):
doc = self.make_template([(TAX_ACCOUNT, 9, 0)])
doc.insert()
self.assertTrue(doc.name.endswith(" - _TC"))
self.assertTrue(doc.name.startswith(doc.title))
def test_duplicate_tax_type_throws(self):
doc = self.make_template([(TAX_ACCOUNT, 9, 0), (TAX_ACCOUNT, 5, 0)])
self.assertRaises(frappe.ValidationError, doc.insert)
def test_account_of_wrong_company_throws(self):
other_account = frappe.db.get_value("Account", {"company": "_Test Company 1", "is_group": 0}, "name")
self.assertTrue(other_account, "need a non-group account in _Test Company 1")
doc = self.make_template([(other_account, 9, 0)])
self.assertRaises(frappe.ValidationError, doc.insert)
def test_disallowed_account_type_throws(self):
# a Receivable account is not Tax/Chargeable/Income/Expense
doc = self.make_template([(RECEIVABLE_ACCOUNT, 9, 0)])
self.assertRaises(frappe.ValidationError, doc.insert)
def test_not_applicable_row_has_rate_zeroed(self):
doc = self.make_template([(TAX_ACCOUNT, 18, 1)])
doc.insert()
self.assertEqual(doc.taxes[0].tax_rate, 0)
def test_negative_tax_rate_is_accepted(self):
# SUSPECTED BUG: validate never bounds tax_rate, so a negative (or >100) rate
# saves silently. Locking the current (wrong) behaviour.
doc = self.make_template([(TAX_ACCOUNT, -5, 0)])
doc.insert()
self.assertEqual(doc.taxes[0].tax_rate, -5)
pass

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More