Compare commits

..

1 Commits

Author SHA1 Message Date
Ankush Menat
265bc4eb6f fix: Add authorization checks on internal functions 2026-06-08 14:49:23 +05:30
1614 changed files with 349441 additions and 945897 deletions

View File

@@ -1,259 +0,0 @@
# PostgreSQL compatibility — review guide
ERPNext targets **both MariaDB and PostgreSQL from a single codebase**. The full server
test suite passes on both, but the PostgreSQL CI job is **label-gated** (it does not run on
every PR), so until it is required this guide is the always-on guard. Greptile loads it as
review context (`.greptile/config.json`).
When reviewing a PR, flag any **new or changed query** (raw `frappe.db.sql`, `frappe.qb`,
`frappe.get_all/get_list/get_value`, report SQL) that would **error on PostgreSQL** or
**return different results on the two engines**.
## The one rule that governs everything
**MariaDB behaviour must not change; PostgreSQL is brought into line with MariaDB — never the
reverse.** A "fix" that changes the value, row count, or ordering MariaDB produced is a
regression, even if the new behaviour looks more correct. The only accepted MariaDB-output
change is replacing a genuinely *undefined/arbitrary* result with a deterministic one (row
count preserved) — and that should be called out explicitly.
There are two failure modes to watch for:
1. **Hard breaks** — PostgreSQL raises an exception; MariaDB is green. Easy to catch in CI,
but the gated job may not run.
2. **Silent divergences** — both engines succeed but return *different* results. CI on one
engine stays green; the bug only shows on a PostgreSQL site. These are the dangerous ones.
---
## 1. Hard breaks — would error on PostgreSQL
Flag a changed query that uses any of these:
- **Loose `GROUP BY`** — selecting/ordering a column that is neither in `GROUP BY` nor wrapped
in an aggregate. MariaDB tolerates it; PostgreSQL errors (`must appear in the GROUP BY
clause or be used in an aggregate function`). This **also covers an aggregate (`Sum`/`Count`/…)
selected alongside bare columns with NO `.groupby()` at all** — MariaDB silently collapses
every row into one arbitrary-valued row (often a *wrong-output* bug there too), PostgreSQL
errors. Fix: add the bare column to `GROUP BY` **if it is functionally dependent on the group
key**, otherwise wrap it in `Max()`/`Min()`. **See §3 — the row-count trap — before suggesting
"add it to GROUP BY".**
- **MySQL-only functions** — `TIMESTAMP(date,time)`, `TIMEDIFF`, `STR_TO_DATE`, `DATE_FORMAT`,
`DATE_ADD/SUB`, `GROUP_CONCAT`, `PERIOD_DIFF`, SQL `IF(cond,a,b)`. Use the portable
`frappe.query_builder.functions` equivalents (`CombineDatetime`, `DateDiff`, `Case`,
`GroupConcat`, …) or a precomputed column (e.g. `posting_datetime`).
- **`UPDATE … JOIN`** — not valid on PostgreSQL. Rewrite as `UPDATE … WHERE name IN (subquery)`.
- **`HAVING` referencing a `SELECT` alias** — PostgreSQL rejects output-column aliases in
`HAVING` (regardless of whether the query has a `GROUP BY`; MariaDB allows them). Repeat the
underlying expression in `HAVING`, or move a non-aggregate predicate into `WHERE`.
- **`SELECT DISTINCT … ORDER BY <expr not in the select list>`** — add the expr to the select
**only if it is single-valued per distinct row**; otherwise it grows the `DISTINCT` key and the
MariaDB row count (see §3) — drop the SQL `ORDER BY` and sort in Python instead.
- **Single-quoted column alias** `AS 'x'` — PostgreSQL reads `'x'` as a string literal. Use an
unquoted (or double-quoted) alias.
- **`varchar | varchar`** (bitwise OR misused as a coalesce) — errors on PostgreSQL. Use
`Coalesce(...)`.
- **Capital-cased identifiers** used as column/field names in `get_value(dt, dn, "Status")`,
`get_all(dt, fields=["Account"])`, and similar — PostgreSQL quotes the identifier and matches
it case-sensitively; a stored column named `status`/`account` won't match `"Status"`/`"Account"`
(`column "Account" does not exist`). Use the exact stored (lower-case) fieldname.
- **Boolean passed where an integer column is expected** — `frappe.db.set_value(dt, dn,
check_field, True)`, `doc.db_set(field, False)`, or `frappe.qb.update(dt).set(check_field, True)`
emit `SET col = true`, which PostgreSQL rejects on a `smallint`/`Check` column
(`column is of type smallint but expression is of type boolean`). Pass `1`/`0`.
- **A direct `.like()`/`.ilike()` on a pypika field (or raw `LIKE`) on a NON-text column** — `idx`,
`docstatus`, a date, etc. frappe maps `.like()` → `ILIKE`, and PostgreSQL has no `bigint ILIKE text`
operator (`operator does not exist: bigint ~~* unknown`). Cast the column to text first —
**`Cast_(col, "varchar")`**, not `Cast(col, "char")` (see below). MariaDB coerces the int
implicitly, so the cast is a no-op there. A `["like", …]` filter passed to `get_all`/`get_list`/
`qb.get_query`/`reportview` needs no cast: the framework casts non-text fields itself
(frappe/frappe#42449).
- **`CAST(… AS CHAR)` / `Cast(x, "char")`** — on PostgreSQL bare `CHAR` is `character(1)`, so
`CAST(12 AS CHAR)` → `'1'` (silently truncates multi-digit values); MariaDB gives the full string.
Use `VARCHAR` / `Cast_(x, "varchar")`.
- **`.rlike()` / raw `RLIKE`** — frappe rewrites `REGEXP` → `~*` on PostgreSQL but does **not**
translate `RLIKE` (no such PostgreSQL operator). Use `.regexp()` (or `.like()` for a simple prefix).
- **`IfNull`/`Coalesce` of a typed column with a different-typed literal** — `IfNull(asset.disposal_date, 0)`
renders `COALESCE("disposal_date", 0)`, coalescing a **DATE** with an **integer**. PostgreSQL requires
`COALESCE` args to share a type (`DatatypeMismatch: COALESCE types date and integer cannot be matched`);
MariaDB's `IFNULL` is permissive. The common shape is `IfNull(date_col, 0) != 0 / == 0` as a presence test —
replace with `date_col.isnotnull()` / `date_col.isnull()` (identical, and valid on both). Otherwise coalesce
to a **same-type** default (`Coalesce(date_col, '1900-01-01')`, `Coalesce(text_col, '')`).
- **Division by a possibly-zero divisor** — `Sum(a) / Sum(b)`, `x / col`, etc. where the
divisor can be `0`/empty. MariaDB returns `NULL` for division by zero; PostgreSQL raises
`division by zero` and aborts the query. Wrap the divisor in `NullIf(divisor, 0)` — that
yields `NULL` on both engines, matching MariaDB's value. (Only the *literal* `/ 0` is a parse
constant; the trap is a divisor that is an aggregate or column the data can drive to zero.)
---
## 2. Silent divergences — succeeds on both, returns different results
These don't error, so a one-engine CI stays green. Flag them:
- **Case sensitivity on text equality** — `==`, `.isin()`, `Strpos`/`Locate` on free-text
columns are case-**sensitive** on PostgreSQL but case-**insensitive** under MariaDB's default
collation. `Lower()` both sides. *(Not `.like()`/`["like", …]` — those already render as
`ILIKE` on PostgreSQL; see §4.)*
- **Case sensitivity in a doc-`name` lookup** — lower-casing a value then using it as a
document name in `get_value`/`get_doc`/`exists` misses on PostgreSQL (names are
case-sensitive). Keep original case for the identifier; lower-case only comparison operands.
- **Empty string vs NULL** — PostgreSQL stores a blank link/data field as `NULL` on some paths
while MariaDB keeps `''`; `Concat`/`Concat_ws` then diverge. Prefer the stored full value, or
`Coalesce(col, '')` per argument.
- **NULL ordering** — MariaDB sorts `NULL` first, PostgreSQL sorts it last. For
`ORDER BY … LIMIT 1`/`[0]` on a nullable column, guard with `Coalesce`/`isnotnull()`.
- **`ORDER BY … LIMIT 1` with no unique tiebreaker** — when rows tie on the ordered column the
two engines may pick different rows. Add a `creation`/`name` tiebreaker **only if it does not
change MariaDB's current pick** (see §4).
- **Integer division** — `int / int` truncates on PostgreSQL but is decimal on MariaDB, e.g.
`COUNT(...) / COUNT(...) * 100` → `0`, or `manufacturing_time_in_mins / 1440` flooring a
lead-time to whole days. Force float: multiply by `100.0`, or make a literal a float
(`/ 1440` → `/ 1440.0`), or cast an operand. (Only SQL-level `/` on integer **columns/literals**
— Python `/` is already float.)
- **`DISTINCT` list ordering** — `frappe.get_all(distinct=True, order_by=…)` /
`SELECT DISTINCT … ORDER BY`: frappe's `db_query` **silently drops `ORDER BY` for distinct
queries on PostgreSQL**, so the result is unordered there. Sort in Python instead — and use
`key=str.casefold`, because bare `sorted()` is case-sensitive (ASCII) while MariaDB's
collation is case-insensitive, so a plain sort reorders MariaDB's output.
- **Engine-specific function rewrites** — e.g. a PostgreSQL `regexp_replace` branch
reimplementing MariaDB's `CAST(SUBSTRING_INDEX(name,' ',-1) AS UNSIGNED)` (leading digits of
the last whitespace token). Verify the rewrite matches MariaDB on edge cases (`"X - 3a"→3`,
`"X - 1.5"→1`) by diffing both engines on literal rows.
- **`UnixTimestamp(date)` / date→epoch** is timezone-dependent (midnight in the DB session TZ),
so a strict `epoch <= now` bound is flaky on PostgreSQL.
---
## 3. The row-count trap — `GROUP BY` **and** `DISTINCT` (the single most important rule)
When making a loose `GROUP BY` PostgreSQL-valid, **do not add a non-functionally-dependent
column to the `GROUP BY` just to satisfy PostgreSQL** — that turns one group row into N and
**changes the MariaDB row count** (a regression). The classic traps are adding the **child/row
primary key** or an **editable per-row field**. Instead **`Max()`/`Min()`-wrap** the offending
column: the row count is preserved and the value goes from arbitrary (MariaDB's old loose pick)
to deterministic.
**Judge functional dependence by the source table, not the column name:**
- A column from a **master joined on the group key** (`t3.x` where `t1.key = t3.name`) is FD →
safe to keep in `GROUP BY`.
- A descriptive field on the **transaction** table (`t1.supplier_name`, `t1.territory`,
`t1.item_name` — fetched/editable, can differ across historical rows for the same key) is
**not** FD even though it looks master-derived → `Max()`-wrap it.
Conversely, do **not** suggest changing a `Max()`/`Min()`-wrapped column to `Sum()` (or vice
versa) to make a number "more correct" — that changes the MariaDB value. The wrap reproduces
MariaDB's prior one-value-per-group output; a different aggregate is a product change, out of
scope for a portability fix.
**The same trap applies to `SELECT DISTINCT`.** To satisfy PostgreSQL's "an `ORDER BY` expr must
appear in the select list under `DISTINCT`" rule, **do not blindly add the ordered column to the
select** — if it is not single-valued per existing distinct row, the `DISTINCT` key grows and
MariaDB returns **more rows** (a regression), exactly as adding a non-FD column to `GROUP BY` does.
Add it only when it is functionally dependent on the existing select columns; otherwise drop the
SQL `ORDER BY` and **sort in Python** (`key=str.casefold`, per §2) so the distinct row set is
unchanged.
### 3.1 Second-order traps — when the `Max()`/`Min()` wrap itself is the bug
The wrap is only a no-op when the column is provably single-valued per group (**"`Max()` means
provably constant"**). When the column can genuinely vary, the wrap is a decision, and a full
audit of these fixes found four recurring mistakes:
- **Incoherent pair** — two semantically-coupled columns (a flag + a link:
`is_phantom_item` + `bom_no`; a discriminator + its value) aggregated with *independent*
`Max()`/`Min()` can pair values from **different rows** — a chimera row that never existed.
MariaDB's loose pick was at least row-coherent. Fix: group by the pair (when consumers
tolerate the extra rows), or select one **representative row** (`Min(child.name)` subquery +
join-back) so every column comes from the same line.
- **NULL-skipping** — `MAX`/`MIN` ignore NULLs, so `Max()` over a mostly-NULL discriminator
(an `original_item`-style column) *deterministically* returns the non-NULL value where
MariaDB could return NULL — deterministically wrong where the old behavior was only
intermittently wrong. Flag it wherever "no value" is a meaningful state (fallback gates,
dict keys).
- **Fabricated arithmetic** — `Sum(x) * Max(y)` where `y` varies within the group invents a
number no row ever had (and `Max` biases it upward) — poisonous when it feeds validation,
budgets, valuation, or GL/stock values. Fix per-row: `Sum(x * y)`.
- **Collation-dependent pick (text columns)** — `Max()`/`Min()` over text is a *sort*, and the two
engines sort text differently: MariaDB's `utf8mb4` collations fold case, PostgreSQL (as CI runs
it) orders by byte value. `MAX('abc', 'ABD')` is `ABD` on MariaDB and `abc` on PostgreSQL. So a
`Max()` over a text column that varies **in case** within its group is a live P2 divergence, not
the arbitrary-pick preservation the wrap is usually justified as. Confirmed on CI; see #56241.
Note a local macOS PostgreSQL gives a **false all-clear** — its collation happens to agree with
MariaDB on case. Fix: take a representative row rather than sorting text.
- **Wrong bound** — where the value has a semantic, pick the bound deliberately:
`Min(schedule_date)` for a "required by", `Min(idx)` for first-line ordering, a qty-weighted
average for a rate. A blind `Max` can understate urgency or overstate a figure.
Review heuristic: **if choosing between `Max` and `Min` would change the answer, the column is
not functionally dependent** — wrapping either is the wrong fix. Group by it, restructure, or
pick a bound for a stated reason, and cover the varying-group case with a test.
---
## 4. False positives — do NOT flag these
These are auto-handled by the framework and are **not** breaks:
- **`.like()` / `["like", …]`** already renders as `ILIKE` on PostgreSQL — not a
case-sensitivity bug. A `["like", …]` filter on a **non-text** field is also cast to text by
the framework. *(Exception: a direct `.like()` on a **non-text** pypika field — `idx`,
`docstatus` — is a hard break, `bigint ILIKE`; see §1.)*
- **Raw `ifnull(...)`** inside `frappe.db.sql()` is rewritten to `coalesce(...)` on all engines.
- **Backticks**, **`LOCATE`**, **`REGEXP`** / **`.regexp()`** in raw SQL are auto-translated on
PostgreSQL (`REGEXP` → `~*`). **But `RLIKE` / `.rlike()` is NOT translated** — that one is a
hard break (see §1).
- **An `ORDER BY … LIMIT 1` tie where the two engines already agree**, or where adding a
tiebreaker would *change* MariaDB's current pick — leave it; "fixing" it would either change
MariaDB or has no observable effect.
---
## 5. Transaction / runtime (not query-shape, still PostgreSQL-only)
- **Catch-and-continue inserts** — on PostgreSQL a failed `insert()` aborts the **whole
transaction**, so code that swallows a duplicate and keeps going dies on the next statement
with `InFailedSqlTransaction` (frappe dropped its blanket per-statement savepoint in
frappe#40075). Such a handler must wrap the fallible insert in `frappe.db.savepoint(name)` +
`rollback(save_point=name)` — unless it re-`throw`s with no DB call before the throw, or the
insert uses `ignore_if_duplicate=True` / `autoname="hash"` (→ `ON CONFLICT DO NOTHING`).
- **Recover the txn with a *scoped* savepoint, not a full `frappe.db.rollback()`, if any prior work
must survive.** A full rollback un-poisons the txn but also discards every row the handler committed
*before* the failure — which MariaDB kept (it has no statement-abort), so it's a **silent MariaDB
regression**. **"The background job / whitelist entrypoint owns the txn" does NOT make a full rollback
safe** if it did multiple inserts in a loop first — it drops the partial results MariaDB retained. A
full rollback is safe only when it (a) immediately re-`throw`s/`raise`s (MariaDB rolls back anyway),
(b) has nothing successful before it (a single op), or (c) the batch is genuinely meant to be
**atomic** (a partial result is an invalid state → rollback + mark *Failed* is correct). Otherwise use
a **per-iteration / per-record savepoint** — and keep the function's success/`None` return contract:
do **not** return the doc when the savepoint was rolled back.
---
## 6. Refactors and raw-SQL→ORM conversions are not automatically 1:1
A commit labeled a **refactor** or a **raw-`frappe.db.sql` → `frappe.qb`/ORM conversion** is meant
to preserve behaviour — but it easily doesn't, and the change passes the static checker and a
one-engine green run. **Diff the `WHERE`/predicate, the `JOIN`/`ON` conditions, and the resulting
row set — not just the `SELECT` shape.** A conversion that silently widens or narrows the filter
changes the rows touched on **both** engines and is a regression hiding under a "refactor" label.
Real example: an `UPDATE` whose bound was `posting_datetime > X` gained an
`OR (posting_datetime == X AND creation > args.creation)` branch during a "`sql` → `qb` refactor",
widening the rows updated on both engines. Even when such a change is a deliberate bug-fix it must
be called out and tested — it is **not** the no-op the refactor label implies. Confirm the
converted query touches exactly the same rows with the same values MariaDB produced before.
---
## How to review
For every changed query: does it (a) use a construct from §1 (would error on PostgreSQL),
(b) match a divergence in §2/§3 (different result across engines), or (c) change the row set under
a refactor/conversion label (§6)? If so, comment with the
portable fix and confirm it leaves **MariaDB output unchanged**. Skip the §4 false positives.
Prefer a comment that names the rule (e.g. "loose GROUP BY — Max()-wrap, don't add to GROUP BY:
splits the row count") so the fix is unambiguous.
The static pre-commit checker (`.github/helper/postgres_compat.py`) catches the *mechanical*
§1 breaks; the **semantic** §2/§3 divergences and the §6 refactor/conversion row-set changes are
exactly what a reviewer (and this guide) must cover, because no static check can see them.

View File

@@ -1,72 +0,0 @@
#!/bin/bash
#
# Hydrate a test shard from the setup job's artifact.
#
# The bench (apps, venv, node_modules, sites) is already on disk at ~/frappe-bench — the
# workflow untar'd it from the artifact the setup job built. So there is NO bench init, no
# asset build, and no reinstall here: just bring the DB up on the baked datadir and start redis
# so tests can run. The whole point is that the expensive work happened ONCE in the setup job.
#
set -e
ci_user="${ERPNEXT_CI_USER:-frappe}"
db_host="${DB_HOST:-127.0.0.1}"
# Re-exec as the ci user (uid 1001) so bench/cache ownership matches the artifact, same as
# install.sh. The workflow untar'd as root with -p, so the files are already owned by ci.
if [ "$(id -u)" = "0" ] && [ "${SKIP_SYSTEM_SETUP:-0}" = "1" ] && [ "$ci_user" != "root" ]; then
exec su -m "$ci_user" -s /bin/bash -c \
"ERPNEXT_CI_USER='$ci_user' DB_HOST='$db_host' DB='${DB:-}' bash '$0'"
fi
cd ~/frappe-bench
# Start the DB on the datadir baked into the artifact. It's already populated (the setup job
# reinstalled into this very datadir), so there is NO restore — the server comes up on the
# existing files. This is what replaces the per-shard SQL replay.
bash ~/frappe-bench/start-db.sh
# Bring up redis (lightmode unit tests need cache + queue). In the self-hosted container we use the
# full `bench start` (web/workers too, like install.sh). On the bare GitHub Postgres shard
# `bench start` (honcho) lagged — it blocks the redis procs behind web/worker procs the lightmode
# suite never uses, so the wait below burned its full timeout (~4m). There, start the two redis
# instances directly: fast and deterministic.
if [ "${DB:-mariadb}" = "postgres" ]; then
# Start redis directly as daemons — reliable and persists across steps. Do NOT route it through
# `bench start`: honcho tears the whole process group down if any one Procfile proc dies on the
# bare shard, which took redis with it (redis @ 13000 refused in Run Tests). Keeping redis
# independent is what makes it survive. The web server (for PDF tests) is NOT started here — a
# backgrounded server doesn't survive into the next step; it's started inside the Run Tests step.
for conf in redis_cache redis_queue; do
[ -f ~/frappe-bench/config/$conf.conf ] && redis-server ~/frappe-bench/config/$conf.conf --daemonize yes
done
else
bench start >> ~/frappe-bench/bench_start.log 2>&1 &
fi
# Wait for redis, failing fast instead of silently burning minutes if it never comes up.
cfg=~/frappe-bench/sites/common_site_config.json
if [ -f "$cfg" ]; then
ports=$(python - "$cfg" <<'PY'
import json, re, sys
try:
cfg = json.load(open(sys.argv[1]))
except Exception:
sys.exit(0)
for key in ("redis_cache", "redis_queue"):
m = re.search(r":(\d+)", str(cfg.get(key, "")))
if m:
print(m.group(1))
PY
)
for port in $ports; do
up=0
for _ in $(seq 1 60); do
if (exec 3<>"/dev/tcp/127.0.0.1/$port") 2>/dev/null; then exec 3>&- 3<&-; up=1; break; fi
sleep 1
done
[ "$up" = "1" ] || { echo "redis did not come up on port $port"; exit 1; }
done
fi
echo "Hydrated: DB up on baked datadir, redis up — ready for tests."

View File

@@ -4,406 +4,75 @@ set -e
cd ~ || exit
# Authenticate git against github.com with the job token: anonymous git-over-HTTPS from the
# runners gets throttled to a 401, which kills whichever clone is in flight — the frappe fetch
# below, or payments under `bench get-app`. See the PR description.
#
# A credential helper rather than a url.insteadOf rewrite, because `git clone` PERSISTS a
# rewritten URL into the new repo's .git/config: an insteadOf would leave the token sitting in
# apps/payments/.git/config on the runner. A helper is consulted only when github.com actually
# challenges, and leaves the stored remote URL untouched. Passing it through GIT_CONFIG_* keeps
# the token out of ~/.gitconfig too, and child processes inherit it (bench shells out to git).
ci_github_token=${CI_GITHUB_TOKEN:-${GITHUB_TOKEN:-}}
if [ -n "$ci_github_token" ]; then
export CI_GITHUB_TOKEN="$ci_github_token"
export GIT_CONFIG_COUNT=3
# Reset first: git runs EVERY configured helper and calls `store` on them after a successful
# auth, so a `credential.helper=store` inherited from the image's gitconfig would write the
# token to ~/.git-credentials. An empty value clears the list before ours is added.
export GIT_CONFIG_KEY_0="credential.helper"
export GIT_CONFIG_VALUE_0=""
export GIT_CONFIG_KEY_1="credential.https://github.com.username"
export GIT_CONFIG_VALUE_1="x-access-token"
export GIT_CONFIG_KEY_2="credential.https://github.com.helper"
# Single-quoted: $CI_GITHUB_TOKEN is expanded by the shell git runs the helper in, so the
# token is read from the environment at call time and never stored anywhere. Answering only
# `get` makes the helper inert for git's `store`/`erase` calls.
export GIT_CONFIG_VALUE_2='!f() { test "$1" = get && echo "password=$CI_GITHUB_TOKEN"; }; f'
fi
sudo apt update
sudo apt remove mysql-server mysql-client
sudo apt install libcups2-dev redis-server mariadb-client libmariadb-dev
# Whatever happens, never sit on a credential prompt: fail fast and legibly instead.
export GIT_TERMINAL_PROMPT=0
pip install frappe-bench
githubbranch=${GITHUB_BASE_REF:-${GITHUB_REF##*/}}
frappeuser=${FRAPPE_USER:-"frappe"}
frappecommitish=${FRAPPE_BRANCH:-}
# A stacked pull request targets another erpnext branch, which has no counterpart in frappe.
# Fall back to develop so the bench is still installed. An explicit FRAPPE_BRANCH is trusted as
# given, since it can be a commit sha rather than a branch.
if [ -z "$frappecommitish" ]; then
frappecommitish=$githubbranch
# git ls-remote --exit-code reports 2 for a branch that is not there and 128 for a remote it
# could not reach. Only the first one is proof of absence; keep the branch on anything else so
# a flaky probe cannot install an unrelated frappe.
probe=0
git ls-remote --exit-code --heads "https://github.com/${frappeuser}/frappe" "$frappecommitish" >/dev/null 2>&1 || probe=$?
if [ "$probe" -eq 2 ]; then
echo "frappe has no branch ${frappecommitish}, falling back to develop"
frappecommitish=develop
elif [ "$probe" -ne 0 ]; then
echo "could not reach frappe to check for branch ${frappecommitish} (git ls-remote exited ${probe}), keeping it"
fi
fi
db_host=${DB_HOST:-"127.0.0.1"}
db_user_host=${DB_USER_HOST:-"localhost"}
wkhtmltox_deb=${WKHTMLTOX_DEB:-"/tmp/wkhtmltox.deb"}
bench_cache_dir=${BENCH_CACHE_DIR:-}
run_as_ci_user_if_needed() {
if [ "$(id -u)" != "0" ] || [ "${SKIP_SYSTEM_SETUP:-0}" != "1" ] || [ "${ERPNEXT_CI_NON_ROOT:-0}" = "1" ]; then
return
fi
local missing_packages=()
if ! command -v pkg-config >/dev/null 2>&1; then
missing_packages+=("pkg-config")
fi
if ! command -v mariadb_config >/dev/null 2>&1 && ! command -v mysql_config >/dev/null 2>&1; then
missing_packages+=("libmariadb-dev")
fi
if ! command -v crontab >/dev/null 2>&1; then
missing_packages+=("cron")
fi
if [ "${#missing_packages[@]}" -gt 0 ]; then
apt-get update
apt-get install -y --no-install-recommends "${missing_packages[@]}"
fi
local ci_user="${ERPNEXT_CI_USER:-frappe}"
if ! id "$ci_user" >/dev/null 2>&1; then
useradd --home-dir "$HOME" --no-create-home --shell /bin/bash "$ci_user"
fi
rm -rf ~/frappe ~/frappe-bench
local ci_dirs=(
"$HOME"
"$GITHUB_WORKSPACE"
"$HOME/.cache"
"${PIP_CACHE_DIR:-$HOME/.cache/pip}"
"${npm_config_cache:-$HOME/.npm}"
"${YARN_CACHE_FOLDER:-$HOME/.cache/yarn}"
"$HOME/.yarn"
"${UV_CACHE_DIR:-$HOME/.cache/uv}"
"$(dirname "$wkhtmltox_deb")"
)
if [ -n "$bench_cache_dir" ]; then
ci_dirs+=("$bench_cache_dir")
fi
# Create + own (non-recursively) the home/cache/workspace dirs before dropping to
# the ci user. We deliberately do NOT wipe the yarn/uv caches here so a persistent
# cache (mounted volume or baked image layer) stays warm across runs.
mkdir -p "${ci_dirs[@]}" "$HOME/.yarn"
chown "$ci_user:$ci_user" "${ci_dirs[@]}" "$HOME/.yarn"
export ERPNEXT_CI_NON_ROOT=1
exec su -m "$ci_user" -s /bin/bash -c "cd '$HOME' && bash '$GITHUB_WORKSPACE/.github/helper/install.sh'"
}
run_as_ci_user_if_needed
run_ci_step() {
local label=$1
shift
echo "::group::${label}"
date -u
local exit_code=0
timeout --foreground "${CI_INSTALL_STEP_TIMEOUT:-1800}" "$@" || exit_code=$?
date -u
echo "::endgroup::"
return "$exit_code"
}
if [ -n "${GITHUB_WORKSPACE:-}" ]; then
git config --global --add safe.directory "$GITHUB_WORKSPACE" || true
git config --global --add safe.directory "$GITHUB_WORKSPACE/.git" || true
fi
rm -rf ~/frappe ~/frappe-bench
# ---------------------------------------------------------------------------
# Phase 1 — parallelise the three slow, independent setup steps:
# a) system packages b) frappe-bench pip install c) frappe git fetch
# ---------------------------------------------------------------------------
if [ "${SKIP_SYSTEM_SETUP:-0}" != "1" ]; then
sudo apt-get update
# apt remove/install must run sequentially but can overlap with pip and git.
sudo apt-get remove -y mysql-server mysql-client
sudo apt-get install -y libcups2-dev redis-server mariadb-client libmariadb-dev &
apt_pid=$!
pip install frappe-bench &
pip_pid=$!
else
apt_pid=
pip_pid=
fi
frappecommitish=${FRAPPE_BRANCH:-$githubbranch}
mkdir frappe
(
cd frappe
git init
git remote add origin "https://github.com/${frappeuser}/frappe"
git fetch origin "${frappecommitish}" --depth 1
) &
clone_pid=$!
if [ -n "$apt_pid" ]; then wait $apt_pid; fi
if [ -n "$pip_pid" ]; then wait $pip_pid; fi
wait $clone_pid
pushd frappe
git init
git remote add origin "https://github.com/${frappeuser}/frappe"
git fetch origin "${frappecommitish}" --depth 1
git checkout FETCH_HEAD
popd
frappe_sha=$(git -C frappe rev-parse HEAD)
get_bench_cache_archive() {
if [ -z "$bench_cache_dir" ]; then
return
fi
bench init --skip-assets --frappe-path ~/frappe --python "$(which python)" frappe-bench
mkdir -p "$bench_cache_dir"
# Keyed on tool versions only (NOT the frappe SHA): any recent base bench works, because
# restore_warm_bench fast-forwards it to the exact live develop SHA. This is what lets a
# constantly-moving develop still hit the cache.
local cache_key
cache_key=$(
{
uname -m
python --version
node --version
bench --version
} | sha256sum | awk '{print $1}'
)
echo "${bench_cache_dir}/frappe-bench-base-${cache_key}.tar.zst"
}
restore_warm_bench() {
bench_cache_archive=$(get_bench_cache_archive)
[ -n "$bench_cache_archive" ] && [ -f "$bench_cache_archive" ] || return 1
echo "Restoring base bench from ${bench_cache_archive}"
tar --use-compress-program=unzstd -xf "$bench_cache_archive" -C ~ || return 1
[ -d ~/frappe-bench/apps/frappe/.git ] || return 1
mkdir -p ~/frappe-bench/sites ~/frappe-bench/logs
[ -f ~/frappe-bench/sites/apps.txt ] || printf "frappe\n" > ~/frappe-bench/sites/apps.txt
[ -f ~/frappe-bench/sites/common_site_config.json ] || printf "{}\n" > ~/frappe-bench/sites/common_site_config.json
# Fast-forward the restored frappe to the EXACT live develop SHA fetched in phase 1, then
# rebuild only what changed. The editable install means the venv tracks the new code with
# no reinstall. Any failure returns non-zero so the caller falls back to a full bench init.
if ! (
cd ~/frappe-bench/apps/frappe || exit 1
# Phase 1 already fetched ~/frappe to the exact live develop SHA. Fetch that commit
# straight from it (bench init names the remote 'upstream', not 'origin', and points
# it at this local clone — so a plain `git fetch origin` does not work).
git fetch --no-tags --update-shallow "$HOME/frappe" HEAD || exit 1
git checkout --force FETCH_HEAD || exit 1
); then
echo "Fast-forward to ${frappe_sha} failed; falling back to full init"
rm -rf ~/frappe-bench
return 1
fi
# Pick up any frappe dependency changes since the base was built (cached → fast if none),
# so a develop commit that bumped requirements doesn't leave a stale venv.
if ! ~/frappe-bench/env/bin/python -m pip install -q -e ~/frappe-bench/apps/frappe; then
echo "frappe dependency refresh failed; falling back to full init"
rm -rf ~/frappe-bench
return 1
fi
( cd ~/frappe-bench && CI=Yes bench build --app frappe ) || { rm -rf ~/frappe-bench; return 1; }
return 0
}
save_warm_bench() {
if [ -z "${bench_cache_archive:-}" ] || [ -f "$bench_cache_archive" ]; then
return
fi
if [ -n "$bench_cache_dir" ] && [ ! -w "$bench_cache_dir" ]; then
echo "Skipping warm bench save because ${bench_cache_dir} is not writable"
return
fi
local tmp_archive
tmp_archive="${bench_cache_archive}.${$}.tmp"
echo "Saving warm bench to ${bench_cache_archive}"
# Keep sites/common_site_config.json (the redis ports live there — dropping it makes the
# restore path fall back to a default redis port that bench start never bound, so reinstall
# fails with "redis ... connection refused"). Only the rebuildable sites/assets is excluded;
# restore_warm_bench runs `bench build` to regenerate it.
tar \
--use-compress-program="zstd -T0 -3" \
--exclude="frappe-bench/logs" \
--exclude="frappe-bench/sites/assets" \
-cf "$tmp_archive" \
-C ~ frappe-bench
mv "$tmp_archive" "$bench_cache_archive"
}
# ---------------------------------------------------------------------------
# Phase 2 — bench init and site setup
# ---------------------------------------------------------------------------
install_whktml() {
# Re-use the .deb if the wkhtmltopdf cache step already restored it.
if [ ! -f "$wkhtmltox_deb" ]; then
wget -O "$wkhtmltox_deb" https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6.1-2/wkhtmltox_0.12.6.1-2.jammy_amd64.deb
fi
sudo apt-get install -y "$wkhtmltox_deb"
}
if [ "${SKIP_WKHTMLTOX_SETUP:-0}" != "1" ]; then
install_whktml &
wkpid=$!
else
wkpid=
fi
if ! restore_warm_bench; then
bench init --skip-assets --frappe-path ~/frappe --python "$(which python)" frappe-bench
cd ~/frappe-bench || exit
sed -i 's/watch:/# watch:/g' Procfile
sed -i 's/schedule:/# schedule:/g' Procfile
sed -i 's/socketio:/# socketio:/g' Procfile
sed -i 's/redis_socketio:/# redis_socketio:/g' Procfile
CI=Yes bench build --app frappe
save_warm_bench
fi
if [ -n "$wkpid" ]; then wait $wkpid; fi
mkdir -p ~/frappe-bench/sites/test_site
mkdir ~/frappe-bench/sites/test_site
if [ "$DB" == "mariadb" ];then
cp -r "${GITHUB_WORKSPACE}/.github/helper/site_config_mariadb.json" ~/frappe-bench/sites/test_site/site_config.json
if [ "$db_host" != "127.0.0.1" ]; then
sed -i "s/\"db_host\": \"127.0.0.1\"/\"db_host\": \"${db_host}\"/" ~/frappe-bench/sites/test_site/site_config.json
fi
else
cp -r "${GITHUB_WORKSPACE}/.github/helper/site_config_postgres.json" ~/frappe-bench/sites/test_site/site_config.json
fi
if [ "$DB" == "mariadb" ];then
for _ in {1..60}; do
if mariadb-admin ping --host "$db_host" --port 3306 -u root -proot --silent; then
break
fi
sleep 1
done
mariadb-admin ping --host "$db_host" --port 3306 -u root -proot --silent
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "SET GLOBAL character_set_server = 'utf8mb4'"
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "SET GLOBAL collation_server = 'utf8mb4_unicode_ci'"
mariadb --host "$db_host" --port 3306 -u root -proot -e "SET GLOBAL character_set_server = 'utf8mb4'"
mariadb --host "$db_host" --port 3306 -u root -proot -e "SET GLOBAL collation_server = 'utf8mb4_unicode_ci'"
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "CREATE USER 'test_frappe'@'localhost' IDENTIFIED BY 'test_frappe'"
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "CREATE DATABASE test_frappe"
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "GRANT ALL PRIVILEGES ON \`test_frappe\`.* TO 'test_frappe'@'localhost'"
# Throwaway-DB durability tuning at runtime. (innodb_doublewrite is read-only on MariaDB
# 10.6, so it can't be disabled here — would need a server startup flag.)
mariadb --host "$db_host" --port 3306 -u root -proot \
-e "SET GLOBAL innodb_flush_log_at_trx_commit=0; SET GLOBAL sync_binlog=0;"
# Opt-in DDL speedup: a shared tablespace avoids a create+fsync per DocType table during
# reinstall — a big win under disk contention. But ROW_FORMAT=DYNAMIC must be accepted in
# the system tablespace on this MariaDB. Enable with CI_INNODB_SHARED_TABLESPACE=1; if
# reinstall then errors on table creation, unset it (off by default — zero risk).
if [ "${CI_INNODB_SHARED_TABLESPACE:-0}" = "1" ]; then
mariadb --host "$db_host" --port 3306 -u root -proot -e "SET GLOBAL innodb_file_per_table=0;"
fi
mariadb --host "$db_host" --port 3306 -u root -proot -e "CREATE USER 'test_frappe'@'${db_user_host}' IDENTIFIED BY 'test_frappe'"
mariadb --host "$db_host" --port 3306 -u root -proot -e "CREATE DATABASE test_frappe"
mariadb --host "$db_host" --port 3306 -u root -proot -e "GRANT ALL PRIVILEGES ON \`test_frappe\`.* TO 'test_frappe'@'${db_user_host}'"
mariadb --host "$db_host" --port 3306 -u root -proot -e "FLUSH PRIVILEGES"
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "FLUSH PRIVILEGES"
fi
if [ "$DB" == "postgres" ];then
echo "travis" | psql -h 127.0.0.1 -p 5432 -c "CREATE DATABASE test_frappe" -U postgres;
echo "travis" | psql -h 127.0.0.1 -p 5432 -c "CREATE USER test_frappe WITH PASSWORD 'test_frappe'" -U postgres;
# Durability-off for speed (no fsync/synchronous_commit/full_page_writes) is applied by
# start-db.sh's postgres `-o` flags on every start — setup job AND each test shard — so it is
# NOT repeated here. The postgres workflow runs in-runner via start-db.sh, not a service
# container.
fi
install_whktml() {
wget -O /tmp/wkhtmltox.deb https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6.1-2/wkhtmltox_0.12.6.1-2.jammy_amd64.deb
sudo apt install /tmp/wkhtmltox.deb
}
install_whktml &
wkpid=$!
cd ~/frappe-bench || exit
run_ci_step "Get payments app" bench get-app payments --branch develop
sed -i 's/watch:/# watch:/g' Procfile
sed -i 's/schedule:/# schedule:/g' Procfile
sed -i 's/socketio:/# socketio:/g' Procfile
sed -i 's/redis_socketio:/# redis_socketio:/g' Procfile
# Opt-in: skip building erpnext's frontend assets. Server tests don't need them, but PDF
# tests (print formats) do — they pass only if the PDF renderer ignores missing assets.
# Enable with CI_SKIP_ERPNEXT_ASSETS=1 to test; if PDF tests fail, unset it.
erpnext_get_app_args=()
if [ "${CI_SKIP_ERPNEXT_ASSETS:-0}" = "1" ]; then erpnext_get_app_args=(--skip-assets); fi
run_ci_step "Get erpnext app" bench get-app erpnext "${GITHUB_WORKSPACE}" "${erpnext_get_app_args[@]}"
bench get-app payments --branch develop
bench get-app erpnext "${GITHUB_WORKSPACE}"
if [ "$TYPE" == "server" ]; then run_ci_step "Setup dev requirements" bench setup requirements --dev; fi
if [ "$TYPE" == "server" ]; then bench setup requirements --dev; fi
bench start >> ~/frappe-bench/bench_start.log 2>&1 &
wait $wkpid
# Under heavy concurrency, gunicorn's startup can delay redis coming up. reinstall and the
# tests need redis, so wait for it (best-effort, bounded) instead of racing — contention
# then slows the job rather than failing it.
wait_for_redis() {
local cfg=~/frappe-bench/sites/common_site_config.json
[ -f "$cfg" ] || return 0
local ports port
ports=$(python - "$cfg" <<'PY'
import json, re, sys
try:
cfg = json.load(open(sys.argv[1]))
except Exception:
sys.exit(0)
for key in ("redis_cache", "redis_queue"):
match = re.search(r":(\d+)", str(cfg.get(key, "")))
if match:
print(match.group(1))
PY
)
for port in $ports; do
local up=0
for _ in $(seq 1 120); do
if (exec 3<>"/dev/tcp/127.0.0.1/$port") 2>/dev/null; then
exec 3>&- 3<&-; up=1
break
fi
sleep 1
done
# Fail clearly instead of letting reinstall die later on a vague socket-connection error
# when redis never bound.
[ "$up" = "1" ] || { echo "redis did not come up on port $port"; return 1; }
done
}
wait_for_redis
# Site setup: build the schema (~1000 DocTypes) into the DB. This is the single-threaded-Python
# bottleneck, but the fan-out amortises it — it runs once here in the setup job, and the test
# shards start the DB on the baked datadir instead of repeating the reinstall.
run_ci_step "Reinstall test site" bench --site test_site reinstall --yes
bench start &>> ~/frappe-bench/bench_start.log &
CI=Yes bench build --app frappe &
bench --site test_site reinstall --yes

View File

@@ -1,241 +0,0 @@
#!/usr/bin/env python3
"""Static guard against MySQL-only SQL that breaks on PostgreSQL.
The Postgres test job is label-gated, so it does not run on every PR. This pre-commit
hook is the always-on first line of defence: it flags the *mechanical* Postgres breaks
that static analysis can catch reliably with a low false-positive rate.
It deliberately does NOT try to catch the *semantic* divergences (loose GROUP BY,
case-sensitive ==/IN, NULL ordering, ORDER BY ... LIMIT 1 tiebreakers, integer-division
intent, division by a possibly-zero divisor, savepoint discipline) — those genuinely need
the test suite or a human/Greptile reviewer. Run the full suite on a Postgres site for those.
Escape hatch: put `# pg-ok` anywhere on the offending statement's line span (e.g. on a
`SHOW INDEX` query that lives inside an `if frappe.db.db_type == "mariadb":` branch).
Usage: postgres_compat.py <file.py> [<file.py> ...] (pre-commit passes staged files)
"""
from __future__ import annotations
import ast
import re
import sys
IGNORE = "pg-ok"
# Strings are only scanned for the patterns below when they have real SQL *structure*
# (not just an English word like "select" or "from"), to keep false positives near zero.
SQL_HINT = re.compile(
r"\bselect\b[\s\S]{0,800}\bfrom\b" # SELECT ... FROM
r"|\bupdate\b[\s\S]{0,400}\bset\b" # UPDATE ... SET
r"|\bdelete\s+from\b"
r"|\binsert\s+into\b"
r"|\bshow\s+(?:index|tables|columns)\b"
r"|\bfrom\s+[\"'`]?tab", # FROM `tabDocType`
re.I,
)
# MySQL-only constructs with NO frappe auto-translation. (frappe.db.sql already rewrites
# ifnull->coalesce on all engines and backtick/locate/REGEXP on Postgres, and .like()
# renders ILIKE — so those are NOT listed here; flagging them would be false positives.)
SQL_PATTERNS: list[tuple[re.Pattern, str]] = [
(re.compile(r"\btimestamp\s*\(\s*[^,()]+,", re.I),
"timestamp(date, time) is MySQL-only -> use CombineDatetime() or a precomputed datetime column"),
(re.compile(r"\btimediff\s*\(", re.I),
"timediff() is MySQL-only -> compute the delta in Python"),
(re.compile(r"\bstr_to_date\s*\(", re.I),
"str_to_date() is MySQL-only -> parse in Python and pass a real date"),
(re.compile(r"\bdate_format\s*\(", re.I),
"date_format() is MySQL-only -> filter on a date range instead"),
(re.compile(r"\bdate_(add|sub)\s*\(", re.I),
"date_add()/date_sub() are MySQL-only -> use Python date math or interval arithmetic"),
(re.compile(r"\bgroup_concat\s*\(", re.I),
"group_concat() is MySQL-only -> use GroupConcat (string_agg) or aggregate in Python"),
(re.compile(r"\bperiod_diff\s*\(", re.I),
"period_diff() is MySQL-only -> compute in Python"),
(re.compile(r"\bshow\s+index\b", re.I),
"SHOW INDEX is MySQL-only -> use frappe.db.has_index() / get_column_index()"),
(re.compile(r"\bshow\s+(tables|columns)\b", re.I),
"SHOW TABLES/COLUMNS is MySQL-only -> use frappe.db.get_tables()/table_columns / information-schema helpers"),
(re.compile(r"\bas\s+'[^']+'", re.I),
"single-quoted column alias breaks on Postgres -> use a bare or double-quoted alias"),
(re.compile(r"\bif\s*\(", re.I),
"SQL IF() is MySQL-only -> use CASE WHEN ... THEN ... ELSE ... END (frappe.qb.Case())"),
(re.compile(r"\brlike\b", re.I),
"RLIKE is MySQL-only -> frappe rewrites REGEXP->~* on Postgres but NOT RLIKE; use REGEXP / .regexp() / ~"),
(re.compile(r"\bcast\s*\(.+?\bas\s+char\b", re.I | re.S), # .+? spans nested parens, e.g. CAST(ABS(x) AS CHAR)
"CAST(... AS CHAR) is character(1) on Postgres and truncates -> CAST AS VARCHAR (frappe Cast_(x, 'varchar'))"),
]
# UPDATE ... JOIN: both keywords in the same SQL string.
UPDATE_JOIN = (re.compile(r"\bupdate\b", re.I), re.compile(r"\bjoin\b", re.I))
MYSQL_RESULT_KEYS = {"Column_name", "Key_name", "Seq_in_index", "Non_unique", "Index_type"}
SET_BOOL_FUNCS = {"set_value", "db_set"}
# query-builder cast helpers: pypika Cast / frappe Cast_. A "char" target type is character(1)
# on Postgres (truncates); "varchar" is the full-length cast.
CAST_FUNCS = {"Cast", "Cast_"}
# frappe.get_all / get_list: frappe's db_query SILENTLY drops ORDER BY for `distinct` queries on
# Postgres (the ORDER BY column must appear in the SELECT-DISTINCT list), so `distinct=True` together
# with a literal `order_by` is a no-op on PG and the result comes back unordered.
DISTINCT_ORDER_FUNCS = {"get_all", "get_list"}
def _docstring_ids(tree: ast.AST) -> set[int]:
"""ids of Constant nodes that are docstrings (so prose describing the rules isn't flagged)."""
ids: set[int] = set()
for node in ast.walk(tree):
if isinstance(node, (ast.Module, ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
body = getattr(node, "body", None)
if body and isinstance(body[0], ast.Expr) and isinstance(body[0].value, ast.Constant) and isinstance(body[0].value.value, str):
ids.add(id(body[0].value))
return ids
class Visitor(ast.NodeVisitor):
def __init__(self, lines: list[str], docstrings: set[int]):
self.lines = lines
self.docstrings = docstrings
self.violations: list[tuple[int, str]] = []
def _ignored(self, node: ast.AST) -> bool:
start = getattr(node, "lineno", 1)
end = getattr(node, "end_lineno", start) or start
# honour `# pg-ok` anywhere on the node's line span, the line just above (the enclosing
# call, e.g. `frappe.db.sql( # pg-ok`), or the line just below (a multi-line call's `) # pg-ok`).
lo = max(0, start - 2)
return any(IGNORE in self.lines[i] for i in range(lo, min(end + 1, len(self.lines))))
def _flag(self, node: ast.AST, msg: str) -> None:
if not self._ignored(node):
self.violations.append((getattr(node, "lineno", 1), msg))
def _scan_sql(self, text: str, node: ast.AST) -> None:
if not SQL_HINT.search(text):
return
for pattern, msg in SQL_PATTERNS:
if pattern.search(text):
self._flag(node, msg)
if UPDATE_JOIN[0].search(text) and UPDATE_JOIN[1].search(text):
self._flag(node, "UPDATE ... JOIN is MySQL-only -> use a correlated subquery (WHERE ... IN/EXISTS)")
def visit_Constant(self, node: ast.Constant) -> None:
# plain string literals, incl. `"...".format()` and `"..." % (...)` templates
if isinstance(node.value, str) and id(node) not in self.docstrings:
self._scan_sql(node.value, node)
self.generic_visit(node)
def visit_JoinedStr(self, node: ast.JoinedStr) -> None:
# f-string: scan its STATIC text (interpolated values become a placeholder) so MySQL-isms
# in dynamic SQL are caught, without flagging safe interpolation of identifiers.
text = "".join(
v.value if isinstance(v, ast.Constant) and isinstance(v.value, str) else " ? "
for v in node.values
)
self._scan_sql(text, node)
# don't recurse: child literal chunks would otherwise be re-scanned individually
def visit_Call(self, node: ast.Call) -> None:
fn = node.func
name = fn.attr if isinstance(fn, ast.Attribute) else (fn.id if isinstance(fn, ast.Name) else "")
# row.get("Column_name") — MySQL SHOW INDEX result key
if name == "get" and node.args and isinstance(node.args[0], ast.Constant) and node.args[0].value in MYSQL_RESULT_KEYS:
self._flag(node, f'"{node.args[0].value}" is a MySQL SHOW INDEX result key -> use frappe.db.has_index()/get_column_index()')
# set_value(..., True) / db_set("field", True) on a Check (int) column.
# Only the field *value* arg carries bool->smallint risk — NOT trailing flags like
# update_modified. db_set(field, value, update_modified, ...) -> value at args[1] (or a dict
# at args[0]); set_value(dt, dn, field, value, ...) -> value at args[3] (or a dict at args[2]).
if name in SET_BOOL_FUNCS:
value_idx, dict_idx = (1, 0) if name == "db_set" else (3, 2)
dict_arg = (
node.args[dict_idx]
if len(node.args) > dict_idx and isinstance(node.args[dict_idx], ast.Dict)
else None
)
if dict_arg is not None:
for v in dict_arg.values:
if isinstance(v, ast.Constant) and isinstance(v.value, bool):
self._flag(node, f"{name}(...) sets an int/Check column with a bool in a dict -> pass 1/0 (Postgres rejects bool->smallint)")
elif len(node.args) > value_idx:
a = node.args[value_idx]
if isinstance(a, ast.Constant) and isinstance(a.value, bool):
self._flag(node, f"{name}(..., {a.value}) sets an int/Check column with a bool -> pass 1/0 (Postgres rejects bool->smallint)")
# frappe.get_all/get_list(..., distinct=True, order_by="<col>") -> ORDER BY is silently dropped
# for distinct queries on Postgres, so the result is unordered there. Sort in python instead
# (e.g. sorted(frappe.get_all(..., distinct=True), key=str.casefold)). An empty order_by="" (the
# explicit "suppress the injected default" idiom) and a dynamic/variable order_by are not flagged.
if name in DISTINCT_ORDER_FUNCS:
has_distinct = any(
kw.arg == "distinct" and isinstance(kw.value, ast.Constant) and kw.value.value
for kw in node.keywords
)
order_kw = next((kw for kw in node.keywords if kw.arg == "order_by"), None)
has_literal_order = (
order_kw is not None
and isinstance(order_kw.value, ast.Constant)
and isinstance(order_kw.value.value, str)
and order_kw.value.value.strip()
)
if has_distinct and has_literal_order:
self._flag(node, f"{name}(distinct=True, order_by=...) -> frappe drops ORDER BY for distinct queries on Postgres; sort in python instead, e.g. sorted(..., key=str.casefold)")
# query-builder .rlike(...): pypika emits the MySQL-only RLIKE operator, which frappe does
# NOT translate for Postgres (it rewrites only REGEXP -> ~*).
if name == "rlike":
self._flag(node, ".rlike() emits MySQL-only RLIKE (not translated on Postgres) -> use .regexp() (rewritten to ~*) or .like()")
# Cast(col, "char") / Cast_(col, "char"): on Postgres a bare CHAR is character(1) and truncates
# (e.g. CAST(12 AS CHAR) -> '1'); use "varchar" for a full-length string cast.
if name in CAST_FUNCS:
for arg in (*node.args, *(kw.value for kw in node.keywords)):
if isinstance(arg, ast.Constant) and isinstance(arg.value, str) and arg.value.strip().lower() == "char":
self._flag(node, f"{name}(..., 'char') is character(1) on Postgres and truncates -> use 'varchar'")
self.generic_visit(node)
def visit_Subscript(self, node: ast.Subscript) -> None:
key = node.slice
if isinstance(key, ast.Constant) and key.value in MYSQL_RESULT_KEYS:
self._flag(node, f'"{key.value}" is a MySQL SHOW INDEX result key -> use frappe.db.has_index()/get_column_index()')
self.generic_visit(node)
def check_file(path: str) -> list[str]:
try:
# nosemgrep: frappe-semgrep-rules.rules.security.frappe-security-file-traversal -- dev-only lint tool; `path` is a source file supplied by pre-commit, not user input
src = open(path, encoding="utf-8").read()
except (OSError, UnicodeDecodeError):
return []
try:
tree = ast.parse(src, filename=path)
except SyntaxError:
return [] # check-ast hook reports real syntax errors
v = Visitor(src.splitlines(), _docstring_ids(tree))
v.visit(tree)
return [f"{path}:{line}: [pg-compat] {msg}" for line, msg in sorted(set(v.violations))]
def main(argv: list[str]) -> int:
out: list[str] = []
for path in argv:
if path.endswith(".py"):
out.extend(check_file(path))
if out:
print("\n".join(out))
print(
f"\n{len(out)} PostgreSQL-incompatibility issue(s). Fix them, or add `# pg-ok` to a "
"line that is intentionally MariaDB-only (e.g. inside an `if frappe.db.db_type == 'mariadb':` branch)."
)
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))

View File

@@ -13,6 +13,6 @@
"root_login": "postgres",
"root_password": "travis",
"host_name": "http://test_site:8000",
"install_apps": ["payments", "erpnext"],
"install_apps": ["erpnext"],
"throttle_user_limit": 100
}

View File

@@ -1,79 +0,0 @@
#!/bin/bash
#
# Run MariaDB INSIDE the runner container, on a datadir we control. Because the datadir can be
# packaged into the bench artifact, test shards start an already-loaded server instead of
# replaying a SQL dump (the ~60s hydrate restore). Each shard gets its own copy → isolation kept.
#
# CI_DB_DATADIR picks the path:
# - setup job: /home/ci/db-data (OUTSIDE the bench, so install.sh's `rm -rf ~/frappe-bench`
# doesn't wipe it; it's moved into the bench just before packaging)
# - test shard: ~/frappe-bench/mariadb-data (where the artifact untar'd it)
#
# Idempotent: inits a fresh datadir if absent (setup), else starts on the existing one (shards).
#
set -e
ci_user="${ERPNEXT_CI_USER:-frappe}"
# Re-exec as the ci user so mariadbd and the datadir are owned consistently (root mariadbd is
# refused anyway). Mirrors install.sh's user switch.
if [ "$(id -u)" = "0" ] && [ "${SKIP_SYSTEM_SETUP:-0}" = "1" ] && [ "$ci_user" != "root" ]; then
exec su -m "$ci_user" -s /bin/bash -c \
"ERPNEXT_CI_USER='$ci_user' CI_DB_DATADIR='${CI_DB_DATADIR:-}' DB='${DB:-}' bash '$0'"
fi
# --- PostgreSQL (GitHub-hosted CI): run in-runner on a PGDATA so it bakes into the artifact,
# same idea as the mariadb datadir. Trust auth (throwaway CI) skips password setup; durability
# off for speed. Postgres is preinstalled on ubuntu-latest under /usr/lib/postgresql/<ver>/bin.
if [ "${DB:-mariadb}" = "postgres" ]; then
PG_BIN=$(ls -d /usr/lib/postgresql/*/bin 2>/dev/null | sort -V | tail -1)
[ -n "$PG_BIN" ] && export PATH="$PG_BIN:$PATH"
PGDATA="${CI_DB_DATADIR:-$HOME/frappe-bench/pgdata}"
if [ ! -d "$PGDATA/base" ]; then
initdb -D "$PGDATA" -U postgres --auth-local=trust --auth-host=trust >/dev/null
echo "host all all 127.0.0.1/32 trust" >> "$PGDATA/pg_hba.conf"
fi
pg_ctl -D "$PGDATA" -w -o "-p 5432 -c listen_addresses=127.0.0.1 -c unix_socket_directories=$PGDATA -c fsync=off -c synchronous_commit=off -c full_page_writes=off" start
echo "PostgreSQL up in-runner (pgdata=$PGDATA)"
exit 0
fi
# --- MariaDB ---
DATADIR="${CI_DB_DATADIR:-$HOME/frappe-bench/mariadb-data}"
SOCK="$DATADIR/mysqld.sock"
fresh=0
if [ ! -d "$DATADIR/mysql" ]; then
mkdir -p "$DATADIR"
mariadb-install-db --no-defaults --datadir="$DATADIR" \
--auth-root-authentication-method=normal --skip-test-db >/dev/null 2>&1
fresh=1
fi
# Throwaway-CI durability off; bind TCP 127.0.0.1:3306 so bench/install.sh connect as usual.
mariadbd --no-defaults --datadir="$DATADIR" --socket="$SOCK" --pid-file="$DATADIR/mysqld.pid" \
--port=3306 --bind-address=127.0.0.1 \
--innodb-flush-log-at-trx-commit=0 --sync-binlog=0 --skip-log-bin \
> "$HOME/mariadb.log" 2>&1 &
up=0
for _ in $(seq 1 60); do
if mariadb-admin --socket="$SOCK" ping --silent 2>/dev/null; then up=1; break; fi
sleep 1
done
# Fail loudly instead of letting the loop fall through (exit 0 of the last `sleep`) into SQL that
# would error with a vague socket-connection failure.
[ "$up" = "1" ] || { echo "mariadbd did not come up on $SOCK"; cat "$HOME/mariadb.log" 2>/dev/null; exit 1; }
if [ "$fresh" = "1" ]; then
# A fresh datadir has only a password-less root@localhost. Give it the password install.sh
# uses, plus a TCP-reachable root@127.0.0.1, so the rest of install.sh works unchanged.
mariadb --no-defaults --socket="$SOCK" -u root <<'SQL'
ALTER USER 'root'@'localhost' IDENTIFIED BY 'root';
CREATE USER IF NOT EXISTS 'root'@'127.0.0.1' IDENTIFIED BY 'root';
GRANT ALL PRIVILEGES ON *.* TO 'root'@'127.0.0.1' WITH GRANT OPTION;
FLUSH PRIVILEGES;
SQL
fi
echo "MariaDB up in-container (datadir=$DATADIR, fresh=$fresh)"

View File

@@ -1,51 +0,0 @@
name: Download translations from Crowdin
on:
schedule:
- cron: "0 4 * * 1"
workflow_dispatch:
concurrency:
group: crowdin-download
cancel-in-progress: false
permissions:
contents: read
jobs:
download-translations:
name: Download translations into ${{ matrix.branch }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
branch: ["develop", "version-16-hotfix"]
steps:
- name: Checkout ${{ matrix.branch }}
uses: actions/checkout@v6
with:
ref: ${{ matrix.branch }}
fetch-depth: 0
persist-credentials: false
- name: Download translations and open PR
uses: crowdin/github-action@8f01d54f70f1713ee3f09d82c2bbb2daeac28689 # v2.17.1
with:
config: crowdin.yml
upload_sources: false
upload_translations: false
download_translations: true
crowdin_branch_name: "[frappe.erpnext] ${{ matrix.branch }}"
skip_ref_checkout: true
localization_branch_name: l10n_crowdin_${{ matrix.branch }}
create_pull_request: true
pull_request_base_branch_name: ${{ matrix.branch }}
commit_message: "fix: sync translations from crowdin"
pull_request_title: "fix: sync translations from crowdin (${{ matrix.branch }})"
pull_request_labels: "translation, skip-release-notes"
pull_request_reviewers: barredterra
env:
GITHUB_TOKEN: ${{ secrets.RELEASE_TOKEN }}
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}

View File

@@ -1,54 +0,0 @@
name: Upload main.pot to Crowdin
on:
push:
branches:
- develop
- version-16-hotfix
paths:
- "erpnext/locale/main.pot"
workflow_dispatch:
concurrency:
group: crowdin-upload-${{ github.ref_name }}
cancel-in-progress: true
permissions:
contents: read
jobs:
upload-sources:
name: Upload sources from ${{ github.ref_name }}
runs-on: ubuntu-latest
steps:
- name: Checkout ${{ github.ref_name }}
uses: actions/checkout@v6
- name: Restore Crowdin cache
uses: actions/cache/restore@v6
with:
path: .crowdin
key: crowdin-${{ github.ref_name }}
restore-keys: crowdin-${{ github.ref_name }}-
- name: Upload main.pot to Crowdin
uses: crowdin/github-action@8f01d54f70f1713ee3f09d82c2bbb2daeac28689 # v2.17.1
with:
config: crowdin.yml
upload_sources: true
upload_translations: false
download_translations: false
create_pull_request: false
crowdin_branch_name: "[frappe.erpnext] ${{ github.ref_name }}"
upload_sources_args: "--cache"
env:
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
- name: Save Crowdin cache
uses: actions/cache/save@v6
if: always()
with:
path: .crowdin
key: crowdin-${{ github.ref_name }}-${{ github.run_id }}

View File

@@ -21,7 +21,7 @@ jobs:
cache: pip
- name: Install and Run Pre-commit
uses: pre-commit/action@v3.0.1
uses: pre-commit/action@v3.0.0
semgrep:
name: semgrep

View File

@@ -1,45 +0,0 @@
name: Notify Support on PR release
on:
issue_comment:
types: [created]
permissions: {}
jobs:
notify-support:
if: >-
github.event.issue.pull_request
&& github.event.comment.user.id == 28699486
&& contains(github.event.comment.body, 'This PR is included in version')
runs-on: ubuntu-latest
timeout-minutes: 2
steps:
- name: Notify support.frappe.io
env:
COMMENT_ID: ${{ github.event.comment.id }}
PR_NUMBER: ${{ github.event.issue.number }}
REPOSITORY: ${{ github.repository }}
SUPPORT_FRAPPE_AUTH: ${{ secrets.SUPPORT_FRAPPE_AUTH }}
run: |
payload="$(
jq -n \
--arg repository "$REPOSITORY" \
--argjson pr_number "$PR_NUMBER" \
--argjson comment_id "$COMMENT_ID" \
'{
repository: $repository,
pr_number: $pr_number,
comment_id: $comment_id
}'
)"
curl --fail-with-body \
--retry 3 \
--retry-all-errors \
--request POST \
--header "Authorization: $SUPPORT_FRAPPE_AUTH" \
--header "Content-Type: application/json" \
--data "$payload" \
"https://support.frappe.io/api/method/notify_pr_release"

View File

@@ -65,22 +65,6 @@ jobs:
- name: Add to Hosts
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
# The v14 baseline backup is a fixed published file — cache it instead of re-downloading
# it from the GitHub release every run.
- name: Cache erpnext v14 backup
id: cache-v14
uses: actions/cache@v4
with:
path: ~/erpnext-v14.sql.gz
key: erpnext-v14-sql-gz
- name: Download erpnext v14 backup
if: steps.cache-v14.outputs.cache-hit != 'true'
run: |
curl -fSL --retry 5 --retry-all-errors --retry-delay 5 \
-o ~/erpnext-v14.sql.gz \
https://github.com/frappe/erpnext/releases/download/v14-baseline/erpnext-v14.sql.gz
- name: Cache pip
uses: actions/cache@v4
with:
@@ -121,8 +105,6 @@ jobs:
env:
DB: mariadb
TYPE: server
# Anonymous git to github.com gets throttled to a 401; authenticate the clones.
CI_GITHUB_TOKEN: ${{ github.token }}
- name: Run Patch Tests
run: |
@@ -131,20 +113,12 @@ jobs:
jq 'del(.install_apps)' ~/frappe-bench/sites/test_site/site_config.json > tmp.json
mv tmp.json ~/frappe-bench/sites/test_site/site_config.json
bench --site test_site --force restore ~/erpnext-v14.sql.gz
wget https://frappe.io/files/erpnext-v14.sql.gz
bench --site test_site --force restore ~/frappe-bench/erpnext-v14.sql.gz
git -C "apps/frappe" remote set-url upstream https://github.com/frappe/frappe.git
git -C "apps/erpnext" remote set-url upstream https://github.com/frappe/erpnext.git
# Start every bench process except the background workers. If workers run during a
# migrate, they pick up the orphan-link cleanup jobs it enqueues and race its schema
# changes, which fails with MySQL 1412 "Table definition has changed". Redis and the
# other services stay up; the queued jobs simply wait and are harmless here.
function start_bench_without_workers() {
local procs
procs=$(awk -F: '/^[a-z_]+:/ && $1 !~ /worker/ {print $1}' ~/frappe-bench/Procfile)
honcho start -f ~/frappe-bench/Procfile $procs &>> ~/frappe-bench/bench_start.log &
}
function update_to_version() {
version=$1
@@ -160,11 +134,10 @@ jobs:
# Resetup env and install apps
pgrep honcho | xargs kill
sleep 10
rm -rf ~/frappe-bench/env
bench -v setup env --python python$2
bench pip install -e ./apps/erpnext
start_bench_without_workers
bench start &>> ~/frappe-bench/bench_start.log &
bench --site test_site migrate
}
@@ -173,30 +146,7 @@ jobs:
update_to_version 16 3.14
echo "Updating to latest version"
fallback_to_develop=0
if [ -n "${GITHUB_BASE_REF:-}" ]; then
frappe_ref="refs/heads/$GITHUB_BASE_REF"
fallback_to_develop=1
elif [ "${GITHUB_REF_TYPE:-}" = "branch" ]; then
frappe_ref="$GITHUB_REF"
fallback_to_develop=1
elif [ "${GITHUB_REF_TYPE:-}" = "tag" ]; then
frappe_ref="$GITHUB_REF"
else
echo "Unsupported GitHub ref type: '${GITHUB_REF_TYPE:-unset}'"
exit 1
fi
ls_remote_status=0
git -C "apps/frappe" ls-remote --exit-code upstream "$frappe_ref" >/dev/null \
|| ls_remote_status=$?
if [ "$ls_remote_status" -eq 2 ] && [ "$fallback_to_develop" -eq 1 ]; then
echo "frappe has no '$frappe_ref'; falling back to develop"
frappe_ref=refs/heads/develop
elif [ "$ls_remote_status" -ne 0 ]; then
exit "$ls_remote_status"
fi
git -C "apps/frappe" fetch --depth 1 upstream "$frappe_ref"
git -C "apps/frappe" fetch --depth 1 upstream "${GITHUB_BASE_REF:-${GITHUB_REF##*/}}"
git -C "apps/frappe" checkout -q -f FETCH_HEAD
git -C "apps/erpnext" checkout -q -f "$GITHUB_SHA"
@@ -204,7 +154,7 @@ jobs:
rm -rf ~/frappe-bench/env
bench -v setup env
bench pip install -e ./apps/erpnext
start_bench_without_workers
bench start &>> ~/frappe-bench/bench_start.log &
bench --site test_site migrate

View File

@@ -22,6 +22,4 @@ jobs:
pull-requests: write
steps:
- uses: alyf-de/po-review-action@5928f84d6bc9094f9ad6e2c5780f01c0044b800e # v1.1.1
with:
hidden-po-files: eo.po
- uses: alyf-de/po-review-action@v1.0.0

View File

@@ -129,8 +129,6 @@ jobs:
TYPE: server
FRAPPE_USER: ${{ github.event.inputs.user }}
FRAPPE_BRANCH: ${{ github.event.inputs.branch }}
# Anonymous git to github.com gets throttled to a 401; authenticate the clones.
CI_GITHUB_TOKEN: ${{ github.token }}
- name: Run Tests
run: |

View File

@@ -13,7 +13,6 @@ on:
- 'crowdin.yml'
- '.coderabbit.yml'
- '.mergify.yml'
- '**.po'
permissions:
contents: read

View File

@@ -13,7 +13,6 @@ on:
- 'crowdin.yml'
- '.coderabbit.yml'
- '.mergify.yml'
- '**.po'
schedule:
# Run everday at midnight UTC / 5:30 IST
- cron: "0 0 * * *"
@@ -32,49 +31,47 @@ on:
permissions:
contents: read
packages: read
concurrency:
group: server-mariadb-develop-${{ github.event_name }}-${{ github.event.number || github.event_name == 'workflow_dispatch' && github.run_id || '' }}
cancel-in-progress: true
# Shared across both jobs. Both run in the SAME CI image so the bench lives at the identical
# path (/home/ci/frappe-bench) on the setup runner and the test shards — that's what makes the
# packaged Python venv portable between them.
env:
TZ: 'Asia/Kolkata'
DEBIAN_FRONTEND: noninteractive
NODE_ENV: "production"
WITH_COVERAGE: ${{ github.event_name != 'pull_request' }}
ERPNEXT_CI_USER: ci
PIP_CACHE_DIR: /home/ci/.cache/pip
npm_config_cache: /home/ci/.cache/npm
YARN_CACHE_FOLDER: /home/ci/.cache/yarn
UV_CACHE_DIR: /home/ci/.cache/uv
jobs:
# Build the bench (clone + pip + yarn + assets) and reinstall test_site ONCE, on a free
# GitHub-hosted runner, then publish the whole bench (with a DB dump baked in) as an artifact.
# The expensive, non-parallelisable work happens here exactly once instead of on every shard.
setup:
name: Build & reinstall (setup)
# Dedicated scale set (fat cpu request) so the build+reinstall runs at full speed, uncontended
# by the many thin test shards. Same CI image + /home/ci path + 127.0.0.1 DB as the shards,
# so the packaged bench (and its venv) transplants cleanly.
runs-on: erpnext-arc-setup
timeout-minutes: 40
container:
image: ghcr.io/frappe/erpnext-ci-mariadb:py3.14-node24
credentials:
username: ${{ secrets.GHCR_USERNAME || github.actor }}
password: ${{ secrets.GHCR_TOKEN || github.token }}
defaults:
run:
shell: bash
test:
runs-on: ubuntu-latest
timeout-minutes: 60
env:
TZ: 'Asia/Kolkata'
NODE_ENV: "production"
WITH_COVERAGE: ${{ github.event_name != 'pull_request' }}
strategy:
fail-fast: false
matrix:
container: [1, 2, 3, 4]
name: Python Unit Tests
services:
mysql:
image: mariadb:10.6
env:
TZ: 'Asia/Kolkata'
MARIADB_ROOT_PASSWORD: 'root'
ports:
- 3306:3306
options: --health-cmd="mariadb-admin ping" --health-interval=5s --health-timeout=2s --health-retries=3
steps:
- name: Clone
uses: actions/checkout@v6
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: '3.14'
- name: Check for valid Python & Merge Conflicts
run: |
python -m compileall -fq "${GITHUB_WORKSPACE}"
@@ -83,17 +80,47 @@ jobs:
exit 1
fi
- name: Setup Node
uses: actions/setup-node@v6
with:
node-version: 24
check-latest: true
- name: Add to Hosts
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
# MariaDB runs in-container on a datadir OUTSIDE the bench, because install.sh's next step
# does `rm -rf ~/frappe-bench`. After the reinstall, the datadir is moved into the bench so
# it ships in the artifact — test shards then start an already-loaded server (no restore).
- name: Start DB
run: bash ${GITHUB_WORKSPACE}/.github/helper/start-db.sh
- name: Cache pip
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/*requirements.txt', '**/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-
${{ runner.os }}-
- name: Cache node modules
uses: actions/cache@v4
env:
SKIP_SYSTEM_SETUP: "1"
CI_DB_DATADIR: /home/ci/db-data
cache-name: cache-node-modules
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
- name: Get yarn cache directory path
id: yarn-cache-dir-path
run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
- uses: actions/cache@v4
id: yarn-cache
with:
path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install
run: bash ${GITHUB_WORKSPACE}/.github/helper/install.sh
@@ -102,116 +129,26 @@ jobs:
TYPE: server
FRAPPE_USER: ${{ github.event.inputs.user }}
FRAPPE_BRANCH: ${{ github.event.client_payload.sha || github.event.inputs.branch }}
# Anonymous git to github.com gets throttled to a 401; authenticate the clones.
CI_GITHUB_TOKEN: ${{ github.token }}
DB_HOST: 127.0.0.1
DB_USER_HOST: '%'
WKHTMLTOX_DEB: /tmp/wkhtmltox.deb
SKIP_SYSTEM_SETUP: "1"
SKIP_WKHTMLTOX_SETUP: "1"
- name: Warm up test data
run: |
su -m "${ERPNEXT_CI_USER:-frappe}" -s /bin/bash <<'EOF'
cd ~/frappe-bench/
bench --site test_site run-tests --lightmode --module erpnext.tests.bootstrap_test_data
EOF
# Clean shutdown (consistent InnoDB datadir), then stage it inside the bench for packaging.
- name: Stop DB and stage datadir
run: |
mariadb-admin -h 127.0.0.1 -P 3306 -u root -proot shutdown || true
for _ in $(seq 1 30); do [ -f /home/ci/db-data/mysqld.pid ] || break; sleep 1; done
# Don't bake a dirty datadir — fail if mariadbd didn't finish stopping, rather than ship
# an inconsistent datadir the shards would have to crash-recover.
[ -f /home/ci/db-data/mysqld.pid ] && { echo "mariadbd did not shut down cleanly"; exit 1; }
mv /home/ci/db-data /home/ci/frappe-bench/mariadb-data
# Package the whole bench (apps, venv, node_modules, sites, the DB dump, and hydrate.sh)
# into one artifact for the test shards to consume.
# Single-node hand-off: stage the bench on a node-local hostPath instead of round-tripping
# through GitHub artifact storage (~60s/shard). Setup and shards share the same disk, so
# the shards just untar it locally. NOTE: this assumes one node — a shard on a different
# node could not read this path (then you'd need GitHub artifacts or an NFS/RWX volume).
- name: Stage bench on node (hostPath)
run: |
cp "${GITHUB_WORKSPACE}/.github/helper/hydrate.sh" /home/ci/frappe-bench/hydrate.sh
cp "${GITHUB_WORKSPACE}/.github/helper/start-db.sh" /home/ci/frappe-bench/start-db.sh
mkdir -p /opt/ci-bench-staging
# self-clean: drop bench tars from runs older than 2h
find /opt/ci-bench-staging -maxdepth 1 -name '*.tar.gz' -mmin +120 -delete 2>/dev/null || true
# Exclude .git/node_modules; the mariadb-data datadir IS included (the pre-loaded DB).
tar czpf "/opt/ci-bench-staging/${GITHUB_RUN_ID}.tar.gz" -C /home/ci \
--exclude='.git' --exclude='node_modules' frappe-bench
ls -lh "/opt/ci-bench-staging/${GITHUB_RUN_ID}.tar.gz"
# Fan-out: each shard downloads the bench, untars it, starts MariaDB on the baked datadir, and
# runs its slice of the suite. No clone, no build, no reinstall, no DB dump restore on the shards.
test:
name: Python Unit Tests
needs: setup
runs-on: erpnext-arc
timeout-minutes: 60
container:
image: ghcr.io/frappe/erpnext-ci-mariadb:py3.14-node24
credentials:
username: ${{ secrets.GHCR_USERNAME || github.actor }}
password: ${{ secrets.GHCR_TOKEN || github.token }}
defaults:
run:
shell: bash
strategy:
fail-fast: false
matrix:
container: [1, 2, 3, 4]
steps:
- name: Add to Hosts
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
# Read the bench straight from the node-local hostPath the setup job staged it on — no
# GitHub download. -p preserves the ci (uid 1001) ownership so bench runs as ci cleanly.
- name: Untar bench from node (hostPath)
run: |
tar xzpf "/opt/ci-bench-staging/${GITHUB_RUN_ID}.tar.gz" -C /home/ci
ls -ld /home/ci/frappe-bench
- name: Hydrate (start DB on baked datadir + bench start)
run: bash /home/ci/frappe-bench/hydrate.sh
env:
DB_HOST: 127.0.0.1
SKIP_SYSTEM_SETUP: "1"
- name: Run Tests
run: |
su -m "${ERPNEXT_CI_USER:-frappe}" -s /bin/bash <<'EOF'
cd ~/frappe-bench/
coverage_flag=""
if [ "$WITH_COVERAGE" = "true" ]; then coverage_flag="--with-coverage"; fi
bench --site test_site run-parallel-tests --lightmode --app erpnext \
--total-builds ${{ strategy.job-total }} \
--build-number ${{ matrix.container }} \
$coverage_flag
EOF
run: 'cd ~/frappe-bench/ && bench --site test_site run-parallel-tests --lightmode --app erpnext --total-builds ${{ strategy.job-total }} --build-number ${{ matrix.container }} --with-coverage'
env:
TYPE: server
- name: Show bench output
if: ${{ always() }}
run: cat ~/frappe-bench/bench_start.log || true
- name: Upload coverage data
if: ${{ env.WITH_COVERAGE == 'true' }}
uses: actions/upload-artifact@v4
with:
name: coverage-${{ matrix.container }}
path: /home/ci/frappe-bench/sites/coverage.xml
path: /home/runner/frappe-bench/sites/coverage.xml
coverage:
name: Coverage Wrap Up
needs: [test]
if: ${{ github.event_name != 'pull_request' }}
needs: test
runs-on: ubuntu-latest
steps:
- name: Clone

View File

@@ -1,12 +1,7 @@
name: Server (Postgres)
on:
schedule:
# 03:00 AM IST daily (21:30 UTC the previous day)
- cron: "30 21 * * *"
pull_request:
# 'labeled' so adding the 'postgres' label to an already-open PR re-triggers the run.
types: [opened, reopened, synchronize, labeled]
paths-ignore:
- '**.js'
- '**.md'
@@ -14,7 +9,7 @@ on:
- 'crowdin.yml'
- '.coderabbit.yml'
- '.mergify.yml'
workflow_dispatch:
types: [opened, labelled, synchronize, reopened]
concurrency:
group: server-postgres-develop-${{ github.event_name }}-${{ github.event.number || github.event_name == 'workflow_dispatch' && github.run_id || '' }}
@@ -23,31 +18,41 @@ concurrency:
permissions:
contents: read
# Postgres CI stays on GitHub-hosted (free, full-speed VM per shard) but follows the same fan-out
# we built for MariaDB: build the bench + reinstall ONCE in the setup job, bake the PostgreSQL
# PGDATA into the artifact, and have 4 test shards start Postgres on that datadir — no per-shard
# clone/build/reinstall/restore. Python is pinned so the venv transplants between VMs.
env:
TZ: 'Asia/Kolkata'
NODE_ENV: "production"
PYTHON_VERSION: '3.14'
jobs:
setup:
name: Build & reinstall (setup)
test:
if: ${{ contains(github.event.pull_request.labels.*.name, 'postgres') }}
runs-on: ubuntu-latest
# Runs on the daily schedule (and workflow_dispatch). On PRs it runs ONLY when the PR carries
# the 'postgres' label — the test job needs setup, so it's skipped too when this is.
if: github.event_name != 'pull_request' || contains(github.event.pull_request.labels.*.name, 'postgres')
timeout-minutes: 40
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
container: [1]
name: Python Unit Tests
services:
postgres:
image: postgres:13.3
env:
POSTGRES_PASSWORD: travis
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
steps:
- name: Clone
uses: actions/checkout@v6
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: ${{ env.PYTHON_VERSION }}
python-version: '3.14'
- name: Check for valid Python & Merge Conflicts
run: |
@@ -66,135 +71,48 @@ jobs:
- name: Add to Hosts
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
- name: Cache deps (uv/pip/npm/yarn)
- name: Cache pip
uses: actions/cache@v4
with:
path: |
~/.cache/uv
~/.cache/pip
~/.npm
~/.cache/yarn
key: ${{ runner.os }}-deps-${{ hashFiles('**/*requirements.txt', '**/pyproject.toml', '**/yarn.lock') }}
restore-keys: ${{ runner.os }}-deps-
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/*requirements.txt', '**/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-
${{ runner.os }}-
# Warm-bench cache (the big one): install.sh saves the built base bench — frappe + env +
# node_modules + assets — here as frappe-bench-base-*.tar.zst. Later runs restore it and only
# fast-forward to the live develop SHA + rebuild the delta, so the bench BUILD is near-free and
# only the test_site reinstall (per-run DB, uncacheable) stays slow — matching the self-hosted
# box. The first run after a deps change populates it; every run after that is fast.
- name: Cache warm bench (base build)
- name: Cache node modules
uses: actions/cache@v4
with:
path: ~/bench-cache
key: ${{ runner.os }}-warmbench-v2-${{ hashFiles('**/*requirements.txt', '**/pyproject.toml', '**/yarn.lock') }}
restore-keys: ${{ runner.os }}-warmbench-v2-
# Postgres runs in-runner on a PGDATA OUTSIDE the bench (install.sh wipes ~/frappe-bench);
# after the reinstall it's moved into the bench so it ships in the artifact.
- name: Start DB
run: bash ${GITHUB_WORKSPACE}/.github/helper/start-db.sh
env:
DB: postgres
CI_DB_DATADIR: /home/runner/pgdata
cache-name: cache-node-modules
with:
path: ~/.npm
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-${{ env.cache-name }}-
${{ runner.os }}-build-
${{ runner.os }}-
- name: Get yarn cache directory path
id: yarn-cache-dir-path
run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
- uses: actions/cache@v4
id: yarn-cache
with:
path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install
run: bash ${GITHUB_WORKSPACE}/.github/helper/install.sh
env:
DB: postgres
TYPE: server
FRAPPE_BRANCH: develop
# Anonymous git to github.com gets throttled to a 401; authenticate the clones.
CI_GITHUB_TOKEN: ${{ github.token }}
BENCH_CACHE_DIR: /home/runner/bench-cache
- name: Warm up test data
run: |
cd ~/frappe-bench/
bench --site test_site run-tests --lightmode --module erpnext.tests.bootstrap_test_data
- name: Stop DB and stage datadir
run: |
PG_BIN=$(ls -d /usr/lib/postgresql/*/bin | sort -V | tail -1)
# Clean shutdown so the baked datadir is consistent. Do NOT swallow a failed stop with
# `|| true`: moving and tarring a still-running cluster ships a torn datadir the shards
# cannot crash-recover (full_page_writes is off). Fail the job instead — mirrors the
# MariaDB sister's "don't bake a dirty datadir" guard.
"$PG_BIN/pg_ctl" -D /home/runner/pgdata -m fast -w stop
mv /home/runner/pgdata /home/runner/frappe-bench/pgdata
- name: Package bench for test shards
run: |
cp "${GITHUB_WORKSPACE}/.github/helper/hydrate.sh" /home/runner/frappe-bench/hydrate.sh
cp "${GITHUB_WORKSPACE}/.github/helper/start-db.sh" /home/runner/frappe-bench/start-db.sh
tar czpf "${GITHUB_WORKSPACE}/bench.tar.gz" -C /home/runner \
--exclude='.git' --exclude='node_modules' frappe-bench
ls -lh "${GITHUB_WORKSPACE}/bench.tar.gz"
- name: Upload bench artifact
uses: actions/upload-artifact@v4
with:
name: bench-pg
path: bench.tar.gz
retention-days: 1
compression-level: 0
test:
name: Python Unit Tests
needs: setup
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
container: [1, 2, 3, 4]
steps:
- name: Download bench artifact
uses: actions/download-artifact@v4
with:
name: bench-pg
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Add to Hosts
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
# The bench CLI (frappe-bench) and redis are global/system tools — not in the bench tarball.
# The setup runner got them via install.sh; the MariaDB shards get them from the arc5 image.
# GitHub-hosted PG shards install them here (cheap vs the build+reinstall that setup did once).
- name: Install shard runtime (bench CLI + redis + wkhtmltopdf)
run: |
pip install frappe-bench
command -v redis-server >/dev/null || { sudo apt-get update -qq && sudo apt-get install -y -qq redis-server; }
# wkhtmltopdf (patched-qt build) for print-format / PDF tests — same .deb install.sh uses.
if ! command -v wkhtmltopdf >/dev/null; then
wget -qO /tmp/wkhtmltox.deb https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6.1-2/wkhtmltox_0.12.6.1-2.jammy_amd64.deb
sudo apt-get install -y -qq /tmp/wkhtmltox.deb
fi
- name: Untar bench
run: |
tar xzpf "${GITHUB_WORKSPACE}/bench.tar.gz" -C /home/runner
ls -ld /home/runner/frappe-bench
- name: Hydrate (start Postgres on the baked datadir)
run: bash /home/runner/frappe-bench/hydrate.sh
env:
DB: postgres
DB_HOST: 127.0.0.1
- name: Run Tests
run: |
cd ~/frappe-bench/
# print-format / PDF tests are engine-independent (they exercise wkhtmltopdf rendering,
# not postgres SQL — the MariaDB CI already covers them). They only fetch the static asset
# bundles from http://test_site:8000/assets/..., so a plain static file server over sites/
# satisfies wkhtmltopdf without the frappe web server (which never bound on a bare runner).
( cd ~/frappe-bench/sites && nohup python3 -m http.server 8000 --bind 127.0.0.1 > ~/frappe-bench/web.log 2>&1 & )
for _ in $(seq 1 15); do (exec 3<>/dev/tcp/127.0.0.1/8000) 2>/dev/null && { exec 3>&- 3<&-; break; }; sleep 1; done
bench --site test_site run-parallel-tests --lightmode --app erpnext \
--total-builds ${{ strategy.job-total }} --build-number ${{ matrix.container }}
run: cd ~/frappe-bench/ && bench --site test_site run-parallel-tests --app erpnext --use-orchestrator
env:
TYPE: server
CI_BUILD_ID: ${{ github.run_id }}
ORCHESTRATOR_URL: http://test-orchestrator.frappe.io

File diff suppressed because one or more lines are too long

View File

@@ -88,8 +88,7 @@ pull_request_rules:
actions:
merge:
method: squash
commit_message_format:
title: pr-title
body: pr-body
merge_queue:
queue_controls_comment: false
commit_message_template: |
{{ title }} (#{{ number }})
{{ body }}

View File

@@ -48,6 +48,7 @@ repos:
cypress/.*|
.*node_modules.*|
.*boilerplate.*|
erpnext/public/js/controllers/.*|
erpnext/templates/pages/order.js|
erpnext/templates/includes/.*
)$
@@ -65,18 +66,6 @@ repos:
- id: ruff-format
name: "Run ruff formatter"
- repo: local
hooks:
- id: postgres-compat
name: "PostgreSQL compatibility (static check)"
description: "Flags MySQL-only SQL that breaks on Postgres; the label-gated PG test job is the backstop for semantic divergences."
entry: .github/helper/postgres_compat.py
language: script
files: ^erpnext/.*\.py$
# patches/ are historical, version-gated migrations (skipped on fresh Postgres installs);
# out of scope for the always-on gate.
exclude: ^erpnext/patches/
ci:
autoupdate_schedule: weekly
skip: []

View File

@@ -7,7 +7,6 @@ erpnext/accounts/ @ruthra-kumar
erpnext/assets/ @khushi8112
erpnext/regional @ruthra-kumar
erpnext/selling @ruthra-kumar
banking/ @nikkothari22
erpnext/buying/ @rohitwaghchaure @mihir-kandoi
erpnext/maintenance/ @rohitwaghchaure @mihir-kandoi

View File

@@ -14,52 +14,52 @@
"@dnd-kit/core": "^6.3.1",
"@dnd-kit/sortable": "^10.0.0",
"@dnd-kit/utilities": "^3.2.2",
"@tailwindcss/vite": "^4.3.2",
"@tailwindcss/vite": "^4.3.0",
"@tanstack/react-table": "^8.21.3",
"@tanstack/react-virtual": "^3.13.24",
"@vitejs/plugin-react": "^6.0.3",
"@vitejs/plugin-react": "^6.0.1",
"chrono-node": "^2.9.1",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"cmdk": "^1.1.1",
"date-fns": "^4.1.0",
"dayjs": "^1.11.20",
"frappe-react-sdk": "^1.17.1",
"frappe-react-sdk": "^1.15.0",
"fuse.js": "^7.3.0",
"jotai": "^2.20.2",
"jotai-family": "^1.1.0",
"jotai": "^2.20.0",
"jotai-family": "^1.0.1",
"lodash.isplainobject": "^4.0.6",
"lucide-react": "^1.14.0",
"radix-ui": "^1.6.1",
"react": "^19.2.7",
"radix-ui": "^1.4.3",
"react": "^19.2.6",
"react-currency-input-field": "^4.0.5",
"react-day-picker": "9.14.0",
"react-dom": "^19.2.7",
"react-dom": "^19.2.6",
"react-dropzone": "^15.0.0",
"react-hook-form": "^7.75.0",
"react-hotkeys-hook": "^5.3.2",
"react-markdown": "^10.1.0",
"react-router": "^8.3.0",
"react-router": "^7.15.0",
"react-router-dom": "^7.15.0",
"rehype-raw": "^7.0.0",
"remark-gfm": "^4.0.1",
"safe-expr-eval": "^1.0.4",
"sonner": "^2.0.7",
"tailwind-merge": "^3.5.0",
"tailwindcss": "^4.3.0",
"tw-animate-css": "^1.4.0",
"usehooks-ts": "^3.1.1",
"vite": "^8.2.1"
"vite": "^8.0.11"
},
"devDependencies": {
"@eslint/js": "^9.39.5",
"@eslint/js": "^9.39.1",
"@types/node": "^25.3.0",
"@types/react": "^19.2.7",
"@types/react-dom": "^19.2.3",
"eslint": "^10.8.1",
"eslint": "^9.39.1",
"eslint-plugin-react-hooks": "^7.1.1",
"eslint-plugin-react-refresh": "^0.5.3",
"eslint-plugin-react-refresh": "^0.4.24",
"globals": "^16.5.0",
"typescript": "~5.9.3",
"typescript-eslint": "^8.67.0"
"typescript-eslint": "^8.48.0"
}
}

View File

@@ -1,5 +1,5 @@
import { lazy, useEffect } from 'react'
import { BrowserRouter, Navigate, Route, Routes } from 'react-router'
import { BrowserRouter, Navigate, Route, Routes } from 'react-router-dom'
import { FrappeProvider } from 'frappe-react-sdk'
import { Toaster } from '@/components/ui/sonner'
import BankReconciliation from '@/pages/BankReconciliation'

View File

@@ -9,7 +9,6 @@ import Fuse from "fuse.js"
import { ChevronDownIcon } from "lucide-react"
import { useLayoutEffect, useMemo, useRef, useState } from "react"
import { FormControl } from "../ui/form"
import useResetScrollOnSearch from "@/hooks/useResetScrollOnSearch"
export interface AccountsDropdownProps {
@@ -105,10 +104,6 @@ const AccountsDropdown = ({ root_type, report_type, account_type, value, onChang
const buttonRef = useRef<HTMLButtonElement>(null)
// Searching replaces the grouped list with a short result list, so pin the scroll back to
// the top - otherwise the auto-selected first result can be out of view.
const listRef = useResetScrollOnSearch(search)
const [width, setWidth] = useState(320)
useLayoutEffect(() => {
@@ -158,7 +153,7 @@ const AccountsDropdown = ({ root_type, report_type, account_type, value, onChang
<PopoverContent className="p-0" style={{ minWidth: width }} align="start">
<Command shouldFilter={false} className="w-full">
<CommandInput placeholder={_("Search account...")} onValueChange={setSearch} value={search} />
<CommandList ref={listRef}>
<CommandList>
<CommandEmpty>{_("No accounts found.")}</CommandEmpty>
{recommendedAccounts.length > 0 && (

View File

@@ -10,7 +10,6 @@ import { ChevronDownIcon, ExternalLink } from "lucide-react";
import { Button } from "../ui/button";
import { cn } from "@/lib/utils";
import { Command, CommandEmpty, CommandGroup, CommandInput, CommandItem, CommandList } from "../ui/command";
import useResetScrollOnSearch from "@/hooks/useResetScrollOnSearch";
import _ from "@/lib/translate";
import ErrorBanner from "../ui/error-banner";
import MarkdownRenderer from "../ui/markdown";
@@ -150,10 +149,6 @@ const LinkFieldCombobox = ({
const buttonRef = useRef<HTMLButtonElement>(null)
// Results change as the search runs, so pin the scroll back to the top to keep the
// auto-selected first result in view.
const listRef = useResetScrollOnSearch(searchInput)
const [width, setWidth] = useState(320)
useLayoutEffect(() => {
@@ -269,7 +264,7 @@ const LinkFieldCombobox = ({
{error && <ErrorBanner error={error} />}
<Command shouldFilter={false} className="w-full">
<CommandInput placeholder={placeholder} onValueChange={setSearchInput} />
<CommandList ref={listRef}>
<CommandList>
<CommandEmpty>{isLoading ? _("Loading...") : _("No results found.")}</CommandEmpty>
<CommandGroup>
{items?.map((result) => (
@@ -277,7 +272,7 @@ const LinkFieldCombobox = ({
<span className="font-medium">
{result.label || result.value}
</span>
{result.description && <span className="text-p-xs text-ink-gray-5">
{result.description && <span className="text-xs text-ink-gray-5">
<MarkdownRenderer content={result.description} />
</span>}
</CommandItem>

View File

@@ -6,13 +6,13 @@ import { Progress } from "@/components/ui/progress"
import { useGetAccountClosingBalance, useGetAccountClosingBalanceAsPerStatement, useGetAccountOpeningBalance, useGetUnreconciledTransactions } from "./utils"
import { flt, formatCurrency } from "@/lib/numbers"
import { Skeleton } from "@/components/ui/skeleton"
import { StatContainer, StatLabel, StatValue } from "@/components/ui/stats"
import { Edit, Info, Trash2 } from "lucide-react"
import { H4, Paragraph } from "@/components/ui/typography"
import { HoverCard, HoverCardContent, HoverCardTrigger } from "@/components/ui/hover-card"
import { getCompanyCurrency } from "@/lib/company"
import _ from "@/lib/translate"
import { cn } from "@/lib/utils"
import { Dialog, DialogClose, DialogContent, DialogDescription, DialogFooter, DialogHeader, DialogTitle } from "@/components/ui/dialog"
import { Dialog, DialogClose, DialogContent, DialogDescription, DialogFooter, DialogHeader, DialogTitle, DialogTrigger } from "@/components/ui/dialog"
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip"
import { formatDate } from "@/lib/date"
import { Form } from "@/components/ui/form"
@@ -26,109 +26,50 @@ import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "@
import { toast } from "sonner"
import ErrorBanner from "@/components/ui/error-banner"
const useBankCurrency = () => {
const bankAccount = useAtomValue(selectedBankAccountAtom)
return bankAccount?.account_currency ?? getCompanyCurrency(bankAccount?.company ?? '')
}
/**
* One line of the balance summary - label on the left, figure right-aligned.
*
* `items-baseline` keeps the figure on the label's FIRST line, so a row carrying a `subLabel`
* (the statement row's "As of <date>" note) doesn't centre its value against both lines.
*/
const BalanceRow = ({ label, info, subLabel, emphasis, children }: {
label: React.ReactNode
info?: React.ReactNode
subLabel?: React.ReactNode
emphasis?: boolean
children: React.ReactNode
}) => (
<div className="flex items-baseline justify-between gap-3">
<span className="flex min-w-0 flex-col gap-1.5">
<span className={cn("flex items-center gap-1 whitespace-nowrap text-xs text-ink-gray-6",
emphasis && "font-medium text-ink-gray-7")}>
{label}
{info}
</span>
{subLabel}
</span>
<div className="flex flex-col items-end">{children}</div>
</div>
)
/**
* Type styles for a figure. Shared so an interactive figure can put them on the <button>
* ITSELF rather than on a nested span: Tailwind's preflight sets `font: inherit` on buttons,
* which resets line-height too, so a button wrapping a `text-sm` span gets a taller strut than
* the span and the row grows - visible as extra space above a baseline-aligned row.
*/
const BALANCE_VALUE_CLASSES = "font-numeric text-sm tabular-nums text-ink-gray-8"
const BalanceValue = ({ children, emphasis, tone, className }: { children: React.ReactNode, emphasis?: boolean, tone?: 'red', className?: string }) => (
<span className={cn(BALANCE_VALUE_CLASSES,
emphasis && "font-semibold",
tone === 'red' && "text-ink-red-3",
className)}>
{children}
</span>
)
const BalanceSkeleton = () => <Skeleton className="h-4 w-24 rounded-sm" />
/**
* Balances and progress for the selected bank account, laid out like the totals block of an
* invoice. This sits beside the bank picker rather than in a row of its own (saves vertical
* space) and outside the picker's horizontal scroll area, so the figures being reconciled
* against can never scroll out of view.
*/
const BankAccountBalancePanel = () => {
const BankBalance = () => {
const bankAccount = useAtomValue(selectedBankAccountAtom)
if (!bankAccount) {
return null
}
return (
<div className="flex w-72 shrink-0 flex-col justify-center gap-2.5 border-s border-outline-gray-2 ps-4">
{/* Names the account these figures belong to - the picker scrolls, so the
highlighted card can't be relied on as the referent. */}
<span
className="truncate text-xs font-medium text-ink-gray-7"
title={bankAccount.account_name}>
{bankAccount.account_name}
</span>
<OpeningBalanceRow />
<SystemClosingBalanceRow />
<StatementClosingBalanceRow />
<Separator />
<DifferenceRow />
<ReconciledRow />
<div className="flex justify-between">
<div className="w-[80%] flex flex-wrap justify-between gap-2 pe-8 border-e-border border-e">
<OpeningBalance />
<ClosingBalance />
<ClosingBalanceAsPerStatement />
<Difference />
</div>
<ReconcileProgress />
</div>
)
}
const OpeningBalanceRow = () => {
const currency = useBankCurrency()
const OpeningBalance = () => {
const bankAccount = useAtomValue(selectedBankAccountAtom)
const { data, isLoading } = useGetAccountOpeningBalance()
return <BalanceRow label={_("Opening Balance")}>
{isLoading ? <BalanceSkeleton /> : <BalanceValue>{formatCurrency(flt(data?.message, 2), currency)}</BalanceValue>}
</BalanceRow>
return <StatContainer className="min-w-48">
<StatLabel>{_("Opening Balance")}</StatLabel>
{isLoading ? <Skeleton className="w-[150px] h-5 rounded-sm" /> : <StatValue className="font-numeric">{formatCurrency(flt(data?.message, 2), bankAccount?.account_currency ?? getCompanyCurrency(bankAccount?.company ?? ''))}</StatValue>}
</StatContainer>
}
const SystemClosingBalanceRow = () => {
const currency = useBankCurrency()
const ClosingBalance = () => {
const bankAccount = useAtomValue(selectedBankAccountAtom)
const { data, isLoading } = useGetAccountClosingBalance()
return (
<BalanceRow
label={_("Closing (system)")}
info={
<StatContainer className="min-w-48">
<div className="flex items-start gap-1">
<StatLabel>
{_("Closing Balance as per system")}
</StatLabel>
<HoverCard openDelay={100}>
<HoverCardTrigger>
<Info className="size-3.5 text-ink-gray-6" />
<Info className="size-3.5 text-ink-gray-6 -mt-px" />
</HoverCardTrigger>
<HoverCardContent className="w-96" align="start" side="right">
<H4 className="text-base">{_("Closing balance as per system")}</H4>
@@ -143,111 +84,15 @@ const SystemClosingBalanceRow = () => {
</Paragraph>
</HoverCardContent>
</HoverCard>
}
>
{isLoading ? <BalanceSkeleton /> : <BalanceValue>{formatCurrency(flt(data?.message, 2), currency)}</BalanceValue>}
</BalanceRow>
</div>
{isLoading ? <Skeleton className="w-[150px] h-5 rounded-sm" /> : <StatValue className="font-numeric">{formatCurrency(flt(data?.message, 2), bankAccount?.account_currency ?? getCompanyCurrency(bankAccount?.company ?? ''))}</StatValue>}
</StatContainer>
)
}
const StatementClosingBalanceRow = () => {
const Difference = () => {
const bankAccount = useAtomValue(selectedBankAccountAtom)
const currency = useBankCurrency()
const dates = useAtomValue(bankRecDateAtom)
const setValue = useSetAtom(bankRecClosingBalanceAtom(bankAccount?.name ?? ''))
const { data, isLoading } = useGetAccountClosingBalanceAsPerStatement({
onSuccess: (data) => {
if (data?.message && data?.message?.balance) {
setValue({
value: data?.message?.balance,
stringValue: data?.message?.balance.toString()
})
}
}
})
const isDateSame = data?.message?.date === dates.toDate
// The server uses the returned date to distinguish an unset balance from a saved zero.
const hasBalance = Boolean(data?.message?.date)
const [isOpen, setIsOpen] = useState(false)
const tooltip = hasBalance
? _("Click to change the closing balance as per statement")
: _("Click to set the closing balance as per statement")
return (
<BalanceRow
label={_("Closing (statement)")}
// The pencil sits beside the label, mirroring the info icon on the row above, so
// the figure stays a plain right-aligned number in line with every other row.
info={
<Tooltip>
<TooltipTrigger asChild>
{/* `p-0`: Tailwind's preflight gives buttons `appearance: button` but
doesn't reset padding, so a bare button picks up the UA's ~1px 6px
and knocks this row out of step with its neighbours. */}
<button
type='button'
aria-label={tooltip}
onClick={() => setIsOpen(true)}
className="cursor-pointer p-0 text-ink-gray-5 transition-colors hover:text-ink-gray-7">
<Edit className="size-3.5" />
</button>
</TooltipTrigger>
<TooltipContent>{tooltip}</TooltipContent>
</Tooltip>
}
subLabel={!isDateSame && data?.message.date
? <span className="whitespace-nowrap text-2xs font-medium text-ink-red-3">
{_("As of {0}", [formatDate(data?.message?.date ?? '', 'Do MMM YYYY')])}
</span>
: undefined}
>
{/* Deliberately NOT a flex container: a flex box's baseline doesn't resolve to its
text, so the row's `items-baseline` couldn't line this up with the label. As a
plain inline button its baseline is the figure's own, like every other row.
"Set" gets the same treatment as a figure - it stands in for one. */}
{isLoading
? <BalanceSkeleton />
: <Tooltip>
<TooltipTrigger asChild>
{/* The figure styles live on the button itself - see
BALANCE_VALUE_CLASSES. `p-0` because preflight leaves the UA's
button padding in place. */}
<button
type='button'
aria-label={tooltip}
onClick={() => setIsOpen(true)}
className={cn(BALANCE_VALUE_CLASSES,
"cursor-pointer p-0 underline decoration-outline-gray-5 decoration-dashed underline-offset-4",
"transition-colors hover:decoration-ink-gray-8")}>
{hasBalance ? formatCurrency(flt(data?.message?.balance, 2), currency) : _("Set")}
</button>
</TooltipTrigger>
<TooltipContent>{tooltip}</TooltipContent>
</Tooltip>}
<Dialog open={isOpen} onOpenChange={setIsOpen}>
<DialogContent className="min-w-xl">
<ClosingBalanceForm
defaultBalance={data?.message?.balance ?? 0}
date={dates.toDate}
bankAccount={bankAccount}
onClose={() => setIsOpen(false)}
/>
</DialogContent>
</Dialog>
</BalanceRow>
)
}
const DifferenceRow = () => {
const bankAccount = useAtomValue(selectedBankAccountAtom)
const currency = useBankCurrency()
const { data, isLoading } = useGetAccountClosingBalance()
@@ -257,15 +102,16 @@ const DifferenceRow = () => {
const isError = difference !== 0
return <BalanceRow label={_("Difference")} emphasis>
{isLoading
? <BalanceSkeleton />
: <BalanceValue emphasis tone={isError ? 'red' : undefined}>{formatCurrency(difference, currency)}</BalanceValue>}
</BalanceRow>
return <StatContainer className="w-fit text-end sm:min-w-56">
<StatLabel className="text-end">{_("Difference")}</StatLabel>
{isLoading ? <Skeleton className="w-[150px] h-5 self-end rounded-sm" /> : <StatValue className={isError ? 'text-ink-red-3 font-numeric' : 'font-numeric'}>
{formatCurrency(difference,
bankAccount?.account_currency ?? getCompanyCurrency(bankAccount?.company ?? ''))
}</StatValue>}
</StatContainer>
}
/** Reconciliation progress through the selected date range: a count plus a slim bar. */
const ReconciledRow = () => {
const ReconcileProgress = () => {
const bankAccount = useAtomValue(selectedBankAccountAtom)
@@ -286,14 +132,75 @@ const ReconciledRow = () => {
const progress = (totalCount ? reconciledCount / totalCount : 0) * 100
return <div className="flex flex-col gap-1.5">
<BalanceRow label={_("Reconciled")}>
<BalanceValue>{reconciledCount} / {totalCount ?? 0}</BalanceValue>
</BalanceRow>
<Progress value={progress} max={100} size="sm" />
return <div className="w-[18%] flex flex-col gap-1 items-end">
<div className="w-full">
<Progress
value={progress}
max={100}
size="md"
label="Progress"
hint
hintText={`${reconciledCount} / ${totalCount} ${_("reconciled")}`} />
</div>
</div>
}
const ClosingBalanceAsPerStatement = () => {
const bankAccount = useAtomValue(selectedBankAccountAtom)
const dates = useAtomValue(bankRecDateAtom)
const setValue = useSetAtom(bankRecClosingBalanceAtom(bankAccount?.name ?? ''))
const { data, isLoading } = useGetAccountClosingBalanceAsPerStatement({
onSuccess: (data) => {
if (data?.message && data?.message?.balance) {
setValue({
value: data?.message?.balance,
stringValue: data?.message?.balance.toString()
})
}
}
})
const isDateSame = data?.message?.date === dates.toDate
const [isOpen, setIsOpen] = useState(false)
return <StatContainer className="min-w-48">
<StatLabel>{_("Closing Balance as per statement")}</StatLabel>
<div className="flex flex-col gap-2 items-start">
<Dialog open={isOpen} onOpenChange={setIsOpen}>
<DialogTrigger>
<Tooltip>
<TooltipTrigger asChild>
<div className="flex items-center gap-4 underline cursor-pointer underline-offset-6" role="button">
{isLoading ? <Skeleton className="w-[150px] h-5 rounded-sm" /> : <StatValue className="font-numeric">{formatCurrency(flt(data?.message?.balance, 2), bankAccount?.account_currency ?? getCompanyCurrency(bankAccount?.company ?? ''))}</StatValue>}
<Edit className="w-4 h-4" />
</div>
</TooltipTrigger>
<TooltipContent>
{_("Click to set the closing balance as per statement")}
</TooltipContent>
</Tooltip>
</DialogTrigger>
<DialogContent className="min-w-xl">
<ClosingBalanceForm
defaultBalance={data?.message?.balance ?? 0}
date={dates.toDate}
bankAccount={bankAccount}
onClose={() => setIsOpen(false)}
/>
</DialogContent>
</Dialog>
{!isDateSame && data?.message.date && <span className="text-xs font-medium text-ink-red-3">{_("As of {0}", [formatDate(data?.message?.date ?? '', 'Do MMM YYYY')])}</span>}
</div>
</StatContainer>
}
const ClosingBalanceForm = ({ defaultBalance, date, bankAccount, onClose }: { defaultBalance: number, date: string, bankAccount: SelectedBank | null, onClose: VoidFunction }) => {
const { mutate } = useSWRConfig()
@@ -343,7 +250,7 @@ const ClosingBalanceForm = ({ defaultBalance, date, bankAccount, onClose }: { de
{_("Enter the closing balance you see in your bank statement for {0} as of the {1}", [bankAccount?.account_name ?? bankAccount?.name ?? '', formatDate(date, 'Do MMM YYYY')])}
</DialogDescription>
</DialogHeader>
{error && <div className="py-2"><ErrorBanner error={error} /></div>}
{error && <ErrorBanner error={error} />}
<div className="py-4">
<CurrencyFormField
name="balance"
@@ -395,7 +302,7 @@ const ClosingBalancesList = ({ bankAccount, date }: { bankAccount: SelectedBank
return <div>
<Separator className="my-8" />
<p className="text-p-sm text-center pb-2">{_("Balances as per bank statement before {0}", [formatDate(date, 'Do MMM YYYY')])}</p>
<p className="text-sm text-center">{_("Balances as per bank statement before {0}", [formatDate(date, 'Do MMM YYYY')])}</p>
<Table>
<TableHeader>
<TableRow>
@@ -424,4 +331,4 @@ const ClosingBalancesList = ({ bankAccount, date }: { bankAccount: SelectedBank
}
export default BankAccountBalancePanel
export default BankBalance

View File

@@ -2,6 +2,7 @@ import { useAtomValue } from "jotai"
import { MissingFiltersBanner } from "./MissingFiltersBanner"
import { bankRecDateAtom, SelectedBank, selectedBankAccountAtom } from "./bankRecAtoms"
import { useCurrentCompany } from "@/hooks/useCurrentCompany"
import { Paragraph } from "@/components/ui/typography"
import type { ColumnDef } from "@tanstack/react-table"
import { useCallback, useMemo, useState } from "react"
import { useFrappeGetCall, useFrappePostCall, useSWRConfig } from "frappe-react-sdk"
@@ -25,7 +26,6 @@ import { Form } from "@/components/ui/form"
import { useForm } from "react-hook-form"
import { DateField } from "@/components/ui/form-elements"
import { Empty, EmptyMedia, EmptyHeader, EmptyTitle, EmptyDescription } from "@/components/ui/empty"
import MarkdownRenderer from "@/components/ui/markdown"
const BankClearanceSummary = () => {
const bankAccount = useAtomValue(selectedBankAccountAtom)
@@ -203,14 +203,14 @@ const BankClearanceSummaryView = () => {
[accountCurrency, bankAccount, companyID, mutate, onCopy],
)
const content = _("Below is a list of all accounting entries posted against the bank account {0} between {1} and {2}.", [`<strong>${bankAccount?.account}</strong>`, `<strong>${formattedFromDate}</strong>`, `<strong>${formattedToDate}</strong>`])
return <div className="space-y-4 py-2">
return <div className="flex min-h-0 flex-1 flex-col space-y-4 py-2">
<div className="shrink-0">
<span className="text-p-sm">
<MarkdownRenderer content={content} />
</span>
<div>
<Paragraph className="text-sm">
<span dangerouslySetInnerHTML={{
__html: _("Below is a list of all accounting entries posted against the bank account {0} between {1} and {2}.", [`<strong>${bankAccount?.account}</strong>`, `<strong>${formattedFromDate}</strong>`, `<strong>${formattedToDate}</strong>`])
}} />
</Paragraph>
</div>
{error && <ErrorBanner error={error} />}
@@ -220,9 +220,8 @@ const BankClearanceSummaryView = () => {
data={data.message.result}
columns={clearanceColumns}
getRowId={(row) => `${row.payment_entry}-${row.posting_date}`}
className="min-h-0 flex-1"
maxHeight="none"
scrollAreaClassName="flex-1"
maxHeight="calc(100vh - 200px)"
scrollAreaClassName="min-h-[calc(100vh-200px)]"
emptyState={_("No rows to display.")}
/>
) : null}

View File

@@ -18,7 +18,6 @@ import { useMultiFileUploadProgress } from "@/hooks/useMultiFileUploadProgress"
import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "@/components/ui/table"
import { Checkbox } from "@/components/ui/checkbox"
import { ArrowDownRight, ArrowUpRight, Plus, Trash2 } from "lucide-react"
import { evaluateAmountFormula } from "@/lib/amountFormula"
import { flt, formatCurrency } from "@/lib/numbers"
import { cn } from "@/lib/utils"
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip"
@@ -216,13 +215,38 @@ const BankEntryForm = ({ selectedTransaction }: { selectedTransaction: Unreconci
})
} else {
const transactionAmount = selectedTransaction.unallocated_amount ?? 0
/**
* The debit and credit amounts can also be expressions - like "transaction_amount * 0.5"
* So we need to compute the value of the expression
* We can use the eval function to do this. But we need to expose certain variables to the expression.
* One of them is transaction_amount which is the unallocated amount of the selected transaction
* @param expression - The expression to compute
* @returns The computed value
*/
const computeExpression = (expression: string) => {
const script = `
const transaction_amount = ${selectedTransaction.unallocated_amount ?? 0}
${expression};
`
let value = 0;
try {
value = window.eval(script);
} catch (error: unknown) {
console.error(error);
value = 0;
}
return value;
}
if (!acc?.debit && !acc?.credit) {
hasTotallyEmptyRowEarlier = true;
}
const computedDebit = acc?.debit ? flt(evaluateAmountFormula(acc.debit, transactionAmount), 2) : 0
const computedCredit = acc?.credit ? flt(evaluateAmountFormula(acc.credit, transactionAmount), 2) : 0
const computedDebit = acc?.debit ? flt(computeExpression(acc.debit), 2) : 0
const computedCredit = acc?.credit ? flt(computeExpression(acc.credit), 2) : 0
totalDebits = flt(totalDebits + computedDebit, 2)
totalCredits = flt(totalCredits + computedCredit, 2)

View File

@@ -74,10 +74,7 @@ const BankPicker = ({ className }: { className?: string }) => {
}
return (
<div
// No trailing padding: it would sit inside the fade region, so the mask would
// spend itself on empty space and the last card would stop short of the balance
// panel instead of fading towards it. The column gap provides the separation.
className={cn("flex gap-3 items-stretch w-full overflow-x-auto scroll-fade-x",
className={cn("flex gap-3 items-stretch w-full overflow-x-auto pe-4",
banks?.length > 4 ? 'pb-2' : '', className,
)}
style={{
@@ -111,12 +108,12 @@ const BankPickerItem = ({ bank }: { bank: SelectedBank }) => {
role="button"
title={`Select ${bank.account_name}`}
onClick={onSelect}
// `shrink-0`: this is a horizontally scrolling row, so cards keep their own width
// instead of being compressed to fit the container.
className={cn('w-60 shrink-0 rounded-md border border-outline-gray-1 p-2 overflow-hidden cursor-pointer transition-colors',
className={cn('rounded-md border border-outline-gray-1 max-w-60 min-w-60 p-2 overflow-hidden cursor-pointer',
isSelected ? 'border-outline-gray-5 bg-surface-gray-1' : 'hover:bg-surface-gray-1'
)}
>
<BankLogo bank={bank} className="mb-2" />
<div className="flex flex-col gap-1">

View File

@@ -5,179 +5,107 @@ import { AVAILABLE_TIME_PERIODS, formatDate, getDatesForTimePeriod, TimePeriod }
import { Button } from '@/components/ui/button'
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover'
import { ChevronDownIcon, ChevronLeftIcon, ChevronRight } from 'lucide-react'
import { Command, CommandGroup, CommandInput, CommandItem, CommandList } from '@/components/ui/command'
import { Command, CommandEmpty, CommandInput, CommandItem, CommandList } from '@/components/ui/command'
import { parse } from "chrono-node"
import { Calendar } from '@/components/ui/calendar'
import useFiscalYear from '@/hooks/useFiscalYear'
import dayjs from 'dayjs'
import _ from '@/lib/translate'
import { useDirection } from '@/components/ui/direction'
import useResetScrollOnSearch from '@/hooks/useResetScrollOnSearch'
const DATE_FORMAT = 'YYYY-MM-DD'
/** Current fiscal year plus this many previous ones, for quarter/year options. */
const PREVIOUS_FISCAL_YEARS = 2
type DateOption = {
/** Stable id - used as the cmdk value and the React key. */
key: string
label: string
translatedLabel: string
fromDate: string
toDate: string
format: string
/** Extra terms to match against, beyond the labels and dates. */
keywords?: string[]
/** Whether to show this option when the search box is empty. */
isDefault?: boolean
}
/**
* Fiscal years keep the same month/day boundaries year on year, so previous years can be
* derived by subtracting whole years instead of fetching them. Works for both Jan-Dec and
* Apr-Mar style fiscal years.
*/
const fiscalYearLabel = (start: dayjs.Dayjs, end: dayjs.Dayjs) =>
start.year() === end.year() ? `${start.year()}` : `${start.year()}-${end.year()}`
const BankRecDateFilter = () => {
const [bankRecDate, setBankRecDate] = useAtom(bankRecDateAtom)
const { fiscalYear } = useFiscalYear()
const { data: fiscalYear } = useFiscalYear()
const today = useMemo(() => dayjs().format(DATE_FORMAT), [])
const allOptions = useMemo(() => {
const standardOptions: DateOption[] = AVAILABLE_TIME_PERIODS.map((period) => {
const timePeriodOptions = useMemo(() => {
const standardOptions = AVAILABLE_TIME_PERIODS.map((period) => {
const dates = getDatesForTimePeriod(period)
return {
key: period,
label: period,
translatedLabel: dates.translatedLabel ?? _(period),
fromDate: dates.fromDate,
toDate: dates.toDate,
format: dates.format,
isDefault: true,
translatedLabel: dates.translatedLabel
}
})
if (!fiscalYear) {
return standardOptions
}
if (fiscalYear?.message) {
// For a fiscal year, we need to replace "Last Year", "This Year", and add options for quarters
const fiscalYearStart = fiscalYear.message.year_start_date
const fiscalYearEnd = fiscalYear.message.year_end_date
const currentStart = dayjs(fiscalYear.year_start_date)
const currentEnd = dayjs(fiscalYear.year_end_date)
const quarterOptions: DateOption[] = []
const fiscalYearOptions: DateOption[] = []
// Static literals so the translation extractor can find them.
const quarterLabels = [_("Q1"), _("Q2"), _("Q3"), _("Q4")]
for (let yearsAgo = 0; yearsAgo <= PREVIOUS_FISCAL_YEARS; yearsAgo++) {
const start = currentStart.subtract(yearsAgo, 'year')
const end = currentEnd.subtract(yearsAgo, 'year')
// Keep the real name for the current year; derive it for the earlier ones.
const yearLabel = yearsAgo === 0 ? fiscalYear.name : fiscalYearLabel(start, end)
for (let quarter = 0; quarter < 4; quarter++) {
const quarterStart = start.add(quarter * 3, 'month')
// End the day before the next quarter starts, clamped to the fiscal year end
// so a short fiscal year can't spill over.
const nextQuarterStart = start.add((quarter + 1) * 3, 'month')
const quarterEnd = nextQuarterStart.subtract(1, 'day').isAfter(end)
? end
: nextQuarterStart.subtract(1, 'day')
if (quarterStart.isAfter(end)) continue
quarterOptions.push({
key: `Q${quarter + 1}-${yearLabel}`,
label: `Q${quarter + 1}: ${yearLabel}`,
translatedLabel: `${quarterLabels[quarter]}: ${yearLabel}`,
fromDate: quarterStart.format(DATE_FORMAT),
toDate: quarterEnd.format(DATE_FORMAT),
format: 'MMM YYYY',
keywords: ['quarter', `q${quarter + 1}`, yearLabel],
// Only the current fiscal year's quarters clutter the default list;
// older ones stay searchable.
isDefault: yearsAgo === 0,
})
const q1 = {
label: `Q1: ${fiscalYear.message.name}`,
translatedLabel: `${_("Q1")}: ${fiscalYear.message.name}`,
fromDate: fiscalYearStart,
toDate: dayjs(fiscalYearStart).add(3, 'month').format('YYYY-MM-DD'),
format: 'MMM YYYY'
}
const label = yearsAgo === 0
? 'This Fiscal Year'
: yearsAgo === 1
? 'Last Fiscal Year'
: `FY ${yearLabel}`
const q2 = {
label: `Q2: ${fiscalYear.message.name}`,
translatedLabel: `${_("Q2")}: ${fiscalYear.message.name}`,
fromDate: dayjs(fiscalYearStart).add(3, 'month').format('YYYY-MM-DD'),
toDate: dayjs(fiscalYearStart).add(6, 'month').format('YYYY-MM-DD'),
format: 'MMM YYYY'
}
fiscalYearOptions.push({
key: `fiscal-year-${yearLabel}`,
label,
translatedLabel: yearsAgo <= 1 ? _(label) : `${_("FY")} ${yearLabel}`,
fromDate: start.format(DATE_FORMAT),
toDate: end.format(DATE_FORMAT),
format: 'MMM YYYY',
keywords: ['fiscal year', yearLabel],
isDefault: yearsAgo <= 1,
const q3 = {
label: `Q3: ${fiscalYear.message.name}`,
translatedLabel: `${_("Q3")}: ${fiscalYear.message.name}`,
fromDate: dayjs(fiscalYearStart).add(6, 'month').format('YYYY-MM-DD'),
toDate: dayjs(fiscalYearStart).add(9, 'month').format('YYYY-MM-DD'),
format: 'MMM YYYY'
}
const q4 = {
label: `Q4: ${fiscalYear.message.name}`,
translatedLabel: `${_("Q4")}: ${fiscalYear.message.name}`,
fromDate: dayjs(fiscalYearStart).add(9, 'month').format('YYYY-MM-DD'),
toDate: fiscalYearEnd,
format: 'MMM YYYY'
}
const thisYear = {
label: `This Fiscal Year`,
translatedLabel: `${_("This Fiscal Year")}`,
fromDate: fiscalYearStart,
toDate: fiscalYearEnd,
format: 'MMM YYYY'
}
const lastYear = {
label: `Last Fiscal Year`,
translatedLabel: `${_("Last Fiscal Year")}`,
fromDate: dayjs(fiscalYearStart).subtract(1, 'year').format('YYYY-MM-DD'),
toDate: dayjs(fiscalYearEnd).subtract(1, 'year').format('YYYY-MM-DD'),
format: 'MMM YYYY'
}
// Sort the options so that we get "This Month", "Last Month", quarters, fiscal year, then the rest of the standard options
const topRankedItems = standardOptions.filter((option) => {
return option.label === "This Month" || option.label === "Last Month"
})
const bottomRankedItems = standardOptions.filter((option) => {
return option.label !== "This Month" && option.label !== "Last Month"
})
return [...topRankedItems, q1, q2, q3, q4, thisYear, lastYear, ...bottomRankedItems]
}
// "This Month"/"Last Month" first, then quarters and fiscal years, then the rest.
const topRanked = standardOptions.filter((o) => o.label === 'This Month' || o.label === 'Last Month')
const bottomRanked = standardOptions.filter((o) => o.label !== 'This Month' && o.label !== 'Last Month')
return [...topRanked, ...quarterOptions, ...fiscalYearOptions, ...bottomRanked]
return standardOptions
}, [fiscalYear])
// Reconciliation only looks backwards, so a period that hasn't started is never useful.
const selectableOptions = useMemo(
() => allOptions.filter((option) => option.fromDate <= today),
[allOptions, today],
)
const [open, setOpen] = useState(false)
const [value, setValue] = useState("")
// We filter ourselves (`shouldFilter={false}`) so that the parsed-date suggestion can be a
// real CommandItem alongside the predefined options, and keyboard navigation covers both.
const filteredOptions = useMemo(() => {
const query = value.trim().toLowerCase()
if (!query) {
return selectableOptions.filter((option) => option.isDefault)
}
const tokens = query.split(/\s+/)
return selectableOptions.filter((option) => {
const haystack = [
option.label,
option.translatedLabel,
...(option.keywords ?? []),
option.fromDate,
option.toDate,
].join(' ').toLowerCase()
return tokens.every((token) => haystack.includes(token))
})
}, [selectableOptions, value])
const parsedOption = useMemo(() => parseDateRange(value), [value])
// Filtering shortens the list, so pin the scroll back to the top to keep the
// auto-selected first option in view.
const listRef = useResetScrollOnSearch(value)
// Don't show a parsed suggestion that duplicates an option already in the list.
const showParsedOption = parsedOption
&& !filteredOptions.some((o) => o.fromDate === parsedOption.fromDate && o.toDate === parsedOption.toDate)
const timePeriod: TimePeriod | string = useMemo(() => {
if (bankRecDate.fromDate && bankRecDate.toDate) {
for (const period of allOptions) {
// Check if the from and to dates match any predefined time period
for (const period of timePeriodOptions) {
if (period.fromDate === bankRecDate.fromDate && period.toDate === bankRecDate.toDate) {
return period.label;
}
@@ -186,11 +114,10 @@ const BankRecDateFilter = () => {
} else {
return "Date Range";
}
}, [bankRecDate.fromDate, bankRecDate.toDate, allOptions]);
}, [bankRecDate.fromDate, bankRecDate.toDate, timePeriodOptions]);
const handleTimePeriodChange = (fromDate: string, toDate: string) => {
setBankRecDate({ fromDate, toDate })
setValue("")
setOpen(false)
}
@@ -203,9 +130,7 @@ const BankRecDateFilter = () => {
const direction = useDirection()
const RangeArrow = direction === 'ltr'
? <ChevronRight className='text-[12px] text-ink-gray-5/70' />
: <ChevronLeftIcon className='text-[12px] text-ink-gray-5/70' />
return <div className='flex items-center'>
<Popover open={open} onOpenChange={setOpen}>
@@ -216,57 +141,30 @@ const BankRecDateFilter = () => {
size='md'
className='rounded-e-none border-e-0'
role="combobox">
{allOptions.find((period) => period.label === timePeriod)?.translatedLabel ?? _(timePeriod)}
{timePeriodOptions.find((period) => period.label === timePeriod)?.translatedLabel ?? _(timePeriod)}
<ChevronDownIcon />
</Button>
</PopoverTrigger>
<PopoverContent className="w-84 p-1" align='start'>
<Command shouldFilter={false}>
<Command>
<CommandInput placeholder={_("e.g. Last 3 weeks, Q1, May 2025")} onValueChange={setValue} value={value} />
<CommandList ref={listRef} className='max-h-80'>
{showParsedOption && parsedOption && (
<CommandGroup heading={_("Matched date")}>
<CommandItem
value='parsed-date-range'
className='flex justify-between'
onSelect={() => handleTimePeriodChange(parsedOption.fromDate, parsedOption.toDate)}>
<span className='max-w-[45%] truncate'>{value}</span>
<span className='text-xs text-ink-gray-5 flex items-center gap-1 text-end whitespace-nowrap'>
{parsedOption.fromDate === parsedOption.toDate
? formatDate(parsedOption.fromDate, 'Do MMM YYYY')
: <>{formatDate(parsedOption.fromDate, 'Do MMM YY')} {RangeArrow} {formatDate(parsedOption.toDate, 'Do MMM YY')}</>}
</span>
</CommandItem>
</CommandGroup>
)}
{filteredOptions.length > 0 && (
<CommandGroup>
{filteredOptions.map((period) => (
<CommandItem
key={period.key}
value={period.key}
className='flex justify-between'
onSelect={() => handleTimePeriodChange(period.fromDate, period.toDate)}>
<span>
{period.translatedLabel}
</span>
<span className='text-xs text-ink-gray-5 flex items-center gap-1 text-end whitespace-nowrap'>
{formatDate(period.fromDate, period.format)} {RangeArrow} {formatDate(period.toDate, period.format)}
</span>
</CommandItem>
))}
</CommandGroup>
)}
{!showParsedOption && filteredOptions.length === 0 && (
<div className='p-2 text-sm text-ink-gray-5'>
{_("No results found")}
</div>
)}
<CommandInput placeholder="e.g. Last 3 weeks" onValueChange={setValue} value={value} />
<CommandList className='max-h-fit'>
<CommandEmpty className='text-start p-2 hover:bg-surface-gray-1'>
<EmptyState onSelect={handleTimePeriodChange} value={value} />
</CommandEmpty>
{timePeriodOptions.map((period) => (
<CommandItem key={period.label} className='flex justify-between' onSelect={() => handleTimePeriodChange(period.fromDate, period.toDate)}>
<span>
{period.translatedLabel ?? _(period.label)}
</span>
<span className='text-xs text-ink-gray-5 flex items-center gap-1 text-end whitespace-nowrap'>
{formatDate(period.fromDate, period.format)} {direction === 'ltr' ? <ChevronRight className='text-[12px] text-ink-gray-5/70' /> : <ChevronLeftIcon className='text-[12px] text-ink-gray-5/70' />} {formatDate(period.toDate, period.format)}
</span>
</CommandItem>
))}
</CommandList>
</Command>
@@ -301,97 +199,77 @@ const BankRecDateFilter = () => {
}
const referentialKeywords = ["last", "this", "next", "previous"]
const EmptyState = ({ onSelect, value }: { onSelect: (fromDate: string, toDate: string) => void, value: string }) => {
/** chrono exposes `knownValues` on ParsingComponents but doesn't type it publicly. */
const knownValuesOf = (components: unknown): Record<string, number> =>
(components as { knownValues?: Record<string, number> })?.knownValues ?? {}
const dates = useMemo(() => {
if (value) {
// Try parsing the value
const parsedDate = parse(value, undefined, { forwardDate: false })
/**
* How far back a parsed date must move to land in the past. Reconciliation only ever looks
* backwards, so an ambiguous input that chrono resolves into the future - "December" typed in
* September, or a bare weekday like "Friday" - is pulled to its most recent past occurrence.
* An explicitly stated year is respected; a range that is still future gets discarded later.
*
* This returns a shift rather than a date so that a range can be moved as a single unit -
* shifting its start and end independently would distort or invert it.
*/
const pastShift = (date: Date, knownValues: Record<string, number>) => {
const today = dayjs()
let candidate = dayjs(date)
if (parsedDate && parsedDate.length > 0) {
const startDate = parsedDate[0].start.date()
const endDate = parsedDate[0].end?.date()
if (!candidate.isAfter(today, 'date') || knownValues.year !== undefined) {
return { amount: 0, unit: 'year' as const }
if (!endDate) {
const today = new Date()
// If today is greater than the start date, use today as the end date
if (startDate.getTime() > today.getTime()) {
return { fromDate: today, toDate: startDate }
} else {
// Check if the user only wants a specific month like "May 2025"
// If the "known values" just has month and year, then we need to get the first day of the month and the last day of the month
// @ts-expect-error - "Known Values" is available in the start "ParsingComponents"
if (parsedDate[0].start.knownValues?.month && !parsedDate[0].start.knownValues?.day) {
return {
fromDate: startDate,
toDate: dayjs(startDate).endOf('month').toDate()
}
// @ts-expect-error - "Known Values" is available in the start "ParsingComponents"
} else if (parsedDate[0].start.knownValues?.month && parsedDate[0].start.knownValues?.day && !referentialKeywords.some(keyword => value.toLowerCase().includes(keyword))) {
// If month and day is known, then we should not assume that the user wants to get everything until today
return {
fromDate: startDate,
toDate: startDate,
}
}
return {
fromDate: startDate,
toDate: today
}
}
} else {
return { fromDate: startDate, toDate: endDate }
}
}
}
}, [value])
const onClick = (fromDate: Date, toDate: Date) => {
onSelect(formatDate(fromDate, 'YYYY-MM-DD'), formatDate(toDate, 'YYYY-MM-DD'))
}
// A bare weekday repeats weekly, everything else (month/day) repeats yearly.
const unit = knownValues.weekday !== undefined && knownValues.day === undefined
? 'day' as const
: 'year' as const
const step = unit === 'day' ? 7 : 1
let amount = 0
const isEqual = dates?.fromDate && dates?.toDate && dayjs(dates.fromDate).isSame(dates.toDate, 'date')
for (let i = 0; i < 200 && candidate.isAfter(today, 'date'); i++) {
candidate = candidate.subtract(step, unit)
amount += step
}
return { amount, unit }
return <div>
{dates ?
<div className='flex gap-2 items-center justify-between cursor-pointer' onClick={() => onClick(dates.fromDate, dates.toDate)}>
<span className='text-sm text-ink-gray-5 max-w-[30%]'>
{value}
</span>
{isEqual ? <span className='text-xs text-ink-gray-5 text-balance flex items-center gap-1'>
{formatDate(dates.fromDate, 'Do MMM YYYY')}
</span> :
<span className='text-xs text-ink-gray-5 flex items-center gap-1'>
{formatDate(dates.fromDate, 'Do MMM YY')} <ChevronRight size='16' className='text-ink-gray-5' /> {formatDate(dates.toDate, 'Do MMM YY')}
</span>}
</div> :
<span className='text-sm text-ink-gray-5'>
No results found
</span>
}
</div>
}
/**
* Parse free text into a past date range, or return undefined when it can't be parsed or
* resolves entirely into the future.
*/
const parseDateRange = (value: string): { fromDate: string, toDate: string } | undefined => {
if (!value.trim()) return undefined
const parsedDate = parse(value, undefined, { forwardDate: false })
if (!parsedDate || parsedDate.length === 0) return undefined
const result = parsedDate[0]
const startKnownValues = knownValuesOf(result.start)
// Anchor the shift on the start and apply it to both ends, so an explicit range like
// "1st Sept to 30th Sept" keeps its shape instead of having only its end rolled back.
const shift = pastShift(result.start.date(), startKnownValues)
const startDate = dayjs(result.start.date()).subtract(shift.amount, shift.unit).toDate()
const endDate = result.end
? dayjs(result.end.date()).subtract(shift.amount, shift.unit).toDate()
: undefined
const today = new Date()
let range: { fromDate: Date, toDate: Date }
if (endDate) {
const endKnownValues = knownValuesOf(result.end)
// chrono ends "Apr 2025 to Jun 2025" on the 1st of June, but the user means all of it.
const rangeEnd = endKnownValues.month && !endKnownValues.day
? dayjs(endDate).endOf('month').toDate()
: endDate
range = { fromDate: startDate, toDate: rangeEnd }
} else if (startKnownValues.month && !startKnownValues.day) {
// The user only wants a specific month like "May 2025" - span the whole month
range = { fromDate: dayjs(startDate).startOf('month').toDate(), toDate: dayjs(startDate).endOf('month').toDate() }
} else if (startKnownValues.month && startKnownValues.day && !referentialKeywords.some(keyword => value.toLowerCase().includes(keyword))) {
// If month and day is known, then we should not assume that the user wants to get everything until today
range = { fromDate: startDate, toDate: startDate }
} else {
range = { fromDate: startDate, toDate: today }
}
// A range that hasn't started yet is never useful for reconciliation. A range that merely
// ends in the future is kept as typed, the same way "This Month" spans the whole month.
if (dayjs(range.fromDate).isAfter(today, 'date')) return undefined
if (dayjs(range.toDate).isBefore(range.fromDate, 'date')) {
range = { fromDate: range.toDate, toDate: range.fromDate }
}
return {
fromDate: dayjs(range.fromDate).format(DATE_FORMAT),
toDate: dayjs(range.toDate).format(DATE_FORMAT),
}
}
export default BankRecDateFilter
export default BankRecDateFilter

View File

@@ -2,6 +2,7 @@ import { useAtomValue } from "jotai"
import { MissingFiltersBanner } from "./MissingFiltersBanner"
import { bankRecDateAtom, selectedBankAccountAtom } from "./bankRecAtoms"
import { useCurrentCompany } from "@/hooks/useCurrentCompany"
import { Paragraph } from "@/components/ui/typography"
import { useCallback, useMemo } from "react"
import type { ColumnDef } from "@tanstack/react-table"
import { useFrappeGetCall } from "frappe-react-sdk"
@@ -18,7 +19,6 @@ import _ from "@/lib/translate"
import { toast } from "sonner"
import { useCopyToClipboard } from "usehooks-ts"
import { Empty, EmptyDescription, EmptyHeader, EmptyMedia, EmptyTitle } from "@/components/ui/empty"
import MarkdownRenderer from "@/components/ui/markdown"
const BankReconciliationStatement = () => {
const bankAccount = useAtomValue(selectedBankAccountAtom)
@@ -189,30 +189,28 @@ const BankReconciliationStatementView = () => {
return data.message.result.filter((row: BankClearanceSummaryEntry) => Boolean(row.payment_entry))
}, [data])
const content = _("Below is a list of all entries posted against the bank account {0} which have not been cleared till {1}.", [`<strong>${bankAccount?.account}</strong>`, `<strong>${formatDate(dates.toDate)}</strong>`])
return <div className="space-y-4 py-2">
return <div className="flex min-h-0 flex-1 flex-col space-y-4 py-2">
<div className="shrink-0">
<span className="text-p-sm">
<MarkdownRenderer content={content} />
</span>
<div>
<Paragraph className="text-sm">
<span dangerouslySetInnerHTML={{
__html: _("Below is a list of all entries posted against the bank account {0} which have not been cleared till {1}.", [`<strong>${bankAccount?.account}</strong>`, `<strong>${formatDate(dates.toDate)}</strong>`])
}} />
</Paragraph>
</div>
{error && <ErrorBanner error={error} />}
{data && <div className="shrink-0"><SummarySection data={data} /></div>}
{data && <SummarySection data={data} />}
{data && data.message.result.length > 0 && (
<div className="flex min-h-0 flex-1 flex-col space-y-2">
<p className="shrink-0 text-ink-gray-5 text-sm">{_("Bank Reconciliation Statement")}</p>
<div className="space-y-2">
<p className="text-ink-gray-5 text-sm">{_("Bank Reconciliation Statement")}</p>
<ListView
data={statementRows}
columns={statementColumns}
getRowId={(row) => row.payment_entry}
className="min-h-0 flex-1"
maxHeight="none"
scrollAreaClassName="flex-1"
maxHeight="min(70vh, 640px)"
emptyState={_("No entries with a payment document in this list.")}
/>
</div>

View File

@@ -1,6 +1,7 @@
import { useAtomValue, useSetAtom } from "jotai"
import { MissingFiltersBanner } from "./MissingFiltersBanner"
import { bankRecDateAtom, bankRecUnreconcileModalAtom, selectedBankAccountAtom } from "./bankRecAtoms"
import { Paragraph } from "@/components/ui/typography"
import { formatDate } from "@/lib/date"
import { ListView, type ListViewColumnMeta } from "@/components/ui/list-view"
import { formatCurrency, getCurrencyFormatInfo } from "@/lib/numbers"
@@ -22,7 +23,6 @@ import { useCallback, useMemo, useState } from "react"
import { Link } from "react-router"
import { Empty, EmptyTitle, EmptyHeader, EmptyMedia, EmptyDescription, EmptyContent } from "@/components/ui/empty"
import { InputGroup, InputGroupAddon } from "@/components/ui/input-group"
import MarkdownRenderer from "@/components/ui/markdown"
const BankTransactions = () => {
const selectedBank = useAtomValue(selectedBankAccountAtom)
@@ -243,14 +243,14 @@ const BankTransactionListView = () => {
}, [data, search, amountFilter, typeFilter, status])
const content = _("Below is a list of all bank transactions imported in the system for the bank account {0} between {1} and {2}.", [`<strong>${bankAccount?.account_name}</strong>`, `<strong>${formattedFromDate}</strong>`, `<strong>${formattedToDate}</strong>`])
return <div className="space-y-2 py-2">
return <div className="flex min-h-0 flex-1 flex-col space-y-2 py-2">
<div className="flex shrink-0 gap-2 justify-between items-center">
<span className="text-p-sm">
<MarkdownRenderer content={content} />
</span>
<div className="flex gap-2 justify-between items-center">
<Paragraph className="text-sm">
<span dangerouslySetInnerHTML={{
__html: _("Below is a list of all bank transactions imported in the system for the bank account {0} between {1} and {2}.", [`<strong>${bankAccount?.account_name}</strong>`, `<strong>${formattedFromDate}</strong>`, `<strong>${formattedToDate}</strong>`])
}} />
</Paragraph>
<Button size='md' variant='subtle' asChild>
<Link to="/statement-importer">
@@ -278,9 +278,8 @@ const BankTransactionListView = () => {
data={filteredResults}
columns={transactionColumns}
getRowId={(row) => row.name}
className="min-h-0 flex-1"
maxHeight="none"
scrollAreaClassName="flex-1"
maxHeight="calc(100vh - 200px)"
scrollAreaClassName="min-h-[calc(100vh-200px)]"
emptyState={<Empty>
<EmptyMedia>
<ListIcon />

View File

@@ -19,22 +19,13 @@ import {
import { cn } from "@/lib/utils"
import _ from "@/lib/translate"
import { selectedBankAccountAtom } from "./bankRecAtoms"
import { useFrappeGetDocList } from "frappe-react-sdk"
import ErrorBanner from "@/components/ui/error-banner"
const CompanySelector = ({ onChange }: { onChange?: (company: string) => void }) => {
const [open, setOpen] = useState(false)
const [searchQuery, setSearchQuery] = useState("")
const { data: companies, error } = useFrappeGetDocList("Company", {
limit: 0,
fields: ["name"],
}, 'company_list', {
revalidateOnFocus: false,
revalidateOnReconnect: false,
})
const options = companies?.map((company: { name: string }) => company.name) || []
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const options = window.frappe?.boot?.docs?.filter((doc: Record<string, any>) => doc.doctype === ":Company").map((company: Record<string, any>) => company.name) || []
const setSelectedCompany = useSetAtom(selectedCompanyAtom)
const setSelectedBankAccount = useSetAtom(selectedBankAccountAtom)
@@ -51,10 +42,6 @@ const CompanySelector = ({ onChange }: { onChange?: (company: string) => void })
}
}
if (error) {
return <ErrorBanner error={error} />
}
return (<Popover open={open} onOpenChange={setOpen}>
<PopoverTrigger asChild>
<Button

View File

@@ -2,6 +2,7 @@ import { useAtomValue } from "jotai"
import { MissingFiltersBanner } from "./MissingFiltersBanner"
import { bankRecDateAtom, selectedBankAccountAtom } from "./bankRecAtoms"
import { useCurrentCompany } from "@/hooks/useCurrentCompany"
import { Paragraph } from "@/components/ui/typography"
import type { ColumnDef } from "@tanstack/react-table"
import { useCallback, useMemo } from "react"
import { useFrappeGetCall, useFrappePostCall } from "frappe-react-sdk"
@@ -17,7 +18,6 @@ import { PartyPopper } from "lucide-react"
import ErrorBanner from "@/components/ui/error-banner"
import _ from "@/lib/translate"
import { Empty, EmptyTitle, EmptyDescription, EmptyMedia, EmptyHeader } from "@/components/ui/empty"
import MarkdownRenderer from "@/components/ui/markdown"
const IncorrectlyClearedEntries = () => {
const companyID = useCurrentCompany()
@@ -177,36 +177,34 @@ const IncorrectlyClearedEntriesView = () => {
[accountCurrency, onClearClick],
)
const content = _("This report shows all entries in the system where the <strong>clearance date is before the posting date</strong> which is incorrect.")
return <div className="space-y-4 py-2">
const entriesContent = _("Entries below have a posting date after {0} but the clearance date is before {1}.", [`<strong>${formattedToDate}</strong>`, `<strong>${formattedToDate}</strong>`])
return <div className="flex min-h-0 flex-1 flex-col space-y-4 py-2">
<div className="shrink-0">
<span className="text-p-sm">
<MarkdownRenderer content={content} />
<div>
<Paragraph className="text-sm">
<span dangerouslySetInnerHTML={{
__html: _("This report shows all entries in the system where the <strong>clearance date is before the posting date</strong> which is incorrect.")
}} />
<br />
{data && data.message.result.length > 0 && <span>
<MarkdownRenderer content={entriesContent} />
<span dangerouslySetInnerHTML={{
__html: _("Entries below have a posting date after {0} but the clearance date is before {1}.", [`<strong>${formattedToDate}</strong>`, `<strong>${formattedToDate}</strong>`])
}} />
<br />
{_("You can reset the clearing dates of these entries here.")}
</span>}
</span>
</Paragraph>
</div>
{error && <ErrorBanner error={error} />}
{data && data.message.result.length > 0 && (
<div className="flex min-h-0 flex-1 flex-col space-y-2">
<p className="shrink-0 text-ink-gray-5 text-sm">{_("Incorrectly cleared entries as per the report.")}</p>
<div className="space-y-2">
<p className="text-ink-gray-5 text-sm">{_("Incorrectly cleared entries as per the report.")}</p>
<ListView
data={data.message.result}
columns={incorrectlyClearedColumns}
getRowId={(row) => `${row.payment_entry}-${row.posting_date}`}
className="min-h-0 flex-1"
maxHeight="none"
scrollAreaClassName="flex-1"
maxHeight="min(70vh, 640px)"
emptyState={_("No rows to display.")}
/>
</div>

View File

@@ -37,7 +37,7 @@ import { Link } from "react-router"
import { Alert, AlertDescription, AlertTitle } from "@/components/ui/alert"
import { InputGroup, InputGroupAddon, InputGroupText } from "@/components/ui/input-group"
const MatchAndReconcile = () => {
const MatchAndReconcile = ({ contentHeight }: { contentHeight: number }) => {
const selectedBank = useAtomValue(selectedBankAccountAtom)
if (!selectedBank) {
@@ -52,15 +52,15 @@ const MatchAndReconcile = () => {
}
return <>
<div className="flex min-h-0 flex-1 items-stretch space-x-2" >
<div className="flex min-h-0 flex-1 flex-col">
<H4 className="shrink-0 text-sm font-medium">{_("Unreconciled Transactions")}</H4>
<UnreconciledTransactions />
<div className={`flex items-start space-x-2`} >
<div className="flex-1">
<H4 className="text-sm font-medium">{_("Unreconciled Transactions")}</H4>
<UnreconciledTransactions contentHeight={contentHeight} />
</div>
<Separator orientation="vertical" className="self-stretch" />
<div className="flex min-h-0 flex-1 flex-col px-1">
<H4 className="shrink-0 text-sm font-medium">{_("Match or Create")}</H4>
<VouchersSection />
<Separator orientation="vertical" style={{ minHeight: `${contentHeight}px` }} />
<div className="flex-1 px-1">
<H4 className="text-sm font-medium">{_("Match or Create")}</H4>
<VouchersSection contentHeight={contentHeight} />
</div>
</div>
<TransferModal />
@@ -69,19 +69,16 @@ const MatchAndReconcile = () => {
</>
}
/**
* TanStack requires `estimateSize` for initial scroll range; `measureElement` on each row sets
* the real height. The scroll container fills its flex parent rather than taking a pixel
* height - the virtualizer observes its own rect, so it stays correct across resizes and any
* layout change above it.
*/
/** TanStack requires `estimateSize` for initial scroll range; `measureElement` on each row sets the real height. */
function VirtualizedListBody<T>({
items,
height,
getItemKey,
children,
estimateSize = 74,
}: {
items: T[]
height: number
getItemKey: (item: T, index: number) => string | number
children: (item: T, index: number) => React.ReactNode
estimateSize?: number
@@ -103,7 +100,8 @@ function VirtualizedListBody<T>({
return (
<div
ref={scrollRef}
className="min-h-0 flex-1 overflow-auto contain-strict"
className="overflow-auto contain-strict"
style={{ height }}
>
<div
className="relative w-full"
@@ -125,7 +123,7 @@ function VirtualizedListBody<T>({
)
}
const UnreconciledTransactions = () => {
const UnreconciledTransactions = ({ contentHeight }: { contentHeight: number }) => {
const bankAccount = useAtomValue(selectedBankAccountAtom)
const currency = bankAccount?.account_currency ?? getCompanyCurrency(bankAccount?.company ?? '')
@@ -189,13 +187,14 @@ const UnreconciledTransactions = () => {
}
const hasFilters = search !== '' || typeFilter !== 'All' || amountFilter.value !== 0
const listHeight = contentHeight - 72
if (isLoading) {
return <UnreconciledTransactionsLoadingState />
}
return <div className="flex min-h-0 flex-1 flex-col space-y-1">
<div className="flex py-2 w-full gap-2 shrink-0">
return <div className="space-y-1">
<div className="flex py-2 w-full gap-2">
<InputGroup variant='outline'>
<label className="sr-only">{_("Search transactions")}</label>
@@ -279,6 +278,7 @@ const UnreconciledTransactions = () => {
<VirtualizedListBody
items={results}
height={listHeight}
estimateSize={74}
getItemKey={(transaction) => transaction.name}
>
@@ -381,7 +381,7 @@ const UnreconciledTransactionItem = ({ transaction }: { transaction: Unreconcile
}
const VouchersSection = () => {
const VouchersSection = ({ contentHeight }: { contentHeight: number }) => {
const selectedBank = useAtomValue(selectedBankAccountAtom)
const selectedTransactions = useAtomValue(bankRecSelectedTransactionAtom(selectedBank?.name || ''))
@@ -402,8 +402,8 @@ const VouchersSection = () => {
return <OptionsForMultipleTransactions transactions={selectedTransactions} />
}
return <div className="mt-2 flex min-h-0 flex-1 flex-col">
<OptionsForSingleTransaction transaction={selectedTransactions[0]} />
return <div style={{ minHeight: contentHeight }} className="mt-2">
<OptionsForSingleTransaction transaction={selectedTransactions[0]} contentHeight={contentHeight} />
</div>
}
@@ -535,11 +535,11 @@ const OptionsForMultipleTransactions = ({ transactions }: { transactions: Unreco
}
const OptionsForSingleTransaction = ({ transaction }: { transaction: UnreconciledTransaction }) => {
const OptionsForSingleTransaction = ({ transaction, contentHeight }: { transaction: UnreconciledTransaction, contentHeight: number }) => {
const { setTransferModalOpen, setRecordPaymentModalOpen, setRecordJournalEntryModalOpen } = useKeyboardShortcuts()
return <div className="flex min-h-0 flex-1 flex-col gap-3">
return <div className="flex flex-col gap-3">
<TooltipProvider>
<div className="flex items-center justify-between pt-2">
<div className="flex gap-4 justify-center">
@@ -602,7 +602,7 @@ const OptionsForSingleTransaction = ({ transaction }: { transaction: Unreconcile
</div>
</TooltipProvider>
{transaction.matched_transaction_rule && <RuleAction transaction={transaction} />}
<VouchersForTransaction transaction={transaction} />
<VouchersForTransaction transaction={transaction} contentHeight={contentHeight} />
</div>
}
@@ -774,11 +774,12 @@ const RuleAction = ({ transaction }: { transaction: UnreconciledTransaction }) =
)
}
const VouchersForTransaction = ({ transaction }: { transaction: UnreconciledTransaction }) => {
const VouchersForTransaction = ({ transaction, contentHeight }: { transaction: UnreconciledTransaction, contentHeight: number }) => {
const { data: vouchers, isLoading, error } = useGetVouchersForTransaction(transaction)
const voucherList = vouchers?.message ?? []
const listHeight = contentHeight - 120
if (error) {
return <ErrorBanner error={error} />
@@ -800,8 +801,8 @@ const VouchersForTransaction = ({ transaction }: { transaction: UnreconciledTran
</div>
}
return <div className="relative flex min-h-0 flex-1 flex-col space-y-2">
<div className="flex shrink-0 items-center gap-2 text-sm text-ink-gray-5">
return <div className="relative space-y-2">
<div className="flex items-center gap-2 text-sm text-ink-gray-5">
<Separator className="flex-1" />
<span>or</span>
<Separator className="flex-1" />
@@ -817,6 +818,7 @@ const VouchersForTransaction = ({ transaction }: { transaction: UnreconciledTran
</Empty>}
<VirtualizedListBody
items={voucherList}
height={listHeight}
estimateSize={121}
getItemKey={(voucher) => voucher.name}
>

View File

@@ -11,7 +11,6 @@ import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "@
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip"
import { H4, Paragraph } from "@/components/ui/typography"
import { today } from "@/lib/date"
import { evaluateAmountFormula } from "@/lib/amountFormula"
import _ from "@/lib/translate"
import { cn } from "@/lib/utils"
import { BankTransactionRule } from "@/types/Accounts/BankTransactionRule"
@@ -446,10 +445,11 @@ const AmountFormulaRenderer = ({ value }: { value?: string }) => {
// If it's a string and cannot be a number, then show it as a formula
if (isNaN(Number(value))) {
let calculatedValue = "";
try {
calculatedValue = String(evaluateAmountFormula(value ?? "", 200));
calculatedValue = window.eval(`const transaction_amount = 200; ${value}`);
} catch (error: unknown) {
console.error(error);
calculatedValue = "Error";

View File

@@ -59,8 +59,8 @@ const SelectedTransactionDetails = ({ transaction, showAccount = false, account
</div>
</div>
<div className='flex flex-col gap-1'>
<span className='text-p-sm'>{transaction.description}</span>
{transaction.reference_number ? <span className='text-p-sm text-ink-gray-5'>{_("Ref")}: {transaction.reference_number}</span> : null}
<span className='text-sm'>{transaction.description}</span>
{transaction.reference_number ? <span className='text-sm text-ink-gray-5'>{_("Ref")}: {transaction.reference_number}</span> : null}
{showAccount && account ? <span className='text-sm text-ink-gray-5'>{_("GL Account")}: {account}</span> : null}
</div>

View File

@@ -490,7 +490,7 @@ const RecommendedTransferAccount = ({ transaction, onAccountChange }: { transact
<Calendar size='16px' />
<span className='text-sm'>{formatDate(data.message.date, 'Do MMM YYYY')}</span>
</div>
<span className='text-p-sm line-clamp-1' title={data.message.description}>{data.message.description}</span>
<span className='text-sm line-clamp-1' title={data.message.description}>{data.message.description}</span>
</div>
</div>
</div>

View File

@@ -231,7 +231,7 @@ export const BANK_LOGOS: { keywords: string[], logo: string, locale?: string[],
{
keywords: ['Federal Bank'],
logo: 'Federal_Bank.png',
logoDark: 'Federal_Bank-Dark.png',
logoDark: 'Federal_Bank-dark.png',
locale: ['India']
},
{

View File

@@ -14,7 +14,7 @@ import { Tooltip, TooltipContent, TooltipTrigger } from '@/components/ui/tooltip
import { useFrappeEventListener, useFrappePostCall } from 'frappe-react-sdk'
import { toast } from 'sonner'
import ErrorBanner from '@/components/ui/error-banner'
import { Link, useNavigate } from 'react-router'
import { Link, useNavigate } from 'react-router-dom'
import { useMemo, useState } from 'react'
import { Progress } from '@/components/ui/progress'
import { useSetAtom } from 'jotai'
@@ -83,13 +83,10 @@ const StatementDetails = ({ data }: Props) => {
}
// `progress` is a percentage (drives the bar); `current`/`total` are actual counts.
const [progress, setProgress] = useState(0)
const [imported, setImported] = useState({ current: 0, total: 0 })
useFrappeEventListener("bank-rec-statement-import-progress", (event) => {
setProgress(event.progress)
setImported({ current: event.current ?? 0, total: event.total ?? 0 })
})
const file_name = data.doc.file.split("/").pop() ?? ""
@@ -115,9 +112,7 @@ const StatementDetails = ({ data }: Props) => {
{data.doc.status === 'Completed' ? <Badge theme='green'>{_("Completed")}</Badge> :
<Button onClick={onImport} disabled={loading || data.final_transactions?.length === 0} size='sm' type='button'>
{loading ? <Loader2Icon className='size-4 animate-spin' /> : null}
{loading ? _("Importing...") : data.final_transactions?.length === 1
? _("Import 1 transaction")
: _("Import {0} transactions", [data.final_transactions?.length?.toString() || "0"])}</Button>
{loading ? _("Importing...") : _("Import {0} transactions", [data.final_transactions?.length?.toString() || "0"])}</Button>
}
</div>
<div className='flex items-start gap-4'>
@@ -134,9 +129,7 @@ const StatementDetails = ({ data }: Props) => {
</div>
{progress > 0 && <div className='flex flex-col gap-2'><Progress value={progress} max={100} size="lg" />
<span className='text-sm'>{imported.total === 1
? _("Importing 1 transaction")
: _("Importing {0} of {1} transactions", [imported.current.toString(), imported.total.toString()])}
<span className='text-sm'>{_("Importing {0} transactions", [progress.toString()])}
</span>
</div>}

View File

@@ -387,7 +387,7 @@ function ListViewInner<TData>({
)}
role="columnheader"
>
<div className="min-w-0 flex-1 truncate leading-snug">
<div className="min-w-0 flex-1 truncate">
{header.isPlaceholder
? null
: flexRender(header.column.columnDef.header, header.getContext())}

View File

@@ -1,9 +1,7 @@
import { useAtomValue } from "jotai"
import { atomWithStorage } from "jotai/utils"
export const selectedCompanyAtom = atomWithStorage<string>('bank-rec-selected-company', window.frappe?.boot?.user?.defaults?.company || '', undefined, {
getOnInit: true,
})
export const selectedCompanyAtom = atomWithStorage<string>('bank-rec-selected-company', window.frappe?.boot?.user?.defaults?.company || '')
export const useCurrentCompany = () => {
const selectedCompany = useAtomValue(selectedCompanyAtom)

View File

@@ -1,58 +1,13 @@
import { useFrappeGetCall } from "frappe-react-sdk"
import { useMemo } from "react"
import dayjs from "dayjs"
import { useCurrentCompany } from "./useCurrentCompany"
export type FiscalYear = {
name: string
year_start_date: string
year_end_date: string
}
/**
* The fiscal year containing today, for the currently selected company.
*
* `company` matters in multi-company setups, where fiscal years can be restricted to
* specific companies. `date` matters because without it `get_fiscal_year` returns the newest
* fiscal year in the system (they're ordered by start date, descending) - which may be one
* created in advance for a year that hasn't started.
*/
const useFiscalYear = () => {
const company = useCurrentCompany()
const { data, ...rest } = useFrappeGetCall<{ message: FiscalYear | [string, string, string] | false }>(
"erpnext.accounts.utils.get_fiscal_year",
{
date: dayjs().format("YYYY-MM-DD"),
company,
as_dict: 1,
// Return nothing instead of throwing/msgprinting when no fiscal year covers today.
raise_on_missing: 0,
verbose: 0,
},
company ? `fiscal_year_${company}` : null,
{
revalidateOnFocus: false,
revalidateIfStale: false,
revalidateOnReconnect: false
}
)
return useFrappeGetCall("erpnext.accounts.utils.get_fiscal_year", undefined, 'fiscal_year', {
revalidateOnFocus: false,
revalidateIfStale: false,
revalidateOnReconnect: false
})
// get_fiscal_year returns a dict with as_dict, a (name, start, end) tuple without it, and
// false when there's no match - normalise all three.
const fiscalYear = useMemo<FiscalYear | undefined>(() => {
const message = data?.message
if (!message) return undefined
if (Array.isArray(message)) {
const [name, year_start_date, year_end_date] = message
return { name, year_start_date, year_end_date }
}
return message
}, [data])
return { fiscalYear, ...rest }
}
export default useFiscalYear
export default useFiscalYear

View File

@@ -1,23 +0,0 @@
import { useLayoutEffect, useRef } from "react"
/**
* Pins a scrollable list back to the top whenever the search term changes.
*
* Dropdowns that do their own filtering (`shouldFilter={false}`) swap a long list for a much
* shorter one while the scroll container keeps its previous offset - which can leave the
* auto-selected first item scrolled out of view.
*
* Returns a ref to attach to the scroll container (e.g. `CommandList`).
*/
const useResetScrollOnSearch = (search: string) => {
const listRef = useRef<HTMLDivElement>(null)
// Layout effect so the reset lands before paint, avoiding a visible jump.
useLayoutEffect(() => {
listRef.current?.scrollTo({ top: 0 })
}, [search])
return listRef
}
export default useResetScrollOnSearch

View File

@@ -1,6 +1,5 @@
@import "tailwindcss";
@import "tw-animate-css";
@import "./styles/scroll-fade.css";
@font-face {
font-family: InterVariable;

View File

@@ -1,26 +0,0 @@
import { Parser } from 'safe-expr-eval'
const parser = new Parser()
const PLAIN_NUMBER_PATTERN = /^-?\d+(\.\d+)?$/
export function evaluateAmountFormula(expression: string, transactionAmount: number): number {
const trimmed = expression.trim()
if (!trimmed) {
return 0
}
if (PLAIN_NUMBER_PATTERN.test(trimmed)) {
return Number(trimmed)
}
try {
const result = parser.parse(trimmed).evaluate({ transaction_amount: transactionAmount })
if (typeof result !== 'number' || !Number.isFinite(result)) {
return 0
}
return result
} catch {
return 0
}
}

View File

@@ -33,16 +33,6 @@ export const getErrorMessages = (error?: FrappeError | null): ParsedErrorMessage
}
})
// @ts-expect-error - some errors have _error_message
if (error?._error_message) {
eMessages.push({
// @ts-expect-error - some errors have _error_message
message: error?._error_message,
title: "Error",
indicator: "red"
})
}
if (eMessages.length === 0) {
// Get the message from the exception by removing the exc_type
const indexOfFirstColon = error?.exception?.indexOf(':')

View File

@@ -1,4 +1,4 @@
import BankAccountBalancePanel from "@/components/features/BankReconciliation/BankBalance"
import BankBalance from "@/components/features/BankReconciliation/BankBalance"
import BankPicker from "@/components/features/BankReconciliation/BankPicker"
import BankRecDateFilter from "@/components/features/BankReconciliation/BankRecDateFilter"
import BankTransactionUnreconcileModal from "@/components/features/BankReconciliation/BankTransactionUnreconcileModal"
@@ -9,9 +9,10 @@ import ActionLog from "@/components/features/ActionLog/ActionLog"
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs"
import { TooltipProvider } from "@/components/ui/tooltip"
import _ from "@/lib/translate"
import { lazy, Suspense } from "react"
import { lazy, Suspense, useLayoutEffect, useRef, useState } from "react"
import { AlertTriangleIcon, CheckCircleIcon, HomeIcon, LandmarkIcon, ListIcon, Loader2Icon, ScrollTextIcon, ShuffleIcon } from "lucide-react"
import { Breadcrumb, BreadcrumbItem, BreadcrumbList, BreadcrumbPage, BreadcrumbSeparator } from "@/components/ui/breadcrumb"
import { Badge } from "@/components/ui/badge"
import { Empty, EmptyContent, EmptyDescription, EmptyHeader, EmptyMedia, EmptyTitle } from "@/components/ui/empty"
import { Button } from "@/components/ui/button"
import { useAtomValue } from "jotai"
@@ -24,13 +25,23 @@ const IncorrectlyClearedEntries = lazy(() => import('@/components/features/BankR
const BankReconciliation = () => {
const [headerHeight, setHeaderHeight] = useState(0)
const ref = useRef<HTMLDivElement>(null)
useLayoutEffect(() => {
if (ref.current) {
setHeaderHeight(ref.current.clientHeight)
}
}, [])
const remainingHeightAfterTabs = window.innerHeight - headerHeight - 220
return (
<div>
{/* The page owns the viewport height and the tabs/lists below fill what's left, so
the virtualizers size themselves from layout instead of a measured pixel value. */}
<div className="px-2 pt-1 flex-col gap-4 md:flex hidden h-dvh">
<div className="flex flex-col gap-4 shrink-0">
<div className="flex justify-between shrink-0">
<div className="p-4 flex-col gap-4 md:flex hidden">
<div ref={ref} className="flex flex-col gap-4">
<div className="flex justify-between">
<div className="flex items-center gap-6">
<Breadcrumb>
<BreadcrumbList>
@@ -43,7 +54,7 @@ const BankReconciliation = () => {
<BreadcrumbItem>
<BreadcrumbPage>
<div className="flex gap-1 items-center">
{_("Banking")}
{_("Banking")} <Badge theme="violet" variant="subtle">{_("Beta")}</Badge>
</div>
</BreadcrumbPage>
@@ -60,8 +71,10 @@ const BankReconciliation = () => {
<BankRecDateFilter />
</div>
</div>
<BankPicker />
<BankBalance />
</div>
<BankRecWorkspace />
<BankRecTabs remainingHeightAfterTabs={remainingHeightAfterTabs} />
<BankTransactionUnreconcileModal />
</div>
<div className="md:hidden flex h-screen items-center justify-between">
@@ -91,53 +104,42 @@ const BankReconciliation = () => {
)
}
const BankRecWorkspace = () => {
const BankRecTabs = ({ remainingHeightAfterTabs }: { remainingHeightAfterTabs: number }) => {
const selectedBankAccount = useAtomValue(selectedBankAccountAtom)
return <Tabs defaultValue="Match and Reconcile" className="min-h-0 flex-1 gap-4">
{/* Picker + tab strip stack on the left, balance panel beside them - the tab strip
fills height the panel needs anyway, so it costs no row of its own. The picker
scrolls horizontally (`min-w-0` lets it shrink so its overflow-x engages) while
the panel stays put, so the figures never scroll away. */}
{/* No gap here: the panel's own `border-s ps-4` supplies the separation, and a gap
would leave dead space the picker's edge fade can't reach. */}
<div className="flex shrink-0 items-stretch">
<div className="flex min-w-0 flex-1 flex-col justify-between gap-3">
<BankPicker />
{selectedBankAccount && <TabsList>
<TabsTrigger value="Match and Reconcile"><ShuffleIcon /> {_("Match and Reconcile")}</TabsTrigger>
<TabsTrigger value="Bank Reconciliation Statement"><ScrollTextIcon /> {_("Reconciliation Statement")}</TabsTrigger>
<TabsTrigger value="Bank Transactions"><ListIcon />{_("Transactions")}</TabsTrigger>
<TabsTrigger value="Bank Clearance Summary"><CheckCircleIcon />{_("Clearance Summary")}</TabsTrigger>
<TabsTrigger value="Incorrectly Cleared Entries"><AlertTriangleIcon /> {_("Incorrectly Cleared")}</TabsTrigger>
</TabsList>}
</div>
{selectedBankAccount && <BankAccountBalancePanel />}
</div>
if (!selectedBankAccount) {
return null
}
{selectedBankAccount && <>
<TabsContent value="Match and Reconcile" className="flex min-h-0 flex-col">
<MatchAndReconcile />
return <Tabs defaultValue="Match and Reconcile">
<TabsList>
<TabsTrigger value="Match and Reconcile"><ShuffleIcon /> {_("Match and Reconcile")}</TabsTrigger>
<TabsTrigger value="Bank Reconciliation Statement"><ScrollTextIcon /> {_("Bank Reconciliation Statement")}</TabsTrigger>
<TabsTrigger value="Bank Transactions"><ListIcon />{_("Bank Transactions")}</TabsTrigger>
<TabsTrigger value="Bank Clearance Summary"><CheckCircleIcon />{_("Bank Clearance Summary")}</TabsTrigger>
<TabsTrigger value="Incorrectly Cleared Entries"><AlertTriangleIcon /> {_("Incorrectly Cleared Entries")}</TabsTrigger>
</TabsList>
<TabsContent value="Match and Reconcile">
<MatchAndReconcile contentHeight={remainingHeightAfterTabs} />
</TabsContent>
<Suspense fallback={
<div className="flex items-center justify-center p-16">
<Loader2Icon className="size-6 animate-spin text-muted-foreground" />
</div>
}>
<TabsContent value="Bank Reconciliation Statement">
<BankReconciliationStatement />
</TabsContent>
<Suspense fallback={
<div className="flex items-center justify-center p-16">
<Loader2Icon className="size-6 animate-spin text-muted-foreground" />
</div>
}>
<TabsContent value="Bank Reconciliation Statement" className="flex min-h-0 flex-col">
<BankReconciliationStatement />
</TabsContent>
<TabsContent value="Bank Transactions" className="flex min-h-0 flex-col">
<BankTransactions />
</TabsContent>
<TabsContent value="Bank Clearance Summary" className="flex min-h-0 flex-col">
<BankClearanceSummary />
</TabsContent>
<TabsContent value="Incorrectly Cleared Entries" className="flex min-h-0 flex-col">
<IncorrectlyClearedEntries />
</TabsContent>
</Suspense>
</>}
<TabsContent value="Bank Transactions">
<BankTransactions />
</TabsContent>
<TabsContent value="Bank Clearance Summary">
<BankClearanceSummary />
</TabsContent>
<TabsContent value="Incorrectly Cleared Entries">
<IncorrectlyClearedEntries />
</TabsContent>
</Suspense>
</Tabs>
}

View File

@@ -226,7 +226,7 @@ const StatementImportLog = () => {
field: "creation",
order: "desc"
},
limit: 20
limit: 10
}, bankAccount ? undefined : null, {
revalidateOnFocus: false
})

View File

@@ -1,94 +0,0 @@
/* Scroll-edge fade mask for horizontal scroll containers (the bank picker strip).
Ported from Raven's `scroll-fade-x`; imported by index.css, since Tailwind processes
`@utility` in imported files the same as in the entry file.
The scroll-timeline keyframes reveal each edge's fade only when there IS content to scroll
in that direction - no fade on the left edge when scrolled fully left, none on the right at
the end. `@property` makes the fade animate smoothly rather than jumping.
Without scroll-timeline support (Firefox) there is deliberately NO fade at all: the fade
vars stay at their 0px initial value and the gradient stops collapse to the edges. A static
both-edges fallback was tried in Raven and removed - on a container with nothing to scroll
it dimmed the edges anyway, promising content that didn't exist. */
@property --scroll-fade-l {
/* length-percentage, NOT length: the fade size is min(12%, …) - a percentage. A <length>
property rejects that value and reverts to initial-value (0px), zeroing the fade. */
syntax: "<length-percentage>";
inherits: false;
initial-value: 0px;
}
@property --scroll-fade-r {
syntax: "<length-percentage>";
inherits: false;
initial-value: 0px;
}
@keyframes scroll-fade-reveal-l {
from {
--scroll-fade-l: 0px;
}
to {
--scroll-fade-l: var(--_scroll-fade-size-l);
}
}
@keyframes scroll-fade-reveal-r {
from {
--scroll-fade-r: var(--_scroll-fade-size-r);
}
to {
--scroll-fade-r: 0px;
}
}
@utility scroll-fade-x {
--_scroll-fade-size-l: var(--scroll-fade-l-size,
var(--scroll-fade-size, min(12%, calc(var(--spacing, 0.25rem) * 10))));
--_scroll-fade-size-r: var(--scroll-fade-r-size,
var(--scroll-fade-size, min(12%, calc(var(--spacing, 0.25rem) * 10))));
/* Eased (smoothstep) alpha ramp, sampled finely so it reads as a smooth curve, NOT fading
all the way to transparent: the edge floors at 0.25 (content dims, never vanishes), ramping
up to a full 1 for the body. The opaque end MUST be 1 or everything would be permanently
dimmed. Stops collapse to the edge when the size animates to 0, so the true first/last card
is never dimmed at rest. Tune the floor - higher (~0.4) = subtler, lower (~0.1) = stronger. */
--scroll-fade-inline: linear-gradient(to right,
rgba(0, 0, 0, 0.25) 0,
rgba(0, 0, 0, 0.282) calc(var(--scroll-fade-l, 0px) * 0.125),
rgba(0, 0, 0, 0.367) calc(var(--scroll-fade-l, 0px) * 0.25),
rgba(0, 0, 0, 0.487) calc(var(--scroll-fade-l, 0px) * 0.375),
rgba(0, 0, 0, 0.625) calc(var(--scroll-fade-l, 0px) * 0.5),
rgba(0, 0, 0, 0.763) calc(var(--scroll-fade-l, 0px) * 0.625),
rgba(0, 0, 0, 0.883) calc(var(--scroll-fade-l, 0px) * 0.75),
rgba(0, 0, 0, 0.968) calc(var(--scroll-fade-l, 0px) * 0.875),
rgba(0, 0, 0, 1) var(--scroll-fade-l, 0px),
rgba(0, 0, 0, 1) calc(100% - var(--scroll-fade-r, 0px)),
rgba(0, 0, 0, 0.968) calc(100% - var(--scroll-fade-r, 0px) * 0.875),
rgba(0, 0, 0, 0.883) calc(100% - var(--scroll-fade-r, 0px) * 0.75),
rgba(0, 0, 0, 0.763) calc(100% - var(--scroll-fade-r, 0px) * 0.625),
rgba(0, 0, 0, 0.625) calc(100% - var(--scroll-fade-r, 0px) * 0.5),
rgba(0, 0, 0, 0.487) calc(100% - var(--scroll-fade-r, 0px) * 0.375),
rgba(0, 0, 0, 0.367) calc(100% - var(--scroll-fade-r, 0px) * 0.25),
rgba(0, 0, 0, 0.282) calc(100% - var(--scroll-fade-r, 0px) * 0.125),
rgba(0, 0, 0, 0.25) 100%);
-webkit-mask-image: var(--scroll-fade-mask, var(--scroll-fade-inline));
mask-image: var(--scroll-fade-mask, var(--scroll-fade-inline));
-webkit-mask-composite: source-in;
mask-composite: intersect;
-webkit-mask-repeat: no-repeat;
mask-repeat: no-repeat;
@supports (animation-timeline: scroll()) {
animation:
scroll-fade-reveal-l 1ms ease-in-out,
scroll-fade-reveal-r 1ms ease-in-out;
animation-timeline: scroll(self x), scroll(self x);
animation-range:
0 var(--scroll-fade-reveal, calc(var(--spacing, 0.25rem) * 24)),
calc(100% - var(--scroll-fade-reveal, calc(var(--spacing, 0.25rem) * 24))) 100%;
animation-fill-mode: both;
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,5 +1,14 @@
preserve_hierarchy: true
files:
- source: /erpnext/locale/main.pot
translation: /erpnext/locale/%two_letters_code%.po
pull_request_title: "fix: sync translations from crowdin"
pull_request_labels:
- translation
- skip-release-notes
pull_request_reviewers:
- barredterra # change to your GitHub username if you copied this file
commit_message: "fix: %language% translations"
append_commit_message: false
languages_mapping:
two_letters_code:
pt-BR: pt_BR

View File

@@ -3,6 +3,8 @@ import inspect
from typing import TypeVar
import frappe
from frappe.model.document import Document
from frappe.utils.user import is_website_user
__version__ = "17.0.0-dev"
@@ -153,8 +155,6 @@ def allow_regional(fn):
def check_app_permission():
from frappe.utils.user import is_website_user
if frappe.session.user == "Administrator":
return True
@@ -175,16 +175,9 @@ def normalize_ctx_input(T: type) -> callable:
- Casting the result to the specified type T
"""
from frappe.model.document import Document
def decorator(func: callable):
# conserve annotations for frappe.utils.typing_validations
@functools.wraps(
func,
assigned=(
a for a in functools.WRAPPER_ASSIGNMENTS if a not in ("__annotations__", "__annotate__")
),
)
@functools.wraps(func, assigned=(a for a in functools.WRAPPER_ASSIGNMENTS if a != "__annotations__"))
def wrapper(ctx: T | Document | dict | str, *args, **kwargs):
if isinstance(ctx, Document):
ctx = T(**ctx.as_dict())

View File

@@ -1,190 +0,0 @@
import frappe
from frappe import _
from frappe.utils import flt
from erpnext.accounts.doctype.payment_entry.payment_entry import (
get_outstanding_reference_documents,
get_payment_entry,
)
@frappe.whitelist(methods=["POST"])
def create_payment_entries(invoices: str | list | None = None):
"""Create draft Payment Entries from AP report invoice selection."""
frappe.has_permission("Payment Entry", "create", throw=True)
names = [d["voucher_no"] for d in frappe.parse_json(invoices or "[]") if d.get("voucher_no")]
if not names:
frappe.throw(_("No Purchase Invoices selected"))
payable, excluded = _partition_payable_invoices(names)
if not payable:
frappe.throw(_("None of the selected invoices are payable"))
# invoices sharing a (supplier, payable account) are combined into one Payment Entry
groups = {}
for d in payable:
key = (d["supplier"], d["party_account"])
groups.setdefault(
key, {"supplier": d["supplier"], "party_account": d["party_account"], "vouchers": []}
)["vouchers"].append(d["voucher_no"])
created, failed = 0, 0
for group in groups.values():
if _create_payment_entry(group):
created += 1
else:
failed += 1
message = _("Created {0} draft Payment Entries").format(created)
if excluded:
message += "" + _("{0} excluded (not payable)").format(len(excluded))
if failed:
message += "" + _("{0} failed (see Error Log)").format(failed)
frappe.msgprint(message, title=_("Bulk Payment Entries"), indicator="green")
@frappe.whitelist()
def get_payable_invoices(invoices: str | list | None = None):
"""Return the live payable subset of the selected invoices for the report dialog."""
frappe.has_permission("Payment Entry", "create", throw=True)
names = [d["voucher_no"] for d in frappe.parse_json(invoices or "[]") if d.get("voucher_no")]
payable, excluded = _partition_payable_invoices(names)
currency = None
if payable:
company = frappe.get_cached_value("Purchase Invoice", payable[0]["voucher_no"], "company")
currency = frappe.get_cached_value("Company", company, "default_currency")
return {"payable": payable, "excluded": excluded, "currency": currency}
def _partition_payable_invoices(names):
"""Split submitted Purchase Invoices into payable ones and excluded ones (with reason).
Returns are debit notes, internal transfers are inter-company, and non-positive
outstanding means already settled — none are valid targets for a supplier payment.
"""
if not names:
return [], []
rows = frappe.get_list(
"Purchase Invoice",
filters={"name": ["in", names], "docstatus": 1},
fields=[
"name",
"supplier",
"credit_to",
"outstanding_amount",
"conversion_rate",
"is_return",
"is_internal_supplier",
],
limit_page_length=0,
)
payable, excluded = [], []
for r in rows:
if r.is_return:
excluded.append({"voucher_no": r.name, "reason": _("Debit Note")})
elif r.is_internal_supplier:
excluded.append({"voucher_no": r.name, "reason": _("Internal Transfer")})
elif flt(r.outstanding_amount) <= 0:
excluded.append({"voucher_no": r.name, "reason": _("Already Paid")})
else:
payable.append(
{
"voucher_no": r.name,
"supplier": r.supplier,
"party_account": r.credit_to,
"outstanding": flt(r.outstanding_amount) * flt(r.conversion_rate or 1),
}
)
# names not returned were cancelled/deleted or no longer readable after the report loaded
found = {r.name for r in rows}
for name in names:
if name not in found:
excluded.append({"voucher_no": name, "reason": _("Not available")})
return payable, excluded
def _create_payment_entry(group):
supplier = group["supplier"]
try:
frappe.db.savepoint("bulk_pe")
if len(group["vouchers"]) == 1:
pe = _build_single_payment_entry(group["vouchers"][0])
else:
pe = _build_grouped_payment_entry(supplier, group["party_account"], group["vouchers"])
if not pe:
frappe.db.rollback(save_point="bulk_pe")
frappe.log_error(
title=_("Bulk Payment Entry skipped for {0}").format(supplier),
message=_("No outstanding amount for the selected invoice(s)."),
)
return False
pe.flags.ignore_validate = True
pe.set_title_field()
pe.insert(ignore_mandatory=True)
return True
except Exception:
frappe.db.rollback(save_point="bulk_pe")
frappe.log_error(title=_("Bulk Payment Entry creation failed for {0}").format(supplier))
return False
def _build_single_payment_entry(name):
pe = get_payment_entry("Purchase Invoice", name)
# guard against a stale report row: nothing to allocate means the invoice is already settled
if not pe.references or not any(flt(r.allocated_amount) for r in pe.references):
return None
return pe
def _build_grouped_payment_entry(supplier, party_account, names):
name_set = set(names)
pe = get_payment_entry("Purchase Invoice", names[0])
pe.set("references", [])
refs = get_outstanding_reference_documents(
{
"party_type": "Supplier",
"party": supplier,
"party_account": party_account,
"company": pe.company,
"vouchers": [frappe._dict(voucher_type="Purchase Invoice", voucher_no=n) for n in names],
}
)
# get_negative_outstanding_invoices ignores the vouchers filter, so bound refs to the selection
for r in refs:
if r.voucher_type != "Purchase Invoice" or r.voucher_no not in name_set:
continue
pe.append(
"references",
{
"reference_doctype": r.voucher_type,
"reference_name": r.voucher_no,
"bill_no": r.get("bill_no"),
"due_date": r.get("due_date"),
"payment_term": r.get("payment_term"),
"total_amount": r.invoice_amount,
"outstanding_amount": r.outstanding_amount,
"allocated_amount": r.outstanding_amount,
"exchange_rate": r.get("exchange_rate") or 1,
},
)
if not pe.references:
return None
# received_amount is in paid_to account currency; convert to paid_from account currency for paid_amount
pe.received_amount = sum(r.allocated_amount for r in pe.references)
pe.paid_amount = flt(pe.received_amount * pe.target_exchange_rate, pe.precision("paid_amount"))
pe.set_amounts()
return pe

View File

@@ -17,7 +17,7 @@ class ERPNextAddress(Address):
def link_address(self):
"""Link address based on owner"""
if self.get("is_your_company_address"):
if self.is_your_company_address:
return
return super().link_address()
@@ -28,9 +28,7 @@ class ERPNextAddress(Address):
self.is_your_company_address = 1
def validate_reference(self):
if self.get("is_your_company_address") and not [
row for row in self.links if row.link_doctype == "Company"
]:
if self.is_your_company_address and not [row for row in self.links if row.link_doctype == "Company"]:
frappe.throw(
_(
"Address needs to be linked to a Company. Please add a row for Company in the Links table."
@@ -71,6 +69,4 @@ def get_shipping_address(company: str, address: str | None = None):
if address:
address_as_dict = address[0]
name, address_template = get_address_templates(address_as_dict)
return address_as_dict.get("name"), frappe.render_template(
address_template, address_as_dict, restrict_globals=True
)
return address_as_dict.get("name"), frappe.render_template(address_template, address_as_dict)

View File

@@ -9,7 +9,7 @@
"idx": 0,
"is_public": 1,
"is_standard": 1,
"modified": "2026-09-04 12:37:31.673782",
"modified": "2025-12-19 12:37:31.673782",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Profit and Loss",
@@ -17,6 +17,7 @@
"owner": "Administrator",
"report_name": "Profit and Loss Statement",
"roles": [],
"show_values_over_chart": 1,
"timeseries": 0,
"type": "Line",
"use_report_chart": 1,

View File

@@ -1,7 +1,6 @@
import frappe
from frappe import _
from frappe.email import sendmail_to_system_managers
from frappe.query_builder.functions import IfNull, Sum
from frappe.utils import (
add_days,
add_months,
@@ -54,24 +53,20 @@ def validate_service_stop_date(doc):
def build_conditions(process_type, account, company):
if process_type == "Income":
item = frappe.qb.DocType("Sales Invoice Item")
parent = frappe.qb.DocType("Sales Invoice")
deferred_account = item.deferred_revenue_account
else:
item = frappe.qb.DocType("Purchase Invoice Item")
parent = frappe.qb.DocType("Purchase Invoice")
deferred_account = item.deferred_expense_account
conditions = ""
deferred_account = (
"item.deferred_revenue_account" if process_type == "Income" else "item.deferred_expense_account"
)
if account:
return deferred_account == account
conditions += f"AND {deferred_account}={frappe.db.escape(account)}"
elif company:
return parent.company == company
conditions += f"AND p.company = {frappe.db.escape(company)}"
return None
return conditions
def convert_deferred_expense_to_expense(deferred_process, start_date=None, end_date=None, conditions=None):
def convert_deferred_expense_to_expense(deferred_process, start_date=None, end_date=None, conditions=""):
# book the expense/income on the last day, but it will be trigger on the 1st of month at 12:00 AM
if not start_date:
@@ -80,25 +75,17 @@ def convert_deferred_expense_to_expense(deferred_process, start_date=None, end_d
end_date = add_days(today(), -1)
# check for the purchase invoice for which GL entries has to be done
item = frappe.qb.DocType("Purchase Invoice Item")
parent = frappe.qb.DocType("Purchase Invoice")
query = (
frappe.qb.from_(item)
.inner_join(parent)
.on(item.parent == parent.name)
.select(item.parent)
.distinct()
.where(
(item.service_start_date <= end_date)
& (item.service_end_date >= start_date)
& (item.enable_deferred_expense == 1)
& (item.docstatus == 1)
& (IfNull(item.amount, 0) > 0)
)
)
if conditions is not None:
query = query.where(conditions)
invoices = query.run(pluck=True)
invoices = frappe.db.sql_list(
f"""
select distinct item.parent
from `tabPurchase Invoice Item` item, `tabPurchase Invoice` p
where item.service_start_date<=%s and item.service_end_date>=%s
and item.enable_deferred_expense = 1 and item.parent=p.name
and item.docstatus = 1 and ifnull(item.amount, 0) > 0
{conditions}
""",
(end_date, start_date),
) # nosec
# For each invoice, book deferred expense
for invoice in invoices:
@@ -109,7 +96,7 @@ def convert_deferred_expense_to_expense(deferred_process, start_date=None, end_d
send_mail(deferred_process)
def convert_deferred_revenue_to_income(deferred_process, start_date=None, end_date=None, conditions=None):
def convert_deferred_revenue_to_income(deferred_process, start_date=None, end_date=None, conditions=""):
# book the expense/income on the last day, but it will be trigger on the 1st of month at 12:00 AM
if not start_date:
@@ -118,25 +105,17 @@ def convert_deferred_revenue_to_income(deferred_process, start_date=None, end_da
end_date = add_days(today(), -1)
# check for the sales invoice for which GL entries has to be done
item = frappe.qb.DocType("Sales Invoice Item")
parent = frappe.qb.DocType("Sales Invoice")
query = (
frappe.qb.from_(item)
.inner_join(parent)
.on(item.parent == parent.name)
.select(item.parent)
.distinct()
.where(
(item.service_start_date <= end_date)
& (item.service_end_date >= start_date)
& (item.enable_deferred_revenue == 1)
& (item.docstatus == 1)
& (IfNull(item.amount, 0) > 0)
)
)
if conditions is not None:
query = query.where(conditions)
invoices = query.run(pluck=True)
invoices = frappe.db.sql_list(
f"""
select distinct item.parent
from `tabSales Invoice Item` item, `tabSales Invoice` p
where item.service_start_date<=%s and item.service_end_date>=%s
and item.enable_deferred_revenue = 1 and item.parent=p.name
and item.docstatus = 1 and ifnull(item.amount, 0) > 0
{conditions}
""",
(end_date, start_date),
) # nosec
for invoice in invoices:
doc = frappe.get_doc("Sales Invoice", invoice)
@@ -157,39 +136,26 @@ def get_booking_dates(doc, item, posting_date=None, prev_posting_date=None):
)
if not prev_posting_date:
prev_gl_entry = frappe.get_all(
"GL Entry",
filters={
"company": doc.company,
"account": item.get(deferred_account),
"voucher_type": doc.doctype,
"voucher_no": doc.name,
"voucher_detail_no": item.name,
"is_cancelled": 0,
},
fields=["name", "posting_date"],
order_by="posting_date desc",
limit=1,
prev_gl_entry = frappe.db.sql(
"""
select name, posting_date from `tabGL Entry` where company=%s and account=%s and
voucher_type=%s and voucher_no=%s and voucher_detail_no=%s
and is_cancelled = 0
order by posting_date desc limit 1
""",
(doc.company, item.get(deferred_account), doc.doctype, doc.name, item.name),
as_dict=True,
)
je = frappe.qb.DocType("Journal Entry")
jea = frappe.qb.DocType("Journal Entry Account")
prev_gl_via_je = (
frappe.qb.from_(je)
.inner_join(jea)
.on(je.name == jea.parent)
.select(je.name, je.posting_date)
.where(
(je.company == doc.company)
& (jea.account == item.get(deferred_account))
& (jea.reference_type == doc.doctype)
& (jea.reference_name == doc.name)
& (jea.reference_detail_no == item.name)
& (jea.docstatus < 2)
)
.orderby(je.posting_date, order=frappe.qb.desc)
.limit(1)
.run(as_dict=True)
prev_gl_via_je = frappe.db.sql(
"""
SELECT p.name, p.posting_date FROM `tabJournal Entry` p, `tabJournal Entry Account` c
WHERE p.name = c.parent and p.company=%s and c.account=%s
and c.reference_type=%s and c.reference_name=%s
and c.reference_detail_no=%s and c.docstatus < 2 order by posting_date desc limit 1
""",
(doc.company, item.get(deferred_account), doc.doctype, doc.name, item.name),
as_dict=True,
)
if prev_gl_via_je:
@@ -311,47 +277,26 @@ def get_already_booked_amount(doc, item):
total_credit_debit, total_credit_debit_currency = "credit", "credit_in_account_currency"
deferred_account = "deferred_expense_account"
gle = frappe.qb.DocType("GL Entry")
gl_entries_details = (
frappe.qb.from_(gle)
.select(
Sum(gle[total_credit_debit]).as_("total_credit"),
Sum(gle[total_credit_debit_currency]).as_("total_credit_in_account_currency"),
gle.voucher_detail_no,
)
.where(
(gle.company == doc.company)
& (gle.account == item.get(deferred_account))
& (gle.voucher_type == doc.doctype)
& (gle.voucher_no == doc.name)
& (gle.voucher_detail_no == item.name)
& (gle.is_cancelled == 0)
)
.groupby(gle.voucher_detail_no)
.run(as_dict=True)
gl_entries_details = frappe.db.sql(
"""
select sum({}) as total_credit, sum({}) as total_credit_in_account_currency, voucher_detail_no
from `tabGL Entry` where company=%s and account=%s and voucher_type=%s and voucher_no=%s and voucher_detail_no=%s
and is_cancelled = 0
group by voucher_detail_no
""".format(total_credit_debit, total_credit_debit_currency),
(doc.company, item.get(deferred_account), doc.doctype, doc.name, item.name),
as_dict=True,
)
je = frappe.qb.DocType("Journal Entry")
jea = frappe.qb.DocType("Journal Entry Account")
journal_entry_details = (
frappe.qb.from_(je)
.inner_join(jea)
.on(je.name == jea.parent)
.select(
Sum(jea[total_credit_debit]).as_("total_credit"),
Sum(jea[total_credit_debit_currency]).as_("total_credit_in_account_currency"),
jea.reference_detail_no,
)
.where(
(je.company == doc.company)
& (jea.account == item.get(deferred_account))
& (jea.reference_type == doc.doctype)
& (jea.reference_name == doc.name)
& (jea.reference_detail_no == item.name)
& (je.docstatus < 2)
)
.groupby(jea.reference_detail_no)
.run(as_dict=True)
journal_entry_details = frappe.db.sql(
"""
SELECT sum(c.{}) as total_credit, sum(c.{}) as total_credit_in_account_currency, reference_detail_no
FROM `tabJournal Entry` p , `tabJournal Entry Account` c WHERE p.name = c.parent and
p.company = %s and c.account=%s and c.reference_type=%s and c.reference_name=%s and c.reference_detail_no=%s
and p.docstatus < 2 group by reference_detail_no
""".format(total_credit_debit, total_credit_debit_currency),
(doc.company, item.get(deferred_account), doc.doctype, doc.name, item.name),
as_dict=True,
)
already_booked_amount = gl_entries_details[0].total_credit if gl_entries_details else 0
@@ -582,7 +527,6 @@ def make_gl_entries(
frappe.db.commit()
except Exception as e:
if frappe.in_test:
frappe.db.rollback()
doc.log_error(f"Error while processing deferred accounting for Invoice {doc.name}")
raise e
else:

View File

@@ -122,7 +122,6 @@
"description": "Setting Account Type helps in selecting this Account in transactions.",
"fieldname": "account_type",
"fieldtype": "Select",
"in_preview": 1,
"in_standard_filter": 1,
"label": "Account Type",
"oldfieldname": "account_type",
@@ -204,7 +203,7 @@
"idx": 1,
"is_tree": 1,
"links": [],
"modified": "2026-09-03 12:59:42.190900",
"modified": "2026-04-14 18:14:42.202065",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Account",
@@ -265,46 +264,6 @@
{
"role": "HR Manager",
"select": 1
},
{
"role": "Delivery Manager",
"select": 1
},
{
"role": "Delivery User",
"select": 1
},
{
"role": "Maintenance Manager",
"select": 1
},
{
"role": "Maintenance User",
"select": 1
},
{
"role": "Manufacturing Manager",
"select": 1
},
{
"role": "Manufacturing User",
"select": 1
},
{
"role": "Purchase Master Manager",
"select": 1
},
{
"role": "Quality Manager",
"select": 1
},
{
"role": "Sales Master Manager",
"select": 1
},
{
"role": "Stock Manager",
"select": 1
}
],
"row_format": "Dynamic",

View File

@@ -121,7 +121,6 @@ class Account(NestedSet):
self.validate_account_currency()
self.validate_root_company_and_sync_account_to_children()
self.validate_receivable_payable_account_type()
self.validate_stock_account_type_change()
def validate_parent_child_account_type(self):
if self.parent_account:
@@ -213,36 +212,6 @@ class Account(NestedSet):
frappe.msgprint(msg)
self.add_comment("Comment", msg)
def validate_stock_account_type_change(self):
doc_before_save = self.get_doc_before_save()
if not (doc_before_save and doc_before_save.account_type == "Stock"):
return
if self.account_type == "Stock":
return
if self.stock_ledger_entry_exists():
frappe.throw(
_(
"The account type of {0} cannot be changed from {1} because stock ledger entries exist against it."
).format(frappe.bold(self.name), frappe.bold(_("Stock")))
)
def stock_ledger_entry_exists(self):
from erpnext.stock import get_warehouse_account_map
warehouse_account = get_warehouse_account_map(self.company)
warehouses = [wh for wh, details in warehouse_account.items() if details.account == self.name]
if not warehouses:
return False
return bool(
frappe.db.count(
"Stock Ledger Entry",
filters={"warehouse": ("in", warehouses), "is_cancelled": 0},
)
)
def validate_root_details(self):
doc_before_save = self.get_doc_before_save()
@@ -265,7 +234,7 @@ class Account(NestedSet):
if not frappe.db.get_value(
"Account", {"account_name": self.account_name, "company": ancestors[0]}, "name"
):
frappe.throw(_("Please add the account to root level Company - {0}").format(ancestors[0]))
frappe.throw(_("Please add the account to root level Company - {}").format(ancestors[0]))
elif self.parent_account:
descendants = get_descendants_of("Company", self.company)
if not descendants:
@@ -690,15 +659,8 @@ def _ensure_idle_system():
last_gl_update = None
try:
if frappe.db.db_type == "postgres":
# The MariaDB branch blocks new GL inserts via the gap lock its for_update read takes;
# a postgres row lock never blocks inserts, so take an EXCLUSIVE table lock instead --
# writers block until the rename commits, readers don't. NOWAIT mirrors wait=False.
frappe.db.sql("LOCK TABLE `tabGL Entry` IN EXCLUSIVE MODE NOWAIT")
last_gl_update = frappe.db.get_value("GL Entry", {}, "modified")
else:
# We also lock inserts to GL entry table with for_update here.
last_gl_update = frappe.db.get_value("GL Entry", {}, "modified", for_update=True, wait=False)
# We also lock inserts to GL entry table with for_update here.
last_gl_update = frappe.db.get_value("GL Entry", {}, "modified", for_update=True, wait=False)
except frappe.QueryTimeoutError:
# wait=False fails immediately if there's an active transaction.
last_gl_update = add_to_date(None, seconds=-1)
@@ -709,7 +671,7 @@ def _ensure_idle_system():
if last_gl_update > add_to_date(None, minutes=-5):
frappe.throw(
_(
"Last GL Entry update was done {0}. This operation is not allowed while system is actively being used. Please wait for 5 minutes before retrying."
"Last GL Entry update was done {}. This operation is not allowed while system is actively being used. Please wait for 5 minutes before retrying."
).format(pretty_date(last_gl_update)),
title=_("System In Use"),
)
@@ -727,12 +689,9 @@ def get_company_default_account_fields():
"stock_delivered_but_not_billed": "Stock Delivered But Not Billed Account",
"stock_adjustment_account": "Stock Adjustment Account",
"write_off_account": "Write Off Account",
"bank_charges_account": "Bank Charges Account",
"default_discount_account": "Default Payment Discount Account",
"unrealized_profit_loss_account": "Unrealized Profit / Loss Account",
"exchange_gain_loss_account": "Exchange Gain / Loss Account",
"exchange_gain_account": "Exchange Gain Account",
"exchange_loss_account": "Exchange Loss Account",
"unrealized_exchange_gain_loss_account": "Unrealized Exchange Gain / Loss Account",
"round_off_account": "Round Off Account",
"default_deferred_revenue_account": "Default Deferred Revenue Account",

View File

@@ -52,42 +52,6 @@ frappe.treeview_settings["Account"] = {
],
root_label: "Accounts",
get_tree_nodes: "erpnext.accounts.utils.get_children",
get_label: function (node) {
// clean display name — the account number renders as a badge (see
// onrender) instead of being glued into the name
return frappe.utils.escape_html(node.data.account_name || node.title || node.label);
},
onrender: function (node) {
if (node.is_root || !node.data) return;
const flags = [];
if (node.data.account_number) {
flags.push(frappe.ui.badge({ label: node.data.account_number }));
}
const company = frappe.treeview_settings["Account"].treeview?.page?.fields_dict?.company?.get_value();
const company_currency = company && erpnext.get_currency(company);
if (
node.data.account_currency &&
company_currency &&
node.data.account_currency !== company_currency
) {
flags.push(frappe.ui.badge({ label: node.data.account_currency, theme: "blue" }));
}
if (node.data.freeze_account === "Yes") {
flags.push(
frappe.ui.badge({
label: __("Frozen"),
icon: "lock",
title: __("Frozen - entries restricted"),
theme: "orange",
})
);
}
erpnext.utils.render_tree_node_flags(node, flags);
},
on_node_render: function (node, deep) {
const render_balances = () => {
for (let account of cur_tree.account_balance_data) {
@@ -268,25 +232,24 @@ frappe.treeview_settings["Account"] = {
frappe.treeview_settings["Account"].treeview["tree"] = treeview.tree;
if (treeview.can_create) {
treeview.page.set_primary_action(
{ label: __("Add Account"), short_label: __("Add") },
__("New"),
function () {
let root_company = treeview.page.fields_dict.root_company.get_value();
if (root_company) {
frappe.throw(
__("Please add the account to root level Company - {0}", [root_company])
);
frappe.throw(__("Please add the account to root level Company - {0}"), [
root_company,
]);
} else {
treeview.new_node();
}
},
"plus"
"add"
);
}
},
toolbar: [
{
label: __("Add Child"),
icon: "plus",
condition: function (node) {
return (
frappe.boot.user.can_create.indexOf("Account") !== -1 &&
@@ -309,7 +272,6 @@ frappe.treeview_settings["Account"] = {
return !node.root && frappe.boot.user.can_read.indexOf("GL Entry") !== -1;
},
label: __("View Ledger"),
icon: "book-open",
click: function (node, btn) {
frappe.route_options = {
from_date: erpnext.utils.get_fiscal_year(frappe.datetime.get_today(), true)[1],
@@ -324,106 +286,6 @@ frappe.treeview_settings["Account"] = {
},
btnClass: "hidden-xs",
},
{
// same label and mechanism as the Account form's Actions button:
// NOT frappe's generic rename (Allow Rename stays off) — this is
// ERPNext's controlled update that rebuilds the derived
// "number - name - abbr" document name
label: __("Update Account Name / Number"),
icon: "text-cursor-input",
condition: function (node) {
return !node.is_root && frappe.model.can_write("Account");
},
click: function (node) {
const dialog = new frappe.ui.Dialog({
title: __("Update Account Number / Name"),
fields: [
{
fieldtype: "Data",
fieldname: "account_name",
label: __("Account Name"),
reqd: 1,
default: node.data.account_name,
},
{
fieldtype: "Data",
fieldname: "account_number",
label: __("Account Number"),
default: node.data.account_number,
},
],
primary_action_label: __("Update"),
primary_action(values) {
dialog.hide();
frappe.dom.freeze(__("Updating {0}", [node.label]));
frappe.call({
method: "erpnext.accounts.doctype.account.account.update_account_number",
args: {
name: node.label,
account_name: values.account_name,
account_number: values.account_number,
},
callback: function (r) {
if (r.exc) return;
const treeview = frappe.views.trees["Account"];
node.parent_node && treeview.tree.load_children(node.parent_node);
},
always: function () {
frappe.dom.unfreeze();
},
});
},
});
dialog.show();
},
},
{
label: __("Convert to Group"),
icon: "folder-tree",
condition: function (node) {
return !node.is_root && !node.expandable && frappe.model.can_write("Account");
},
click: function (node) {
erpnext.accounts.convert_tree_node("Account", node, "convert_ledger_to_group");
},
},
{
label: __("Convert to Non-Group"),
icon: "file-text",
condition: function (node) {
// only on groups the user has opened and found empty — a
// group with children can't convert, so don't offer it
return (
!node.is_root &&
node.expandable &&
node.loaded &&
!node.$ul.children().length &&
frappe.model.can_write("Account")
);
},
click: function (node) {
erpnext.accounts.convert_tree_node("Account", node, "convert_group_to_ledger");
},
},
],
extend_toolbar: true,
};
frappe.provide("erpnext.accounts");
// shared by the Account and Cost Center tree views (defined in both files,
// whichever loads first wins): run the doctype's whitelisted convert method,
// then re-render the branch so the node's group/leaf state updates
erpnext.accounts.convert_tree_node =
erpnext.accounts.convert_tree_node ||
function (doctype, node, method) {
frappe.call({
method: "run_doc_method",
args: { dt: doctype, dn: node.label, method: method },
callback: function (r) {
if (r.exc) return;
const treeview = frappe.views.trees[doctype];
node.parent_node && treeview.tree.load_children(node.parent_node);
frappe.show_alert({ message: __("{0} converted", [node.label]), indicator: "green" });
},
});
};

View File

@@ -102,8 +102,6 @@ def identify_is_group(child):
def get_chart(chart_template: str | None, existing_company: str | None = None):
chart = {}
if existing_company:
frappe.has_permission("Company", doc=existing_company, throw=True)
return get_account_tree_from_existing_company(existing_company)
elif chart_template == "Standard":
@@ -139,7 +137,7 @@ def get_charts_for_country(country: str, with_standard: bool = False):
def _get_chart_name(content):
if content:
content = frappe.parse_json(content)
content = json.loads(content)
if (
content and content.get("disabled", "No") == "No"
) or frappe.local.flags.allow_unverified_charts:

View File

@@ -24,8 +24,7 @@
"account_number": "11530"
},
"account_number": "115",
"is_group": 1,
"account_type": "Bank"
"is_group": 1
},
"Trade Receivables": {
"Trade Debtors": {
@@ -530,13 +529,6 @@
"account_number": "630",
"is_group": 1
},
"Accrued Manufacturing Expenses": {
"Accrued Expenses - Manufacturing": {
"account_number": "63510"
},
"account_number": "635",
"is_group": 1
},
"account_number": "63",
"is_group": 1
},
@@ -822,4 +814,4 @@
"root_type": "Expense"
}
}
}
}

View File

@@ -37,10 +37,6 @@
"account_type": "Stock",
"account_category": "Stock Assets"
},
"Stock Delivered But Not Billed": {
"account_type": "Stock Delivered But Not Billed",
"account_category": "Stock Assets"
},
"account_type": "Stock",
"account_category": "Stock Assets"
},
@@ -179,9 +175,6 @@
},
"Impairment": {
"account_category": "Operating Expenses"
},
"Exchange Loss": {
"account_category": "Operating Expenses"
}
},
"root_type": "Expense"
@@ -199,10 +192,6 @@
"account_type": "Income Account"
},
"Indirect Income": {
"Exchange Gain": {
"account_type": "Income Account",
"account_category": "Other Operating Income"
},
"account_type": "Income Account",
"is_group": 1
},
@@ -234,6 +223,10 @@
"Stock Received But Not Billed": {
"account_type": "Stock Received But Not Billed",
"account_category": "Trade Payables"
},
"Stock Delivered But Not Billed": {
"account_type": "Stock Delivered But Not Billed",
"account_category": "Trade Payables"
}
},
"Duties and Taxes": {

View File

@@ -22,12 +22,12 @@
"account_type": "Cash"
},
"Petty Cash Fund": {
"account_number": "1110",
"account_number": "1200",
"is_group": 1,
"root_type": "Asset",
"account_type": "Cash",
"Petty Cash Fund": {
"account_number": "1111",
"account_number": "1201",
"is_group": 0,
"root_type": "Asset",
"account_type": "Cash"
@@ -35,16 +35,10 @@
}
},
"Bank Accounts": {
"account_number": "1200",
"account_number": "1102",
"is_group": 1,
"root_type": "Asset",
"account_type": "Bank",
"Cash in Bank - Checking Account": {
"account_number": "1201",
"is_group": 0,
"root_type": "Asset",
"account_type": "Bank"
}
"account_type": "Bank"
},
"Advances to Officers & Employees": {
"account_number": "1290",
@@ -110,20 +104,25 @@
"account_number": "1511",
"is_group": 0,
"root_type": "Asset"
},
"Factory Overhead Variance": {
"account_number": "1512",
"is_group": 0,
"root_type": "Asset"
}
},
"Finished Goods": {
"account_number": "1540",
"account_number": "1520",
"is_group": 1,
"root_type": "Asset",
"Finished Goods Inventory": {
"account_number": "1541",
"account_number": "1531",
"is_group": 0,
"root_type": "Asset",
"account_type": "Stock"
},
"Inventory in Transit": {
"account_number": "1542",
"account_number": "1532",
"is_group": 0,
"root_type": "Asset",
"account_type": "Stock Adjustment"
@@ -269,7 +268,7 @@
"root_type": "Asset"
}
},
"Intangible Assets": {
"System Development": {
"account_number": "1940",
"is_group": 1,
"root_type": "Asset",
@@ -278,17 +277,6 @@
"is_group": 0,
"root_type": "Asset"
}
},
"Accumulated Amortization - Intangible Assets": {
"account_number": "1950",
"is_group": 1,
"root_type": "Asset",
"Accum Amortization - System Development": {
"account_number": "1951",
"is_group": 0,
"root_type": "Asset",
"account_type": "Accumulated Depreciation"
}
}
}
},
@@ -418,7 +406,8 @@
"Customer Deposits": {
"account_number": "2500",
"is_group": 0,
"root_type": "Liability"
"root_type": "Liability",
"account_type": "Payable"
}
},
"Non Current Liabilities": {
@@ -574,28 +563,6 @@
"is_group": 0,
"root_type": "Income"
}
},
"Exchange Gain": {
"account_number": "6030",
"is_group": 1,
"root_type": "Income",
"Exchange Gain - Detail": {
"account_number": "6031",
"is_group": 0,
"root_type": "Income",
"account_type": "Indirect Income"
}
},
"Gain on Asset Disposal": {
"account_number": "6040",
"is_group": 1,
"root_type": "Income",
"Gain on Asset Disposal - Detail": {
"account_number": "6041",
"is_group": 0,
"root_type": "Income",
"account_type": "Indirect Income"
}
}
}
},
@@ -608,7 +575,7 @@
"is_group": 1,
"root_type": "Expense",
"Cost of Goods Sold": {
"account_number": "5002",
"account_number": "5010",
"is_group": 0,
"root_type": "Expense",
"account_type": "Cost of Goods Sold"
@@ -861,61 +828,20 @@
"root_type": "Expense"
}
},
"Other Expenses": {
"account_number": "5200",
"is_group": 1,
"root_type": "Expense",
"Bank Charges": {
"account_number": "5201",
"is_group": 0,
"root_type": "Expense",
"account_type": "Indirect Expense"
},
"Interest Expenses Bank": {
"account_number": "5202",
"is_group": 0,
"root_type": "Expense",
"account_type": "Indirect Expense"
},
"Write Off": {
"account_number": "5203",
"is_group": 0,
"root_type": "Expense",
"account_type": "Indirect Expense"
},
"Exchange Loss": {
"account_number": "5204",
"is_group": 0,
"root_type": "Expense",
"account_type": "Indirect Expense"
},
"Loss on Asset Disposal": {
"account_number": "5205",
"is_group": 0,
"root_type": "Expense",
"account_type": "Indirect Expense"
}
},
"Provision For Income Tax": {
"account_number": "5300",
"is_group": 0,
"root_type": "Expense",
"account_type": "Tax"
},
"Stock Adjustment": {
"account_number": "5400",
"account_number": "5200",
"is_group": 0,
"root_type": "Expense",
"account_type": "Stock Adjustment"
},
"Round Off": {
"account_number": "5500",
"account_number": "5300",
"is_group": 0,
"root_type": "Expense",
"account_type": "Round Off"
},
"Expenses Included In Valuation": {
"account_number": "5600",
"account_number": "5400",
"is_group": 0,
"root_type": "Expense",
"account_type": "Expenses Included In Valuation"

View File

@@ -138,7 +138,6 @@ def get():
_("Gain/Loss on Asset Disposal"): {"account_category": "Other Operating Income"},
_("Impairment"): {"account_category": "Operating Expenses"},
_("Tax Expense"): {"account_category": "Tax Expense"},
_("Exchange Loss"): {"account_category": "Operating Expenses"},
},
"root_type": "Expense",
},
@@ -150,7 +149,6 @@ def get():
_("Indirect Income"): {
_("Interest Income"): {"account_category": "Investment Income"},
_("Interest on Fixed Deposits"): {"account_category": "Investment Income"},
_("Exchange Gain"): {"account_category": "Other Operating Income"},
"is_group": 1,
},
"root_type": "Income",

View File

@@ -233,7 +233,6 @@ def get():
},
_("Impairment"): {"account_number": "5224", "account_category": "Operating Expenses"},
_("Tax Expense"): {"account_number": "5225", "account_category": "Tax Expense"},
_("Exchange Loss"): {"account_number": "5226", "account_category": "Operating Expenses"},
"account_number": "5200",
},
"root_type": "Expense",
@@ -251,10 +250,6 @@ def get():
"account_number": "4220",
"account_category": "Investment Income",
},
_("Exchange Gain"): {
"account_number": "4230",
"account_category": "Other Operating Income",
},
"is_group": 1,
"account_number": "4200",
},

View File

@@ -306,31 +306,6 @@ class TestAccount(ERPNextTestSuite):
acc.account_currency = "USD"
self.assertRaises(frappe.ValidationError, acc.save)
def test_stock_account_type_change_with_ledger_entries(self):
from erpnext.stock.doctype.stock_entry.stock_entry_utils import make_stock_entry
company = "_Test Company with perpetual inventory"
warehouse = "Stores - TCP1"
stock_account = get_warehouse_account(frappe.get_doc("Warehouse", warehouse))
make_stock_entry(
item_code="_Test Item",
target=warehouse,
company=company,
qty=5,
basic_rate=100,
)
account = frappe.get_doc("Account", stock_account)
self.assertEqual(account.account_type, "Stock")
account.account_type = ""
self.assertRaises(frappe.ValidationError, account.save)
account.reload()
account.account_name = f"{account.account_name} Updated"
account.save() # non-type change stays allowed
def test_account_balance(self):
from erpnext.accounts.utils import get_balance_on

View File

@@ -1,59 +1,10 @@
# Copyright (c) 2026, Frappe Technologies Pvt. Ltd. and Contributors
# Copyright (c) 2023, Frappe Technologies Pvt. Ltd. and Contributors
# See license.txt
from erpnext.accounts.doctype.account_closing_balance.account_closing_balance import (
aggregate_with_last_account_closing_balance,
generate_key,
)
# import frappe
from erpnext.tests.utils import ERPNextTestSuite
def entry(**overrides):
row = {"debit": 0, "credit": 0, "debit_in_account_currency": 0, "credit_in_account_currency": 0}
row.update(overrides)
return row
class TestAccountClosingBalance(ERPNextTestSuite):
"""The closing-balance snapshot is built by merging this period's entries with the
previous period's. These lock the merge/key logic that drives that carry-forward."""
def test_matching_entries_are_summed(self):
# this is how a prior-period balance carries forward into the current one
merged = aggregate_with_last_account_closing_balance(
[
entry(account="Cash - _TC", debit=100, debit_in_account_currency=100),
entry(
account="Cash - _TC",
debit=50,
credit=20,
debit_in_account_currency=50,
credit_in_account_currency=20,
),
],
[],
)
self.assertEqual(len(merged), 1)
row = next(iter(merged.values()))
self.assertEqual(row["debit"], 150)
self.assertEqual(row["credit"], 20)
# the account-currency columns are accumulated in the same pass
self.assertEqual(row["debit_in_account_currency"], 150)
self.assertEqual(row["credit_in_account_currency"], 20)
def test_entries_are_kept_separate_per_dimension(self):
merged = aggregate_with_last_account_closing_balance(
[
entry(account="Cash - _TC", cost_center="CC1", debit=100, debit_in_account_currency=100),
entry(account="Cash - _TC", cost_center="CC2", debit=40, debit_in_account_currency=40),
],
[],
)
self.assertEqual(len(merged), 2)
def test_period_closing_flag_is_part_of_the_key(self):
# a P&L reversal (flag 0) and a closing-account entry (flag 1) for the same
# account must not merge, so the flag has to distinguish their keys
key_reversal, _ = generate_key(entry(account="Sales - _TC", is_period_closing_voucher_entry=0), [])
key_closing, _ = generate_key(entry(account="Sales - _TC", is_period_closing_voucher_entry=1), [])
self.assertNotEqual(key_reversal, key_closing)
pass

View File

@@ -16,8 +16,6 @@ frappe.ui.form.on("Accounting Dimension", {
return {
filters: {
name: ["not in", invalid_doctypes],
istable: 0,
issingle: 0,
},
};
});

View File

@@ -60,14 +60,6 @@ class AccountingDimension(Document):
msg = _("Not allowed to create accounting dimension for {0}").format(self.document_type)
frappe.throw(msg)
meta = frappe.get_meta(self.document_type)
if meta.istable or meta.issingle:
frappe.throw(
_(
"{0} cannot be used as an accounting dimension as it is not a standalone document type."
).format(frappe.bold(self.document_type))
)
exists = frappe.db.get_value("Accounting Dimension", {"document_type": self.document_type}, ["name"])
if exists and self.is_new():
@@ -232,7 +224,7 @@ def disable_dimension(doc: str):
def toggle_disabling(doc):
doc = frappe.parse_json(doc)
doc = json.loads(doc)
if doc.get("disabled"):
df = {"read_only": 1}
@@ -367,13 +359,3 @@ def create_accounting_dimensions_for_doctype(doctype):
create_custom_field(doctype, df, ignore_validate=True)
frappe.clear_cache(doctype=doctype)
def get_dimension_fieldname(dim_doctype: str) -> str:
"""
Return the `GL Entry` fieldname for a given dimension.
"""
if dim_doctype in ("Cost Center", "Project"):
return frappe.scrub(dim_doctype)
return frappe.db.get_value("Accounting Dimension", {"document_type": dim_doctype}, "fieldname")

View File

@@ -51,23 +51,6 @@ class TestAccountingDimension(ERPNextTestSuite):
self.assertEqual(gle.get("department"), "_Test Department - _TC")
self.assertEqual(gle1.get("department"), "_Test Department - _TC")
def test_child_table_not_allowed_as_dimension(self):
dimension = frappe.get_doc({"doctype": "Accounting Dimension", "document_type": "Sales Team"})
self.assertRaises(frappe.ValidationError, dimension.insert)
def test_single_doctype_not_allowed_as_dimension(self):
dimension = frappe.get_doc({"doctype": "Accounting Dimension", "document_type": "Selling Settings"})
self.assertRaises(frappe.ValidationError, dimension.insert)
def test_non_scalar_dimension_value_skipped_in_gl_dict(self):
si = create_sales_invoice(do_not_save=1)
si.department = "_Test Department - _TC"
self.assertEqual(si.get_gl_dict({}).get("department"), "_Test Department - _TC")
si.department = ["_Test Department - _TC"]
self.assertNotIn("department", si.get_gl_dict({}))
def test_mandatory(self):
location = frappe.get_doc("Accounting Dimension", "Location")
location.dimension_defaults[0].mandatory_for_bs = True

View File

@@ -6,7 +6,7 @@ frappe.ui.form.on("Accounting Dimension Filter", {
let help_content = `<table class="table table-bordered" style="background-color: var(--scrollbar-track-color);">
<tr><td>
<p>
<svg class="icon icon-sm"><use href="#icon-info"></use></svg>
<i class="fa fa-hand-right"></i>
{{__('Note: On checking Is Mandatory the accounting dimension will become mandatory against that specific account for all accounting transactions')}}
</p>
</td></tr>

View File

@@ -5,7 +5,6 @@
import frappe
from frappe import _
from frappe.model.document import Document
from frappe.utils import getdate, nowdate
class OverlapError(frappe.ValidationError):
@@ -37,20 +36,8 @@ class AccountingPeriod(Document):
# end: auto-generated types
def validate(self):
self.validate_dates()
self.validate_overlap()
def validate_dates(self):
if getdate(self.start_date) > getdate(self.end_date):
frappe.throw(_("Start Date cannot be after End Date"))
if getdate(self.end_date) > getdate(nowdate()):
frappe.throw(
_(
"Accounting Period cannot be created for a future date. End Date {0} is after today."
).format(frappe.bold(frappe.format(self.end_date, "Date")))
)
def before_insert(self):
self.bootstrap_doctypes_for_closing()

View File

@@ -2,7 +2,7 @@
# See license.txt
import frappe
from frappe.utils import nowdate
from frappe.utils import add_months, nowdate
from erpnext.accounts.doctype.accounting_period.accounting_period import (
ClosedAccountingPeriod,
@@ -93,7 +93,7 @@ def create_accounting_period(**args):
accounting_period = frappe.new_doc("Accounting Period")
accounting_period.start_date = args.start_date or nowdate()
accounting_period.end_date = args.end_date or nowdate()
accounting_period.end_date = args.end_date or add_months(nowdate(), 1)
accounting_period.company = args.company or "_Test Company"
accounting_period.period_name = args.period_name or "_Test_Period_Name_1"
accounting_period.append("closed_documents", {"document_type": "Sales Invoice", "closed": 1})

View File

@@ -10,7 +10,7 @@ frappe.ui.form.on("Accounts Settings", {
},
};
});
if (!frm.naming_controller) frm.naming_controller = new frappe.ui.NamingSeriesController(frm);
if (!frm.naming_controller) frm.naming_controller = new erpnext.NamingSeriesController(frm);
frm.naming_controller.render_table("transaction_naming_html", get_transactions(frm));
},

View File

@@ -22,8 +22,6 @@
"allow_multi_currency_invoices_against_single_party_account",
"confirm_before_resetting_posting_date",
"preview_mode",
"stock_expense_section",
"book_stock_expense_gl_entries",
"analytics_section",
"enable_discounts_and_margin",
"enable_accounting_dimensions",
@@ -78,8 +76,6 @@
"over_billing_allowance",
"credit_controller",
"role_allowed_to_over_bill",
"enable_overdue_billing_threshold",
"role_allowed_to_bypass_overdue_billing",
"column_break_11",
"assets_tab",
"asset_settings_section",
@@ -91,18 +87,16 @@
"period_closing_settings_section",
"ignore_account_closing_balance",
"use_legacy_controller_for_pcv",
"pcv_job_timeout",
"column_break_25",
"reports_tab",
"remarks_section",
"disable_include_dimensions",
"column_break_lvjk",
"general_ledger_remarks_length",
"receivable_payable_remarks_length",
"column_break_lvjk",
"accounts_receivable_payable_tuning_section",
"receivable_payable_fetch_method",
"default_ageing_range",
"column_break_ntmi",
"receivable_payable_remarks_length",
"legacy_section",
"ignore_is_opening_check_for_reporting",
"tab_break_dpet",
@@ -200,12 +194,10 @@
},
{
"default": "1",
"description": "System will use the latest saved Currency Exchange rate on or before the transaction date, however old it is. <br>\nUncheck to ignore rates older than Stale Days and fetch a fresh rate from the exchange rate provider instead.",
"fieldname": "allow_stale",
"fieldtype": "Check",
"in_list_view": 1,
"label": "Allow Stale Exchange Rates",
"show_description_on_click": 1
"label": "Allow Stale Exchange Rates"
},
{
"default": "1",
@@ -225,8 +217,7 @@
"description": "The percentage you are allowed to bill more against the amount ordered. For example, if the order value is $100 for an item and tolerance is set as 10%, then you are allowed to bill up to $110 ",
"fieldname": "over_billing_allowance",
"fieldtype": "Currency",
"label": "Over Billing Allowance (%)",
"non_negative": 1
"label": "Over Billing Allowance (%)"
},
{
"default": "1",
@@ -280,21 +271,6 @@
"label": "Role Allowed to over bill ",
"options": "Role"
},
{
"default": "0",
"description": "Enabling this option prevents the creation of a new Sales Invoice when the customer has an overdue limit set and their outstanding overdue amount exceeds that limit.",
"fieldname": "enable_overdue_billing_threshold",
"fieldtype": "Check",
"label": "Prevent Sales Invoice when Customer is Overdue"
},
{
"depends_on": "eval:doc.enable_overdue_billing_threshold",
"description": "Users with this role can still submit invoices for customers who have crossed their Overdue Limit.",
"fieldname": "role_allowed_to_bypass_overdue_billing",
"fieldtype": "Link",
"label": "Role Allowed to Bypass Over Billing Restriction",
"options": "Role"
},
{
"fieldname": "period_closing_settings_section",
"fieldtype": "Section Break"
@@ -479,7 +455,7 @@
{
"fieldname": "remarks_section",
"fieldtype": "Section Break",
"label": "General Ledger Report"
"label": "Remarks Column Length"
},
{
"default": "0",
@@ -553,7 +529,7 @@
{
"fieldname": "accounts_receivable_payable_tuning_section",
"fieldtype": "Section Break",
"label": "Accounts Receivable / Payable Report"
"label": "Accounts Receivable / Payable Tuning"
},
{
"fieldname": "legacy_section",
@@ -636,14 +612,6 @@
"fieldtype": "Check",
"label": "Use legacy controller for Period Closing Voucher"
},
{
"default": "3600",
"depends_on": "eval: !doc.use_legacy_controller_for_pcv",
"description": "Timeout (in seconds) for each background job enqueued by Process Period Closing Voucher",
"fieldname": "pcv_job_timeout",
"fieldtype": "Int",
"label": "PCV Job Timeout (seconds)"
},
{
"description": "Users with this role will be notified if the asset depreciation gets failed",
"fieldname": "role_to_notify_on_depreciation_failure",
@@ -780,24 +748,6 @@
"description": "Changing the account in any transaction of the DocTypes listed below will trigger a repost. To prevent reposting, remove the relevant DocType from the list.",
"fieldname": "column_break_mfor",
"fieldtype": "Column Break"
},
{
"fieldname": "stock_expense_section",
"fieldtype": "Section Break",
"label": "Stock Expense Accounting"
},
{
"default": "0",
"description": "Books Purchase Expense and Expenses Added To Stock account pairs against stock value. On enabling this, the accounts become mandatory in Company or Item Defaults for Purchase Receipt, Purchase Invoice, Stock Entry, Stock Reconciliation and Landed Cost Voucher",
"fieldname": "book_stock_expense_gl_entries",
"fieldtype": "Check",
"label": "Book Stock Expense GL Entries"
},
{
"default": "0",
"fieldname": "disable_include_dimensions",
"fieldtype": "Check",
"label": "Disable \"Consider Accounting Dimension\" Filter"
}
],
"grid_page_length": 50,
@@ -806,7 +756,7 @@
"index_web_pages_for_search": 1,
"issingle": 1,
"links": [],
"modified": "2026-09-04 10:08:30.115003",
"modified": "2026-06-03 13:11:54.721495",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Accounts Settings",

View File

@@ -62,7 +62,6 @@ class AccountsSettings(Document):
book_asset_depreciation_entry_automatically: DF.Check
book_deferred_entries_based_on: DF.Literal["Days", "Months"]
book_deferred_entries_via_journal_entry: DF.Check
book_stock_expense_gl_entries: DF.Check
book_tax_discount_loss: DF.Check
calculate_depr_using_total_days: DF.Check
check_supplier_invoice_uniqueness: DF.Check
@@ -72,14 +71,12 @@ class AccountsSettings(Document):
default_ageing_range: DF.Data | None
delete_linked_ledger_entries: DF.Check
determine_address_tax_category_from: DF.Literal["Billing Address", "Shipping Address"]
disable_include_dimensions: DF.Check
enable_accounting_dimensions: DF.Check
enable_common_party_accounting: DF.Check
enable_discounts_and_margin: DF.Check
enable_fuzzy_matching: DF.Check
enable_immutable_ledger: DF.Check
enable_loyalty_point_program: DF.Check
enable_overdue_billing_threshold: DF.Check
enable_party_matching: DF.Check
enable_subscription: DF.Check
exchange_gain_loss_posting_date: DF.Literal["Invoice", "Payment", "Reconciliation Date"]
@@ -93,13 +90,11 @@ class AccountsSettings(Document):
make_payment_via_journal_entry: DF.Check
merge_similar_account_heads: DF.Check
over_billing_allowance: DF.Currency
pcv_job_timeout: DF.Int
preview_mode: DF.Check
receivable_payable_fetch_method: DF.Literal["Buffered Cursor", "UnBuffered Cursor"]
receivable_payable_remarks_length: DF.Int
reconciliation_queue_size: DF.Int
repost_allowed_types: DF.Table[RepostAllowedTypes]
role_allowed_to_bypass_overdue_billing: DF.Link | None
role_allowed_to_over_bill: DF.Link | None
role_to_notify_on_depreciation_failure: DF.Link | None
role_to_override_stop_action: DF.Link | None
@@ -155,10 +150,6 @@ class AccountsSettings(Document):
toggle_subscription_sections(not self.enable_subscription)
clear_cache = True
if old_doc.enable_overdue_billing_threshold != self.enable_overdue_billing_threshold:
toggle_overdue_billing_threshold_field(not self.enable_overdue_billing_threshold)
clear_cache = True
if clear_cache:
frappe.clear_cache()
@@ -204,8 +195,8 @@ class AccountsSettings(Document):
if self.add_taxes_from_item_tax_template and self.add_taxes_from_taxes_and_charges_template:
frappe.throw(
_("You cannot enable both the settings '{0}' and '{1}'.").format(
frappe.bold(self.meta.get_translated_label("add_taxes_from_item_tax_template")),
frappe.bold(self.meta.get_translated_label("add_taxes_from_taxes_and_charges_template")),
frappe.bold(_(self.meta.get_label("add_taxes_from_item_tax_template"))),
frappe.bold(_(self.meta.get_label("add_taxes_from_taxes_and_charges_template"))),
),
title=_("Auto Tax Settings Error"),
)
@@ -250,10 +241,6 @@ def toggle_subscription_sections(hide):
create_property_setter_for_hiding_field(doctype, "subscription_section", hide)
def toggle_overdue_billing_threshold_field(hide):
create_property_setter_for_hiding_field("Customer Credit Limit", "overdue_billing_threshold", hide)
def create_property_setter_for_hiding_field(doctype, field_name, hide):
make_property_setter(
doctype,

View File

@@ -22,13 +22,11 @@ class TestAdvancePaymentLedgerEntry(ERPNextTestSuite, AccountsTestMixin):
"""
def setUp(self):
self.company = "_Test Company"
self.customer = "_Test Customer"
self.supplier = "_Test Supplier"
self.item = "_Test Item"
self.cash = "Cash - _TC"
self.debtors_usd = "_Test Receivable USD - _TC"
self.creditors_usd = "_Test Payable USD - _TC"
self.create_company()
self.create_usd_receivable_account()
self.create_usd_payable_account()
self.create_item()
self.clear_old_entries()
def create_sales_order(self, qty=1, rate=100, currency="INR", do_not_submit=False):
"""

View File

@@ -101,7 +101,7 @@
}
],
"links": [],
"modified": "2026-08-21 23:11:39.423431",
"modified": "2024-03-27 13:06:36.896195",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Bank",
@@ -118,20 +118,11 @@
"role": "System Manager",
"share": 1,
"write": 1
},
{
"role": "Accounts Manager",
"select": 1
},
{
"role": "Accounts User",
"select": 1
}
],
"quick_entry": 1,
"row_format": "Dynamic",
"sort_field": "creation",
"sort_order": "DESC",
"states": [],
"track_changes": 1
}
}

View File

@@ -269,7 +269,7 @@
"link_fieldname": "default_bank_account"
}
],
"modified": "2026-08-21 23:11:39.585456",
"modified": "2026-04-11 19:46:27.609994",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Bank Account",
@@ -299,22 +299,6 @@
"role": "Accounts User",
"share": 1,
"write": 1
},
{
"role": "Purchase Manager",
"select": 1
},
{
"role": "Purchase Master Manager",
"select": 1
},
{
"role": "Sales Master Manager",
"select": 1
},
{
"role": "Sales User",
"select": 1
}
],
"row_format": "Dynamic",

View File

@@ -107,7 +107,7 @@ def get_party_bank_account(party_type, party):
)
def get_default_company_bank_account(company, party_type, party, ignore_permissions=True):
def get_default_company_bank_account(company, party_type, party):
default_company_bank_account = frappe.db.get_value(party_type, party, "default_bank_account")
if default_company_bank_account:
if company != frappe.get_cached_value("Bank Account", default_company_bank_account, "company"):
@@ -118,14 +118,6 @@ def get_default_company_bank_account(company, party_type, party, ignore_permissi
"Bank Account", {"company": company, "is_company_account": 1, "is_default": 1}
)
if not ignore_permissions:
default_company_bank_account = (
default_company_bank_account
if default_company_bank_account
and frappe.get_cached_doc("Bank Account", default_company_bank_account).has_permission("select")
else None
)
return default_company_bank_account
@@ -196,7 +188,7 @@ def get_closing_balance_as_per_statement(bank_account: str, date: str):
return {"balance": 0, "date": None}
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def set_closing_balance_as_per_statement(bank_account: str, date: str | datetime.date, balance: float):
"""
Set the closing balance as per statement for a bank account and date

View File

@@ -1,6 +1,5 @@
{
"actions": [],
"allow_bulk_edit": 1,
"allow_rename": 1,
"creation": "2026-04-11 19:48:13.622253",
"doctype": "DocType",
@@ -8,8 +7,7 @@
"field_order": [
"bank_account",
"date",
"balance",
"company"
"balance"
],
"fields": [
{
@@ -33,20 +31,12 @@
"in_list_view": 1,
"label": "Balance",
"reqd": 1
},
{
"fetch_from": "bank_account.company",
"fieldname": "company",
"fieldtype": "Link",
"label": "Company",
"options": "Company",
"read_only": 1
}
],
"grid_page_length": 50,
"index_web_pages_for_search": 1,
"links": [],
"modified": "2026-06-16 22:17:48.007982",
"modified": "2026-04-11 19:49:45.374695",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Bank Account Balance",

View File

@@ -16,7 +16,6 @@ class BankAccountBalance(Document):
balance: DF.Currency
bank_account: DF.Link
company: DF.Link | None
date: DF.Date
# end: auto-generated types

View File

@@ -7,7 +7,7 @@ from frappe import _, msgprint
from frappe.model.document import Document
from frappe.query_builder import Case
from frappe.query_builder.custom import ConstantColumn
from frappe.query_builder.functions import Coalesce, Max, Sum
from frappe.query_builder.functions import Coalesce, Sum
from frappe.utils import cint, flt, fmt_money, getdate
from pypika import Order
@@ -94,7 +94,6 @@ class BankClearance(Document):
invalid_document = []
invalid_cheque_date = []
entries_to_update = []
self.check_permission("write")
def validate_entry(d):
is_valid = True
@@ -195,17 +194,14 @@ def get_payment_entries_for_bank_clearance(
.select(
ConstantColumn("Journal Entry").as_("payment_document"),
journal_entry.name.as_("payment_entry"),
# non-grouped columns are constant per grouped JE name / account (against_account is
# arbitrary per group on MySQL) -> Max() keeps the GROUP BY valid on postgres with the
# same value MySQL picked.
Max(journal_entry.cheque_no).as_("cheque_number"),
Max(journal_entry.cheque_date).as_("cheque_date"),
journal_entry.cheque_no.as_("cheque_number"),
journal_entry.cheque_date,
Sum(journal_entry_account.debit_in_account_currency).as_("debit"),
Sum(journal_entry_account.credit_in_account_currency).as_("credit"),
Max(journal_entry.posting_date).as_("posting_date"),
Max(journal_entry_account.against_account).as_("against_account"),
Max(journal_entry.clearance_date).as_("clearance_date"),
Max(journal_entry_account.account_currency).as_("account_currency"),
journal_entry.posting_date,
journal_entry_account.against_account,
journal_entry.clearance_date,
journal_entry_account.account_currency,
)
.where(
(journal_entry_account.account == account)
@@ -218,13 +214,12 @@ def get_payment_entries_for_bank_clearance(
if not include_reconciled_entries:
journal_entry_query = journal_entry_query.where(
(journal_entry.clearance_date.isnull())
| (journal_entry.clearance_date == ("0000-00-00" if frappe.db.db_type != "postgres" else None))
(journal_entry.clearance_date.isnull()) | (journal_entry.clearance_date == "0000-00-00")
)
journal_entries = (
journal_entry_query.groupby(journal_entry_account.account, journal_entry.name)
.orderby(Max(journal_entry.posting_date))
.orderby(journal_entry.posting_date)
.orderby(journal_entry.name, order=Order.desc)
).run(as_dict=True)
@@ -294,8 +289,7 @@ def get_payment_entries_for_bank_clearance(
if not include_reconciled_entries:
payment_entry_query = payment_entry_query.where(
(pe.clearance_date.isnull())
| (pe.clearance_date == ("0000-00-00" if frappe.db.db_type != "postgres" else None))
(pe.clearance_date.isnull()) | (pe.clearance_date == "0000-00-00")
)
payment_entries = (payment_entry_query.orderby(pe.posting_date).orderby(pe.name, order=Order.desc)).run(
@@ -332,8 +326,7 @@ def get_payment_entries_for_bank_clearance(
if not include_reconciled_entries:
paid_purchase_invoices_query = paid_purchase_invoices_query.where(
(pi.clearance_date.isnull())
| (pi.clearance_date == ("0000-00-00" if frappe.db.db_type != "postgres" else None))
(pi.clearance_date.isnull()) | (pi.clearance_date == "0000-00-00")
)
paid_purchase_invoices = (
@@ -373,8 +366,7 @@ def get_payment_entries_for_bank_clearance(
if not include_reconciled_entries:
pos_sales_invoices_query = pos_sales_invoices_query.where(
(si_payment.clearance_date.isnull())
| (si_payment.clearance_date == ("0000-00-00" if frappe.db.db_type != "postgres" else None))
(si_payment.clearance_date.isnull()) | (si_payment.clearance_date == "0000-00-00")
)
pos_sales_invoices = (

View File

@@ -4,18 +4,29 @@
import frappe
from frappe.utils import add_months, getdate
from erpnext.accounts.doctype.cost_center.test_cost_center import create_cost_center
from erpnext.accounts.doctype.mode_of_payment.test_mode_of_payment import (
set_default_account_for_mode_of_payment,
)
from erpnext.accounts.doctype.payment_entry.test_payment_entry import get_payment_entry
from erpnext.accounts.doctype.purchase_invoice.test_purchase_invoice import make_purchase_invoice
from erpnext.accounts.doctype.sales_invoice.test_sales_invoice import create_sales_invoice
from erpnext.stock.doctype.item.test_item import create_item
from erpnext.stock.doctype.warehouse.test_warehouse import create_warehouse
from erpnext.tests.utils import ERPNextTestSuite, if_lending_app_installed, if_lending_app_not_installed
class TestBankClearance(ERPNextTestSuite):
def setUp(self):
frappe.clear_cache()
create_warehouse(
warehouse_name="_Test Warehouse",
properties={"parent_warehouse": "All Warehouses - _TC"},
company="_Test Company",
)
create_item("_Test Item")
create_cost_center(cost_center_name="_Test Cost Center", company="_Test Company")
make_bank_account()
add_transactions()
@@ -128,8 +139,11 @@ def add_transactions():
def make_payment_entry():
from erpnext.buying.doctype.supplier.test_supplier import create_supplier
supplier = create_supplier(supplier_name="_Test Supplier")
pi = make_purchase_invoice(
supplier="_Test Supplier",
supplier=supplier.name,
supplier_warehouse="_Test Warehouse - _TC",
expense_account="Cost of Goods Sold - _TC",
uom="Nos",
@@ -144,6 +158,10 @@ def make_payment_entry():
def make_pos_sales_invoice():
from erpnext.accounts.doctype.opening_invoice_creation_tool.test_opening_invoice_creation_tool import (
make_customer,
)
mode_of_payment = frappe.get_doc({"doctype": "Mode of Payment", "name": "Cash"})
if not frappe.db.get_value("Mode of Payment Account", {"company": "_Test Company", "parent": "Cash"}):
@@ -152,13 +170,13 @@ def make_pos_sales_invoice():
)
mode_of_payment.save()
customer = make_customer(customer="_Test Customer")
mode_of_payment = frappe.get_doc("Mode of Payment", "Wire Transfer")
set_default_account_for_mode_of_payment(mode_of_payment, "_Test Company", "_Test Bank Clearance - _TC")
si = create_sales_invoice(
customer="_Test Customer", item="_Test Item", is_pos=1, qty=1, rate=1000, do_not_save=1
)
si = create_sales_invoice(customer=customer, item="_Test Item", is_pos=1, qty=1, rate=1000, do_not_save=1)
si.set("payments", [])
si.append("payments", {"mode_of_payment": "Wire Transfer", "amount": 1000})
si.insert()

View File

@@ -9,13 +9,6 @@ cur_frm.add_fetch("bank", "swift_number", "swift_number");
frappe.ui.form.on("Bank Guarantee", {
setup: function (frm) {
frm.set_query("reference_doctype", function () {
return {
filters: {
name: ["in", ["Sales Order", "Purchase Order"]],
},
};
});
frm.set_query("bank_account", function () {
return {
filters: {

View File

@@ -1,6 +1,5 @@
{
"actions": [],
"allow_bulk_edit": 1,
"autoname": "ACC-BG-.YYYY.-.#####",
"creation": "2016-12-17 10:43:35.731631",
"doctype": "DocType",
@@ -51,7 +50,8 @@
"fieldname": "reference_doctype",
"fieldtype": "Link",
"label": "Reference Document Type",
"options": "DocType"
"options": "DocType",
"read_only": 1
},
{
"fieldname": "reference_docname",
@@ -60,14 +60,14 @@
"options": "reference_doctype"
},
{
"depends_on": "eval: doc.reference_doctype == \"Sales Order\"",
"depends_on": "eval: doc.bg_type == \"Receiving\"",
"fieldname": "customer",
"fieldtype": "Link",
"label": "Customer",
"options": "Customer"
},
{
"depends_on": "eval: doc.reference_doctype == \"Purchase Order\"",
"depends_on": "eval: doc.bg_type == \"Providing\"",
"fieldname": "supplier",
"fieldtype": "Link",
"label": "Supplier",
@@ -218,11 +218,10 @@
"grid_page_length": 50,
"is_submittable": 1,
"links": [],
"modified": "2026-05-25 18:12:10.768835",
"modified": "2025-08-29 11:52:33.550847",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Bank Guarantee",
"naming_rule": "Expression",
"owner": "Administrator",
"permissions": [
{

View File

@@ -1,76 +1,8 @@
# Copyright (c) 2026, Frappe Technologies Pvt. Ltd. and Contributors
# Copyright (c) 2018, Frappe Technologies Pvt. Ltd. and Contributors
# See license.txt
import frappe
from frappe.utils import flt
from erpnext.accounts.doctype.bank_guarantee.bank_guarantee import get_voucher_details
from erpnext.selling.doctype.sales_order.test_sales_order import make_sales_order
from erpnext.tests.utils import ERPNextTestSuite
BANK = "_Test BG Bank"
class TestBankGuarantee(ERPNextTestSuite):
"""Bank Guarantee records a guarantee issued/received against a customer or
supplier. validate() needs a party; on_submit() needs the bank details filled in."""
def setUp(self):
frappe.set_user("Administrator")
if not frappe.db.exists("Bank", BANK):
frappe.get_doc({"doctype": "Bank", "bank_name": BANK}).insert()
def make_bg(self, **args):
args = frappe._dict(args)
doc = frappe.new_doc("Bank Guarantee")
doc.bg_type = args.bg_type or "Receiving"
doc.amount = args.amount if args.amount is not None else 1000
doc.start_date = args.start_date or "2026-06-01"
if args.end_date:
doc.end_date = args.end_date
doc.customer = args.get("customer", "_Test Customer")
doc.supplier = args.get("supplier")
# fields on_submit requires — present by default, cleared per-test to assert the guard
doc.bank_guarantee_number = args.get("bank_guarantee_number", "BG-001")
doc.name_of_beneficiary = args.get("name_of_beneficiary", "Test Beneficiary")
doc.bank = args.get("bank", BANK)
return doc
def test_validate_requires_customer_or_supplier(self):
doc = self.make_bg(customer=None)
self.assertRaises(frappe.ValidationError, doc.insert)
def test_submit_requires_guarantee_number(self):
doc = self.make_bg(bank_guarantee_number="")
doc.insert()
self.assertRaises(frappe.ValidationError, doc.submit)
def test_submit_requires_beneficiary_name(self):
doc = self.make_bg(name_of_beneficiary="")
doc.insert()
self.assertRaises(frappe.ValidationError, doc.submit)
def test_submit_requires_bank(self):
doc = self.make_bg(bank="")
doc.insert()
self.assertRaises(frappe.ValidationError, doc.submit)
def test_valid_guarantee_submits(self):
doc = self.make_bg()
doc.insert()
doc.submit()
self.assertEqual(frappe.db.get_value("Bank Guarantee", doc.name, "docstatus"), 1)
def test_get_voucher_details_for_receiving(self):
so = make_sales_order()
details = get_voucher_details("Receiving", so.name)
self.assertEqual(details.customer, so.customer)
self.assertEqual(flt(details.grand_total), flt(so.grand_total))
def test_end_date_before_start_date_is_not_validated(self):
# SUSPECTED BUG: validate() never checks that end_date >= start_date, so a
# guarantee that expires before it starts saves cleanly. Locking the current
# (wrong) behaviour so a future fix that adds the check trips this test.
doc = self.make_bg(start_date="2026-06-30", end_date="2026-06-01")
doc.insert()
self.assertTrue(frappe.db.exists("Bank Guarantee", doc.name))
pass

View File

@@ -8,7 +8,7 @@ import frappe
from frappe import _
from frappe.model.document import Document
from frappe.query_builder.custom import ConstantColumn
from frappe.query_builder.functions import Max, Sum
from frappe.query_builder.functions import Sum
from frappe.utils import cint, create_batch, flt
from erpnext import get_default_cost_center
@@ -116,7 +116,7 @@ def get_account_balance(bank_account: str, till_date: str | date, company: str):
return flt(balance_as_per_system) - flt(total_debit) + flt(total_credit) + amounts_not_reflected_in_system
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def update_bank_transaction(
bank_transaction_name: str, reference_number: str, party_type: str | None = None, party: str | None = None
):
@@ -146,7 +146,7 @@ def update_bank_transaction(
)[0]
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def create_journal_entry_bts(
bank_transaction_name: str,
reference_number: str | None = None,
@@ -305,7 +305,7 @@ def create_journal_entry_bts(
return reconcile_vouchers(bank_transaction_name, vouchers, is_new_voucher=True)
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def create_payment_entry_bts(
bank_transaction_name: str,
reference_number: str | None = None,
@@ -500,7 +500,7 @@ def create_bulk_internal_transfer(bank_transaction_names: list[str | int], bank_
return output
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def create_internal_transfer(
bank_transaction_name: str | int,
posting_date: str | date,
@@ -518,7 +518,6 @@ def create_internal_transfer(
"""
bank_transaction = frappe.get_doc("Bank Transaction", bank_transaction_name)
bank_transaction.check_permission("write")
bank_account = frappe.get_cached_value("Bank Account", bank_transaction.bank_account, "account")
company = frappe.get_cached_value("Account", bank_account, "company")
@@ -779,6 +778,7 @@ def create_bulk_payment_entry_and_reconcile(
"""
Create a payment entry and reconcile it with the bank transaction
"""
output = []
for bank_transaction_name in bank_transaction_names:
@@ -912,7 +912,7 @@ def search_for_transfer_transaction(transaction_id: str | int):
days = frappe.db.get_single_value("Accounts Settings", "transfer_match_days")
if days is None:
if not days:
days = 3
min_date = frappe.utils.add_days(date, -days)
@@ -1057,10 +1057,10 @@ def get_auto_reconcile_message(partially_reconciled, reconciled):
return alert_message, indicator
@frappe.whitelist(methods=["POST"])
def reconcile_vouchers(bank_transaction_name: str | int, vouchers: str | list, is_new_voucher: bool = False):
@frappe.whitelist()
def reconcile_vouchers(bank_transaction_name: str | int, vouchers: str, is_new_voucher: bool = False):
# updated clear date of all the vouchers based on the bank transaction
vouchers = frappe.parse_json(vouchers)
vouchers = json.loads(vouchers)
transaction = frappe.get_doc("Bank Transaction", bank_transaction_name)
transaction.add_payment_entries(vouchers, is_new_voucher)
transaction.validate_duplicate_references()
@@ -1336,11 +1336,9 @@ def get_pe_matching_query(
ref_condition = pe.reference_no == transaction.reference_number
ref_rank = frappe.qb.terms.Case().when(ref_condition, 1).else_(0)
amount_field = pe.received_amount_after_tax if account_from_to == "paid_to" else pe.paid_amount_after_tax
amount_equality = amount_field == transaction.unallocated_amount
amount_equality = pe.paid_amount == transaction.unallocated_amount
amount_rank = frappe.qb.terms.Case().when(amount_equality, 1).else_(0)
amount_condition = amount_equality if exact_match else amount_field > 0.0
amount_condition = amount_equality if exact_match else pe.paid_amount > 0.0
party_condition = (
(pe.party_type == transaction.party_type) & (pe.party == transaction.party) & pe.party.isnotnull()
@@ -1357,7 +1355,7 @@ def get_pe_matching_query(
(ref_rank + amount_rank + party_rank + 1).as_("rank"),
ConstantColumn("Payment Entry").as_("doctype"),
pe.name,
amount_field.as_("paid_amount"),
pe.base_paid_amount_after_tax.as_("paid_amount"),
pe.reference_no,
pe.reference_date,
pe.party,
@@ -1412,14 +1410,12 @@ def get_je_matching_query(
Sum(getattr(jea, amount_field)).as_("paid_amount"),
ConstantColumn("Journal Entry").as_("doctype"),
je.name,
# non-grouped columns are constant per grouped JE name (party_type/currency come from the
# single bank-account line) -> Max() keeps the GROUP BY valid on postgres with the same value
Max(je.cheque_no).as_("reference_no"),
Max(je.cheque_date).as_("reference_date"),
Max(je.pay_to_recd_from).as_("party"),
Max(jea.party_type).as_("party_type"),
Max(je.posting_date).as_("posting_date"),
Max(jea.account_currency).as_("currency"),
je.cheque_no.as_("reference_no"),
je.cheque_date.as_("reference_date"),
je.pay_to_recd_from.as_("party"),
jea.party_type,
je.posting_date,
jea.account_currency.as_("currency"),
)
.where(je.docstatus == 1)
.where(je.voucher_type != "Opening Entry")
@@ -1427,7 +1423,7 @@ def get_je_matching_query(
.where(jea.account == common_filters.bank_account)
.where(filter_by_date)
.groupby(je.name)
.orderby(Max(je.cheque_date) if cint(filter_by_reference_date) else Max(je.posting_date))
.orderby(je.cheque_date if cint(filter_by_reference_date) else je.posting_date)
)
if frappe.flags.auto_reconcile_vouchers is True:

View File

@@ -8,9 +8,7 @@ from frappe.utils import add_days, today
from erpnext.accounts.doctype.bank_reconciliation_tool.bank_reconciliation_tool import (
auto_reconcile_vouchers,
get_auto_reconcile_message,
get_bank_transactions,
get_linked_payments,
)
from erpnext.accounts.doctype.payment_entry.test_payment_entry import create_payment_entry
from erpnext.accounts.test.accounts_mixin import AccountsTestMixin
@@ -19,10 +17,9 @@ from erpnext.tests.utils import ERPNextTestSuite
class TestBankReconciliationTool(ERPNextTestSuite, AccountsTestMixin):
def setUp(self):
self.company = "_Test Company"
self.customer = "_Test Customer"
self.bank = "HDFC - _TC"
self.debit_to = "Debtors - _TC"
self.create_company()
self.create_customer()
self.clear_old_entries()
bank_dt = qb.DocType("Bank")
qb.from_(bank_dt).delete().where(bank_dt.name == "HDFC").run()
self.create_bank_account()
@@ -99,103 +96,3 @@ class TestBankReconciliationTool(ERPNextTestSuite, AccountsTestMixin):
# assert API output post reconciliation
transactions = get_bank_transactions(self.bank_account, from_date, to_date)
self.assertEqual(len(transactions), 0)
def make_bank_transaction(self, date, deposit=100, withdrawal=0):
return (
frappe.get_doc(
{
"doctype": "Bank Transaction",
"date": date,
"deposit": deposit,
"withdrawal": withdrawal,
"bank_account": self.bank_account,
"currency": "INR",
}
)
.save()
.submit()
)
def get_matching_payment_entries(self, bank_transaction, exact_match=False):
document_types = ["payment_entry", "exact_match"] if exact_match else ["payment_entry"]
vouchers = get_linked_payments(
bank_transaction,
document_types,
from_date=add_days(today(), -1),
to_date=today(),
)
return [v for v in vouchers if v.get("doctype") == "Payment Entry"]
def test_get_bank_transactions_excludes_dates_after_to_date(self):
self.make_bank_transaction(date=today())
names = [t.name for t in get_bank_transactions(self.bank_account, to_date=add_days(today(), -1))]
self.assertEqual(names, [])
def test_deposit_matches_amount_received_in_bank_account(self):
# money leaves another bank account and lands here minus a charge, so the two sides differ
payment = frappe.get_doc(
{
"doctype": "Payment Entry",
"payment_type": "Internal Transfer",
"company": self.company,
"posting_date": today(),
"paid_from": "_Test Bank - _TC",
"paid_to": self.bank,
"paid_amount": 3537.64,
"received_amount": 3460.52,
"reference_no": "TRF-001",
"reference_date": today(),
}
)
payment.set_missing_values()
payment.set_exchange_rate()
payment.set_amounts()
payment.deductions[-1].account = "_Test Exchange Gain/Loss - _TC"
payment.deductions[-1].cost_center = "_Test Cost Center - _TC"
payment = payment.save().submit()
transaction = self.make_bank_transaction(date=today(), deposit=3460.52)
# the received side is what reached this bank account, so that is what is shown
matches = self.get_matching_payment_entries(transaction.name)
self.assertEqual([m["name"] for m in matches], [payment.name])
self.assertEqual(matches[0]["paid_amount"], 3460.52)
# and what the exact match compares against
exact_matches = self.get_matching_payment_entries(transaction.name, exact_match=True)
self.assertEqual([m["name"] for m in exact_matches], [payment.name])
def test_withdrawal_matches_amount_paid_from_bank_account(self):
payment = create_payment_entry(
company=self.company,
payment_type="Pay",
party_type="Supplier",
party="_Test Supplier",
paid_from=self.bank,
paid_to="Creditors - _TC",
paid_amount=1250,
)
payment = payment.save().submit()
transaction = self.make_bank_transaction(date=today(), deposit=0, withdrawal=1250)
exact_matches = self.get_matching_payment_entries(transaction.name, exact_match=True)
self.assertEqual([m["name"] for m in exact_matches], [payment.name])
self.assertEqual(exact_matches[0]["paid_amount"], 1250)
def test_auto_reconcile_message_for_no_matches(self):
message, indicator = get_auto_reconcile_message([], [])
self.assertEqual(indicator, "blue")
self.assertIn("No matches", message)
def test_auto_reconcile_message_counts_and_pluralizes(self):
# reconciled count is reported and the indicator turns green
message, indicator = get_auto_reconcile_message([], ["t1", "t2"])
self.assertEqual(indicator, "green")
self.assertIn("2 Transaction(s) Reconciled", message)
# partially-reconciled label is singular for one, plural for many
singular, _ = get_auto_reconcile_message(["p1"], [])
self.assertIn("1 Transaction Partially Reconciled", singular)
plural, _ = get_auto_reconcile_message(["p1", "p2"], [])
self.assertIn("2 Transactions Partially Reconciled", plural)

View File

@@ -221,12 +221,12 @@
"default": "0",
"fieldname": "import_mt940_fromat",
"fieldtype": "Check",
"label": "Import MT940 Format"
"label": "Import MT940 Fromat"
}
],
"hide_toolbar": 1,
"links": [],
"modified": "2026-06-19 14:18:00.000000",
"modified": "2026-05-31 00:41:11.251215",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Bank Statement Import",

View File

@@ -142,34 +142,9 @@ def preprocess_mt940_content(content: str) -> str:
return processed_content
MT940_CUSTOMER_REFERENCE_MAX_LEN = 16
def get_transaction_reference(txn_data: dict) -> str:
"""Extract the per-transaction reference from an MT940 :61: tag.
The mt940 library exposes ``transaction_reference`` from the :20: tag, which is the
statement-level reference and identical for every transaction in a statement. The
real per-transaction reference is ``customer_reference`` (with any overflow captured
into ``extra_details`` when a bank emits a single-line :61: longer than 16 chars).
"""
customer_reference = (txn_data.get("customer_reference") or "").strip()
if len(customer_reference) == MT940_CUSTOMER_REFERENCE_MAX_LEN:
customer_reference += (txn_data.get("extra_details") or "").strip()
if customer_reference and customer_reference.upper() != "NONREF":
return customer_reference
return (txn_data.get("bank_reference") or "").strip() or (
txn_data.get("transaction_reference") or ""
).strip()
@frappe.whitelist(methods=["POST"])
@frappe.whitelist()
def convert_mt940_to_csv(data_import: str, mt940_file_path: str):
doc = frappe.get_doc("Bank Statement Import", data_import)
doc.check_permission("write")
_file_doc, content = get_file(mt940_file_path)
@@ -214,8 +189,8 @@ def convert_mt940_to_csv(data_import: str, mt940_file_path: str):
deposit = amount_value if amount_value > 0 else ""
withdrawal = abs(amount_value) if amount_value < 0 else ""
description = txn.data.get("transaction_details") or txn.data.get("extra_details") or ""
reference = get_transaction_reference(txn.data)
description = txn.data.get("extra_details") or ""
reference = txn.data.get("transaction_reference") or ""
currency = txn.data.get("currency", "")
writer.writerow([date_str, deposit, withdrawal, description, reference, doc.bank_account, currency])
@@ -236,30 +211,26 @@ def convert_mt940_to_csv(data_import: str, mt940_file_path: str):
def get_preview_from_template(
data_import: str, import_file: str | None = None, google_sheets_url: str | None = None
):
bsi = frappe.get_doc("Bank Statement Import", data_import)
bsi.check_permission()
return bsi.get_preview_from_template(import_file, google_sheets_url)
return frappe.get_doc("Bank Statement Import", data_import).get_preview_from_template(
import_file, google_sheets_url
)
@frappe.whitelist()
def form_start_import(data_import: str):
bsi = frappe.get_doc("Bank Statement Import", data_import)
bsi.check_permission("write")
return bsi.start_import()
job_id = frappe.get_doc("Bank Statement Import", data_import).start_import()
return job_id is not None
@frappe.whitelist()
def download_errored_template(data_import_name: str):
data_import = frappe.get_doc("Bank Statement Import", data_import_name)
data_import.check_permission()
data_import.export_errored_rows()
@frappe.whitelist()
def download_import_log(data_import_name: str):
bsi = frappe.get_doc("Bank Statement Import", data_import_name)
bsi.check_permission()
return bsi.download_import_log()
return frappe.get_doc("Bank Statement Import", data_import_name).download_import_log()
def is_mt940_format(content: str) -> bool:
@@ -319,7 +290,7 @@ def update_mapping_db(bank, template_options):
for d in bank.bank_transaction_mapping:
d.delete()
for d in frappe.parse_json(template_options)["column_to_field_map"].items():
for d in json.loads(template_options)["column_to_field_map"].items():
bank.append("bank_transaction_mapping", {"bank_transaction_field": d[1], "file_field": d[0]})
bank.save()
@@ -336,7 +307,7 @@ def add_bank_account(data, bank_account):
bank_account_loc = loc
for row in data[1:]:
if bank_account_loc is not None:
if bank_account_loc:
row[bank_account_loc] = bank_account
else:
row.append(bank_account)
@@ -398,7 +369,6 @@ def get_import_status(docname: str):
import_status = {}
data_import = frappe.get_doc("Bank Statement Import", docname)
data_import.check_permission()
import_status["status"] = data_import.status
logs = frappe.get_all(

View File

@@ -1,10 +1,7 @@
# Copyright (c) 2020, Frappe Technologies and Contributors
# See license.txt
import mt940
from erpnext.accounts.doctype.bank_statement_import.bank_statement_import import (
get_transaction_reference,
is_mt940_format,
preprocess_mt940_content,
)
@@ -191,135 +188,6 @@ class TestBankStatementImport(ERPNextTestSuite):
self.assertIn(":20:STMTREF167619", result) # Reference should remain unchanged
self.assertIn("UPI/TEST USER/123456789/PaidViaTestApp", result)
def test_get_transaction_reference_uses_customer_reference(self):
"""Per-transaction reference must come from :61: customer_reference, not :20:."""
self.assertEqual(
get_transaction_reference(
{"customer_reference": "UPI-100000000001", "transaction_reference": "STMTREF12345"}
),
"UPI-100000000001",
)
def test_get_transaction_reference_rejoins_overflow(self):
"""When a bank emits a single-line :61: with >16-char reference, the regex
splits the tail into extra_details. We must rejoin them."""
self.assertEqual(
get_transaction_reference(
{
"customer_reference": "NEFTINW-12345678",
"extra_details": "90",
"transaction_reference": "STMTREF12345",
}
),
"NEFTINW-1234567890",
)
def test_get_transaction_reference_falls_back_to_bank_reference_on_nonref(self):
"""NONREF is the MT940 'no customer reference' sentinel; prefer bank_reference."""
self.assertEqual(
get_transaction_reference(
{
"customer_reference": "NONREF",
"bank_reference": "1234567890123456",
"transaction_reference": "STMTREF12345",
}
),
"1234567890123456",
)
def test_get_transaction_reference_falls_back_to_bank_reference_on_nonref_with_extra_details(self):
"""NONREF sentinel must trigger the bank_reference fallback even when
extra_details is populated. Without the 16-char gate, the old naive concat
would produce a junk reference like 'NONREFsome info' and bypass the check."""
self.assertEqual(
get_transaction_reference(
{
"customer_reference": "NONREF",
"extra_details": "some info",
"bank_reference": "1234567890123456",
"transaction_reference": "STMTREF12345",
}
),
"1234567890123456",
)
def test_get_transaction_reference_does_not_append_extra_details_below_16_chars(self):
"""When customer_reference is below the 16-char cap, extra_details is a
genuine supplementary-info field from :61: — not overflow — and must not
be appended to the reference."""
self.assertEqual(
get_transaction_reference(
{
"customer_reference": "TBMS-123456789",
"extra_details": "note field",
"transaction_reference": "STMTREF12345",
}
),
"TBMS-123456789",
)
def test_get_transaction_reference_keeps_noref_literal(self):
"""Bare 'NOREF' (without bank_reference) stays as-is; still better than the
statement-level reference which is identical across all transactions."""
self.assertEqual(
get_transaction_reference(
{
"customer_reference": "NOREF",
"bank_reference": None,
"transaction_reference": "STMTREF12345",
}
),
"NOREF",
)
def test_mt940_parse_per_transaction_reference_mapping(self):
"""End-to-end: every transaction in a statement must get its own distinct
reference from :61: customer_reference, never the statement-level :20: reference."""
mt940_content = """{1:F0112345678901X0000000000}{2:I94012345678901XN}{4:
:20:STMTREF12345
:25:1234567890
:28C:12345/1
:60F:C250716INR88123,38
:61:2509280928D5000,00NMSCUPI-100000000001
:86:UPI/TEST PAYEE ONE/111111111111/TestApp
:61:2509190919D2606,00NMSCUPI-100000000002
:86:UPI/TEST PAYEE TWO/222222222222/TestApp
:61:2509190919D900,00NMSCUPI-100000000003
:86:UPI/TEST PAYEE THREE/333333333333/TestApp
:61:2508140814D5000,00NMSCUPI-100000000004
:86:UPI/TEST PAYEE FOUR/444444444444/TestApp
:61:2508060806D2000,00NMSCUPI-100000000005
:86:UPI/TEST PAYEE FIVE/555555555555/TestApp
:61:2508030803D1066,00NMSC123456789012
:86:PCD/1234/TEST MERCHANT/01234567890123/12:00
:61:2507310731D305,62NMSCTBMS-123456789
:86:Chrg: Debit Card Annual Fee 1234 for 2025
:61:2507240724C1,00NMSCNEFTINW-1234567890
:86:NEFT TEST123456789 TEST SERVICES
:61:2507170717C100000,00NMSCNOREF
:86:BY CLG INST 123456/01-01-25/TESTBANK/TESTCITY
:62F:C250930INR100000,00
-}"""
transactions = list(mt940.parse(preprocess_mt940_content(mt940_content)))
references = [get_transaction_reference(t.data) for t in transactions]
self.assertEqual(
references,
[
"UPI-100000000001",
"UPI-100000000002",
"UPI-100000000003",
"UPI-100000000004",
"UPI-100000000005",
"123456789012",
"TBMS-123456789",
"NEFTINW-1234567890",
"NOREF",
],
)
# No transaction should carry the statement-level reference from :20:
self.assertNotIn("STMTREF12345", references)
def test_preprocess_mt940_content_whitespace_variants(self):
"""Test handling of whitespace and different line endings"""
# Test with trailing spaces

View File

@@ -54,6 +54,7 @@
"fieldtype": "Currency",
"in_list_view": 1,
"label": "Closing Balance",
"non_negative": 1,
"options": "currency"
},
{
@@ -190,7 +191,7 @@
"grid_page_length": 50,
"index_web_pages_for_search": 1,
"links": [],
"modified": "2026-07-09 17:55:25.615942",
"modified": "2026-05-08 17:55:25.615942",
"modified_by": "Administrator",
"module": "Accounts",
"name": "Bank Statement Import Log",

View File

@@ -375,7 +375,8 @@ class BankStatementImportLog(Document):
table["column_mapping"] = guess_column_mapping_by_content(table["rows"])
final_transactions, table["date_format"], table["amount_format"] = build_table_transactions(table)
table["included"] = should_include_table(table, final_transactions)
# Tables with no detectable transactions (ads, summaries, headers) start excluded.
table["included"] = bool(final_transactions)
self.pdf_tables = json.dumps(tables)
return tables
@@ -541,8 +542,6 @@ class BankStatementImportLog(Document):
"bank-rec-statement-import-progress",
{
"progress": round(progress / total_transactions * 100),
"current": progress,
"total": total_transactions,
},
doctype="Bank Statement Import Log",
docname=self.name,
@@ -552,14 +551,13 @@ class BankStatementImportLog(Document):
"bank-rec-statement-import-progress",
{
"progress": 100,
"current": total_transactions,
"total": total_transactions,
},
doctype="Bank Statement Import Log",
docname=self.name,
)
if self.closing_balance is not None and self.end_date:
if self.closing_balance and self.closing_balance > 0 and self.end_date:
set_closing_balance_as_per_statement(
self.bank_account, frappe.utils.getdate(self.end_date), self.closing_balance
)
@@ -823,15 +821,6 @@ def compute_final_transactions(transaction_rows: list, date_format: str, amount_
"""Pure version of the final-transaction builder (date normalized, amount split)."""
final_transactions = []
# Which marker does this statement actually write? A statement that only ever says "Cr"
# is marking the credits as its exceptions, so an unmarked row is a withdrawal; one that
# only ever says "Dr" means the opposite. With both markers present an unmarked row is
# genuinely undetermined, so it stays a withdrawal.
unmarked_is_deposit = False
if amount_format == 'Amount column has "CR"/"DR" values':
markers = {get_amount_cr_dr_marker(row.get("amount")) for row in transaction_rows}
unmarked_is_deposit = markers - {None} == {"dr"}
def parse_amount(transaction_row: dict):
if amount_format == "Separate columns for withdrawal and deposit":
return get_float_amount(transaction_row.get("withdrawal")), get_float_amount(
@@ -840,43 +829,42 @@ def compute_final_transactions(transaction_rows: list, date_format: str, amount_
if amount_format == 'Amount column has "CR"/"DR" values':
amount = transaction_row.get("amount")
marker = get_amount_cr_dr_marker(amount)
# The marker carries the direction, so the amount's own sign is ignored.
signed_amount = get_float_amount(amount) or 0
float_amount = get_float_amount(amount)
if "cr" in amount.lower():
return 0, float_amount
else:
return float_amount, 0
if marker:
return (0, abs(signed_amount)) if marker == "cr" else (abs(signed_amount), 0)
# An unmarked row takes the opposite direction to the marker this statement
# uses. A negative amount reverses that again (a refund).
is_deposit = unmarked_is_deposit
if signed_amount < 0:
is_deposit = not is_deposit
return (0, abs(signed_amount)) if is_deposit else (abs(signed_amount), 0)
# `or 0` below: get_float_amount returns None for an unparseable cell, and a blank
# transaction-type cell comes through as None. Both used to raise.
if amount_format == "Amount column has positive/negative values":
amount = get_float_amount(transaction_row.get("amount", "0")) or 0
amount = get_float_amount(transaction_row.get("amount", "0"))
if amount > 0:
return 0, abs(amount)
else:
return abs(amount), 0
transaction_type = str(transaction_row.get("debit_credit") or "").strip().lower()
amount = abs(get_float_amount(transaction_row.get("amount", "0")) or 0)
if amount_format == 'Transaction type column has "CR"/"DR" values':
# "credit" contains "cr". "debit" does not contain "dr", so it correctly falls
# through to the withdrawal side.
return (0, amount) if "cr" in transaction_type else (amount, 0)
transaction_type = transaction_row.get("debit_credit")
amount = get_float_amount(transaction_row.get("amount", "0"))
if "cr" in transaction_type.lower():
return 0, abs(amount)
else:
return abs(amount), 0
if amount_format == 'Transaction type column has "C"/"D" values':
return (0, amount) if transaction_type == "c" else (amount, 0)
transaction_type = transaction_row.get("debit_credit")
amount = get_float_amount(transaction_row.get("amount", "0"))
if transaction_type.lower().strip() == "c":
return 0, abs(amount)
else:
return abs(amount), 0
if amount_format == 'Transaction type column has "Deposit"/"Withdrawal" values':
return (0, amount) if "deposit" in transaction_type else (amount, 0)
transaction_type = transaction_row.get("debit_credit")
amount = get_float_amount(transaction_row.get("amount", "0"))
if "deposit" in transaction_type.lower():
return 0, abs(amount)
else:
return abs(amount), 0
return 0, 0
@@ -920,26 +908,6 @@ def build_table_transactions(table: dict):
return final_transactions, date_format, amount_format
def should_include_table(table: dict, final_transactions: list) -> bool:
"""
Whether a freshly extracted PDF table should START as included - only the default state
of the checkbox, which the user can change afterwards.
It must have yielded transactions, and it must have a Description column mapped. A
transaction table always carries a narration; the summary boxes printed around it -
payment due, credit limit, reward points - are dates and figures only. Otherwise the
HDFC credit-card "Payment Due Date / Total Dues / Minimum Amount Due" box parses as one
transaction and imports a phantom row.
A description is NOT needed to import (it is not mandatory on Bank Transaction), so a
bank that omits narration still works - its table just starts unticked.
"""
if not final_transactions:
return False
return any(column.get("maps_to") == "Description" for column in table.get("column_mapping", []))
def _clean_cell(cell) -> str:
"""Normalize a pdfplumber cell: None -> '', collapse wrapped newlines, strip."""
if cell is None:
@@ -964,18 +932,14 @@ def extract_pdf_tables(content: bytes, password: str | None = None) -> list[dict
from pypdf import PdfReader
reader = PdfReader(io.BytesIO(content))
if reader.is_encrypted:
# Try opening the PDF with a password - if no password is provided, try with a blank password
if not password:
password = ""
if not reader.decrypt(password):
frappe.throw(
_(
"This PDF is password protected. Please set the correct statement password on the"
" Bank Account and try again."
),
title=_("Password Required"),
)
if reader.is_encrypted and (not password or not reader.decrypt(password)):
frappe.throw(
_(
"This PDF is password protected. Please set the correct statement password on the"
" Bank Account and try again."
),
title=_("Password Required"),
)
text_settings = {"vertical_strategy": "text", "horizontal_strategy": "text"}
tables = []
@@ -1085,43 +1049,6 @@ def get_float_amount(amount):
return amount
# A "CR"/"DR" marker on the amount itself, at either end: "2,378.00Cr", "Cr 100",
# "INR 50.90 Cr.", "DR 1,234.50".
# `(?![a-zA-Z])` rather than `\b` on the leading form: there is no word boundary between
# the "r" of "Cr100" and the digit, but there IS one inside "CREDIT" and "DRAFT".
AMOUNT_CR_DR_PATTERN = re.compile(r"^\s*(cr|dr)(?![a-zA-Z])\.?|(?:^|[\s\d.)])(cr|dr)\b\.?\s*$", re.IGNORECASE)
def get_amount_cr_dr_marker(amount) -> str | None:
"""
Return "cr" or "dr" if the amount cell carries a direction marker of its own, else None.
What is left after removing the marker has to look like an amount - it must hold a digit
and at most a short currency token - so that text which merely starts or ends with the
letters is not read as a marker. That guard is what separates "Cr 100" from a
description that bled into the amount column, like "Dr Smith Clinic 500".
"""
if not isinstance(amount, str):
return None
match = AMOUNT_CR_DR_PATTERN.search(amount)
if not match:
return None
# Only the marker itself is removed - the surrounding character the pattern needed to
# anchor on (a digit, say) stays part of the remainder.
group = 1 if match.group(1) else 2
start, end = match.span(group)
remainder = amount[:start] + amount[end:]
if not any(char.isdigit() for char in remainder):
return None
if sum(char.isalpha() for char in remainder) > 3:
return None
return match.group(group).lower()
def get_file_properties(transactions: list):
"""
From the transaction rows, try to figure out the following:
@@ -1142,8 +1069,6 @@ def get_file_properties(transactions: list):
'Transaction type column has "C"/"D" values': 0,
}
amount_column_has_cr_dr = False
for transaction in transactions:
date_format = transaction.get("date_format")
@@ -1161,40 +1086,33 @@ def get_file_properties(transactions: list):
if not amount:
continue
debit_credit = str(transaction.get("debit_credit") or "").strip().lower()
# One vote per row, most specific signal first. Order matters: "withdrawal" contains
# "dr", so it must be matched before the loose cr/dr check or a Deposit/Withdrawal
# column reads as CR/DR. "debit" needs listing because, unlike "credit", it does not
# contain "dr". The final else means every row votes, even an unrecognised type.
if get_amount_cr_dr_marker(amount):
amount_column_has_cr_dr = True
if isinstance(amount, str) and ("cr" in amount.lower() or "dr" in amount.lower()):
amount_format_frequency['Amount column has "CR"/"DR" values'] += 1
elif "deposit" in debit_credit or "withdrawal" in debit_credit:
amount_format_frequency['Transaction type column has "Deposit"/"Withdrawal" values'] += 1
elif debit_credit in ("c", "d"):
amount_format_frequency['Transaction type column has "C"/"D" values'] += 1
elif any(token in debit_credit for token in ("cr", "dr", "debit")):
amount_format_frequency['Transaction type column has "CR"/"DR" values'] += 1
# Check if there's a debit_credit column containing "cr"/"dr"
if transaction.get("debit_credit", None):
if (
"cr" in transaction.get("debit_credit", "").lower()
or "dr" in transaction.get("debit_credit", "").lower()
):
amount_format_frequency['Transaction type column has "CR"/"DR" values'] += 1
elif (
"deposit" in transaction.get("debit_credit", "").lower()
or "withdrawal" in transaction.get("debit_credit", "").lower()
):
amount_format_frequency['Transaction type column has "Deposit"/"Withdrawal" values'] += 1
elif (transaction.get("debit_credit", "").lower().strip() == "c") or (
transaction.get("debit_credit", "").lower().strip() == "d"
):
amount_format_frequency['Transaction type column has "C"/"D" values'] += 1
# Else assume that the amount is expressed as positive/negative value
else:
# Nothing said which direction this is, so assume the amount carries the sign.
amount_format_frequency["Amount column has positive/negative values"] += 1
most_common_date_format = max(date_format_frequency, key=date_format_frequency.get)
most_common_amount_format = max(amount_format_frequency, key=amount_format_frequency.get)
# With no votes at all (no rows, or every amount blank) max() would return whichever key
# happens to be first in the dict. Say what we mean instead.
if not amount_format_frequency[most_common_amount_format]:
most_common_amount_format = "Amount column has positive/negative values"
# A CR/DR amount column is proved by a single marker, not by a majority: both formats
# describe the same column, and an unmarked row is only the default direction, not
# evidence against the notation. Statements mark just the exceptions - one HDFC
# credit-card page has 18 rows and a single "50.90Cr".
if amount_column_has_cr_dr and most_common_amount_format == "Amount column has positive/negative values":
most_common_amount_format = 'Amount column has "CR"/"DR" values'
return most_common_date_format, most_common_amount_format
@@ -1265,7 +1183,8 @@ def update_pdf_tables(statement_import_id: str, tables: list | str):
if doc.status == "Completed":
frappe.throw(_("This statement has already been imported."), title=_("Already Imported"))
tables = frappe.parse_json(tables)
if isinstance(tables, str):
tables = json.loads(tables)
doc.apply_pdf_tables(tables)
@@ -1285,7 +1204,8 @@ def reextract_pdf_table(statement_import_id: str, page: int, table_index: int, b
if doc.status == "Completed":
frappe.throw(_("This statement has already been imported."), title=_("Already Imported"))
bbox = frappe.parse_json(bbox)
if isinstance(bbox, str):
bbox = json.loads(bbox)
page = int(page)
table_index = int(table_index)
@@ -1370,7 +1290,8 @@ def update_column_mapping(statement_import_id: str, column_mapping: list | str):
if doc.status == "Completed":
frappe.throw(_("This statement has already been imported."), title=_("Already Imported"))
column_mapping = frappe.parse_json(column_mapping)
if isinstance(column_mapping, str):
column_mapping = json.loads(column_mapping)
doc.apply_column_mapping(column_mapping)
doc.save()

View File

@@ -11,14 +11,12 @@ from erpnext.accounts.doctype.bank_statement_import_log.bank_statement_import_lo
detect_column_mapping,
detect_header_row,
extract_pdf_tables,
get_amount_cr_dr_marker,
get_float_amount,
get_statement_details,
guess_column_mapping_by_content,
reextract_pdf_table,
set_header_index,
set_pdf_table_header,
should_include_table,
update_column_mapping,
update_pdf_tables,
)
@@ -28,9 +26,9 @@ from erpnext.tests.utils import ERPNextTestSuite
class TestBankStatementImportLog(ERPNextTestSuite, AccountsTestMixin):
def setUp(self):
self.company = "_Test Company"
self.customer = "_Test Customer"
self.bank = "HDFC - _TC"
self.create_company()
self.create_customer()
self.clear_old_entries()
bank_dt = qb.DocType("Bank")
qb.from_(bank_dt).delete().where(bank_dt.name == "HDFC").run()
self.create_bank_account()
@@ -126,184 +124,6 @@ class TestBankStatementImportLog(ERPNextTestSuite, AccountsTestMixin):
self.assertIsNone(get_float_amount("ABCD"))
self.assertIsNone(get_float_amount("****"))
# ------------------------------------------------------------------ #
# Amount format detection
# ------------------------------------------------------------------ #
def test_amount_cr_dr_marker(self):
"""The marker is read at either end of the cell, but only next to the amount."""
for amount in ("2,378.00Cr", "50.90 CR", "INR 50.90 Cr.", "1000cr", "5cr", "(100) Cr"):
self.assertEqual(get_amount_cr_dr_marker(amount), "cr", amount)
for amount in ("2,378.00Dr", "50.90 DR", "1000dr", "-100 Dr"):
self.assertEqual(get_amount_cr_dr_marker(amount), "dr", amount)
# Some banks put the marker in front of the digits instead.
for amount in ("Cr 100", "Cr100", "CR INR 100", "cr 0.00"):
self.assertEqual(get_amount_cr_dr_marker(amount), "cr", amount)
for amount in ("Dr 100", "Dr100", "Dr. 1,234.50"):
self.assertEqual(get_amount_cr_dr_marker(amount), "dr", amount)
for amount in ("100.00", "-2,000.00", "INR 25,236.00", "", None, 100.0):
self.assertIsNone(get_amount_cr_dr_marker(amount), amount)
# Text that merely starts or ends with the letters must not be read as a marker, or
# a description that bled into the amount column would reclassify the statement.
for amount in (
"CREDIT CARD PAYMENT 500",
"DRAFT 100",
"Dr Smith Clinic 500",
"DR AMBEDKAR ROAD BRANCH 500",
"500 CRC",
"Cheque Dr",
"Cr",
):
self.assertIsNone(get_amount_cr_dr_marker(amount), amount)
def test_sparsely_marked_cr_dr_amount_column(self):
"""One marker is enough to prove a CR/DR amount column - it is not a majority vote.
A real HDFC credit-card page carries 18 rows and a single "50.90Cr": the unmarked
rows are ordinary purchases, and only the exceptions are marked. A frequency vote
therefore picked "positive/negative" 17-1 and imported that lone credit as a debit.
"""
doc = self._create_bank_statement_import_log(
[
["Date", "Transaction Description", "Amount (in Rs.)"],
["21/07/2026", "ITC MAURYA NEW DELHI", "2,495.00"],
["22/07/2026", "ZOMATO LIMITED Gurugram", "1,288.68"],
["23/07/2026", "SWIGGY Bangalore", "532.00"],
["26/07/2026", "SWIGGY Bangalore", "1,043.00"],
["27/07/2026", "PETRO SURCHARGE WAIVER", "50.90Cr"],
]
)
self.assertEqual(doc.detected_amount_format, 'Amount column has "CR"/"DR" values')
# Only "Cr" appears, so it is the marked exception and unmarked rows are debits.
self.assertEqual(doc.total_credits, 50.90)
self.assertEqual(doc.total_credit_transactions, 1)
self.assertEqual(doc.total_debits, 5358.68)
self.assertEqual(doc.total_debit_transactions, 4)
def test_dr_only_statement_treats_unmarked_rows_as_deposits(self):
"""The mirror image of a Cr-only statement: only withdrawals are marked.
The unmarked default cannot be hardcoded to the debit, because which side gets
marked varies by bank. It is derived from the markers the statement actually uses -
here only "Dr" appears, so "Dr" is the exception and everything unmarked is a
deposit.
"""
doc = self._create_bank_statement_import_log(
[
["Date", "Narration", "Amount"],
["01/04/2026", "ATM WITHDRAWAL", "2,000.00Dr"],
["03/04/2026", "SALARY", "20,000.00"],
["05/04/2026", "INTEREST", "150.00"],
]
)
self.assertEqual(doc.detected_amount_format, 'Amount column has "CR"/"DR" values')
self.assertEqual(doc.total_debits, 2000.0)
self.assertEqual(doc.total_debit_transactions, 1)
self.assertEqual(doc.total_credits, 20150.0)
self.assertEqual(doc.total_credit_transactions, 2)
def test_leading_cr_dr_markers(self):
"""Some banks print the marker in front of the amount."""
doc = self._create_bank_statement_import_log(
[
["Date", "Narration", "Amount"],
["01/04/2026", "ATM WITHDRAWAL", "Dr 2,000.00"],
["03/04/2026", "SALARY", "Cr 20,000.00"],
]
)
self.assertEqual(doc.detected_amount_format, 'Amount column has "CR"/"DR" values')
self.assertEqual(doc.total_debits, 2000.0)
self.assertEqual(doc.total_credits, 20000.0)
def test_partially_marked_cr_dr_amount_column(self):
"""A CR/DR amount column stays CR/DR even when some rows carry no marker.
Every unmarked row used to also vote for "positive/negative", so an ordinary
statement with a few unmarked rows was detected as positive/negative and a
"2000.00Dr" was then imported as a deposit.
"""
doc = self._create_bank_statement_import_log(
[
["Date", "Narration", "Amount", "Balance"],
["01/04/2026", "OPENING FEE", "100.00", "9,900.00"],
["03/04/2026", "SALARY", "20000.00Cr", "29,900.00"],
["05/04/2026", "ATM WDL", "2000.00Dr", "27,900.00"],
]
)
self.assertEqual(doc.detected_amount_format, 'Amount column has "CR"/"DR" values')
# Both markers appear, so an unmarked row is undetermined and stays a debit.
self.assertEqual(doc.total_debits, 2100.0)
self.assertEqual(doc.total_debit_transactions, 2)
self.assertEqual(doc.total_credits, 20000.0)
self.assertEqual(doc.total_credit_transactions, 1)
def test_deposit_withdrawal_type_column(self):
"""The word Withdrawal contains "dr", so a loose CR/DR check claims this column first.
It then reads "Deposit" (which has no "cr" in it) as a withdrawal, flipping the
direction of every credit in the statement.
"""
doc = self._create_bank_statement_import_log(
[
["Date", "Narration", "Transaction Type", "Amount"],
["01/04/2026", "ATM WDL", "Withdrawal", "2,000.00"],
["03/04/2026", "SALARY", "Deposit", "20,000.00"],
["05/04/2026", "ATM WDL", "Withdrawal", "500.00"],
]
)
self.assertEqual(
doc.detected_amount_format, 'Transaction type column has "Deposit"/"Withdrawal" values'
)
self.assertEqual(doc.total_debits, 2500.0)
self.assertEqual(doc.total_debit_transactions, 2)
self.assertEqual(doc.total_credits, 20000.0)
self.assertEqual(doc.total_credit_transactions, 1)
def test_unrecognised_type_column_falls_back_to_signed_amount(self):
"""An unrecognised transaction type must not stop the amount being read.
No tally was incremented for these rows, so max() returned the first key -
"Separate columns for withdrawal and deposit" - and, with no such columns in the
file, every amount came through as None.
"""
doc = self._create_bank_statement_import_log(
[
["Date", "Narration", "Transaction Type", "Amount"],
["01/04/2026", "ATM WDL", "NEFT", "-2,000.00"],
["03/04/2026", "SALARY", "IMPS", "20,000.00"],
]
)
self.assertEqual(doc.detected_amount_format, "Amount column has positive/negative values")
self.assertEqual(doc.total_debits, 2000.0)
self.assertEqual(doc.total_credits, 20000.0)
def test_blank_transaction_type_cell(self):
"""A blank type cell used to raise - `None.lower()` - instead of parsing the row."""
doc = self._create_bank_statement_import_log(
[
["Date", "Narration", "Transaction Type", "Amount"],
["01/04/2026", "ATM WDL", "Dr", "2,000.00"],
["03/04/2026", "SALARY", "Cr", "20,000.00"],
["05/04/2026", "UNKNOWN", None, "500.00"],
]
)
self.assertEqual(doc.detected_amount_format, 'Transaction type column has "CR"/"DR" values')
# The unmarked row has no direction of its own, so it counts as a withdrawal.
self.assertEqual(doc.total_debits, 2500.0)
self.assertEqual(doc.total_credits, 20000.0)
# ------------------------------------------------------------------ #
# PDF statement import
# ------------------------------------------------------------------ #
@@ -339,8 +159,7 @@ class TestBankStatementImportLog(ERPNextTestSuite, AccountsTestMixin):
else:
table["header_index"] = None
table["column_mapping"] = guess_column_mapping_by_content(table["rows"])
final_transactions, _df, _af = build_table_transactions(table)
table["included"] = should_include_table(table, final_transactions)
table["included"] = True
return table
def test_pdf_multi_page_kept_separate_and_unioned(self):
@@ -378,74 +197,6 @@ class TestBankStatementImportLog(ERPNextTestSuite, AccountsTestMixin):
final, _df, _af = build_table_transactions(ad_table)
self.assertEqual(final, [])
def test_pdf_summary_box_not_auto_included(self):
"""A summary box that happens to parse as one transaction must not start included.
The "Payment Due Date / Total Dues / Minimum Amount Due" block on an HDFC
credit-card statement has a date column and a figures column, so it yields a single
transaction - the due date and the minimum amount - and used to import as a phantom
row. What it does not have, and a real transaction table always does, is a narration.
"""
summary_box = {
"header_index": 1,
"rows": [
["Statement Date:17/08/2025", "Card No: 4341 55XX XXXX 2754", ""],
["Payment Due Date", "Total Dues", "Minimum Amount Due"],
["06/09/2025", "73,200.00", "3,660.00"],
["Credit Limit", "Available Credit Limit", "Available Cash Limit"],
["", "32,800", ""],
],
"column_mapping": [
{"index": 0, "header_text": "Payment Due Date", "variable": "a", "maps_to": "Date"},
{"index": 1, "header_text": "Total Dues", "variable": "b", "maps_to": "Do not import"},
{"index": 2, "header_text": "Minimum Amount Due", "variable": "c", "maps_to": "Amount"},
],
}
final, _df, _af = build_table_transactions(summary_box)
# It really does parse as a transaction - that is why the previous check missed it.
self.assertEqual(len(final), 1)
self.assertFalse(should_include_table(summary_box, final))
# The transaction table beside it, which does carry a narration, still starts included.
transactions = self._auto_map(
{
"rows": [
["Date", "Transaction Description", "Amount (in Rs.)"],
["21/07/2025", "ITC MAURYA NEW DELHI", "2,495.00"],
["27/07/2025", "PETRO SURCHARGE WAIVER", "50.90Cr"],
]
}
)
self.assertTrue(transactions["included"])
def test_pdf_table_without_description_still_importable(self):
"""No narration column means "starts unticked", NOT "cannot be imported".
`description` is not mandatory on Bank Transaction, so a bank that omits narration
must still import once the user ticks the table.
"""
table = {
"header_index": 0,
"rows": [
["Date", "Amount", "Balance"],
["01/04/2025", "500.00", "9,500.00"],
["03/04/2025", "20000.00", "29,500.00"],
],
"column_mapping": [
{"index": 0, "header_text": "Date", "variable": "a", "maps_to": "Date"},
{"index": 1, "header_text": "Amount", "variable": "b", "maps_to": "Amount"},
{"index": 2, "header_text": "Balance", "variable": "c", "maps_to": "Balance"},
],
}
final, _df, _af = build_table_transactions(table)
self.assertFalse(should_include_table(table, final))
# The transactions themselves are intact and importable.
self.assertEqual(len(final), 2)
self.assertEqual([t["date"] for t in final], ["2025-04-01", "2025-04-03"])
def test_headerless_content_mapping(self):
"""Without a header row, columns are guessed from their contents."""
rows = [

Some files were not shown because too many files have changed in this diff Show More