mirror of
https://github.com/frappe/erpnext.git
synced 2026-09-09 07:19:31 +00:00
Compare commits
1 Commits
codex/seri
...
auth_did
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
265bc4eb6f |
259
.github/POSTGRES_COMPATIBILITY.md
vendored
259
.github/POSTGRES_COMPATIBILITY.md
vendored
@@ -1,259 +0,0 @@
|
||||
# PostgreSQL compatibility — review guide
|
||||
|
||||
ERPNext targets **both MariaDB and PostgreSQL from a single codebase**. The full server
|
||||
test suite passes on both, but the PostgreSQL CI job is **label-gated** (it does not run on
|
||||
every PR), so until it is required this guide is the always-on guard. Greptile loads it as
|
||||
review context (`.greptile/config.json`).
|
||||
|
||||
When reviewing a PR, flag any **new or changed query** (raw `frappe.db.sql`, `frappe.qb`,
|
||||
`frappe.get_all/get_list/get_value`, report SQL) that would **error on PostgreSQL** or
|
||||
**return different results on the two engines**.
|
||||
|
||||
## The one rule that governs everything
|
||||
|
||||
**MariaDB behaviour must not change; PostgreSQL is brought into line with MariaDB — never the
|
||||
reverse.** A "fix" that changes the value, row count, or ordering MariaDB produced is a
|
||||
regression, even if the new behaviour looks more correct. The only accepted MariaDB-output
|
||||
change is replacing a genuinely *undefined/arbitrary* result with a deterministic one (row
|
||||
count preserved) — and that should be called out explicitly.
|
||||
|
||||
There are two failure modes to watch for:
|
||||
1. **Hard breaks** — PostgreSQL raises an exception; MariaDB is green. Easy to catch in CI,
|
||||
but the gated job may not run.
|
||||
2. **Silent divergences** — both engines succeed but return *different* results. CI on one
|
||||
engine stays green; the bug only shows on a PostgreSQL site. These are the dangerous ones.
|
||||
|
||||
---
|
||||
|
||||
## 1. Hard breaks — would error on PostgreSQL
|
||||
|
||||
Flag a changed query that uses any of these:
|
||||
|
||||
- **Loose `GROUP BY`** — selecting/ordering a column that is neither in `GROUP BY` nor wrapped
|
||||
in an aggregate. MariaDB tolerates it; PostgreSQL errors (`must appear in the GROUP BY
|
||||
clause or be used in an aggregate function`). This **also covers an aggregate (`Sum`/`Count`/…)
|
||||
selected alongside bare columns with NO `.groupby()` at all** — MariaDB silently collapses
|
||||
every row into one arbitrary-valued row (often a *wrong-output* bug there too), PostgreSQL
|
||||
errors. Fix: add the bare column to `GROUP BY` **if it is functionally dependent on the group
|
||||
key**, otherwise wrap it in `Max()`/`Min()`. **See §3 — the row-count trap — before suggesting
|
||||
"add it to GROUP BY".**
|
||||
- **MySQL-only functions** — `TIMESTAMP(date,time)`, `TIMEDIFF`, `STR_TO_DATE`, `DATE_FORMAT`,
|
||||
`DATE_ADD/SUB`, `GROUP_CONCAT`, `PERIOD_DIFF`, SQL `IF(cond,a,b)`. Use the portable
|
||||
`frappe.query_builder.functions` equivalents (`CombineDatetime`, `DateDiff`, `Case`,
|
||||
`GroupConcat`, …) or a precomputed column (e.g. `posting_datetime`).
|
||||
- **`UPDATE … JOIN`** — not valid on PostgreSQL. Rewrite as `UPDATE … WHERE name IN (subquery)`.
|
||||
- **`HAVING` referencing a `SELECT` alias** — PostgreSQL rejects output-column aliases in
|
||||
`HAVING` (regardless of whether the query has a `GROUP BY`; MariaDB allows them). Repeat the
|
||||
underlying expression in `HAVING`, or move a non-aggregate predicate into `WHERE`.
|
||||
- **`SELECT DISTINCT … ORDER BY <expr not in the select list>`** — add the expr to the select
|
||||
**only if it is single-valued per distinct row**; otherwise it grows the `DISTINCT` key and the
|
||||
MariaDB row count (see §3) — drop the SQL `ORDER BY` and sort in Python instead.
|
||||
- **Single-quoted column alias** `AS 'x'` — PostgreSQL reads `'x'` as a string literal. Use an
|
||||
unquoted (or double-quoted) alias.
|
||||
- **`varchar | varchar`** (bitwise OR misused as a coalesce) — errors on PostgreSQL. Use
|
||||
`Coalesce(...)`.
|
||||
- **Capital-cased identifiers** used as column/field names in `get_value(dt, dn, "Status")`,
|
||||
`get_all(dt, fields=["Account"])`, and similar — PostgreSQL quotes the identifier and matches
|
||||
it case-sensitively; a stored column named `status`/`account` won't match `"Status"`/`"Account"`
|
||||
(`column "Account" does not exist`). Use the exact stored (lower-case) fieldname.
|
||||
- **Boolean passed where an integer column is expected** — `frappe.db.set_value(dt, dn,
|
||||
check_field, True)`, `doc.db_set(field, False)`, or `frappe.qb.update(dt).set(check_field, True)`
|
||||
emit `SET col = true`, which PostgreSQL rejects on a `smallint`/`Check` column
|
||||
(`column is of type smallint but expression is of type boolean`). Pass `1`/`0`.
|
||||
- **A direct `.like()`/`.ilike()` on a pypika field (or raw `LIKE`) on a NON-text column** — `idx`,
|
||||
`docstatus`, a date, etc. frappe maps `.like()` → `ILIKE`, and PostgreSQL has no `bigint ILIKE text`
|
||||
operator (`operator does not exist: bigint ~~* unknown`). Cast the column to text first —
|
||||
**`Cast_(col, "varchar")`**, not `Cast(col, "char")` (see below). MariaDB coerces the int
|
||||
implicitly, so the cast is a no-op there. A `["like", …]` filter passed to `get_all`/`get_list`/
|
||||
`qb.get_query`/`reportview` needs no cast: the framework casts non-text fields itself
|
||||
(frappe/frappe#42449).
|
||||
- **`CAST(… AS CHAR)` / `Cast(x, "char")`** — on PostgreSQL bare `CHAR` is `character(1)`, so
|
||||
`CAST(12 AS CHAR)` → `'1'` (silently truncates multi-digit values); MariaDB gives the full string.
|
||||
Use `VARCHAR` / `Cast_(x, "varchar")`.
|
||||
- **`.rlike()` / raw `RLIKE`** — frappe rewrites `REGEXP` → `~*` on PostgreSQL but does **not**
|
||||
translate `RLIKE` (no such PostgreSQL operator). Use `.regexp()` (or `.like()` for a simple prefix).
|
||||
- **`IfNull`/`Coalesce` of a typed column with a different-typed literal** — `IfNull(asset.disposal_date, 0)`
|
||||
renders `COALESCE("disposal_date", 0)`, coalescing a **DATE** with an **integer**. PostgreSQL requires
|
||||
`COALESCE` args to share a type (`DatatypeMismatch: COALESCE types date and integer cannot be matched`);
|
||||
MariaDB's `IFNULL` is permissive. The common shape is `IfNull(date_col, 0) != 0 / == 0` as a presence test —
|
||||
replace with `date_col.isnotnull()` / `date_col.isnull()` (identical, and valid on both). Otherwise coalesce
|
||||
to a **same-type** default (`Coalesce(date_col, '1900-01-01')`, `Coalesce(text_col, '')`).
|
||||
- **Division by a possibly-zero divisor** — `Sum(a) / Sum(b)`, `x / col`, etc. where the
|
||||
divisor can be `0`/empty. MariaDB returns `NULL` for division by zero; PostgreSQL raises
|
||||
`division by zero` and aborts the query. Wrap the divisor in `NullIf(divisor, 0)` — that
|
||||
yields `NULL` on both engines, matching MariaDB's value. (Only the *literal* `/ 0` is a parse
|
||||
constant; the trap is a divisor that is an aggregate or column the data can drive to zero.)
|
||||
|
||||
---
|
||||
|
||||
## 2. Silent divergences — succeeds on both, returns different results
|
||||
|
||||
These don't error, so a one-engine CI stays green. Flag them:
|
||||
|
||||
- **Case sensitivity on text equality** — `==`, `.isin()`, `Strpos`/`Locate` on free-text
|
||||
columns are case-**sensitive** on PostgreSQL but case-**insensitive** under MariaDB's default
|
||||
collation. `Lower()` both sides. *(Not `.like()`/`["like", …]` — those already render as
|
||||
`ILIKE` on PostgreSQL; see §4.)*
|
||||
- **Case sensitivity in a doc-`name` lookup** — lower-casing a value then using it as a
|
||||
document name in `get_value`/`get_doc`/`exists` misses on PostgreSQL (names are
|
||||
case-sensitive). Keep original case for the identifier; lower-case only comparison operands.
|
||||
- **Empty string vs NULL** — PostgreSQL stores a blank link/data field as `NULL` on some paths
|
||||
while MariaDB keeps `''`; `Concat`/`Concat_ws` then diverge. Prefer the stored full value, or
|
||||
`Coalesce(col, '')` per argument.
|
||||
- **NULL ordering** — MariaDB sorts `NULL` first, PostgreSQL sorts it last. For
|
||||
`ORDER BY … LIMIT 1`/`[0]` on a nullable column, guard with `Coalesce`/`isnotnull()`.
|
||||
- **`ORDER BY … LIMIT 1` with no unique tiebreaker** — when rows tie on the ordered column the
|
||||
two engines may pick different rows. Add a `creation`/`name` tiebreaker **only if it does not
|
||||
change MariaDB's current pick** (see §4).
|
||||
- **Integer division** — `int / int` truncates on PostgreSQL but is decimal on MariaDB, e.g.
|
||||
`COUNT(...) / COUNT(...) * 100` → `0`, or `manufacturing_time_in_mins / 1440` flooring a
|
||||
lead-time to whole days. Force float: multiply by `100.0`, or make a literal a float
|
||||
(`/ 1440` → `/ 1440.0`), or cast an operand. (Only SQL-level `/` on integer **columns/literals**
|
||||
— Python `/` is already float.)
|
||||
- **`DISTINCT` list ordering** — `frappe.get_all(distinct=True, order_by=…)` /
|
||||
`SELECT DISTINCT … ORDER BY`: frappe's `db_query` **silently drops `ORDER BY` for distinct
|
||||
queries on PostgreSQL**, so the result is unordered there. Sort in Python instead — and use
|
||||
`key=str.casefold`, because bare `sorted()` is case-sensitive (ASCII) while MariaDB's
|
||||
collation is case-insensitive, so a plain sort reorders MariaDB's output.
|
||||
- **Engine-specific function rewrites** — e.g. a PostgreSQL `regexp_replace` branch
|
||||
reimplementing MariaDB's `CAST(SUBSTRING_INDEX(name,' ',-1) AS UNSIGNED)` (leading digits of
|
||||
the last whitespace token). Verify the rewrite matches MariaDB on edge cases (`"X - 3a"→3`,
|
||||
`"X - 1.5"→1`) by diffing both engines on literal rows.
|
||||
- **`UnixTimestamp(date)` / date→epoch** is timezone-dependent (midnight in the DB session TZ),
|
||||
so a strict `epoch <= now` bound is flaky on PostgreSQL.
|
||||
|
||||
---
|
||||
|
||||
## 3. The row-count trap — `GROUP BY` **and** `DISTINCT` (the single most important rule)
|
||||
|
||||
When making a loose `GROUP BY` PostgreSQL-valid, **do not add a non-functionally-dependent
|
||||
column to the `GROUP BY` just to satisfy PostgreSQL** — that turns one group row into N and
|
||||
**changes the MariaDB row count** (a regression). The classic traps are adding the **child/row
|
||||
primary key** or an **editable per-row field**. Instead **`Max()`/`Min()`-wrap** the offending
|
||||
column: the row count is preserved and the value goes from arbitrary (MariaDB's old loose pick)
|
||||
to deterministic.
|
||||
|
||||
**Judge functional dependence by the source table, not the column name:**
|
||||
- A column from a **master joined on the group key** (`t3.x` where `t1.key = t3.name`) is FD →
|
||||
safe to keep in `GROUP BY`.
|
||||
- A descriptive field on the **transaction** table (`t1.supplier_name`, `t1.territory`,
|
||||
`t1.item_name` — fetched/editable, can differ across historical rows for the same key) is
|
||||
**not** FD even though it looks master-derived → `Max()`-wrap it.
|
||||
|
||||
Conversely, do **not** suggest changing a `Max()`/`Min()`-wrapped column to `Sum()` (or vice
|
||||
versa) to make a number "more correct" — that changes the MariaDB value. The wrap reproduces
|
||||
MariaDB's prior one-value-per-group output; a different aggregate is a product change, out of
|
||||
scope for a portability fix.
|
||||
|
||||
**The same trap applies to `SELECT DISTINCT`.** To satisfy PostgreSQL's "an `ORDER BY` expr must
|
||||
appear in the select list under `DISTINCT`" rule, **do not blindly add the ordered column to the
|
||||
select** — if it is not single-valued per existing distinct row, the `DISTINCT` key grows and
|
||||
MariaDB returns **more rows** (a regression), exactly as adding a non-FD column to `GROUP BY` does.
|
||||
Add it only when it is functionally dependent on the existing select columns; otherwise drop the
|
||||
SQL `ORDER BY` and **sort in Python** (`key=str.casefold`, per §2) so the distinct row set is
|
||||
unchanged.
|
||||
|
||||
### 3.1 Second-order traps — when the `Max()`/`Min()` wrap itself is the bug
|
||||
|
||||
The wrap is only a no-op when the column is provably single-valued per group (**"`Max()` means
|
||||
provably constant"**). When the column can genuinely vary, the wrap is a decision, and a full
|
||||
audit of these fixes found four recurring mistakes:
|
||||
|
||||
- **Incoherent pair** — two semantically-coupled columns (a flag + a link:
|
||||
`is_phantom_item` + `bom_no`; a discriminator + its value) aggregated with *independent*
|
||||
`Max()`/`Min()` can pair values from **different rows** — a chimera row that never existed.
|
||||
MariaDB's loose pick was at least row-coherent. Fix: group by the pair (when consumers
|
||||
tolerate the extra rows), or select one **representative row** (`Min(child.name)` subquery +
|
||||
join-back) so every column comes from the same line.
|
||||
- **NULL-skipping** — `MAX`/`MIN` ignore NULLs, so `Max()` over a mostly-NULL discriminator
|
||||
(an `original_item`-style column) *deterministically* returns the non-NULL value where
|
||||
MariaDB could return NULL — deterministically wrong where the old behavior was only
|
||||
intermittently wrong. Flag it wherever "no value" is a meaningful state (fallback gates,
|
||||
dict keys).
|
||||
- **Fabricated arithmetic** — `Sum(x) * Max(y)` where `y` varies within the group invents a
|
||||
number no row ever had (and `Max` biases it upward) — poisonous when it feeds validation,
|
||||
budgets, valuation, or GL/stock values. Fix per-row: `Sum(x * y)`.
|
||||
- **Collation-dependent pick (text columns)** — `Max()`/`Min()` over text is a *sort*, and the two
|
||||
engines sort text differently: MariaDB's `utf8mb4` collations fold case, PostgreSQL (as CI runs
|
||||
it) orders by byte value. `MAX('abc', 'ABD')` is `ABD` on MariaDB and `abc` on PostgreSQL. So a
|
||||
`Max()` over a text column that varies **in case** within its group is a live P2 divergence, not
|
||||
the arbitrary-pick preservation the wrap is usually justified as. Confirmed on CI; see #56241.
|
||||
Note a local macOS PostgreSQL gives a **false all-clear** — its collation happens to agree with
|
||||
MariaDB on case. Fix: take a representative row rather than sorting text.
|
||||
- **Wrong bound** — where the value has a semantic, pick the bound deliberately:
|
||||
`Min(schedule_date)` for a "required by", `Min(idx)` for first-line ordering, a qty-weighted
|
||||
average for a rate. A blind `Max` can understate urgency or overstate a figure.
|
||||
|
||||
Review heuristic: **if choosing between `Max` and `Min` would change the answer, the column is
|
||||
not functionally dependent** — wrapping either is the wrong fix. Group by it, restructure, or
|
||||
pick a bound for a stated reason, and cover the varying-group case with a test.
|
||||
|
||||
---
|
||||
|
||||
## 4. False positives — do NOT flag these
|
||||
|
||||
These are auto-handled by the framework and are **not** breaks:
|
||||
|
||||
- **`.like()` / `["like", …]`** already renders as `ILIKE` on PostgreSQL — not a
|
||||
case-sensitivity bug. A `["like", …]` filter on a **non-text** field is also cast to text by
|
||||
the framework. *(Exception: a direct `.like()` on a **non-text** pypika field — `idx`,
|
||||
`docstatus` — is a hard break, `bigint ILIKE`; see §1.)*
|
||||
- **Raw `ifnull(...)`** inside `frappe.db.sql()` is rewritten to `coalesce(...)` on all engines.
|
||||
- **Backticks**, **`LOCATE`**, **`REGEXP`** / **`.regexp()`** in raw SQL are auto-translated on
|
||||
PostgreSQL (`REGEXP` → `~*`). **But `RLIKE` / `.rlike()` is NOT translated** — that one is a
|
||||
hard break (see §1).
|
||||
- **An `ORDER BY … LIMIT 1` tie where the two engines already agree**, or where adding a
|
||||
tiebreaker would *change* MariaDB's current pick — leave it; "fixing" it would either change
|
||||
MariaDB or has no observable effect.
|
||||
|
||||
---
|
||||
|
||||
## 5. Transaction / runtime (not query-shape, still PostgreSQL-only)
|
||||
|
||||
- **Catch-and-continue inserts** — on PostgreSQL a failed `insert()` aborts the **whole
|
||||
transaction**, so code that swallows a duplicate and keeps going dies on the next statement
|
||||
with `InFailedSqlTransaction` (frappe dropped its blanket per-statement savepoint in
|
||||
frappe#40075). Such a handler must wrap the fallible insert in `frappe.db.savepoint(name)` +
|
||||
`rollback(save_point=name)` — unless it re-`throw`s with no DB call before the throw, or the
|
||||
insert uses `ignore_if_duplicate=True` / `autoname="hash"` (→ `ON CONFLICT DO NOTHING`).
|
||||
- **Recover the txn with a *scoped* savepoint, not a full `frappe.db.rollback()`, if any prior work
|
||||
must survive.** A full rollback un-poisons the txn but also discards every row the handler committed
|
||||
*before* the failure — which MariaDB kept (it has no statement-abort), so it's a **silent MariaDB
|
||||
regression**. **"The background job / whitelist entrypoint owns the txn" does NOT make a full rollback
|
||||
safe** if it did multiple inserts in a loop first — it drops the partial results MariaDB retained. A
|
||||
full rollback is safe only when it (a) immediately re-`throw`s/`raise`s (MariaDB rolls back anyway),
|
||||
(b) has nothing successful before it (a single op), or (c) the batch is genuinely meant to be
|
||||
**atomic** (a partial result is an invalid state → rollback + mark *Failed* is correct). Otherwise use
|
||||
a **per-iteration / per-record savepoint** — and keep the function's success/`None` return contract:
|
||||
do **not** return the doc when the savepoint was rolled back.
|
||||
|
||||
---
|
||||
|
||||
## 6. Refactors and raw-SQL→ORM conversions are not automatically 1:1
|
||||
|
||||
A commit labeled a **refactor** or a **raw-`frappe.db.sql` → `frappe.qb`/ORM conversion** is meant
|
||||
to preserve behaviour — but it easily doesn't, and the change passes the static checker and a
|
||||
one-engine green run. **Diff the `WHERE`/predicate, the `JOIN`/`ON` conditions, and the resulting
|
||||
row set — not just the `SELECT` shape.** A conversion that silently widens or narrows the filter
|
||||
changes the rows touched on **both** engines and is a regression hiding under a "refactor" label.
|
||||
|
||||
Real example: an `UPDATE` whose bound was `posting_datetime > X` gained an
|
||||
`OR (posting_datetime == X AND creation > args.creation)` branch during a "`sql` → `qb` refactor",
|
||||
widening the rows updated on both engines. Even when such a change is a deliberate bug-fix it must
|
||||
be called out and tested — it is **not** the no-op the refactor label implies. Confirm the
|
||||
converted query touches exactly the same rows with the same values MariaDB produced before.
|
||||
|
||||
---
|
||||
|
||||
## How to review
|
||||
|
||||
For every changed query: does it (a) use a construct from §1 (would error on PostgreSQL),
|
||||
(b) match a divergence in §2/§3 (different result across engines), or (c) change the row set under
|
||||
a refactor/conversion label (§6)? If so, comment with the
|
||||
portable fix and confirm it leaves **MariaDB output unchanged**. Skip the §4 false positives.
|
||||
Prefer a comment that names the rule (e.g. "loose GROUP BY — Max()-wrap, don't add to GROUP BY:
|
||||
splits the row count") so the fix is unambiguous.
|
||||
|
||||
The static pre-commit checker (`.github/helper/postgres_compat.py`) catches the *mechanical*
|
||||
§1 breaks; the **semantic** §2/§3 divergences and the §6 refactor/conversion row-set changes are
|
||||
exactly what a reviewer (and this guide) must cover, because no static check can see them.
|
||||
72
.github/helper/hydrate.sh
vendored
72
.github/helper/hydrate.sh
vendored
@@ -1,72 +0,0 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Hydrate a test shard from the setup job's artifact.
|
||||
#
|
||||
# The bench (apps, venv, node_modules, sites) is already on disk at ~/frappe-bench — the
|
||||
# workflow untar'd it from the artifact the setup job built. So there is NO bench init, no
|
||||
# asset build, and no reinstall here: just bring the DB up on the baked datadir and start redis
|
||||
# so tests can run. The whole point is that the expensive work happened ONCE in the setup job.
|
||||
#
|
||||
set -e
|
||||
|
||||
ci_user="${ERPNEXT_CI_USER:-frappe}"
|
||||
db_host="${DB_HOST:-127.0.0.1}"
|
||||
|
||||
# Re-exec as the ci user (uid 1001) so bench/cache ownership matches the artifact, same as
|
||||
# install.sh. The workflow untar'd as root with -p, so the files are already owned by ci.
|
||||
if [ "$(id -u)" = "0" ] && [ "${SKIP_SYSTEM_SETUP:-0}" = "1" ] && [ "$ci_user" != "root" ]; then
|
||||
exec su -m "$ci_user" -s /bin/bash -c \
|
||||
"ERPNEXT_CI_USER='$ci_user' DB_HOST='$db_host' DB='${DB:-}' bash '$0'"
|
||||
fi
|
||||
|
||||
cd ~/frappe-bench
|
||||
|
||||
# Start the DB on the datadir baked into the artifact. It's already populated (the setup job
|
||||
# reinstalled into this very datadir), so there is NO restore — the server comes up on the
|
||||
# existing files. This is what replaces the per-shard SQL replay.
|
||||
bash ~/frappe-bench/start-db.sh
|
||||
|
||||
# Bring up redis (lightmode unit tests need cache + queue). In the self-hosted container we use the
|
||||
# full `bench start` (web/workers too, like install.sh). On the bare GitHub Postgres shard
|
||||
# `bench start` (honcho) lagged — it blocks the redis procs behind web/worker procs the lightmode
|
||||
# suite never uses, so the wait below burned its full timeout (~4m). There, start the two redis
|
||||
# instances directly: fast and deterministic.
|
||||
if [ "${DB:-mariadb}" = "postgres" ]; then
|
||||
# Start redis directly as daemons — reliable and persists across steps. Do NOT route it through
|
||||
# `bench start`: honcho tears the whole process group down if any one Procfile proc dies on the
|
||||
# bare shard, which took redis with it (redis @ 13000 refused in Run Tests). Keeping redis
|
||||
# independent is what makes it survive. The web server (for PDF tests) is NOT started here — a
|
||||
# backgrounded server doesn't survive into the next step; it's started inside the Run Tests step.
|
||||
for conf in redis_cache redis_queue; do
|
||||
[ -f ~/frappe-bench/config/$conf.conf ] && redis-server ~/frappe-bench/config/$conf.conf --daemonize yes
|
||||
done
|
||||
else
|
||||
bench start >> ~/frappe-bench/bench_start.log 2>&1 &
|
||||
fi
|
||||
|
||||
# Wait for redis, failing fast instead of silently burning minutes if it never comes up.
|
||||
cfg=~/frappe-bench/sites/common_site_config.json
|
||||
if [ -f "$cfg" ]; then
|
||||
ports=$(python - "$cfg" <<'PY'
|
||||
import json, re, sys
|
||||
try:
|
||||
cfg = json.load(open(sys.argv[1]))
|
||||
except Exception:
|
||||
sys.exit(0)
|
||||
for key in ("redis_cache", "redis_queue"):
|
||||
m = re.search(r":(\d+)", str(cfg.get(key, "")))
|
||||
if m:
|
||||
print(m.group(1))
|
||||
PY
|
||||
)
|
||||
for port in $ports; do
|
||||
up=0
|
||||
for _ in $(seq 1 60); do
|
||||
if (exec 3<>"/dev/tcp/127.0.0.1/$port") 2>/dev/null; then exec 3>&- 3<&-; up=1; break; fi
|
||||
sleep 1
|
||||
done
|
||||
[ "$up" = "1" ] || { echo "redis did not come up on port $port"; exit 1; }
|
||||
done
|
||||
fi
|
||||
|
||||
echo "Hydrated: DB up on baked datadir, redis up — ready for tests."
|
||||
405
.github/helper/install.sh
vendored
405
.github/helper/install.sh
vendored
@@ -4,406 +4,75 @@ set -e
|
||||
|
||||
cd ~ || exit
|
||||
|
||||
# Authenticate git against github.com with the job token: anonymous git-over-HTTPS from the
|
||||
# runners gets throttled to a 401, which kills whichever clone is in flight — the frappe fetch
|
||||
# below, or payments under `bench get-app`. See the PR description.
|
||||
#
|
||||
# A credential helper rather than a url.insteadOf rewrite, because `git clone` PERSISTS a
|
||||
# rewritten URL into the new repo's .git/config: an insteadOf would leave the token sitting in
|
||||
# apps/payments/.git/config on the runner. A helper is consulted only when github.com actually
|
||||
# challenges, and leaves the stored remote URL untouched. Passing it through GIT_CONFIG_* keeps
|
||||
# the token out of ~/.gitconfig too, and child processes inherit it (bench shells out to git).
|
||||
ci_github_token=${CI_GITHUB_TOKEN:-${GITHUB_TOKEN:-}}
|
||||
if [ -n "$ci_github_token" ]; then
|
||||
export CI_GITHUB_TOKEN="$ci_github_token"
|
||||
export GIT_CONFIG_COUNT=3
|
||||
# Reset first: git runs EVERY configured helper and calls `store` on them after a successful
|
||||
# auth, so a `credential.helper=store` inherited from the image's gitconfig would write the
|
||||
# token to ~/.git-credentials. An empty value clears the list before ours is added.
|
||||
export GIT_CONFIG_KEY_0="credential.helper"
|
||||
export GIT_CONFIG_VALUE_0=""
|
||||
export GIT_CONFIG_KEY_1="credential.https://github.com.username"
|
||||
export GIT_CONFIG_VALUE_1="x-access-token"
|
||||
export GIT_CONFIG_KEY_2="credential.https://github.com.helper"
|
||||
# Single-quoted: $CI_GITHUB_TOKEN is expanded by the shell git runs the helper in, so the
|
||||
# token is read from the environment at call time and never stored anywhere. Answering only
|
||||
# `get` makes the helper inert for git's `store`/`erase` calls.
|
||||
export GIT_CONFIG_VALUE_2='!f() { test "$1" = get && echo "password=$CI_GITHUB_TOKEN"; }; f'
|
||||
fi
|
||||
sudo apt update
|
||||
sudo apt remove mysql-server mysql-client
|
||||
sudo apt install libcups2-dev redis-server mariadb-client libmariadb-dev
|
||||
|
||||
# Whatever happens, never sit on a credential prompt: fail fast and legibly instead.
|
||||
export GIT_TERMINAL_PROMPT=0
|
||||
pip install frappe-bench
|
||||
|
||||
githubbranch=${GITHUB_BASE_REF:-${GITHUB_REF##*/}}
|
||||
frappeuser=${FRAPPE_USER:-"frappe"}
|
||||
frappecommitish=${FRAPPE_BRANCH:-}
|
||||
|
||||
# A stacked pull request targets another erpnext branch, which has no counterpart in frappe.
|
||||
# Fall back to develop so the bench is still installed. An explicit FRAPPE_BRANCH is trusted as
|
||||
# given, since it can be a commit sha rather than a branch.
|
||||
if [ -z "$frappecommitish" ]; then
|
||||
frappecommitish=$githubbranch
|
||||
|
||||
# git ls-remote --exit-code reports 2 for a branch that is not there and 128 for a remote it
|
||||
# could not reach. Only the first one is proof of absence; keep the branch on anything else so
|
||||
# a flaky probe cannot install an unrelated frappe.
|
||||
probe=0
|
||||
git ls-remote --exit-code --heads "https://github.com/${frappeuser}/frappe" "$frappecommitish" >/dev/null 2>&1 || probe=$?
|
||||
|
||||
if [ "$probe" -eq 2 ]; then
|
||||
echo "frappe has no branch ${frappecommitish}, falling back to develop"
|
||||
frappecommitish=develop
|
||||
elif [ "$probe" -ne 0 ]; then
|
||||
echo "could not reach frappe to check for branch ${frappecommitish} (git ls-remote exited ${probe}), keeping it"
|
||||
fi
|
||||
fi
|
||||
db_host=${DB_HOST:-"127.0.0.1"}
|
||||
db_user_host=${DB_USER_HOST:-"localhost"}
|
||||
wkhtmltox_deb=${WKHTMLTOX_DEB:-"/tmp/wkhtmltox.deb"}
|
||||
bench_cache_dir=${BENCH_CACHE_DIR:-}
|
||||
|
||||
run_as_ci_user_if_needed() {
|
||||
if [ "$(id -u)" != "0" ] || [ "${SKIP_SYSTEM_SETUP:-0}" != "1" ] || [ "${ERPNEXT_CI_NON_ROOT:-0}" = "1" ]; then
|
||||
return
|
||||
fi
|
||||
|
||||
local missing_packages=()
|
||||
if ! command -v pkg-config >/dev/null 2>&1; then
|
||||
missing_packages+=("pkg-config")
|
||||
fi
|
||||
if ! command -v mariadb_config >/dev/null 2>&1 && ! command -v mysql_config >/dev/null 2>&1; then
|
||||
missing_packages+=("libmariadb-dev")
|
||||
fi
|
||||
if ! command -v crontab >/dev/null 2>&1; then
|
||||
missing_packages+=("cron")
|
||||
fi
|
||||
|
||||
if [ "${#missing_packages[@]}" -gt 0 ]; then
|
||||
apt-get update
|
||||
apt-get install -y --no-install-recommends "${missing_packages[@]}"
|
||||
fi
|
||||
|
||||
local ci_user="${ERPNEXT_CI_USER:-frappe}"
|
||||
|
||||
if ! id "$ci_user" >/dev/null 2>&1; then
|
||||
useradd --home-dir "$HOME" --no-create-home --shell /bin/bash "$ci_user"
|
||||
fi
|
||||
|
||||
rm -rf ~/frappe ~/frappe-bench
|
||||
|
||||
local ci_dirs=(
|
||||
"$HOME"
|
||||
"$GITHUB_WORKSPACE"
|
||||
"$HOME/.cache"
|
||||
"${PIP_CACHE_DIR:-$HOME/.cache/pip}"
|
||||
"${npm_config_cache:-$HOME/.npm}"
|
||||
"${YARN_CACHE_FOLDER:-$HOME/.cache/yarn}"
|
||||
"$HOME/.yarn"
|
||||
"${UV_CACHE_DIR:-$HOME/.cache/uv}"
|
||||
"$(dirname "$wkhtmltox_deb")"
|
||||
)
|
||||
if [ -n "$bench_cache_dir" ]; then
|
||||
ci_dirs+=("$bench_cache_dir")
|
||||
fi
|
||||
|
||||
# Create + own (non-recursively) the home/cache/workspace dirs before dropping to
|
||||
# the ci user. We deliberately do NOT wipe the yarn/uv caches here so a persistent
|
||||
# cache (mounted volume or baked image layer) stays warm across runs.
|
||||
mkdir -p "${ci_dirs[@]}" "$HOME/.yarn"
|
||||
chown "$ci_user:$ci_user" "${ci_dirs[@]}" "$HOME/.yarn"
|
||||
|
||||
export ERPNEXT_CI_NON_ROOT=1
|
||||
exec su -m "$ci_user" -s /bin/bash -c "cd '$HOME' && bash '$GITHUB_WORKSPACE/.github/helper/install.sh'"
|
||||
}
|
||||
|
||||
run_as_ci_user_if_needed
|
||||
|
||||
run_ci_step() {
|
||||
local label=$1
|
||||
shift
|
||||
|
||||
echo "::group::${label}"
|
||||
date -u
|
||||
local exit_code=0
|
||||
timeout --foreground "${CI_INSTALL_STEP_TIMEOUT:-1800}" "$@" || exit_code=$?
|
||||
date -u
|
||||
echo "::endgroup::"
|
||||
return "$exit_code"
|
||||
}
|
||||
|
||||
if [ -n "${GITHUB_WORKSPACE:-}" ]; then
|
||||
git config --global --add safe.directory "$GITHUB_WORKSPACE" || true
|
||||
git config --global --add safe.directory "$GITHUB_WORKSPACE/.git" || true
|
||||
fi
|
||||
|
||||
rm -rf ~/frappe ~/frappe-bench
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 1 — parallelise the three slow, independent setup steps:
|
||||
# a) system packages b) frappe-bench pip install c) frappe git fetch
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
if [ "${SKIP_SYSTEM_SETUP:-0}" != "1" ]; then
|
||||
sudo apt-get update
|
||||
|
||||
# apt remove/install must run sequentially but can overlap with pip and git.
|
||||
sudo apt-get remove -y mysql-server mysql-client
|
||||
sudo apt-get install -y libcups2-dev redis-server mariadb-client libmariadb-dev &
|
||||
apt_pid=$!
|
||||
|
||||
pip install frappe-bench &
|
||||
pip_pid=$!
|
||||
else
|
||||
apt_pid=
|
||||
pip_pid=
|
||||
fi
|
||||
frappecommitish=${FRAPPE_BRANCH:-$githubbranch}
|
||||
|
||||
mkdir frappe
|
||||
(
|
||||
cd frappe
|
||||
git init
|
||||
git remote add origin "https://github.com/${frappeuser}/frappe"
|
||||
git fetch origin "${frappecommitish}" --depth 1
|
||||
) &
|
||||
clone_pid=$!
|
||||
|
||||
if [ -n "$apt_pid" ]; then wait $apt_pid; fi
|
||||
if [ -n "$pip_pid" ]; then wait $pip_pid; fi
|
||||
wait $clone_pid
|
||||
|
||||
pushd frappe
|
||||
git init
|
||||
git remote add origin "https://github.com/${frappeuser}/frappe"
|
||||
git fetch origin "${frappecommitish}" --depth 1
|
||||
git checkout FETCH_HEAD
|
||||
popd
|
||||
frappe_sha=$(git -C frappe rev-parse HEAD)
|
||||
|
||||
get_bench_cache_archive() {
|
||||
if [ -z "$bench_cache_dir" ]; then
|
||||
return
|
||||
fi
|
||||
bench init --skip-assets --frappe-path ~/frappe --python "$(which python)" frappe-bench
|
||||
|
||||
mkdir -p "$bench_cache_dir"
|
||||
|
||||
# Keyed on tool versions only (NOT the frappe SHA): any recent base bench works, because
|
||||
# restore_warm_bench fast-forwards it to the exact live develop SHA. This is what lets a
|
||||
# constantly-moving develop still hit the cache.
|
||||
local cache_key
|
||||
cache_key=$(
|
||||
{
|
||||
uname -m
|
||||
python --version
|
||||
node --version
|
||||
bench --version
|
||||
} | sha256sum | awk '{print $1}'
|
||||
)
|
||||
|
||||
echo "${bench_cache_dir}/frappe-bench-base-${cache_key}.tar.zst"
|
||||
}
|
||||
|
||||
restore_warm_bench() {
|
||||
bench_cache_archive=$(get_bench_cache_archive)
|
||||
[ -n "$bench_cache_archive" ] && [ -f "$bench_cache_archive" ] || return 1
|
||||
|
||||
echo "Restoring base bench from ${bench_cache_archive}"
|
||||
tar --use-compress-program=unzstd -xf "$bench_cache_archive" -C ~ || return 1
|
||||
[ -d ~/frappe-bench/apps/frappe/.git ] || return 1
|
||||
mkdir -p ~/frappe-bench/sites ~/frappe-bench/logs
|
||||
[ -f ~/frappe-bench/sites/apps.txt ] || printf "frappe\n" > ~/frappe-bench/sites/apps.txt
|
||||
[ -f ~/frappe-bench/sites/common_site_config.json ] || printf "{}\n" > ~/frappe-bench/sites/common_site_config.json
|
||||
|
||||
# Fast-forward the restored frappe to the EXACT live develop SHA fetched in phase 1, then
|
||||
# rebuild only what changed. The editable install means the venv tracks the new code with
|
||||
# no reinstall. Any failure returns non-zero so the caller falls back to a full bench init.
|
||||
if ! (
|
||||
cd ~/frappe-bench/apps/frappe || exit 1
|
||||
# Phase 1 already fetched ~/frappe to the exact live develop SHA. Fetch that commit
|
||||
# straight from it (bench init names the remote 'upstream', not 'origin', and points
|
||||
# it at this local clone — so a plain `git fetch origin` does not work).
|
||||
git fetch --no-tags --update-shallow "$HOME/frappe" HEAD || exit 1
|
||||
git checkout --force FETCH_HEAD || exit 1
|
||||
); then
|
||||
echo "Fast-forward to ${frappe_sha} failed; falling back to full init"
|
||||
rm -rf ~/frappe-bench
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Pick up any frappe dependency changes since the base was built (cached → fast if none),
|
||||
# so a develop commit that bumped requirements doesn't leave a stale venv.
|
||||
if ! ~/frappe-bench/env/bin/python -m pip install -q -e ~/frappe-bench/apps/frappe; then
|
||||
echo "frappe dependency refresh failed; falling back to full init"
|
||||
rm -rf ~/frappe-bench
|
||||
return 1
|
||||
fi
|
||||
|
||||
( cd ~/frappe-bench && CI=Yes bench build --app frappe ) || { rm -rf ~/frappe-bench; return 1; }
|
||||
return 0
|
||||
}
|
||||
|
||||
save_warm_bench() {
|
||||
if [ -z "${bench_cache_archive:-}" ] || [ -f "$bench_cache_archive" ]; then
|
||||
return
|
||||
fi
|
||||
|
||||
if [ -n "$bench_cache_dir" ] && [ ! -w "$bench_cache_dir" ]; then
|
||||
echo "Skipping warm bench save because ${bench_cache_dir} is not writable"
|
||||
return
|
||||
fi
|
||||
|
||||
local tmp_archive
|
||||
tmp_archive="${bench_cache_archive}.${$}.tmp"
|
||||
|
||||
echo "Saving warm bench to ${bench_cache_archive}"
|
||||
# Keep sites/common_site_config.json (the redis ports live there — dropping it makes the
|
||||
# restore path fall back to a default redis port that bench start never bound, so reinstall
|
||||
# fails with "redis ... connection refused"). Only the rebuildable sites/assets is excluded;
|
||||
# restore_warm_bench runs `bench build` to regenerate it.
|
||||
tar \
|
||||
--use-compress-program="zstd -T0 -3" \
|
||||
--exclude="frappe-bench/logs" \
|
||||
--exclude="frappe-bench/sites/assets" \
|
||||
-cf "$tmp_archive" \
|
||||
-C ~ frappe-bench
|
||||
mv "$tmp_archive" "$bench_cache_archive"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Phase 2 — bench init and site setup
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
install_whktml() {
|
||||
# Re-use the .deb if the wkhtmltopdf cache step already restored it.
|
||||
if [ ! -f "$wkhtmltox_deb" ]; then
|
||||
wget -O "$wkhtmltox_deb" https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6.1-2/wkhtmltox_0.12.6.1-2.jammy_amd64.deb
|
||||
fi
|
||||
sudo apt-get install -y "$wkhtmltox_deb"
|
||||
}
|
||||
if [ "${SKIP_WKHTMLTOX_SETUP:-0}" != "1" ]; then
|
||||
install_whktml &
|
||||
wkpid=$!
|
||||
else
|
||||
wkpid=
|
||||
fi
|
||||
|
||||
if ! restore_warm_bench; then
|
||||
bench init --skip-assets --frappe-path ~/frappe --python "$(which python)" frappe-bench
|
||||
|
||||
cd ~/frappe-bench || exit
|
||||
|
||||
sed -i 's/watch:/# watch:/g' Procfile
|
||||
sed -i 's/schedule:/# schedule:/g' Procfile
|
||||
sed -i 's/socketio:/# socketio:/g' Procfile
|
||||
sed -i 's/redis_socketio:/# redis_socketio:/g' Procfile
|
||||
|
||||
CI=Yes bench build --app frappe
|
||||
save_warm_bench
|
||||
fi
|
||||
|
||||
if [ -n "$wkpid" ]; then wait $wkpid; fi
|
||||
|
||||
mkdir -p ~/frappe-bench/sites/test_site
|
||||
mkdir ~/frappe-bench/sites/test_site
|
||||
|
||||
if [ "$DB" == "mariadb" ];then
|
||||
cp -r "${GITHUB_WORKSPACE}/.github/helper/site_config_mariadb.json" ~/frappe-bench/sites/test_site/site_config.json
|
||||
if [ "$db_host" != "127.0.0.1" ]; then
|
||||
sed -i "s/\"db_host\": \"127.0.0.1\"/\"db_host\": \"${db_host}\"/" ~/frappe-bench/sites/test_site/site_config.json
|
||||
fi
|
||||
else
|
||||
cp -r "${GITHUB_WORKSPACE}/.github/helper/site_config_postgres.json" ~/frappe-bench/sites/test_site/site_config.json
|
||||
fi
|
||||
|
||||
|
||||
if [ "$DB" == "mariadb" ];then
|
||||
for _ in {1..60}; do
|
||||
if mariadb-admin ping --host "$db_host" --port 3306 -u root -proot --silent; then
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
mariadb-admin ping --host "$db_host" --port 3306 -u root -proot --silent
|
||||
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "SET GLOBAL character_set_server = 'utf8mb4'"
|
||||
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "SET GLOBAL collation_server = 'utf8mb4_unicode_ci'"
|
||||
|
||||
mariadb --host "$db_host" --port 3306 -u root -proot -e "SET GLOBAL character_set_server = 'utf8mb4'"
|
||||
mariadb --host "$db_host" --port 3306 -u root -proot -e "SET GLOBAL collation_server = 'utf8mb4_unicode_ci'"
|
||||
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "CREATE USER 'test_frappe'@'localhost' IDENTIFIED BY 'test_frappe'"
|
||||
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "CREATE DATABASE test_frappe"
|
||||
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "GRANT ALL PRIVILEGES ON \`test_frappe\`.* TO 'test_frappe'@'localhost'"
|
||||
|
||||
# Throwaway-DB durability tuning at runtime. (innodb_doublewrite is read-only on MariaDB
|
||||
# 10.6, so it can't be disabled here — would need a server startup flag.)
|
||||
mariadb --host "$db_host" --port 3306 -u root -proot \
|
||||
-e "SET GLOBAL innodb_flush_log_at_trx_commit=0; SET GLOBAL sync_binlog=0;"
|
||||
|
||||
# Opt-in DDL speedup: a shared tablespace avoids a create+fsync per DocType table during
|
||||
# reinstall — a big win under disk contention. But ROW_FORMAT=DYNAMIC must be accepted in
|
||||
# the system tablespace on this MariaDB. Enable with CI_INNODB_SHARED_TABLESPACE=1; if
|
||||
# reinstall then errors on table creation, unset it (off by default — zero risk).
|
||||
if [ "${CI_INNODB_SHARED_TABLESPACE:-0}" = "1" ]; then
|
||||
mariadb --host "$db_host" --port 3306 -u root -proot -e "SET GLOBAL innodb_file_per_table=0;"
|
||||
fi
|
||||
|
||||
mariadb --host "$db_host" --port 3306 -u root -proot -e "CREATE USER 'test_frappe'@'${db_user_host}' IDENTIFIED BY 'test_frappe'"
|
||||
mariadb --host "$db_host" --port 3306 -u root -proot -e "CREATE DATABASE test_frappe"
|
||||
mariadb --host "$db_host" --port 3306 -u root -proot -e "GRANT ALL PRIVILEGES ON \`test_frappe\`.* TO 'test_frappe'@'${db_user_host}'"
|
||||
|
||||
mariadb --host "$db_host" --port 3306 -u root -proot -e "FLUSH PRIVILEGES"
|
||||
mariadb --host 127.0.0.1 --port 3306 -u root -proot -e "FLUSH PRIVILEGES"
|
||||
fi
|
||||
|
||||
if [ "$DB" == "postgres" ];then
|
||||
echo "travis" | psql -h 127.0.0.1 -p 5432 -c "CREATE DATABASE test_frappe" -U postgres;
|
||||
echo "travis" | psql -h 127.0.0.1 -p 5432 -c "CREATE USER test_frappe WITH PASSWORD 'test_frappe'" -U postgres;
|
||||
|
||||
# Durability-off for speed (no fsync/synchronous_commit/full_page_writes) is applied by
|
||||
# start-db.sh's postgres `-o` flags on every start — setup job AND each test shard — so it is
|
||||
# NOT repeated here. The postgres workflow runs in-runner via start-db.sh, not a service
|
||||
# container.
|
||||
fi
|
||||
|
||||
|
||||
install_whktml() {
|
||||
wget -O /tmp/wkhtmltox.deb https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6.1-2/wkhtmltox_0.12.6.1-2.jammy_amd64.deb
|
||||
sudo apt install /tmp/wkhtmltox.deb
|
||||
|
||||
}
|
||||
install_whktml &
|
||||
wkpid=$!
|
||||
|
||||
|
||||
cd ~/frappe-bench || exit
|
||||
|
||||
run_ci_step "Get payments app" bench get-app payments --branch develop
|
||||
sed -i 's/watch:/# watch:/g' Procfile
|
||||
sed -i 's/schedule:/# schedule:/g' Procfile
|
||||
sed -i 's/socketio:/# socketio:/g' Procfile
|
||||
sed -i 's/redis_socketio:/# redis_socketio:/g' Procfile
|
||||
|
||||
# Opt-in: skip building erpnext's frontend assets. Server tests don't need them, but PDF
|
||||
# tests (print formats) do — they pass only if the PDF renderer ignores missing assets.
|
||||
# Enable with CI_SKIP_ERPNEXT_ASSETS=1 to test; if PDF tests fail, unset it.
|
||||
erpnext_get_app_args=()
|
||||
if [ "${CI_SKIP_ERPNEXT_ASSETS:-0}" = "1" ]; then erpnext_get_app_args=(--skip-assets); fi
|
||||
run_ci_step "Get erpnext app" bench get-app erpnext "${GITHUB_WORKSPACE}" "${erpnext_get_app_args[@]}"
|
||||
bench get-app payments --branch develop
|
||||
bench get-app erpnext "${GITHUB_WORKSPACE}"
|
||||
|
||||
if [ "$TYPE" == "server" ]; then run_ci_step "Setup dev requirements" bench setup requirements --dev; fi
|
||||
if [ "$TYPE" == "server" ]; then bench setup requirements --dev; fi
|
||||
|
||||
bench start >> ~/frappe-bench/bench_start.log 2>&1 &
|
||||
wait $wkpid
|
||||
|
||||
# Under heavy concurrency, gunicorn's startup can delay redis coming up. reinstall and the
|
||||
# tests need redis, so wait for it (best-effort, bounded) instead of racing — contention
|
||||
# then slows the job rather than failing it.
|
||||
wait_for_redis() {
|
||||
local cfg=~/frappe-bench/sites/common_site_config.json
|
||||
[ -f "$cfg" ] || return 0
|
||||
local ports port
|
||||
ports=$(python - "$cfg" <<'PY'
|
||||
import json, re, sys
|
||||
try:
|
||||
cfg = json.load(open(sys.argv[1]))
|
||||
except Exception:
|
||||
sys.exit(0)
|
||||
for key in ("redis_cache", "redis_queue"):
|
||||
match = re.search(r":(\d+)", str(cfg.get(key, "")))
|
||||
if match:
|
||||
print(match.group(1))
|
||||
PY
|
||||
)
|
||||
for port in $ports; do
|
||||
local up=0
|
||||
for _ in $(seq 1 120); do
|
||||
if (exec 3<>"/dev/tcp/127.0.0.1/$port") 2>/dev/null; then
|
||||
exec 3>&- 3<&-; up=1
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
# Fail clearly instead of letting reinstall die later on a vague socket-connection error
|
||||
# when redis never bound.
|
||||
[ "$up" = "1" ] || { echo "redis did not come up on port $port"; return 1; }
|
||||
done
|
||||
}
|
||||
wait_for_redis
|
||||
|
||||
# Site setup: build the schema (~1000 DocTypes) into the DB. This is the single-threaded-Python
|
||||
# bottleneck, but the fan-out amortises it — it runs once here in the setup job, and the test
|
||||
# shards start the DB on the baked datadir instead of repeating the reinstall.
|
||||
run_ci_step "Reinstall test site" bench --site test_site reinstall --yes
|
||||
bench start &>> ~/frappe-bench/bench_start.log &
|
||||
CI=Yes bench build --app frappe &
|
||||
bench --site test_site reinstall --yes
|
||||
|
||||
241
.github/helper/postgres_compat.py
vendored
241
.github/helper/postgres_compat.py
vendored
@@ -1,241 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Static guard against MySQL-only SQL that breaks on PostgreSQL.
|
||||
|
||||
The Postgres test job is label-gated, so it does not run on every PR. This pre-commit
|
||||
hook is the always-on first line of defence: it flags the *mechanical* Postgres breaks
|
||||
that static analysis can catch reliably with a low false-positive rate.
|
||||
|
||||
It deliberately does NOT try to catch the *semantic* divergences (loose GROUP BY,
|
||||
case-sensitive ==/IN, NULL ordering, ORDER BY ... LIMIT 1 tiebreakers, integer-division
|
||||
intent, division by a possibly-zero divisor, savepoint discipline) — those genuinely need
|
||||
the test suite or a human/Greptile reviewer. Run the full suite on a Postgres site for those.
|
||||
|
||||
Escape hatch: put `# pg-ok` anywhere on the offending statement's line span (e.g. on a
|
||||
`SHOW INDEX` query that lives inside an `if frappe.db.db_type == "mariadb":` branch).
|
||||
|
||||
Usage: postgres_compat.py <file.py> [<file.py> ...] (pre-commit passes staged files)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import re
|
||||
import sys
|
||||
|
||||
IGNORE = "pg-ok"
|
||||
|
||||
# Strings are only scanned for the patterns below when they have real SQL *structure*
|
||||
# (not just an English word like "select" or "from"), to keep false positives near zero.
|
||||
SQL_HINT = re.compile(
|
||||
r"\bselect\b[\s\S]{0,800}\bfrom\b" # SELECT ... FROM
|
||||
r"|\bupdate\b[\s\S]{0,400}\bset\b" # UPDATE ... SET
|
||||
r"|\bdelete\s+from\b"
|
||||
r"|\binsert\s+into\b"
|
||||
r"|\bshow\s+(?:index|tables|columns)\b"
|
||||
r"|\bfrom\s+[\"'`]?tab", # FROM `tabDocType`
|
||||
re.I,
|
||||
)
|
||||
|
||||
# MySQL-only constructs with NO frappe auto-translation. (frappe.db.sql already rewrites
|
||||
# ifnull->coalesce on all engines and backtick/locate/REGEXP on Postgres, and .like()
|
||||
# renders ILIKE — so those are NOT listed here; flagging them would be false positives.)
|
||||
SQL_PATTERNS: list[tuple[re.Pattern, str]] = [
|
||||
(re.compile(r"\btimestamp\s*\(\s*[^,()]+,", re.I),
|
||||
"timestamp(date, time) is MySQL-only -> use CombineDatetime() or a precomputed datetime column"),
|
||||
(re.compile(r"\btimediff\s*\(", re.I),
|
||||
"timediff() is MySQL-only -> compute the delta in Python"),
|
||||
(re.compile(r"\bstr_to_date\s*\(", re.I),
|
||||
"str_to_date() is MySQL-only -> parse in Python and pass a real date"),
|
||||
(re.compile(r"\bdate_format\s*\(", re.I),
|
||||
"date_format() is MySQL-only -> filter on a date range instead"),
|
||||
(re.compile(r"\bdate_(add|sub)\s*\(", re.I),
|
||||
"date_add()/date_sub() are MySQL-only -> use Python date math or interval arithmetic"),
|
||||
(re.compile(r"\bgroup_concat\s*\(", re.I),
|
||||
"group_concat() is MySQL-only -> use GroupConcat (string_agg) or aggregate in Python"),
|
||||
(re.compile(r"\bperiod_diff\s*\(", re.I),
|
||||
"period_diff() is MySQL-only -> compute in Python"),
|
||||
(re.compile(r"\bshow\s+index\b", re.I),
|
||||
"SHOW INDEX is MySQL-only -> use frappe.db.has_index() / get_column_index()"),
|
||||
(re.compile(r"\bshow\s+(tables|columns)\b", re.I),
|
||||
"SHOW TABLES/COLUMNS is MySQL-only -> use frappe.db.get_tables()/table_columns / information-schema helpers"),
|
||||
(re.compile(r"\bas\s+'[^']+'", re.I),
|
||||
"single-quoted column alias breaks on Postgres -> use a bare or double-quoted alias"),
|
||||
(re.compile(r"\bif\s*\(", re.I),
|
||||
"SQL IF() is MySQL-only -> use CASE WHEN ... THEN ... ELSE ... END (frappe.qb.Case())"),
|
||||
(re.compile(r"\brlike\b", re.I),
|
||||
"RLIKE is MySQL-only -> frappe rewrites REGEXP->~* on Postgres but NOT RLIKE; use REGEXP / .regexp() / ~"),
|
||||
(re.compile(r"\bcast\s*\(.+?\bas\s+char\b", re.I | re.S), # .+? spans nested parens, e.g. CAST(ABS(x) AS CHAR)
|
||||
"CAST(... AS CHAR) is character(1) on Postgres and truncates -> CAST AS VARCHAR (frappe Cast_(x, 'varchar'))"),
|
||||
]
|
||||
|
||||
# UPDATE ... JOIN: both keywords in the same SQL string.
|
||||
UPDATE_JOIN = (re.compile(r"\bupdate\b", re.I), re.compile(r"\bjoin\b", re.I))
|
||||
|
||||
MYSQL_RESULT_KEYS = {"Column_name", "Key_name", "Seq_in_index", "Non_unique", "Index_type"}
|
||||
|
||||
SET_BOOL_FUNCS = {"set_value", "db_set"}
|
||||
|
||||
# query-builder cast helpers: pypika Cast / frappe Cast_. A "char" target type is character(1)
|
||||
# on Postgres (truncates); "varchar" is the full-length cast.
|
||||
CAST_FUNCS = {"Cast", "Cast_"}
|
||||
|
||||
# frappe.get_all / get_list: frappe's db_query SILENTLY drops ORDER BY for `distinct` queries on
|
||||
# Postgres (the ORDER BY column must appear in the SELECT-DISTINCT list), so `distinct=True` together
|
||||
# with a literal `order_by` is a no-op on PG and the result comes back unordered.
|
||||
DISTINCT_ORDER_FUNCS = {"get_all", "get_list"}
|
||||
|
||||
|
||||
def _docstring_ids(tree: ast.AST) -> set[int]:
|
||||
"""ids of Constant nodes that are docstrings (so prose describing the rules isn't flagged)."""
|
||||
ids: set[int] = set()
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, (ast.Module, ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
|
||||
body = getattr(node, "body", None)
|
||||
if body and isinstance(body[0], ast.Expr) and isinstance(body[0].value, ast.Constant) and isinstance(body[0].value.value, str):
|
||||
ids.add(id(body[0].value))
|
||||
return ids
|
||||
|
||||
|
||||
class Visitor(ast.NodeVisitor):
|
||||
def __init__(self, lines: list[str], docstrings: set[int]):
|
||||
self.lines = lines
|
||||
self.docstrings = docstrings
|
||||
self.violations: list[tuple[int, str]] = []
|
||||
|
||||
def _ignored(self, node: ast.AST) -> bool:
|
||||
start = getattr(node, "lineno", 1)
|
||||
end = getattr(node, "end_lineno", start) or start
|
||||
# honour `# pg-ok` anywhere on the node's line span, the line just above (the enclosing
|
||||
# call, e.g. `frappe.db.sql( # pg-ok`), or the line just below (a multi-line call's `) # pg-ok`).
|
||||
lo = max(0, start - 2)
|
||||
return any(IGNORE in self.lines[i] for i in range(lo, min(end + 1, len(self.lines))))
|
||||
|
||||
def _flag(self, node: ast.AST, msg: str) -> None:
|
||||
if not self._ignored(node):
|
||||
self.violations.append((getattr(node, "lineno", 1), msg))
|
||||
|
||||
def _scan_sql(self, text: str, node: ast.AST) -> None:
|
||||
if not SQL_HINT.search(text):
|
||||
return
|
||||
for pattern, msg in SQL_PATTERNS:
|
||||
if pattern.search(text):
|
||||
self._flag(node, msg)
|
||||
if UPDATE_JOIN[0].search(text) and UPDATE_JOIN[1].search(text):
|
||||
self._flag(node, "UPDATE ... JOIN is MySQL-only -> use a correlated subquery (WHERE ... IN/EXISTS)")
|
||||
|
||||
def visit_Constant(self, node: ast.Constant) -> None:
|
||||
# plain string literals, incl. `"...".format()` and `"..." % (...)` templates
|
||||
if isinstance(node.value, str) and id(node) not in self.docstrings:
|
||||
self._scan_sql(node.value, node)
|
||||
self.generic_visit(node)
|
||||
|
||||
def visit_JoinedStr(self, node: ast.JoinedStr) -> None:
|
||||
# f-string: scan its STATIC text (interpolated values become a placeholder) so MySQL-isms
|
||||
# in dynamic SQL are caught, without flagging safe interpolation of identifiers.
|
||||
text = "".join(
|
||||
v.value if isinstance(v, ast.Constant) and isinstance(v.value, str) else " ? "
|
||||
for v in node.values
|
||||
)
|
||||
self._scan_sql(text, node)
|
||||
# don't recurse: child literal chunks would otherwise be re-scanned individually
|
||||
|
||||
def visit_Call(self, node: ast.Call) -> None:
|
||||
fn = node.func
|
||||
name = fn.attr if isinstance(fn, ast.Attribute) else (fn.id if isinstance(fn, ast.Name) else "")
|
||||
|
||||
# row.get("Column_name") — MySQL SHOW INDEX result key
|
||||
if name == "get" and node.args and isinstance(node.args[0], ast.Constant) and node.args[0].value in MYSQL_RESULT_KEYS:
|
||||
self._flag(node, f'"{node.args[0].value}" is a MySQL SHOW INDEX result key -> use frappe.db.has_index()/get_column_index()')
|
||||
|
||||
# set_value(..., True) / db_set("field", True) on a Check (int) column.
|
||||
# Only the field *value* arg carries bool->smallint risk — NOT trailing flags like
|
||||
# update_modified. db_set(field, value, update_modified, ...) -> value at args[1] (or a dict
|
||||
# at args[0]); set_value(dt, dn, field, value, ...) -> value at args[3] (or a dict at args[2]).
|
||||
if name in SET_BOOL_FUNCS:
|
||||
value_idx, dict_idx = (1, 0) if name == "db_set" else (3, 2)
|
||||
dict_arg = (
|
||||
node.args[dict_idx]
|
||||
if len(node.args) > dict_idx and isinstance(node.args[dict_idx], ast.Dict)
|
||||
else None
|
||||
)
|
||||
if dict_arg is not None:
|
||||
for v in dict_arg.values:
|
||||
if isinstance(v, ast.Constant) and isinstance(v.value, bool):
|
||||
self._flag(node, f"{name}(...) sets an int/Check column with a bool in a dict -> pass 1/0 (Postgres rejects bool->smallint)")
|
||||
elif len(node.args) > value_idx:
|
||||
a = node.args[value_idx]
|
||||
if isinstance(a, ast.Constant) and isinstance(a.value, bool):
|
||||
self._flag(node, f"{name}(..., {a.value}) sets an int/Check column with a bool -> pass 1/0 (Postgres rejects bool->smallint)")
|
||||
|
||||
# frappe.get_all/get_list(..., distinct=True, order_by="<col>") -> ORDER BY is silently dropped
|
||||
# for distinct queries on Postgres, so the result is unordered there. Sort in python instead
|
||||
# (e.g. sorted(frappe.get_all(..., distinct=True), key=str.casefold)). An empty order_by="" (the
|
||||
# explicit "suppress the injected default" idiom) and a dynamic/variable order_by are not flagged.
|
||||
if name in DISTINCT_ORDER_FUNCS:
|
||||
has_distinct = any(
|
||||
kw.arg == "distinct" and isinstance(kw.value, ast.Constant) and kw.value.value
|
||||
for kw in node.keywords
|
||||
)
|
||||
order_kw = next((kw for kw in node.keywords if kw.arg == "order_by"), None)
|
||||
has_literal_order = (
|
||||
order_kw is not None
|
||||
and isinstance(order_kw.value, ast.Constant)
|
||||
and isinstance(order_kw.value.value, str)
|
||||
and order_kw.value.value.strip()
|
||||
)
|
||||
if has_distinct and has_literal_order:
|
||||
self._flag(node, f"{name}(distinct=True, order_by=...) -> frappe drops ORDER BY for distinct queries on Postgres; sort in python instead, e.g. sorted(..., key=str.casefold)")
|
||||
|
||||
# query-builder .rlike(...): pypika emits the MySQL-only RLIKE operator, which frappe does
|
||||
# NOT translate for Postgres (it rewrites only REGEXP -> ~*).
|
||||
if name == "rlike":
|
||||
self._flag(node, ".rlike() emits MySQL-only RLIKE (not translated on Postgres) -> use .regexp() (rewritten to ~*) or .like()")
|
||||
|
||||
# Cast(col, "char") / Cast_(col, "char"): on Postgres a bare CHAR is character(1) and truncates
|
||||
# (e.g. CAST(12 AS CHAR) -> '1'); use "varchar" for a full-length string cast.
|
||||
if name in CAST_FUNCS:
|
||||
for arg in (*node.args, *(kw.value for kw in node.keywords)):
|
||||
if isinstance(arg, ast.Constant) and isinstance(arg.value, str) and arg.value.strip().lower() == "char":
|
||||
self._flag(node, f"{name}(..., 'char') is character(1) on Postgres and truncates -> use 'varchar'")
|
||||
|
||||
self.generic_visit(node)
|
||||
|
||||
def visit_Subscript(self, node: ast.Subscript) -> None:
|
||||
key = node.slice
|
||||
if isinstance(key, ast.Constant) and key.value in MYSQL_RESULT_KEYS:
|
||||
self._flag(node, f'"{key.value}" is a MySQL SHOW INDEX result key -> use frappe.db.has_index()/get_column_index()')
|
||||
self.generic_visit(node)
|
||||
|
||||
|
||||
def check_file(path: str) -> list[str]:
|
||||
try:
|
||||
# nosemgrep: frappe-semgrep-rules.rules.security.frappe-security-file-traversal -- dev-only lint tool; `path` is a source file supplied by pre-commit, not user input
|
||||
src = open(path, encoding="utf-8").read()
|
||||
except (OSError, UnicodeDecodeError):
|
||||
return []
|
||||
try:
|
||||
tree = ast.parse(src, filename=path)
|
||||
except SyntaxError:
|
||||
return [] # check-ast hook reports real syntax errors
|
||||
v = Visitor(src.splitlines(), _docstring_ids(tree))
|
||||
v.visit(tree)
|
||||
return [f"{path}:{line}: [pg-compat] {msg}" for line, msg in sorted(set(v.violations))]
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
out: list[str] = []
|
||||
for path in argv:
|
||||
if path.endswith(".py"):
|
||||
out.extend(check_file(path))
|
||||
if out:
|
||||
print("\n".join(out))
|
||||
print(
|
||||
f"\n{len(out)} PostgreSQL-incompatibility issue(s). Fix them, or add `# pg-ok` to a "
|
||||
"line that is intentionally MariaDB-only (e.g. inside an `if frappe.db.db_type == 'mariadb':` branch)."
|
||||
)
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main(sys.argv[1:]))
|
||||
2
.github/helper/site_config_postgres.json
vendored
2
.github/helper/site_config_postgres.json
vendored
@@ -13,6 +13,6 @@
|
||||
"root_login": "postgres",
|
||||
"root_password": "travis",
|
||||
"host_name": "http://test_site:8000",
|
||||
"install_apps": ["payments", "erpnext"],
|
||||
"install_apps": ["erpnext"],
|
||||
"throttle_user_limit": 100
|
||||
}
|
||||
|
||||
79
.github/helper/start-db.sh
vendored
79
.github/helper/start-db.sh
vendored
@@ -1,79 +0,0 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Run MariaDB INSIDE the runner container, on a datadir we control. Because the datadir can be
|
||||
# packaged into the bench artifact, test shards start an already-loaded server instead of
|
||||
# replaying a SQL dump (the ~60s hydrate restore). Each shard gets its own copy → isolation kept.
|
||||
#
|
||||
# CI_DB_DATADIR picks the path:
|
||||
# - setup job: /home/ci/db-data (OUTSIDE the bench, so install.sh's `rm -rf ~/frappe-bench`
|
||||
# doesn't wipe it; it's moved into the bench just before packaging)
|
||||
# - test shard: ~/frappe-bench/mariadb-data (where the artifact untar'd it)
|
||||
#
|
||||
# Idempotent: inits a fresh datadir if absent (setup), else starts on the existing one (shards).
|
||||
#
|
||||
set -e
|
||||
|
||||
ci_user="${ERPNEXT_CI_USER:-frappe}"
|
||||
|
||||
# Re-exec as the ci user so mariadbd and the datadir are owned consistently (root mariadbd is
|
||||
# refused anyway). Mirrors install.sh's user switch.
|
||||
if [ "$(id -u)" = "0" ] && [ "${SKIP_SYSTEM_SETUP:-0}" = "1" ] && [ "$ci_user" != "root" ]; then
|
||||
exec su -m "$ci_user" -s /bin/bash -c \
|
||||
"ERPNEXT_CI_USER='$ci_user' CI_DB_DATADIR='${CI_DB_DATADIR:-}' DB='${DB:-}' bash '$0'"
|
||||
fi
|
||||
|
||||
# --- PostgreSQL (GitHub-hosted CI): run in-runner on a PGDATA so it bakes into the artifact,
|
||||
# same idea as the mariadb datadir. Trust auth (throwaway CI) skips password setup; durability
|
||||
# off for speed. Postgres is preinstalled on ubuntu-latest under /usr/lib/postgresql/<ver>/bin.
|
||||
if [ "${DB:-mariadb}" = "postgres" ]; then
|
||||
PG_BIN=$(ls -d /usr/lib/postgresql/*/bin 2>/dev/null | sort -V | tail -1)
|
||||
[ -n "$PG_BIN" ] && export PATH="$PG_BIN:$PATH"
|
||||
PGDATA="${CI_DB_DATADIR:-$HOME/frappe-bench/pgdata}"
|
||||
if [ ! -d "$PGDATA/base" ]; then
|
||||
initdb -D "$PGDATA" -U postgres --auth-local=trust --auth-host=trust >/dev/null
|
||||
echo "host all all 127.0.0.1/32 trust" >> "$PGDATA/pg_hba.conf"
|
||||
fi
|
||||
pg_ctl -D "$PGDATA" -w -o "-p 5432 -c listen_addresses=127.0.0.1 -c unix_socket_directories=$PGDATA -c fsync=off -c synchronous_commit=off -c full_page_writes=off" start
|
||||
echo "PostgreSQL up in-runner (pgdata=$PGDATA)"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# --- MariaDB ---
|
||||
DATADIR="${CI_DB_DATADIR:-$HOME/frappe-bench/mariadb-data}"
|
||||
SOCK="$DATADIR/mysqld.sock"
|
||||
fresh=0
|
||||
|
||||
if [ ! -d "$DATADIR/mysql" ]; then
|
||||
mkdir -p "$DATADIR"
|
||||
mariadb-install-db --no-defaults --datadir="$DATADIR" \
|
||||
--auth-root-authentication-method=normal --skip-test-db >/dev/null 2>&1
|
||||
fresh=1
|
||||
fi
|
||||
|
||||
# Throwaway-CI durability off; bind TCP 127.0.0.1:3306 so bench/install.sh connect as usual.
|
||||
mariadbd --no-defaults --datadir="$DATADIR" --socket="$SOCK" --pid-file="$DATADIR/mysqld.pid" \
|
||||
--port=3306 --bind-address=127.0.0.1 \
|
||||
--innodb-flush-log-at-trx-commit=0 --sync-binlog=0 --skip-log-bin \
|
||||
> "$HOME/mariadb.log" 2>&1 &
|
||||
|
||||
up=0
|
||||
for _ in $(seq 1 60); do
|
||||
if mariadb-admin --socket="$SOCK" ping --silent 2>/dev/null; then up=1; break; fi
|
||||
sleep 1
|
||||
done
|
||||
# Fail loudly instead of letting the loop fall through (exit 0 of the last `sleep`) into SQL that
|
||||
# would error with a vague socket-connection failure.
|
||||
[ "$up" = "1" ] || { echo "mariadbd did not come up on $SOCK"; cat "$HOME/mariadb.log" 2>/dev/null; exit 1; }
|
||||
|
||||
if [ "$fresh" = "1" ]; then
|
||||
# A fresh datadir has only a password-less root@localhost. Give it the password install.sh
|
||||
# uses, plus a TCP-reachable root@127.0.0.1, so the rest of install.sh works unchanged.
|
||||
mariadb --no-defaults --socket="$SOCK" -u root <<'SQL'
|
||||
ALTER USER 'root'@'localhost' IDENTIFIED BY 'root';
|
||||
CREATE USER IF NOT EXISTS 'root'@'127.0.0.1' IDENTIFIED BY 'root';
|
||||
GRANT ALL PRIVILEGES ON *.* TO 'root'@'127.0.0.1' WITH GRANT OPTION;
|
||||
FLUSH PRIVILEGES;
|
||||
SQL
|
||||
fi
|
||||
|
||||
echo "MariaDB up in-container (datadir=$DATADIR, fresh=$fresh)"
|
||||
@@ -1,51 +0,0 @@
|
||||
name: Download translations from Crowdin
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "0 4 * * 1"
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: crowdin-download
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
download-translations:
|
||||
name: Download translations into ${{ matrix.branch }}
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
branch: ["develop", "version-16-hotfix"]
|
||||
|
||||
steps:
|
||||
- name: Checkout ${{ matrix.branch }}
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ matrix.branch }}
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Download translations and open PR
|
||||
uses: crowdin/github-action@8f01d54f70f1713ee3f09d82c2bbb2daeac28689 # v2.17.1
|
||||
with:
|
||||
config: crowdin.yml
|
||||
upload_sources: false
|
||||
upload_translations: false
|
||||
download_translations: true
|
||||
crowdin_branch_name: "[frappe.erpnext] ${{ matrix.branch }}"
|
||||
skip_ref_checkout: true
|
||||
localization_branch_name: l10n_crowdin_${{ matrix.branch }}
|
||||
create_pull_request: true
|
||||
pull_request_base_branch_name: ${{ matrix.branch }}
|
||||
commit_message: "fix: sync translations from crowdin"
|
||||
pull_request_title: "fix: sync translations from crowdin (${{ matrix.branch }})"
|
||||
pull_request_labels: "translation, skip-release-notes"
|
||||
pull_request_reviewers: barredterra
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.RELEASE_TOKEN }}
|
||||
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
|
||||
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
|
||||
@@ -1,54 +0,0 @@
|
||||
name: Upload main.pot to Crowdin
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- develop
|
||||
- version-16-hotfix
|
||||
paths:
|
||||
- "erpnext/locale/main.pot"
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: crowdin-upload-${{ github.ref_name }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
upload-sources:
|
||||
name: Upload sources from ${{ github.ref_name }}
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout ${{ github.ref_name }}
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Restore Crowdin cache
|
||||
uses: actions/cache/restore@v6
|
||||
with:
|
||||
path: .crowdin
|
||||
key: crowdin-${{ github.ref_name }}
|
||||
restore-keys: crowdin-${{ github.ref_name }}-
|
||||
|
||||
- name: Upload main.pot to Crowdin
|
||||
uses: crowdin/github-action@8f01d54f70f1713ee3f09d82c2bbb2daeac28689 # v2.17.1
|
||||
with:
|
||||
config: crowdin.yml
|
||||
upload_sources: true
|
||||
upload_translations: false
|
||||
download_translations: false
|
||||
create_pull_request: false
|
||||
crowdin_branch_name: "[frappe.erpnext] ${{ github.ref_name }}"
|
||||
upload_sources_args: "--cache"
|
||||
env:
|
||||
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
|
||||
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
|
||||
|
||||
- name: Save Crowdin cache
|
||||
uses: actions/cache/save@v6
|
||||
if: always()
|
||||
with:
|
||||
path: .crowdin
|
||||
key: crowdin-${{ github.ref_name }}-${{ github.run_id }}
|
||||
2
.github/workflows/linters.yml
vendored
2
.github/workflows/linters.yml
vendored
@@ -21,7 +21,7 @@ jobs:
|
||||
cache: pip
|
||||
|
||||
- name: Install and Run Pre-commit
|
||||
uses: pre-commit/action@v3.0.1
|
||||
uses: pre-commit/action@v3.0.0
|
||||
|
||||
semgrep:
|
||||
name: semgrep
|
||||
|
||||
45
.github/workflows/notify-support-on-release.yml
vendored
45
.github/workflows/notify-support-on-release.yml
vendored
@@ -1,45 +0,0 @@
|
||||
name: Notify Support on PR release
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
notify-support:
|
||||
if: >-
|
||||
github.event.issue.pull_request
|
||||
&& github.event.comment.user.id == 28699486
|
||||
&& contains(github.event.comment.body, 'This PR is included in version')
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 2
|
||||
|
||||
steps:
|
||||
- name: Notify support.frappe.io
|
||||
env:
|
||||
COMMENT_ID: ${{ github.event.comment.id }}
|
||||
PR_NUMBER: ${{ github.event.issue.number }}
|
||||
REPOSITORY: ${{ github.repository }}
|
||||
SUPPORT_FRAPPE_AUTH: ${{ secrets.SUPPORT_FRAPPE_AUTH }}
|
||||
run: |
|
||||
payload="$(
|
||||
jq -n \
|
||||
--arg repository "$REPOSITORY" \
|
||||
--argjson pr_number "$PR_NUMBER" \
|
||||
--argjson comment_id "$COMMENT_ID" \
|
||||
'{
|
||||
repository: $repository,
|
||||
pr_number: $pr_number,
|
||||
comment_id: $comment_id
|
||||
}'
|
||||
)"
|
||||
|
||||
curl --fail-with-body \
|
||||
--retry 3 \
|
||||
--retry-all-errors \
|
||||
--request POST \
|
||||
--header "Authorization: $SUPPORT_FRAPPE_AUTH" \
|
||||
--header "Content-Type: application/json" \
|
||||
--data "$payload" \
|
||||
"https://support.frappe.io/api/method/notify_pr_release"
|
||||
60
.github/workflows/patch.yml
vendored
60
.github/workflows/patch.yml
vendored
@@ -65,22 +65,6 @@ jobs:
|
||||
- name: Add to Hosts
|
||||
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
|
||||
|
||||
# The v14 baseline backup is a fixed published file — cache it instead of re-downloading
|
||||
# it from the GitHub release every run.
|
||||
- name: Cache erpnext v14 backup
|
||||
id: cache-v14
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/erpnext-v14.sql.gz
|
||||
key: erpnext-v14-sql-gz
|
||||
|
||||
- name: Download erpnext v14 backup
|
||||
if: steps.cache-v14.outputs.cache-hit != 'true'
|
||||
run: |
|
||||
curl -fSL --retry 5 --retry-all-errors --retry-delay 5 \
|
||||
-o ~/erpnext-v14.sql.gz \
|
||||
https://github.com/frappe/erpnext/releases/download/v14-baseline/erpnext-v14.sql.gz
|
||||
|
||||
- name: Cache pip
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
@@ -121,8 +105,6 @@ jobs:
|
||||
env:
|
||||
DB: mariadb
|
||||
TYPE: server
|
||||
# Anonymous git to github.com gets throttled to a 401; authenticate the clones.
|
||||
CI_GITHUB_TOKEN: ${{ github.token }}
|
||||
|
||||
- name: Run Patch Tests
|
||||
run: |
|
||||
@@ -131,20 +113,12 @@ jobs:
|
||||
jq 'del(.install_apps)' ~/frappe-bench/sites/test_site/site_config.json > tmp.json
|
||||
mv tmp.json ~/frappe-bench/sites/test_site/site_config.json
|
||||
|
||||
bench --site test_site --force restore ~/erpnext-v14.sql.gz
|
||||
wget https://frappe.io/files/erpnext-v14.sql.gz
|
||||
bench --site test_site --force restore ~/frappe-bench/erpnext-v14.sql.gz
|
||||
|
||||
git -C "apps/frappe" remote set-url upstream https://github.com/frappe/frappe.git
|
||||
git -C "apps/erpnext" remote set-url upstream https://github.com/frappe/erpnext.git
|
||||
|
||||
# Start every bench process except the background workers. If workers run during a
|
||||
# migrate, they pick up the orphan-link cleanup jobs it enqueues and race its schema
|
||||
# changes, which fails with MySQL 1412 "Table definition has changed". Redis and the
|
||||
# other services stay up; the queued jobs simply wait and are harmless here.
|
||||
function start_bench_without_workers() {
|
||||
local procs
|
||||
procs=$(awk -F: '/^[a-z_]+:/ && $1 !~ /worker/ {print $1}' ~/frappe-bench/Procfile)
|
||||
honcho start -f ~/frappe-bench/Procfile $procs &>> ~/frappe-bench/bench_start.log &
|
||||
}
|
||||
|
||||
function update_to_version() {
|
||||
version=$1
|
||||
@@ -160,11 +134,10 @@ jobs:
|
||||
|
||||
# Resetup env and install apps
|
||||
pgrep honcho | xargs kill
|
||||
sleep 10
|
||||
rm -rf ~/frappe-bench/env
|
||||
bench -v setup env --python python$2
|
||||
bench pip install -e ./apps/erpnext
|
||||
start_bench_without_workers
|
||||
bench start &>> ~/frappe-bench/bench_start.log &
|
||||
|
||||
bench --site test_site migrate
|
||||
}
|
||||
@@ -173,30 +146,7 @@ jobs:
|
||||
update_to_version 16 3.14
|
||||
|
||||
echo "Updating to latest version"
|
||||
fallback_to_develop=0
|
||||
if [ -n "${GITHUB_BASE_REF:-}" ]; then
|
||||
frappe_ref="refs/heads/$GITHUB_BASE_REF"
|
||||
fallback_to_develop=1
|
||||
elif [ "${GITHUB_REF_TYPE:-}" = "branch" ]; then
|
||||
frappe_ref="$GITHUB_REF"
|
||||
fallback_to_develop=1
|
||||
elif [ "${GITHUB_REF_TYPE:-}" = "tag" ]; then
|
||||
frappe_ref="$GITHUB_REF"
|
||||
else
|
||||
echo "Unsupported GitHub ref type: '${GITHUB_REF_TYPE:-unset}'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
ls_remote_status=0
|
||||
git -C "apps/frappe" ls-remote --exit-code upstream "$frappe_ref" >/dev/null \
|
||||
|| ls_remote_status=$?
|
||||
if [ "$ls_remote_status" -eq 2 ] && [ "$fallback_to_develop" -eq 1 ]; then
|
||||
echo "frappe has no '$frappe_ref'; falling back to develop"
|
||||
frappe_ref=refs/heads/develop
|
||||
elif [ "$ls_remote_status" -ne 0 ]; then
|
||||
exit "$ls_remote_status"
|
||||
fi
|
||||
git -C "apps/frappe" fetch --depth 1 upstream "$frappe_ref"
|
||||
git -C "apps/frappe" fetch --depth 1 upstream "${GITHUB_BASE_REF:-${GITHUB_REF##*/}}"
|
||||
git -C "apps/frappe" checkout -q -f FETCH_HEAD
|
||||
git -C "apps/erpnext" checkout -q -f "$GITHUB_SHA"
|
||||
|
||||
@@ -204,7 +154,7 @@ jobs:
|
||||
rm -rf ~/frappe-bench/env
|
||||
bench -v setup env
|
||||
bench pip install -e ./apps/erpnext
|
||||
start_bench_without_workers
|
||||
bench start &>> ~/frappe-bench/bench_start.log &
|
||||
|
||||
bench --site test_site migrate
|
||||
|
||||
|
||||
@@ -22,6 +22,4 @@ jobs:
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- uses: alyf-de/po-review-action@5928f84d6bc9094f9ad6e2c5780f01c0044b800e # v1.1.1
|
||||
with:
|
||||
hidden-po-files: eo.po
|
||||
- uses: alyf-de/po-review-action@v1.0.0
|
||||
|
||||
2
.github/workflows/run-individual-tests.yml
vendored
2
.github/workflows/run-individual-tests.yml
vendored
@@ -129,8 +129,6 @@ jobs:
|
||||
TYPE: server
|
||||
FRAPPE_USER: ${{ github.event.inputs.user }}
|
||||
FRAPPE_BRANCH: ${{ github.event.inputs.branch }}
|
||||
# Anonymous git to github.com gets throttled to a 401; authenticate the clones.
|
||||
CI_GITHUB_TOKEN: ${{ github.token }}
|
||||
|
||||
- name: Run Tests
|
||||
run: |
|
||||
|
||||
@@ -13,7 +13,6 @@ on:
|
||||
- 'crowdin.yml'
|
||||
- '.coderabbit.yml'
|
||||
- '.mergify.yml'
|
||||
- '**.po'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
207
.github/workflows/server-tests-mariadb.yml
vendored
207
.github/workflows/server-tests-mariadb.yml
vendored
@@ -13,7 +13,6 @@ on:
|
||||
- 'crowdin.yml'
|
||||
- '.coderabbit.yml'
|
||||
- '.mergify.yml'
|
||||
- '**.po'
|
||||
schedule:
|
||||
# Run everday at midnight UTC / 5:30 IST
|
||||
- cron: "0 0 * * *"
|
||||
@@ -32,49 +31,47 @@ on:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
packages: read
|
||||
|
||||
concurrency:
|
||||
group: server-mariadb-develop-${{ github.event_name }}-${{ github.event.number || github.event_name == 'workflow_dispatch' && github.run_id || '' }}
|
||||
cancel-in-progress: true
|
||||
|
||||
# Shared across both jobs. Both run in the SAME CI image so the bench lives at the identical
|
||||
# path (/home/ci/frappe-bench) on the setup runner and the test shards — that's what makes the
|
||||
# packaged Python venv portable between them.
|
||||
env:
|
||||
TZ: 'Asia/Kolkata'
|
||||
DEBIAN_FRONTEND: noninteractive
|
||||
NODE_ENV: "production"
|
||||
WITH_COVERAGE: ${{ github.event_name != 'pull_request' }}
|
||||
ERPNEXT_CI_USER: ci
|
||||
PIP_CACHE_DIR: /home/ci/.cache/pip
|
||||
npm_config_cache: /home/ci/.cache/npm
|
||||
YARN_CACHE_FOLDER: /home/ci/.cache/yarn
|
||||
UV_CACHE_DIR: /home/ci/.cache/uv
|
||||
|
||||
jobs:
|
||||
# Build the bench (clone + pip + yarn + assets) and reinstall test_site ONCE, on a free
|
||||
# GitHub-hosted runner, then publish the whole bench (with a DB dump baked in) as an artifact.
|
||||
# The expensive, non-parallelisable work happens here exactly once instead of on every shard.
|
||||
setup:
|
||||
name: Build & reinstall (setup)
|
||||
# Dedicated scale set (fat cpu request) so the build+reinstall runs at full speed, uncontended
|
||||
# by the many thin test shards. Same CI image + /home/ci path + 127.0.0.1 DB as the shards,
|
||||
# so the packaged bench (and its venv) transplants cleanly.
|
||||
runs-on: erpnext-arc-setup
|
||||
timeout-minutes: 40
|
||||
container:
|
||||
image: ghcr.io/frappe/erpnext-ci-mariadb:py3.14-node24
|
||||
credentials:
|
||||
username: ${{ secrets.GHCR_USERNAME || github.actor }}
|
||||
password: ${{ secrets.GHCR_TOKEN || github.token }}
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
TZ: 'Asia/Kolkata'
|
||||
NODE_ENV: "production"
|
||||
WITH_COVERAGE: ${{ github.event_name != 'pull_request' }}
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
|
||||
matrix:
|
||||
container: [1, 2, 3, 4]
|
||||
|
||||
name: Python Unit Tests
|
||||
|
||||
services:
|
||||
mysql:
|
||||
image: mariadb:10.6
|
||||
env:
|
||||
TZ: 'Asia/Kolkata'
|
||||
MARIADB_ROOT_PASSWORD: 'root'
|
||||
ports:
|
||||
- 3306:3306
|
||||
options: --health-cmd="mariadb-admin ping" --health-interval=5s --health-timeout=2s --health-retries=3
|
||||
|
||||
steps:
|
||||
- name: Clone
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Setup Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3.14'
|
||||
|
||||
- name: Check for valid Python & Merge Conflicts
|
||||
run: |
|
||||
python -m compileall -fq "${GITHUB_WORKSPACE}"
|
||||
@@ -83,17 +80,47 @@ jobs:
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Setup Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 24
|
||||
check-latest: true
|
||||
|
||||
- name: Add to Hosts
|
||||
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
|
||||
|
||||
# MariaDB runs in-container on a datadir OUTSIDE the bench, because install.sh's next step
|
||||
# does `rm -rf ~/frappe-bench`. After the reinstall, the datadir is moved into the bench so
|
||||
# it ships in the artifact — test shards then start an already-loaded server (no restore).
|
||||
- name: Start DB
|
||||
run: bash ${GITHUB_WORKSPACE}/.github/helper/start-db.sh
|
||||
- name: Cache pip
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cache/pip
|
||||
key: ${{ runner.os }}-pip-${{ hashFiles('**/*requirements.txt', '**/pyproject.toml') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pip-
|
||||
${{ runner.os }}-
|
||||
|
||||
- name: Cache node modules
|
||||
uses: actions/cache@v4
|
||||
env:
|
||||
SKIP_SYSTEM_SETUP: "1"
|
||||
CI_DB_DATADIR: /home/ci/db-data
|
||||
cache-name: cache-node-modules
|
||||
with:
|
||||
path: ~/.npm
|
||||
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-build-${{ env.cache-name }}-
|
||||
${{ runner.os }}-build-
|
||||
${{ runner.os }}-
|
||||
|
||||
- name: Get yarn cache directory path
|
||||
id: yarn-cache-dir-path
|
||||
run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
|
||||
|
||||
- uses: actions/cache@v4
|
||||
id: yarn-cache
|
||||
with:
|
||||
path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
|
||||
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-yarn-
|
||||
|
||||
- name: Install
|
||||
run: bash ${GITHUB_WORKSPACE}/.github/helper/install.sh
|
||||
@@ -102,116 +129,26 @@ jobs:
|
||||
TYPE: server
|
||||
FRAPPE_USER: ${{ github.event.inputs.user }}
|
||||
FRAPPE_BRANCH: ${{ github.event.client_payload.sha || github.event.inputs.branch }}
|
||||
# Anonymous git to github.com gets throttled to a 401; authenticate the clones.
|
||||
CI_GITHUB_TOKEN: ${{ github.token }}
|
||||
DB_HOST: 127.0.0.1
|
||||
DB_USER_HOST: '%'
|
||||
WKHTMLTOX_DEB: /tmp/wkhtmltox.deb
|
||||
SKIP_SYSTEM_SETUP: "1"
|
||||
SKIP_WKHTMLTOX_SETUP: "1"
|
||||
|
||||
- name: Warm up test data
|
||||
run: |
|
||||
su -m "${ERPNEXT_CI_USER:-frappe}" -s /bin/bash <<'EOF'
|
||||
cd ~/frappe-bench/
|
||||
bench --site test_site run-tests --lightmode --module erpnext.tests.bootstrap_test_data
|
||||
EOF
|
||||
|
||||
# Clean shutdown (consistent InnoDB datadir), then stage it inside the bench for packaging.
|
||||
- name: Stop DB and stage datadir
|
||||
run: |
|
||||
mariadb-admin -h 127.0.0.1 -P 3306 -u root -proot shutdown || true
|
||||
for _ in $(seq 1 30); do [ -f /home/ci/db-data/mysqld.pid ] || break; sleep 1; done
|
||||
# Don't bake a dirty datadir — fail if mariadbd didn't finish stopping, rather than ship
|
||||
# an inconsistent datadir the shards would have to crash-recover.
|
||||
[ -f /home/ci/db-data/mysqld.pid ] && { echo "mariadbd did not shut down cleanly"; exit 1; }
|
||||
mv /home/ci/db-data /home/ci/frappe-bench/mariadb-data
|
||||
|
||||
# Package the whole bench (apps, venv, node_modules, sites, the DB dump, and hydrate.sh)
|
||||
# into one artifact for the test shards to consume.
|
||||
# Single-node hand-off: stage the bench on a node-local hostPath instead of round-tripping
|
||||
# through GitHub artifact storage (~60s/shard). Setup and shards share the same disk, so
|
||||
# the shards just untar it locally. NOTE: this assumes one node — a shard on a different
|
||||
# node could not read this path (then you'd need GitHub artifacts or an NFS/RWX volume).
|
||||
- name: Stage bench on node (hostPath)
|
||||
run: |
|
||||
cp "${GITHUB_WORKSPACE}/.github/helper/hydrate.sh" /home/ci/frappe-bench/hydrate.sh
|
||||
cp "${GITHUB_WORKSPACE}/.github/helper/start-db.sh" /home/ci/frappe-bench/start-db.sh
|
||||
mkdir -p /opt/ci-bench-staging
|
||||
# self-clean: drop bench tars from runs older than 2h
|
||||
find /opt/ci-bench-staging -maxdepth 1 -name '*.tar.gz' -mmin +120 -delete 2>/dev/null || true
|
||||
# Exclude .git/node_modules; the mariadb-data datadir IS included (the pre-loaded DB).
|
||||
tar czpf "/opt/ci-bench-staging/${GITHUB_RUN_ID}.tar.gz" -C /home/ci \
|
||||
--exclude='.git' --exclude='node_modules' frappe-bench
|
||||
ls -lh "/opt/ci-bench-staging/${GITHUB_RUN_ID}.tar.gz"
|
||||
|
||||
# Fan-out: each shard downloads the bench, untars it, starts MariaDB on the baked datadir, and
|
||||
# runs its slice of the suite. No clone, no build, no reinstall, no DB dump restore on the shards.
|
||||
test:
|
||||
name: Python Unit Tests
|
||||
needs: setup
|
||||
runs-on: erpnext-arc
|
||||
timeout-minutes: 60
|
||||
container:
|
||||
image: ghcr.io/frappe/erpnext-ci-mariadb:py3.14-node24
|
||||
credentials:
|
||||
username: ${{ secrets.GHCR_USERNAME || github.actor }}
|
||||
password: ${{ secrets.GHCR_TOKEN || github.token }}
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
container: [1, 2, 3, 4]
|
||||
|
||||
steps:
|
||||
- name: Add to Hosts
|
||||
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
|
||||
|
||||
# Read the bench straight from the node-local hostPath the setup job staged it on — no
|
||||
# GitHub download. -p preserves the ci (uid 1001) ownership so bench runs as ci cleanly.
|
||||
- name: Untar bench from node (hostPath)
|
||||
run: |
|
||||
tar xzpf "/opt/ci-bench-staging/${GITHUB_RUN_ID}.tar.gz" -C /home/ci
|
||||
ls -ld /home/ci/frappe-bench
|
||||
|
||||
- name: Hydrate (start DB on baked datadir + bench start)
|
||||
run: bash /home/ci/frappe-bench/hydrate.sh
|
||||
env:
|
||||
DB_HOST: 127.0.0.1
|
||||
SKIP_SYSTEM_SETUP: "1"
|
||||
|
||||
- name: Run Tests
|
||||
run: |
|
||||
su -m "${ERPNEXT_CI_USER:-frappe}" -s /bin/bash <<'EOF'
|
||||
cd ~/frappe-bench/
|
||||
coverage_flag=""
|
||||
if [ "$WITH_COVERAGE" = "true" ]; then coverage_flag="--with-coverage"; fi
|
||||
bench --site test_site run-parallel-tests --lightmode --app erpnext \
|
||||
--total-builds ${{ strategy.job-total }} \
|
||||
--build-number ${{ matrix.container }} \
|
||||
$coverage_flag
|
||||
EOF
|
||||
run: 'cd ~/frappe-bench/ && bench --site test_site run-parallel-tests --lightmode --app erpnext --total-builds ${{ strategy.job-total }} --build-number ${{ matrix.container }} --with-coverage'
|
||||
env:
|
||||
TYPE: server
|
||||
|
||||
|
||||
- name: Show bench output
|
||||
if: ${{ always() }}
|
||||
run: cat ~/frappe-bench/bench_start.log || true
|
||||
|
||||
- name: Upload coverage data
|
||||
if: ${{ env.WITH_COVERAGE == 'true' }}
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: coverage-${{ matrix.container }}
|
||||
path: /home/ci/frappe-bench/sites/coverage.xml
|
||||
path: /home/runner/frappe-bench/sites/coverage.xml
|
||||
|
||||
coverage:
|
||||
name: Coverage Wrap Up
|
||||
needs: [test]
|
||||
if: ${{ github.event_name != 'pull_request' }}
|
||||
needs: test
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Clone
|
||||
|
||||
196
.github/workflows/server-tests-postgres.yml
vendored
196
.github/workflows/server-tests-postgres.yml
vendored
@@ -1,12 +1,7 @@
|
||||
name: Server (Postgres)
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# 03:00 AM IST daily (21:30 UTC the previous day)
|
||||
- cron: "30 21 * * *"
|
||||
pull_request:
|
||||
# 'labeled' so adding the 'postgres' label to an already-open PR re-triggers the run.
|
||||
types: [opened, reopened, synchronize, labeled]
|
||||
paths-ignore:
|
||||
- '**.js'
|
||||
- '**.md'
|
||||
@@ -14,7 +9,7 @@ on:
|
||||
- 'crowdin.yml'
|
||||
- '.coderabbit.yml'
|
||||
- '.mergify.yml'
|
||||
workflow_dispatch:
|
||||
types: [opened, labelled, synchronize, reopened]
|
||||
|
||||
concurrency:
|
||||
group: server-postgres-develop-${{ github.event_name }}-${{ github.event.number || github.event_name == 'workflow_dispatch' && github.run_id || '' }}
|
||||
@@ -23,31 +18,41 @@ concurrency:
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
# Postgres CI stays on GitHub-hosted (free, full-speed VM per shard) but follows the same fan-out
|
||||
# we built for MariaDB: build the bench + reinstall ONCE in the setup job, bake the PostgreSQL
|
||||
# PGDATA into the artifact, and have 4 test shards start Postgres on that datadir — no per-shard
|
||||
# clone/build/reinstall/restore. Python is pinned so the venv transplants between VMs.
|
||||
env:
|
||||
TZ: 'Asia/Kolkata'
|
||||
NODE_ENV: "production"
|
||||
PYTHON_VERSION: '3.14'
|
||||
|
||||
jobs:
|
||||
setup:
|
||||
name: Build & reinstall (setup)
|
||||
test:
|
||||
if: ${{ contains(github.event.pull_request.labels.*.name, 'postgres') }}
|
||||
runs-on: ubuntu-latest
|
||||
# Runs on the daily schedule (and workflow_dispatch). On PRs it runs ONLY when the PR carries
|
||||
# the 'postgres' label — the test job needs setup, so it's skipped too when this is.
|
||||
if: github.event_name != 'pull_request' || contains(github.event.pull_request.labels.*.name, 'postgres')
|
||||
timeout-minutes: 40
|
||||
timeout-minutes: 60
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
container: [1]
|
||||
|
||||
name: Python Unit Tests
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:13.3
|
||||
env:
|
||||
POSTGRES_PASSWORD: travis
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
ports:
|
||||
- 5432:5432
|
||||
|
||||
steps:
|
||||
|
||||
- name: Clone
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Setup Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
python-version: '3.14'
|
||||
|
||||
- name: Check for valid Python & Merge Conflicts
|
||||
run: |
|
||||
@@ -66,135 +71,48 @@ jobs:
|
||||
- name: Add to Hosts
|
||||
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
|
||||
|
||||
- name: Cache deps (uv/pip/npm/yarn)
|
||||
- name: Cache pip
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cache/uv
|
||||
~/.cache/pip
|
||||
~/.npm
|
||||
~/.cache/yarn
|
||||
key: ${{ runner.os }}-deps-${{ hashFiles('**/*requirements.txt', '**/pyproject.toml', '**/yarn.lock') }}
|
||||
restore-keys: ${{ runner.os }}-deps-
|
||||
path: ~/.cache/pip
|
||||
key: ${{ runner.os }}-pip-${{ hashFiles('**/*requirements.txt', '**/pyproject.toml') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pip-
|
||||
${{ runner.os }}-
|
||||
|
||||
# Warm-bench cache (the big one): install.sh saves the built base bench — frappe + env +
|
||||
# node_modules + assets — here as frappe-bench-base-*.tar.zst. Later runs restore it and only
|
||||
# fast-forward to the live develop SHA + rebuild the delta, so the bench BUILD is near-free and
|
||||
# only the test_site reinstall (per-run DB, uncacheable) stays slow — matching the self-hosted
|
||||
# box. The first run after a deps change populates it; every run after that is fast.
|
||||
- name: Cache warm bench (base build)
|
||||
- name: Cache node modules
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/bench-cache
|
||||
key: ${{ runner.os }}-warmbench-v2-${{ hashFiles('**/*requirements.txt', '**/pyproject.toml', '**/yarn.lock') }}
|
||||
restore-keys: ${{ runner.os }}-warmbench-v2-
|
||||
|
||||
# Postgres runs in-runner on a PGDATA OUTSIDE the bench (install.sh wipes ~/frappe-bench);
|
||||
# after the reinstall it's moved into the bench so it ships in the artifact.
|
||||
- name: Start DB
|
||||
run: bash ${GITHUB_WORKSPACE}/.github/helper/start-db.sh
|
||||
env:
|
||||
DB: postgres
|
||||
CI_DB_DATADIR: /home/runner/pgdata
|
||||
cache-name: cache-node-modules
|
||||
with:
|
||||
path: ~/.npm
|
||||
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-build-${{ env.cache-name }}-
|
||||
${{ runner.os }}-build-
|
||||
${{ runner.os }}-
|
||||
|
||||
- name: Get yarn cache directory path
|
||||
id: yarn-cache-dir-path
|
||||
run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
|
||||
|
||||
- uses: actions/cache@v4
|
||||
id: yarn-cache
|
||||
with:
|
||||
path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
|
||||
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-yarn-
|
||||
|
||||
- name: Install
|
||||
run: bash ${GITHUB_WORKSPACE}/.github/helper/install.sh
|
||||
env:
|
||||
DB: postgres
|
||||
TYPE: server
|
||||
FRAPPE_BRANCH: develop
|
||||
# Anonymous git to github.com gets throttled to a 401; authenticate the clones.
|
||||
CI_GITHUB_TOKEN: ${{ github.token }}
|
||||
BENCH_CACHE_DIR: /home/runner/bench-cache
|
||||
|
||||
- name: Warm up test data
|
||||
run: |
|
||||
cd ~/frappe-bench/
|
||||
bench --site test_site run-tests --lightmode --module erpnext.tests.bootstrap_test_data
|
||||
|
||||
- name: Stop DB and stage datadir
|
||||
run: |
|
||||
PG_BIN=$(ls -d /usr/lib/postgresql/*/bin | sort -V | tail -1)
|
||||
# Clean shutdown so the baked datadir is consistent. Do NOT swallow a failed stop with
|
||||
# `|| true`: moving and tarring a still-running cluster ships a torn datadir the shards
|
||||
# cannot crash-recover (full_page_writes is off). Fail the job instead — mirrors the
|
||||
# MariaDB sister's "don't bake a dirty datadir" guard.
|
||||
"$PG_BIN/pg_ctl" -D /home/runner/pgdata -m fast -w stop
|
||||
mv /home/runner/pgdata /home/runner/frappe-bench/pgdata
|
||||
|
||||
- name: Package bench for test shards
|
||||
run: |
|
||||
cp "${GITHUB_WORKSPACE}/.github/helper/hydrate.sh" /home/runner/frappe-bench/hydrate.sh
|
||||
cp "${GITHUB_WORKSPACE}/.github/helper/start-db.sh" /home/runner/frappe-bench/start-db.sh
|
||||
tar czpf "${GITHUB_WORKSPACE}/bench.tar.gz" -C /home/runner \
|
||||
--exclude='.git' --exclude='node_modules' frappe-bench
|
||||
ls -lh "${GITHUB_WORKSPACE}/bench.tar.gz"
|
||||
|
||||
- name: Upload bench artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: bench-pg
|
||||
path: bench.tar.gz
|
||||
retention-days: 1
|
||||
compression-level: 0
|
||||
|
||||
test:
|
||||
name: Python Unit Tests
|
||||
needs: setup
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 60
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
container: [1, 2, 3, 4]
|
||||
steps:
|
||||
- name: Download bench artifact
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: bench-pg
|
||||
|
||||
- name: Setup Python
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
|
||||
- name: Add to Hosts
|
||||
run: echo "127.0.0.1 test_site" | sudo tee -a /etc/hosts
|
||||
|
||||
# The bench CLI (frappe-bench) and redis are global/system tools — not in the bench tarball.
|
||||
# The setup runner got them via install.sh; the MariaDB shards get them from the arc5 image.
|
||||
# GitHub-hosted PG shards install them here (cheap vs the build+reinstall that setup did once).
|
||||
- name: Install shard runtime (bench CLI + redis + wkhtmltopdf)
|
||||
run: |
|
||||
pip install frappe-bench
|
||||
command -v redis-server >/dev/null || { sudo apt-get update -qq && sudo apt-get install -y -qq redis-server; }
|
||||
# wkhtmltopdf (patched-qt build) for print-format / PDF tests — same .deb install.sh uses.
|
||||
if ! command -v wkhtmltopdf >/dev/null; then
|
||||
wget -qO /tmp/wkhtmltox.deb https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6.1-2/wkhtmltox_0.12.6.1-2.jammy_amd64.deb
|
||||
sudo apt-get install -y -qq /tmp/wkhtmltox.deb
|
||||
fi
|
||||
|
||||
- name: Untar bench
|
||||
run: |
|
||||
tar xzpf "${GITHUB_WORKSPACE}/bench.tar.gz" -C /home/runner
|
||||
ls -ld /home/runner/frappe-bench
|
||||
|
||||
- name: Hydrate (start Postgres on the baked datadir)
|
||||
run: bash /home/runner/frappe-bench/hydrate.sh
|
||||
env:
|
||||
DB: postgres
|
||||
DB_HOST: 127.0.0.1
|
||||
|
||||
- name: Run Tests
|
||||
run: |
|
||||
cd ~/frappe-bench/
|
||||
# print-format / PDF tests are engine-independent (they exercise wkhtmltopdf rendering,
|
||||
# not postgres SQL — the MariaDB CI already covers them). They only fetch the static asset
|
||||
# bundles from http://test_site:8000/assets/..., so a plain static file server over sites/
|
||||
# satisfies wkhtmltopdf without the frappe web server (which never bound on a bare runner).
|
||||
( cd ~/frappe-bench/sites && nohup python3 -m http.server 8000 --bind 127.0.0.1 > ~/frappe-bench/web.log 2>&1 & )
|
||||
for _ in $(seq 1 15); do (exec 3<>/dev/tcp/127.0.0.1/8000) 2>/dev/null && { exec 3>&- 3<&-; break; }; sleep 1; done
|
||||
bench --site test_site run-parallel-tests --lightmode --app erpnext \
|
||||
--total-builds ${{ strategy.job-total }} --build-number ${{ matrix.container }}
|
||||
run: cd ~/frappe-bench/ && bench --site test_site run-parallel-tests --app erpnext --use-orchestrator
|
||||
env:
|
||||
TYPE: server
|
||||
CI_BUILD_ID: ${{ github.run_id }}
|
||||
ORCHESTRATOR_URL: http://test-orchestrator.frappe.io
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -88,8 +88,7 @@ pull_request_rules:
|
||||
actions:
|
||||
merge:
|
||||
method: squash
|
||||
commit_message_format:
|
||||
title: pr-title
|
||||
body: pr-body
|
||||
merge_queue:
|
||||
queue_controls_comment: false
|
||||
commit_message_template: |
|
||||
{{ title }} (#{{ number }})
|
||||
|
||||
{{ body }}
|
||||
|
||||
@@ -48,6 +48,7 @@ repos:
|
||||
cypress/.*|
|
||||
.*node_modules.*|
|
||||
.*boilerplate.*|
|
||||
erpnext/public/js/controllers/.*|
|
||||
erpnext/templates/pages/order.js|
|
||||
erpnext/templates/includes/.*
|
||||
)$
|
||||
@@ -65,18 +66,6 @@ repos:
|
||||
- id: ruff-format
|
||||
name: "Run ruff formatter"
|
||||
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: postgres-compat
|
||||
name: "PostgreSQL compatibility (static check)"
|
||||
description: "Flags MySQL-only SQL that breaks on Postgres; the label-gated PG test job is the backstop for semantic divergences."
|
||||
entry: .github/helper/postgres_compat.py
|
||||
language: script
|
||||
files: ^erpnext/.*\.py$
|
||||
# patches/ are historical, version-gated migrations (skipped on fresh Postgres installs);
|
||||
# out of scope for the always-on gate.
|
||||
exclude: ^erpnext/patches/
|
||||
|
||||
ci:
|
||||
autoupdate_schedule: weekly
|
||||
skip: []
|
||||
|
||||
@@ -7,7 +7,6 @@ erpnext/accounts/ @ruthra-kumar
|
||||
erpnext/assets/ @khushi8112
|
||||
erpnext/regional @ruthra-kumar
|
||||
erpnext/selling @ruthra-kumar
|
||||
banking/ @nikkothari22
|
||||
|
||||
erpnext/buying/ @rohitwaghchaure @mihir-kandoi
|
||||
erpnext/maintenance/ @rohitwaghchaure @mihir-kandoi
|
||||
|
||||
@@ -14,52 +14,52 @@
|
||||
"@dnd-kit/core": "^6.3.1",
|
||||
"@dnd-kit/sortable": "^10.0.0",
|
||||
"@dnd-kit/utilities": "^3.2.2",
|
||||
"@tailwindcss/vite": "^4.3.2",
|
||||
"@tailwindcss/vite": "^4.3.0",
|
||||
"@tanstack/react-table": "^8.21.3",
|
||||
"@tanstack/react-virtual": "^3.13.24",
|
||||
"@vitejs/plugin-react": "^6.0.3",
|
||||
"@vitejs/plugin-react": "^6.0.1",
|
||||
"chrono-node": "^2.9.1",
|
||||
"class-variance-authority": "^0.7.1",
|
||||
"clsx": "^2.1.1",
|
||||
"cmdk": "^1.1.1",
|
||||
"date-fns": "^4.1.0",
|
||||
"dayjs": "^1.11.20",
|
||||
"frappe-react-sdk": "^1.17.1",
|
||||
"frappe-react-sdk": "^1.15.0",
|
||||
"fuse.js": "^7.3.0",
|
||||
"jotai": "^2.20.2",
|
||||
"jotai-family": "^1.1.0",
|
||||
"jotai": "^2.20.0",
|
||||
"jotai-family": "^1.0.1",
|
||||
"lodash.isplainobject": "^4.0.6",
|
||||
"lucide-react": "^1.14.0",
|
||||
"radix-ui": "^1.6.1",
|
||||
"react": "^19.2.7",
|
||||
"radix-ui": "^1.4.3",
|
||||
"react": "^19.2.6",
|
||||
"react-currency-input-field": "^4.0.5",
|
||||
"react-day-picker": "9.14.0",
|
||||
"react-dom": "^19.2.7",
|
||||
"react-dom": "^19.2.6",
|
||||
"react-dropzone": "^15.0.0",
|
||||
"react-hook-form": "^7.75.0",
|
||||
"react-hotkeys-hook": "^5.3.2",
|
||||
"react-markdown": "^10.1.0",
|
||||
"react-router": "^8.3.0",
|
||||
"react-router": "^7.15.0",
|
||||
"react-router-dom": "^7.15.0",
|
||||
"rehype-raw": "^7.0.0",
|
||||
"remark-gfm": "^4.0.1",
|
||||
"safe-expr-eval": "^1.0.4",
|
||||
"sonner": "^2.0.7",
|
||||
"tailwind-merge": "^3.5.0",
|
||||
"tailwindcss": "^4.3.0",
|
||||
"tw-animate-css": "^1.4.0",
|
||||
"usehooks-ts": "^3.1.1",
|
||||
"vite": "^8.2.1"
|
||||
"vite": "^8.0.11"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@eslint/js": "^9.39.5",
|
||||
"@eslint/js": "^9.39.1",
|
||||
"@types/node": "^25.3.0",
|
||||
"@types/react": "^19.2.7",
|
||||
"@types/react-dom": "^19.2.3",
|
||||
"eslint": "^10.8.1",
|
||||
"eslint": "^9.39.1",
|
||||
"eslint-plugin-react-hooks": "^7.1.1",
|
||||
"eslint-plugin-react-refresh": "^0.5.3",
|
||||
"eslint-plugin-react-refresh": "^0.4.24",
|
||||
"globals": "^16.5.0",
|
||||
"typescript": "~5.9.3",
|
||||
"typescript-eslint": "^8.67.0"
|
||||
"typescript-eslint": "^8.48.0"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { lazy, useEffect } from 'react'
|
||||
import { BrowserRouter, Navigate, Route, Routes } from 'react-router'
|
||||
import { BrowserRouter, Navigate, Route, Routes } from 'react-router-dom'
|
||||
import { FrappeProvider } from 'frappe-react-sdk'
|
||||
import { Toaster } from '@/components/ui/sonner'
|
||||
import BankReconciliation from '@/pages/BankReconciliation'
|
||||
|
||||
@@ -9,7 +9,6 @@ import Fuse from "fuse.js"
|
||||
import { ChevronDownIcon } from "lucide-react"
|
||||
import { useLayoutEffect, useMemo, useRef, useState } from "react"
|
||||
import { FormControl } from "../ui/form"
|
||||
import useResetScrollOnSearch from "@/hooks/useResetScrollOnSearch"
|
||||
|
||||
|
||||
export interface AccountsDropdownProps {
|
||||
@@ -105,10 +104,6 @@ const AccountsDropdown = ({ root_type, report_type, account_type, value, onChang
|
||||
|
||||
const buttonRef = useRef<HTMLButtonElement>(null)
|
||||
|
||||
// Searching replaces the grouped list with a short result list, so pin the scroll back to
|
||||
// the top - otherwise the auto-selected first result can be out of view.
|
||||
const listRef = useResetScrollOnSearch(search)
|
||||
|
||||
const [width, setWidth] = useState(320)
|
||||
|
||||
useLayoutEffect(() => {
|
||||
@@ -158,7 +153,7 @@ const AccountsDropdown = ({ root_type, report_type, account_type, value, onChang
|
||||
<PopoverContent className="p-0" style={{ minWidth: width }} align="start">
|
||||
<Command shouldFilter={false} className="w-full">
|
||||
<CommandInput placeholder={_("Search account...")} onValueChange={setSearch} value={search} />
|
||||
<CommandList ref={listRef}>
|
||||
<CommandList>
|
||||
<CommandEmpty>{_("No accounts found.")}</CommandEmpty>
|
||||
|
||||
{recommendedAccounts.length > 0 && (
|
||||
|
||||
@@ -10,7 +10,6 @@ import { ChevronDownIcon, ExternalLink } from "lucide-react";
|
||||
import { Button } from "../ui/button";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { Command, CommandEmpty, CommandGroup, CommandInput, CommandItem, CommandList } from "../ui/command";
|
||||
import useResetScrollOnSearch from "@/hooks/useResetScrollOnSearch";
|
||||
import _ from "@/lib/translate";
|
||||
import ErrorBanner from "../ui/error-banner";
|
||||
import MarkdownRenderer from "../ui/markdown";
|
||||
@@ -150,10 +149,6 @@ const LinkFieldCombobox = ({
|
||||
|
||||
const buttonRef = useRef<HTMLButtonElement>(null)
|
||||
|
||||
// Results change as the search runs, so pin the scroll back to the top to keep the
|
||||
// auto-selected first result in view.
|
||||
const listRef = useResetScrollOnSearch(searchInput)
|
||||
|
||||
const [width, setWidth] = useState(320)
|
||||
|
||||
useLayoutEffect(() => {
|
||||
@@ -269,7 +264,7 @@ const LinkFieldCombobox = ({
|
||||
{error && <ErrorBanner error={error} />}
|
||||
<Command shouldFilter={false} className="w-full">
|
||||
<CommandInput placeholder={placeholder} onValueChange={setSearchInput} />
|
||||
<CommandList ref={listRef}>
|
||||
<CommandList>
|
||||
<CommandEmpty>{isLoading ? _("Loading...") : _("No results found.")}</CommandEmpty>
|
||||
<CommandGroup>
|
||||
{items?.map((result) => (
|
||||
@@ -277,7 +272,7 @@ const LinkFieldCombobox = ({
|
||||
<span className="font-medium">
|
||||
{result.label || result.value}
|
||||
</span>
|
||||
{result.description && <span className="text-p-xs text-ink-gray-5">
|
||||
{result.description && <span className="text-xs text-ink-gray-5">
|
||||
<MarkdownRenderer content={result.description} />
|
||||
</span>}
|
||||
</CommandItem>
|
||||
|
||||
@@ -6,13 +6,13 @@ import { Progress } from "@/components/ui/progress"
|
||||
import { useGetAccountClosingBalance, useGetAccountClosingBalanceAsPerStatement, useGetAccountOpeningBalance, useGetUnreconciledTransactions } from "./utils"
|
||||
import { flt, formatCurrency } from "@/lib/numbers"
|
||||
import { Skeleton } from "@/components/ui/skeleton"
|
||||
import { StatContainer, StatLabel, StatValue } from "@/components/ui/stats"
|
||||
import { Edit, Info, Trash2 } from "lucide-react"
|
||||
import { H4, Paragraph } from "@/components/ui/typography"
|
||||
import { HoverCard, HoverCardContent, HoverCardTrigger } from "@/components/ui/hover-card"
|
||||
import { getCompanyCurrency } from "@/lib/company"
|
||||
import _ from "@/lib/translate"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { Dialog, DialogClose, DialogContent, DialogDescription, DialogFooter, DialogHeader, DialogTitle } from "@/components/ui/dialog"
|
||||
import { Dialog, DialogClose, DialogContent, DialogDescription, DialogFooter, DialogHeader, DialogTitle, DialogTrigger } from "@/components/ui/dialog"
|
||||
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip"
|
||||
import { formatDate } from "@/lib/date"
|
||||
import { Form } from "@/components/ui/form"
|
||||
@@ -26,109 +26,50 @@ import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "@
|
||||
import { toast } from "sonner"
|
||||
import ErrorBanner from "@/components/ui/error-banner"
|
||||
|
||||
const useBankCurrency = () => {
|
||||
const bankAccount = useAtomValue(selectedBankAccountAtom)
|
||||
return bankAccount?.account_currency ?? getCompanyCurrency(bankAccount?.company ?? '')
|
||||
}
|
||||
|
||||
/**
|
||||
* One line of the balance summary - label on the left, figure right-aligned.
|
||||
*
|
||||
* `items-baseline` keeps the figure on the label's FIRST line, so a row carrying a `subLabel`
|
||||
* (the statement row's "As of <date>" note) doesn't centre its value against both lines.
|
||||
*/
|
||||
const BalanceRow = ({ label, info, subLabel, emphasis, children }: {
|
||||
label: React.ReactNode
|
||||
info?: React.ReactNode
|
||||
subLabel?: React.ReactNode
|
||||
emphasis?: boolean
|
||||
children: React.ReactNode
|
||||
}) => (
|
||||
<div className="flex items-baseline justify-between gap-3">
|
||||
<span className="flex min-w-0 flex-col gap-1.5">
|
||||
<span className={cn("flex items-center gap-1 whitespace-nowrap text-xs text-ink-gray-6",
|
||||
emphasis && "font-medium text-ink-gray-7")}>
|
||||
{label}
|
||||
{info}
|
||||
</span>
|
||||
{subLabel}
|
||||
</span>
|
||||
<div className="flex flex-col items-end">{children}</div>
|
||||
</div>
|
||||
)
|
||||
|
||||
/**
|
||||
* Type styles for a figure. Shared so an interactive figure can put them on the <button>
|
||||
* ITSELF rather than on a nested span: Tailwind's preflight sets `font: inherit` on buttons,
|
||||
* which resets line-height too, so a button wrapping a `text-sm` span gets a taller strut than
|
||||
* the span and the row grows - visible as extra space above a baseline-aligned row.
|
||||
*/
|
||||
const BALANCE_VALUE_CLASSES = "font-numeric text-sm tabular-nums text-ink-gray-8"
|
||||
|
||||
const BalanceValue = ({ children, emphasis, tone, className }: { children: React.ReactNode, emphasis?: boolean, tone?: 'red', className?: string }) => (
|
||||
<span className={cn(BALANCE_VALUE_CLASSES,
|
||||
emphasis && "font-semibold",
|
||||
tone === 'red' && "text-ink-red-3",
|
||||
className)}>
|
||||
{children}
|
||||
</span>
|
||||
)
|
||||
|
||||
const BalanceSkeleton = () => <Skeleton className="h-4 w-24 rounded-sm" />
|
||||
|
||||
/**
|
||||
* Balances and progress for the selected bank account, laid out like the totals block of an
|
||||
* invoice. This sits beside the bank picker rather than in a row of its own (saves vertical
|
||||
* space) and outside the picker's horizontal scroll area, so the figures being reconciled
|
||||
* against can never scroll out of view.
|
||||
*/
|
||||
const BankAccountBalancePanel = () => {
|
||||
const BankBalance = () => {
|
||||
|
||||
const bankAccount = useAtomValue(selectedBankAccountAtom)
|
||||
|
||||
if (!bankAccount) {
|
||||
return null
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex w-72 shrink-0 flex-col justify-center gap-2.5 border-s border-outline-gray-2 ps-4">
|
||||
{/* Names the account these figures belong to - the picker scrolls, so the
|
||||
highlighted card can't be relied on as the referent. */}
|
||||
<span
|
||||
className="truncate text-xs font-medium text-ink-gray-7"
|
||||
title={bankAccount.account_name}>
|
||||
{bankAccount.account_name}
|
||||
</span>
|
||||
<OpeningBalanceRow />
|
||||
<SystemClosingBalanceRow />
|
||||
<StatementClosingBalanceRow />
|
||||
<Separator />
|
||||
<DifferenceRow />
|
||||
<ReconciledRow />
|
||||
<div className="flex justify-between">
|
||||
<div className="w-[80%] flex flex-wrap justify-between gap-2 pe-8 border-e-border border-e">
|
||||
<OpeningBalance />
|
||||
<ClosingBalance />
|
||||
<ClosingBalanceAsPerStatement />
|
||||
<Difference />
|
||||
</div>
|
||||
|
||||
<ReconcileProgress />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
const OpeningBalanceRow = () => {
|
||||
const currency = useBankCurrency()
|
||||
const OpeningBalance = () => {
|
||||
const bankAccount = useAtomValue(selectedBankAccountAtom)
|
||||
const { data, isLoading } = useGetAccountOpeningBalance()
|
||||
|
||||
return <BalanceRow label={_("Opening Balance")}>
|
||||
{isLoading ? <BalanceSkeleton /> : <BalanceValue>{formatCurrency(flt(data?.message, 2), currency)}</BalanceValue>}
|
||||
</BalanceRow>
|
||||
return <StatContainer className="min-w-48">
|
||||
<StatLabel>{_("Opening Balance")}</StatLabel>
|
||||
{isLoading ? <Skeleton className="w-[150px] h-5 rounded-sm" /> : <StatValue className="font-numeric">{formatCurrency(flt(data?.message, 2), bankAccount?.account_currency ?? getCompanyCurrency(bankAccount?.company ?? ''))}</StatValue>}
|
||||
</StatContainer>
|
||||
}
|
||||
|
||||
const SystemClosingBalanceRow = () => {
|
||||
const currency = useBankCurrency()
|
||||
const ClosingBalance = () => {
|
||||
const bankAccount = useAtomValue(selectedBankAccountAtom)
|
||||
const { data, isLoading } = useGetAccountClosingBalance()
|
||||
|
||||
return (
|
||||
<BalanceRow
|
||||
label={_("Closing (system)")}
|
||||
info={
|
||||
<StatContainer className="min-w-48">
|
||||
<div className="flex items-start gap-1">
|
||||
<StatLabel>
|
||||
{_("Closing Balance as per system")}
|
||||
</StatLabel>
|
||||
<HoverCard openDelay={100}>
|
||||
<HoverCardTrigger>
|
||||
<Info className="size-3.5 text-ink-gray-6" />
|
||||
<Info className="size-3.5 text-ink-gray-6 -mt-px" />
|
||||
</HoverCardTrigger>
|
||||
<HoverCardContent className="w-96" align="start" side="right">
|
||||
<H4 className="text-base">{_("Closing balance as per system")}</H4>
|
||||
@@ -143,111 +84,15 @@ const SystemClosingBalanceRow = () => {
|
||||
</Paragraph>
|
||||
</HoverCardContent>
|
||||
</HoverCard>
|
||||
}
|
||||
>
|
||||
{isLoading ? <BalanceSkeleton /> : <BalanceValue>{formatCurrency(flt(data?.message, 2), currency)}</BalanceValue>}
|
||||
</BalanceRow>
|
||||
|
||||
</div>
|
||||
{isLoading ? <Skeleton className="w-[150px] h-5 rounded-sm" /> : <StatValue className="font-numeric">{formatCurrency(flt(data?.message, 2), bankAccount?.account_currency ?? getCompanyCurrency(bankAccount?.company ?? ''))}</StatValue>}
|
||||
</StatContainer>
|
||||
)
|
||||
}
|
||||
|
||||
const StatementClosingBalanceRow = () => {
|
||||
|
||||
const Difference = () => {
|
||||
const bankAccount = useAtomValue(selectedBankAccountAtom)
|
||||
const currency = useBankCurrency()
|
||||
const dates = useAtomValue(bankRecDateAtom)
|
||||
const setValue = useSetAtom(bankRecClosingBalanceAtom(bankAccount?.name ?? ''))
|
||||
|
||||
const { data, isLoading } = useGetAccountClosingBalanceAsPerStatement({
|
||||
onSuccess: (data) => {
|
||||
if (data?.message && data?.message?.balance) {
|
||||
setValue({
|
||||
value: data?.message?.balance,
|
||||
stringValue: data?.message?.balance.toString()
|
||||
})
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
const isDateSame = data?.message?.date === dates.toDate
|
||||
|
||||
// The server uses the returned date to distinguish an unset balance from a saved zero.
|
||||
const hasBalance = Boolean(data?.message?.date)
|
||||
|
||||
const [isOpen, setIsOpen] = useState(false)
|
||||
|
||||
const tooltip = hasBalance
|
||||
? _("Click to change the closing balance as per statement")
|
||||
: _("Click to set the closing balance as per statement")
|
||||
|
||||
return (
|
||||
<BalanceRow
|
||||
label={_("Closing (statement)")}
|
||||
// The pencil sits beside the label, mirroring the info icon on the row above, so
|
||||
// the figure stays a plain right-aligned number in line with every other row.
|
||||
info={
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
{/* `p-0`: Tailwind's preflight gives buttons `appearance: button` but
|
||||
doesn't reset padding, so a bare button picks up the UA's ~1px 6px
|
||||
and knocks this row out of step with its neighbours. */}
|
||||
<button
|
||||
type='button'
|
||||
aria-label={tooltip}
|
||||
onClick={() => setIsOpen(true)}
|
||||
className="cursor-pointer p-0 text-ink-gray-5 transition-colors hover:text-ink-gray-7">
|
||||
<Edit className="size-3.5" />
|
||||
</button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent>{tooltip}</TooltipContent>
|
||||
</Tooltip>
|
||||
}
|
||||
subLabel={!isDateSame && data?.message.date
|
||||
? <span className="whitespace-nowrap text-2xs font-medium text-ink-red-3">
|
||||
{_("As of {0}", [formatDate(data?.message?.date ?? '', 'Do MMM YYYY')])}
|
||||
</span>
|
||||
: undefined}
|
||||
>
|
||||
{/* Deliberately NOT a flex container: a flex box's baseline doesn't resolve to its
|
||||
text, so the row's `items-baseline` couldn't line this up with the label. As a
|
||||
plain inline button its baseline is the figure's own, like every other row.
|
||||
"Set" gets the same treatment as a figure - it stands in for one. */}
|
||||
{isLoading
|
||||
? <BalanceSkeleton />
|
||||
: <Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
{/* The figure styles live on the button itself - see
|
||||
BALANCE_VALUE_CLASSES. `p-0` because preflight leaves the UA's
|
||||
button padding in place. */}
|
||||
<button
|
||||
type='button'
|
||||
aria-label={tooltip}
|
||||
onClick={() => setIsOpen(true)}
|
||||
className={cn(BALANCE_VALUE_CLASSES,
|
||||
"cursor-pointer p-0 underline decoration-outline-gray-5 decoration-dashed underline-offset-4",
|
||||
"transition-colors hover:decoration-ink-gray-8")}>
|
||||
{hasBalance ? formatCurrency(flt(data?.message?.balance, 2), currency) : _("Set")}
|
||||
</button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent>{tooltip}</TooltipContent>
|
||||
</Tooltip>}
|
||||
|
||||
<Dialog open={isOpen} onOpenChange={setIsOpen}>
|
||||
<DialogContent className="min-w-xl">
|
||||
<ClosingBalanceForm
|
||||
defaultBalance={data?.message?.balance ?? 0}
|
||||
date={dates.toDate}
|
||||
bankAccount={bankAccount}
|
||||
onClose={() => setIsOpen(false)}
|
||||
/>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
</BalanceRow>
|
||||
)
|
||||
}
|
||||
|
||||
const DifferenceRow = () => {
|
||||
const bankAccount = useAtomValue(selectedBankAccountAtom)
|
||||
const currency = useBankCurrency()
|
||||
|
||||
const { data, isLoading } = useGetAccountClosingBalance()
|
||||
|
||||
@@ -257,15 +102,16 @@ const DifferenceRow = () => {
|
||||
|
||||
const isError = difference !== 0
|
||||
|
||||
return <BalanceRow label={_("Difference")} emphasis>
|
||||
{isLoading
|
||||
? <BalanceSkeleton />
|
||||
: <BalanceValue emphasis tone={isError ? 'red' : undefined}>{formatCurrency(difference, currency)}</BalanceValue>}
|
||||
</BalanceRow>
|
||||
return <StatContainer className="w-fit text-end sm:min-w-56">
|
||||
<StatLabel className="text-end">{_("Difference")}</StatLabel>
|
||||
{isLoading ? <Skeleton className="w-[150px] h-5 self-end rounded-sm" /> : <StatValue className={isError ? 'text-ink-red-3 font-numeric' : 'font-numeric'}>
|
||||
{formatCurrency(difference,
|
||||
bankAccount?.account_currency ?? getCompanyCurrency(bankAccount?.company ?? ''))
|
||||
}</StatValue>}
|
||||
</StatContainer>
|
||||
}
|
||||
|
||||
/** Reconciliation progress through the selected date range: a count plus a slim bar. */
|
||||
const ReconciledRow = () => {
|
||||
const ReconcileProgress = () => {
|
||||
|
||||
const bankAccount = useAtomValue(selectedBankAccountAtom)
|
||||
|
||||
@@ -286,14 +132,75 @@ const ReconciledRow = () => {
|
||||
|
||||
const progress = (totalCount ? reconciledCount / totalCount : 0) * 100
|
||||
|
||||
return <div className="flex flex-col gap-1.5">
|
||||
<BalanceRow label={_("Reconciled")}>
|
||||
<BalanceValue>{reconciledCount} / {totalCount ?? 0}</BalanceValue>
|
||||
</BalanceRow>
|
||||
<Progress value={progress} max={100} size="sm" />
|
||||
return <div className="w-[18%] flex flex-col gap-1 items-end">
|
||||
<div className="w-full">
|
||||
<Progress
|
||||
value={progress}
|
||||
max={100}
|
||||
size="md"
|
||||
label="Progress"
|
||||
hint
|
||||
hintText={`${reconciledCount} / ${totalCount} ${_("reconciled")}`} />
|
||||
</div>
|
||||
</div>
|
||||
}
|
||||
|
||||
const ClosingBalanceAsPerStatement = () => {
|
||||
|
||||
const bankAccount = useAtomValue(selectedBankAccountAtom)
|
||||
const dates = useAtomValue(bankRecDateAtom)
|
||||
const setValue = useSetAtom(bankRecClosingBalanceAtom(bankAccount?.name ?? ''))
|
||||
|
||||
const { data, isLoading } = useGetAccountClosingBalanceAsPerStatement({
|
||||
onSuccess: (data) => {
|
||||
if (data?.message && data?.message?.balance) {
|
||||
setValue({
|
||||
value: data?.message?.balance,
|
||||
stringValue: data?.message?.balance.toString()
|
||||
})
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
const isDateSame = data?.message?.date === dates.toDate
|
||||
|
||||
const [isOpen, setIsOpen] = useState(false)
|
||||
|
||||
|
||||
return <StatContainer className="min-w-48">
|
||||
<StatLabel>{_("Closing Balance as per statement")}</StatLabel>
|
||||
<div className="flex flex-col gap-2 items-start">
|
||||
<Dialog open={isOpen} onOpenChange={setIsOpen}>
|
||||
<DialogTrigger>
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<div className="flex items-center gap-4 underline cursor-pointer underline-offset-6" role="button">
|
||||
{isLoading ? <Skeleton className="w-[150px] h-5 rounded-sm" /> : <StatValue className="font-numeric">{formatCurrency(flt(data?.message?.balance, 2), bankAccount?.account_currency ?? getCompanyCurrency(bankAccount?.company ?? ''))}</StatValue>}
|
||||
<Edit className="w-4 h-4" />
|
||||
</div>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent>
|
||||
{_("Click to set the closing balance as per statement")}
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
</DialogTrigger>
|
||||
<DialogContent className="min-w-xl">
|
||||
<ClosingBalanceForm
|
||||
defaultBalance={data?.message?.balance ?? 0}
|
||||
date={dates.toDate}
|
||||
bankAccount={bankAccount}
|
||||
onClose={() => setIsOpen(false)}
|
||||
/>
|
||||
|
||||
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
{!isDateSame && data?.message.date && <span className="text-xs font-medium text-ink-red-3">{_("As of {0}", [formatDate(data?.message?.date ?? '', 'Do MMM YYYY')])}</span>}
|
||||
</div>
|
||||
</StatContainer>
|
||||
|
||||
}
|
||||
|
||||
const ClosingBalanceForm = ({ defaultBalance, date, bankAccount, onClose }: { defaultBalance: number, date: string, bankAccount: SelectedBank | null, onClose: VoidFunction }) => {
|
||||
|
||||
const { mutate } = useSWRConfig()
|
||||
@@ -343,7 +250,7 @@ const ClosingBalanceForm = ({ defaultBalance, date, bankAccount, onClose }: { de
|
||||
{_("Enter the closing balance you see in your bank statement for {0} as of the {1}", [bankAccount?.account_name ?? bankAccount?.name ?? '', formatDate(date, 'Do MMM YYYY')])}
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
{error && <div className="py-2"><ErrorBanner error={error} /></div>}
|
||||
{error && <ErrorBanner error={error} />}
|
||||
<div className="py-4">
|
||||
<CurrencyFormField
|
||||
name="balance"
|
||||
@@ -395,7 +302,7 @@ const ClosingBalancesList = ({ bankAccount, date }: { bankAccount: SelectedBank
|
||||
|
||||
return <div>
|
||||
<Separator className="my-8" />
|
||||
<p className="text-p-sm text-center pb-2">{_("Balances as per bank statement before {0}", [formatDate(date, 'Do MMM YYYY')])}</p>
|
||||
<p className="text-sm text-center">{_("Balances as per bank statement before {0}", [formatDate(date, 'Do MMM YYYY')])}</p>
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
@@ -424,4 +331,4 @@ const ClosingBalancesList = ({ bankAccount, date }: { bankAccount: SelectedBank
|
||||
|
||||
}
|
||||
|
||||
export default BankAccountBalancePanel
|
||||
export default BankBalance
|
||||
@@ -2,6 +2,7 @@ import { useAtomValue } from "jotai"
|
||||
import { MissingFiltersBanner } from "./MissingFiltersBanner"
|
||||
import { bankRecDateAtom, SelectedBank, selectedBankAccountAtom } from "./bankRecAtoms"
|
||||
import { useCurrentCompany } from "@/hooks/useCurrentCompany"
|
||||
import { Paragraph } from "@/components/ui/typography"
|
||||
import type { ColumnDef } from "@tanstack/react-table"
|
||||
import { useCallback, useMemo, useState } from "react"
|
||||
import { useFrappeGetCall, useFrappePostCall, useSWRConfig } from "frappe-react-sdk"
|
||||
@@ -25,7 +26,6 @@ import { Form } from "@/components/ui/form"
|
||||
import { useForm } from "react-hook-form"
|
||||
import { DateField } from "@/components/ui/form-elements"
|
||||
import { Empty, EmptyMedia, EmptyHeader, EmptyTitle, EmptyDescription } from "@/components/ui/empty"
|
||||
import MarkdownRenderer from "@/components/ui/markdown"
|
||||
|
||||
const BankClearanceSummary = () => {
|
||||
const bankAccount = useAtomValue(selectedBankAccountAtom)
|
||||
@@ -203,14 +203,14 @@ const BankClearanceSummaryView = () => {
|
||||
[accountCurrency, bankAccount, companyID, mutate, onCopy],
|
||||
)
|
||||
|
||||
const content = _("Below is a list of all accounting entries posted against the bank account {0} between {1} and {2}.", [`<strong>${bankAccount?.account}</strong>`, `<strong>${formattedFromDate}</strong>`, `<strong>${formattedToDate}</strong>`])
|
||||
return <div className="space-y-4 py-2">
|
||||
|
||||
return <div className="flex min-h-0 flex-1 flex-col space-y-4 py-2">
|
||||
|
||||
<div className="shrink-0">
|
||||
<span className="text-p-sm">
|
||||
<MarkdownRenderer content={content} />
|
||||
</span>
|
||||
<div>
|
||||
<Paragraph className="text-sm">
|
||||
<span dangerouslySetInnerHTML={{
|
||||
__html: _("Below is a list of all accounting entries posted against the bank account {0} between {1} and {2}.", [`<strong>${bankAccount?.account}</strong>`, `<strong>${formattedFromDate}</strong>`, `<strong>${formattedToDate}</strong>`])
|
||||
}} />
|
||||
</Paragraph>
|
||||
</div>
|
||||
|
||||
{error && <ErrorBanner error={error} />}
|
||||
@@ -220,9 +220,8 @@ const BankClearanceSummaryView = () => {
|
||||
data={data.message.result}
|
||||
columns={clearanceColumns}
|
||||
getRowId={(row) => `${row.payment_entry}-${row.posting_date}`}
|
||||
className="min-h-0 flex-1"
|
||||
maxHeight="none"
|
||||
scrollAreaClassName="flex-1"
|
||||
maxHeight="calc(100vh - 200px)"
|
||||
scrollAreaClassName="min-h-[calc(100vh-200px)]"
|
||||
emptyState={_("No rows to display.")}
|
||||
/>
|
||||
) : null}
|
||||
|
||||
@@ -18,7 +18,6 @@ import { useMultiFileUploadProgress } from "@/hooks/useMultiFileUploadProgress"
|
||||
import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "@/components/ui/table"
|
||||
import { Checkbox } from "@/components/ui/checkbox"
|
||||
import { ArrowDownRight, ArrowUpRight, Plus, Trash2 } from "lucide-react"
|
||||
import { evaluateAmountFormula } from "@/lib/amountFormula"
|
||||
import { flt, formatCurrency } from "@/lib/numbers"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip"
|
||||
@@ -216,13 +215,38 @@ const BankEntryForm = ({ selectedTransaction }: { selectedTransaction: Unreconci
|
||||
})
|
||||
} else {
|
||||
|
||||
const transactionAmount = selectedTransaction.unallocated_amount ?? 0
|
||||
/**
|
||||
* The debit and credit amounts can also be expressions - like "transaction_amount * 0.5"
|
||||
* So we need to compute the value of the expression
|
||||
* We can use the eval function to do this. But we need to expose certain variables to the expression.
|
||||
* One of them is transaction_amount which is the unallocated amount of the selected transaction
|
||||
* @param expression - The expression to compute
|
||||
* @returns The computed value
|
||||
*/
|
||||
const computeExpression = (expression: string) => {
|
||||
|
||||
const script = `
|
||||
const transaction_amount = ${selectedTransaction.unallocated_amount ?? 0}
|
||||
${expression};
|
||||
`
|
||||
|
||||
let value = 0;
|
||||
|
||||
try {
|
||||
value = window.eval(script);
|
||||
} catch (error: unknown) {
|
||||
console.error(error);
|
||||
value = 0;
|
||||
}
|
||||
|
||||
return value;
|
||||
}
|
||||
if (!acc?.debit && !acc?.credit) {
|
||||
hasTotallyEmptyRowEarlier = true;
|
||||
}
|
||||
|
||||
const computedDebit = acc?.debit ? flt(evaluateAmountFormula(acc.debit, transactionAmount), 2) : 0
|
||||
const computedCredit = acc?.credit ? flt(evaluateAmountFormula(acc.credit, transactionAmount), 2) : 0
|
||||
const computedDebit = acc?.debit ? flt(computeExpression(acc.debit), 2) : 0
|
||||
const computedCredit = acc?.credit ? flt(computeExpression(acc.credit), 2) : 0
|
||||
|
||||
totalDebits = flt(totalDebits + computedDebit, 2)
|
||||
totalCredits = flt(totalCredits + computedCredit, 2)
|
||||
|
||||
@@ -74,10 +74,7 @@ const BankPicker = ({ className }: { className?: string }) => {
|
||||
}
|
||||
return (
|
||||
<div
|
||||
// No trailing padding: it would sit inside the fade region, so the mask would
|
||||
// spend itself on empty space and the last card would stop short of the balance
|
||||
// panel instead of fading towards it. The column gap provides the separation.
|
||||
className={cn("flex gap-3 items-stretch w-full overflow-x-auto scroll-fade-x",
|
||||
className={cn("flex gap-3 items-stretch w-full overflow-x-auto pe-4",
|
||||
banks?.length > 4 ? 'pb-2' : '', className,
|
||||
)}
|
||||
style={{
|
||||
@@ -111,12 +108,12 @@ const BankPickerItem = ({ bank }: { bank: SelectedBank }) => {
|
||||
role="button"
|
||||
title={`Select ${bank.account_name}`}
|
||||
onClick={onSelect}
|
||||
// `shrink-0`: this is a horizontally scrolling row, so cards keep their own width
|
||||
// instead of being compressed to fit the container.
|
||||
className={cn('w-60 shrink-0 rounded-md border border-outline-gray-1 p-2 overflow-hidden cursor-pointer transition-colors',
|
||||
className={cn('rounded-md border border-outline-gray-1 max-w-60 min-w-60 p-2 overflow-hidden cursor-pointer',
|
||||
isSelected ? 'border-outline-gray-5 bg-surface-gray-1' : 'hover:bg-surface-gray-1'
|
||||
)}
|
||||
>
|
||||
|
||||
|
||||
<BankLogo bank={bank} className="mb-2" />
|
||||
|
||||
<div className="flex flex-col gap-1">
|
||||
|
||||
@@ -5,179 +5,107 @@ import { AVAILABLE_TIME_PERIODS, formatDate, getDatesForTimePeriod, TimePeriod }
|
||||
import { Button } from '@/components/ui/button'
|
||||
import { Popover, PopoverContent, PopoverTrigger } from '@/components/ui/popover'
|
||||
import { ChevronDownIcon, ChevronLeftIcon, ChevronRight } from 'lucide-react'
|
||||
import { Command, CommandGroup, CommandInput, CommandItem, CommandList } from '@/components/ui/command'
|
||||
import { Command, CommandEmpty, CommandInput, CommandItem, CommandList } from '@/components/ui/command'
|
||||
import { parse } from "chrono-node"
|
||||
import { Calendar } from '@/components/ui/calendar'
|
||||
import useFiscalYear from '@/hooks/useFiscalYear'
|
||||
import dayjs from 'dayjs'
|
||||
import _ from '@/lib/translate'
|
||||
import { useDirection } from '@/components/ui/direction'
|
||||
import useResetScrollOnSearch from '@/hooks/useResetScrollOnSearch'
|
||||
|
||||
const DATE_FORMAT = 'YYYY-MM-DD'
|
||||
|
||||
/** Current fiscal year plus this many previous ones, for quarter/year options. */
|
||||
const PREVIOUS_FISCAL_YEARS = 2
|
||||
|
||||
type DateOption = {
|
||||
/** Stable id - used as the cmdk value and the React key. */
|
||||
key: string
|
||||
label: string
|
||||
translatedLabel: string
|
||||
fromDate: string
|
||||
toDate: string
|
||||
format: string
|
||||
/** Extra terms to match against, beyond the labels and dates. */
|
||||
keywords?: string[]
|
||||
/** Whether to show this option when the search box is empty. */
|
||||
isDefault?: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Fiscal years keep the same month/day boundaries year on year, so previous years can be
|
||||
* derived by subtracting whole years instead of fetching them. Works for both Jan-Dec and
|
||||
* Apr-Mar style fiscal years.
|
||||
*/
|
||||
const fiscalYearLabel = (start: dayjs.Dayjs, end: dayjs.Dayjs) =>
|
||||
start.year() === end.year() ? `${start.year()}` : `${start.year()}-${end.year()}`
|
||||
|
||||
const BankRecDateFilter = () => {
|
||||
|
||||
const [bankRecDate, setBankRecDate] = useAtom(bankRecDateAtom)
|
||||
|
||||
const { fiscalYear } = useFiscalYear()
|
||||
const { data: fiscalYear } = useFiscalYear()
|
||||
|
||||
const today = useMemo(() => dayjs().format(DATE_FORMAT), [])
|
||||
|
||||
const allOptions = useMemo(() => {
|
||||
const standardOptions: DateOption[] = AVAILABLE_TIME_PERIODS.map((period) => {
|
||||
const timePeriodOptions = useMemo(() => {
|
||||
const standardOptions = AVAILABLE_TIME_PERIODS.map((period) => {
|
||||
const dates = getDatesForTimePeriod(period)
|
||||
return {
|
||||
key: period,
|
||||
label: period,
|
||||
translatedLabel: dates.translatedLabel ?? _(period),
|
||||
fromDate: dates.fromDate,
|
||||
toDate: dates.toDate,
|
||||
format: dates.format,
|
||||
isDefault: true,
|
||||
translatedLabel: dates.translatedLabel
|
||||
}
|
||||
})
|
||||
|
||||
if (!fiscalYear) {
|
||||
return standardOptions
|
||||
}
|
||||
if (fiscalYear?.message) {
|
||||
// For a fiscal year, we need to replace "Last Year", "This Year", and add options for quarters
|
||||
const fiscalYearStart = fiscalYear.message.year_start_date
|
||||
const fiscalYearEnd = fiscalYear.message.year_end_date
|
||||
|
||||
const currentStart = dayjs(fiscalYear.year_start_date)
|
||||
const currentEnd = dayjs(fiscalYear.year_end_date)
|
||||
|
||||
const quarterOptions: DateOption[] = []
|
||||
const fiscalYearOptions: DateOption[] = []
|
||||
|
||||
// Static literals so the translation extractor can find them.
|
||||
const quarterLabels = [_("Q1"), _("Q2"), _("Q3"), _("Q4")]
|
||||
|
||||
for (let yearsAgo = 0; yearsAgo <= PREVIOUS_FISCAL_YEARS; yearsAgo++) {
|
||||
const start = currentStart.subtract(yearsAgo, 'year')
|
||||
const end = currentEnd.subtract(yearsAgo, 'year')
|
||||
// Keep the real name for the current year; derive it for the earlier ones.
|
||||
const yearLabel = yearsAgo === 0 ? fiscalYear.name : fiscalYearLabel(start, end)
|
||||
|
||||
for (let quarter = 0; quarter < 4; quarter++) {
|
||||
const quarterStart = start.add(quarter * 3, 'month')
|
||||
// End the day before the next quarter starts, clamped to the fiscal year end
|
||||
// so a short fiscal year can't spill over.
|
||||
const nextQuarterStart = start.add((quarter + 1) * 3, 'month')
|
||||
const quarterEnd = nextQuarterStart.subtract(1, 'day').isAfter(end)
|
||||
? end
|
||||
: nextQuarterStart.subtract(1, 'day')
|
||||
|
||||
if (quarterStart.isAfter(end)) continue
|
||||
|
||||
quarterOptions.push({
|
||||
key: `Q${quarter + 1}-${yearLabel}`,
|
||||
label: `Q${quarter + 1}: ${yearLabel}`,
|
||||
translatedLabel: `${quarterLabels[quarter]}: ${yearLabel}`,
|
||||
fromDate: quarterStart.format(DATE_FORMAT),
|
||||
toDate: quarterEnd.format(DATE_FORMAT),
|
||||
format: 'MMM YYYY',
|
||||
keywords: ['quarter', `q${quarter + 1}`, yearLabel],
|
||||
// Only the current fiscal year's quarters clutter the default list;
|
||||
// older ones stay searchable.
|
||||
isDefault: yearsAgo === 0,
|
||||
})
|
||||
const q1 = {
|
||||
label: `Q1: ${fiscalYear.message.name}`,
|
||||
translatedLabel: `${_("Q1")}: ${fiscalYear.message.name}`,
|
||||
fromDate: fiscalYearStart,
|
||||
toDate: dayjs(fiscalYearStart).add(3, 'month').format('YYYY-MM-DD'),
|
||||
format: 'MMM YYYY'
|
||||
}
|
||||
|
||||
const label = yearsAgo === 0
|
||||
? 'This Fiscal Year'
|
||||
: yearsAgo === 1
|
||||
? 'Last Fiscal Year'
|
||||
: `FY ${yearLabel}`
|
||||
const q2 = {
|
||||
label: `Q2: ${fiscalYear.message.name}`,
|
||||
translatedLabel: `${_("Q2")}: ${fiscalYear.message.name}`,
|
||||
fromDate: dayjs(fiscalYearStart).add(3, 'month').format('YYYY-MM-DD'),
|
||||
toDate: dayjs(fiscalYearStart).add(6, 'month').format('YYYY-MM-DD'),
|
||||
format: 'MMM YYYY'
|
||||
}
|
||||
|
||||
fiscalYearOptions.push({
|
||||
key: `fiscal-year-${yearLabel}`,
|
||||
label,
|
||||
translatedLabel: yearsAgo <= 1 ? _(label) : `${_("FY")} ${yearLabel}`,
|
||||
fromDate: start.format(DATE_FORMAT),
|
||||
toDate: end.format(DATE_FORMAT),
|
||||
format: 'MMM YYYY',
|
||||
keywords: ['fiscal year', yearLabel],
|
||||
isDefault: yearsAgo <= 1,
|
||||
const q3 = {
|
||||
label: `Q3: ${fiscalYear.message.name}`,
|
||||
translatedLabel: `${_("Q3")}: ${fiscalYear.message.name}`,
|
||||
fromDate: dayjs(fiscalYearStart).add(6, 'month').format('YYYY-MM-DD'),
|
||||
toDate: dayjs(fiscalYearStart).add(9, 'month').format('YYYY-MM-DD'),
|
||||
format: 'MMM YYYY'
|
||||
}
|
||||
|
||||
const q4 = {
|
||||
label: `Q4: ${fiscalYear.message.name}`,
|
||||
translatedLabel: `${_("Q4")}: ${fiscalYear.message.name}`,
|
||||
fromDate: dayjs(fiscalYearStart).add(9, 'month').format('YYYY-MM-DD'),
|
||||
toDate: fiscalYearEnd,
|
||||
format: 'MMM YYYY'
|
||||
}
|
||||
|
||||
const thisYear = {
|
||||
label: `This Fiscal Year`,
|
||||
translatedLabel: `${_("This Fiscal Year")}`,
|
||||
fromDate: fiscalYearStart,
|
||||
toDate: fiscalYearEnd,
|
||||
format: 'MMM YYYY'
|
||||
}
|
||||
|
||||
const lastYear = {
|
||||
label: `Last Fiscal Year`,
|
||||
translatedLabel: `${_("Last Fiscal Year")}`,
|
||||
fromDate: dayjs(fiscalYearStart).subtract(1, 'year').format('YYYY-MM-DD'),
|
||||
toDate: dayjs(fiscalYearEnd).subtract(1, 'year').format('YYYY-MM-DD'),
|
||||
format: 'MMM YYYY'
|
||||
}
|
||||
// Sort the options so that we get "This Month", "Last Month", quarters, fiscal year, then the rest of the standard options
|
||||
|
||||
const topRankedItems = standardOptions.filter((option) => {
|
||||
return option.label === "This Month" || option.label === "Last Month"
|
||||
})
|
||||
|
||||
const bottomRankedItems = standardOptions.filter((option) => {
|
||||
return option.label !== "This Month" && option.label !== "Last Month"
|
||||
})
|
||||
|
||||
return [...topRankedItems, q1, q2, q3, q4, thisYear, lastYear, ...bottomRankedItems]
|
||||
}
|
||||
|
||||
// "This Month"/"Last Month" first, then quarters and fiscal years, then the rest.
|
||||
const topRanked = standardOptions.filter((o) => o.label === 'This Month' || o.label === 'Last Month')
|
||||
const bottomRanked = standardOptions.filter((o) => o.label !== 'This Month' && o.label !== 'Last Month')
|
||||
|
||||
return [...topRanked, ...quarterOptions, ...fiscalYearOptions, ...bottomRanked]
|
||||
return standardOptions
|
||||
}, [fiscalYear])
|
||||
|
||||
// Reconciliation only looks backwards, so a period that hasn't started is never useful.
|
||||
const selectableOptions = useMemo(
|
||||
() => allOptions.filter((option) => option.fromDate <= today),
|
||||
[allOptions, today],
|
||||
)
|
||||
|
||||
const [open, setOpen] = useState(false)
|
||||
const [value, setValue] = useState("")
|
||||
|
||||
// We filter ourselves (`shouldFilter={false}`) so that the parsed-date suggestion can be a
|
||||
// real CommandItem alongside the predefined options, and keyboard navigation covers both.
|
||||
const filteredOptions = useMemo(() => {
|
||||
const query = value.trim().toLowerCase()
|
||||
|
||||
if (!query) {
|
||||
return selectableOptions.filter((option) => option.isDefault)
|
||||
}
|
||||
|
||||
const tokens = query.split(/\s+/)
|
||||
|
||||
return selectableOptions.filter((option) => {
|
||||
const haystack = [
|
||||
option.label,
|
||||
option.translatedLabel,
|
||||
...(option.keywords ?? []),
|
||||
option.fromDate,
|
||||
option.toDate,
|
||||
].join(' ').toLowerCase()
|
||||
|
||||
return tokens.every((token) => haystack.includes(token))
|
||||
})
|
||||
}, [selectableOptions, value])
|
||||
|
||||
const parsedOption = useMemo(() => parseDateRange(value), [value])
|
||||
|
||||
// Filtering shortens the list, so pin the scroll back to the top to keep the
|
||||
// auto-selected first option in view.
|
||||
const listRef = useResetScrollOnSearch(value)
|
||||
|
||||
// Don't show a parsed suggestion that duplicates an option already in the list.
|
||||
const showParsedOption = parsedOption
|
||||
&& !filteredOptions.some((o) => o.fromDate === parsedOption.fromDate && o.toDate === parsedOption.toDate)
|
||||
|
||||
const timePeriod: TimePeriod | string = useMemo(() => {
|
||||
if (bankRecDate.fromDate && bankRecDate.toDate) {
|
||||
for (const period of allOptions) {
|
||||
// Check if the from and to dates match any predefined time period
|
||||
for (const period of timePeriodOptions) {
|
||||
if (period.fromDate === bankRecDate.fromDate && period.toDate === bankRecDate.toDate) {
|
||||
return period.label;
|
||||
}
|
||||
@@ -186,11 +114,10 @@ const BankRecDateFilter = () => {
|
||||
} else {
|
||||
return "Date Range";
|
||||
}
|
||||
}, [bankRecDate.fromDate, bankRecDate.toDate, allOptions]);
|
||||
}, [bankRecDate.fromDate, bankRecDate.toDate, timePeriodOptions]);
|
||||
|
||||
const handleTimePeriodChange = (fromDate: string, toDate: string) => {
|
||||
setBankRecDate({ fromDate, toDate })
|
||||
setValue("")
|
||||
setOpen(false)
|
||||
}
|
||||
|
||||
@@ -203,9 +130,7 @@ const BankRecDateFilter = () => {
|
||||
|
||||
const direction = useDirection()
|
||||
|
||||
const RangeArrow = direction === 'ltr'
|
||||
? <ChevronRight className='text-[12px] text-ink-gray-5/70' />
|
||||
: <ChevronLeftIcon className='text-[12px] text-ink-gray-5/70' />
|
||||
|
||||
|
||||
return <div className='flex items-center'>
|
||||
<Popover open={open} onOpenChange={setOpen}>
|
||||
@@ -216,57 +141,30 @@ const BankRecDateFilter = () => {
|
||||
size='md'
|
||||
className='rounded-e-none border-e-0'
|
||||
role="combobox">
|
||||
{allOptions.find((period) => period.label === timePeriod)?.translatedLabel ?? _(timePeriod)}
|
||||
{timePeriodOptions.find((period) => period.label === timePeriod)?.translatedLabel ?? _(timePeriod)}
|
||||
|
||||
<ChevronDownIcon />
|
||||
</Button>
|
||||
</PopoverTrigger>
|
||||
|
||||
<PopoverContent className="w-84 p-1" align='start'>
|
||||
<Command shouldFilter={false}>
|
||||
<Command>
|
||||
|
||||
<CommandInput placeholder={_("e.g. Last 3 weeks, Q1, May 2025")} onValueChange={setValue} value={value} />
|
||||
<CommandList ref={listRef} className='max-h-80'>
|
||||
{showParsedOption && parsedOption && (
|
||||
<CommandGroup heading={_("Matched date")}>
|
||||
<CommandItem
|
||||
value='parsed-date-range'
|
||||
className='flex justify-between'
|
||||
onSelect={() => handleTimePeriodChange(parsedOption.fromDate, parsedOption.toDate)}>
|
||||
<span className='max-w-[45%] truncate'>{value}</span>
|
||||
<span className='text-xs text-ink-gray-5 flex items-center gap-1 text-end whitespace-nowrap'>
|
||||
{parsedOption.fromDate === parsedOption.toDate
|
||||
? formatDate(parsedOption.fromDate, 'Do MMM YYYY')
|
||||
: <>{formatDate(parsedOption.fromDate, 'Do MMM YY')} {RangeArrow} {formatDate(parsedOption.toDate, 'Do MMM YY')}</>}
|
||||
</span>
|
||||
</CommandItem>
|
||||
</CommandGroup>
|
||||
)}
|
||||
|
||||
{filteredOptions.length > 0 && (
|
||||
<CommandGroup>
|
||||
{filteredOptions.map((period) => (
|
||||
<CommandItem
|
||||
key={period.key}
|
||||
value={period.key}
|
||||
className='flex justify-between'
|
||||
onSelect={() => handleTimePeriodChange(period.fromDate, period.toDate)}>
|
||||
<span>
|
||||
{period.translatedLabel}
|
||||
</span>
|
||||
<span className='text-xs text-ink-gray-5 flex items-center gap-1 text-end whitespace-nowrap'>
|
||||
{formatDate(period.fromDate, period.format)} {RangeArrow} {formatDate(period.toDate, period.format)}
|
||||
</span>
|
||||
</CommandItem>
|
||||
))}
|
||||
</CommandGroup>
|
||||
)}
|
||||
|
||||
{!showParsedOption && filteredOptions.length === 0 && (
|
||||
<div className='p-2 text-sm text-ink-gray-5'>
|
||||
{_("No results found")}
|
||||
</div>
|
||||
)}
|
||||
<CommandInput placeholder="e.g. Last 3 weeks" onValueChange={setValue} value={value} />
|
||||
<CommandList className='max-h-fit'>
|
||||
<CommandEmpty className='text-start p-2 hover:bg-surface-gray-1'>
|
||||
<EmptyState onSelect={handleTimePeriodChange} value={value} />
|
||||
</CommandEmpty>
|
||||
{timePeriodOptions.map((period) => (
|
||||
<CommandItem key={period.label} className='flex justify-between' onSelect={() => handleTimePeriodChange(period.fromDate, period.toDate)}>
|
||||
<span>
|
||||
{period.translatedLabel ?? _(period.label)}
|
||||
</span>
|
||||
<span className='text-xs text-ink-gray-5 flex items-center gap-1 text-end whitespace-nowrap'>
|
||||
{formatDate(period.fromDate, period.format)} {direction === 'ltr' ? <ChevronRight className='text-[12px] text-ink-gray-5/70' /> : <ChevronLeftIcon className='text-[12px] text-ink-gray-5/70' />} {formatDate(period.toDate, period.format)}
|
||||
</span>
|
||||
</CommandItem>
|
||||
))}
|
||||
</CommandList>
|
||||
</Command>
|
||||
|
||||
@@ -301,97 +199,77 @@ const BankRecDateFilter = () => {
|
||||
}
|
||||
|
||||
const referentialKeywords = ["last", "this", "next", "previous"]
|
||||
const EmptyState = ({ onSelect, value }: { onSelect: (fromDate: string, toDate: string) => void, value: string }) => {
|
||||
|
||||
/** chrono exposes `knownValues` on ParsingComponents but doesn't type it publicly. */
|
||||
const knownValuesOf = (components: unknown): Record<string, number> =>
|
||||
(components as { knownValues?: Record<string, number> })?.knownValues ?? {}
|
||||
const dates = useMemo(() => {
|
||||
if (value) {
|
||||
// Try parsing the value
|
||||
const parsedDate = parse(value, undefined, { forwardDate: false })
|
||||
|
||||
/**
|
||||
* How far back a parsed date must move to land in the past. Reconciliation only ever looks
|
||||
* backwards, so an ambiguous input that chrono resolves into the future - "December" typed in
|
||||
* September, or a bare weekday like "Friday" - is pulled to its most recent past occurrence.
|
||||
* An explicitly stated year is respected; a range that is still future gets discarded later.
|
||||
*
|
||||
* This returns a shift rather than a date so that a range can be moved as a single unit -
|
||||
* shifting its start and end independently would distort or invert it.
|
||||
*/
|
||||
const pastShift = (date: Date, knownValues: Record<string, number>) => {
|
||||
const today = dayjs()
|
||||
let candidate = dayjs(date)
|
||||
if (parsedDate && parsedDate.length > 0) {
|
||||
const startDate = parsedDate[0].start.date()
|
||||
const endDate = parsedDate[0].end?.date()
|
||||
|
||||
if (!candidate.isAfter(today, 'date') || knownValues.year !== undefined) {
|
||||
return { amount: 0, unit: 'year' as const }
|
||||
if (!endDate) {
|
||||
const today = new Date()
|
||||
// If today is greater than the start date, use today as the end date
|
||||
if (startDate.getTime() > today.getTime()) {
|
||||
return { fromDate: today, toDate: startDate }
|
||||
} else {
|
||||
// Check if the user only wants a specific month like "May 2025"
|
||||
// If the "known values" just has month and year, then we need to get the first day of the month and the last day of the month
|
||||
// @ts-expect-error - "Known Values" is available in the start "ParsingComponents"
|
||||
if (parsedDate[0].start.knownValues?.month && !parsedDate[0].start.knownValues?.day) {
|
||||
return {
|
||||
fromDate: startDate,
|
||||
toDate: dayjs(startDate).endOf('month').toDate()
|
||||
}
|
||||
// @ts-expect-error - "Known Values" is available in the start "ParsingComponents"
|
||||
} else if (parsedDate[0].start.knownValues?.month && parsedDate[0].start.knownValues?.day && !referentialKeywords.some(keyword => value.toLowerCase().includes(keyword))) {
|
||||
// If month and day is known, then we should not assume that the user wants to get everything until today
|
||||
return {
|
||||
fromDate: startDate,
|
||||
toDate: startDate,
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
fromDate: startDate,
|
||||
toDate: today
|
||||
}
|
||||
}
|
||||
} else {
|
||||
return { fromDate: startDate, toDate: endDate }
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
}, [value])
|
||||
|
||||
const onClick = (fromDate: Date, toDate: Date) => {
|
||||
onSelect(formatDate(fromDate, 'YYYY-MM-DD'), formatDate(toDate, 'YYYY-MM-DD'))
|
||||
}
|
||||
|
||||
// A bare weekday repeats weekly, everything else (month/day) repeats yearly.
|
||||
const unit = knownValues.weekday !== undefined && knownValues.day === undefined
|
||||
? 'day' as const
|
||||
: 'year' as const
|
||||
const step = unit === 'day' ? 7 : 1
|
||||
let amount = 0
|
||||
const isEqual = dates?.fromDate && dates?.toDate && dayjs(dates.fromDate).isSame(dates.toDate, 'date')
|
||||
|
||||
for (let i = 0; i < 200 && candidate.isAfter(today, 'date'); i++) {
|
||||
candidate = candidate.subtract(step, unit)
|
||||
amount += step
|
||||
}
|
||||
|
||||
return { amount, unit }
|
||||
return <div>
|
||||
{dates ?
|
||||
<div className='flex gap-2 items-center justify-between cursor-pointer' onClick={() => onClick(dates.fromDate, dates.toDate)}>
|
||||
<span className='text-sm text-ink-gray-5 max-w-[30%]'>
|
||||
{value}
|
||||
</span>
|
||||
{isEqual ? <span className='text-xs text-ink-gray-5 text-balance flex items-center gap-1'>
|
||||
{formatDate(dates.fromDate, 'Do MMM YYYY')}
|
||||
</span> :
|
||||
<span className='text-xs text-ink-gray-5 flex items-center gap-1'>
|
||||
{formatDate(dates.fromDate, 'Do MMM YY')} <ChevronRight size='16' className='text-ink-gray-5' /> {formatDate(dates.toDate, 'Do MMM YY')}
|
||||
</span>}
|
||||
</div> :
|
||||
<span className='text-sm text-ink-gray-5'>
|
||||
No results found
|
||||
</span>
|
||||
}
|
||||
</div>
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse free text into a past date range, or return undefined when it can't be parsed or
|
||||
* resolves entirely into the future.
|
||||
*/
|
||||
const parseDateRange = (value: string): { fromDate: string, toDate: string } | undefined => {
|
||||
if (!value.trim()) return undefined
|
||||
|
||||
const parsedDate = parse(value, undefined, { forwardDate: false })
|
||||
|
||||
if (!parsedDate || parsedDate.length === 0) return undefined
|
||||
|
||||
const result = parsedDate[0]
|
||||
const startKnownValues = knownValuesOf(result.start)
|
||||
|
||||
// Anchor the shift on the start and apply it to both ends, so an explicit range like
|
||||
// "1st Sept to 30th Sept" keeps its shape instead of having only its end rolled back.
|
||||
const shift = pastShift(result.start.date(), startKnownValues)
|
||||
const startDate = dayjs(result.start.date()).subtract(shift.amount, shift.unit).toDate()
|
||||
const endDate = result.end
|
||||
? dayjs(result.end.date()).subtract(shift.amount, shift.unit).toDate()
|
||||
: undefined
|
||||
|
||||
const today = new Date()
|
||||
let range: { fromDate: Date, toDate: Date }
|
||||
|
||||
if (endDate) {
|
||||
const endKnownValues = knownValuesOf(result.end)
|
||||
// chrono ends "Apr 2025 to Jun 2025" on the 1st of June, but the user means all of it.
|
||||
const rangeEnd = endKnownValues.month && !endKnownValues.day
|
||||
? dayjs(endDate).endOf('month').toDate()
|
||||
: endDate
|
||||
range = { fromDate: startDate, toDate: rangeEnd }
|
||||
} else if (startKnownValues.month && !startKnownValues.day) {
|
||||
// The user only wants a specific month like "May 2025" - span the whole month
|
||||
range = { fromDate: dayjs(startDate).startOf('month').toDate(), toDate: dayjs(startDate).endOf('month').toDate() }
|
||||
} else if (startKnownValues.month && startKnownValues.day && !referentialKeywords.some(keyword => value.toLowerCase().includes(keyword))) {
|
||||
// If month and day is known, then we should not assume that the user wants to get everything until today
|
||||
range = { fromDate: startDate, toDate: startDate }
|
||||
} else {
|
||||
range = { fromDate: startDate, toDate: today }
|
||||
}
|
||||
|
||||
// A range that hasn't started yet is never useful for reconciliation. A range that merely
|
||||
// ends in the future is kept as typed, the same way "This Month" spans the whole month.
|
||||
if (dayjs(range.fromDate).isAfter(today, 'date')) return undefined
|
||||
|
||||
if (dayjs(range.toDate).isBefore(range.fromDate, 'date')) {
|
||||
range = { fromDate: range.toDate, toDate: range.fromDate }
|
||||
}
|
||||
|
||||
return {
|
||||
fromDate: dayjs(range.fromDate).format(DATE_FORMAT),
|
||||
toDate: dayjs(range.toDate).format(DATE_FORMAT),
|
||||
}
|
||||
}
|
||||
|
||||
export default BankRecDateFilter
|
||||
export default BankRecDateFilter
|
||||
@@ -2,6 +2,7 @@ import { useAtomValue } from "jotai"
|
||||
import { MissingFiltersBanner } from "./MissingFiltersBanner"
|
||||
import { bankRecDateAtom, selectedBankAccountAtom } from "./bankRecAtoms"
|
||||
import { useCurrentCompany } from "@/hooks/useCurrentCompany"
|
||||
import { Paragraph } from "@/components/ui/typography"
|
||||
import { useCallback, useMemo } from "react"
|
||||
import type { ColumnDef } from "@tanstack/react-table"
|
||||
import { useFrappeGetCall } from "frappe-react-sdk"
|
||||
@@ -18,7 +19,6 @@ import _ from "@/lib/translate"
|
||||
import { toast } from "sonner"
|
||||
import { useCopyToClipboard } from "usehooks-ts"
|
||||
import { Empty, EmptyDescription, EmptyHeader, EmptyMedia, EmptyTitle } from "@/components/ui/empty"
|
||||
import MarkdownRenderer from "@/components/ui/markdown"
|
||||
|
||||
const BankReconciliationStatement = () => {
|
||||
const bankAccount = useAtomValue(selectedBankAccountAtom)
|
||||
@@ -189,30 +189,28 @@ const BankReconciliationStatementView = () => {
|
||||
return data.message.result.filter((row: BankClearanceSummaryEntry) => Boolean(row.payment_entry))
|
||||
}, [data])
|
||||
|
||||
const content = _("Below is a list of all entries posted against the bank account {0} which have not been cleared till {1}.", [`<strong>${bankAccount?.account}</strong>`, `<strong>${formatDate(dates.toDate)}</strong>`])
|
||||
return <div className="space-y-4 py-2">
|
||||
|
||||
return <div className="flex min-h-0 flex-1 flex-col space-y-4 py-2">
|
||||
|
||||
<div className="shrink-0">
|
||||
<span className="text-p-sm">
|
||||
<MarkdownRenderer content={content} />
|
||||
</span>
|
||||
<div>
|
||||
<Paragraph className="text-sm">
|
||||
<span dangerouslySetInnerHTML={{
|
||||
__html: _("Below is a list of all entries posted against the bank account {0} which have not been cleared till {1}.", [`<strong>${bankAccount?.account}</strong>`, `<strong>${formatDate(dates.toDate)}</strong>`])
|
||||
}} />
|
||||
</Paragraph>
|
||||
</div>
|
||||
|
||||
{error && <ErrorBanner error={error} />}
|
||||
|
||||
{data && <div className="shrink-0"><SummarySection data={data} /></div>}
|
||||
{data && <SummarySection data={data} />}
|
||||
|
||||
{data && data.message.result.length > 0 && (
|
||||
<div className="flex min-h-0 flex-1 flex-col space-y-2">
|
||||
<p className="shrink-0 text-ink-gray-5 text-sm">{_("Bank Reconciliation Statement")}</p>
|
||||
<div className="space-y-2">
|
||||
<p className="text-ink-gray-5 text-sm">{_("Bank Reconciliation Statement")}</p>
|
||||
<ListView
|
||||
data={statementRows}
|
||||
columns={statementColumns}
|
||||
getRowId={(row) => row.payment_entry}
|
||||
className="min-h-0 flex-1"
|
||||
maxHeight="none"
|
||||
scrollAreaClassName="flex-1"
|
||||
maxHeight="min(70vh, 640px)"
|
||||
emptyState={_("No entries with a payment document in this list.")}
|
||||
/>
|
||||
</div>
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { useAtomValue, useSetAtom } from "jotai"
|
||||
import { MissingFiltersBanner } from "./MissingFiltersBanner"
|
||||
import { bankRecDateAtom, bankRecUnreconcileModalAtom, selectedBankAccountAtom } from "./bankRecAtoms"
|
||||
import { Paragraph } from "@/components/ui/typography"
|
||||
import { formatDate } from "@/lib/date"
|
||||
import { ListView, type ListViewColumnMeta } from "@/components/ui/list-view"
|
||||
import { formatCurrency, getCurrencyFormatInfo } from "@/lib/numbers"
|
||||
@@ -22,7 +23,6 @@ import { useCallback, useMemo, useState } from "react"
|
||||
import { Link } from "react-router"
|
||||
import { Empty, EmptyTitle, EmptyHeader, EmptyMedia, EmptyDescription, EmptyContent } from "@/components/ui/empty"
|
||||
import { InputGroup, InputGroupAddon } from "@/components/ui/input-group"
|
||||
import MarkdownRenderer from "@/components/ui/markdown"
|
||||
|
||||
const BankTransactions = () => {
|
||||
const selectedBank = useAtomValue(selectedBankAccountAtom)
|
||||
@@ -243,14 +243,14 @@ const BankTransactionListView = () => {
|
||||
|
||||
}, [data, search, amountFilter, typeFilter, status])
|
||||
|
||||
const content = _("Below is a list of all bank transactions imported in the system for the bank account {0} between {1} and {2}.", [`<strong>${bankAccount?.account_name}</strong>`, `<strong>${formattedFromDate}</strong>`, `<strong>${formattedToDate}</strong>`])
|
||||
return <div className="space-y-2 py-2">
|
||||
|
||||
return <div className="flex min-h-0 flex-1 flex-col space-y-2 py-2">
|
||||
|
||||
<div className="flex shrink-0 gap-2 justify-between items-center">
|
||||
<span className="text-p-sm">
|
||||
<MarkdownRenderer content={content} />
|
||||
</span>
|
||||
<div className="flex gap-2 justify-between items-center">
|
||||
<Paragraph className="text-sm">
|
||||
<span dangerouslySetInnerHTML={{
|
||||
__html: _("Below is a list of all bank transactions imported in the system for the bank account {0} between {1} and {2}.", [`<strong>${bankAccount?.account_name}</strong>`, `<strong>${formattedFromDate}</strong>`, `<strong>${formattedToDate}</strong>`])
|
||||
}} />
|
||||
</Paragraph>
|
||||
|
||||
<Button size='md' variant='subtle' asChild>
|
||||
<Link to="/statement-importer">
|
||||
@@ -278,9 +278,8 @@ const BankTransactionListView = () => {
|
||||
data={filteredResults}
|
||||
columns={transactionColumns}
|
||||
getRowId={(row) => row.name}
|
||||
className="min-h-0 flex-1"
|
||||
maxHeight="none"
|
||||
scrollAreaClassName="flex-1"
|
||||
maxHeight="calc(100vh - 200px)"
|
||||
scrollAreaClassName="min-h-[calc(100vh-200px)]"
|
||||
emptyState={<Empty>
|
||||
<EmptyMedia>
|
||||
<ListIcon />
|
||||
|
||||
@@ -19,22 +19,13 @@ import {
|
||||
import { cn } from "@/lib/utils"
|
||||
import _ from "@/lib/translate"
|
||||
import { selectedBankAccountAtom } from "./bankRecAtoms"
|
||||
import { useFrappeGetDocList } from "frappe-react-sdk"
|
||||
import ErrorBanner from "@/components/ui/error-banner"
|
||||
|
||||
const CompanySelector = ({ onChange }: { onChange?: (company: string) => void }) => {
|
||||
const [open, setOpen] = useState(false)
|
||||
const [searchQuery, setSearchQuery] = useState("")
|
||||
|
||||
const { data: companies, error } = useFrappeGetDocList("Company", {
|
||||
limit: 0,
|
||||
fields: ["name"],
|
||||
}, 'company_list', {
|
||||
revalidateOnFocus: false,
|
||||
revalidateOnReconnect: false,
|
||||
})
|
||||
|
||||
const options = companies?.map((company: { name: string }) => company.name) || []
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
const options = window.frappe?.boot?.docs?.filter((doc: Record<string, any>) => doc.doctype === ":Company").map((company: Record<string, any>) => company.name) || []
|
||||
|
||||
const setSelectedCompany = useSetAtom(selectedCompanyAtom)
|
||||
const setSelectedBankAccount = useSetAtom(selectedBankAccountAtom)
|
||||
@@ -51,10 +42,6 @@ const CompanySelector = ({ onChange }: { onChange?: (company: string) => void })
|
||||
}
|
||||
}
|
||||
|
||||
if (error) {
|
||||
return <ErrorBanner error={error} />
|
||||
}
|
||||
|
||||
return (<Popover open={open} onOpenChange={setOpen}>
|
||||
<PopoverTrigger asChild>
|
||||
<Button
|
||||
|
||||
@@ -2,6 +2,7 @@ import { useAtomValue } from "jotai"
|
||||
import { MissingFiltersBanner } from "./MissingFiltersBanner"
|
||||
import { bankRecDateAtom, selectedBankAccountAtom } from "./bankRecAtoms"
|
||||
import { useCurrentCompany } from "@/hooks/useCurrentCompany"
|
||||
import { Paragraph } from "@/components/ui/typography"
|
||||
import type { ColumnDef } from "@tanstack/react-table"
|
||||
import { useCallback, useMemo } from "react"
|
||||
import { useFrappeGetCall, useFrappePostCall } from "frappe-react-sdk"
|
||||
@@ -17,7 +18,6 @@ import { PartyPopper } from "lucide-react"
|
||||
import ErrorBanner from "@/components/ui/error-banner"
|
||||
import _ from "@/lib/translate"
|
||||
import { Empty, EmptyTitle, EmptyDescription, EmptyMedia, EmptyHeader } from "@/components/ui/empty"
|
||||
import MarkdownRenderer from "@/components/ui/markdown"
|
||||
|
||||
const IncorrectlyClearedEntries = () => {
|
||||
const companyID = useCurrentCompany()
|
||||
@@ -177,36 +177,34 @@ const IncorrectlyClearedEntriesView = () => {
|
||||
[accountCurrency, onClearClick],
|
||||
)
|
||||
|
||||
const content = _("This report shows all entries in the system where the <strong>clearance date is before the posting date</strong> which is incorrect.")
|
||||
return <div className="space-y-4 py-2">
|
||||
|
||||
const entriesContent = _("Entries below have a posting date after {0} but the clearance date is before {1}.", [`<strong>${formattedToDate}</strong>`, `<strong>${formattedToDate}</strong>`])
|
||||
|
||||
return <div className="flex min-h-0 flex-1 flex-col space-y-4 py-2">
|
||||
|
||||
<div className="shrink-0">
|
||||
<span className="text-p-sm">
|
||||
<MarkdownRenderer content={content} />
|
||||
<div>
|
||||
<Paragraph className="text-sm">
|
||||
<span dangerouslySetInnerHTML={{
|
||||
__html: _("This report shows all entries in the system where the <strong>clearance date is before the posting date</strong> which is incorrect.")
|
||||
}} />
|
||||
<br />
|
||||
{data && data.message.result.length > 0 && <span>
|
||||
<MarkdownRenderer content={entriesContent} />
|
||||
<span dangerouslySetInnerHTML={{
|
||||
__html: _("Entries below have a posting date after {0} but the clearance date is before {1}.", [`<strong>${formattedToDate}</strong>`, `<strong>${formattedToDate}</strong>`])
|
||||
}} />
|
||||
<br />
|
||||
{_("You can reset the clearing dates of these entries here.")}
|
||||
</span>}
|
||||
</span>
|
||||
</Paragraph>
|
||||
</div>
|
||||
|
||||
{error && <ErrorBanner error={error} />}
|
||||
|
||||
{data && data.message.result.length > 0 && (
|
||||
<div className="flex min-h-0 flex-1 flex-col space-y-2">
|
||||
<p className="shrink-0 text-ink-gray-5 text-sm">{_("Incorrectly cleared entries as per the report.")}</p>
|
||||
<div className="space-y-2">
|
||||
<p className="text-ink-gray-5 text-sm">{_("Incorrectly cleared entries as per the report.")}</p>
|
||||
<ListView
|
||||
data={data.message.result}
|
||||
columns={incorrectlyClearedColumns}
|
||||
getRowId={(row) => `${row.payment_entry}-${row.posting_date}`}
|
||||
className="min-h-0 flex-1"
|
||||
maxHeight="none"
|
||||
scrollAreaClassName="flex-1"
|
||||
maxHeight="min(70vh, 640px)"
|
||||
emptyState={_("No rows to display.")}
|
||||
/>
|
||||
</div>
|
||||
|
||||
@@ -37,7 +37,7 @@ import { Link } from "react-router"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/ui/alert"
|
||||
import { InputGroup, InputGroupAddon, InputGroupText } from "@/components/ui/input-group"
|
||||
|
||||
const MatchAndReconcile = () => {
|
||||
const MatchAndReconcile = ({ contentHeight }: { contentHeight: number }) => {
|
||||
const selectedBank = useAtomValue(selectedBankAccountAtom)
|
||||
|
||||
if (!selectedBank) {
|
||||
@@ -52,15 +52,15 @@ const MatchAndReconcile = () => {
|
||||
}
|
||||
|
||||
return <>
|
||||
<div className="flex min-h-0 flex-1 items-stretch space-x-2" >
|
||||
<div className="flex min-h-0 flex-1 flex-col">
|
||||
<H4 className="shrink-0 text-sm font-medium">{_("Unreconciled Transactions")}</H4>
|
||||
<UnreconciledTransactions />
|
||||
<div className={`flex items-start space-x-2`} >
|
||||
<div className="flex-1">
|
||||
<H4 className="text-sm font-medium">{_("Unreconciled Transactions")}</H4>
|
||||
<UnreconciledTransactions contentHeight={contentHeight} />
|
||||
</div>
|
||||
<Separator orientation="vertical" className="self-stretch" />
|
||||
<div className="flex min-h-0 flex-1 flex-col px-1">
|
||||
<H4 className="shrink-0 text-sm font-medium">{_("Match or Create")}</H4>
|
||||
<VouchersSection />
|
||||
<Separator orientation="vertical" style={{ minHeight: `${contentHeight}px` }} />
|
||||
<div className="flex-1 px-1">
|
||||
<H4 className="text-sm font-medium">{_("Match or Create")}</H4>
|
||||
<VouchersSection contentHeight={contentHeight} />
|
||||
</div>
|
||||
</div>
|
||||
<TransferModal />
|
||||
@@ -69,19 +69,16 @@ const MatchAndReconcile = () => {
|
||||
</>
|
||||
}
|
||||
|
||||
/**
|
||||
* TanStack requires `estimateSize` for initial scroll range; `measureElement` on each row sets
|
||||
* the real height. The scroll container fills its flex parent rather than taking a pixel
|
||||
* height - the virtualizer observes its own rect, so it stays correct across resizes and any
|
||||
* layout change above it.
|
||||
*/
|
||||
/** TanStack requires `estimateSize` for initial scroll range; `measureElement` on each row sets the real height. */
|
||||
function VirtualizedListBody<T>({
|
||||
items,
|
||||
height,
|
||||
getItemKey,
|
||||
children,
|
||||
estimateSize = 74,
|
||||
}: {
|
||||
items: T[]
|
||||
height: number
|
||||
getItemKey: (item: T, index: number) => string | number
|
||||
children: (item: T, index: number) => React.ReactNode
|
||||
estimateSize?: number
|
||||
@@ -103,7 +100,8 @@ function VirtualizedListBody<T>({
|
||||
return (
|
||||
<div
|
||||
ref={scrollRef}
|
||||
className="min-h-0 flex-1 overflow-auto contain-strict"
|
||||
className="overflow-auto contain-strict"
|
||||
style={{ height }}
|
||||
>
|
||||
<div
|
||||
className="relative w-full"
|
||||
@@ -125,7 +123,7 @@ function VirtualizedListBody<T>({
|
||||
)
|
||||
}
|
||||
|
||||
const UnreconciledTransactions = () => {
|
||||
const UnreconciledTransactions = ({ contentHeight }: { contentHeight: number }) => {
|
||||
const bankAccount = useAtomValue(selectedBankAccountAtom)
|
||||
|
||||
const currency = bankAccount?.account_currency ?? getCompanyCurrency(bankAccount?.company ?? '')
|
||||
@@ -189,13 +187,14 @@ const UnreconciledTransactions = () => {
|
||||
}
|
||||
|
||||
const hasFilters = search !== '' || typeFilter !== 'All' || amountFilter.value !== 0
|
||||
const listHeight = contentHeight - 72
|
||||
|
||||
if (isLoading) {
|
||||
return <UnreconciledTransactionsLoadingState />
|
||||
}
|
||||
|
||||
return <div className="flex min-h-0 flex-1 flex-col space-y-1">
|
||||
<div className="flex py-2 w-full gap-2 shrink-0">
|
||||
return <div className="space-y-1">
|
||||
<div className="flex py-2 w-full gap-2">
|
||||
|
||||
<InputGroup variant='outline'>
|
||||
<label className="sr-only">{_("Search transactions")}</label>
|
||||
@@ -279,6 +278,7 @@ const UnreconciledTransactions = () => {
|
||||
|
||||
<VirtualizedListBody
|
||||
items={results}
|
||||
height={listHeight}
|
||||
estimateSize={74}
|
||||
getItemKey={(transaction) => transaction.name}
|
||||
>
|
||||
@@ -381,7 +381,7 @@ const UnreconciledTransactionItem = ({ transaction }: { transaction: Unreconcile
|
||||
}
|
||||
|
||||
|
||||
const VouchersSection = () => {
|
||||
const VouchersSection = ({ contentHeight }: { contentHeight: number }) => {
|
||||
|
||||
const selectedBank = useAtomValue(selectedBankAccountAtom)
|
||||
const selectedTransactions = useAtomValue(bankRecSelectedTransactionAtom(selectedBank?.name || ''))
|
||||
@@ -402,8 +402,8 @@ const VouchersSection = () => {
|
||||
return <OptionsForMultipleTransactions transactions={selectedTransactions} />
|
||||
}
|
||||
|
||||
return <div className="mt-2 flex min-h-0 flex-1 flex-col">
|
||||
<OptionsForSingleTransaction transaction={selectedTransactions[0]} />
|
||||
return <div style={{ minHeight: contentHeight }} className="mt-2">
|
||||
<OptionsForSingleTransaction transaction={selectedTransactions[0]} contentHeight={contentHeight} />
|
||||
</div>
|
||||
}
|
||||
|
||||
@@ -535,11 +535,11 @@ const OptionsForMultipleTransactions = ({ transactions }: { transactions: Unreco
|
||||
}
|
||||
|
||||
|
||||
const OptionsForSingleTransaction = ({ transaction }: { transaction: UnreconciledTransaction }) => {
|
||||
const OptionsForSingleTransaction = ({ transaction, contentHeight }: { transaction: UnreconciledTransaction, contentHeight: number }) => {
|
||||
|
||||
const { setTransferModalOpen, setRecordPaymentModalOpen, setRecordJournalEntryModalOpen } = useKeyboardShortcuts()
|
||||
|
||||
return <div className="flex min-h-0 flex-1 flex-col gap-3">
|
||||
return <div className="flex flex-col gap-3">
|
||||
<TooltipProvider>
|
||||
<div className="flex items-center justify-between pt-2">
|
||||
<div className="flex gap-4 justify-center">
|
||||
@@ -602,7 +602,7 @@ const OptionsForSingleTransaction = ({ transaction }: { transaction: Unreconcile
|
||||
</div>
|
||||
</TooltipProvider>
|
||||
{transaction.matched_transaction_rule && <RuleAction transaction={transaction} />}
|
||||
<VouchersForTransaction transaction={transaction} />
|
||||
<VouchersForTransaction transaction={transaction} contentHeight={contentHeight} />
|
||||
</div>
|
||||
}
|
||||
|
||||
@@ -774,11 +774,12 @@ const RuleAction = ({ transaction }: { transaction: UnreconciledTransaction }) =
|
||||
)
|
||||
}
|
||||
|
||||
const VouchersForTransaction = ({ transaction }: { transaction: UnreconciledTransaction }) => {
|
||||
const VouchersForTransaction = ({ transaction, contentHeight }: { transaction: UnreconciledTransaction, contentHeight: number }) => {
|
||||
|
||||
const { data: vouchers, isLoading, error } = useGetVouchersForTransaction(transaction)
|
||||
|
||||
const voucherList = vouchers?.message ?? []
|
||||
const listHeight = contentHeight - 120
|
||||
|
||||
if (error) {
|
||||
return <ErrorBanner error={error} />
|
||||
@@ -800,8 +801,8 @@ const VouchersForTransaction = ({ transaction }: { transaction: UnreconciledTran
|
||||
</div>
|
||||
}
|
||||
|
||||
return <div className="relative flex min-h-0 flex-1 flex-col space-y-2">
|
||||
<div className="flex shrink-0 items-center gap-2 text-sm text-ink-gray-5">
|
||||
return <div className="relative space-y-2">
|
||||
<div className="flex items-center gap-2 text-sm text-ink-gray-5">
|
||||
<Separator className="flex-1" />
|
||||
<span>or</span>
|
||||
<Separator className="flex-1" />
|
||||
@@ -817,6 +818,7 @@ const VouchersForTransaction = ({ transaction }: { transaction: UnreconciledTran
|
||||
</Empty>}
|
||||
<VirtualizedListBody
|
||||
items={voucherList}
|
||||
height={listHeight}
|
||||
estimateSize={121}
|
||||
getItemKey={(voucher) => voucher.name}
|
||||
>
|
||||
|
||||
@@ -11,7 +11,6 @@ import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "@
|
||||
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip"
|
||||
import { H4, Paragraph } from "@/components/ui/typography"
|
||||
import { today } from "@/lib/date"
|
||||
import { evaluateAmountFormula } from "@/lib/amountFormula"
|
||||
import _ from "@/lib/translate"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { BankTransactionRule } from "@/types/Accounts/BankTransactionRule"
|
||||
@@ -446,10 +445,11 @@ const AmountFormulaRenderer = ({ value }: { value?: string }) => {
|
||||
// If it's a string and cannot be a number, then show it as a formula
|
||||
|
||||
if (isNaN(Number(value))) {
|
||||
|
||||
let calculatedValue = "";
|
||||
|
||||
try {
|
||||
calculatedValue = String(evaluateAmountFormula(value ?? "", 200));
|
||||
calculatedValue = window.eval(`const transaction_amount = 200; ${value}`);
|
||||
} catch (error: unknown) {
|
||||
console.error(error);
|
||||
calculatedValue = "Error";
|
||||
|
||||
@@ -59,8 +59,8 @@ const SelectedTransactionDetails = ({ transaction, showAccount = false, account
|
||||
</div>
|
||||
</div>
|
||||
<div className='flex flex-col gap-1'>
|
||||
<span className='text-p-sm'>{transaction.description}</span>
|
||||
{transaction.reference_number ? <span className='text-p-sm text-ink-gray-5'>{_("Ref")}: {transaction.reference_number}</span> : null}
|
||||
<span className='text-sm'>{transaction.description}</span>
|
||||
{transaction.reference_number ? <span className='text-sm text-ink-gray-5'>{_("Ref")}: {transaction.reference_number}</span> : null}
|
||||
{showAccount && account ? <span className='text-sm text-ink-gray-5'>{_("GL Account")}: {account}</span> : null}
|
||||
</div>
|
||||
|
||||
|
||||
@@ -490,7 +490,7 @@ const RecommendedTransferAccount = ({ transaction, onAccountChange }: { transact
|
||||
<Calendar size='16px' />
|
||||
<span className='text-sm'>{formatDate(data.message.date, 'Do MMM YYYY')}</span>
|
||||
</div>
|
||||
<span className='text-p-sm line-clamp-1' title={data.message.description}>{data.message.description}</span>
|
||||
<span className='text-sm line-clamp-1' title={data.message.description}>{data.message.description}</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -231,7 +231,7 @@ export const BANK_LOGOS: { keywords: string[], logo: string, locale?: string[],
|
||||
{
|
||||
keywords: ['Federal Bank'],
|
||||
logo: 'Federal_Bank.png',
|
||||
logoDark: 'Federal_Bank-Dark.png',
|
||||
logoDark: 'Federal_Bank-dark.png',
|
||||
locale: ['India']
|
||||
},
|
||||
{
|
||||
|
||||
@@ -14,7 +14,7 @@ import { Tooltip, TooltipContent, TooltipTrigger } from '@/components/ui/tooltip
|
||||
import { useFrappeEventListener, useFrappePostCall } from 'frappe-react-sdk'
|
||||
import { toast } from 'sonner'
|
||||
import ErrorBanner from '@/components/ui/error-banner'
|
||||
import { Link, useNavigate } from 'react-router'
|
||||
import { Link, useNavigate } from 'react-router-dom'
|
||||
import { useMemo, useState } from 'react'
|
||||
import { Progress } from '@/components/ui/progress'
|
||||
import { useSetAtom } from 'jotai'
|
||||
@@ -83,13 +83,10 @@ const StatementDetails = ({ data }: Props) => {
|
||||
|
||||
}
|
||||
|
||||
// `progress` is a percentage (drives the bar); `current`/`total` are actual counts.
|
||||
const [progress, setProgress] = useState(0)
|
||||
const [imported, setImported] = useState({ current: 0, total: 0 })
|
||||
|
||||
useFrappeEventListener("bank-rec-statement-import-progress", (event) => {
|
||||
setProgress(event.progress)
|
||||
setImported({ current: event.current ?? 0, total: event.total ?? 0 })
|
||||
})
|
||||
|
||||
const file_name = data.doc.file.split("/").pop() ?? ""
|
||||
@@ -115,9 +112,7 @@ const StatementDetails = ({ data }: Props) => {
|
||||
{data.doc.status === 'Completed' ? <Badge theme='green'>{_("Completed")}</Badge> :
|
||||
<Button onClick={onImport} disabled={loading || data.final_transactions?.length === 0} size='sm' type='button'>
|
||||
{loading ? <Loader2Icon className='size-4 animate-spin' /> : null}
|
||||
{loading ? _("Importing...") : data.final_transactions?.length === 1
|
||||
? _("Import 1 transaction")
|
||||
: _("Import {0} transactions", [data.final_transactions?.length?.toString() || "0"])}</Button>
|
||||
{loading ? _("Importing...") : _("Import {0} transactions", [data.final_transactions?.length?.toString() || "0"])}</Button>
|
||||
}
|
||||
</div>
|
||||
<div className='flex items-start gap-4'>
|
||||
@@ -134,9 +129,7 @@ const StatementDetails = ({ data }: Props) => {
|
||||
</div>
|
||||
|
||||
{progress > 0 && <div className='flex flex-col gap-2'><Progress value={progress} max={100} size="lg" />
|
||||
<span className='text-sm'>{imported.total === 1
|
||||
? _("Importing 1 transaction")
|
||||
: _("Importing {0} of {1} transactions", [imported.current.toString(), imported.total.toString()])}
|
||||
<span className='text-sm'>{_("Importing {0} transactions", [progress.toString()])}
|
||||
</span>
|
||||
</div>}
|
||||
|
||||
|
||||
@@ -387,7 +387,7 @@ function ListViewInner<TData>({
|
||||
)}
|
||||
role="columnheader"
|
||||
>
|
||||
<div className="min-w-0 flex-1 truncate leading-snug">
|
||||
<div className="min-w-0 flex-1 truncate">
|
||||
{header.isPlaceholder
|
||||
? null
|
||||
: flexRender(header.column.columnDef.header, header.getContext())}
|
||||
|
||||
@@ -1,9 +1,7 @@
|
||||
import { useAtomValue } from "jotai"
|
||||
import { atomWithStorage } from "jotai/utils"
|
||||
|
||||
export const selectedCompanyAtom = atomWithStorage<string>('bank-rec-selected-company', window.frappe?.boot?.user?.defaults?.company || '', undefined, {
|
||||
getOnInit: true,
|
||||
})
|
||||
export const selectedCompanyAtom = atomWithStorage<string>('bank-rec-selected-company', window.frappe?.boot?.user?.defaults?.company || '')
|
||||
|
||||
export const useCurrentCompany = () => {
|
||||
const selectedCompany = useAtomValue(selectedCompanyAtom)
|
||||
|
||||
@@ -1,58 +1,13 @@
|
||||
import { useFrappeGetCall } from "frappe-react-sdk"
|
||||
import { useMemo } from "react"
|
||||
import dayjs from "dayjs"
|
||||
import { useCurrentCompany } from "./useCurrentCompany"
|
||||
|
||||
export type FiscalYear = {
|
||||
name: string
|
||||
year_start_date: string
|
||||
year_end_date: string
|
||||
}
|
||||
|
||||
/**
|
||||
* The fiscal year containing today, for the currently selected company.
|
||||
*
|
||||
* `company` matters in multi-company setups, where fiscal years can be restricted to
|
||||
* specific companies. `date` matters because without it `get_fiscal_year` returns the newest
|
||||
* fiscal year in the system (they're ordered by start date, descending) - which may be one
|
||||
* created in advance for a year that hasn't started.
|
||||
*/
|
||||
const useFiscalYear = () => {
|
||||
const company = useCurrentCompany()
|
||||
|
||||
const { data, ...rest } = useFrappeGetCall<{ message: FiscalYear | [string, string, string] | false }>(
|
||||
"erpnext.accounts.utils.get_fiscal_year",
|
||||
{
|
||||
date: dayjs().format("YYYY-MM-DD"),
|
||||
company,
|
||||
as_dict: 1,
|
||||
// Return nothing instead of throwing/msgprinting when no fiscal year covers today.
|
||||
raise_on_missing: 0,
|
||||
verbose: 0,
|
||||
},
|
||||
company ? `fiscal_year_${company}` : null,
|
||||
{
|
||||
revalidateOnFocus: false,
|
||||
revalidateIfStale: false,
|
||||
revalidateOnReconnect: false
|
||||
}
|
||||
)
|
||||
return useFrappeGetCall("erpnext.accounts.utils.get_fiscal_year", undefined, 'fiscal_year', {
|
||||
revalidateOnFocus: false,
|
||||
revalidateIfStale: false,
|
||||
revalidateOnReconnect: false
|
||||
})
|
||||
|
||||
// get_fiscal_year returns a dict with as_dict, a (name, start, end) tuple without it, and
|
||||
// false when there's no match - normalise all three.
|
||||
const fiscalYear = useMemo<FiscalYear | undefined>(() => {
|
||||
const message = data?.message
|
||||
if (!message) return undefined
|
||||
|
||||
if (Array.isArray(message)) {
|
||||
const [name, year_start_date, year_end_date] = message
|
||||
return { name, year_start_date, year_end_date }
|
||||
}
|
||||
|
||||
return message
|
||||
}, [data])
|
||||
|
||||
return { fiscalYear, ...rest }
|
||||
}
|
||||
|
||||
export default useFiscalYear
|
||||
export default useFiscalYear
|
||||
@@ -1,23 +0,0 @@
|
||||
import { useLayoutEffect, useRef } from "react"
|
||||
|
||||
/**
|
||||
* Pins a scrollable list back to the top whenever the search term changes.
|
||||
*
|
||||
* Dropdowns that do their own filtering (`shouldFilter={false}`) swap a long list for a much
|
||||
* shorter one while the scroll container keeps its previous offset - which can leave the
|
||||
* auto-selected first item scrolled out of view.
|
||||
*
|
||||
* Returns a ref to attach to the scroll container (e.g. `CommandList`).
|
||||
*/
|
||||
const useResetScrollOnSearch = (search: string) => {
|
||||
const listRef = useRef<HTMLDivElement>(null)
|
||||
|
||||
// Layout effect so the reset lands before paint, avoiding a visible jump.
|
||||
useLayoutEffect(() => {
|
||||
listRef.current?.scrollTo({ top: 0 })
|
||||
}, [search])
|
||||
|
||||
return listRef
|
||||
}
|
||||
|
||||
export default useResetScrollOnSearch
|
||||
@@ -1,6 +1,5 @@
|
||||
@import "tailwindcss";
|
||||
@import "tw-animate-css";
|
||||
@import "./styles/scroll-fade.css";
|
||||
|
||||
@font-face {
|
||||
font-family: InterVariable;
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
import { Parser } from 'safe-expr-eval'
|
||||
|
||||
const parser = new Parser()
|
||||
|
||||
const PLAIN_NUMBER_PATTERN = /^-?\d+(\.\d+)?$/
|
||||
|
||||
export function evaluateAmountFormula(expression: string, transactionAmount: number): number {
|
||||
const trimmed = expression.trim()
|
||||
if (!trimmed) {
|
||||
return 0
|
||||
}
|
||||
|
||||
if (PLAIN_NUMBER_PATTERN.test(trimmed)) {
|
||||
return Number(trimmed)
|
||||
}
|
||||
|
||||
try {
|
||||
const result = parser.parse(trimmed).evaluate({ transaction_amount: transactionAmount })
|
||||
if (typeof result !== 'number' || !Number.isFinite(result)) {
|
||||
return 0
|
||||
}
|
||||
return result
|
||||
} catch {
|
||||
return 0
|
||||
}
|
||||
}
|
||||
@@ -33,16 +33,6 @@ export const getErrorMessages = (error?: FrappeError | null): ParsedErrorMessage
|
||||
}
|
||||
})
|
||||
|
||||
// @ts-expect-error - some errors have _error_message
|
||||
if (error?._error_message) {
|
||||
eMessages.push({
|
||||
// @ts-expect-error - some errors have _error_message
|
||||
message: error?._error_message,
|
||||
title: "Error",
|
||||
indicator: "red"
|
||||
})
|
||||
}
|
||||
|
||||
if (eMessages.length === 0) {
|
||||
// Get the message from the exception by removing the exc_type
|
||||
const indexOfFirstColon = error?.exception?.indexOf(':')
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import BankAccountBalancePanel from "@/components/features/BankReconciliation/BankBalance"
|
||||
import BankBalance from "@/components/features/BankReconciliation/BankBalance"
|
||||
import BankPicker from "@/components/features/BankReconciliation/BankPicker"
|
||||
import BankRecDateFilter from "@/components/features/BankReconciliation/BankRecDateFilter"
|
||||
import BankTransactionUnreconcileModal from "@/components/features/BankReconciliation/BankTransactionUnreconcileModal"
|
||||
@@ -9,9 +9,10 @@ import ActionLog from "@/components/features/ActionLog/ActionLog"
|
||||
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs"
|
||||
import { TooltipProvider } from "@/components/ui/tooltip"
|
||||
import _ from "@/lib/translate"
|
||||
import { lazy, Suspense } from "react"
|
||||
import { lazy, Suspense, useLayoutEffect, useRef, useState } from "react"
|
||||
import { AlertTriangleIcon, CheckCircleIcon, HomeIcon, LandmarkIcon, ListIcon, Loader2Icon, ScrollTextIcon, ShuffleIcon } from "lucide-react"
|
||||
import { Breadcrumb, BreadcrumbItem, BreadcrumbList, BreadcrumbPage, BreadcrumbSeparator } from "@/components/ui/breadcrumb"
|
||||
import { Badge } from "@/components/ui/badge"
|
||||
import { Empty, EmptyContent, EmptyDescription, EmptyHeader, EmptyMedia, EmptyTitle } from "@/components/ui/empty"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { useAtomValue } from "jotai"
|
||||
@@ -24,13 +25,23 @@ const IncorrectlyClearedEntries = lazy(() => import('@/components/features/BankR
|
||||
|
||||
const BankReconciliation = () => {
|
||||
|
||||
const [headerHeight, setHeaderHeight] = useState(0)
|
||||
|
||||
const ref = useRef<HTMLDivElement>(null)
|
||||
|
||||
useLayoutEffect(() => {
|
||||
if (ref.current) {
|
||||
setHeaderHeight(ref.current.clientHeight)
|
||||
}
|
||||
}, [])
|
||||
|
||||
const remainingHeightAfterTabs = window.innerHeight - headerHeight - 220
|
||||
|
||||
return (
|
||||
<div>
|
||||
{/* The page owns the viewport height and the tabs/lists below fill what's left, so
|
||||
the virtualizers size themselves from layout instead of a measured pixel value. */}
|
||||
<div className="px-2 pt-1 flex-col gap-4 md:flex hidden h-dvh">
|
||||
<div className="flex flex-col gap-4 shrink-0">
|
||||
<div className="flex justify-between shrink-0">
|
||||
<div className="p-4 flex-col gap-4 md:flex hidden">
|
||||
<div ref={ref} className="flex flex-col gap-4">
|
||||
<div className="flex justify-between">
|
||||
<div className="flex items-center gap-6">
|
||||
<Breadcrumb>
|
||||
<BreadcrumbList>
|
||||
@@ -43,7 +54,7 @@ const BankReconciliation = () => {
|
||||
<BreadcrumbItem>
|
||||
<BreadcrumbPage>
|
||||
<div className="flex gap-1 items-center">
|
||||
{_("Banking")}
|
||||
{_("Banking")} <Badge theme="violet" variant="subtle">{_("Beta")}</Badge>
|
||||
</div>
|
||||
|
||||
</BreadcrumbPage>
|
||||
@@ -60,8 +71,10 @@ const BankReconciliation = () => {
|
||||
<BankRecDateFilter />
|
||||
</div>
|
||||
</div>
|
||||
<BankPicker />
|
||||
<BankBalance />
|
||||
</div>
|
||||
<BankRecWorkspace />
|
||||
<BankRecTabs remainingHeightAfterTabs={remainingHeightAfterTabs} />
|
||||
<BankTransactionUnreconcileModal />
|
||||
</div>
|
||||
<div className="md:hidden flex h-screen items-center justify-between">
|
||||
@@ -91,53 +104,42 @@ const BankReconciliation = () => {
|
||||
)
|
||||
}
|
||||
|
||||
const BankRecWorkspace = () => {
|
||||
const BankRecTabs = ({ remainingHeightAfterTabs }: { remainingHeightAfterTabs: number }) => {
|
||||
const selectedBankAccount = useAtomValue(selectedBankAccountAtom)
|
||||
|
||||
return <Tabs defaultValue="Match and Reconcile" className="min-h-0 flex-1 gap-4">
|
||||
{/* Picker + tab strip stack on the left, balance panel beside them - the tab strip
|
||||
fills height the panel needs anyway, so it costs no row of its own. The picker
|
||||
scrolls horizontally (`min-w-0` lets it shrink so its overflow-x engages) while
|
||||
the panel stays put, so the figures never scroll away. */}
|
||||
{/* No gap here: the panel's own `border-s ps-4` supplies the separation, and a gap
|
||||
would leave dead space the picker's edge fade can't reach. */}
|
||||
<div className="flex shrink-0 items-stretch">
|
||||
<div className="flex min-w-0 flex-1 flex-col justify-between gap-3">
|
||||
<BankPicker />
|
||||
{selectedBankAccount && <TabsList>
|
||||
<TabsTrigger value="Match and Reconcile"><ShuffleIcon /> {_("Match and Reconcile")}</TabsTrigger>
|
||||
<TabsTrigger value="Bank Reconciliation Statement"><ScrollTextIcon /> {_("Reconciliation Statement")}</TabsTrigger>
|
||||
<TabsTrigger value="Bank Transactions"><ListIcon />{_("Transactions")}</TabsTrigger>
|
||||
<TabsTrigger value="Bank Clearance Summary"><CheckCircleIcon />{_("Clearance Summary")}</TabsTrigger>
|
||||
<TabsTrigger value="Incorrectly Cleared Entries"><AlertTriangleIcon /> {_("Incorrectly Cleared")}</TabsTrigger>
|
||||
</TabsList>}
|
||||
</div>
|
||||
{selectedBankAccount && <BankAccountBalancePanel />}
|
||||
</div>
|
||||
if (!selectedBankAccount) {
|
||||
return null
|
||||
}
|
||||
|
||||
{selectedBankAccount && <>
|
||||
<TabsContent value="Match and Reconcile" className="flex min-h-0 flex-col">
|
||||
<MatchAndReconcile />
|
||||
return <Tabs defaultValue="Match and Reconcile">
|
||||
<TabsList>
|
||||
<TabsTrigger value="Match and Reconcile"><ShuffleIcon /> {_("Match and Reconcile")}</TabsTrigger>
|
||||
<TabsTrigger value="Bank Reconciliation Statement"><ScrollTextIcon /> {_("Bank Reconciliation Statement")}</TabsTrigger>
|
||||
<TabsTrigger value="Bank Transactions"><ListIcon />{_("Bank Transactions")}</TabsTrigger>
|
||||
<TabsTrigger value="Bank Clearance Summary"><CheckCircleIcon />{_("Bank Clearance Summary")}</TabsTrigger>
|
||||
<TabsTrigger value="Incorrectly Cleared Entries"><AlertTriangleIcon /> {_("Incorrectly Cleared Entries")}</TabsTrigger>
|
||||
</TabsList>
|
||||
<TabsContent value="Match and Reconcile">
|
||||
<MatchAndReconcile contentHeight={remainingHeightAfterTabs} />
|
||||
</TabsContent>
|
||||
<Suspense fallback={
|
||||
<div className="flex items-center justify-center p-16">
|
||||
<Loader2Icon className="size-6 animate-spin text-muted-foreground" />
|
||||
</div>
|
||||
}>
|
||||
<TabsContent value="Bank Reconciliation Statement">
|
||||
<BankReconciliationStatement />
|
||||
</TabsContent>
|
||||
<Suspense fallback={
|
||||
<div className="flex items-center justify-center p-16">
|
||||
<Loader2Icon className="size-6 animate-spin text-muted-foreground" />
|
||||
</div>
|
||||
}>
|
||||
<TabsContent value="Bank Reconciliation Statement" className="flex min-h-0 flex-col">
|
||||
<BankReconciliationStatement />
|
||||
</TabsContent>
|
||||
<TabsContent value="Bank Transactions" className="flex min-h-0 flex-col">
|
||||
<BankTransactions />
|
||||
</TabsContent>
|
||||
<TabsContent value="Bank Clearance Summary" className="flex min-h-0 flex-col">
|
||||
<BankClearanceSummary />
|
||||
</TabsContent>
|
||||
<TabsContent value="Incorrectly Cleared Entries" className="flex min-h-0 flex-col">
|
||||
<IncorrectlyClearedEntries />
|
||||
</TabsContent>
|
||||
</Suspense>
|
||||
</>}
|
||||
<TabsContent value="Bank Transactions">
|
||||
<BankTransactions />
|
||||
</TabsContent>
|
||||
<TabsContent value="Bank Clearance Summary">
|
||||
<BankClearanceSummary />
|
||||
</TabsContent>
|
||||
<TabsContent value="Incorrectly Cleared Entries">
|
||||
<IncorrectlyClearedEntries />
|
||||
</TabsContent>
|
||||
</Suspense>
|
||||
</Tabs>
|
||||
}
|
||||
|
||||
|
||||
@@ -226,7 +226,7 @@ const StatementImportLog = () => {
|
||||
field: "creation",
|
||||
order: "desc"
|
||||
},
|
||||
limit: 20
|
||||
limit: 10
|
||||
}, bankAccount ? undefined : null, {
|
||||
revalidateOnFocus: false
|
||||
})
|
||||
|
||||
@@ -1,94 +0,0 @@
|
||||
/* Scroll-edge fade mask for horizontal scroll containers (the bank picker strip).
|
||||
Ported from Raven's `scroll-fade-x`; imported by index.css, since Tailwind processes
|
||||
`@utility` in imported files the same as in the entry file.
|
||||
|
||||
The scroll-timeline keyframes reveal each edge's fade only when there IS content to scroll
|
||||
in that direction - no fade on the left edge when scrolled fully left, none on the right at
|
||||
the end. `@property` makes the fade animate smoothly rather than jumping.
|
||||
|
||||
Without scroll-timeline support (Firefox) there is deliberately NO fade at all: the fade
|
||||
vars stay at their 0px initial value and the gradient stops collapse to the edges. A static
|
||||
both-edges fallback was tried in Raven and removed - on a container with nothing to scroll
|
||||
it dimmed the edges anyway, promising content that didn't exist. */
|
||||
|
||||
@property --scroll-fade-l {
|
||||
/* length-percentage, NOT length: the fade size is min(12%, …) - a percentage. A <length>
|
||||
property rejects that value and reverts to initial-value (0px), zeroing the fade. */
|
||||
syntax: "<length-percentage>";
|
||||
inherits: false;
|
||||
initial-value: 0px;
|
||||
}
|
||||
|
||||
@property --scroll-fade-r {
|
||||
syntax: "<length-percentage>";
|
||||
inherits: false;
|
||||
initial-value: 0px;
|
||||
}
|
||||
|
||||
@keyframes scroll-fade-reveal-l {
|
||||
from {
|
||||
--scroll-fade-l: 0px;
|
||||
}
|
||||
|
||||
to {
|
||||
--scroll-fade-l: var(--_scroll-fade-size-l);
|
||||
}
|
||||
}
|
||||
|
||||
@keyframes scroll-fade-reveal-r {
|
||||
from {
|
||||
--scroll-fade-r: var(--_scroll-fade-size-r);
|
||||
}
|
||||
|
||||
to {
|
||||
--scroll-fade-r: 0px;
|
||||
}
|
||||
}
|
||||
|
||||
@utility scroll-fade-x {
|
||||
--_scroll-fade-size-l: var(--scroll-fade-l-size,
|
||||
var(--scroll-fade-size, min(12%, calc(var(--spacing, 0.25rem) * 10))));
|
||||
--_scroll-fade-size-r: var(--scroll-fade-r-size,
|
||||
var(--scroll-fade-size, min(12%, calc(var(--spacing, 0.25rem) * 10))));
|
||||
/* Eased (smoothstep) alpha ramp, sampled finely so it reads as a smooth curve, NOT fading
|
||||
all the way to transparent: the edge floors at 0.25 (content dims, never vanishes), ramping
|
||||
up to a full 1 for the body. The opaque end MUST be 1 or everything would be permanently
|
||||
dimmed. Stops collapse to the edge when the size animates to 0, so the true first/last card
|
||||
is never dimmed at rest. Tune the floor - higher (~0.4) = subtler, lower (~0.1) = stronger. */
|
||||
--scroll-fade-inline: linear-gradient(to right,
|
||||
rgba(0, 0, 0, 0.25) 0,
|
||||
rgba(0, 0, 0, 0.282) calc(var(--scroll-fade-l, 0px) * 0.125),
|
||||
rgba(0, 0, 0, 0.367) calc(var(--scroll-fade-l, 0px) * 0.25),
|
||||
rgba(0, 0, 0, 0.487) calc(var(--scroll-fade-l, 0px) * 0.375),
|
||||
rgba(0, 0, 0, 0.625) calc(var(--scroll-fade-l, 0px) * 0.5),
|
||||
rgba(0, 0, 0, 0.763) calc(var(--scroll-fade-l, 0px) * 0.625),
|
||||
rgba(0, 0, 0, 0.883) calc(var(--scroll-fade-l, 0px) * 0.75),
|
||||
rgba(0, 0, 0, 0.968) calc(var(--scroll-fade-l, 0px) * 0.875),
|
||||
rgba(0, 0, 0, 1) var(--scroll-fade-l, 0px),
|
||||
rgba(0, 0, 0, 1) calc(100% - var(--scroll-fade-r, 0px)),
|
||||
rgba(0, 0, 0, 0.968) calc(100% - var(--scroll-fade-r, 0px) * 0.875),
|
||||
rgba(0, 0, 0, 0.883) calc(100% - var(--scroll-fade-r, 0px) * 0.75),
|
||||
rgba(0, 0, 0, 0.763) calc(100% - var(--scroll-fade-r, 0px) * 0.625),
|
||||
rgba(0, 0, 0, 0.625) calc(100% - var(--scroll-fade-r, 0px) * 0.5),
|
||||
rgba(0, 0, 0, 0.487) calc(100% - var(--scroll-fade-r, 0px) * 0.375),
|
||||
rgba(0, 0, 0, 0.367) calc(100% - var(--scroll-fade-r, 0px) * 0.25),
|
||||
rgba(0, 0, 0, 0.282) calc(100% - var(--scroll-fade-r, 0px) * 0.125),
|
||||
rgba(0, 0, 0, 0.25) 100%);
|
||||
-webkit-mask-image: var(--scroll-fade-mask, var(--scroll-fade-inline));
|
||||
mask-image: var(--scroll-fade-mask, var(--scroll-fade-inline));
|
||||
-webkit-mask-composite: source-in;
|
||||
mask-composite: intersect;
|
||||
-webkit-mask-repeat: no-repeat;
|
||||
mask-repeat: no-repeat;
|
||||
|
||||
@supports (animation-timeline: scroll()) {
|
||||
animation:
|
||||
scroll-fade-reveal-l 1ms ease-in-out,
|
||||
scroll-fade-reveal-r 1ms ease-in-out;
|
||||
animation-timeline: scroll(self x), scroll(self x);
|
||||
animation-range:
|
||||
0 var(--scroll-fade-reveal, calc(var(--spacing, 0.25rem) * 24)),
|
||||
calc(100% - var(--scroll-fade-reveal, calc(var(--spacing, 0.25rem) * 24))) 100%;
|
||||
animation-fill-mode: both;
|
||||
}
|
||||
}
|
||||
2904
banking/yarn.lock
2904
banking/yarn.lock
File diff suppressed because it is too large
Load Diff
13
crowdin.yml
13
crowdin.yml
@@ -1,5 +1,14 @@
|
||||
preserve_hierarchy: true
|
||||
|
||||
files:
|
||||
- source: /erpnext/locale/main.pot
|
||||
translation: /erpnext/locale/%two_letters_code%.po
|
||||
pull_request_title: "fix: sync translations from crowdin"
|
||||
pull_request_labels:
|
||||
- translation
|
||||
- skip-release-notes
|
||||
pull_request_reviewers:
|
||||
- barredterra # change to your GitHub username if you copied this file
|
||||
commit_message: "fix: %language% translations"
|
||||
append_commit_message: false
|
||||
languages_mapping:
|
||||
two_letters_code:
|
||||
pt-BR: pt_BR
|
||||
|
||||
@@ -3,6 +3,8 @@ import inspect
|
||||
from typing import TypeVar
|
||||
|
||||
import frappe
|
||||
from frappe.model.document import Document
|
||||
from frappe.utils.user import is_website_user
|
||||
|
||||
__version__ = "17.0.0-dev"
|
||||
|
||||
@@ -153,8 +155,6 @@ def allow_regional(fn):
|
||||
|
||||
|
||||
def check_app_permission():
|
||||
from frappe.utils.user import is_website_user
|
||||
|
||||
if frappe.session.user == "Administrator":
|
||||
return True
|
||||
|
||||
@@ -175,16 +175,9 @@ def normalize_ctx_input(T: type) -> callable:
|
||||
- Casting the result to the specified type T
|
||||
"""
|
||||
|
||||
from frappe.model.document import Document
|
||||
|
||||
def decorator(func: callable):
|
||||
# conserve annotations for frappe.utils.typing_validations
|
||||
@functools.wraps(
|
||||
func,
|
||||
assigned=(
|
||||
a for a in functools.WRAPPER_ASSIGNMENTS if a not in ("__annotations__", "__annotate__")
|
||||
),
|
||||
)
|
||||
@functools.wraps(func, assigned=(a for a in functools.WRAPPER_ASSIGNMENTS if a != "__annotations__"))
|
||||
def wrapper(ctx: T | Document | dict | str, *args, **kwargs):
|
||||
if isinstance(ctx, Document):
|
||||
ctx = T(**ctx.as_dict())
|
||||
|
||||
@@ -1,190 +0,0 @@
|
||||
import frappe
|
||||
from frappe import _
|
||||
from frappe.utils import flt
|
||||
|
||||
from erpnext.accounts.doctype.payment_entry.payment_entry import (
|
||||
get_outstanding_reference_documents,
|
||||
get_payment_entry,
|
||||
)
|
||||
|
||||
|
||||
@frappe.whitelist(methods=["POST"])
|
||||
def create_payment_entries(invoices: str | list | None = None):
|
||||
"""Create draft Payment Entries from AP report invoice selection."""
|
||||
frappe.has_permission("Payment Entry", "create", throw=True)
|
||||
|
||||
names = [d["voucher_no"] for d in frappe.parse_json(invoices or "[]") if d.get("voucher_no")]
|
||||
if not names:
|
||||
frappe.throw(_("No Purchase Invoices selected"))
|
||||
|
||||
payable, excluded = _partition_payable_invoices(names)
|
||||
if not payable:
|
||||
frappe.throw(_("None of the selected invoices are payable"))
|
||||
|
||||
# invoices sharing a (supplier, payable account) are combined into one Payment Entry
|
||||
groups = {}
|
||||
for d in payable:
|
||||
key = (d["supplier"], d["party_account"])
|
||||
groups.setdefault(
|
||||
key, {"supplier": d["supplier"], "party_account": d["party_account"], "vouchers": []}
|
||||
)["vouchers"].append(d["voucher_no"])
|
||||
|
||||
created, failed = 0, 0
|
||||
for group in groups.values():
|
||||
if _create_payment_entry(group):
|
||||
created += 1
|
||||
else:
|
||||
failed += 1
|
||||
|
||||
message = _("Created {0} draft Payment Entries").format(created)
|
||||
if excluded:
|
||||
message += " — " + _("{0} excluded (not payable)").format(len(excluded))
|
||||
if failed:
|
||||
message += " — " + _("{0} failed (see Error Log)").format(failed)
|
||||
frappe.msgprint(message, title=_("Bulk Payment Entries"), indicator="green")
|
||||
|
||||
|
||||
@frappe.whitelist()
|
||||
def get_payable_invoices(invoices: str | list | None = None):
|
||||
"""Return the live payable subset of the selected invoices for the report dialog."""
|
||||
frappe.has_permission("Payment Entry", "create", throw=True)
|
||||
|
||||
names = [d["voucher_no"] for d in frappe.parse_json(invoices or "[]") if d.get("voucher_no")]
|
||||
payable, excluded = _partition_payable_invoices(names)
|
||||
|
||||
currency = None
|
||||
if payable:
|
||||
company = frappe.get_cached_value("Purchase Invoice", payable[0]["voucher_no"], "company")
|
||||
currency = frappe.get_cached_value("Company", company, "default_currency")
|
||||
|
||||
return {"payable": payable, "excluded": excluded, "currency": currency}
|
||||
|
||||
|
||||
def _partition_payable_invoices(names):
|
||||
"""Split submitted Purchase Invoices into payable ones and excluded ones (with reason).
|
||||
|
||||
Returns are debit notes, internal transfers are inter-company, and non-positive
|
||||
outstanding means already settled — none are valid targets for a supplier payment.
|
||||
"""
|
||||
if not names:
|
||||
return [], []
|
||||
|
||||
rows = frappe.get_list(
|
||||
"Purchase Invoice",
|
||||
filters={"name": ["in", names], "docstatus": 1},
|
||||
fields=[
|
||||
"name",
|
||||
"supplier",
|
||||
"credit_to",
|
||||
"outstanding_amount",
|
||||
"conversion_rate",
|
||||
"is_return",
|
||||
"is_internal_supplier",
|
||||
],
|
||||
limit_page_length=0,
|
||||
)
|
||||
|
||||
payable, excluded = [], []
|
||||
for r in rows:
|
||||
if r.is_return:
|
||||
excluded.append({"voucher_no": r.name, "reason": _("Debit Note")})
|
||||
elif r.is_internal_supplier:
|
||||
excluded.append({"voucher_no": r.name, "reason": _("Internal Transfer")})
|
||||
elif flt(r.outstanding_amount) <= 0:
|
||||
excluded.append({"voucher_no": r.name, "reason": _("Already Paid")})
|
||||
else:
|
||||
payable.append(
|
||||
{
|
||||
"voucher_no": r.name,
|
||||
"supplier": r.supplier,
|
||||
"party_account": r.credit_to,
|
||||
"outstanding": flt(r.outstanding_amount) * flt(r.conversion_rate or 1),
|
||||
}
|
||||
)
|
||||
|
||||
# names not returned were cancelled/deleted or no longer readable after the report loaded
|
||||
found = {r.name for r in rows}
|
||||
for name in names:
|
||||
if name not in found:
|
||||
excluded.append({"voucher_no": name, "reason": _("Not available")})
|
||||
|
||||
return payable, excluded
|
||||
|
||||
|
||||
def _create_payment_entry(group):
|
||||
supplier = group["supplier"]
|
||||
try:
|
||||
frappe.db.savepoint("bulk_pe")
|
||||
if len(group["vouchers"]) == 1:
|
||||
pe = _build_single_payment_entry(group["vouchers"][0])
|
||||
else:
|
||||
pe = _build_grouped_payment_entry(supplier, group["party_account"], group["vouchers"])
|
||||
|
||||
if not pe:
|
||||
frappe.db.rollback(save_point="bulk_pe")
|
||||
frappe.log_error(
|
||||
title=_("Bulk Payment Entry skipped for {0}").format(supplier),
|
||||
message=_("No outstanding amount for the selected invoice(s)."),
|
||||
)
|
||||
return False
|
||||
|
||||
pe.flags.ignore_validate = True
|
||||
pe.set_title_field()
|
||||
pe.insert(ignore_mandatory=True)
|
||||
return True
|
||||
except Exception:
|
||||
frappe.db.rollback(save_point="bulk_pe")
|
||||
frappe.log_error(title=_("Bulk Payment Entry creation failed for {0}").format(supplier))
|
||||
return False
|
||||
|
||||
|
||||
def _build_single_payment_entry(name):
|
||||
pe = get_payment_entry("Purchase Invoice", name)
|
||||
# guard against a stale report row: nothing to allocate means the invoice is already settled
|
||||
if not pe.references or not any(flt(r.allocated_amount) for r in pe.references):
|
||||
return None
|
||||
return pe
|
||||
|
||||
|
||||
def _build_grouped_payment_entry(supplier, party_account, names):
|
||||
name_set = set(names)
|
||||
pe = get_payment_entry("Purchase Invoice", names[0])
|
||||
pe.set("references", [])
|
||||
|
||||
refs = get_outstanding_reference_documents(
|
||||
{
|
||||
"party_type": "Supplier",
|
||||
"party": supplier,
|
||||
"party_account": party_account,
|
||||
"company": pe.company,
|
||||
"vouchers": [frappe._dict(voucher_type="Purchase Invoice", voucher_no=n) for n in names],
|
||||
}
|
||||
)
|
||||
|
||||
# get_negative_outstanding_invoices ignores the vouchers filter, so bound refs to the selection
|
||||
for r in refs:
|
||||
if r.voucher_type != "Purchase Invoice" or r.voucher_no not in name_set:
|
||||
continue
|
||||
pe.append(
|
||||
"references",
|
||||
{
|
||||
"reference_doctype": r.voucher_type,
|
||||
"reference_name": r.voucher_no,
|
||||
"bill_no": r.get("bill_no"),
|
||||
"due_date": r.get("due_date"),
|
||||
"payment_term": r.get("payment_term"),
|
||||
"total_amount": r.invoice_amount,
|
||||
"outstanding_amount": r.outstanding_amount,
|
||||
"allocated_amount": r.outstanding_amount,
|
||||
"exchange_rate": r.get("exchange_rate") or 1,
|
||||
},
|
||||
)
|
||||
|
||||
if not pe.references:
|
||||
return None
|
||||
|
||||
# received_amount is in paid_to account currency; convert to paid_from account currency for paid_amount
|
||||
pe.received_amount = sum(r.allocated_amount for r in pe.references)
|
||||
pe.paid_amount = flt(pe.received_amount * pe.target_exchange_rate, pe.precision("paid_amount"))
|
||||
pe.set_amounts()
|
||||
return pe
|
||||
@@ -17,7 +17,7 @@ class ERPNextAddress(Address):
|
||||
|
||||
def link_address(self):
|
||||
"""Link address based on owner"""
|
||||
if self.get("is_your_company_address"):
|
||||
if self.is_your_company_address:
|
||||
return
|
||||
|
||||
return super().link_address()
|
||||
@@ -28,9 +28,7 @@ class ERPNextAddress(Address):
|
||||
self.is_your_company_address = 1
|
||||
|
||||
def validate_reference(self):
|
||||
if self.get("is_your_company_address") and not [
|
||||
row for row in self.links if row.link_doctype == "Company"
|
||||
]:
|
||||
if self.is_your_company_address and not [row for row in self.links if row.link_doctype == "Company"]:
|
||||
frappe.throw(
|
||||
_(
|
||||
"Address needs to be linked to a Company. Please add a row for Company in the Links table."
|
||||
@@ -71,6 +69,4 @@ def get_shipping_address(company: str, address: str | None = None):
|
||||
if address:
|
||||
address_as_dict = address[0]
|
||||
name, address_template = get_address_templates(address_as_dict)
|
||||
return address_as_dict.get("name"), frappe.render_template(
|
||||
address_template, address_as_dict, restrict_globals=True
|
||||
)
|
||||
return address_as_dict.get("name"), frappe.render_template(address_template, address_as_dict)
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
"idx": 0,
|
||||
"is_public": 1,
|
||||
"is_standard": 1,
|
||||
"modified": "2026-09-04 12:37:31.673782",
|
||||
"modified": "2025-12-19 12:37:31.673782",
|
||||
"modified_by": "Administrator",
|
||||
"module": "Accounts",
|
||||
"name": "Profit and Loss",
|
||||
@@ -17,6 +17,7 @@
|
||||
"owner": "Administrator",
|
||||
"report_name": "Profit and Loss Statement",
|
||||
"roles": [],
|
||||
"show_values_over_chart": 1,
|
||||
"timeseries": 0,
|
||||
"type": "Line",
|
||||
"use_report_chart": 1,
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import frappe
|
||||
from frappe import _
|
||||
from frappe.email import sendmail_to_system_managers
|
||||
from frappe.query_builder.functions import IfNull, Sum
|
||||
from frappe.utils import (
|
||||
add_days,
|
||||
add_months,
|
||||
@@ -54,24 +53,20 @@ def validate_service_stop_date(doc):
|
||||
|
||||
|
||||
def build_conditions(process_type, account, company):
|
||||
if process_type == "Income":
|
||||
item = frappe.qb.DocType("Sales Invoice Item")
|
||||
parent = frappe.qb.DocType("Sales Invoice")
|
||||
deferred_account = item.deferred_revenue_account
|
||||
else:
|
||||
item = frappe.qb.DocType("Purchase Invoice Item")
|
||||
parent = frappe.qb.DocType("Purchase Invoice")
|
||||
deferred_account = item.deferred_expense_account
|
||||
conditions = ""
|
||||
deferred_account = (
|
||||
"item.deferred_revenue_account" if process_type == "Income" else "item.deferred_expense_account"
|
||||
)
|
||||
|
||||
if account:
|
||||
return deferred_account == account
|
||||
conditions += f"AND {deferred_account}={frappe.db.escape(account)}"
|
||||
elif company:
|
||||
return parent.company == company
|
||||
conditions += f"AND p.company = {frappe.db.escape(company)}"
|
||||
|
||||
return None
|
||||
return conditions
|
||||
|
||||
|
||||
def convert_deferred_expense_to_expense(deferred_process, start_date=None, end_date=None, conditions=None):
|
||||
def convert_deferred_expense_to_expense(deferred_process, start_date=None, end_date=None, conditions=""):
|
||||
# book the expense/income on the last day, but it will be trigger on the 1st of month at 12:00 AM
|
||||
|
||||
if not start_date:
|
||||
@@ -80,25 +75,17 @@ def convert_deferred_expense_to_expense(deferred_process, start_date=None, end_d
|
||||
end_date = add_days(today(), -1)
|
||||
|
||||
# check for the purchase invoice for which GL entries has to be done
|
||||
item = frappe.qb.DocType("Purchase Invoice Item")
|
||||
parent = frappe.qb.DocType("Purchase Invoice")
|
||||
query = (
|
||||
frappe.qb.from_(item)
|
||||
.inner_join(parent)
|
||||
.on(item.parent == parent.name)
|
||||
.select(item.parent)
|
||||
.distinct()
|
||||
.where(
|
||||
(item.service_start_date <= end_date)
|
||||
& (item.service_end_date >= start_date)
|
||||
& (item.enable_deferred_expense == 1)
|
||||
& (item.docstatus == 1)
|
||||
& (IfNull(item.amount, 0) > 0)
|
||||
)
|
||||
)
|
||||
if conditions is not None:
|
||||
query = query.where(conditions)
|
||||
invoices = query.run(pluck=True)
|
||||
invoices = frappe.db.sql_list(
|
||||
f"""
|
||||
select distinct item.parent
|
||||
from `tabPurchase Invoice Item` item, `tabPurchase Invoice` p
|
||||
where item.service_start_date<=%s and item.service_end_date>=%s
|
||||
and item.enable_deferred_expense = 1 and item.parent=p.name
|
||||
and item.docstatus = 1 and ifnull(item.amount, 0) > 0
|
||||
{conditions}
|
||||
""",
|
||||
(end_date, start_date),
|
||||
) # nosec
|
||||
|
||||
# For each invoice, book deferred expense
|
||||
for invoice in invoices:
|
||||
@@ -109,7 +96,7 @@ def convert_deferred_expense_to_expense(deferred_process, start_date=None, end_d
|
||||
send_mail(deferred_process)
|
||||
|
||||
|
||||
def convert_deferred_revenue_to_income(deferred_process, start_date=None, end_date=None, conditions=None):
|
||||
def convert_deferred_revenue_to_income(deferred_process, start_date=None, end_date=None, conditions=""):
|
||||
# book the expense/income on the last day, but it will be trigger on the 1st of month at 12:00 AM
|
||||
|
||||
if not start_date:
|
||||
@@ -118,25 +105,17 @@ def convert_deferred_revenue_to_income(deferred_process, start_date=None, end_da
|
||||
end_date = add_days(today(), -1)
|
||||
|
||||
# check for the sales invoice for which GL entries has to be done
|
||||
item = frappe.qb.DocType("Sales Invoice Item")
|
||||
parent = frappe.qb.DocType("Sales Invoice")
|
||||
query = (
|
||||
frappe.qb.from_(item)
|
||||
.inner_join(parent)
|
||||
.on(item.parent == parent.name)
|
||||
.select(item.parent)
|
||||
.distinct()
|
||||
.where(
|
||||
(item.service_start_date <= end_date)
|
||||
& (item.service_end_date >= start_date)
|
||||
& (item.enable_deferred_revenue == 1)
|
||||
& (item.docstatus == 1)
|
||||
& (IfNull(item.amount, 0) > 0)
|
||||
)
|
||||
)
|
||||
if conditions is not None:
|
||||
query = query.where(conditions)
|
||||
invoices = query.run(pluck=True)
|
||||
invoices = frappe.db.sql_list(
|
||||
f"""
|
||||
select distinct item.parent
|
||||
from `tabSales Invoice Item` item, `tabSales Invoice` p
|
||||
where item.service_start_date<=%s and item.service_end_date>=%s
|
||||
and item.enable_deferred_revenue = 1 and item.parent=p.name
|
||||
and item.docstatus = 1 and ifnull(item.amount, 0) > 0
|
||||
{conditions}
|
||||
""",
|
||||
(end_date, start_date),
|
||||
) # nosec
|
||||
|
||||
for invoice in invoices:
|
||||
doc = frappe.get_doc("Sales Invoice", invoice)
|
||||
@@ -157,39 +136,26 @@ def get_booking_dates(doc, item, posting_date=None, prev_posting_date=None):
|
||||
)
|
||||
|
||||
if not prev_posting_date:
|
||||
prev_gl_entry = frappe.get_all(
|
||||
"GL Entry",
|
||||
filters={
|
||||
"company": doc.company,
|
||||
"account": item.get(deferred_account),
|
||||
"voucher_type": doc.doctype,
|
||||
"voucher_no": doc.name,
|
||||
"voucher_detail_no": item.name,
|
||||
"is_cancelled": 0,
|
||||
},
|
||||
fields=["name", "posting_date"],
|
||||
order_by="posting_date desc",
|
||||
limit=1,
|
||||
prev_gl_entry = frappe.db.sql(
|
||||
"""
|
||||
select name, posting_date from `tabGL Entry` where company=%s and account=%s and
|
||||
voucher_type=%s and voucher_no=%s and voucher_detail_no=%s
|
||||
and is_cancelled = 0
|
||||
order by posting_date desc limit 1
|
||||
""",
|
||||
(doc.company, item.get(deferred_account), doc.doctype, doc.name, item.name),
|
||||
as_dict=True,
|
||||
)
|
||||
|
||||
je = frappe.qb.DocType("Journal Entry")
|
||||
jea = frappe.qb.DocType("Journal Entry Account")
|
||||
prev_gl_via_je = (
|
||||
frappe.qb.from_(je)
|
||||
.inner_join(jea)
|
||||
.on(je.name == jea.parent)
|
||||
.select(je.name, je.posting_date)
|
||||
.where(
|
||||
(je.company == doc.company)
|
||||
& (jea.account == item.get(deferred_account))
|
||||
& (jea.reference_type == doc.doctype)
|
||||
& (jea.reference_name == doc.name)
|
||||
& (jea.reference_detail_no == item.name)
|
||||
& (jea.docstatus < 2)
|
||||
)
|
||||
.orderby(je.posting_date, order=frappe.qb.desc)
|
||||
.limit(1)
|
||||
.run(as_dict=True)
|
||||
prev_gl_via_je = frappe.db.sql(
|
||||
"""
|
||||
SELECT p.name, p.posting_date FROM `tabJournal Entry` p, `tabJournal Entry Account` c
|
||||
WHERE p.name = c.parent and p.company=%s and c.account=%s
|
||||
and c.reference_type=%s and c.reference_name=%s
|
||||
and c.reference_detail_no=%s and c.docstatus < 2 order by posting_date desc limit 1
|
||||
""",
|
||||
(doc.company, item.get(deferred_account), doc.doctype, doc.name, item.name),
|
||||
as_dict=True,
|
||||
)
|
||||
|
||||
if prev_gl_via_je:
|
||||
@@ -311,47 +277,26 @@ def get_already_booked_amount(doc, item):
|
||||
total_credit_debit, total_credit_debit_currency = "credit", "credit_in_account_currency"
|
||||
deferred_account = "deferred_expense_account"
|
||||
|
||||
gle = frappe.qb.DocType("GL Entry")
|
||||
gl_entries_details = (
|
||||
frappe.qb.from_(gle)
|
||||
.select(
|
||||
Sum(gle[total_credit_debit]).as_("total_credit"),
|
||||
Sum(gle[total_credit_debit_currency]).as_("total_credit_in_account_currency"),
|
||||
gle.voucher_detail_no,
|
||||
)
|
||||
.where(
|
||||
(gle.company == doc.company)
|
||||
& (gle.account == item.get(deferred_account))
|
||||
& (gle.voucher_type == doc.doctype)
|
||||
& (gle.voucher_no == doc.name)
|
||||
& (gle.voucher_detail_no == item.name)
|
||||
& (gle.is_cancelled == 0)
|
||||
)
|
||||
.groupby(gle.voucher_detail_no)
|
||||
.run(as_dict=True)
|
||||
gl_entries_details = frappe.db.sql(
|
||||
"""
|
||||
select sum({}) as total_credit, sum({}) as total_credit_in_account_currency, voucher_detail_no
|
||||
from `tabGL Entry` where company=%s and account=%s and voucher_type=%s and voucher_no=%s and voucher_detail_no=%s
|
||||
and is_cancelled = 0
|
||||
group by voucher_detail_no
|
||||
""".format(total_credit_debit, total_credit_debit_currency),
|
||||
(doc.company, item.get(deferred_account), doc.doctype, doc.name, item.name),
|
||||
as_dict=True,
|
||||
)
|
||||
|
||||
je = frappe.qb.DocType("Journal Entry")
|
||||
jea = frappe.qb.DocType("Journal Entry Account")
|
||||
journal_entry_details = (
|
||||
frappe.qb.from_(je)
|
||||
.inner_join(jea)
|
||||
.on(je.name == jea.parent)
|
||||
.select(
|
||||
Sum(jea[total_credit_debit]).as_("total_credit"),
|
||||
Sum(jea[total_credit_debit_currency]).as_("total_credit_in_account_currency"),
|
||||
jea.reference_detail_no,
|
||||
)
|
||||
.where(
|
||||
(je.company == doc.company)
|
||||
& (jea.account == item.get(deferred_account))
|
||||
& (jea.reference_type == doc.doctype)
|
||||
& (jea.reference_name == doc.name)
|
||||
& (jea.reference_detail_no == item.name)
|
||||
& (je.docstatus < 2)
|
||||
)
|
||||
.groupby(jea.reference_detail_no)
|
||||
.run(as_dict=True)
|
||||
journal_entry_details = frappe.db.sql(
|
||||
"""
|
||||
SELECT sum(c.{}) as total_credit, sum(c.{}) as total_credit_in_account_currency, reference_detail_no
|
||||
FROM `tabJournal Entry` p , `tabJournal Entry Account` c WHERE p.name = c.parent and
|
||||
p.company = %s and c.account=%s and c.reference_type=%s and c.reference_name=%s and c.reference_detail_no=%s
|
||||
and p.docstatus < 2 group by reference_detail_no
|
||||
""".format(total_credit_debit, total_credit_debit_currency),
|
||||
(doc.company, item.get(deferred_account), doc.doctype, doc.name, item.name),
|
||||
as_dict=True,
|
||||
)
|
||||
|
||||
already_booked_amount = gl_entries_details[0].total_credit if gl_entries_details else 0
|
||||
@@ -582,7 +527,6 @@ def make_gl_entries(
|
||||
frappe.db.commit()
|
||||
except Exception as e:
|
||||
if frappe.in_test:
|
||||
frappe.db.rollback()
|
||||
doc.log_error(f"Error while processing deferred accounting for Invoice {doc.name}")
|
||||
raise e
|
||||
else:
|
||||
|
||||
@@ -122,7 +122,6 @@
|
||||
"description": "Setting Account Type helps in selecting this Account in transactions.",
|
||||
"fieldname": "account_type",
|
||||
"fieldtype": "Select",
|
||||
"in_preview": 1,
|
||||
"in_standard_filter": 1,
|
||||
"label": "Account Type",
|
||||
"oldfieldname": "account_type",
|
||||
@@ -204,7 +203,7 @@
|
||||
"idx": 1,
|
||||
"is_tree": 1,
|
||||
"links": [],
|
||||
"modified": "2026-09-03 12:59:42.190900",
|
||||
"modified": "2026-04-14 18:14:42.202065",
|
||||
"modified_by": "Administrator",
|
||||
"module": "Accounts",
|
||||
"name": "Account",
|
||||
@@ -265,46 +264,6 @@
|
||||
{
|
||||
"role": "HR Manager",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Delivery Manager",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Delivery User",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Maintenance Manager",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Maintenance User",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Manufacturing Manager",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Manufacturing User",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Purchase Master Manager",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Quality Manager",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Sales Master Manager",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Stock Manager",
|
||||
"select": 1
|
||||
}
|
||||
],
|
||||
"row_format": "Dynamic",
|
||||
|
||||
@@ -121,7 +121,6 @@ class Account(NestedSet):
|
||||
self.validate_account_currency()
|
||||
self.validate_root_company_and_sync_account_to_children()
|
||||
self.validate_receivable_payable_account_type()
|
||||
self.validate_stock_account_type_change()
|
||||
|
||||
def validate_parent_child_account_type(self):
|
||||
if self.parent_account:
|
||||
@@ -213,36 +212,6 @@ class Account(NestedSet):
|
||||
frappe.msgprint(msg)
|
||||
self.add_comment("Comment", msg)
|
||||
|
||||
def validate_stock_account_type_change(self):
|
||||
doc_before_save = self.get_doc_before_save()
|
||||
if not (doc_before_save and doc_before_save.account_type == "Stock"):
|
||||
return
|
||||
|
||||
if self.account_type == "Stock":
|
||||
return
|
||||
|
||||
if self.stock_ledger_entry_exists():
|
||||
frappe.throw(
|
||||
_(
|
||||
"The account type of {0} cannot be changed from {1} because stock ledger entries exist against it."
|
||||
).format(frappe.bold(self.name), frappe.bold(_("Stock")))
|
||||
)
|
||||
|
||||
def stock_ledger_entry_exists(self):
|
||||
from erpnext.stock import get_warehouse_account_map
|
||||
|
||||
warehouse_account = get_warehouse_account_map(self.company)
|
||||
warehouses = [wh for wh, details in warehouse_account.items() if details.account == self.name]
|
||||
if not warehouses:
|
||||
return False
|
||||
|
||||
return bool(
|
||||
frappe.db.count(
|
||||
"Stock Ledger Entry",
|
||||
filters={"warehouse": ("in", warehouses), "is_cancelled": 0},
|
||||
)
|
||||
)
|
||||
|
||||
def validate_root_details(self):
|
||||
doc_before_save = self.get_doc_before_save()
|
||||
|
||||
@@ -265,7 +234,7 @@ class Account(NestedSet):
|
||||
if not frappe.db.get_value(
|
||||
"Account", {"account_name": self.account_name, "company": ancestors[0]}, "name"
|
||||
):
|
||||
frappe.throw(_("Please add the account to root level Company - {0}").format(ancestors[0]))
|
||||
frappe.throw(_("Please add the account to root level Company - {}").format(ancestors[0]))
|
||||
elif self.parent_account:
|
||||
descendants = get_descendants_of("Company", self.company)
|
||||
if not descendants:
|
||||
@@ -690,15 +659,8 @@ def _ensure_idle_system():
|
||||
|
||||
last_gl_update = None
|
||||
try:
|
||||
if frappe.db.db_type == "postgres":
|
||||
# The MariaDB branch blocks new GL inserts via the gap lock its for_update read takes;
|
||||
# a postgres row lock never blocks inserts, so take an EXCLUSIVE table lock instead --
|
||||
# writers block until the rename commits, readers don't. NOWAIT mirrors wait=False.
|
||||
frappe.db.sql("LOCK TABLE `tabGL Entry` IN EXCLUSIVE MODE NOWAIT")
|
||||
last_gl_update = frappe.db.get_value("GL Entry", {}, "modified")
|
||||
else:
|
||||
# We also lock inserts to GL entry table with for_update here.
|
||||
last_gl_update = frappe.db.get_value("GL Entry", {}, "modified", for_update=True, wait=False)
|
||||
# We also lock inserts to GL entry table with for_update here.
|
||||
last_gl_update = frappe.db.get_value("GL Entry", {}, "modified", for_update=True, wait=False)
|
||||
except frappe.QueryTimeoutError:
|
||||
# wait=False fails immediately if there's an active transaction.
|
||||
last_gl_update = add_to_date(None, seconds=-1)
|
||||
@@ -709,7 +671,7 @@ def _ensure_idle_system():
|
||||
if last_gl_update > add_to_date(None, minutes=-5):
|
||||
frappe.throw(
|
||||
_(
|
||||
"Last GL Entry update was done {0}. This operation is not allowed while system is actively being used. Please wait for 5 minutes before retrying."
|
||||
"Last GL Entry update was done {}. This operation is not allowed while system is actively being used. Please wait for 5 minutes before retrying."
|
||||
).format(pretty_date(last_gl_update)),
|
||||
title=_("System In Use"),
|
||||
)
|
||||
@@ -727,12 +689,9 @@ def get_company_default_account_fields():
|
||||
"stock_delivered_but_not_billed": "Stock Delivered But Not Billed Account",
|
||||
"stock_adjustment_account": "Stock Adjustment Account",
|
||||
"write_off_account": "Write Off Account",
|
||||
"bank_charges_account": "Bank Charges Account",
|
||||
"default_discount_account": "Default Payment Discount Account",
|
||||
"unrealized_profit_loss_account": "Unrealized Profit / Loss Account",
|
||||
"exchange_gain_loss_account": "Exchange Gain / Loss Account",
|
||||
"exchange_gain_account": "Exchange Gain Account",
|
||||
"exchange_loss_account": "Exchange Loss Account",
|
||||
"unrealized_exchange_gain_loss_account": "Unrealized Exchange Gain / Loss Account",
|
||||
"round_off_account": "Round Off Account",
|
||||
"default_deferred_revenue_account": "Default Deferred Revenue Account",
|
||||
|
||||
@@ -52,42 +52,6 @@ frappe.treeview_settings["Account"] = {
|
||||
],
|
||||
root_label: "Accounts",
|
||||
get_tree_nodes: "erpnext.accounts.utils.get_children",
|
||||
get_label: function (node) {
|
||||
// clean display name — the account number renders as a badge (see
|
||||
// onrender) instead of being glued into the name
|
||||
return frappe.utils.escape_html(node.data.account_name || node.title || node.label);
|
||||
},
|
||||
onrender: function (node) {
|
||||
if (node.is_root || !node.data) return;
|
||||
|
||||
const flags = [];
|
||||
if (node.data.account_number) {
|
||||
flags.push(frappe.ui.badge({ label: node.data.account_number }));
|
||||
}
|
||||
|
||||
const company = frappe.treeview_settings["Account"].treeview?.page?.fields_dict?.company?.get_value();
|
||||
const company_currency = company && erpnext.get_currency(company);
|
||||
if (
|
||||
node.data.account_currency &&
|
||||
company_currency &&
|
||||
node.data.account_currency !== company_currency
|
||||
) {
|
||||
flags.push(frappe.ui.badge({ label: node.data.account_currency, theme: "blue" }));
|
||||
}
|
||||
|
||||
if (node.data.freeze_account === "Yes") {
|
||||
flags.push(
|
||||
frappe.ui.badge({
|
||||
label: __("Frozen"),
|
||||
icon: "lock",
|
||||
title: __("Frozen - entries restricted"),
|
||||
theme: "orange",
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
erpnext.utils.render_tree_node_flags(node, flags);
|
||||
},
|
||||
on_node_render: function (node, deep) {
|
||||
const render_balances = () => {
|
||||
for (let account of cur_tree.account_balance_data) {
|
||||
@@ -268,25 +232,24 @@ frappe.treeview_settings["Account"] = {
|
||||
frappe.treeview_settings["Account"].treeview["tree"] = treeview.tree;
|
||||
if (treeview.can_create) {
|
||||
treeview.page.set_primary_action(
|
||||
{ label: __("Add Account"), short_label: __("Add") },
|
||||
__("New"),
|
||||
function () {
|
||||
let root_company = treeview.page.fields_dict.root_company.get_value();
|
||||
if (root_company) {
|
||||
frappe.throw(
|
||||
__("Please add the account to root level Company - {0}", [root_company])
|
||||
);
|
||||
frappe.throw(__("Please add the account to root level Company - {0}"), [
|
||||
root_company,
|
||||
]);
|
||||
} else {
|
||||
treeview.new_node();
|
||||
}
|
||||
},
|
||||
"plus"
|
||||
"add"
|
||||
);
|
||||
}
|
||||
},
|
||||
toolbar: [
|
||||
{
|
||||
label: __("Add Child"),
|
||||
icon: "plus",
|
||||
condition: function (node) {
|
||||
return (
|
||||
frappe.boot.user.can_create.indexOf("Account") !== -1 &&
|
||||
@@ -309,7 +272,6 @@ frappe.treeview_settings["Account"] = {
|
||||
return !node.root && frappe.boot.user.can_read.indexOf("GL Entry") !== -1;
|
||||
},
|
||||
label: __("View Ledger"),
|
||||
icon: "book-open",
|
||||
click: function (node, btn) {
|
||||
frappe.route_options = {
|
||||
from_date: erpnext.utils.get_fiscal_year(frappe.datetime.get_today(), true)[1],
|
||||
@@ -324,106 +286,6 @@ frappe.treeview_settings["Account"] = {
|
||||
},
|
||||
btnClass: "hidden-xs",
|
||||
},
|
||||
{
|
||||
// same label and mechanism as the Account form's Actions button:
|
||||
// NOT frappe's generic rename (Allow Rename stays off) — this is
|
||||
// ERPNext's controlled update that rebuilds the derived
|
||||
// "number - name - abbr" document name
|
||||
label: __("Update Account Name / Number"),
|
||||
icon: "text-cursor-input",
|
||||
condition: function (node) {
|
||||
return !node.is_root && frappe.model.can_write("Account");
|
||||
},
|
||||
click: function (node) {
|
||||
const dialog = new frappe.ui.Dialog({
|
||||
title: __("Update Account Number / Name"),
|
||||
fields: [
|
||||
{
|
||||
fieldtype: "Data",
|
||||
fieldname: "account_name",
|
||||
label: __("Account Name"),
|
||||
reqd: 1,
|
||||
default: node.data.account_name,
|
||||
},
|
||||
{
|
||||
fieldtype: "Data",
|
||||
fieldname: "account_number",
|
||||
label: __("Account Number"),
|
||||
default: node.data.account_number,
|
||||
},
|
||||
],
|
||||
primary_action_label: __("Update"),
|
||||
primary_action(values) {
|
||||
dialog.hide();
|
||||
frappe.dom.freeze(__("Updating {0}", [node.label]));
|
||||
frappe.call({
|
||||
method: "erpnext.accounts.doctype.account.account.update_account_number",
|
||||
args: {
|
||||
name: node.label,
|
||||
account_name: values.account_name,
|
||||
account_number: values.account_number,
|
||||
},
|
||||
callback: function (r) {
|
||||
if (r.exc) return;
|
||||
const treeview = frappe.views.trees["Account"];
|
||||
node.parent_node && treeview.tree.load_children(node.parent_node);
|
||||
},
|
||||
always: function () {
|
||||
frappe.dom.unfreeze();
|
||||
},
|
||||
});
|
||||
},
|
||||
});
|
||||
dialog.show();
|
||||
},
|
||||
},
|
||||
{
|
||||
label: __("Convert to Group"),
|
||||
icon: "folder-tree",
|
||||
condition: function (node) {
|
||||
return !node.is_root && !node.expandable && frappe.model.can_write("Account");
|
||||
},
|
||||
click: function (node) {
|
||||
erpnext.accounts.convert_tree_node("Account", node, "convert_ledger_to_group");
|
||||
},
|
||||
},
|
||||
{
|
||||
label: __("Convert to Non-Group"),
|
||||
icon: "file-text",
|
||||
condition: function (node) {
|
||||
// only on groups the user has opened and found empty — a
|
||||
// group with children can't convert, so don't offer it
|
||||
return (
|
||||
!node.is_root &&
|
||||
node.expandable &&
|
||||
node.loaded &&
|
||||
!node.$ul.children().length &&
|
||||
frappe.model.can_write("Account")
|
||||
);
|
||||
},
|
||||
click: function (node) {
|
||||
erpnext.accounts.convert_tree_node("Account", node, "convert_group_to_ledger");
|
||||
},
|
||||
},
|
||||
],
|
||||
extend_toolbar: true,
|
||||
};
|
||||
|
||||
frappe.provide("erpnext.accounts");
|
||||
// shared by the Account and Cost Center tree views (defined in both files,
|
||||
// whichever loads first wins): run the doctype's whitelisted convert method,
|
||||
// then re-render the branch so the node's group/leaf state updates
|
||||
erpnext.accounts.convert_tree_node =
|
||||
erpnext.accounts.convert_tree_node ||
|
||||
function (doctype, node, method) {
|
||||
frappe.call({
|
||||
method: "run_doc_method",
|
||||
args: { dt: doctype, dn: node.label, method: method },
|
||||
callback: function (r) {
|
||||
if (r.exc) return;
|
||||
const treeview = frappe.views.trees[doctype];
|
||||
node.parent_node && treeview.tree.load_children(node.parent_node);
|
||||
frappe.show_alert({ message: __("{0} converted", [node.label]), indicator: "green" });
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
@@ -102,8 +102,6 @@ def identify_is_group(child):
|
||||
def get_chart(chart_template: str | None, existing_company: str | None = None):
|
||||
chart = {}
|
||||
if existing_company:
|
||||
frappe.has_permission("Company", doc=existing_company, throw=True)
|
||||
|
||||
return get_account_tree_from_existing_company(existing_company)
|
||||
|
||||
elif chart_template == "Standard":
|
||||
@@ -139,7 +137,7 @@ def get_charts_for_country(country: str, with_standard: bool = False):
|
||||
|
||||
def _get_chart_name(content):
|
||||
if content:
|
||||
content = frappe.parse_json(content)
|
||||
content = json.loads(content)
|
||||
if (
|
||||
content and content.get("disabled", "No") == "No"
|
||||
) or frappe.local.flags.allow_unverified_charts:
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -24,8 +24,7 @@
|
||||
"account_number": "11530"
|
||||
},
|
||||
"account_number": "115",
|
||||
"is_group": 1,
|
||||
"account_type": "Bank"
|
||||
"is_group": 1
|
||||
},
|
||||
"Trade Receivables": {
|
||||
"Trade Debtors": {
|
||||
@@ -530,13 +529,6 @@
|
||||
"account_number": "630",
|
||||
"is_group": 1
|
||||
},
|
||||
"Accrued Manufacturing Expenses": {
|
||||
"Accrued Expenses - Manufacturing": {
|
||||
"account_number": "63510"
|
||||
},
|
||||
"account_number": "635",
|
||||
"is_group": 1
|
||||
},
|
||||
"account_number": "63",
|
||||
"is_group": 1
|
||||
},
|
||||
@@ -822,4 +814,4 @@
|
||||
"root_type": "Expense"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -37,10 +37,6 @@
|
||||
"account_type": "Stock",
|
||||
"account_category": "Stock Assets"
|
||||
},
|
||||
"Stock Delivered But Not Billed": {
|
||||
"account_type": "Stock Delivered But Not Billed",
|
||||
"account_category": "Stock Assets"
|
||||
},
|
||||
"account_type": "Stock",
|
||||
"account_category": "Stock Assets"
|
||||
},
|
||||
@@ -179,9 +175,6 @@
|
||||
},
|
||||
"Impairment": {
|
||||
"account_category": "Operating Expenses"
|
||||
},
|
||||
"Exchange Loss": {
|
||||
"account_category": "Operating Expenses"
|
||||
}
|
||||
},
|
||||
"root_type": "Expense"
|
||||
@@ -199,10 +192,6 @@
|
||||
"account_type": "Income Account"
|
||||
},
|
||||
"Indirect Income": {
|
||||
"Exchange Gain": {
|
||||
"account_type": "Income Account",
|
||||
"account_category": "Other Operating Income"
|
||||
},
|
||||
"account_type": "Income Account",
|
||||
"is_group": 1
|
||||
},
|
||||
@@ -234,6 +223,10 @@
|
||||
"Stock Received But Not Billed": {
|
||||
"account_type": "Stock Received But Not Billed",
|
||||
"account_category": "Trade Payables"
|
||||
},
|
||||
"Stock Delivered But Not Billed": {
|
||||
"account_type": "Stock Delivered But Not Billed",
|
||||
"account_category": "Trade Payables"
|
||||
}
|
||||
},
|
||||
"Duties and Taxes": {
|
||||
|
||||
@@ -22,12 +22,12 @@
|
||||
"account_type": "Cash"
|
||||
},
|
||||
"Petty Cash Fund": {
|
||||
"account_number": "1110",
|
||||
"account_number": "1200",
|
||||
"is_group": 1,
|
||||
"root_type": "Asset",
|
||||
"account_type": "Cash",
|
||||
"Petty Cash Fund": {
|
||||
"account_number": "1111",
|
||||
"account_number": "1201",
|
||||
"is_group": 0,
|
||||
"root_type": "Asset",
|
||||
"account_type": "Cash"
|
||||
@@ -35,16 +35,10 @@
|
||||
}
|
||||
},
|
||||
"Bank Accounts": {
|
||||
"account_number": "1200",
|
||||
"account_number": "1102",
|
||||
"is_group": 1,
|
||||
"root_type": "Asset",
|
||||
"account_type": "Bank",
|
||||
"Cash in Bank - Checking Account": {
|
||||
"account_number": "1201",
|
||||
"is_group": 0,
|
||||
"root_type": "Asset",
|
||||
"account_type": "Bank"
|
||||
}
|
||||
"account_type": "Bank"
|
||||
},
|
||||
"Advances to Officers & Employees": {
|
||||
"account_number": "1290",
|
||||
@@ -110,20 +104,25 @@
|
||||
"account_number": "1511",
|
||||
"is_group": 0,
|
||||
"root_type": "Asset"
|
||||
},
|
||||
"Factory Overhead Variance": {
|
||||
"account_number": "1512",
|
||||
"is_group": 0,
|
||||
"root_type": "Asset"
|
||||
}
|
||||
},
|
||||
"Finished Goods": {
|
||||
"account_number": "1540",
|
||||
"account_number": "1520",
|
||||
"is_group": 1,
|
||||
"root_type": "Asset",
|
||||
"Finished Goods Inventory": {
|
||||
"account_number": "1541",
|
||||
"account_number": "1531",
|
||||
"is_group": 0,
|
||||
"root_type": "Asset",
|
||||
"account_type": "Stock"
|
||||
},
|
||||
"Inventory in Transit": {
|
||||
"account_number": "1542",
|
||||
"account_number": "1532",
|
||||
"is_group": 0,
|
||||
"root_type": "Asset",
|
||||
"account_type": "Stock Adjustment"
|
||||
@@ -269,7 +268,7 @@
|
||||
"root_type": "Asset"
|
||||
}
|
||||
},
|
||||
"Intangible Assets": {
|
||||
"System Development": {
|
||||
"account_number": "1940",
|
||||
"is_group": 1,
|
||||
"root_type": "Asset",
|
||||
@@ -278,17 +277,6 @@
|
||||
"is_group": 0,
|
||||
"root_type": "Asset"
|
||||
}
|
||||
},
|
||||
"Accumulated Amortization - Intangible Assets": {
|
||||
"account_number": "1950",
|
||||
"is_group": 1,
|
||||
"root_type": "Asset",
|
||||
"Accum Amortization - System Development": {
|
||||
"account_number": "1951",
|
||||
"is_group": 0,
|
||||
"root_type": "Asset",
|
||||
"account_type": "Accumulated Depreciation"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -418,7 +406,8 @@
|
||||
"Customer Deposits": {
|
||||
"account_number": "2500",
|
||||
"is_group": 0,
|
||||
"root_type": "Liability"
|
||||
"root_type": "Liability",
|
||||
"account_type": "Payable"
|
||||
}
|
||||
},
|
||||
"Non Current Liabilities": {
|
||||
@@ -574,28 +563,6 @@
|
||||
"is_group": 0,
|
||||
"root_type": "Income"
|
||||
}
|
||||
},
|
||||
"Exchange Gain": {
|
||||
"account_number": "6030",
|
||||
"is_group": 1,
|
||||
"root_type": "Income",
|
||||
"Exchange Gain - Detail": {
|
||||
"account_number": "6031",
|
||||
"is_group": 0,
|
||||
"root_type": "Income",
|
||||
"account_type": "Indirect Income"
|
||||
}
|
||||
},
|
||||
"Gain on Asset Disposal": {
|
||||
"account_number": "6040",
|
||||
"is_group": 1,
|
||||
"root_type": "Income",
|
||||
"Gain on Asset Disposal - Detail": {
|
||||
"account_number": "6041",
|
||||
"is_group": 0,
|
||||
"root_type": "Income",
|
||||
"account_type": "Indirect Income"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -608,7 +575,7 @@
|
||||
"is_group": 1,
|
||||
"root_type": "Expense",
|
||||
"Cost of Goods Sold": {
|
||||
"account_number": "5002",
|
||||
"account_number": "5010",
|
||||
"is_group": 0,
|
||||
"root_type": "Expense",
|
||||
"account_type": "Cost of Goods Sold"
|
||||
@@ -861,61 +828,20 @@
|
||||
"root_type": "Expense"
|
||||
}
|
||||
},
|
||||
"Other Expenses": {
|
||||
"account_number": "5200",
|
||||
"is_group": 1,
|
||||
"root_type": "Expense",
|
||||
"Bank Charges": {
|
||||
"account_number": "5201",
|
||||
"is_group": 0,
|
||||
"root_type": "Expense",
|
||||
"account_type": "Indirect Expense"
|
||||
},
|
||||
"Interest Expenses Bank": {
|
||||
"account_number": "5202",
|
||||
"is_group": 0,
|
||||
"root_type": "Expense",
|
||||
"account_type": "Indirect Expense"
|
||||
},
|
||||
"Write Off": {
|
||||
"account_number": "5203",
|
||||
"is_group": 0,
|
||||
"root_type": "Expense",
|
||||
"account_type": "Indirect Expense"
|
||||
},
|
||||
"Exchange Loss": {
|
||||
"account_number": "5204",
|
||||
"is_group": 0,
|
||||
"root_type": "Expense",
|
||||
"account_type": "Indirect Expense"
|
||||
},
|
||||
"Loss on Asset Disposal": {
|
||||
"account_number": "5205",
|
||||
"is_group": 0,
|
||||
"root_type": "Expense",
|
||||
"account_type": "Indirect Expense"
|
||||
}
|
||||
},
|
||||
"Provision For Income Tax": {
|
||||
"account_number": "5300",
|
||||
"is_group": 0,
|
||||
"root_type": "Expense",
|
||||
"account_type": "Tax"
|
||||
},
|
||||
"Stock Adjustment": {
|
||||
"account_number": "5400",
|
||||
"account_number": "5200",
|
||||
"is_group": 0,
|
||||
"root_type": "Expense",
|
||||
"account_type": "Stock Adjustment"
|
||||
},
|
||||
"Round Off": {
|
||||
"account_number": "5500",
|
||||
"account_number": "5300",
|
||||
"is_group": 0,
|
||||
"root_type": "Expense",
|
||||
"account_type": "Round Off"
|
||||
},
|
||||
"Expenses Included In Valuation": {
|
||||
"account_number": "5600",
|
||||
"account_number": "5400",
|
||||
"is_group": 0,
|
||||
"root_type": "Expense",
|
||||
"account_type": "Expenses Included In Valuation"
|
||||
|
||||
@@ -138,7 +138,6 @@ def get():
|
||||
_("Gain/Loss on Asset Disposal"): {"account_category": "Other Operating Income"},
|
||||
_("Impairment"): {"account_category": "Operating Expenses"},
|
||||
_("Tax Expense"): {"account_category": "Tax Expense"},
|
||||
_("Exchange Loss"): {"account_category": "Operating Expenses"},
|
||||
},
|
||||
"root_type": "Expense",
|
||||
},
|
||||
@@ -150,7 +149,6 @@ def get():
|
||||
_("Indirect Income"): {
|
||||
_("Interest Income"): {"account_category": "Investment Income"},
|
||||
_("Interest on Fixed Deposits"): {"account_category": "Investment Income"},
|
||||
_("Exchange Gain"): {"account_category": "Other Operating Income"},
|
||||
"is_group": 1,
|
||||
},
|
||||
"root_type": "Income",
|
||||
|
||||
@@ -233,7 +233,6 @@ def get():
|
||||
},
|
||||
_("Impairment"): {"account_number": "5224", "account_category": "Operating Expenses"},
|
||||
_("Tax Expense"): {"account_number": "5225", "account_category": "Tax Expense"},
|
||||
_("Exchange Loss"): {"account_number": "5226", "account_category": "Operating Expenses"},
|
||||
"account_number": "5200",
|
||||
},
|
||||
"root_type": "Expense",
|
||||
@@ -251,10 +250,6 @@ def get():
|
||||
"account_number": "4220",
|
||||
"account_category": "Investment Income",
|
||||
},
|
||||
_("Exchange Gain"): {
|
||||
"account_number": "4230",
|
||||
"account_category": "Other Operating Income",
|
||||
},
|
||||
"is_group": 1,
|
||||
"account_number": "4200",
|
||||
},
|
||||
|
||||
@@ -306,31 +306,6 @@ class TestAccount(ERPNextTestSuite):
|
||||
acc.account_currency = "USD"
|
||||
self.assertRaises(frappe.ValidationError, acc.save)
|
||||
|
||||
def test_stock_account_type_change_with_ledger_entries(self):
|
||||
from erpnext.stock.doctype.stock_entry.stock_entry_utils import make_stock_entry
|
||||
|
||||
company = "_Test Company with perpetual inventory"
|
||||
warehouse = "Stores - TCP1"
|
||||
stock_account = get_warehouse_account(frappe.get_doc("Warehouse", warehouse))
|
||||
|
||||
make_stock_entry(
|
||||
item_code="_Test Item",
|
||||
target=warehouse,
|
||||
company=company,
|
||||
qty=5,
|
||||
basic_rate=100,
|
||||
)
|
||||
|
||||
account = frappe.get_doc("Account", stock_account)
|
||||
self.assertEqual(account.account_type, "Stock")
|
||||
|
||||
account.account_type = ""
|
||||
self.assertRaises(frappe.ValidationError, account.save)
|
||||
|
||||
account.reload()
|
||||
account.account_name = f"{account.account_name} Updated"
|
||||
account.save() # non-type change stays allowed
|
||||
|
||||
def test_account_balance(self):
|
||||
from erpnext.accounts.utils import get_balance_on
|
||||
|
||||
|
||||
@@ -1,59 +1,10 @@
|
||||
# Copyright (c) 2026, Frappe Technologies Pvt. Ltd. and Contributors
|
||||
# Copyright (c) 2023, Frappe Technologies Pvt. Ltd. and Contributors
|
||||
# See license.txt
|
||||
|
||||
from erpnext.accounts.doctype.account_closing_balance.account_closing_balance import (
|
||||
aggregate_with_last_account_closing_balance,
|
||||
generate_key,
|
||||
)
|
||||
# import frappe
|
||||
|
||||
from erpnext.tests.utils import ERPNextTestSuite
|
||||
|
||||
|
||||
def entry(**overrides):
|
||||
row = {"debit": 0, "credit": 0, "debit_in_account_currency": 0, "credit_in_account_currency": 0}
|
||||
row.update(overrides)
|
||||
return row
|
||||
|
||||
|
||||
class TestAccountClosingBalance(ERPNextTestSuite):
|
||||
"""The closing-balance snapshot is built by merging this period's entries with the
|
||||
previous period's. These lock the merge/key logic that drives that carry-forward."""
|
||||
|
||||
def test_matching_entries_are_summed(self):
|
||||
# this is how a prior-period balance carries forward into the current one
|
||||
merged = aggregate_with_last_account_closing_balance(
|
||||
[
|
||||
entry(account="Cash - _TC", debit=100, debit_in_account_currency=100),
|
||||
entry(
|
||||
account="Cash - _TC",
|
||||
debit=50,
|
||||
credit=20,
|
||||
debit_in_account_currency=50,
|
||||
credit_in_account_currency=20,
|
||||
),
|
||||
],
|
||||
[],
|
||||
)
|
||||
self.assertEqual(len(merged), 1)
|
||||
row = next(iter(merged.values()))
|
||||
self.assertEqual(row["debit"], 150)
|
||||
self.assertEqual(row["credit"], 20)
|
||||
# the account-currency columns are accumulated in the same pass
|
||||
self.assertEqual(row["debit_in_account_currency"], 150)
|
||||
self.assertEqual(row["credit_in_account_currency"], 20)
|
||||
|
||||
def test_entries_are_kept_separate_per_dimension(self):
|
||||
merged = aggregate_with_last_account_closing_balance(
|
||||
[
|
||||
entry(account="Cash - _TC", cost_center="CC1", debit=100, debit_in_account_currency=100),
|
||||
entry(account="Cash - _TC", cost_center="CC2", debit=40, debit_in_account_currency=40),
|
||||
],
|
||||
[],
|
||||
)
|
||||
self.assertEqual(len(merged), 2)
|
||||
|
||||
def test_period_closing_flag_is_part_of_the_key(self):
|
||||
# a P&L reversal (flag 0) and a closing-account entry (flag 1) for the same
|
||||
# account must not merge, so the flag has to distinguish their keys
|
||||
key_reversal, _ = generate_key(entry(account="Sales - _TC", is_period_closing_voucher_entry=0), [])
|
||||
key_closing, _ = generate_key(entry(account="Sales - _TC", is_period_closing_voucher_entry=1), [])
|
||||
self.assertNotEqual(key_reversal, key_closing)
|
||||
pass
|
||||
|
||||
@@ -16,8 +16,6 @@ frappe.ui.form.on("Accounting Dimension", {
|
||||
return {
|
||||
filters: {
|
||||
name: ["not in", invalid_doctypes],
|
||||
istable: 0,
|
||||
issingle: 0,
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
@@ -60,14 +60,6 @@ class AccountingDimension(Document):
|
||||
msg = _("Not allowed to create accounting dimension for {0}").format(self.document_type)
|
||||
frappe.throw(msg)
|
||||
|
||||
meta = frappe.get_meta(self.document_type)
|
||||
if meta.istable or meta.issingle:
|
||||
frappe.throw(
|
||||
_(
|
||||
"{0} cannot be used as an accounting dimension as it is not a standalone document type."
|
||||
).format(frappe.bold(self.document_type))
|
||||
)
|
||||
|
||||
exists = frappe.db.get_value("Accounting Dimension", {"document_type": self.document_type}, ["name"])
|
||||
|
||||
if exists and self.is_new():
|
||||
@@ -232,7 +224,7 @@ def disable_dimension(doc: str):
|
||||
|
||||
|
||||
def toggle_disabling(doc):
|
||||
doc = frappe.parse_json(doc)
|
||||
doc = json.loads(doc)
|
||||
|
||||
if doc.get("disabled"):
|
||||
df = {"read_only": 1}
|
||||
@@ -367,13 +359,3 @@ def create_accounting_dimensions_for_doctype(doctype):
|
||||
create_custom_field(doctype, df, ignore_validate=True)
|
||||
|
||||
frappe.clear_cache(doctype=doctype)
|
||||
|
||||
|
||||
def get_dimension_fieldname(dim_doctype: str) -> str:
|
||||
"""
|
||||
Return the `GL Entry` fieldname for a given dimension.
|
||||
"""
|
||||
if dim_doctype in ("Cost Center", "Project"):
|
||||
return frappe.scrub(dim_doctype)
|
||||
|
||||
return frappe.db.get_value("Accounting Dimension", {"document_type": dim_doctype}, "fieldname")
|
||||
|
||||
@@ -51,23 +51,6 @@ class TestAccountingDimension(ERPNextTestSuite):
|
||||
self.assertEqual(gle.get("department"), "_Test Department - _TC")
|
||||
self.assertEqual(gle1.get("department"), "_Test Department - _TC")
|
||||
|
||||
def test_child_table_not_allowed_as_dimension(self):
|
||||
dimension = frappe.get_doc({"doctype": "Accounting Dimension", "document_type": "Sales Team"})
|
||||
self.assertRaises(frappe.ValidationError, dimension.insert)
|
||||
|
||||
def test_single_doctype_not_allowed_as_dimension(self):
|
||||
dimension = frappe.get_doc({"doctype": "Accounting Dimension", "document_type": "Selling Settings"})
|
||||
self.assertRaises(frappe.ValidationError, dimension.insert)
|
||||
|
||||
def test_non_scalar_dimension_value_skipped_in_gl_dict(self):
|
||||
si = create_sales_invoice(do_not_save=1)
|
||||
|
||||
si.department = "_Test Department - _TC"
|
||||
self.assertEqual(si.get_gl_dict({}).get("department"), "_Test Department - _TC")
|
||||
|
||||
si.department = ["_Test Department - _TC"]
|
||||
self.assertNotIn("department", si.get_gl_dict({}))
|
||||
|
||||
def test_mandatory(self):
|
||||
location = frappe.get_doc("Accounting Dimension", "Location")
|
||||
location.dimension_defaults[0].mandatory_for_bs = True
|
||||
|
||||
@@ -6,7 +6,7 @@ frappe.ui.form.on("Accounting Dimension Filter", {
|
||||
let help_content = `<table class="table table-bordered" style="background-color: var(--scrollbar-track-color);">
|
||||
<tr><td>
|
||||
<p>
|
||||
<svg class="icon icon-sm"><use href="#icon-info"></use></svg>
|
||||
<i class="fa fa-hand-right"></i>
|
||||
{{__('Note: On checking Is Mandatory the accounting dimension will become mandatory against that specific account for all accounting transactions')}}
|
||||
</p>
|
||||
</td></tr>
|
||||
|
||||
@@ -5,7 +5,6 @@
|
||||
import frappe
|
||||
from frappe import _
|
||||
from frappe.model.document import Document
|
||||
from frappe.utils import getdate, nowdate
|
||||
|
||||
|
||||
class OverlapError(frappe.ValidationError):
|
||||
@@ -37,20 +36,8 @@ class AccountingPeriod(Document):
|
||||
# end: auto-generated types
|
||||
|
||||
def validate(self):
|
||||
self.validate_dates()
|
||||
self.validate_overlap()
|
||||
|
||||
def validate_dates(self):
|
||||
if getdate(self.start_date) > getdate(self.end_date):
|
||||
frappe.throw(_("Start Date cannot be after End Date"))
|
||||
|
||||
if getdate(self.end_date) > getdate(nowdate()):
|
||||
frappe.throw(
|
||||
_(
|
||||
"Accounting Period cannot be created for a future date. End Date {0} is after today."
|
||||
).format(frappe.bold(frappe.format(self.end_date, "Date")))
|
||||
)
|
||||
|
||||
def before_insert(self):
|
||||
self.bootstrap_doctypes_for_closing()
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
# See license.txt
|
||||
|
||||
import frappe
|
||||
from frappe.utils import nowdate
|
||||
from frappe.utils import add_months, nowdate
|
||||
|
||||
from erpnext.accounts.doctype.accounting_period.accounting_period import (
|
||||
ClosedAccountingPeriod,
|
||||
@@ -93,7 +93,7 @@ def create_accounting_period(**args):
|
||||
|
||||
accounting_period = frappe.new_doc("Accounting Period")
|
||||
accounting_period.start_date = args.start_date or nowdate()
|
||||
accounting_period.end_date = args.end_date or nowdate()
|
||||
accounting_period.end_date = args.end_date or add_months(nowdate(), 1)
|
||||
accounting_period.company = args.company or "_Test Company"
|
||||
accounting_period.period_name = args.period_name or "_Test_Period_Name_1"
|
||||
accounting_period.append("closed_documents", {"document_type": "Sales Invoice", "closed": 1})
|
||||
|
||||
@@ -10,7 +10,7 @@ frappe.ui.form.on("Accounts Settings", {
|
||||
},
|
||||
};
|
||||
});
|
||||
if (!frm.naming_controller) frm.naming_controller = new frappe.ui.NamingSeriesController(frm);
|
||||
if (!frm.naming_controller) frm.naming_controller = new erpnext.NamingSeriesController(frm);
|
||||
|
||||
frm.naming_controller.render_table("transaction_naming_html", get_transactions(frm));
|
||||
},
|
||||
|
||||
@@ -22,8 +22,6 @@
|
||||
"allow_multi_currency_invoices_against_single_party_account",
|
||||
"confirm_before_resetting_posting_date",
|
||||
"preview_mode",
|
||||
"stock_expense_section",
|
||||
"book_stock_expense_gl_entries",
|
||||
"analytics_section",
|
||||
"enable_discounts_and_margin",
|
||||
"enable_accounting_dimensions",
|
||||
@@ -78,8 +76,6 @@
|
||||
"over_billing_allowance",
|
||||
"credit_controller",
|
||||
"role_allowed_to_over_bill",
|
||||
"enable_overdue_billing_threshold",
|
||||
"role_allowed_to_bypass_overdue_billing",
|
||||
"column_break_11",
|
||||
"assets_tab",
|
||||
"asset_settings_section",
|
||||
@@ -91,18 +87,16 @@
|
||||
"period_closing_settings_section",
|
||||
"ignore_account_closing_balance",
|
||||
"use_legacy_controller_for_pcv",
|
||||
"pcv_job_timeout",
|
||||
"column_break_25",
|
||||
"reports_tab",
|
||||
"remarks_section",
|
||||
"disable_include_dimensions",
|
||||
"column_break_lvjk",
|
||||
"general_ledger_remarks_length",
|
||||
"receivable_payable_remarks_length",
|
||||
"column_break_lvjk",
|
||||
"accounts_receivable_payable_tuning_section",
|
||||
"receivable_payable_fetch_method",
|
||||
"default_ageing_range",
|
||||
"column_break_ntmi",
|
||||
"receivable_payable_remarks_length",
|
||||
"legacy_section",
|
||||
"ignore_is_opening_check_for_reporting",
|
||||
"tab_break_dpet",
|
||||
@@ -200,12 +194,10 @@
|
||||
},
|
||||
{
|
||||
"default": "1",
|
||||
"description": "System will use the latest saved Currency Exchange rate on or before the transaction date, however old it is. <br>\nUncheck to ignore rates older than Stale Days and fetch a fresh rate from the exchange rate provider instead.",
|
||||
"fieldname": "allow_stale",
|
||||
"fieldtype": "Check",
|
||||
"in_list_view": 1,
|
||||
"label": "Allow Stale Exchange Rates",
|
||||
"show_description_on_click": 1
|
||||
"label": "Allow Stale Exchange Rates"
|
||||
},
|
||||
{
|
||||
"default": "1",
|
||||
@@ -225,8 +217,7 @@
|
||||
"description": "The percentage you are allowed to bill more against the amount ordered. For example, if the order value is $100 for an item and tolerance is set as 10%, then you are allowed to bill up to $110 ",
|
||||
"fieldname": "over_billing_allowance",
|
||||
"fieldtype": "Currency",
|
||||
"label": "Over Billing Allowance (%)",
|
||||
"non_negative": 1
|
||||
"label": "Over Billing Allowance (%)"
|
||||
},
|
||||
{
|
||||
"default": "1",
|
||||
@@ -280,21 +271,6 @@
|
||||
"label": "Role Allowed to over bill ",
|
||||
"options": "Role"
|
||||
},
|
||||
{
|
||||
"default": "0",
|
||||
"description": "Enabling this option prevents the creation of a new Sales Invoice when the customer has an overdue limit set and their outstanding overdue amount exceeds that limit.",
|
||||
"fieldname": "enable_overdue_billing_threshold",
|
||||
"fieldtype": "Check",
|
||||
"label": "Prevent Sales Invoice when Customer is Overdue"
|
||||
},
|
||||
{
|
||||
"depends_on": "eval:doc.enable_overdue_billing_threshold",
|
||||
"description": "Users with this role can still submit invoices for customers who have crossed their Overdue Limit.",
|
||||
"fieldname": "role_allowed_to_bypass_overdue_billing",
|
||||
"fieldtype": "Link",
|
||||
"label": "Role Allowed to Bypass Over Billing Restriction",
|
||||
"options": "Role"
|
||||
},
|
||||
{
|
||||
"fieldname": "period_closing_settings_section",
|
||||
"fieldtype": "Section Break"
|
||||
@@ -479,7 +455,7 @@
|
||||
{
|
||||
"fieldname": "remarks_section",
|
||||
"fieldtype": "Section Break",
|
||||
"label": "General Ledger Report"
|
||||
"label": "Remarks Column Length"
|
||||
},
|
||||
{
|
||||
"default": "0",
|
||||
@@ -553,7 +529,7 @@
|
||||
{
|
||||
"fieldname": "accounts_receivable_payable_tuning_section",
|
||||
"fieldtype": "Section Break",
|
||||
"label": "Accounts Receivable / Payable Report"
|
||||
"label": "Accounts Receivable / Payable Tuning"
|
||||
},
|
||||
{
|
||||
"fieldname": "legacy_section",
|
||||
@@ -636,14 +612,6 @@
|
||||
"fieldtype": "Check",
|
||||
"label": "Use legacy controller for Period Closing Voucher"
|
||||
},
|
||||
{
|
||||
"default": "3600",
|
||||
"depends_on": "eval: !doc.use_legacy_controller_for_pcv",
|
||||
"description": "Timeout (in seconds) for each background job enqueued by Process Period Closing Voucher",
|
||||
"fieldname": "pcv_job_timeout",
|
||||
"fieldtype": "Int",
|
||||
"label": "PCV Job Timeout (seconds)"
|
||||
},
|
||||
{
|
||||
"description": "Users with this role will be notified if the asset depreciation gets failed",
|
||||
"fieldname": "role_to_notify_on_depreciation_failure",
|
||||
@@ -780,24 +748,6 @@
|
||||
"description": "Changing the account in any transaction of the DocTypes listed below will trigger a repost. To prevent reposting, remove the relevant DocType from the list.",
|
||||
"fieldname": "column_break_mfor",
|
||||
"fieldtype": "Column Break"
|
||||
},
|
||||
{
|
||||
"fieldname": "stock_expense_section",
|
||||
"fieldtype": "Section Break",
|
||||
"label": "Stock Expense Accounting"
|
||||
},
|
||||
{
|
||||
"default": "0",
|
||||
"description": "Books Purchase Expense and Expenses Added To Stock account pairs against stock value. On enabling this, the accounts become mandatory in Company or Item Defaults for Purchase Receipt, Purchase Invoice, Stock Entry, Stock Reconciliation and Landed Cost Voucher",
|
||||
"fieldname": "book_stock_expense_gl_entries",
|
||||
"fieldtype": "Check",
|
||||
"label": "Book Stock Expense GL Entries"
|
||||
},
|
||||
{
|
||||
"default": "0",
|
||||
"fieldname": "disable_include_dimensions",
|
||||
"fieldtype": "Check",
|
||||
"label": "Disable \"Consider Accounting Dimension\" Filter"
|
||||
}
|
||||
],
|
||||
"grid_page_length": 50,
|
||||
@@ -806,7 +756,7 @@
|
||||
"index_web_pages_for_search": 1,
|
||||
"issingle": 1,
|
||||
"links": [],
|
||||
"modified": "2026-09-04 10:08:30.115003",
|
||||
"modified": "2026-06-03 13:11:54.721495",
|
||||
"modified_by": "Administrator",
|
||||
"module": "Accounts",
|
||||
"name": "Accounts Settings",
|
||||
|
||||
@@ -62,7 +62,6 @@ class AccountsSettings(Document):
|
||||
book_asset_depreciation_entry_automatically: DF.Check
|
||||
book_deferred_entries_based_on: DF.Literal["Days", "Months"]
|
||||
book_deferred_entries_via_journal_entry: DF.Check
|
||||
book_stock_expense_gl_entries: DF.Check
|
||||
book_tax_discount_loss: DF.Check
|
||||
calculate_depr_using_total_days: DF.Check
|
||||
check_supplier_invoice_uniqueness: DF.Check
|
||||
@@ -72,14 +71,12 @@ class AccountsSettings(Document):
|
||||
default_ageing_range: DF.Data | None
|
||||
delete_linked_ledger_entries: DF.Check
|
||||
determine_address_tax_category_from: DF.Literal["Billing Address", "Shipping Address"]
|
||||
disable_include_dimensions: DF.Check
|
||||
enable_accounting_dimensions: DF.Check
|
||||
enable_common_party_accounting: DF.Check
|
||||
enable_discounts_and_margin: DF.Check
|
||||
enable_fuzzy_matching: DF.Check
|
||||
enable_immutable_ledger: DF.Check
|
||||
enable_loyalty_point_program: DF.Check
|
||||
enable_overdue_billing_threshold: DF.Check
|
||||
enable_party_matching: DF.Check
|
||||
enable_subscription: DF.Check
|
||||
exchange_gain_loss_posting_date: DF.Literal["Invoice", "Payment", "Reconciliation Date"]
|
||||
@@ -93,13 +90,11 @@ class AccountsSettings(Document):
|
||||
make_payment_via_journal_entry: DF.Check
|
||||
merge_similar_account_heads: DF.Check
|
||||
over_billing_allowance: DF.Currency
|
||||
pcv_job_timeout: DF.Int
|
||||
preview_mode: DF.Check
|
||||
receivable_payable_fetch_method: DF.Literal["Buffered Cursor", "UnBuffered Cursor"]
|
||||
receivable_payable_remarks_length: DF.Int
|
||||
reconciliation_queue_size: DF.Int
|
||||
repost_allowed_types: DF.Table[RepostAllowedTypes]
|
||||
role_allowed_to_bypass_overdue_billing: DF.Link | None
|
||||
role_allowed_to_over_bill: DF.Link | None
|
||||
role_to_notify_on_depreciation_failure: DF.Link | None
|
||||
role_to_override_stop_action: DF.Link | None
|
||||
@@ -155,10 +150,6 @@ class AccountsSettings(Document):
|
||||
toggle_subscription_sections(not self.enable_subscription)
|
||||
clear_cache = True
|
||||
|
||||
if old_doc.enable_overdue_billing_threshold != self.enable_overdue_billing_threshold:
|
||||
toggle_overdue_billing_threshold_field(not self.enable_overdue_billing_threshold)
|
||||
clear_cache = True
|
||||
|
||||
if clear_cache:
|
||||
frappe.clear_cache()
|
||||
|
||||
@@ -204,8 +195,8 @@ class AccountsSettings(Document):
|
||||
if self.add_taxes_from_item_tax_template and self.add_taxes_from_taxes_and_charges_template:
|
||||
frappe.throw(
|
||||
_("You cannot enable both the settings '{0}' and '{1}'.").format(
|
||||
frappe.bold(self.meta.get_translated_label("add_taxes_from_item_tax_template")),
|
||||
frappe.bold(self.meta.get_translated_label("add_taxes_from_taxes_and_charges_template")),
|
||||
frappe.bold(_(self.meta.get_label("add_taxes_from_item_tax_template"))),
|
||||
frappe.bold(_(self.meta.get_label("add_taxes_from_taxes_and_charges_template"))),
|
||||
),
|
||||
title=_("Auto Tax Settings Error"),
|
||||
)
|
||||
@@ -250,10 +241,6 @@ def toggle_subscription_sections(hide):
|
||||
create_property_setter_for_hiding_field(doctype, "subscription_section", hide)
|
||||
|
||||
|
||||
def toggle_overdue_billing_threshold_field(hide):
|
||||
create_property_setter_for_hiding_field("Customer Credit Limit", "overdue_billing_threshold", hide)
|
||||
|
||||
|
||||
def create_property_setter_for_hiding_field(doctype, field_name, hide):
|
||||
make_property_setter(
|
||||
doctype,
|
||||
|
||||
@@ -22,13 +22,11 @@ class TestAdvancePaymentLedgerEntry(ERPNextTestSuite, AccountsTestMixin):
|
||||
"""
|
||||
|
||||
def setUp(self):
|
||||
self.company = "_Test Company"
|
||||
self.customer = "_Test Customer"
|
||||
self.supplier = "_Test Supplier"
|
||||
self.item = "_Test Item"
|
||||
self.cash = "Cash - _TC"
|
||||
self.debtors_usd = "_Test Receivable USD - _TC"
|
||||
self.creditors_usd = "_Test Payable USD - _TC"
|
||||
self.create_company()
|
||||
self.create_usd_receivable_account()
|
||||
self.create_usd_payable_account()
|
||||
self.create_item()
|
||||
self.clear_old_entries()
|
||||
|
||||
def create_sales_order(self, qty=1, rate=100, currency="INR", do_not_submit=False):
|
||||
"""
|
||||
|
||||
@@ -101,7 +101,7 @@
|
||||
}
|
||||
],
|
||||
"links": [],
|
||||
"modified": "2026-08-21 23:11:39.423431",
|
||||
"modified": "2024-03-27 13:06:36.896195",
|
||||
"modified_by": "Administrator",
|
||||
"module": "Accounts",
|
||||
"name": "Bank",
|
||||
@@ -118,20 +118,11 @@
|
||||
"role": "System Manager",
|
||||
"share": 1,
|
||||
"write": 1
|
||||
},
|
||||
{
|
||||
"role": "Accounts Manager",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Accounts User",
|
||||
"select": 1
|
||||
}
|
||||
],
|
||||
"quick_entry": 1,
|
||||
"row_format": "Dynamic",
|
||||
"sort_field": "creation",
|
||||
"sort_order": "DESC",
|
||||
"states": [],
|
||||
"track_changes": 1
|
||||
}
|
||||
}
|
||||
@@ -269,7 +269,7 @@
|
||||
"link_fieldname": "default_bank_account"
|
||||
}
|
||||
],
|
||||
"modified": "2026-08-21 23:11:39.585456",
|
||||
"modified": "2026-04-11 19:46:27.609994",
|
||||
"modified_by": "Administrator",
|
||||
"module": "Accounts",
|
||||
"name": "Bank Account",
|
||||
@@ -299,22 +299,6 @@
|
||||
"role": "Accounts User",
|
||||
"share": 1,
|
||||
"write": 1
|
||||
},
|
||||
{
|
||||
"role": "Purchase Manager",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Purchase Master Manager",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Sales Master Manager",
|
||||
"select": 1
|
||||
},
|
||||
{
|
||||
"role": "Sales User",
|
||||
"select": 1
|
||||
}
|
||||
],
|
||||
"row_format": "Dynamic",
|
||||
|
||||
@@ -107,7 +107,7 @@ def get_party_bank_account(party_type, party):
|
||||
)
|
||||
|
||||
|
||||
def get_default_company_bank_account(company, party_type, party, ignore_permissions=True):
|
||||
def get_default_company_bank_account(company, party_type, party):
|
||||
default_company_bank_account = frappe.db.get_value(party_type, party, "default_bank_account")
|
||||
if default_company_bank_account:
|
||||
if company != frappe.get_cached_value("Bank Account", default_company_bank_account, "company"):
|
||||
@@ -118,14 +118,6 @@ def get_default_company_bank_account(company, party_type, party, ignore_permissi
|
||||
"Bank Account", {"company": company, "is_company_account": 1, "is_default": 1}
|
||||
)
|
||||
|
||||
if not ignore_permissions:
|
||||
default_company_bank_account = (
|
||||
default_company_bank_account
|
||||
if default_company_bank_account
|
||||
and frappe.get_cached_doc("Bank Account", default_company_bank_account).has_permission("select")
|
||||
else None
|
||||
)
|
||||
|
||||
return default_company_bank_account
|
||||
|
||||
|
||||
@@ -196,7 +188,7 @@ def get_closing_balance_as_per_statement(bank_account: str, date: str):
|
||||
return {"balance": 0, "date": None}
|
||||
|
||||
|
||||
@frappe.whitelist(methods=["POST"])
|
||||
@frappe.whitelist()
|
||||
def set_closing_balance_as_per_statement(bank_account: str, date: str | datetime.date, balance: float):
|
||||
"""
|
||||
Set the closing balance as per statement for a bank account and date
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
{
|
||||
"actions": [],
|
||||
"allow_bulk_edit": 1,
|
||||
"allow_rename": 1,
|
||||
"creation": "2026-04-11 19:48:13.622253",
|
||||
"doctype": "DocType",
|
||||
@@ -8,8 +7,7 @@
|
||||
"field_order": [
|
||||
"bank_account",
|
||||
"date",
|
||||
"balance",
|
||||
"company"
|
||||
"balance"
|
||||
],
|
||||
"fields": [
|
||||
{
|
||||
@@ -33,20 +31,12 @@
|
||||
"in_list_view": 1,
|
||||
"label": "Balance",
|
||||
"reqd": 1
|
||||
},
|
||||
{
|
||||
"fetch_from": "bank_account.company",
|
||||
"fieldname": "company",
|
||||
"fieldtype": "Link",
|
||||
"label": "Company",
|
||||
"options": "Company",
|
||||
"read_only": 1
|
||||
}
|
||||
],
|
||||
"grid_page_length": 50,
|
||||
"index_web_pages_for_search": 1,
|
||||
"links": [],
|
||||
"modified": "2026-06-16 22:17:48.007982",
|
||||
"modified": "2026-04-11 19:49:45.374695",
|
||||
"modified_by": "Administrator",
|
||||
"module": "Accounts",
|
||||
"name": "Bank Account Balance",
|
||||
|
||||
@@ -16,7 +16,6 @@ class BankAccountBalance(Document):
|
||||
|
||||
balance: DF.Currency
|
||||
bank_account: DF.Link
|
||||
company: DF.Link | None
|
||||
date: DF.Date
|
||||
# end: auto-generated types
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ from frappe import _, msgprint
|
||||
from frappe.model.document import Document
|
||||
from frappe.query_builder import Case
|
||||
from frappe.query_builder.custom import ConstantColumn
|
||||
from frappe.query_builder.functions import Coalesce, Max, Sum
|
||||
from frappe.query_builder.functions import Coalesce, Sum
|
||||
from frappe.utils import cint, flt, fmt_money, getdate
|
||||
from pypika import Order
|
||||
|
||||
@@ -94,7 +94,6 @@ class BankClearance(Document):
|
||||
invalid_document = []
|
||||
invalid_cheque_date = []
|
||||
entries_to_update = []
|
||||
self.check_permission("write")
|
||||
|
||||
def validate_entry(d):
|
||||
is_valid = True
|
||||
@@ -195,17 +194,14 @@ def get_payment_entries_for_bank_clearance(
|
||||
.select(
|
||||
ConstantColumn("Journal Entry").as_("payment_document"),
|
||||
journal_entry.name.as_("payment_entry"),
|
||||
# non-grouped columns are constant per grouped JE name / account (against_account is
|
||||
# arbitrary per group on MySQL) -> Max() keeps the GROUP BY valid on postgres with the
|
||||
# same value MySQL picked.
|
||||
Max(journal_entry.cheque_no).as_("cheque_number"),
|
||||
Max(journal_entry.cheque_date).as_("cheque_date"),
|
||||
journal_entry.cheque_no.as_("cheque_number"),
|
||||
journal_entry.cheque_date,
|
||||
Sum(journal_entry_account.debit_in_account_currency).as_("debit"),
|
||||
Sum(journal_entry_account.credit_in_account_currency).as_("credit"),
|
||||
Max(journal_entry.posting_date).as_("posting_date"),
|
||||
Max(journal_entry_account.against_account).as_("against_account"),
|
||||
Max(journal_entry.clearance_date).as_("clearance_date"),
|
||||
Max(journal_entry_account.account_currency).as_("account_currency"),
|
||||
journal_entry.posting_date,
|
||||
journal_entry_account.against_account,
|
||||
journal_entry.clearance_date,
|
||||
journal_entry_account.account_currency,
|
||||
)
|
||||
.where(
|
||||
(journal_entry_account.account == account)
|
||||
@@ -218,13 +214,12 @@ def get_payment_entries_for_bank_clearance(
|
||||
|
||||
if not include_reconciled_entries:
|
||||
journal_entry_query = journal_entry_query.where(
|
||||
(journal_entry.clearance_date.isnull())
|
||||
| (journal_entry.clearance_date == ("0000-00-00" if frappe.db.db_type != "postgres" else None))
|
||||
(journal_entry.clearance_date.isnull()) | (journal_entry.clearance_date == "0000-00-00")
|
||||
)
|
||||
|
||||
journal_entries = (
|
||||
journal_entry_query.groupby(journal_entry_account.account, journal_entry.name)
|
||||
.orderby(Max(journal_entry.posting_date))
|
||||
.orderby(journal_entry.posting_date)
|
||||
.orderby(journal_entry.name, order=Order.desc)
|
||||
).run(as_dict=True)
|
||||
|
||||
@@ -294,8 +289,7 @@ def get_payment_entries_for_bank_clearance(
|
||||
|
||||
if not include_reconciled_entries:
|
||||
payment_entry_query = payment_entry_query.where(
|
||||
(pe.clearance_date.isnull())
|
||||
| (pe.clearance_date == ("0000-00-00" if frappe.db.db_type != "postgres" else None))
|
||||
(pe.clearance_date.isnull()) | (pe.clearance_date == "0000-00-00")
|
||||
)
|
||||
|
||||
payment_entries = (payment_entry_query.orderby(pe.posting_date).orderby(pe.name, order=Order.desc)).run(
|
||||
@@ -332,8 +326,7 @@ def get_payment_entries_for_bank_clearance(
|
||||
|
||||
if not include_reconciled_entries:
|
||||
paid_purchase_invoices_query = paid_purchase_invoices_query.where(
|
||||
(pi.clearance_date.isnull())
|
||||
| (pi.clearance_date == ("0000-00-00" if frappe.db.db_type != "postgres" else None))
|
||||
(pi.clearance_date.isnull()) | (pi.clearance_date == "0000-00-00")
|
||||
)
|
||||
|
||||
paid_purchase_invoices = (
|
||||
@@ -373,8 +366,7 @@ def get_payment_entries_for_bank_clearance(
|
||||
|
||||
if not include_reconciled_entries:
|
||||
pos_sales_invoices_query = pos_sales_invoices_query.where(
|
||||
(si_payment.clearance_date.isnull())
|
||||
| (si_payment.clearance_date == ("0000-00-00" if frappe.db.db_type != "postgres" else None))
|
||||
(si_payment.clearance_date.isnull()) | (si_payment.clearance_date == "0000-00-00")
|
||||
)
|
||||
|
||||
pos_sales_invoices = (
|
||||
|
||||
@@ -4,18 +4,29 @@
|
||||
import frappe
|
||||
from frappe.utils import add_months, getdate
|
||||
|
||||
from erpnext.accounts.doctype.cost_center.test_cost_center import create_cost_center
|
||||
from erpnext.accounts.doctype.mode_of_payment.test_mode_of_payment import (
|
||||
set_default_account_for_mode_of_payment,
|
||||
)
|
||||
from erpnext.accounts.doctype.payment_entry.test_payment_entry import get_payment_entry
|
||||
from erpnext.accounts.doctype.purchase_invoice.test_purchase_invoice import make_purchase_invoice
|
||||
from erpnext.accounts.doctype.sales_invoice.test_sales_invoice import create_sales_invoice
|
||||
from erpnext.stock.doctype.item.test_item import create_item
|
||||
from erpnext.stock.doctype.warehouse.test_warehouse import create_warehouse
|
||||
from erpnext.tests.utils import ERPNextTestSuite, if_lending_app_installed, if_lending_app_not_installed
|
||||
|
||||
|
||||
class TestBankClearance(ERPNextTestSuite):
|
||||
def setUp(self):
|
||||
frappe.clear_cache()
|
||||
create_warehouse(
|
||||
warehouse_name="_Test Warehouse",
|
||||
properties={"parent_warehouse": "All Warehouses - _TC"},
|
||||
company="_Test Company",
|
||||
)
|
||||
create_item("_Test Item")
|
||||
create_cost_center(cost_center_name="_Test Cost Center", company="_Test Company")
|
||||
|
||||
make_bank_account()
|
||||
add_transactions()
|
||||
|
||||
@@ -128,8 +139,11 @@ def add_transactions():
|
||||
|
||||
|
||||
def make_payment_entry():
|
||||
from erpnext.buying.doctype.supplier.test_supplier import create_supplier
|
||||
|
||||
supplier = create_supplier(supplier_name="_Test Supplier")
|
||||
pi = make_purchase_invoice(
|
||||
supplier="_Test Supplier",
|
||||
supplier=supplier.name,
|
||||
supplier_warehouse="_Test Warehouse - _TC",
|
||||
expense_account="Cost of Goods Sold - _TC",
|
||||
uom="Nos",
|
||||
@@ -144,6 +158,10 @@ def make_payment_entry():
|
||||
|
||||
|
||||
def make_pos_sales_invoice():
|
||||
from erpnext.accounts.doctype.opening_invoice_creation_tool.test_opening_invoice_creation_tool import (
|
||||
make_customer,
|
||||
)
|
||||
|
||||
mode_of_payment = frappe.get_doc({"doctype": "Mode of Payment", "name": "Cash"})
|
||||
|
||||
if not frappe.db.get_value("Mode of Payment Account", {"company": "_Test Company", "parent": "Cash"}):
|
||||
@@ -152,13 +170,13 @@ def make_pos_sales_invoice():
|
||||
)
|
||||
mode_of_payment.save()
|
||||
|
||||
customer = make_customer(customer="_Test Customer")
|
||||
|
||||
mode_of_payment = frappe.get_doc("Mode of Payment", "Wire Transfer")
|
||||
|
||||
set_default_account_for_mode_of_payment(mode_of_payment, "_Test Company", "_Test Bank Clearance - _TC")
|
||||
|
||||
si = create_sales_invoice(
|
||||
customer="_Test Customer", item="_Test Item", is_pos=1, qty=1, rate=1000, do_not_save=1
|
||||
)
|
||||
si = create_sales_invoice(customer=customer, item="_Test Item", is_pos=1, qty=1, rate=1000, do_not_save=1)
|
||||
si.set("payments", [])
|
||||
si.append("payments", {"mode_of_payment": "Wire Transfer", "amount": 1000})
|
||||
si.insert()
|
||||
|
||||
@@ -9,13 +9,6 @@ cur_frm.add_fetch("bank", "swift_number", "swift_number");
|
||||
|
||||
frappe.ui.form.on("Bank Guarantee", {
|
||||
setup: function (frm) {
|
||||
frm.set_query("reference_doctype", function () {
|
||||
return {
|
||||
filters: {
|
||||
name: ["in", ["Sales Order", "Purchase Order"]],
|
||||
},
|
||||
};
|
||||
});
|
||||
frm.set_query("bank_account", function () {
|
||||
return {
|
||||
filters: {
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
{
|
||||
"actions": [],
|
||||
"allow_bulk_edit": 1,
|
||||
"autoname": "ACC-BG-.YYYY.-.#####",
|
||||
"creation": "2016-12-17 10:43:35.731631",
|
||||
"doctype": "DocType",
|
||||
@@ -51,7 +50,8 @@
|
||||
"fieldname": "reference_doctype",
|
||||
"fieldtype": "Link",
|
||||
"label": "Reference Document Type",
|
||||
"options": "DocType"
|
||||
"options": "DocType",
|
||||
"read_only": 1
|
||||
},
|
||||
{
|
||||
"fieldname": "reference_docname",
|
||||
@@ -60,14 +60,14 @@
|
||||
"options": "reference_doctype"
|
||||
},
|
||||
{
|
||||
"depends_on": "eval: doc.reference_doctype == \"Sales Order\"",
|
||||
"depends_on": "eval: doc.bg_type == \"Receiving\"",
|
||||
"fieldname": "customer",
|
||||
"fieldtype": "Link",
|
||||
"label": "Customer",
|
||||
"options": "Customer"
|
||||
},
|
||||
{
|
||||
"depends_on": "eval: doc.reference_doctype == \"Purchase Order\"",
|
||||
"depends_on": "eval: doc.bg_type == \"Providing\"",
|
||||
"fieldname": "supplier",
|
||||
"fieldtype": "Link",
|
||||
"label": "Supplier",
|
||||
@@ -218,11 +218,10 @@
|
||||
"grid_page_length": 50,
|
||||
"is_submittable": 1,
|
||||
"links": [],
|
||||
"modified": "2026-05-25 18:12:10.768835",
|
||||
"modified": "2025-08-29 11:52:33.550847",
|
||||
"modified_by": "Administrator",
|
||||
"module": "Accounts",
|
||||
"name": "Bank Guarantee",
|
||||
"naming_rule": "Expression",
|
||||
"owner": "Administrator",
|
||||
"permissions": [
|
||||
{
|
||||
|
||||
@@ -1,76 +1,8 @@
|
||||
# Copyright (c) 2026, Frappe Technologies Pvt. Ltd. and Contributors
|
||||
# Copyright (c) 2018, Frappe Technologies Pvt. Ltd. and Contributors
|
||||
# See license.txt
|
||||
|
||||
import frappe
|
||||
from frappe.utils import flt
|
||||
|
||||
from erpnext.accounts.doctype.bank_guarantee.bank_guarantee import get_voucher_details
|
||||
from erpnext.selling.doctype.sales_order.test_sales_order import make_sales_order
|
||||
from erpnext.tests.utils import ERPNextTestSuite
|
||||
|
||||
BANK = "_Test BG Bank"
|
||||
|
||||
|
||||
class TestBankGuarantee(ERPNextTestSuite):
|
||||
"""Bank Guarantee records a guarantee issued/received against a customer or
|
||||
supplier. validate() needs a party; on_submit() needs the bank details filled in."""
|
||||
|
||||
def setUp(self):
|
||||
frappe.set_user("Administrator")
|
||||
if not frappe.db.exists("Bank", BANK):
|
||||
frappe.get_doc({"doctype": "Bank", "bank_name": BANK}).insert()
|
||||
|
||||
def make_bg(self, **args):
|
||||
args = frappe._dict(args)
|
||||
doc = frappe.new_doc("Bank Guarantee")
|
||||
doc.bg_type = args.bg_type or "Receiving"
|
||||
doc.amount = args.amount if args.amount is not None else 1000
|
||||
doc.start_date = args.start_date or "2026-06-01"
|
||||
if args.end_date:
|
||||
doc.end_date = args.end_date
|
||||
doc.customer = args.get("customer", "_Test Customer")
|
||||
doc.supplier = args.get("supplier")
|
||||
# fields on_submit requires — present by default, cleared per-test to assert the guard
|
||||
doc.bank_guarantee_number = args.get("bank_guarantee_number", "BG-001")
|
||||
doc.name_of_beneficiary = args.get("name_of_beneficiary", "Test Beneficiary")
|
||||
doc.bank = args.get("bank", BANK)
|
||||
return doc
|
||||
|
||||
def test_validate_requires_customer_or_supplier(self):
|
||||
doc = self.make_bg(customer=None)
|
||||
self.assertRaises(frappe.ValidationError, doc.insert)
|
||||
|
||||
def test_submit_requires_guarantee_number(self):
|
||||
doc = self.make_bg(bank_guarantee_number="")
|
||||
doc.insert()
|
||||
self.assertRaises(frappe.ValidationError, doc.submit)
|
||||
|
||||
def test_submit_requires_beneficiary_name(self):
|
||||
doc = self.make_bg(name_of_beneficiary="")
|
||||
doc.insert()
|
||||
self.assertRaises(frappe.ValidationError, doc.submit)
|
||||
|
||||
def test_submit_requires_bank(self):
|
||||
doc = self.make_bg(bank="")
|
||||
doc.insert()
|
||||
self.assertRaises(frappe.ValidationError, doc.submit)
|
||||
|
||||
def test_valid_guarantee_submits(self):
|
||||
doc = self.make_bg()
|
||||
doc.insert()
|
||||
doc.submit()
|
||||
self.assertEqual(frappe.db.get_value("Bank Guarantee", doc.name, "docstatus"), 1)
|
||||
|
||||
def test_get_voucher_details_for_receiving(self):
|
||||
so = make_sales_order()
|
||||
details = get_voucher_details("Receiving", so.name)
|
||||
self.assertEqual(details.customer, so.customer)
|
||||
self.assertEqual(flt(details.grand_total), flt(so.grand_total))
|
||||
|
||||
def test_end_date_before_start_date_is_not_validated(self):
|
||||
# SUSPECTED BUG: validate() never checks that end_date >= start_date, so a
|
||||
# guarantee that expires before it starts saves cleanly. Locking the current
|
||||
# (wrong) behaviour so a future fix that adds the check trips this test.
|
||||
doc = self.make_bg(start_date="2026-06-30", end_date="2026-06-01")
|
||||
doc.insert()
|
||||
self.assertTrue(frappe.db.exists("Bank Guarantee", doc.name))
|
||||
pass
|
||||
|
||||
@@ -8,7 +8,7 @@ import frappe
|
||||
from frappe import _
|
||||
from frappe.model.document import Document
|
||||
from frappe.query_builder.custom import ConstantColumn
|
||||
from frappe.query_builder.functions import Max, Sum
|
||||
from frappe.query_builder.functions import Sum
|
||||
from frappe.utils import cint, create_batch, flt
|
||||
|
||||
from erpnext import get_default_cost_center
|
||||
@@ -116,7 +116,7 @@ def get_account_balance(bank_account: str, till_date: str | date, company: str):
|
||||
return flt(balance_as_per_system) - flt(total_debit) + flt(total_credit) + amounts_not_reflected_in_system
|
||||
|
||||
|
||||
@frappe.whitelist(methods=["POST"])
|
||||
@frappe.whitelist()
|
||||
def update_bank_transaction(
|
||||
bank_transaction_name: str, reference_number: str, party_type: str | None = None, party: str | None = None
|
||||
):
|
||||
@@ -146,7 +146,7 @@ def update_bank_transaction(
|
||||
)[0]
|
||||
|
||||
|
||||
@frappe.whitelist(methods=["POST"])
|
||||
@frappe.whitelist()
|
||||
def create_journal_entry_bts(
|
||||
bank_transaction_name: str,
|
||||
reference_number: str | None = None,
|
||||
@@ -305,7 +305,7 @@ def create_journal_entry_bts(
|
||||
return reconcile_vouchers(bank_transaction_name, vouchers, is_new_voucher=True)
|
||||
|
||||
|
||||
@frappe.whitelist(methods=["POST"])
|
||||
@frappe.whitelist()
|
||||
def create_payment_entry_bts(
|
||||
bank_transaction_name: str,
|
||||
reference_number: str | None = None,
|
||||
@@ -500,7 +500,7 @@ def create_bulk_internal_transfer(bank_transaction_names: list[str | int], bank_
|
||||
return output
|
||||
|
||||
|
||||
@frappe.whitelist(methods=["POST"])
|
||||
@frappe.whitelist()
|
||||
def create_internal_transfer(
|
||||
bank_transaction_name: str | int,
|
||||
posting_date: str | date,
|
||||
@@ -518,7 +518,6 @@ def create_internal_transfer(
|
||||
"""
|
||||
|
||||
bank_transaction = frappe.get_doc("Bank Transaction", bank_transaction_name)
|
||||
bank_transaction.check_permission("write")
|
||||
|
||||
bank_account = frappe.get_cached_value("Bank Account", bank_transaction.bank_account, "account")
|
||||
company = frappe.get_cached_value("Account", bank_account, "company")
|
||||
@@ -779,6 +778,7 @@ def create_bulk_payment_entry_and_reconcile(
|
||||
"""
|
||||
Create a payment entry and reconcile it with the bank transaction
|
||||
"""
|
||||
|
||||
output = []
|
||||
|
||||
for bank_transaction_name in bank_transaction_names:
|
||||
@@ -912,7 +912,7 @@ def search_for_transfer_transaction(transaction_id: str | int):
|
||||
|
||||
days = frappe.db.get_single_value("Accounts Settings", "transfer_match_days")
|
||||
|
||||
if days is None:
|
||||
if not days:
|
||||
days = 3
|
||||
|
||||
min_date = frappe.utils.add_days(date, -days)
|
||||
@@ -1057,10 +1057,10 @@ def get_auto_reconcile_message(partially_reconciled, reconciled):
|
||||
return alert_message, indicator
|
||||
|
||||
|
||||
@frappe.whitelist(methods=["POST"])
|
||||
def reconcile_vouchers(bank_transaction_name: str | int, vouchers: str | list, is_new_voucher: bool = False):
|
||||
@frappe.whitelist()
|
||||
def reconcile_vouchers(bank_transaction_name: str | int, vouchers: str, is_new_voucher: bool = False):
|
||||
# updated clear date of all the vouchers based on the bank transaction
|
||||
vouchers = frappe.parse_json(vouchers)
|
||||
vouchers = json.loads(vouchers)
|
||||
transaction = frappe.get_doc("Bank Transaction", bank_transaction_name)
|
||||
transaction.add_payment_entries(vouchers, is_new_voucher)
|
||||
transaction.validate_duplicate_references()
|
||||
@@ -1336,11 +1336,9 @@ def get_pe_matching_query(
|
||||
ref_condition = pe.reference_no == transaction.reference_number
|
||||
ref_rank = frappe.qb.terms.Case().when(ref_condition, 1).else_(0)
|
||||
|
||||
amount_field = pe.received_amount_after_tax if account_from_to == "paid_to" else pe.paid_amount_after_tax
|
||||
|
||||
amount_equality = amount_field == transaction.unallocated_amount
|
||||
amount_equality = pe.paid_amount == transaction.unallocated_amount
|
||||
amount_rank = frappe.qb.terms.Case().when(amount_equality, 1).else_(0)
|
||||
amount_condition = amount_equality if exact_match else amount_field > 0.0
|
||||
amount_condition = amount_equality if exact_match else pe.paid_amount > 0.0
|
||||
|
||||
party_condition = (
|
||||
(pe.party_type == transaction.party_type) & (pe.party == transaction.party) & pe.party.isnotnull()
|
||||
@@ -1357,7 +1355,7 @@ def get_pe_matching_query(
|
||||
(ref_rank + amount_rank + party_rank + 1).as_("rank"),
|
||||
ConstantColumn("Payment Entry").as_("doctype"),
|
||||
pe.name,
|
||||
amount_field.as_("paid_amount"),
|
||||
pe.base_paid_amount_after_tax.as_("paid_amount"),
|
||||
pe.reference_no,
|
||||
pe.reference_date,
|
||||
pe.party,
|
||||
@@ -1412,14 +1410,12 @@ def get_je_matching_query(
|
||||
Sum(getattr(jea, amount_field)).as_("paid_amount"),
|
||||
ConstantColumn("Journal Entry").as_("doctype"),
|
||||
je.name,
|
||||
# non-grouped columns are constant per grouped JE name (party_type/currency come from the
|
||||
# single bank-account line) -> Max() keeps the GROUP BY valid on postgres with the same value
|
||||
Max(je.cheque_no).as_("reference_no"),
|
||||
Max(je.cheque_date).as_("reference_date"),
|
||||
Max(je.pay_to_recd_from).as_("party"),
|
||||
Max(jea.party_type).as_("party_type"),
|
||||
Max(je.posting_date).as_("posting_date"),
|
||||
Max(jea.account_currency).as_("currency"),
|
||||
je.cheque_no.as_("reference_no"),
|
||||
je.cheque_date.as_("reference_date"),
|
||||
je.pay_to_recd_from.as_("party"),
|
||||
jea.party_type,
|
||||
je.posting_date,
|
||||
jea.account_currency.as_("currency"),
|
||||
)
|
||||
.where(je.docstatus == 1)
|
||||
.where(je.voucher_type != "Opening Entry")
|
||||
@@ -1427,7 +1423,7 @@ def get_je_matching_query(
|
||||
.where(jea.account == common_filters.bank_account)
|
||||
.where(filter_by_date)
|
||||
.groupby(je.name)
|
||||
.orderby(Max(je.cheque_date) if cint(filter_by_reference_date) else Max(je.posting_date))
|
||||
.orderby(je.cheque_date if cint(filter_by_reference_date) else je.posting_date)
|
||||
)
|
||||
|
||||
if frappe.flags.auto_reconcile_vouchers is True:
|
||||
|
||||
@@ -8,9 +8,7 @@ from frappe.utils import add_days, today
|
||||
|
||||
from erpnext.accounts.doctype.bank_reconciliation_tool.bank_reconciliation_tool import (
|
||||
auto_reconcile_vouchers,
|
||||
get_auto_reconcile_message,
|
||||
get_bank_transactions,
|
||||
get_linked_payments,
|
||||
)
|
||||
from erpnext.accounts.doctype.payment_entry.test_payment_entry import create_payment_entry
|
||||
from erpnext.accounts.test.accounts_mixin import AccountsTestMixin
|
||||
@@ -19,10 +17,9 @@ from erpnext.tests.utils import ERPNextTestSuite
|
||||
|
||||
class TestBankReconciliationTool(ERPNextTestSuite, AccountsTestMixin):
|
||||
def setUp(self):
|
||||
self.company = "_Test Company"
|
||||
self.customer = "_Test Customer"
|
||||
self.bank = "HDFC - _TC"
|
||||
self.debit_to = "Debtors - _TC"
|
||||
self.create_company()
|
||||
self.create_customer()
|
||||
self.clear_old_entries()
|
||||
bank_dt = qb.DocType("Bank")
|
||||
qb.from_(bank_dt).delete().where(bank_dt.name == "HDFC").run()
|
||||
self.create_bank_account()
|
||||
@@ -99,103 +96,3 @@ class TestBankReconciliationTool(ERPNextTestSuite, AccountsTestMixin):
|
||||
# assert API output post reconciliation
|
||||
transactions = get_bank_transactions(self.bank_account, from_date, to_date)
|
||||
self.assertEqual(len(transactions), 0)
|
||||
|
||||
def make_bank_transaction(self, date, deposit=100, withdrawal=0):
|
||||
return (
|
||||
frappe.get_doc(
|
||||
{
|
||||
"doctype": "Bank Transaction",
|
||||
"date": date,
|
||||
"deposit": deposit,
|
||||
"withdrawal": withdrawal,
|
||||
"bank_account": self.bank_account,
|
||||
"currency": "INR",
|
||||
}
|
||||
)
|
||||
.save()
|
||||
.submit()
|
||||
)
|
||||
|
||||
def get_matching_payment_entries(self, bank_transaction, exact_match=False):
|
||||
document_types = ["payment_entry", "exact_match"] if exact_match else ["payment_entry"]
|
||||
vouchers = get_linked_payments(
|
||||
bank_transaction,
|
||||
document_types,
|
||||
from_date=add_days(today(), -1),
|
||||
to_date=today(),
|
||||
)
|
||||
return [v for v in vouchers if v.get("doctype") == "Payment Entry"]
|
||||
|
||||
def test_get_bank_transactions_excludes_dates_after_to_date(self):
|
||||
self.make_bank_transaction(date=today())
|
||||
names = [t.name for t in get_bank_transactions(self.bank_account, to_date=add_days(today(), -1))]
|
||||
self.assertEqual(names, [])
|
||||
|
||||
def test_deposit_matches_amount_received_in_bank_account(self):
|
||||
# money leaves another bank account and lands here minus a charge, so the two sides differ
|
||||
payment = frappe.get_doc(
|
||||
{
|
||||
"doctype": "Payment Entry",
|
||||
"payment_type": "Internal Transfer",
|
||||
"company": self.company,
|
||||
"posting_date": today(),
|
||||
"paid_from": "_Test Bank - _TC",
|
||||
"paid_to": self.bank,
|
||||
"paid_amount": 3537.64,
|
||||
"received_amount": 3460.52,
|
||||
"reference_no": "TRF-001",
|
||||
"reference_date": today(),
|
||||
}
|
||||
)
|
||||
payment.set_missing_values()
|
||||
payment.set_exchange_rate()
|
||||
payment.set_amounts()
|
||||
payment.deductions[-1].account = "_Test Exchange Gain/Loss - _TC"
|
||||
payment.deductions[-1].cost_center = "_Test Cost Center - _TC"
|
||||
payment = payment.save().submit()
|
||||
|
||||
transaction = self.make_bank_transaction(date=today(), deposit=3460.52)
|
||||
|
||||
# the received side is what reached this bank account, so that is what is shown
|
||||
matches = self.get_matching_payment_entries(transaction.name)
|
||||
self.assertEqual([m["name"] for m in matches], [payment.name])
|
||||
self.assertEqual(matches[0]["paid_amount"], 3460.52)
|
||||
|
||||
# and what the exact match compares against
|
||||
exact_matches = self.get_matching_payment_entries(transaction.name, exact_match=True)
|
||||
self.assertEqual([m["name"] for m in exact_matches], [payment.name])
|
||||
|
||||
def test_withdrawal_matches_amount_paid_from_bank_account(self):
|
||||
payment = create_payment_entry(
|
||||
company=self.company,
|
||||
payment_type="Pay",
|
||||
party_type="Supplier",
|
||||
party="_Test Supplier",
|
||||
paid_from=self.bank,
|
||||
paid_to="Creditors - _TC",
|
||||
paid_amount=1250,
|
||||
)
|
||||
payment = payment.save().submit()
|
||||
|
||||
transaction = self.make_bank_transaction(date=today(), deposit=0, withdrawal=1250)
|
||||
|
||||
exact_matches = self.get_matching_payment_entries(transaction.name, exact_match=True)
|
||||
self.assertEqual([m["name"] for m in exact_matches], [payment.name])
|
||||
self.assertEqual(exact_matches[0]["paid_amount"], 1250)
|
||||
|
||||
def test_auto_reconcile_message_for_no_matches(self):
|
||||
message, indicator = get_auto_reconcile_message([], [])
|
||||
self.assertEqual(indicator, "blue")
|
||||
self.assertIn("No matches", message)
|
||||
|
||||
def test_auto_reconcile_message_counts_and_pluralizes(self):
|
||||
# reconciled count is reported and the indicator turns green
|
||||
message, indicator = get_auto_reconcile_message([], ["t1", "t2"])
|
||||
self.assertEqual(indicator, "green")
|
||||
self.assertIn("2 Transaction(s) Reconciled", message)
|
||||
|
||||
# partially-reconciled label is singular for one, plural for many
|
||||
singular, _ = get_auto_reconcile_message(["p1"], [])
|
||||
self.assertIn("1 Transaction Partially Reconciled", singular)
|
||||
plural, _ = get_auto_reconcile_message(["p1", "p2"], [])
|
||||
self.assertIn("2 Transactions Partially Reconciled", plural)
|
||||
|
||||
@@ -221,12 +221,12 @@
|
||||
"default": "0",
|
||||
"fieldname": "import_mt940_fromat",
|
||||
"fieldtype": "Check",
|
||||
"label": "Import MT940 Format"
|
||||
"label": "Import MT940 Fromat"
|
||||
}
|
||||
],
|
||||
"hide_toolbar": 1,
|
||||
"links": [],
|
||||
"modified": "2026-06-19 14:18:00.000000",
|
||||
"modified": "2026-05-31 00:41:11.251215",
|
||||
"modified_by": "Administrator",
|
||||
"module": "Accounts",
|
||||
"name": "Bank Statement Import",
|
||||
|
||||
@@ -142,34 +142,9 @@ def preprocess_mt940_content(content: str) -> str:
|
||||
return processed_content
|
||||
|
||||
|
||||
MT940_CUSTOMER_REFERENCE_MAX_LEN = 16
|
||||
|
||||
|
||||
def get_transaction_reference(txn_data: dict) -> str:
|
||||
"""Extract the per-transaction reference from an MT940 :61: tag.
|
||||
|
||||
The mt940 library exposes ``transaction_reference`` from the :20: tag, which is the
|
||||
statement-level reference and identical for every transaction in a statement. The
|
||||
real per-transaction reference is ``customer_reference`` (with any overflow captured
|
||||
into ``extra_details`` when a bank emits a single-line :61: longer than 16 chars).
|
||||
"""
|
||||
customer_reference = (txn_data.get("customer_reference") or "").strip()
|
||||
|
||||
if len(customer_reference) == MT940_CUSTOMER_REFERENCE_MAX_LEN:
|
||||
customer_reference += (txn_data.get("extra_details") or "").strip()
|
||||
|
||||
if customer_reference and customer_reference.upper() != "NONREF":
|
||||
return customer_reference
|
||||
|
||||
return (txn_data.get("bank_reference") or "").strip() or (
|
||||
txn_data.get("transaction_reference") or ""
|
||||
).strip()
|
||||
|
||||
|
||||
@frappe.whitelist(methods=["POST"])
|
||||
@frappe.whitelist()
|
||||
def convert_mt940_to_csv(data_import: str, mt940_file_path: str):
|
||||
doc = frappe.get_doc("Bank Statement Import", data_import)
|
||||
doc.check_permission("write")
|
||||
|
||||
_file_doc, content = get_file(mt940_file_path)
|
||||
|
||||
@@ -214,8 +189,8 @@ def convert_mt940_to_csv(data_import: str, mt940_file_path: str):
|
||||
|
||||
deposit = amount_value if amount_value > 0 else ""
|
||||
withdrawal = abs(amount_value) if amount_value < 0 else ""
|
||||
description = txn.data.get("transaction_details") or txn.data.get("extra_details") or ""
|
||||
reference = get_transaction_reference(txn.data)
|
||||
description = txn.data.get("extra_details") or ""
|
||||
reference = txn.data.get("transaction_reference") or ""
|
||||
currency = txn.data.get("currency", "")
|
||||
|
||||
writer.writerow([date_str, deposit, withdrawal, description, reference, doc.bank_account, currency])
|
||||
@@ -236,30 +211,26 @@ def convert_mt940_to_csv(data_import: str, mt940_file_path: str):
|
||||
def get_preview_from_template(
|
||||
data_import: str, import_file: str | None = None, google_sheets_url: str | None = None
|
||||
):
|
||||
bsi = frappe.get_doc("Bank Statement Import", data_import)
|
||||
bsi.check_permission()
|
||||
return bsi.get_preview_from_template(import_file, google_sheets_url)
|
||||
return frappe.get_doc("Bank Statement Import", data_import).get_preview_from_template(
|
||||
import_file, google_sheets_url
|
||||
)
|
||||
|
||||
|
||||
@frappe.whitelist()
|
||||
def form_start_import(data_import: str):
|
||||
bsi = frappe.get_doc("Bank Statement Import", data_import)
|
||||
bsi.check_permission("write")
|
||||
return bsi.start_import()
|
||||
job_id = frappe.get_doc("Bank Statement Import", data_import).start_import()
|
||||
return job_id is not None
|
||||
|
||||
|
||||
@frappe.whitelist()
|
||||
def download_errored_template(data_import_name: str):
|
||||
data_import = frappe.get_doc("Bank Statement Import", data_import_name)
|
||||
data_import.check_permission()
|
||||
data_import.export_errored_rows()
|
||||
|
||||
|
||||
@frappe.whitelist()
|
||||
def download_import_log(data_import_name: str):
|
||||
bsi = frappe.get_doc("Bank Statement Import", data_import_name)
|
||||
bsi.check_permission()
|
||||
return bsi.download_import_log()
|
||||
return frappe.get_doc("Bank Statement Import", data_import_name).download_import_log()
|
||||
|
||||
|
||||
def is_mt940_format(content: str) -> bool:
|
||||
@@ -319,7 +290,7 @@ def update_mapping_db(bank, template_options):
|
||||
for d in bank.bank_transaction_mapping:
|
||||
d.delete()
|
||||
|
||||
for d in frappe.parse_json(template_options)["column_to_field_map"].items():
|
||||
for d in json.loads(template_options)["column_to_field_map"].items():
|
||||
bank.append("bank_transaction_mapping", {"bank_transaction_field": d[1], "file_field": d[0]})
|
||||
|
||||
bank.save()
|
||||
@@ -336,7 +307,7 @@ def add_bank_account(data, bank_account):
|
||||
bank_account_loc = loc
|
||||
|
||||
for row in data[1:]:
|
||||
if bank_account_loc is not None:
|
||||
if bank_account_loc:
|
||||
row[bank_account_loc] = bank_account
|
||||
else:
|
||||
row.append(bank_account)
|
||||
@@ -398,7 +369,6 @@ def get_import_status(docname: str):
|
||||
import_status = {}
|
||||
|
||||
data_import = frappe.get_doc("Bank Statement Import", docname)
|
||||
data_import.check_permission()
|
||||
import_status["status"] = data_import.status
|
||||
|
||||
logs = frappe.get_all(
|
||||
|
||||
@@ -1,10 +1,7 @@
|
||||
# Copyright (c) 2020, Frappe Technologies and Contributors
|
||||
# See license.txt
|
||||
|
||||
import mt940
|
||||
|
||||
from erpnext.accounts.doctype.bank_statement_import.bank_statement_import import (
|
||||
get_transaction_reference,
|
||||
is_mt940_format,
|
||||
preprocess_mt940_content,
|
||||
)
|
||||
@@ -191,135 +188,6 @@ class TestBankStatementImport(ERPNextTestSuite):
|
||||
self.assertIn(":20:STMTREF167619", result) # Reference should remain unchanged
|
||||
self.assertIn("UPI/TEST USER/123456789/PaidViaTestApp", result)
|
||||
|
||||
def test_get_transaction_reference_uses_customer_reference(self):
|
||||
"""Per-transaction reference must come from :61: customer_reference, not :20:."""
|
||||
self.assertEqual(
|
||||
get_transaction_reference(
|
||||
{"customer_reference": "UPI-100000000001", "transaction_reference": "STMTREF12345"}
|
||||
),
|
||||
"UPI-100000000001",
|
||||
)
|
||||
|
||||
def test_get_transaction_reference_rejoins_overflow(self):
|
||||
"""When a bank emits a single-line :61: with >16-char reference, the regex
|
||||
splits the tail into extra_details. We must rejoin them."""
|
||||
self.assertEqual(
|
||||
get_transaction_reference(
|
||||
{
|
||||
"customer_reference": "NEFTINW-12345678",
|
||||
"extra_details": "90",
|
||||
"transaction_reference": "STMTREF12345",
|
||||
}
|
||||
),
|
||||
"NEFTINW-1234567890",
|
||||
)
|
||||
|
||||
def test_get_transaction_reference_falls_back_to_bank_reference_on_nonref(self):
|
||||
"""NONREF is the MT940 'no customer reference' sentinel; prefer bank_reference."""
|
||||
self.assertEqual(
|
||||
get_transaction_reference(
|
||||
{
|
||||
"customer_reference": "NONREF",
|
||||
"bank_reference": "1234567890123456",
|
||||
"transaction_reference": "STMTREF12345",
|
||||
}
|
||||
),
|
||||
"1234567890123456",
|
||||
)
|
||||
|
||||
def test_get_transaction_reference_falls_back_to_bank_reference_on_nonref_with_extra_details(self):
|
||||
"""NONREF sentinel must trigger the bank_reference fallback even when
|
||||
extra_details is populated. Without the 16-char gate, the old naive concat
|
||||
would produce a junk reference like 'NONREFsome info' and bypass the check."""
|
||||
self.assertEqual(
|
||||
get_transaction_reference(
|
||||
{
|
||||
"customer_reference": "NONREF",
|
||||
"extra_details": "some info",
|
||||
"bank_reference": "1234567890123456",
|
||||
"transaction_reference": "STMTREF12345",
|
||||
}
|
||||
),
|
||||
"1234567890123456",
|
||||
)
|
||||
|
||||
def test_get_transaction_reference_does_not_append_extra_details_below_16_chars(self):
|
||||
"""When customer_reference is below the 16-char cap, extra_details is a
|
||||
genuine supplementary-info field from :61: — not overflow — and must not
|
||||
be appended to the reference."""
|
||||
self.assertEqual(
|
||||
get_transaction_reference(
|
||||
{
|
||||
"customer_reference": "TBMS-123456789",
|
||||
"extra_details": "note field",
|
||||
"transaction_reference": "STMTREF12345",
|
||||
}
|
||||
),
|
||||
"TBMS-123456789",
|
||||
)
|
||||
|
||||
def test_get_transaction_reference_keeps_noref_literal(self):
|
||||
"""Bare 'NOREF' (without bank_reference) stays as-is; still better than the
|
||||
statement-level reference which is identical across all transactions."""
|
||||
self.assertEqual(
|
||||
get_transaction_reference(
|
||||
{
|
||||
"customer_reference": "NOREF",
|
||||
"bank_reference": None,
|
||||
"transaction_reference": "STMTREF12345",
|
||||
}
|
||||
),
|
||||
"NOREF",
|
||||
)
|
||||
|
||||
def test_mt940_parse_per_transaction_reference_mapping(self):
|
||||
"""End-to-end: every transaction in a statement must get its own distinct
|
||||
reference from :61: customer_reference, never the statement-level :20: reference."""
|
||||
mt940_content = """{1:F0112345678901X0000000000}{2:I94012345678901XN}{4:
|
||||
:20:STMTREF12345
|
||||
:25:1234567890
|
||||
:28C:12345/1
|
||||
:60F:C250716INR88123,38
|
||||
:61:2509280928D5000,00NMSCUPI-100000000001
|
||||
:86:UPI/TEST PAYEE ONE/111111111111/TestApp
|
||||
:61:2509190919D2606,00NMSCUPI-100000000002
|
||||
:86:UPI/TEST PAYEE TWO/222222222222/TestApp
|
||||
:61:2509190919D900,00NMSCUPI-100000000003
|
||||
:86:UPI/TEST PAYEE THREE/333333333333/TestApp
|
||||
:61:2508140814D5000,00NMSCUPI-100000000004
|
||||
:86:UPI/TEST PAYEE FOUR/444444444444/TestApp
|
||||
:61:2508060806D2000,00NMSCUPI-100000000005
|
||||
:86:UPI/TEST PAYEE FIVE/555555555555/TestApp
|
||||
:61:2508030803D1066,00NMSC123456789012
|
||||
:86:PCD/1234/TEST MERCHANT/01234567890123/12:00
|
||||
:61:2507310731D305,62NMSCTBMS-123456789
|
||||
:86:Chrg: Debit Card Annual Fee 1234 for 2025
|
||||
:61:2507240724C1,00NMSCNEFTINW-1234567890
|
||||
:86:NEFT TEST123456789 TEST SERVICES
|
||||
:61:2507170717C100000,00NMSCNOREF
|
||||
:86:BY CLG INST 123456/01-01-25/TESTBANK/TESTCITY
|
||||
:62F:C250930INR100000,00
|
||||
-}"""
|
||||
transactions = list(mt940.parse(preprocess_mt940_content(mt940_content)))
|
||||
references = [get_transaction_reference(t.data) for t in transactions]
|
||||
|
||||
self.assertEqual(
|
||||
references,
|
||||
[
|
||||
"UPI-100000000001",
|
||||
"UPI-100000000002",
|
||||
"UPI-100000000003",
|
||||
"UPI-100000000004",
|
||||
"UPI-100000000005",
|
||||
"123456789012",
|
||||
"TBMS-123456789",
|
||||
"NEFTINW-1234567890",
|
||||
"NOREF",
|
||||
],
|
||||
)
|
||||
# No transaction should carry the statement-level reference from :20:
|
||||
self.assertNotIn("STMTREF12345", references)
|
||||
|
||||
def test_preprocess_mt940_content_whitespace_variants(self):
|
||||
"""Test handling of whitespace and different line endings"""
|
||||
# Test with trailing spaces
|
||||
|
||||
@@ -54,6 +54,7 @@
|
||||
"fieldtype": "Currency",
|
||||
"in_list_view": 1,
|
||||
"label": "Closing Balance",
|
||||
"non_negative": 1,
|
||||
"options": "currency"
|
||||
},
|
||||
{
|
||||
@@ -190,7 +191,7 @@
|
||||
"grid_page_length": 50,
|
||||
"index_web_pages_for_search": 1,
|
||||
"links": [],
|
||||
"modified": "2026-07-09 17:55:25.615942",
|
||||
"modified": "2026-05-08 17:55:25.615942",
|
||||
"modified_by": "Administrator",
|
||||
"module": "Accounts",
|
||||
"name": "Bank Statement Import Log",
|
||||
|
||||
@@ -375,7 +375,8 @@ class BankStatementImportLog(Document):
|
||||
table["column_mapping"] = guess_column_mapping_by_content(table["rows"])
|
||||
|
||||
final_transactions, table["date_format"], table["amount_format"] = build_table_transactions(table)
|
||||
table["included"] = should_include_table(table, final_transactions)
|
||||
# Tables with no detectable transactions (ads, summaries, headers) start excluded.
|
||||
table["included"] = bool(final_transactions)
|
||||
|
||||
self.pdf_tables = json.dumps(tables)
|
||||
return tables
|
||||
@@ -541,8 +542,6 @@ class BankStatementImportLog(Document):
|
||||
"bank-rec-statement-import-progress",
|
||||
{
|
||||
"progress": round(progress / total_transactions * 100),
|
||||
"current": progress,
|
||||
"total": total_transactions,
|
||||
},
|
||||
doctype="Bank Statement Import Log",
|
||||
docname=self.name,
|
||||
@@ -552,14 +551,13 @@ class BankStatementImportLog(Document):
|
||||
"bank-rec-statement-import-progress",
|
||||
{
|
||||
"progress": 100,
|
||||
"current": total_transactions,
|
||||
"total": total_transactions,
|
||||
},
|
||||
doctype="Bank Statement Import Log",
|
||||
docname=self.name,
|
||||
)
|
||||
|
||||
if self.closing_balance is not None and self.end_date:
|
||||
if self.closing_balance and self.closing_balance > 0 and self.end_date:
|
||||
set_closing_balance_as_per_statement(
|
||||
self.bank_account, frappe.utils.getdate(self.end_date), self.closing_balance
|
||||
)
|
||||
@@ -823,15 +821,6 @@ def compute_final_transactions(transaction_rows: list, date_format: str, amount_
|
||||
"""Pure version of the final-transaction builder (date normalized, amount split)."""
|
||||
final_transactions = []
|
||||
|
||||
# Which marker does this statement actually write? A statement that only ever says "Cr"
|
||||
# is marking the credits as its exceptions, so an unmarked row is a withdrawal; one that
|
||||
# only ever says "Dr" means the opposite. With both markers present an unmarked row is
|
||||
# genuinely undetermined, so it stays a withdrawal.
|
||||
unmarked_is_deposit = False
|
||||
if amount_format == 'Amount column has "CR"/"DR" values':
|
||||
markers = {get_amount_cr_dr_marker(row.get("amount")) for row in transaction_rows}
|
||||
unmarked_is_deposit = markers - {None} == {"dr"}
|
||||
|
||||
def parse_amount(transaction_row: dict):
|
||||
if amount_format == "Separate columns for withdrawal and deposit":
|
||||
return get_float_amount(transaction_row.get("withdrawal")), get_float_amount(
|
||||
@@ -840,43 +829,42 @@ def compute_final_transactions(transaction_rows: list, date_format: str, amount_
|
||||
|
||||
if amount_format == 'Amount column has "CR"/"DR" values':
|
||||
amount = transaction_row.get("amount")
|
||||
marker = get_amount_cr_dr_marker(amount)
|
||||
# The marker carries the direction, so the amount's own sign is ignored.
|
||||
signed_amount = get_float_amount(amount) or 0
|
||||
float_amount = get_float_amount(amount)
|
||||
if "cr" in amount.lower():
|
||||
return 0, float_amount
|
||||
else:
|
||||
return float_amount, 0
|
||||
|
||||
if marker:
|
||||
return (0, abs(signed_amount)) if marker == "cr" else (abs(signed_amount), 0)
|
||||
|
||||
# An unmarked row takes the opposite direction to the marker this statement
|
||||
# uses. A negative amount reverses that again (a refund).
|
||||
is_deposit = unmarked_is_deposit
|
||||
if signed_amount < 0:
|
||||
is_deposit = not is_deposit
|
||||
|
||||
return (0, abs(signed_amount)) if is_deposit else (abs(signed_amount), 0)
|
||||
|
||||
# `or 0` below: get_float_amount returns None for an unparseable cell, and a blank
|
||||
# transaction-type cell comes through as None. Both used to raise.
|
||||
if amount_format == "Amount column has positive/negative values":
|
||||
amount = get_float_amount(transaction_row.get("amount", "0")) or 0
|
||||
amount = get_float_amount(transaction_row.get("amount", "0"))
|
||||
if amount > 0:
|
||||
return 0, abs(amount)
|
||||
else:
|
||||
return abs(amount), 0
|
||||
|
||||
transaction_type = str(transaction_row.get("debit_credit") or "").strip().lower()
|
||||
amount = abs(get_float_amount(transaction_row.get("amount", "0")) or 0)
|
||||
|
||||
if amount_format == 'Transaction type column has "CR"/"DR" values':
|
||||
# "credit" contains "cr". "debit" does not contain "dr", so it correctly falls
|
||||
# through to the withdrawal side.
|
||||
return (0, amount) if "cr" in transaction_type else (amount, 0)
|
||||
transaction_type = transaction_row.get("debit_credit")
|
||||
amount = get_float_amount(transaction_row.get("amount", "0"))
|
||||
if "cr" in transaction_type.lower():
|
||||
return 0, abs(amount)
|
||||
else:
|
||||
return abs(amount), 0
|
||||
|
||||
if amount_format == 'Transaction type column has "C"/"D" values':
|
||||
return (0, amount) if transaction_type == "c" else (amount, 0)
|
||||
transaction_type = transaction_row.get("debit_credit")
|
||||
amount = get_float_amount(transaction_row.get("amount", "0"))
|
||||
if transaction_type.lower().strip() == "c":
|
||||
return 0, abs(amount)
|
||||
else:
|
||||
return abs(amount), 0
|
||||
|
||||
if amount_format == 'Transaction type column has "Deposit"/"Withdrawal" values':
|
||||
return (0, amount) if "deposit" in transaction_type else (amount, 0)
|
||||
transaction_type = transaction_row.get("debit_credit")
|
||||
amount = get_float_amount(transaction_row.get("amount", "0"))
|
||||
if "deposit" in transaction_type.lower():
|
||||
return 0, abs(amount)
|
||||
else:
|
||||
return abs(amount), 0
|
||||
|
||||
return 0, 0
|
||||
|
||||
@@ -920,26 +908,6 @@ def build_table_transactions(table: dict):
|
||||
return final_transactions, date_format, amount_format
|
||||
|
||||
|
||||
def should_include_table(table: dict, final_transactions: list) -> bool:
|
||||
"""
|
||||
Whether a freshly extracted PDF table should START as included - only the default state
|
||||
of the checkbox, which the user can change afterwards.
|
||||
|
||||
It must have yielded transactions, and it must have a Description column mapped. A
|
||||
transaction table always carries a narration; the summary boxes printed around it -
|
||||
payment due, credit limit, reward points - are dates and figures only. Otherwise the
|
||||
HDFC credit-card "Payment Due Date / Total Dues / Minimum Amount Due" box parses as one
|
||||
transaction and imports a phantom row.
|
||||
|
||||
A description is NOT needed to import (it is not mandatory on Bank Transaction), so a
|
||||
bank that omits narration still works - its table just starts unticked.
|
||||
"""
|
||||
if not final_transactions:
|
||||
return False
|
||||
|
||||
return any(column.get("maps_to") == "Description" for column in table.get("column_mapping", []))
|
||||
|
||||
|
||||
def _clean_cell(cell) -> str:
|
||||
"""Normalize a pdfplumber cell: None -> '', collapse wrapped newlines, strip."""
|
||||
if cell is None:
|
||||
@@ -964,18 +932,14 @@ def extract_pdf_tables(content: bytes, password: str | None = None) -> list[dict
|
||||
from pypdf import PdfReader
|
||||
|
||||
reader = PdfReader(io.BytesIO(content))
|
||||
if reader.is_encrypted:
|
||||
# Try opening the PDF with a password - if no password is provided, try with a blank password
|
||||
if not password:
|
||||
password = ""
|
||||
if not reader.decrypt(password):
|
||||
frappe.throw(
|
||||
_(
|
||||
"This PDF is password protected. Please set the correct statement password on the"
|
||||
" Bank Account and try again."
|
||||
),
|
||||
title=_("Password Required"),
|
||||
)
|
||||
if reader.is_encrypted and (not password or not reader.decrypt(password)):
|
||||
frappe.throw(
|
||||
_(
|
||||
"This PDF is password protected. Please set the correct statement password on the"
|
||||
" Bank Account and try again."
|
||||
),
|
||||
title=_("Password Required"),
|
||||
)
|
||||
|
||||
text_settings = {"vertical_strategy": "text", "horizontal_strategy": "text"}
|
||||
tables = []
|
||||
@@ -1085,43 +1049,6 @@ def get_float_amount(amount):
|
||||
return amount
|
||||
|
||||
|
||||
# A "CR"/"DR" marker on the amount itself, at either end: "2,378.00Cr", "Cr 100",
|
||||
# "INR 50.90 Cr.", "DR 1,234.50".
|
||||
# `(?![a-zA-Z])` rather than `\b` on the leading form: there is no word boundary between
|
||||
# the "r" of "Cr100" and the digit, but there IS one inside "CREDIT" and "DRAFT".
|
||||
AMOUNT_CR_DR_PATTERN = re.compile(r"^\s*(cr|dr)(?![a-zA-Z])\.?|(?:^|[\s\d.)])(cr|dr)\b\.?\s*$", re.IGNORECASE)
|
||||
|
||||
|
||||
def get_amount_cr_dr_marker(amount) -> str | None:
|
||||
"""
|
||||
Return "cr" or "dr" if the amount cell carries a direction marker of its own, else None.
|
||||
|
||||
What is left after removing the marker has to look like an amount - it must hold a digit
|
||||
and at most a short currency token - so that text which merely starts or ends with the
|
||||
letters is not read as a marker. That guard is what separates "Cr 100" from a
|
||||
description that bled into the amount column, like "Dr Smith Clinic 500".
|
||||
"""
|
||||
if not isinstance(amount, str):
|
||||
return None
|
||||
|
||||
match = AMOUNT_CR_DR_PATTERN.search(amount)
|
||||
if not match:
|
||||
return None
|
||||
|
||||
# Only the marker itself is removed - the surrounding character the pattern needed to
|
||||
# anchor on (a digit, say) stays part of the remainder.
|
||||
group = 1 if match.group(1) else 2
|
||||
start, end = match.span(group)
|
||||
remainder = amount[:start] + amount[end:]
|
||||
|
||||
if not any(char.isdigit() for char in remainder):
|
||||
return None
|
||||
if sum(char.isalpha() for char in remainder) > 3:
|
||||
return None
|
||||
|
||||
return match.group(group).lower()
|
||||
|
||||
|
||||
def get_file_properties(transactions: list):
|
||||
"""
|
||||
From the transaction rows, try to figure out the following:
|
||||
@@ -1142,8 +1069,6 @@ def get_file_properties(transactions: list):
|
||||
'Transaction type column has "C"/"D" values': 0,
|
||||
}
|
||||
|
||||
amount_column_has_cr_dr = False
|
||||
|
||||
for transaction in transactions:
|
||||
date_format = transaction.get("date_format")
|
||||
|
||||
@@ -1161,40 +1086,33 @@ def get_file_properties(transactions: list):
|
||||
if not amount:
|
||||
continue
|
||||
|
||||
debit_credit = str(transaction.get("debit_credit") or "").strip().lower()
|
||||
|
||||
# One vote per row, most specific signal first. Order matters: "withdrawal" contains
|
||||
# "dr", so it must be matched before the loose cr/dr check or a Deposit/Withdrawal
|
||||
# column reads as CR/DR. "debit" needs listing because, unlike "credit", it does not
|
||||
# contain "dr". The final else means every row votes, even an unrecognised type.
|
||||
if get_amount_cr_dr_marker(amount):
|
||||
amount_column_has_cr_dr = True
|
||||
if isinstance(amount, str) and ("cr" in amount.lower() or "dr" in amount.lower()):
|
||||
amount_format_frequency['Amount column has "CR"/"DR" values'] += 1
|
||||
elif "deposit" in debit_credit or "withdrawal" in debit_credit:
|
||||
amount_format_frequency['Transaction type column has "Deposit"/"Withdrawal" values'] += 1
|
||||
elif debit_credit in ("c", "d"):
|
||||
amount_format_frequency['Transaction type column has "C"/"D" values'] += 1
|
||||
elif any(token in debit_credit for token in ("cr", "dr", "debit")):
|
||||
amount_format_frequency['Transaction type column has "CR"/"DR" values'] += 1
|
||||
|
||||
# Check if there's a debit_credit column containing "cr"/"dr"
|
||||
if transaction.get("debit_credit", None):
|
||||
if (
|
||||
"cr" in transaction.get("debit_credit", "").lower()
|
||||
or "dr" in transaction.get("debit_credit", "").lower()
|
||||
):
|
||||
amount_format_frequency['Transaction type column has "CR"/"DR" values'] += 1
|
||||
elif (
|
||||
"deposit" in transaction.get("debit_credit", "").lower()
|
||||
or "withdrawal" in transaction.get("debit_credit", "").lower()
|
||||
):
|
||||
amount_format_frequency['Transaction type column has "Deposit"/"Withdrawal" values'] += 1
|
||||
elif (transaction.get("debit_credit", "").lower().strip() == "c") or (
|
||||
transaction.get("debit_credit", "").lower().strip() == "d"
|
||||
):
|
||||
amount_format_frequency['Transaction type column has "C"/"D" values'] += 1
|
||||
|
||||
# Else assume that the amount is expressed as positive/negative value
|
||||
else:
|
||||
# Nothing said which direction this is, so assume the amount carries the sign.
|
||||
amount_format_frequency["Amount column has positive/negative values"] += 1
|
||||
|
||||
most_common_date_format = max(date_format_frequency, key=date_format_frequency.get)
|
||||
most_common_amount_format = max(amount_format_frequency, key=amount_format_frequency.get)
|
||||
|
||||
# With no votes at all (no rows, or every amount blank) max() would return whichever key
|
||||
# happens to be first in the dict. Say what we mean instead.
|
||||
if not amount_format_frequency[most_common_amount_format]:
|
||||
most_common_amount_format = "Amount column has positive/negative values"
|
||||
|
||||
# A CR/DR amount column is proved by a single marker, not by a majority: both formats
|
||||
# describe the same column, and an unmarked row is only the default direction, not
|
||||
# evidence against the notation. Statements mark just the exceptions - one HDFC
|
||||
# credit-card page has 18 rows and a single "50.90Cr".
|
||||
if amount_column_has_cr_dr and most_common_amount_format == "Amount column has positive/negative values":
|
||||
most_common_amount_format = 'Amount column has "CR"/"DR" values'
|
||||
|
||||
return most_common_date_format, most_common_amount_format
|
||||
|
||||
|
||||
@@ -1265,7 +1183,8 @@ def update_pdf_tables(statement_import_id: str, tables: list | str):
|
||||
if doc.status == "Completed":
|
||||
frappe.throw(_("This statement has already been imported."), title=_("Already Imported"))
|
||||
|
||||
tables = frappe.parse_json(tables)
|
||||
if isinstance(tables, str):
|
||||
tables = json.loads(tables)
|
||||
|
||||
doc.apply_pdf_tables(tables)
|
||||
|
||||
@@ -1285,7 +1204,8 @@ def reextract_pdf_table(statement_import_id: str, page: int, table_index: int, b
|
||||
if doc.status == "Completed":
|
||||
frappe.throw(_("This statement has already been imported."), title=_("Already Imported"))
|
||||
|
||||
bbox = frappe.parse_json(bbox)
|
||||
if isinstance(bbox, str):
|
||||
bbox = json.loads(bbox)
|
||||
|
||||
page = int(page)
|
||||
table_index = int(table_index)
|
||||
@@ -1370,7 +1290,8 @@ def update_column_mapping(statement_import_id: str, column_mapping: list | str):
|
||||
if doc.status == "Completed":
|
||||
frappe.throw(_("This statement has already been imported."), title=_("Already Imported"))
|
||||
|
||||
column_mapping = frappe.parse_json(column_mapping)
|
||||
if isinstance(column_mapping, str):
|
||||
column_mapping = json.loads(column_mapping)
|
||||
|
||||
doc.apply_column_mapping(column_mapping)
|
||||
doc.save()
|
||||
|
||||
@@ -11,14 +11,12 @@ from erpnext.accounts.doctype.bank_statement_import_log.bank_statement_import_lo
|
||||
detect_column_mapping,
|
||||
detect_header_row,
|
||||
extract_pdf_tables,
|
||||
get_amount_cr_dr_marker,
|
||||
get_float_amount,
|
||||
get_statement_details,
|
||||
guess_column_mapping_by_content,
|
||||
reextract_pdf_table,
|
||||
set_header_index,
|
||||
set_pdf_table_header,
|
||||
should_include_table,
|
||||
update_column_mapping,
|
||||
update_pdf_tables,
|
||||
)
|
||||
@@ -28,9 +26,9 @@ from erpnext.tests.utils import ERPNextTestSuite
|
||||
|
||||
class TestBankStatementImportLog(ERPNextTestSuite, AccountsTestMixin):
|
||||
def setUp(self):
|
||||
self.company = "_Test Company"
|
||||
self.customer = "_Test Customer"
|
||||
self.bank = "HDFC - _TC"
|
||||
self.create_company()
|
||||
self.create_customer()
|
||||
self.clear_old_entries()
|
||||
bank_dt = qb.DocType("Bank")
|
||||
qb.from_(bank_dt).delete().where(bank_dt.name == "HDFC").run()
|
||||
self.create_bank_account()
|
||||
@@ -126,184 +124,6 @@ class TestBankStatementImportLog(ERPNextTestSuite, AccountsTestMixin):
|
||||
self.assertIsNone(get_float_amount("ABCD"))
|
||||
self.assertIsNone(get_float_amount("****"))
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# Amount format detection
|
||||
# ------------------------------------------------------------------ #
|
||||
|
||||
def test_amount_cr_dr_marker(self):
|
||||
"""The marker is read at either end of the cell, but only next to the amount."""
|
||||
for amount in ("2,378.00Cr", "50.90 CR", "INR 50.90 Cr.", "1000cr", "5cr", "(100) Cr"):
|
||||
self.assertEqual(get_amount_cr_dr_marker(amount), "cr", amount)
|
||||
|
||||
for amount in ("2,378.00Dr", "50.90 DR", "1000dr", "-100 Dr"):
|
||||
self.assertEqual(get_amount_cr_dr_marker(amount), "dr", amount)
|
||||
|
||||
# Some banks put the marker in front of the digits instead.
|
||||
for amount in ("Cr 100", "Cr100", "CR INR 100", "cr 0.00"):
|
||||
self.assertEqual(get_amount_cr_dr_marker(amount), "cr", amount)
|
||||
|
||||
for amount in ("Dr 100", "Dr100", "Dr. 1,234.50"):
|
||||
self.assertEqual(get_amount_cr_dr_marker(amount), "dr", amount)
|
||||
|
||||
for amount in ("100.00", "-2,000.00", "INR 25,236.00", "", None, 100.0):
|
||||
self.assertIsNone(get_amount_cr_dr_marker(amount), amount)
|
||||
|
||||
# Text that merely starts or ends with the letters must not be read as a marker, or
|
||||
# a description that bled into the amount column would reclassify the statement.
|
||||
for amount in (
|
||||
"CREDIT CARD PAYMENT 500",
|
||||
"DRAFT 100",
|
||||
"Dr Smith Clinic 500",
|
||||
"DR AMBEDKAR ROAD BRANCH 500",
|
||||
"500 CRC",
|
||||
"Cheque Dr",
|
||||
"Cr",
|
||||
):
|
||||
self.assertIsNone(get_amount_cr_dr_marker(amount), amount)
|
||||
|
||||
def test_sparsely_marked_cr_dr_amount_column(self):
|
||||
"""One marker is enough to prove a CR/DR amount column - it is not a majority vote.
|
||||
|
||||
A real HDFC credit-card page carries 18 rows and a single "50.90Cr": the unmarked
|
||||
rows are ordinary purchases, and only the exceptions are marked. A frequency vote
|
||||
therefore picked "positive/negative" 17-1 and imported that lone credit as a debit.
|
||||
"""
|
||||
doc = self._create_bank_statement_import_log(
|
||||
[
|
||||
["Date", "Transaction Description", "Amount (in Rs.)"],
|
||||
["21/07/2026", "ITC MAURYA NEW DELHI", "2,495.00"],
|
||||
["22/07/2026", "ZOMATO LIMITED Gurugram", "1,288.68"],
|
||||
["23/07/2026", "SWIGGY Bangalore", "532.00"],
|
||||
["26/07/2026", "SWIGGY Bangalore", "1,043.00"],
|
||||
["27/07/2026", "PETRO SURCHARGE WAIVER", "50.90Cr"],
|
||||
]
|
||||
)
|
||||
|
||||
self.assertEqual(doc.detected_amount_format, 'Amount column has "CR"/"DR" values')
|
||||
# Only "Cr" appears, so it is the marked exception and unmarked rows are debits.
|
||||
self.assertEqual(doc.total_credits, 50.90)
|
||||
self.assertEqual(doc.total_credit_transactions, 1)
|
||||
self.assertEqual(doc.total_debits, 5358.68)
|
||||
self.assertEqual(doc.total_debit_transactions, 4)
|
||||
|
||||
def test_dr_only_statement_treats_unmarked_rows_as_deposits(self):
|
||||
"""The mirror image of a Cr-only statement: only withdrawals are marked.
|
||||
|
||||
The unmarked default cannot be hardcoded to the debit, because which side gets
|
||||
marked varies by bank. It is derived from the markers the statement actually uses -
|
||||
here only "Dr" appears, so "Dr" is the exception and everything unmarked is a
|
||||
deposit.
|
||||
"""
|
||||
doc = self._create_bank_statement_import_log(
|
||||
[
|
||||
["Date", "Narration", "Amount"],
|
||||
["01/04/2026", "ATM WITHDRAWAL", "2,000.00Dr"],
|
||||
["03/04/2026", "SALARY", "20,000.00"],
|
||||
["05/04/2026", "INTEREST", "150.00"],
|
||||
]
|
||||
)
|
||||
|
||||
self.assertEqual(doc.detected_amount_format, 'Amount column has "CR"/"DR" values')
|
||||
self.assertEqual(doc.total_debits, 2000.0)
|
||||
self.assertEqual(doc.total_debit_transactions, 1)
|
||||
self.assertEqual(doc.total_credits, 20150.0)
|
||||
self.assertEqual(doc.total_credit_transactions, 2)
|
||||
|
||||
def test_leading_cr_dr_markers(self):
|
||||
"""Some banks print the marker in front of the amount."""
|
||||
doc = self._create_bank_statement_import_log(
|
||||
[
|
||||
["Date", "Narration", "Amount"],
|
||||
["01/04/2026", "ATM WITHDRAWAL", "Dr 2,000.00"],
|
||||
["03/04/2026", "SALARY", "Cr 20,000.00"],
|
||||
]
|
||||
)
|
||||
|
||||
self.assertEqual(doc.detected_amount_format, 'Amount column has "CR"/"DR" values')
|
||||
self.assertEqual(doc.total_debits, 2000.0)
|
||||
self.assertEqual(doc.total_credits, 20000.0)
|
||||
|
||||
def test_partially_marked_cr_dr_amount_column(self):
|
||||
"""A CR/DR amount column stays CR/DR even when some rows carry no marker.
|
||||
|
||||
Every unmarked row used to also vote for "positive/negative", so an ordinary
|
||||
statement with a few unmarked rows was detected as positive/negative and a
|
||||
"2000.00Dr" was then imported as a deposit.
|
||||
"""
|
||||
doc = self._create_bank_statement_import_log(
|
||||
[
|
||||
["Date", "Narration", "Amount", "Balance"],
|
||||
["01/04/2026", "OPENING FEE", "100.00", "9,900.00"],
|
||||
["03/04/2026", "SALARY", "20000.00Cr", "29,900.00"],
|
||||
["05/04/2026", "ATM WDL", "2000.00Dr", "27,900.00"],
|
||||
]
|
||||
)
|
||||
|
||||
self.assertEqual(doc.detected_amount_format, 'Amount column has "CR"/"DR" values')
|
||||
# Both markers appear, so an unmarked row is undetermined and stays a debit.
|
||||
self.assertEqual(doc.total_debits, 2100.0)
|
||||
self.assertEqual(doc.total_debit_transactions, 2)
|
||||
self.assertEqual(doc.total_credits, 20000.0)
|
||||
self.assertEqual(doc.total_credit_transactions, 1)
|
||||
|
||||
def test_deposit_withdrawal_type_column(self):
|
||||
"""The word Withdrawal contains "dr", so a loose CR/DR check claims this column first.
|
||||
|
||||
It then reads "Deposit" (which has no "cr" in it) as a withdrawal, flipping the
|
||||
direction of every credit in the statement.
|
||||
"""
|
||||
doc = self._create_bank_statement_import_log(
|
||||
[
|
||||
["Date", "Narration", "Transaction Type", "Amount"],
|
||||
["01/04/2026", "ATM WDL", "Withdrawal", "2,000.00"],
|
||||
["03/04/2026", "SALARY", "Deposit", "20,000.00"],
|
||||
["05/04/2026", "ATM WDL", "Withdrawal", "500.00"],
|
||||
]
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
doc.detected_amount_format, 'Transaction type column has "Deposit"/"Withdrawal" values'
|
||||
)
|
||||
self.assertEqual(doc.total_debits, 2500.0)
|
||||
self.assertEqual(doc.total_debit_transactions, 2)
|
||||
self.assertEqual(doc.total_credits, 20000.0)
|
||||
self.assertEqual(doc.total_credit_transactions, 1)
|
||||
|
||||
def test_unrecognised_type_column_falls_back_to_signed_amount(self):
|
||||
"""An unrecognised transaction type must not stop the amount being read.
|
||||
|
||||
No tally was incremented for these rows, so max() returned the first key -
|
||||
"Separate columns for withdrawal and deposit" - and, with no such columns in the
|
||||
file, every amount came through as None.
|
||||
"""
|
||||
doc = self._create_bank_statement_import_log(
|
||||
[
|
||||
["Date", "Narration", "Transaction Type", "Amount"],
|
||||
["01/04/2026", "ATM WDL", "NEFT", "-2,000.00"],
|
||||
["03/04/2026", "SALARY", "IMPS", "20,000.00"],
|
||||
]
|
||||
)
|
||||
|
||||
self.assertEqual(doc.detected_amount_format, "Amount column has positive/negative values")
|
||||
self.assertEqual(doc.total_debits, 2000.0)
|
||||
self.assertEqual(doc.total_credits, 20000.0)
|
||||
|
||||
def test_blank_transaction_type_cell(self):
|
||||
"""A blank type cell used to raise - `None.lower()` - instead of parsing the row."""
|
||||
doc = self._create_bank_statement_import_log(
|
||||
[
|
||||
["Date", "Narration", "Transaction Type", "Amount"],
|
||||
["01/04/2026", "ATM WDL", "Dr", "2,000.00"],
|
||||
["03/04/2026", "SALARY", "Cr", "20,000.00"],
|
||||
["05/04/2026", "UNKNOWN", None, "500.00"],
|
||||
]
|
||||
)
|
||||
|
||||
self.assertEqual(doc.detected_amount_format, 'Transaction type column has "CR"/"DR" values')
|
||||
# The unmarked row has no direction of its own, so it counts as a withdrawal.
|
||||
self.assertEqual(doc.total_debits, 2500.0)
|
||||
self.assertEqual(doc.total_credits, 20000.0)
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# PDF statement import
|
||||
# ------------------------------------------------------------------ #
|
||||
@@ -339,8 +159,7 @@ class TestBankStatementImportLog(ERPNextTestSuite, AccountsTestMixin):
|
||||
else:
|
||||
table["header_index"] = None
|
||||
table["column_mapping"] = guess_column_mapping_by_content(table["rows"])
|
||||
final_transactions, _df, _af = build_table_transactions(table)
|
||||
table["included"] = should_include_table(table, final_transactions)
|
||||
table["included"] = True
|
||||
return table
|
||||
|
||||
def test_pdf_multi_page_kept_separate_and_unioned(self):
|
||||
@@ -378,74 +197,6 @@ class TestBankStatementImportLog(ERPNextTestSuite, AccountsTestMixin):
|
||||
final, _df, _af = build_table_transactions(ad_table)
|
||||
self.assertEqual(final, [])
|
||||
|
||||
def test_pdf_summary_box_not_auto_included(self):
|
||||
"""A summary box that happens to parse as one transaction must not start included.
|
||||
|
||||
The "Payment Due Date / Total Dues / Minimum Amount Due" block on an HDFC
|
||||
credit-card statement has a date column and a figures column, so it yields a single
|
||||
transaction - the due date and the minimum amount - and used to import as a phantom
|
||||
row. What it does not have, and a real transaction table always does, is a narration.
|
||||
"""
|
||||
summary_box = {
|
||||
"header_index": 1,
|
||||
"rows": [
|
||||
["Statement Date:17/08/2025", "Card No: 4341 55XX XXXX 2754", ""],
|
||||
["Payment Due Date", "Total Dues", "Minimum Amount Due"],
|
||||
["06/09/2025", "73,200.00", "3,660.00"],
|
||||
["Credit Limit", "Available Credit Limit", "Available Cash Limit"],
|
||||
["", "32,800", ""],
|
||||
],
|
||||
"column_mapping": [
|
||||
{"index": 0, "header_text": "Payment Due Date", "variable": "a", "maps_to": "Date"},
|
||||
{"index": 1, "header_text": "Total Dues", "variable": "b", "maps_to": "Do not import"},
|
||||
{"index": 2, "header_text": "Minimum Amount Due", "variable": "c", "maps_to": "Amount"},
|
||||
],
|
||||
}
|
||||
|
||||
final, _df, _af = build_table_transactions(summary_box)
|
||||
# It really does parse as a transaction - that is why the previous check missed it.
|
||||
self.assertEqual(len(final), 1)
|
||||
self.assertFalse(should_include_table(summary_box, final))
|
||||
|
||||
# The transaction table beside it, which does carry a narration, still starts included.
|
||||
transactions = self._auto_map(
|
||||
{
|
||||
"rows": [
|
||||
["Date", "Transaction Description", "Amount (in Rs.)"],
|
||||
["21/07/2025", "ITC MAURYA NEW DELHI", "2,495.00"],
|
||||
["27/07/2025", "PETRO SURCHARGE WAIVER", "50.90Cr"],
|
||||
]
|
||||
}
|
||||
)
|
||||
self.assertTrue(transactions["included"])
|
||||
|
||||
def test_pdf_table_without_description_still_importable(self):
|
||||
"""No narration column means "starts unticked", NOT "cannot be imported".
|
||||
|
||||
`description` is not mandatory on Bank Transaction, so a bank that omits narration
|
||||
must still import once the user ticks the table.
|
||||
"""
|
||||
table = {
|
||||
"header_index": 0,
|
||||
"rows": [
|
||||
["Date", "Amount", "Balance"],
|
||||
["01/04/2025", "500.00", "9,500.00"],
|
||||
["03/04/2025", "20000.00", "29,500.00"],
|
||||
],
|
||||
"column_mapping": [
|
||||
{"index": 0, "header_text": "Date", "variable": "a", "maps_to": "Date"},
|
||||
{"index": 1, "header_text": "Amount", "variable": "b", "maps_to": "Amount"},
|
||||
{"index": 2, "header_text": "Balance", "variable": "c", "maps_to": "Balance"},
|
||||
],
|
||||
}
|
||||
|
||||
final, _df, _af = build_table_transactions(table)
|
||||
self.assertFalse(should_include_table(table, final))
|
||||
|
||||
# The transactions themselves are intact and importable.
|
||||
self.assertEqual(len(final), 2)
|
||||
self.assertEqual([t["date"] for t in final], ["2025-04-01", "2025-04-03"])
|
||||
|
||||
def test_headerless_content_mapping(self):
|
||||
"""Without a header row, columns are guessed from their contents."""
|
||||
rows = [
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user