set_expired_status passed filters= and fieldname= kwargs that
frappe.db.set_value does not accept, so the daily scheduled task threw
TypeError on every run and quotations were never marked Expired. Pass
the filter dict as the positional docname argument, and scope it to
submitted documents so draft quotations aren't wrongly expired (matching
the selling Quotation behaviour).
Adds coverage for valid-till validation, the expiry task, and the
RFQ quote-status round-trip on submit/cancel.
Address Greptile review:
- customContext.files scope was **/*.py only, so Query Report SQL in .js/.sql/report .json
files didn't get the guide attached as context (the global instructions still applied).
Widen to .py/.js/.sql/report **/*.json.
- The guide's HAVING-alias rule said "with no GROUP BY"; PostgreSQL rejects a SELECT-alias in
HAVING regardless of GROUP BY. Reworded to match (repeat the expression, or move a
non-aggregate predicate to WHERE).
The PostgreSQL server-test job is label-gated, so until it is required the Greptile
PR-review bot is the always-on guard against cross-engine breaks. Extend
.greptile/config.json with `instructions` (and a `customContext` reference to a new
guide) so every review flags new/changed queries that would error on PostgreSQL or
silently diverge from MariaDB, under the prime rule that MariaDB output must not change.
- .github/POSTGRES_COMPATIBILITY.md — the catalogue the bot (and contributors) follow:
hard breaks (loose GROUP BY, MySQL-only funcs, UPDATE..JOIN, HAVING-on-alias,
DISTINCT+ORDER BY, single-quoted alias, varchar bitwise OR, capital identifiers,
set_value(Check,bool)), silent divergences (text case-sensitivity, name-lookup case,
empty-string↔NULL, NULL ordering, ORDER BY..LIMIT 1 tiebreakers, integer division,
distinct-drops-ORDER-BY-on-PG + casefold sorting, function-rewrite parity, UnixTimestamp
TZ), the GROUP BY row-count trap (Max()-wrap vs add-to-GROUP-BY; FD-by-source-table),
the InFailedSqlTransaction/savepoint rule, and the false positives NOT to flag
(.like→ILIKE, ifnull/backtick/LOCATE/REGEXP auto-translation, MariaDB-changing tiebreakers).
- Existing disabledLabels and frappe/frappe context are preserved.
The `skip_transfer` and transfer branches of `validate_manufacture` ran the
same per-item validation loop — look the row up or throw "not a part of",
check overconsumption, guard against duplicates, record — differing only in
the data source (SCIO Received Item vs Work Order Item), the available-qty
basis, the source-warehouse check (skip_transfer only) and the message text.
Split each branch into a small method that builds a normalised
`{item_code: {consumed_qty, available_qty}}` lookup, and share the loop via
`_validate_customer_provided_consumption`. Branch-specific throw messages are
passed as callbacks so the user-facing strings (and their translations) are
unchanged, and the order in which checks fire is preserved. Also drops the
unused `name` column from the skip_transfer query.
Adds a test for the non-skip-transfer manufacture flow (Material Transfer for
Manufacture -> Manufacture), which exercises the Work Order branch that the
existing suite — all of whose manufacture tests set skip_transfer=1 — never
covered. Full subcontracting-inward suite passes on MariaDB and PostgreSQL.
`get_fg_reference_names` built its LIKE filter with old-style
`"%%%s%%" % txt`. Use an f-string (`f"%{txt}%"`) for readability; the value
is still passed as a parameterised filter, so behaviour is unchanged.
Each new child row was given `idx=frappe.db.count(...) + 1`, issuing a count
query per inserted row across three insert loops (received items on receipt,
self-procured RM on manufacture, secondary items on manufacture). Compute the
starting index once before each loop and increment a local counter, producing
the same idx sequence with a single count query.
`update_inward_order_received_items_for_manufacture` unpacks
`zip(*item_code_wh.keys(), strict=True)`. When the manufacture entry has no
raw-material rows (all rows are finished/secondary/scrap), `item_code_wh` is
empty and the unpack raises `ValueError: not enough values to unpack`.
Return early when there are no such rows, mirroring the `if secondary_items:`
guard already present in `update_inward_order_secondary_items`.
In `validate_manufacture`, `customer_warehouse` is read only inside the
`skip_transfer` branch but was fetched unconditionally, wasting a lookup on
the non-skip-transfer path. Move it inside the branch that uses it.
`validate_material_transfer` ran the `Work Order Item` query and rebuilt
`wo_item_dict` inside the per-item loop, even though both depend only on
`self.work_order`. For an entry with N customer-provided rows that meant N
identical queries. Build the lookup once before the loop.
`validate_manufacture` already builds the analogous dict once up front, so
this also aligns the two methods.
In `update_inward_order_item`, the walrus assignment `scio_item_name :=` is
already part of the truthy `if` condition, so the nested `if scio_item_name:`
is always true. Remove it and dedent the body.
`validate_delivery_on_save` imported `pypika.terms.ValueWrapper` inside its
per-item loop, re-running the import on every iteration. Move it to the
module-level imports.
The "exceeds quantity available" throw in `validate_manufacture` passes a
third positional arg (`item.transfer_qty`), but the message only has `{0}`
and `{1}` placeholders, so `str.format` silently discards it. Remove the
dead argument; no behaviour change.
`validate_manufacture` builds its "Target Warehouse for Finished Good must
be same as Finished Good Warehouse ..." message with placeholders `{1}` and
`{2}`, but only passes two positional args (indices 0 and 1). `str.format`
raises `IndexError: Replacement index 2 out of range` instead of rendering
the message, so a user who sets the wrong FG target warehouse gets an opaque
traceback rather than the intended validation error.
Renumber the placeholders to `{0}` and `{1}` to match the args.
get_account_columns fetched the dynamic expense / unrealized-P&L account lists with
frappe.get_all(distinct=True, order_by=...). frappe silently drops ORDER BY for
distinct queries on postgres (db_query), so the generated account columns came back
in arbitrary order on Postgres while MariaDB kept them ordered — a cross-engine
parity gap (the sibling Sales Register had already moved to a python sort).
Sort the lists in python with key=str.casefold (dropping the ignored order_by) so the
column order is deterministic, case-insensitive (matching MariaDB's collation), and
identical on both engines. Add a regression test with two case-colliding expense
account names asserting the casefold column order on both engines.
get_account_columns sorts the dynamic income / unrealized-P&L account columns with
python sorted() (the original raw SQL used ORDER BY, which frappe drops for distinct
queries on postgres). Plain sorted() is case-sensitive (ASCII), so it reordered the
columns versus the pre-effort MariaDB output, whose ORDER BY ran under the
case-insensitive utf8mb4 collation.
Sort with key=str.casefold so the column order matches MariaDB's collation and is
identical on MariaDB and Postgres. Add a regression test with two case-colliding
account names ("aaa ..." / "ZZZ ...") that fails on case-sensitive sort and passes
after, on both engines.
get_customer_name's Postgres branch extracted the PURE TRAILING digits of the
name (regexp '^.*?(\d*)$'), while the MariaDB branch uses
CAST(SUBSTRING_INDEX(name, ' ', -1) AS UNSIGNED) — the LEADING digits of the last
whitespace token. For a scanned name like "<base> - 3a" MariaDB yields 3 but
Postgres yielded NULL→0, so the next de-duplicated number (and thus the generated
Customer name) diverged between engines.
Make the Postgres branch take the last whitespace token then its leading digits,
mirroring MariaDB exactly ("X - 3a"->3, "X - 1.5"->1, "X - Foo"->0). Add a
regression test with a "<base> - 3a" name asserting the next name is "<base> - 4"
on both engines (it produced "<base> - 1" on the old Postgres regex).
The Postgres-portability change added the Purchase Order Item PK (child.name) to
get_po_entries' GROUP BY. material_request_item is blank for PO lines not sourced
from a Material Request, so a multi-line PO previously collapsed to ONE row per
(PO, blank) on MariaDB but now produced one row PER LINE — changing the MariaDB
row count (and the add_total_row totals).
Group only by (PO, material_request_item) — the pre-effort key — and Max()-
aggregate the other selected columns so the query stays valid on Postgres while
restoring the prior one-row-per-group MariaDB output (per-column arbitrary→
deterministic, row count preserved). Add a regression test with a two-line PO
that fails on the multi-column GROUP BY (2 rows) and passes after (1 row), on
both MariaDB and Postgres.
Add a regression test for the one-row-per-item invariant: a BOM that lists the
same raw item on two lines at different qty must still be counted once in the
report ("# Req'd Items" == 1). The test fails on the pre-fix multi-column GROUP
BY (which split the item into one row per distinct stock_qty -> 2) and passes
after the fix, on both MariaDB and Postgres.
The earlier parity fix aggregated the non-key descriptive columns for the Item
and Customer based-on paths but left Supplier grouping by all three selected
columns (supplier, supplier_name, supplier_group). supplier_name is a stored
per-transaction field, so historical purchase docs holding a divergent value for
the same supplier would split one supplier into multiple rows — diverging from
the original MariaDB output, which grouped by t1.supplier only.
Aggregate supplier_name with Max() and keep only supplier + the FD master column
supplier_group in GROUP BY, restoring one row per supplier on both engines.
Add regression tests for the Supplier (purchase) and Customer (sales) paths that
assert a single row per key even when stored descriptive fields diverge; both
fail on the pre-fix multi-column GROUP BY and pass after the fix, on MariaDB and
Postgres.
frappe.local is request-scoped, not test-scoped — it survives
across unit-test methods. Two tests calling get_standard_rated_
expenses_total({"company": "_Test Company UAE VAT"}) hit the
same cache key, so the second test (foreign-currency PI, expected
917.5) was seeing 250 carried over from the first.
Short-circuit @_cached on frappe.flags.in_test so each test method
queries fresh. Production callers run one execute() per request and
have the cache cleared at the top of that call, so the optimisation
still applies there.
#56192 made the trends queries Postgres-strict-GROUP-BY-valid by widening based_on_group_by
to include the selected descriptive columns. For Item it added t2.item_name, for Customer
t1.territory (and customer_name) — but item_name is an editable per-line field and territory an
editable per-document field, not functionally dependent on the item_code/customer key. On MariaDB
(ONLY_FULL_GROUP_BY off) this SPLITS the single row per key into one row per distinct
(key, item_name)/(key, territory), so a customer transacting across two territories (or an item
with an edited item_name) now shows duplicate rows with fractured per-period subtotals.
Group by the KEY only and aggregate the non-key descriptive columns with Max(): one row per
based-on key (identical to the pre-#56192 MariaDB output) and still Postgres-valid. Supplier
columns are master-joined / fetch-locked (functionally dependent) so they stay unchanged.
The JS button only renders the Generate/Retry action for Draft and
Error; the REST endpoint, however, still let an authenticated caller
silently overwrite the attached CSV on a Generated FAF. Tighten the
server-side guard to match the UI lifecycle so the destructive
action has to be explicit (delete and create a new doc to regenerate).
- Gate generate_faf() and mark_as_submitted() on write permission so
REST callers without write access can no longer trigger state
changes via the whitelisted endpoints.
- Drop test_generate_faf_excise_not_yet_implemented; the Excise file
type is no longer a valid Select option, so doc.insert() now fails
before generate_faf() is reached.
- Stream GL Entry rows in pages of GL_PAGE_SIZE to bound memory on
multi-year exports against large companies; running balance,
account-name cache, and totals carry across batches so output is
byte-identical to the single-fetch implementation.
- Move the VAT 201 helper cache from a module-level dict to
frappe.local so concurrent requests on threaded workers no longer
race or leak data across users.
ERPNextTestSuite already calls frappe.db.rollback() in its base
tearDown; overriding (even with the same call) trips the
semgrep "Dont-override-teardown" rule.
