get_opening_balance_for_inv_dimension selected item_code and warehouse alongside Sum() aggregates with no GROUP BY, which PostgreSQL rejects ('column ...item_code must appear in the GROUP BY clause'). Add GROUP BY item_code, warehouse. The query already returns early unless a single item and warehouse is selected, so this stays one row with identical values on MariaDB while becoming valid on Postgres.
The batch-number link picker (get_batch_no) had two Postgres-only defects in
both of its query builders (get_batches_from_stock_ledger_entries and
get_batches_from_serial_and_batch_bundle):
1. GROUP BY. They group by Stock Ledger Entry / Serial-and-Batch-Entry columns
while selecting un-aggregated Batch-master columns (manufacturing_date,
expiry_date, search fields). PostgreSQL only accepts that when the Batch
primary key is in the GROUP BY, so the picker raised GroupingError. Adding
batch_table.name (equal to the grouped batch_no via the join) keeps the
group count - and the MariaDB result - unchanged while making it valid.
2. CONCAT over nullable dates. "MFG-"/"EXP-" labels were built with
Concat("MFG-", manufacturing_date). When the date is NULL, MariaDB CONCAT
returns NULL but Postgres CONCAT drops the NULL and yields a bare "MFG-"/
"EXP-". Guard each with Case().when(date.isnotnull(), ...) so a missing date
is NULL on both engines (matching MariaDB, fixing Postgres).
Both leave MariaDB output unchanged. test_get_batch_no_search_returns_batches
exercises both builders directly and asserts no bare "MFG-"/"EXP-" leaks;
reverting either fix makes it fail on Postgres.
get_picked_batches summed Serial-and-Batch-Entry qty while selecting bare
batch_no and warehouse with no GROUP BY. PostgreSQL rejects this outright:
column "tabSerial and Batch Entry.batch_no" must appear in the GROUP BY clause
MariaDB does not error but collapses every picked row into a single result -
the grand-total qty pinned to one arbitrary batch - so the caller, which keys
the result by (batch_no, warehouse) to subtract already-picked stock, under-counts
whenever more than one batch is picked.
Add GROUP BY batch_no, warehouse so the query returns one correct row per batch
on both engines (this corrects the MariaDB result, not just Postgres validity).
get_material_requests_based_on_supplier deduplicated requests with
SELECT DISTINCT (name, transaction_date, company) while ordering by
mr_item.item_code, which is not in the select list. MariaDB allows this;
PostgreSQL rejects it:
psycopg2.errors.InvalidColumnReference: for SELECT DISTINCT,
ORDER BY expressions must appear in select list
so the picker errored out there.
Group by the three selected columns (equivalent to the DISTINCT, so the
same set of requests is returned) and order by Min(item_code). The order
key stays item_code but is now a well-defined aggregate, making the query
valid - and the ordering deterministic and identical - on both engines.
frappe's db_query SILENTLY drops ORDER BY for distinct queries on Postgres (the ORDER BY
column must appear in the SELECT-DISTINCT list), so `get_all/get_list(distinct=True,
order_by="<col>")` is a no-op there and the result comes back unordered — the root cause of
the Sales Register, Purchase Register and Sales Analytics ordering fixes. Add an AST rule to
.github/helper/postgres_compat.py that flags this (literal order_by only; an empty order_by=""
suppression and a dynamic/variable order_by are not flagged). `# pg-ok` escape hatch as usual.
Grandfather the three pre-existing low-impact sites the rule surfaces (paging/iteration order
only, not data): job_card operation autocomplete, inventory_dimension config list, and a
work_order test loop.
commission_rate is no_copy so it is not carried on Duplicate/amend, but the
mapper also skips no_copy fields, leaving the mapped Delivery Note / Sales
Invoice showing 0 commission until saved (it only re-fetched from the sales
partner on save). Map commission_rate explicitly in the SO->DN, SO->SI and
DN->SI mappers so it carries over immediately; Duplicate still does not copy
it.
Across Sales Order, Delivery Note, Sales Invoice and POS Invoice, the
Commission section's commission_rate, total_commission and
amount_eligible_for_commission are sales-partner commission fields:
- depends_on eval:doc.sales_partner so they only show when a Sales Partner
is set;
- no_copy so a duplicated/amended document does not carry a stale commission
rate or computed commission amount (the sales partner itself still copies).
depends_on is client-only, so the server-side commission calculation is
unchanged. Add a Sales Order test for the no_copy behaviour.
get_timeline_data uses UnixTimestamp(posting_date); on Postgres that is the date's
midnight epoch in the DB session timezone, which can sit up to a day ahead of the
Python time.time() instant when the app timezone is ahead of UTC. The strict
'<= now' upper bound is therefore flaky on Postgres. Allow a day of slack on the
upper bound; MariaDB's UNIX_TIMESTAMP stays <= now so its pass/fail is unchanged.
The deliberate UniqueValidationError when re-adding a barcode aborts the
transaction on Postgres, so the next frappe.get_doc() failed with
InFailedSqlTransaction. Wrap the expected-failure save in a savepoint and roll
back to it. No-op on MariaDB.
The deliberate UniqueValidationError from the second Bin insert aborts the
transaction on Postgres, so the following _create_bin() (which takes its own
savepoint) failed with InFailedSqlTransaction. Wrap the expected-failure insert in
a savepoint and roll back to it, mirroring _create_bin's 'preserve transaction in
postgres' pattern. No-op on MariaDB.
It never references `self`. The deterministic-serial-value test added in #56249
called it as `get_incoming_value_for_serial_nos(None, sle, serial_nos)` — passing
None for self, which is fragile: a future `self.*` access would fail with an opaque
AttributeError. Declaring it @staticmethod makes the call honest
(`get_incoming_value_for_serial_nos(sle, serial_nos)`) and is backward compatible —
the method has no in-repo callers besides that test, and any `self.`-style call still
binds correctly to a staticmethod.
Addresses Greptile review feedback on #56249.
The repo-wide query audit fixed runtime/source queries, but test files carry their
own raw SQL helpers that were never swept and only fail when the suite runs on
Postgres. Port the staging branch's already-green fixes for them:
- timestamp(posting_date, posting_time) (raw + qb Timestamp) -> posting_datetime /
CombineDatetime (test_stock_ledger_entry, test_stock_balance, test_utils)
- HAVING <select-alias> -> qb .having(<expr>) (test_asset_capitalization, test_purchase_order)
- capital-cased identifiers ("Status", "Name") -> lowercase (test_delivery_note,
test_purchase_order, test_employee)
- raw GL/SLE select helpers -> frappe.get_all / qb, with order-independent
comparisons where account ordering is collation-dependent across engines
(test_purchase_invoice, test_sales_invoice, test_payment_entry, test_asset,
test_purchase_receipt, test_payment_request, test_repost_accounting_ledger,
test_journal_entry)
All changes are test-only and behaviour-identical on MariaDB (lowercase column names
resolve the same; posting_datetime == timestamp(posting_date, posting_time); HAVING on
the expression is the same computation). Verified: the heavy modules pass on both
MariaDB and Postgres, and MariaDB output is unchanged.
The serial-no filter used serial_batch_entry.serial_no.isin(serial_nos), which is
case-sensitive on Postgres -- a differently-cased serial no missed Serial and Batch
Entry rows that MariaDB (case-insensitive collation) matches (the OR'd regexp branch
only covers the legacy Stock Ledger Entry.serial_no text, empty for bundle-tracked
serials). Lower() both sides: MariaDB result unchanged, Postgres now matches too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
test_stock_entry.get_sle ordered by `timestamp(posting_date, posting_time)`, a
MySQL-only two-arg function that errors on Postgres ("function timestamp(date, time)
does not exist"), so every test using get_sle (test_fifo, test_stock_entry_qty, ...)
failed to run on Postgres. Order by the precomputed `posting_datetime` column instead
(identical value on MariaDB, valid on both engines).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
batch.get_batches(item_code, warehouse, ...) was added by #55647 and has no callers
anywhere in erpnext, frappe, or payments (not whitelisted, not referenced from JS/hooks).
It is also obsolete: it joins Stock Ledger Entry on `batch_no`, which the Serial and
Batch Bundle system no longer populates, so it returns nothing even on MariaDB. Its
query was additionally Postgres-invalid (GROUP BY batch_id with ORDER BY expiry_date/
creation -> GroupingError, since batch_id is not the primary key).
Remove the dead function (and its now-unused CurDate/Sum import) rather than fix a query
that nothing can reach. Live batch-quantity lookups go through get_batch_qty() /
get_auto_batch_nos(), which use the bundle model.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
test_index_creation used `frappe.db.sql("show index from tabItem")` and the MySQL-only
result key "Column_name". "SHOW INDEX" errors on Postgres, so the test could not run
there. Use the db-agnostic frappe.db.get_column_index("tabItem", column) (checking both
unique and non-unique single-column indexes) instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
test_index_exists used `frappe.db.sql("show index from tabBin ...")`. "SHOW INDEX"
is MySQL-only syntax and errors on Postgres (syntax error at "from"), so the test
could not run there. Use the db-agnostic frappe.db.has_index("tabBin",
"unique_item_warehouse") instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_vendor_invoice_query filtered unclaimed invoices with
.having(unclaimed_amount > 0), but the query has no GROUP BY/aggregate and
unclaimed_amount is a SELECT alias. Postgres rejects HAVING on a SELECT alias
(and HAVING without GROUP BY on a non-aggregated column); MariaDB allowed it.
Move the threshold into WHERE on the underlying expression.
Behaviour is identical on MariaDB (same rows); fixes a hard error on Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_used_alternative_items built its WHERE with f-string interpolation of
subcontract_order / subcontract_order_field / work_order (a SQL-injection risk)
and used a raw implicit comma cross-join. Convert to frappe.qb with an
inner_join on sted.parent == ste.name and parameterised conditions. The raw
SELECT listed sted.conversion_factor twice; the qb version selects it once.
Engine-portable and MariaDB-identical.
Surgical re-apply: the rest of stock_entry.py (the services/ package layout and
other develop-only logic) is untouched.
Adds a test that substitutes an alternative item in a work order's transfer
entry and asserts get_used_alternative_items returns the mapping, on MariaDB
and Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert the raw frappe.db.sql statements in Item to frappe.qb / ORM:
validate_barcode duplicate check (-> frappe.get_all), stock_ledger_created
(-> frappe.db.exists), update_item_price + the BOM/BOM Item/BOM Explosion
description updates + check_stock_uom_with_bin's Bin UOM update (-> frappe.qb
.update), on_trash Bin/Item Price deletes (-> frappe.db.delete),
check_stock_uom_with_bin's bin lookup (-> frappe.get_all with or_filters), and
get_uom_conv_factor's self-join (-> frappe.qb).
The one genuine Postgres break is validate_duplicate_item_in_stock_reconciliation:
its raw query used `HAVING records > 1`, referencing the SELECT alias, which
Postgres rejects. The qb version uses `HAVING Count("*") > 1`.
Surgical re-apply (not a whole-file port): develop's opening-stock-reconciliation
flow (set_opening_stock / create_opening_stock_reconciliation /
make_opening_stock_entry) is preserved, and get_timeline_data keeps develop's
CurDate()-Interval form (valid on both engines), so the Interval/CurDate/
SerialBatchCreation imports are retained.
Verified: test_item 38/38 on MariaDB. Added a merge-rename test exercising the
HAVING query (validate_duplicate_item_in_stock_reconciliation) which passes on
MariaDB and Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert four raw frappe.db.sql statements to frappe.qb:
- set_as_cancel (UPDATE -> frappe.qb.update)
- the invalid-serial-no incoming_rate lookup
- get_valuation_rate's last-valuation lookup
- get_future_sle_with_negative_qty
The serial-no comparisons (invalid-serial lookup and the get_stock_ledger_entries
condition builder, which stays raw) are wrapped in lower()/Lower() so serial
matching is case-insensitive on Postgres too -- MariaDB's collation already is,
so this is a no-op there. Deterministic creation/name tiebreakers are added to
the "ORDER BY posting_date DESC LIMIT 1" lookups so Postgres picks the same row
MariaDB did.
Surgical re-apply (not a whole-file port): develop's reposting valuation-recalc
clause (`recalculate_valuation_rate`) in update_entries_after and the
already-shipped Min()-wrapped get_items_to_be_repost GROUP BY are preserved. The
dynamic-condition / row-locking raw queries (get_previous_sle,
get_stock_ledger_entries builder, get_future_sle_with_negative_batch_qty, the
qty_shift UPDATE) are intentionally left raw.
Verified: full test_stock_ledger_entry suite 22/22 on MariaDB; added focused
tests for set_as_cancel / get_valuation_rate / get_future_sle_with_negative_qty
that pass on MariaDB and Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
deduplicate_similar_repost used a raw UPDATE with the MySQL-only two-arg
TIMESTAMP(posting_date, posting_time) constructor, which is invalid on Postgres.
Convert the UPDATE to frappe.qb and replace TIMESTAMP() with CombineDatetime
on the column (portable, and preserves the original NULL semantics so rows with
a NULL posting_time stay excluded); the right-hand side is this document's own
always-set posting datetime, computed in Python via get_combine_datetime to
avoid wrapping literals in a SQL datetime function.
Surgical re-apply: develop's recalculate_valuation_rate field /
_recalculate_valuation_rate method / repost() branch are left intact.
The existing test_repost_item_valuation.test_deduplication directly exercises
this UPDATE; it errors on develop's Postgres and now passes on MariaDB and
Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
validate_exising_items() used a raw frappe.db.sql implicit-join to find
variant items using the attribute. Convert it to a frappe.qb inner join
(engine-portable, MariaDB-identical) so it no longer relies on raw SQL.
Only this query is converted; develop's update_variant_attribute_values
on_update hook and its imports are left intact (the staging branch's
whole-file version predated and would have reverted them).
Adds a focused test that creates a variant and asserts validate_exising_items
finds it (the validation only raises if the converted query returned the
variant row). Passes on MariaDB and Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both get_batchwise_data_from_stock_ledger and
get_batchwise_data_from_serial_batch_bundle select Batch columns
(expiry_date, and item_name when show_item_name is set) while grouping
only by Stock Ledger Entry columns. MariaDB arbitrary-picks the Batch
columns; Postgres rejects the query with "column ... must appear in the
GROUP BY clause".
Add the Batch PK (batch.name) to both GROUP BYs. batch.name is 1:1 with
the grouped batch_no (the join condition), so groups are unchanged and the
result is identical on MariaDB.
The serial-batch-bundle query additionally grouped by ch_table.warehouse
while selecting table.warehouse; group by the selected (SLE) warehouse so
the grouped and selected columns match (also required by Postgres).
Adds a test (no test file existed) that receives batch stock and asserts
the report lists it with the correct balance, exercising the GROUP BY on
both engines (with show_item_name set to force the extra Batch column).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_items_from_manufacture_stock_entry aggregated Stock Entry Detail rows by
item_code while selecting item_name/description/warehouses/etc. (and an orderby
on the non-grouped idx) -> arbitrary pick on MariaDB, GroupingError on Postgres.
Wrap the non-grouped columns in Max() (Min(idx) for the orderby), preserving the
one-row-per-item shape the disassembly expects; an item plays one role with one
uom/warehouse across the WO's manufacture entries, so MariaDB output is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_secondary_items_from_job_card selected item_name/description/stock_uom/
bom_secondary_item alongside `group by item_code, secondary_item_type` (and an
orderby on the non-grouped idx) -> arbitrary pick on MariaDB, GroupingError on
Postgres. Wrap the non-grouped columns in Max() (Min(idx) for the orderby);
they are item attributes / the secondary-item BOM link, constant per group, so
MariaDB output is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
validate_expense_account SLE existence check -> frappe.db.get_all(limit=1);
get_items_for_stock_reco's two comma-join SELECTs -> frappe.qb inner_joins, with
the correlated Warehouse-subtree EXISTS replaced by a precomputed
warehouses_in_tree subquery + isin and ifnull(disabled,0)=0 -> disabled==0|isnull.
The Item-Default query's `group by i.name` is dropped (sound: validate_item_defaults
enforces one Item Default per (item, company), so the company-filtered query already
returns one row per item; the downstream (item_code, warehouse) de-dup is unchanged).
Same result on MariaDB; valid under Postgres.
Tests: get_items_for_stock_reco Bin branch (stocked item) and Item-Default branch
(default_warehouse, no stock).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_items_to_be_repost selected posting_date/posting_time/creation/posting_datetime
alongside `group_by item_code, warehouse` with no aggregation -> arbitrary pick on
MariaDB, GroupingError on Postgres. Wrap the four columns in `Min()` (earliest row
per item+warehouse, the correct repost-start point; a single voucher's SLEs share
posting_date/time per group -> MariaDB-identical). This is reached by every stock
transaction submit/cancel via repost_future_sle_and_gle, so it unblocks the whole
transaction-heavy stock suite on Postgres (e.g. test_purchase_receipt 105/105).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert the `update tabSerial No set purchase_rate ... where name in (...)` to
frappe.qb.update(isin). Also fix the #39 Postgres bug in
set_landed_cost_voucher_amount: `.select(Sum(applicable_charges), cost_center)`
selected a non-grouped column with no GROUP BY (GroupingError on PG) -> wrap it
in `Max(cost_center)` (deterministic representative; per (receipt_document,
receipt_item) the matching LCV items share a cost_center -> MariaDB-identical).
Covered by the existing landed-cost tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the raw `select pr_detail, qty from Purchase Invoice Item` (summed in
Python) with a frappe.qb GROUP BY Sum(qty) per pr_detail, matching the sibling
get_returned_qty_map. Same result on MariaDB; valid under Postgres. Covered by
the existing make_purchase_invoice tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
