test_index_exists used `frappe.db.sql("show index from tabBin ...")`. "SHOW INDEX"
is MySQL-only syntax and errors on Postgres (syntax error at "from"), so the test
could not run there. Use the db-agnostic frappe.db.has_index("tabBin",
"unique_item_warehouse") instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_purchase_details grouped Purchase Order Item by (item_code, warehouse)
while selecting `qty` ungrouped/unaggregated. MariaDB arbitrary-picks one
row's qty; Postgres rejects the query ("must appear in the GROUP BY clause"),
so the report is broken on Postgres.
Sum the qty per item+warehouse ({"SUM": "qty"}). The column is the "Arrival
Qty" (quantity on order arriving) display figure; summing the open PO lines is
the meaningful planning number, and is deterministic vs MariaDB's arbitrary
single-line pick (which only differed when an item+warehouse had multiple open
PO lines).
Adds a test (no test file existed) that creates a Work Order plus two PO lines
for a BOM raw material and asserts the report runs and reports arrival_qty = 7
(3 + 4), on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_used_alternative_items built its WHERE with f-string interpolation of
subcontract_order / subcontract_order_field / work_order (a SQL-injection risk)
and used a raw implicit comma cross-join. Convert to frappe.qb with an
inner_join on sted.parent == ste.name and parameterised conditions. The raw
SELECT listed sted.conversion_factor twice; the qb version selects it once.
Engine-portable and MariaDB-identical.
Surgical re-apply: the rest of stock_entry.py (the services/ package layout and
other develop-only logic) is untouched.
Adds a test that substitutes an alternative item in a work order's transfer
entry and asserts get_used_alternative_items returns the mapping, on MariaDB
and Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert the raw frappe.db.sql statements in Item to frappe.qb / ORM:
validate_barcode duplicate check (-> frappe.get_all), stock_ledger_created
(-> frappe.db.exists), update_item_price + the BOM/BOM Item/BOM Explosion
description updates + check_stock_uom_with_bin's Bin UOM update (-> frappe.qb
.update), on_trash Bin/Item Price deletes (-> frappe.db.delete),
check_stock_uom_with_bin's bin lookup (-> frappe.get_all with or_filters), and
get_uom_conv_factor's self-join (-> frappe.qb).
The one genuine Postgres break is validate_duplicate_item_in_stock_reconciliation:
its raw query used `HAVING records > 1`, referencing the SELECT alias, which
Postgres rejects. The qb version uses `HAVING Count("*") > 1`.
Surgical re-apply (not a whole-file port): develop's opening-stock-reconciliation
flow (set_opening_stock / create_opening_stock_reconciliation /
make_opening_stock_entry) is preserved, and get_timeline_data keeps develop's
CurDate()-Interval form (valid on both engines), so the Interval/CurDate/
SerialBatchCreation imports are retained.
Verified: test_item 38/38 on MariaDB. Added a merge-rename test exercising the
HAVING query (validate_duplicate_item_in_stock_reconciliation) which passes on
MariaDB and Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert four raw frappe.db.sql statements to frappe.qb:
- set_as_cancel (UPDATE -> frappe.qb.update)
- the invalid-serial-no incoming_rate lookup
- get_valuation_rate's last-valuation lookup
- get_future_sle_with_negative_qty
The serial-no comparisons (invalid-serial lookup and the get_stock_ledger_entries
condition builder, which stays raw) are wrapped in lower()/Lower() so serial
matching is case-insensitive on Postgres too -- MariaDB's collation already is,
so this is a no-op there. Deterministic creation/name tiebreakers are added to
the "ORDER BY posting_date DESC LIMIT 1" lookups so Postgres picks the same row
MariaDB did.
Surgical re-apply (not a whole-file port): develop's reposting valuation-recalc
clause (`recalculate_valuation_rate`) in update_entries_after and the
already-shipped Min()-wrapped get_items_to_be_repost GROUP BY are preserved. The
dynamic-condition / row-locking raw queries (get_previous_sle,
get_stock_ledger_entries builder, get_future_sle_with_negative_batch_qty, the
qty_shift UPDATE) are intentionally left raw.
Verified: full test_stock_ledger_entry suite 22/22 on MariaDB; added focused
tests for set_as_cancel / get_valuation_rate / get_future_sle_with_negative_qty
that pass on MariaDB and Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
deduplicate_similar_repost used a raw UPDATE with the MySQL-only two-arg
TIMESTAMP(posting_date, posting_time) constructor, which is invalid on Postgres.
Convert the UPDATE to frappe.qb and replace TIMESTAMP() with CombineDatetime
on the column (portable, and preserves the original NULL semantics so rows with
a NULL posting_time stay excluded); the right-hand side is this document's own
always-set posting datetime, computed in Python via get_combine_datetime to
avoid wrapping literals in a SQL datetime function.
Surgical re-apply: develop's recalculate_valuation_rate field /
_recalculate_valuation_rate method / repost() branch are left intact.
The existing test_repost_item_valuation.test_deduplication directly exercises
this UPDATE; it errors on develop's Postgres and now passes on MariaDB and
Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The *_trends reports (Sales/Purchase Order/Invoice, Delivery Note, etc.) built
raw SQL that is invalid on Postgres:
- `SUM(IF(...))` -> `SUM(CASE WHEN ... ELSE NULL END)` (IF is MySQL-only).
- Loose GROUP BY: each based_on `group by` listed only the key column while the
SELECT also returned name/territory/group/currency columns. Widen the GROUP BY
to include every selected non-aggregated column so the query is valid on
Postgres.
- Add a based_on_key (the first group-by column) for the group-by detail
subqueries, which equate against a single column (a multi-column group_by
spliced into an equality produced malformed SQL on both engines).
Behaviour note: widening the GROUP BY can split one based-on group into multiple
report rows when the snapshot columns (territory, renamed customer/item) differ
across transactions, vs MariaDB's previous one-arbitrary-row-per-group. Grand
totals are unchanged (calculate_total_row); per-group subtotals become
deterministic partial sums. This is the accepted widen-vs-arbitrary-pick
tradeoff.
Adds a test (no test file existed) running Sales Order Trends with a group_by,
exercising the widened GROUP BY / based_on_key / SUM(CASE) on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
make_variant_item_code used a raw frappe.db.sql left join over Item Attribute /
Item Attribute Value. Convert to frappe.qb. The attribute_value comparison
casts the param with cstr() so Postgres does not error on `varchar = numeric`
for numeric attributes (where that side is irrelevant, since numeric_values == 1
already satisfies the OR). MariaDB-identical.
Surgical re-apply: develop's get_attribute_value_renames /
update_variant_attribute_values helpers and the Case import are preserved.
Covered by test_item_variant on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Asset Movement deletion: raw implicit-join select -> frappe.get_all on
Asset Movement Item (pluck="parent").
- validate_item_type: raw `name in (...)` select -> frappe.get_all with an
`in` filter (pluck="item_code").
Both are engine-portable, MariaDB-identical. Surgical re-apply: develop's
actual-tax distribution rewrite (distribute_actual_tax_amount / get_tax_details)
is preserved (the staging branch predated it).
validate_item_type runs on every Purchase Receipt validation (covered by
test_asset.test_purchase_asset on both engines); the Asset Movement deletion
is covered by the asset cancellation flow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert eight raw frappe.db.sql search handlers to frappe.qb / frappe.qb.get_query
(which applies user-permission match conditions): employee_query, lead_query,
tax_account_query, bom, warehouse_query, get_batch_numbers, get_purchase_receipts
and get_purchase_invoices. Removes the MySQL-only get_match_cond/get_filters_cond
string building and ifnull usage.
The genuine Postgres break is get_project_name: it used CustomFunction("IF")
which emits a literal IF() (invalid on Postgres). Switch it to Case().
Surgical re-apply (not a whole-file port): develop's case-insensitive
Lower() ordering in item_query and get_project_name is preserved (the staging
branch reverted it), item_query is otherwise left untouched, and the Lower
import is retained.
Existing test_queries tests cover the converted handlers and now pass on
Postgres (test_project_query errors on develop). Adds smoke tests for the
three previously-untested handlers (batch numbers / purchase receipts /
purchase invoices).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- make_gl_entries_on_cancel: raw GL Entry existence select -> frappe.db.exists.
- future_sle_exists: raw GROUP BY count -> frappe.qb Count with Criterion.any,
and get_conditions_to_validate_future_sle builds qb Criterion objects
(warehouse == x & item_code.isin(...)) instead of escaped SQL strings.
Parity-preserving and valid on Postgres. Surgical re-apply: develop's
check_item_quality_inspection fix (`return items if doctype == "Stock Entry"
else []`) is preserved (the staging branch predated and would have reverted
it).
Adds a test asserting future_sle_exists detects a later SLE for the same
item/warehouse on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
validate_returned_items used a raw frappe.db.sql with a string-built column
list (and a separate Packed Item select); get_already_returned_items used a
raw GROUP BY sum. Convert both to frappe.get_all / frappe.qb (Sum(Abs(...))
with an explicit groupby). The qb GROUP BY mirrors the original
`group by item_code, <field>`, so it is parity-preserving (not a behaviour
change) and valid on Postgres.
Surgical re-apply: develop's `is_debit_note = 0` credit-note fix in
make_return_doc is preserved (the staging branch predated and would have
reverted it).
Adds a test (Delivery Note -> sales return) exercising validate_returned_items
and get_already_returned_items on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
daily_reminder/email_sending used raw frappe.db.sql with two portability
and correctness problems:
- The update query selected `progress` and `progress_details` from
`tabProject Update`, but those columns do not exist on the Project
Update doctype, so the query raised on BOTH MariaDB and Postgres
(the function is whitelisted-only, so the bug was latent). Drop the
non-existent columns and the corresponding "Project Status"/"Notes"
cells from the summary table.
- `DATE_ADD(CURRENT_DATE, INTERVAL -1 DAY)` (MySQL-only) and a
`CURRENT_DATE` Holiday lookup are not valid on Postgres.
Convert to ORM: frappe.get_all for Project/Project Update/Project User,
frappe.db.count for drafts, frappe.db.exists for the holiday check, and
add_days(today(), -1) for the date filter. Also str() the frequency in the
message so a NULL/empty frequency (Postgres returns None) does not raise.
Adds a test (the file was an empty stub) that creates a project + an update
dated yesterday and asserts the reminder finds it and runs end to end on
both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The max-allowed-qty Case used `... | ValueWrapper(allow_delivery_of_overproduced_qty)`
where the flag is an int (0/1). Postgres rejects `OR <integer>` ("argument of
OR must be type boolean"). Wrap it in bool() so the literal renders as
true/false. MariaDB behaviour is unchanged.
Surgical: only the bool() wrap is applied; develop's weighted-average rate
logic and the internal/whitelisted status-helper split are left intact (the
staging branch predated both).
Covered by test_subcontracting_inward_order.test_over_production_delivery,
which now passes on Postgres and is unchanged on MariaDB.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Material Request requested-amount query selects
`Sum(stock_qty - ordered_qty) * mri.rate` -- an implicit aggregate with no
GROUP BY, where mri.rate is neither grouped nor aggregated. MariaDB
arbitrary-picks the rate; Postgres rejects it ("must appear in the GROUP BY
clause"). Wrap the rate in Max(mri.rate) so the SELECT is a pure aggregate.
Behaviour note: for matched MR items with differing rates, Max() picks the
highest (vs MariaDB's arbitrary single rate). The underlying Sum(qty) * rate
is a pre-existing single-rate aggregation; this preserves it under the
accepted arbitrary-pick convention.
Covered by erpnext.accounts.doctype.budget.test_budget
.test_monthly_budget_crossed_for_mr, which now passes on Postgres (it errors
on develop) and is unchanged on MariaDB.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_list_context built the enabled-currency symbol map with a raw
frappe.db.sql select. Convert to frappe.get_all (as_list). MariaDB-identical.
Adds a test asserting the currency-symbol map is built and contains a known
enabled currency, on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_customer_name's Postgres branch used `Substring(Customer.name, r"\d+$")`,
but pypika's Substring is a start/length function, not a regex extractor, so
it raised `TypeError: Substring.__init__() missing 1 required positional
argument: 'stop'` at query-build time. Creating a second Customer with an
existing name therefore failed outright on Postgres.
Extract the trailing digits with regexp_replace + NULLIF + CAST instead. A
non-numeric trailing token strips to an empty string, which NULLIF turns into
NULL so MAX() skips it and COALESCE floors to 0 -- matching MariaDB's
CAST(... AS UNSIGNED) -> 0. MariaDB behaviour is unchanged (its branch is
untouched). Drops the now-unused Substring import.
Adds a test that creates "<name>" and "<name> - 3" and asserts the next
de-duplicated name is "<name> - 4" on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Replace MySQL-only `ifnull(...)` with `coalesce(...)` in the two
source/second-source percentage subqueries that remain raw (they
interpolate dynamic table/field names).
- zero_amount_refdocs: raw `sql_list` → `frappe.get_all(pluck="name")`.
- update_billing_status: two raw `ifnull(sum(qty), 0)` selects → frappe.qb
`Sum`; an empty result yields None and flt(None) == 0, matching the old
ifnull behaviour.
Behaviour is unchanged on MariaDB. The percentage path (coalesce subquery)
is exercised by test_selling_controller's Sales Order -> Delivery Note
per_delivered test on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_already_delivered_qty used two raw frappe.db.sql sums (Delivery Note
Item, and Sales Invoice Item joined to Sales Invoice) and
get_so_qty_and_warehouse used a raw select. Convert to frappe.qb (Sum) and
frappe.db.get_value. Engine-portable and MariaDB-identical.
Adds a test (Sales Order -> partial Delivery Note) that asserts
per_delivered, exercising get_already_delivered_qty / get_so_qty_and_warehouse
(and the StatusUpdater percentage path) on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
boot_session used raw `frappe.db.sql`, including a MySQL-only
`ifnull(account_type, '')` over Party Type that is invalid on Postgres.
- customer_count: `SELECT count(*)` → `frappe.db.count`
- setup_complete: `SELECT name ... LIMIT 1` → `frappe.db.get_all(limit=1)`
- companies: raw select → `frappe.get_all`, preserving the `:Company`
virtual-doc marker
- party_account_types: `ifnull(account_type,'')` → `frappe.get_all` with a
Python `account_type or ""`, which collapses NULL→'' and ''→''
identically on both engines (handles Postgres storing '' as NULL)
Adds a test (no test file existed) that runs boot_session and asserts the
company list and party_account_types are populated, on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
validate_exising_items() used a raw frappe.db.sql implicit-join to find
variant items using the attribute. Convert it to a frappe.qb inner join
(engine-portable, MariaDB-identical) so it no longer relies on raw SQL.
Only this query is converted; develop's update_variant_attribute_values
on_update hook and its imports are left intact (the staging branch's
whole-file version predated and would have reverted them).
Adds a focused test that creates a variant and asserts validate_exising_items
finds it (the validation only raises if the converted query returned the
variant row). Passes on MariaDB and Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both get_batchwise_data_from_stock_ledger and
get_batchwise_data_from_serial_batch_bundle select Batch columns
(expiry_date, and item_name when show_item_name is set) while grouping
only by Stock Ledger Entry columns. MariaDB arbitrary-picks the Batch
columns; Postgres rejects the query with "column ... must appear in the
GROUP BY clause".
Add the Batch PK (batch.name) to both GROUP BYs. batch.name is 1:1 with
the grouped batch_no (the join condition), so groups are unchanged and the
result is identical on MariaDB.
The serial-batch-bundle query additionally grouped by ch_table.warehouse
while selecting table.warehouse; group by the selected (SLE) warehouse so
the grouped and selected columns match (also required by Postgres).
Adds a test (no test file existed) that receives batch stock and asserts
the report lists it with the correct balance, exercising the GROUP BY on
both engines (with show_item_name set to force the extra Batch column).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
authorization_control.py used MySQL-only `ifnull()` in its raw rule
lookups (invalid on Postgres) and several raw `frappe.db.sql` selects.
- Replace every `ifnull(...)` with the portable `coalesce(...)` in the
rule-lookup statements that remain raw (they interpolate dynamic
conditions and rely on Frappe's Postgres backtick translation).
- Convert the user/role based_on lookups in validate_approving_authority
and the four value-based lookups in get_value_based_rule to frappe.qb
(Coalesce, isin, and a fresh Employee-designation subquery per use).
Behaviour is unchanged on MariaDB; the queries now run on Postgres.
Adds a test (no test file existed): a not-authorized case that exercises
the based_on + coalesce rule lookups (run as a non-admin user, since
Administrator implicitly holds every role), and a get_value_based_rule
call that exercises all four query-builder lookups.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_item_list() computes build_qty as IfNull(bin.actual_qty * bom.quantity
/ bom_item.stock_qty, 0) while grouping only by bom_item.item_code. The
three operand columns are neither grouped nor aggregated, so MariaDB
arbitrary-picks them but Postgres rejects the query with "column ... must
appear in the GROUP BY clause".
Add bom.quantity, bom_item.stock_qty and bin.actual_qty to the GROUP BY.
The WHERE pins bom/item and the join pins warehouse to single rows (Bin is
unique per item+warehouse), so the result stays one row per item and
MariaDB behaviour is unchanged.
Adds a test (no test file existed) that runs the report against a Work
Order and asserts it is listed, exercising the query on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_data() grouped only by Purchase Order Item while selecting Purchase
Order parent columns. MariaDB allows this loose GROUP BY; Postgres rejects
it with "column ... must appear in the GROUP BY clause".
Add the Purchase Order PK (po.name) to the GROUP BY. po.name is 1:1 with
the already-grouped po_item.name, so groups are unchanged and the result
is identical on MariaDB.
Adds a test (no test file existed) that runs the report and asserts the PO
is listed, exercising the GROUP BY on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_po_entries() grouped only by (Purchase Order, material_request_item)
while selecting other Purchase Order Item columns. MariaDB allows this
loose GROUP BY (arbitrary-picking the extra columns); Postgres rejects it
with "column ... must appear in the GROUP BY clause".
Add the Purchase Order Item PK (child.name) to the GROUP BY so the
selected child columns are functionally determined by a grouped key.
Behaviour note: this is not a MariaDB no-op. When one PO has multiple
items sharing the same/blank material_request_item, MariaDB collapsed
them into one arbitrary row; now there is one row per PO line. The
downstream report already keys rows by purchase_order, so totals are
unaffected and the per-line breakdown is more correct.
Adds a test that runs the report and asserts the PO is listed, exercising
the GROUP BY on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_query_bom_items / _build_base_bom_items_query / _add_*_item_columns selected
non-grouped columns (idx, item_name, image, project, item-default fields, BOM
Item attributes) alongside `group by item_code` -> arbitrary pick on MariaDB,
GroupingError on Postgres. Wrap them in Max() (Min() for idx, preserving the
original ordering). Every wrapped column is functionally dependent on the
grouped item_code (item attributes / the single BOM's project / one Item
Default per item+company), so Max()/Min() returns exactly the value MySQL
picked arbitrarily -> MariaDB output unchanged.
This was previously shipped in #56008 and reverted with that batch; re-applied
in isolation here. Verified: test_work_order 85/85 on BOTH MariaDB (no change)
and Postgres (was 85/85 failing on this query).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_items_from_manufacture_stock_entry aggregated Stock Entry Detail rows by
item_code while selecting item_name/description/warehouses/etc. (and an orderby
on the non-grouped idx) -> arbitrary pick on MariaDB, GroupingError on Postgres.
Wrap the non-grouped columns in Max() (Min(idx) for the orderby), preserving the
one-row-per-item shape the disassembly expects; an item plays one role with one
uom/warehouse across the WO's manufacture entries, so MariaDB output is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_secondary_items_from_job_card selected item_name/description/stock_uom/
bom_secondary_item alongside `group by item_code, secondary_item_type` (and an
orderby on the non-grouped idx) -> arbitrary pick on MariaDB, GroupingError on
Postgres. Wrap the non-grouped columns in Max() (Min(idx) for the orderby);
they are item attributes / the secondary-item BOM link, constant per group, so
MariaDB output is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
