When Accounts Settings -> general_ledger_remarks_length is set, the GL report
adds `substr(remarks, 1, n) as 'remarks'` to its raw SQL. Postgres treats a
single-quoted column alias as a string literal and raises a syntax error, so
the General Ledger report is broken on Postgres whenever that setting is on.
Use a bare alias (`as remarks`). substr() itself is portable.
Adds a test that sets general_ledger_remarks_length and runs the report,
asserting it executes (and returns rows) on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_vendor_invoice_query filtered unclaimed invoices with
.having(unclaimed_amount > 0), but the query has no GROUP BY/aggregate and
unclaimed_amount is a SELECT alias. Postgres rejects HAVING on a SELECT alias
(and HAVING without GROUP BY on a non-aggregated column); MariaDB allowed it.
Move the threshold into WHERE on the underlying expression.
Behaviour is identical on MariaDB (same rows); fixes a hard error on Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
update_variant_attribute_values (propagates renamed Item Attribute Values to
variant items) used a qb UPDATE ... JOIN. Postgres has no UPDATE..JOIN syntax,
so renaming an Item Attribute Value errored on Postgres.
Rewrite as a correlated UPDATE that restricts to variant items via a subquery
on the parent (item_variant_table.parent.isin(variant Items)) instead of
joining the Item table. MariaDB behaviour is unchanged.
Covered by the existing test_item.test_rename_attribute_value_updates_variants
and test_swapped_attribute_value_renames_update_variants, which errored on
Postgres before and now pass on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_timeline_data grouped Timesheet Detail by Date(from_time) but selected
UnixTimestamp(from_time) (the full timestamp, ungrouped). MariaDB
arbitrary-picks a row's timestamp; Postgres rejects it ("must appear in the
GROUP BY clause"), so the Project timeline (calendar heatmap) is broken on PG.
Select UnixTimestamp(Date(from_time)) — the day's epoch — which is the
timeline key and matches the GROUP BY. CurDate() - Interval(years=1) is
portable and kept as-is.
Adds a test (no coverage existed) that records a timesheet against a project
and asserts get_timeline_data returns day-bucketed counts, on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_purchase_details grouped Purchase Order Item by (item_code, warehouse)
while selecting `qty` ungrouped/unaggregated. MariaDB arbitrary-picks one
row's qty; Postgres rejects the query ("must appear in the GROUP BY clause"),
so the report is broken on Postgres.
Sum the qty per item+warehouse ({"SUM": "qty"}). The column is the "Arrival
Qty" (quantity on order arriving) display figure; summing the open PO lines is
the meaningful planning number, and is deterministic vs MariaDB's arbitrary
single-line pick (which only differed when an item+warehouse had multiple open
PO lines).
Adds a test (no test file existed) that creates a Work Order plus two PO lines
for a BOM raw material and asserts the report runs and reports arrival_qty = 7
(3 + 4), on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_used_alternative_items built its WHERE with f-string interpolation of
subcontract_order / subcontract_order_field / work_order (a SQL-injection risk)
and used a raw implicit comma cross-join. Convert to frappe.qb with an
inner_join on sted.parent == ste.name and parameterised conditions. The raw
SELECT listed sted.conversion_factor twice; the qb version selects it once.
Engine-portable and MariaDB-identical.
Surgical re-apply: the rest of stock_entry.py (the services/ package layout and
other develop-only logic) is untouched.
Adds a test that substitutes an alternative item in a work order's transfer
entry and asserts get_used_alternative_items returns the mapping, on MariaDB
and Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert the raw frappe.db.sql statements in Item to frappe.qb / ORM:
validate_barcode duplicate check (-> frappe.get_all), stock_ledger_created
(-> frappe.db.exists), update_item_price + the BOM/BOM Item/BOM Explosion
description updates + check_stock_uom_with_bin's Bin UOM update (-> frappe.qb
.update), on_trash Bin/Item Price deletes (-> frappe.db.delete),
check_stock_uom_with_bin's bin lookup (-> frappe.get_all with or_filters), and
get_uom_conv_factor's self-join (-> frappe.qb).
The one genuine Postgres break is validate_duplicate_item_in_stock_reconciliation:
its raw query used `HAVING records > 1`, referencing the SELECT alias, which
Postgres rejects. The qb version uses `HAVING Count("*") > 1`.
Surgical re-apply (not a whole-file port): develop's opening-stock-reconciliation
flow (set_opening_stock / create_opening_stock_reconciliation /
make_opening_stock_entry) is preserved, and get_timeline_data keeps develop's
CurDate()-Interval form (valid on both engines), so the Interval/CurDate/
SerialBatchCreation imports are retained.
Verified: test_item 38/38 on MariaDB. Added a merge-rename test exercising the
HAVING query (validate_duplicate_item_in_stock_reconciliation) which passes on
MariaDB and Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert four raw frappe.db.sql statements to frappe.qb:
- set_as_cancel (UPDATE -> frappe.qb.update)
- the invalid-serial-no incoming_rate lookup
- get_valuation_rate's last-valuation lookup
- get_future_sle_with_negative_qty
The serial-no comparisons (invalid-serial lookup and the get_stock_ledger_entries
condition builder, which stays raw) are wrapped in lower()/Lower() so serial
matching is case-insensitive on Postgres too -- MariaDB's collation already is,
so this is a no-op there. Deterministic creation/name tiebreakers are added to
the "ORDER BY posting_date DESC LIMIT 1" lookups so Postgres picks the same row
MariaDB did.
Surgical re-apply (not a whole-file port): develop's reposting valuation-recalc
clause (`recalculate_valuation_rate`) in update_entries_after and the
already-shipped Min()-wrapped get_items_to_be_repost GROUP BY are preserved. The
dynamic-condition / row-locking raw queries (get_previous_sle,
get_stock_ledger_entries builder, get_future_sle_with_negative_batch_qty, the
qty_shift UPDATE) are intentionally left raw.
Verified: full test_stock_ledger_entry suite 22/22 on MariaDB; added focused
tests for set_as_cancel / get_valuation_rate / get_future_sle_with_negative_qty
that pass on MariaDB and Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
deduplicate_similar_repost used a raw UPDATE with the MySQL-only two-arg
TIMESTAMP(posting_date, posting_time) constructor, which is invalid on Postgres.
Convert the UPDATE to frappe.qb and replace TIMESTAMP() with CombineDatetime
on the column (portable, and preserves the original NULL semantics so rows with
a NULL posting_time stay excluded); the right-hand side is this document's own
always-set posting datetime, computed in Python via get_combine_datetime to
avoid wrapping literals in a SQL datetime function.
Surgical re-apply: develop's recalculate_valuation_rate field /
_recalculate_valuation_rate method / repost() branch are left intact.
The existing test_repost_item_valuation.test_deduplication directly exercises
this UPDATE; it errors on develop's Postgres and now passes on MariaDB and
Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The *_trends reports (Sales/Purchase Order/Invoice, Delivery Note, etc.) built
raw SQL that is invalid on Postgres:
- `SUM(IF(...))` -> `SUM(CASE WHEN ... ELSE NULL END)` (IF is MySQL-only).
- Loose GROUP BY: each based_on `group by` listed only the key column while the
SELECT also returned name/territory/group/currency columns. Widen the GROUP BY
to include every selected non-aggregated column so the query is valid on
Postgres.
- Add a based_on_key (the first group-by column) for the group-by detail
subqueries, which equate against a single column (a multi-column group_by
spliced into an equality produced malformed SQL on both engines).
Behaviour note: widening the GROUP BY can split one based-on group into multiple
report rows when the snapshot columns (territory, renamed customer/item) differ
across transactions, vs MariaDB's previous one-arbitrary-row-per-group. Grand
totals are unchanged (calculate_total_row); per-group subtotals become
deterministic partial sums. This is the accepted widen-vs-arbitrary-pick
tradeoff.
Adds a test (no test file existed) running Sales Order Trends with a group_by,
exercising the widened GROUP BY / based_on_key / SUM(CASE) on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
make_variant_item_code used a raw frappe.db.sql left join over Item Attribute /
Item Attribute Value. Convert to frappe.qb. The attribute_value comparison
casts the param with cstr() so Postgres does not error on `varchar = numeric`
for numeric attributes (where that side is irrelevant, since numeric_values == 1
already satisfies the OR). MariaDB-identical.
Surgical re-apply: develop's get_attribute_value_renames /
update_variant_attribute_values helpers and the Case import are preserved.
Covered by test_item_variant on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Asset Movement deletion: raw implicit-join select -> frappe.get_all on
Asset Movement Item (pluck="parent").
- validate_item_type: raw `name in (...)` select -> frappe.get_all with an
`in` filter (pluck="item_code").
Both are engine-portable, MariaDB-identical. Surgical re-apply: develop's
actual-tax distribution rewrite (distribute_actual_tax_amount / get_tax_details)
is preserved (the staging branch predated it).
validate_item_type runs on every Purchase Receipt validation (covered by
test_asset.test_purchase_asset on both engines); the Asset Movement deletion
is covered by the asset cancellation flow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert eight raw frappe.db.sql search handlers to frappe.qb / frappe.qb.get_query
(which applies user-permission match conditions): employee_query, lead_query,
tax_account_query, bom, warehouse_query, get_batch_numbers, get_purchase_receipts
and get_purchase_invoices. Removes the MySQL-only get_match_cond/get_filters_cond
string building and ifnull usage.
The genuine Postgres break is get_project_name: it used CustomFunction("IF")
which emits a literal IF() (invalid on Postgres). Switch it to Case().
Surgical re-apply (not a whole-file port): develop's case-insensitive
Lower() ordering in item_query and get_project_name is preserved (the staging
branch reverted it), item_query is otherwise left untouched, and the Lower
import is retained.
Existing test_queries tests cover the converted handlers and now pass on
Postgres (test_project_query errors on develop). Adds smoke tests for the
three previously-untested handlers (batch numbers / purchase receipts /
purchase invoices).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- make_gl_entries_on_cancel: raw GL Entry existence select -> frappe.db.exists.
- future_sle_exists: raw GROUP BY count -> frappe.qb Count with Criterion.any,
and get_conditions_to_validate_future_sle builds qb Criterion objects
(warehouse == x & item_code.isin(...)) instead of escaped SQL strings.
Parity-preserving and valid on Postgres. Surgical re-apply: develop's
check_item_quality_inspection fix (`return items if doctype == "Stock Entry"
else []`) is preserved (the staging branch predated and would have reverted
it).
Adds a test asserting future_sle_exists detects a later SLE for the same
item/warehouse on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
validate_returned_items used a raw frappe.db.sql with a string-built column
list (and a separate Packed Item select); get_already_returned_items used a
raw GROUP BY sum. Convert both to frappe.get_all / frappe.qb (Sum(Abs(...))
with an explicit groupby). The qb GROUP BY mirrors the original
`group by item_code, <field>`, so it is parity-preserving (not a behaviour
change) and valid on Postgres.
Surgical re-apply: develop's `is_debit_note = 0` credit-note fix in
make_return_doc is preserved (the staging branch predated and would have
reverted it).
Adds a test (Delivery Note -> sales return) exercising validate_returned_items
and get_already_returned_items on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
daily_reminder/email_sending used raw frappe.db.sql with two portability
and correctness problems:
- The update query selected `progress` and `progress_details` from
`tabProject Update`, but those columns do not exist on the Project
Update doctype, so the query raised on BOTH MariaDB and Postgres
(the function is whitelisted-only, so the bug was latent). Drop the
non-existent columns and the corresponding "Project Status"/"Notes"
cells from the summary table.
- `DATE_ADD(CURRENT_DATE, INTERVAL -1 DAY)` (MySQL-only) and a
`CURRENT_DATE` Holiday lookup are not valid on Postgres.
Convert to ORM: frappe.get_all for Project/Project Update/Project User,
frappe.db.count for drafts, frappe.db.exists for the holiday check, and
add_days(today(), -1) for the date filter. Also str() the frequency in the
message so a NULL/empty frequency (Postgres returns None) does not raise.
Adds a test (the file was an empty stub) that creates a project + an update
dated yesterday and asserts the reminder finds it and runs end to end on
both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The max-allowed-qty Case used `... | ValueWrapper(allow_delivery_of_overproduced_qty)`
where the flag is an int (0/1). Postgres rejects `OR <integer>` ("argument of
OR must be type boolean"). Wrap it in bool() so the literal renders as
true/false. MariaDB behaviour is unchanged.
Surgical: only the bool() wrap is applied; develop's weighted-average rate
logic and the internal/whitelisted status-helper split are left intact (the
staging branch predated both).
Covered by test_subcontracting_inward_order.test_over_production_delivery,
which now passes on Postgres and is unchanged on MariaDB.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Material Request requested-amount query selects
`Sum(stock_qty - ordered_qty) * mri.rate` -- an implicit aggregate with no
GROUP BY, where mri.rate is neither grouped nor aggregated. MariaDB
arbitrary-picks the rate; Postgres rejects it ("must appear in the GROUP BY
clause"). Wrap the rate in Max(mri.rate) so the SELECT is a pure aggregate.
Behaviour note: for matched MR items with differing rates, Max() picks the
highest (vs MariaDB's arbitrary single rate). The underlying Sum(qty) * rate
is a pre-existing single-rate aggregation; this preserves it under the
accepted arbitrary-pick convention.
Covered by erpnext.accounts.doctype.budget.test_budget
.test_monthly_budget_crossed_for_mr, which now passes on Postgres (it errors
on develop) and is unchanged on MariaDB.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_list_context built the enabled-currency symbol map with a raw
frappe.db.sql select. Convert to frappe.get_all (as_list). MariaDB-identical.
Adds a test asserting the currency-symbol map is built and contains a known
enabled currency, on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_customer_name's Postgres branch used `Substring(Customer.name, r"\d+$")`,
but pypika's Substring is a start/length function, not a regex extractor, so
it raised `TypeError: Substring.__init__() missing 1 required positional
argument: 'stop'` at query-build time. Creating a second Customer with an
existing name therefore failed outright on Postgres.
Extract the trailing digits with regexp_replace + NULLIF + CAST instead. A
non-numeric trailing token strips to an empty string, which NULLIF turns into
NULL so MAX() skips it and COALESCE floors to 0 -- matching MariaDB's
CAST(... AS UNSIGNED) -> 0. MariaDB behaviour is unchanged (its branch is
untouched). Drops the now-unused Substring import.
Adds a test that creates "<name>" and "<name> - 3" and asserts the next
de-duplicated name is "<name> - 4" on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Replace MySQL-only `ifnull(...)` with `coalesce(...)` in the two
source/second-source percentage subqueries that remain raw (they
interpolate dynamic table/field names).
- zero_amount_refdocs: raw `sql_list` → `frappe.get_all(pluck="name")`.
- update_billing_status: two raw `ifnull(sum(qty), 0)` selects → frappe.qb
`Sum`; an empty result yields None and flt(None) == 0, matching the old
ifnull behaviour.
Behaviour is unchanged on MariaDB. The percentage path (coalesce subquery)
is exercised by test_selling_controller's Sales Order -> Delivery Note
per_delivered test on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_already_delivered_qty used two raw frappe.db.sql sums (Delivery Note
Item, and Sales Invoice Item joined to Sales Invoice) and
get_so_qty_and_warehouse used a raw select. Convert to frappe.qb (Sum) and
frappe.db.get_value. Engine-portable and MariaDB-identical.
Adds a test (Sales Order -> partial Delivery Note) that asserts
per_delivered, exercising get_already_delivered_qty / get_so_qty_and_warehouse
(and the StatusUpdater percentage path) on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
boot_session used raw `frappe.db.sql`, including a MySQL-only
`ifnull(account_type, '')` over Party Type that is invalid on Postgres.
- customer_count: `SELECT count(*)` → `frappe.db.count`
- setup_complete: `SELECT name ... LIMIT 1` → `frappe.db.get_all(limit=1)`
- companies: raw select → `frappe.get_all`, preserving the `:Company`
virtual-doc marker
- party_account_types: `ifnull(account_type,'')` → `frappe.get_all` with a
Python `account_type or ""`, which collapses NULL→'' and ''→''
identically on both engines (handles Postgres storing '' as NULL)
Adds a test (no test file existed) that runs boot_session and asserts the
company list and party_account_types are populated, on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
validate_exising_items() used a raw frappe.db.sql implicit-join to find
variant items using the attribute. Convert it to a frappe.qb inner join
(engine-portable, MariaDB-identical) so it no longer relies on raw SQL.
Only this query is converted; develop's update_variant_attribute_values
on_update hook and its imports are left intact (the staging branch's
whole-file version predated and would have reverted them).
Adds a focused test that creates a variant and asserts validate_exising_items
finds it (the validation only raises if the converted query returned the
variant row). Passes on MariaDB and Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both get_batchwise_data_from_stock_ledger and
get_batchwise_data_from_serial_batch_bundle select Batch columns
(expiry_date, and item_name when show_item_name is set) while grouping
only by Stock Ledger Entry columns. MariaDB arbitrary-picks the Batch
columns; Postgres rejects the query with "column ... must appear in the
GROUP BY clause".
Add the Batch PK (batch.name) to both GROUP BYs. batch.name is 1:1 with
the grouped batch_no (the join condition), so groups are unchanged and the
result is identical on MariaDB.
The serial-batch-bundle query additionally grouped by ch_table.warehouse
while selecting table.warehouse; group by the selected (SLE) warehouse so
the grouped and selected columns match (also required by Postgres).
Adds a test (no test file existed) that receives batch stock and asserts
the report lists it with the correct balance, exercising the GROUP BY on
both engines (with show_item_name set to force the extra Batch column).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
authorization_control.py used MySQL-only `ifnull()` in its raw rule
lookups (invalid on Postgres) and several raw `frappe.db.sql` selects.
- Replace every `ifnull(...)` with the portable `coalesce(...)` in the
rule-lookup statements that remain raw (they interpolate dynamic
conditions and rely on Frappe's Postgres backtick translation).
- Convert the user/role based_on lookups in validate_approving_authority
and the four value-based lookups in get_value_based_rule to frappe.qb
(Coalesce, isin, and a fresh Employee-designation subquery per use).
Behaviour is unchanged on MariaDB; the queries now run on Postgres.
Adds a test (no test file existed): a not-authorized case that exercises
the based_on + coalesce rule lookups (run as a non-admin user, since
Administrator implicitly holds every role), and a get_value_based_rule
call that exercises all four query-builder lookups.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_item_list() computes build_qty as IfNull(bin.actual_qty * bom.quantity
/ bom_item.stock_qty, 0) while grouping only by bom_item.item_code. The
three operand columns are neither grouped nor aggregated, so MariaDB
arbitrary-picks them but Postgres rejects the query with "column ... must
appear in the GROUP BY clause".
Add bom.quantity, bom_item.stock_qty and bin.actual_qty to the GROUP BY.
The WHERE pins bom/item and the join pins warehouse to single rows (Bin is
unique per item+warehouse), so the result stays one row per item and
MariaDB behaviour is unchanged.
Adds a test (no test file existed) that runs the report against a Work
Order and asserts it is listed, exercising the query on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_data() grouped only by Purchase Order Item while selecting Purchase
Order parent columns. MariaDB allows this loose GROUP BY; Postgres rejects
it with "column ... must appear in the GROUP BY clause".
Add the Purchase Order PK (po.name) to the GROUP BY. po.name is 1:1 with
the already-grouped po_item.name, so groups are unchanged and the result
is identical on MariaDB.
Adds a test (no test file existed) that runs the report and asserts the PO
is listed, exercising the GROUP BY on both engines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
