from_detailed_data inserts tax templates/accounts before update_regional_tax_settings in the same transaction; a full frappe.db.rollback() on regional-setup failure discarded those templates while the wizard continued. Take a savepoint before the regional call and roll back only to it.
Regional tax-template setup writes docs; on failure the except calls frappe.log_error with no rollback -> InFailedSqlTransaction on Postgres. Full rollback before log_error. No-op on MariaDB.
Regional fixture setup writes docs; on failure the except calls frappe.log_error before frappe.throw with no rollback -> InFailedSqlTransaction on Postgres. Full rollback before log_error. No-op on MariaDB.
get_exchange_rate orders Currency Exchange by 'date desc' LIMIT 1 with no unique tiebreaker. Currency Exchange autoname {date}-{from}-{to}-{purpose} allows multiple same-date rows (different purpose) for one currency pair; on the no-purpose-filter path all match, so MariaDB and Postgres can return a different exchange_rate for the same inputs. Add 'name desc' so both engines pick the same row. MariaDB row count unchanged.
The lead/address cleanup pre-escaped each address name before passing the
list into a query-builder .isin() filter, which escapes again. The
double-escaping produced `name IN ('''Addr''')`, matching nothing, so
Lead-linked Addresses were never deleted on either MariaDB or Postgres.
Pass the raw list straight into .isin() so the builder escapes once.
A deeper re-audit (with an adversarial skeptic) of the queries left raw
in the prior commit found more that have exact ORM equivalents:
- scalar SUM/MAX -> frappe.qb + Sum/Max .run()[0][0]
- SUM ... GROUP BY -> frappe.qb .groupby().select(Sum().as_()) run(as_dict)
- name IN (values) -> get_all(filters={'f': ['in', ...]})
- sql_list(select col) -> get_all(pluck='col')
- bulk UPDATE ... = NULL/value -> frappe.db.set_value(filters, field, val)
- positional as_list reads -> get_all(..., as_list=True) (+ sorted())
Note: get_value(dt, filters, 'sum(x)') and get_all(fields=['sum(x)'])
are rejected by frappe ('SQL functions are not allowed as strings'), so
aggregates go through frappe.qb. get_all(as_list=True) returns a tuple
(not a list), so consumers that mutate use sorted().
All affected test modules pass on MariaDB.
Conservative cleanup of frappe.throw/msgprint messages per the message style
guide; meaning, severity, and .format() arguments are unchanged:
- index bare {} placeholders as {0}/{1}/... so translators can reorder
- move f-strings / .format() / concatenation out of _() (they break gettext
extraction and never translate)
- wrap translatable dynamic values (DocType/Select labels) in _()
- fix grammar and colloquialisms
- drop no-op _() wrapping runtime-built strings
Part of #53976.
Conservative cleanup of frappe.throw/msgprint messages per the message style
guide; meaning, severity, and .format() arguments are unchanged:
- index bare {} placeholders as {0}/{1}/... so translators can reorder
- move f-strings / .format() / concatenation out of _() (they break gettext
extraction and never translate)
- wrap translatable dynamic values (DocType/Select labels) in _()
- fix grammar and colloquialisms
- drop no-op _() wrapping runtime-built strings
Part of #53976.
deactivate_sales_person looks up `frappe.db.get_value("Sales Person",
{"Employee": employee})`. The Sales Person field is `employee` (lower case);
the lookup runs with ignore_permissions, so the capital-cased key reaches the
query as the column `"Employee"`. PostgreSQL matches quoted identifiers
case-sensitively and errors:
column "Employee" does not exist
MariaDB resolves `Employee` to the `employee` column regardless of case, so
using the stored `{"employee": employee}` selects the same row on MariaDB and
is valid on PostgreSQL.
commission_rate was a free-text Data field on the Sales Person master and the
Sales Team child, storing percentages as strings. Convert both to Percent.
A pre_model_sync patch sanitizes the existing values first (empty / NULL /
non-numeric -> 0, others normalised via flt) so the Data -> Percent column
change casts cleanly under strict SQL mode, where Percent is a NOT NULL
decimal column. The patch is idempotent and avoids db-specific SQL so it works
on both MariaDB and Postgres.
create_bank_account() inserts a bank Account and swallows DuplicateEntryError
('bank account same as a CoA entry'). On Postgres the failed insert aborts the
transaction, so the rest of company setup ran against a poisoned transaction.
Take a savepoint and roll back to it in the handler. No-op on MariaDB.
The repo-wide query audit fixed runtime/source queries, but test files carry their
own raw SQL helpers that were never swept and only fail when the suite runs on
Postgres. Port the staging branch's already-green fixes for them:
- timestamp(posting_date, posting_time) (raw + qb Timestamp) -> posting_datetime /
CombineDatetime (test_stock_ledger_entry, test_stock_balance, test_utils)
- HAVING <select-alias> -> qb .having(<expr>) (test_asset_capitalization, test_purchase_order)
- capital-cased identifiers ("Status", "Name") -> lowercase (test_delivery_note,
test_purchase_order, test_employee)
- raw GL/SLE select helpers -> frappe.get_all / qb, with order-independent
comparisons where account ordering is collation-dependent across engines
(test_purchase_invoice, test_sales_invoice, test_payment_entry, test_asset,
test_purchase_receipt, test_payment_request, test_repost_accounting_ledger,
test_journal_entry)
All changes are test-only and behaviour-identical on MariaDB (lowercase column names
resolve the same; posting_datetime == timestamp(posting_date, posting_time); HAVING on
the expression is the same computation). Verified: the heavy modules pass on both
MariaDB and Postgres, and MariaDB output is unchanged.
authorization_control.py used MySQL-only `ifnull()` in its raw rule
lookups (invalid on Postgres) and several raw `frappe.db.sql` selects.
- Replace every `ifnull(...)` with the portable `coalesce(...)` in the
rule-lookup statements that remain raw (they interpolate dynamic
conditions and rely on Frappe's Postgres backtick translation).
- Convert the user/role based_on lookups in validate_approving_authority
and the four value-based lookups in get_value_based_rule to frappe.qb
(Coalesce, isin, and a fresh Employee-designation subquery per use).
Behaviour is unchanged on MariaDB; the queries now run on Postgres.
Adds a test (no test file existed): a not-authorized case that exercises
the based_on + coalesce rule lookups (run as a non-admin user, since
Administrator implicitly holds every role), and a get_value_based_rule
call that exercises all four query-builder lookups.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert raw `frappe.db.sql` in the Setup, Utilities, Templates and Regional
areas to `frappe.qb` / the ORM so the same code runs on MariaDB and Postgres.
Behaviour is preserved on MariaDB; the conversions also make these paths valid
under Postgres' stricter SQL (GROUP BY, case-sensitivity, reserved words).
Conversions of note (behaviour kept identical to the MariaDB original):
- email_digest: ToDo ordering replicated with a CASE that mirrors MySQL
`field(priority,'High','Medium','Low')` (unknown/NULL -> 0, sorts first),
NULL-date-first and a `name` tie-break for a deterministic LIMIT.
- company.get_all_transactions_annual_history: the cross-DocType UNION + GROUP BY
is replaced by one grouped query per DocType merged with a Counter, so two
different DocTypes sharing a transaction_date still collapse into one bucket.
- templates/utils.send_message: contact lookup wraps both sides in LOWER() to
keep MariaDB's case-insensitive email match on case-sensitive Postgres.
- regional/irs_1099 & uae_vat_201: address ranking and emirate aggregation
rebuilt with CASE/aggregate selects that satisfy Postgres GROUP BY, with a
deterministic tie-break on the LIMIT-1 address lookups.
- utilities/product.get_item_codes_by_attributes: numeric attribute values are
cast with cstr() so Postgres doesn't reject `varchar = numeric`.
Tests (run on both MariaDB and Postgres, --lightmode):
- New: company merge test, authorization_rule duplicate-check, youtube report,
templates/utils, and utilities/templates page reports (partners, rfq,
material_request_info, product, utilities __init__).
- Existing suites kept green: company, email_digest, transaction_deletion_record,
irs_1099, uae_vat_201.
Deferred (tracked separately):
- setup/doctype/authorization_control.py still has raw `.format()` SELECTs;
left for its own PR.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace per-test company creation in setUp() with persistent master data
from BootStrapTestData. Add Test PCV Company to test_records.json so it
becomes a persistent fixture rather than a throwaway created per test run.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Display the actual company name in bold within the confirmation dialog
label so users immediately know which company they must type to confirm,
reducing the risk of accidental data loss.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
