The Italy regional setup created Custom Fields first_name/last_name on
Customer. Since #46281 added standard quick-entry fields with the same
names, every Italian site carries duplicate field definitions:
- the setup wizard creates the duplicates silently because it skips
validation, and any later Custom Field on Customer then raises
UniqueFieldnameError (#50915)
- without the duplicates, creating an Italian company aborts inside
install_country_fixtures; on MariaDB an interrupted fixture run
persists Custom Field documents whose columns were never added, after
which every Company insert fails with "Unknown column
'fiscal_regime'" (#57215)
Re-land the rename from #50921 (reverted in #53409): the fields become
italy_customer_first_name/italy_customer_last_name and the e-invoice
template reads the new names. The migration patch runs only on sites
with Italy fixtures, re-runs them, explicitly syncs the schema of every
affected doctype (create_custom_fields skips unchanged fields, so its
own schema sync cannot restore missing columns), copies the old column
values wherever the new field is empty (also on sites that removed the
duplicate fields with the documented manual workaround), and deletes
the duplicate Custom Fields last so an interrupted run stays resumable.
The old insert_after anchor "salutation" no longer exists on Customer;
the renamed fields anchor after customer_type.
create_purchase_invoice caught its own failure and then ran frappe.db.set_value + log_error in the SAME transaction. On Postgres a failed insert/save aborts the whole transaction, so the error-marking died with InFailedSqlTransaction and the failure cascaded through prepare_data_for_import's per-file loop, killing the entire import; MariaDB recovers per-statement and continues.
Let create_purchase_invoice raise, and wrap each call in prepare_data_for_import in frappe.db.savepoint + rollback(save_point=...). On failure the savepoint rollback un-poisons the transaction, the error is logged, and the per-file status is set to Error and committed (self.db_set(commit=True), matching the existing process_file_data status commit) so an interrupted import durably reflects Error instead of staying at the already-committed Processing File Data; the loop then continues to the next file. The savepoint is taken after create_supplier/create_address so those are preserved exactly as before.
Behaviour change (MariaDB): a failed invoice's partially-created draft Purchase Invoice is now rolled back on BOTH engines instead of being left as an orphan draft on MariaDB. Deliberate and more correct - a failed import should not leave a partial invoice; release-note worthy.
Conservative cleanup of frappe.throw/msgprint messages per the message style
guide; meaning, severity, and .format() arguments are unchanged:
- index bare {} placeholders as {0}/{1}/... so translators can reorder
- move f-strings / .format() / concatenation out of _() (they break gettext
extraction and never translate)
- wrap translatable dynamic values (DocType/Select labels) in _()
- fix grammar and colloquialisms
- drop no-op _() wrapping runtime-built strings
Part of #53976.
get_tax_accounts fetched fields=['Account'] but the UAE VAT Account fieldname is lowercase account. PostgreSQL treats the double-quoted identifier case-sensitively ('column "Account" does not exist'); MariaDB identifiers are case-insensitive so it worked there. Use the real fieldname account; output unchanged on MariaDB.
get_payer_address_html picks one company address with ORDER BY (Postal DESC, Billing DESC)
LIMIT 1 and no column tie-break. When a company has two addresses of the same address_type
the two CASE keys tie, so the LIMIT-1 row is implementation-defined and MariaDB and PostgreSQL
can return a different address.name — i.e. a different payer address on the rendered IRS-1099
form for identical data.
Add a final .orderby(address.name), mirroring the sibling get_street_address_html in the same
file (which already carries the "deterministic LIMIT-1 tie-break across engines" order). The
pick is now the lexicographically-smallest name on both engines.
frappe.local is request-scoped, not test-scoped — it survives
across unit-test methods. Two tests calling get_standard_rated_
expenses_total({"company": "_Test Company UAE VAT"}) hit the
same cache key, so the second test (foreign-currency PI, expected
917.5) was seeing 250 carried over from the first.
Short-circuit @_cached on frappe.flags.in_test so each test method
queries fresh. Production callers run one execute() per request and
have the cache cleared at the top of that call, so the optimisation
still applies there.
The JS button only renders the Generate/Retry action for Draft and
Error; the REST endpoint, however, still let an authenticated caller
silently overwrite the attached CSV on a Generated FAF. Tighten the
server-side guard to match the UI lifecycle so the destructive
action has to be explicit (delete and create a new doc to regenerate).
- Gate generate_faf() and mark_as_submitted() on write permission so
REST callers without write access can no longer trigger state
changes via the whitelisted endpoints.
- Drop test_generate_faf_excise_not_yet_implemented; the Excise file
type is no longer a valid Select option, so doc.insert() now fails
before generate_faf() is reached.
- Stream GL Entry rows in pages of GL_PAGE_SIZE to bound memory on
multi-year exports against large companies; running balance,
account-name cache, and totals carry across batches so output is
byte-identical to the single-fetch implementation.
- Move the VAT 201 helper cache from a module-level dict to
frappe.local so concurrent requests on threaded workers no longer
race or leak data across users.
ERPNextTestSuite already calls frappe.db.rollback() in its base
tearDown; overriding (even with the same call) trips the
semgrep "Dont-override-teardown" rule.
- Updated the UAE VAT 201 report HTML to improve layout and styling for better readability.
- Modified the JavaScript for the UAE VAT 201 report to include additional formatting for VAT legends.
- Enhanced the Python logic in the UAE VAT 201 report to include caching for performance improvements and added calculations for net VAT due.
- Introduced a new UAE VAT Register report with filters for company, date range, document type, and item-wise details.
- Implemented SQL queries in the UAE VAT Register to fetch sales and purchase invoice data based on selected filters.
- Added a new field for "Company Name in Arabic" in the Company doctype for compliance with local regulations.
Convert raw `frappe.db.sql` in the Setup, Utilities, Templates and Regional
areas to `frappe.qb` / the ORM so the same code runs on MariaDB and Postgres.
Behaviour is preserved on MariaDB; the conversions also make these paths valid
under Postgres' stricter SQL (GROUP BY, case-sensitivity, reserved words).
Conversions of note (behaviour kept identical to the MariaDB original):
- email_digest: ToDo ordering replicated with a CASE that mirrors MySQL
`field(priority,'High','Medium','Low')` (unknown/NULL -> 0, sorts first),
NULL-date-first and a `name` tie-break for a deterministic LIMIT.
- company.get_all_transactions_annual_history: the cross-DocType UNION + GROUP BY
is replaced by one grouped query per DocType merged with a Counter, so two
different DocTypes sharing a transaction_date still collapse into one bucket.
- templates/utils.send_message: contact lookup wraps both sides in LOWER() to
keep MariaDB's case-insensitive email match on case-sensitive Postgres.
- regional/irs_1099 & uae_vat_201: address ranking and emirate aggregation
rebuilt with CASE/aggregate selects that satisfy Postgres GROUP BY, with a
deterministic tie-break on the LIMIT-1 address lookups.
- utilities/product.get_item_codes_by_attributes: numeric attribute values are
cast with cstr() so Postgres doesn't reject `varchar = numeric`.
Tests (run on both MariaDB and Postgres, --lightmode):
- New: company merge test, authorization_rule duplicate-check, youtube report,
templates/utils, and utilities/templates page reports (partners, rfq,
material_request_info, product, utilities __init__).
- Existing suites kept green: company, email_digest, transaction_deletion_record,
irs_1099, uae_vat_201.
Deferred (tracked separately):
- setup/doctype/authorization_control.py still has raw `.format()` SELECTs;
left for its own PR.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace per-test company creation in setUp() with persistent master data
from BootStrapTestData. Add Test PCV Company to test_records.json so it
becomes a persistent fixture rather than a throwaway created per test run.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduce PaymentScheduleService and BillingValidationService classes so
call sites read PaymentScheduleService(doc).set_payment_schedule() instead
of the opaque self.set_payment_schedule() shim. Removes 15 shim methods
from AccountsController and updates all 11 call sites across the codebase.
