docs: add architecture, design spec, and security notes

Analyze the NS App and document what it does: NMI card payments on
Sales Invoices, AutoPay vaulting, multi-invoice settlement, guided
Customer Quick Entry, and branded print formats.

- docs/ARCHITECTURE.md  — functional overview, component map, payment
  flow, data model, dependencies, config, known tech debt
- docs/DESIGN_SPEC.md   — goals, requirements, API contracts, failure
  modes, acceptance criteria
- docs/SECURITY_NOTES.md — posture + hardening list
- docs/README.md        — docs index; link from root README

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
norman
2026-07-02 07:33:32 -04:00
parent 4e0acde164
commit 8bb8954ff6
5 changed files with 496 additions and 0 deletions

209
docs/ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,209 @@
# NS App — Architecture & Functional Overview
> Custom ERPNext / Frappe application by **NS Innovations** that extends the
> standard Sales workflow with an embedded card-payment experience, AutoPay
> vaulting, a streamlined customer onboarding dialog, and branded print
> formats.
---
## 1. What the app does
NS App layers four capabilities on top of a stock ERPNext v-current install:
| Capability | Where it lives | Summary |
|------------|----------------|---------|
| **In-form card payments** | `ns_app/public/js/sales_invoice.js` + `ns_app/api/payments.py` | Adds a **Run Payment** button to submitted, unpaid Sales Invoices. Card data is tokenized client-side by NMI Collect.js and charged server-side via the NMI gateway. A Payment Entry is created and submitted automatically on success. |
| **AutoPay (card vaulting)** | `ns_app/api/payments.py`, Customer custom fields | A card can be saved to the NMI Customer Vault. The returned vault ID is stored on the Customer, enabling one-click recurring charges and webhook-driven payments. |
| **Multi-invoice payment** | `sales_invoice.js`, `get_unpaid_invoices`, `run_token_payment` | A single card charge can settle several of a customer's outstanding invoices at once, producing one Payment Entry allocated across them. |
| **Customer Quick Entry** | `ns_app/public/js/customer_quick_entry.js`, `custom.js`, `ns_app/api/customer.py` | Replaces ERPNext's default "New Customer" quick-entry dialog with a guided form that creates Customer + Contact + Address atomically, with ZIP-based city/state autofill. |
| **Branded print formats** | `ns_app/print_formats/` (shipped as fixtures) | Double-window envelope layouts for Invoice, Sales Order, Quotation, and Dunning. |
---
## 2. Component map
```
ns_app/
├── hooks.py # App manifest: JS injection points + fixtures
├── api/
│ ├── payments.py # Payment gateway integration (NMI)
│ └── customer.py # Atomic customer creation endpoint
├── public/js/
│ ├── customer_quick_entry.js # Overrides CustomerQuickEntryForm (global)
│ ├── custom.js # Legacy/alt quick-entry enhancer
│ └── sales_invoice.js # Payment UI on the Sales Invoice form
└── print_formats/print_formats/ # HTML/Jinja print templates (fixtures)
```
### Injection points (`hooks.py`)
- `app_include_js``customer_quick_entry.js` loads on **every** desk page
(needed because Customer quick-entry can be triggered from many forms).
- `doctype_js["Sales Invoice"]``sales_invoice.js` loads only on the Sales
Invoice form.
- `fixtures` → three Print Formats (`NS Invoice`, `NS Sales Order`,
`NS Quotation`) are version-controlled and synced on migrate.
> Note: `custom.js` is **not** referenced in `hooks.py`. It is an earlier
> iteration of the quick-entry enhancement, superseded by
> `customer_quick_entry.js`. See [Known issues](#8-known-issues--tech-debt).
---
## 3. Payment flow (end-to-end)
```
┌─ Sales Invoice form (submitted, outstanding > 0) ─────────────────────────┐
│ "Run Payment" button → check_autopay(customer) │
└────────────┬──────────────────────────────────────────────────────────────┘
┌───────┴────────┐
│ AutoPay on? │
└───┬────────┬───┘
yes no
│ │
▼ ▼
run_autopay_ open_manual_payment_form() ── Collect.js renders NMI-hosted
payment() │ iframe fields (PCI out of scope)
│ │
│ user enters card → CollectJS.startPaymentRequest() → payment_token
│ │
│ ▼
│ run_token_payment(invoice, token, invoice_names[], …)
│ │
▼ ▼
call_payment POST https://secure.nmi.com/api/transact.php (type=sale)
_api() │
│ response=1 ? ── no ──► error surfaced to UI, no Payment Entry
│ │
│ yes
└───────┬───────┘
create_payment_entry(invoices[], transaction_id, mode_of_payment)
│ (dedup on reference_no == transaction_id)
Payment Entry inserted + submitted → invoice outstanding updates
▼ (only if "Save for Auto Pay" checked AND enable_autopay_signup=1)
customer_vault=add_customer → vault_id stored on Customer
```
### The three server entry points to a charge
1. **`run_token_payment`** — interactive, one-time or multi-invoice card
charge from the manual dialog. Optionally vaults the card.
2. **`run_autopay_payment`** → **`call_payment_api`** — charges a previously
vaulted card (`customer_vault_id`) with no card entry.
3. **`crystalclear_webhook`** (`allow_guest=True`) — gateway-initiated
confirmation that creates a Payment Entry for the referenced invoice.
All three converge on **`create_payment_entry`**, which is idempotent on the
transaction ID (`reference_no`).
---
## 4. Data model (custom fields)
The app relies on custom fields on **Customer** (created outside this repo —
via ERPNext Customize Form / Custom Field, not shipped as fixtures here):
| Field | Type | Purpose |
|-------|------|---------|
| `custom_auto_pay_status` | Check | Whether AutoPay is enabled |
| `custom_auto_pay_id` | Data | NMI Customer Vault ID |
| `custom_auto_pay_first_name` | Data | Cardholder first name (vault) |
| `custom_auto_pay_last_name` | Data | Cardholder last name (vault) |
| `custom_auto_pay_company` | Data | Company on the vault record |
| `custom_auto_pay_zip` | Data | Billing ZIP on the vault record |
| `custom_send_via` | Select | Preferred delivery method (mail/email/fax) |
> ⚠️ These fields are a **required dependency** that is not tracked in this
> repository. See [Known issues](#8-known-issues--tech-debt).
---
## 5. Customer Quick Entry
`customer_quick_entry.js` subclasses `frappe.ui.form.CustomerQuickEntryForm`
and overrides `render_dialog()` to present a custom dialog instead of
ERPNext's. Key design points:
- **Preserves `this.after_insert`** — the originating link-field callback — so
that after creation the new customer is written back into the field that
triggered quick entry (e.g. Customer on a Sales Order).
- **Polls** for `frappe.ui.form.make_quick_entry` to exist, then patches it to
re-assert the override on every `Customer` invocation (defends against bundle
load-order races).
- Submits to **`ns_app.api.customer.create_customer_full`**, which creates
**Customer + Contact + Address** inside one DB transaction
(`begin`/`commit`/`rollback`).
- **ZIP autofill** via the public `api.zippopotam.us` service populates
city/state/country.
---
## 6. External dependencies
| Dependency | Used for | Notes |
|------------|----------|-------|
| **NMI Gateway** (`secure.nmi.com/api/transact.php`) | Sale + vault transactions | Requires `nmi_security_key` in `site_config.json` |
| **NMI Collect.js** (`secure.nmi.com/token/Collect.js`) | Client-side card tokenization | Tokenization key is currently hard-coded in `sales_invoice.js` |
| **api.zippopotam.us** | ZIP → city/state/country autofill | Public, unauthenticated, US only |
---
## 7. Configuration
`site_config.json`:
```json
{
"nmi_security_key": "your_nmi_security_key",
"enable_autopay_signup": 0
}
```
- `nmi_security_key`**required** for all charge and vault calls.
- `enable_autopay_signup` — feature flag. When falsy, the "Save for Auto Pay"
checkbox is ignored server-side and no vault entry is created, even if the
user checks the box.
Hard-coded values worth noting:
- `paid_to` account for card/ACH payments: **`"ENB Bank Account - NIL"`**
(company-abbreviation specific — see `create_payment_entry`).
- Collect.js tokenization key in `sales_invoice.js`.
---
## 8. Known issues / tech debt
- **Undeclared custom-field dependency.** `custom_auto_pay_*` and
`custom_send_via` on Customer are required but not shipped as fixtures.
A fresh install will fail until they are created manually.
- **Duplicate quick-entry logic.** `custom.js` and `customer_quick_entry.js`
both override `make_quick_entry`; only the latter is wired in `hooks.py`.
`custom.js` appears to be dead code.
- **Hard-coded account & keys.** `"ENB Bank Account - NIL"` and the Collect.js
tokenization key are not configurable.
- **Version drift.** `setup.py` declares `0.0.1` while `__init__.py` declares
`0.1.0`.
- **Verbose payment logging.** `payments.py` writes request/response snippets
via `frappe.log_error` as a debug channel; ensure no PII/PAN leakage and
consider a proper logger + log level.
- **Webhook trust.** `crystalclear_webhook` is `allow_guest=True` and does not
verify a signature/shared secret before creating Payment Entries.
---
## 9. Security model
- Card numbers and CVV are entered into **NMI-hosted iframes** (Collect.js) and
never touch ERPNext's DOM or backend — only a single-use `payment_token`
does. This keeps PCI scope minimal.
- ERPNext stores only the **vault ID**, never card data.
- The `nmi_security_key` lives in `site_config.json` (server-side only).
- All gateway calls are HTTPS.
See [SECURITY_NOTES.md](./SECURITY_NOTES.md) for hardening recommendations.

217
docs/DESIGN_SPEC.md Normal file
View File

@@ -0,0 +1,217 @@
# NS App — Design Specification
**Status:** Living document · reverse-engineered from the current
implementation (branch `main`, commit `4e0acde`).
**Owner:** NS Innovations Engineering
**Applies to:** `ns_app` Frappe/ERPNext custom application.
This spec describes the intended behavior, contracts, and design constraints of
NS App so the system can be maintained, extended, and re-implemented
consistently. For a component tour see [ARCHITECTURE.md](./ARCHITECTURE.md).
---
## 1. Goals & non-goals
### Goals
- Let staff take a card payment **without leaving the Sales Invoice**, and have
the ledger (Payment Entry) update automatically and correctly.
- Support **saved cards (AutoPay)** for frictionless repeat/recurring billing.
- Allow one card charge to settle **multiple outstanding invoices**.
- Keep the app **PCI-light**: card data never transits ERPNext.
- Speed up **customer onboarding** with a single guided dialog that produces a
complete, linked Customer/Contact/Address.
### Non-goals
- The app is **not** a general payment-gateway abstraction — it targets NMI
specifically.
- It does **not** manage subscriptions/scheduling itself; "AutoPay" here means a
vaulted card that can be charged on demand or via webhook, not a scheduler.
- It does **not** own the custom-field schema on Customer (assumed present).
---
## 2. Personas & primary use cases
| Persona | Use case |
|---------|----------|
| **AR / billing clerk** | Opens an unpaid invoice, clicks *Run Payment*, keys the customer's card, optionally saves it for AutoPay. |
| **Clerk (repeat customer)** | Opens an unpaid invoice for a customer with AutoPay; confirms a one-click charge of the saved card. |
| **Clerk (bulk settle)** | Charges one card for several of a customer's open invoices at once. |
| **Sales user** | Creates a new customer from any link field via the guided Quick Entry dialog. |
| **Payment gateway (system)** | Posts an async confirmation to the webhook, which reconciles a Payment Entry. |
---
## 3. Functional requirements
### 3.1 Payment button visibility
- **Shown** only when: `docstatus == 1` (submitted) **and** a `customer` is set
**and** `outstanding_amount > 0`.
- When `outstanding_amount <= 0`: show a green **Paid** dashboard indicator, no
button.
- Otherwise: show a red **Unpaid** indicator plus **Run Payment** under
*Actions*.
### 3.2 AutoPay-vs-manual branching
- On *Run Payment*, call `check_autopay(customer)`.
- If `autopay_enabled` and `autopay_id` present → confirm dialog → charge the
vaulted card (`run_autopay_payment`).
- Else → open the manual card-entry dialog.
### 3.3 Manual payment dialog
- Collects: first name, last name, company (optional), billing ZIP.
- Renders **Collect.js inline fields** for card number / expiry / CVV.
- Prefills name/company/ZIP from the invoice's customer where possible.
- Optional **Save for Auto Pay** checkbox.
- Optional **Pay Additional Invoices** toggle → loads the customer's other open
invoices (`get_unpaid_invoices`) into a selectable table with a running
selected-total and a select-all control.
- **Pay** button label reflects the current selected total.
### 3.4 Charge semantics (server)
`run_token_payment` must:
1. Resolve `invoice_names` (JSON string → list; fall back to `[invoice]`).
2. Force `save_autopay = 0` when `enable_autopay_signup` is falsy.
3. Load every selected invoice; **reject** if any is not submitted or already
fully paid.
4. Sum `outstanding_amount` across selected invoices as the charge amount.
5. Generate a unique `orderid` (`<invoice-label>-<hash>`).
6. POST a `type=sale` transaction to NMI with the `payment_token`.
7. On `response == "1"`:
- **Dedup**: if a Payment Entry already exists with
`reference_no == transactionid`, return `duplicate: true` and do nothing.
- Else create **one** Payment Entry allocated across all selected invoices.
- If vaulting requested and a `vault_id` returned, persist AutoPay fields on
the Customer.
8. On failure: return `{success: False, error}` and **create no Payment Entry**.
### 3.5 AutoPay charge (server)
`run_autopay_payment``call_payment_api`:
- Requires `custom_auto_pay_status` and `custom_auto_pay_id`.
- POSTs `type=sale` with `customer_vault_id` (no token, no card entry).
- Derives `mode_of_payment` from the response `type` (`check` → ACH, else Credit
Card).
- Same dedup + `create_payment_entry` path.
### 3.6 Payment Entry creation (invariant)
`create_payment_entry(invoices[], transaction_id, mode_of_payment)`:
- Idempotent on `reference_no == transaction_id`.
- `paid_to` = `"ENB Bank Account - NIL"` for ACH/Credit Card, else the company's
default cash account; throw if none resolved.
- `payment_type = Receive`, party = customer of the first invoice.
- One `references` row per invoice, `allocated_amount = outstanding_amount`.
- Insert **and submit** with `ignore_permissions=True`.
### 3.7 Webhook
`crystalclear_webhook` (`allow_guest=True`):
- Ignore unless `response == "1"`.
- Map `orderid` → Sales Invoice, create a Payment Entry via the shared path.
- Always return a short string ack.
### 3.8 Customer Quick Entry
- Override ERPNext's Customer quick-entry dialog globally, preserving the
originating `after_insert` link-field callback.
- `create_customer_full` requires: `customer_name`, `customer_type`,
`customer_group`, `mobile_no`, `address_line1`, `pincode`, `country`.
- Enforce: if `custom_auto_pay_enabled` then `custom_auto_pay_id` required.
- Create Customer + Contact + Address in a single transaction; rollback on any
error and log.
- Return the new customer name; caller writes it back into the triggering field
via `after_insert({ name })`.
---
## 4. Interface contracts (server API)
All are `@frappe.whitelist()` unless noted. Return values are dicts consumed by
`frappe.call` on the client.
| Method | Args | Returns |
|--------|------|---------|
| `check_autopay` | `customer` | `{autopay_enabled: bool, autopay_id: str\|None}` |
| `get_unpaid_invoices` | `customer` | `[{name, posting_date, customer_name, outstanding_amount}]` |
| `run_token_payment` | `invoice, token, invoice_names?, first_name?, last_name?, company?, billing_zip?, save_autopay?` | `{success, transaction_id?, vault_id?, duplicate?, error?}` |
| `run_autopay_payment` | `invoice` | `{success, message, transaction_id}` or throws |
| `save_to_autopay` | `customer, token, first_name?, last_name?, company?, billing_zip?` | `{success, vault_id?}` / `{success:False, error}` |
| `crystalclear_webhook` | form dict (guest) | `"ok"` / `"ignored"` |
| `create_customer_full` | `**data` (see 3.8) | new customer `name` (str) or throws |
### Error conventions
- **Validation / preconditions** → `frappe.throw` (surfaces as a msgprint).
- **Gateway / recoverable** → `{success: False, error: <message>}`.
- **Post-charge Payment Entry failures** are caught and logged (the charge
already succeeded) — they must **not** raise to the client.
---
## 5. Design constraints & rationale
| Constraint | Rationale |
|------------|-----------|
| Card fields via Collect.js iframes only | Keep PAN/CVV out of ERPNext → minimal PCI scope. |
| Dedup on gateway `transactionid` | The charge is the source of truth; retries/webhook races must not double-post to the ledger. |
| Charge-then-record ordering, with PE errors logged not raised | Never lose money already captured at the gateway; reconcile a missing PE manually rather than re-charging. |
| `enable_autopay_signup` feature flag | Ship vaulting code dark; enable per-site only after testing. |
| Preserve `after_insert` in quick entry | Only correct way to resume the originating document flow without racing the create transaction. |
| Single DB transaction in `create_customer_full` | Never leave an orphan Customer without Contact/Address. |
---
## 6. Idempotency, concurrency & failure modes
- **Double-click / double-submit:** client guards with
`window.ns_payment_processing`; server guards with the `reference_no` dedup.
- **Charge succeeds, PE fails:** logged under
`PAYMENT ENTRY FAILURE AFTER SUCCESSFUL CHARGE`; invoice stays unpaid in
ERPNext until reconciled. **Recovery:** re-run `create_payment_entry` using
the logged `transaction_id` (idempotent).
- **Webhook after interactive PE:** dedup prevents a second PE.
- **Gateway unreachable:** returns a generic error; no PE created.
- **Multi-invoice partial validity:** if any selected invoice is unpaid-invalid
(not submitted / zero balance) the whole request is rejected **before**
charging.
---
## 7. Security requirements
- `nmi_security_key` server-side only (`site_config.json`); never sent to the
client.
- HTTPS for every outbound gateway call.
- Store only vault IDs, never card data.
- **Recommended hardening (not yet implemented):**
- Authenticate the webhook (shared secret / signature) before creating PEs.
- Move the Collect.js tokenization key and `paid_to` account into config.
- Scrub gateway response logging to guarantee no PAN/PII is written.
- Rate-limit / permission-check payment endpoints.
---
## 8. Open questions / future work
- Should AutoPay include a **scheduler** (true recurring billing) rather than
on-demand vault charges only?
- Ship the Customer `custom_*` fields as **fixtures** so installs are
self-contained.
- Consolidate `custom.js` and `customer_quick_entry.js`.
- Make `paid_to` company-aware instead of the hard-coded ENB account.
- Add automated tests around `create_payment_entry` idempotency and
multi-invoice allocation.
---
## 9. Acceptance criteria (smoke test)
1. Submitted unpaid invoice shows **Run Payment**; paid invoice shows **Paid**.
2. Manual charge with a test card creates exactly one submitted Payment Entry;
invoice outstanding goes to 0.
3. Repeating the same gateway transaction (webhook replay) creates **no** second
PE.
4. AutoPay customer: *Run Payment* → confirm → one-click charge succeeds.
5. Multi-invoice: selecting N invoices produces one PE with N allocations
summing to the charged amount.
6. Quick Entry creates a linked Customer/Contact/Address and populates the
originating link field.
7. With `enable_autopay_signup = 0`, checking *Save for Auto Pay* creates no
vault entry.

30
docs/README.md Normal file
View File

@@ -0,0 +1,30 @@
# NS App — Documentation
Reference documentation for the **NS App** ERPNext/Frappe custom application
(NS Innovations). This app extends the Sales workflow with embedded card
payments, AutoPay vaulting, multi-invoice settlement, guided customer
onboarding, and branded print formats.
## Contents
| Document | What's in it |
|----------|--------------|
| [ARCHITECTURE.md](./ARCHITECTURE.md) | What the app does, component map, payment flow diagram, data model, external dependencies, configuration, and known tech debt. |
| [DESIGN_SPEC.md](./DESIGN_SPEC.md) | Goals/non-goals, personas, functional requirements, server API contracts, design constraints, failure modes, and acceptance criteria. |
| [SECURITY_NOTES.md](./SECURITY_NOTES.md) | Current security posture and a prioritized hardening list. |
## Quick orientation
- **Backend:** `ns_app/api/payments.py` (NMI gateway), `ns_app/api/customer.py`
(atomic customer creation).
- **Frontend:** `ns_app/public/js/sales_invoice.js` (payment UI),
`ns_app/public/js/customer_quick_entry.js` (customer quick entry).
- **Wiring:** `ns_app/hooks.py`.
- **Print formats:** `ns_app/print_formats/` (shipped as fixtures).
## Related
- Root [README.md](../README.md) — install & configuration guide.
- `ns_app/api/payment_flow_documentation.md` — original narrative walkthrough of
the payment flow (kept for historical context; superseded by
[ARCHITECTURE.md](./ARCHITECTURE.md) §3).

33
docs/SECURITY_NOTES.md Normal file
View File

@@ -0,0 +1,33 @@
# NS App — Security Notes & Hardening
Companion to [ARCHITECTURE.md](./ARCHITECTURE.md) §9 and
[DESIGN_SPEC.md](./DESIGN_SPEC.md) §7.
## Current posture (as implemented)
- **PCI scope minimized.** Card number, expiry, and CVV are entered into
NMI-hosted Collect.js iframes. ERPNext receives only a single-use
`payment_token`. Raw card data never reaches the browser JS context or the
server.
- **No card storage.** Only the NMI Customer Vault ID is persisted on the
Customer (`custom_auto_pay_id`).
- **Secret handling.** `nmi_security_key` is read from `site_config.json`
(server-side) and sent only in server→NMI requests.
- **Transport.** All gateway calls use HTTPS to `secure.nmi.com`.
## Gaps & recommended hardening
| # | Issue | Recommendation |
|---|-------|----------------|
| 1 | **Unauthenticated webhook.** `crystalclear_webhook` is `allow_guest=True` and creates Payment Entries from any POST whose `orderid` matches an invoice. | Require a shared secret / HMAC signature; verify before writing. Optionally allow-list source IPs. |
| 2 | **Debug logging of gateway payloads.** `payments.py` writes request/response snippets via `frappe.log_error`. | Confirm no PAN/PII is ever logged; use a dedicated logger at an appropriate level; consider truncation/redaction and log retention limits. |
| 3 | **Hard-coded tokenization key** in `sales_invoice.js`. | Move to a server-provided value / site config; makes key rotation and per-environment keys possible. |
| 4 | **Hard-coded ledger account** `"ENB Bank Account - NIL"`. | Make company-aware via config or a Company-level custom field. |
| 5 | **Permissions.** Payment endpoints are whitelisted to any logged-in user. | Add role checks (as `create_customer_full` does with `frappe.only_for`) and/or rate limiting. |
| 6 | **`ignore_permissions=True`** on Payment Entry and Customer writes. | Acceptable for a system flow, but document the trust boundary and ensure the whitelisted entry points are themselves access-controlled. |
## Operational reminders
- Keep `enable_autopay_signup = 0` in production until vaulting is fully tested.
- Never commit `site_config.json` or the NMI security key to version control.
- Rotate the NMI security key and Collect.js key on any suspected exposure.