1 Commits

Author SHA1 Message Date
norman
8bb8954ff6 docs: add architecture, design spec, and security notes
Analyze the NS App and document what it does: NMI card payments on
Sales Invoices, AutoPay vaulting, multi-invoice settlement, guided
Customer Quick Entry, and branded print formats.

- docs/ARCHITECTURE.md  — functional overview, component map, payment
  flow, data model, dependencies, config, known tech debt
- docs/DESIGN_SPEC.md   — goals, requirements, API contracts, failure
  modes, acceptance criteria
- docs/SECURITY_NOTES.md — posture + hardening list
- docs/README.md        — docs index; link from root README

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 07:33:32 -04:00
5 changed files with 496 additions and 0 deletions

View File

@@ -11,6 +11,13 @@ Storing the returned vault ID on the ERPNext Customer for future autopay use
This part of the app is designed to integrate directly into the Sales Invoice workflow.
Documentation
Full reference docs live in the docs/ folder:
- docs/ARCHITECTURE.md what the app does, components, payment flow, data model
- docs/DESIGN_SPEC.md requirements, API contracts, failure modes, acceptance criteria
- docs/SECURITY_NOTES.md security posture and hardening list
Features
Secure card entry using NMI Collect.js (tokenization)
Manual payment dialog inside ERPNext

209
docs/ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,209 @@
# NS App — Architecture & Functional Overview
> Custom ERPNext / Frappe application by **NS Innovations** that extends the
> standard Sales workflow with an embedded card-payment experience, AutoPay
> vaulting, a streamlined customer onboarding dialog, and branded print
> formats.
---
## 1. What the app does
NS App layers four capabilities on top of a stock ERPNext v-current install:
| Capability | Where it lives | Summary |
|------------|----------------|---------|
| **In-form card payments** | `ns_app/public/js/sales_invoice.js` + `ns_app/api/payments.py` | Adds a **Run Payment** button to submitted, unpaid Sales Invoices. Card data is tokenized client-side by NMI Collect.js and charged server-side via the NMI gateway. A Payment Entry is created and submitted automatically on success. |
| **AutoPay (card vaulting)** | `ns_app/api/payments.py`, Customer custom fields | A card can be saved to the NMI Customer Vault. The returned vault ID is stored on the Customer, enabling one-click recurring charges and webhook-driven payments. |
| **Multi-invoice payment** | `sales_invoice.js`, `get_unpaid_invoices`, `run_token_payment` | A single card charge can settle several of a customer's outstanding invoices at once, producing one Payment Entry allocated across them. |
| **Customer Quick Entry** | `ns_app/public/js/customer_quick_entry.js`, `custom.js`, `ns_app/api/customer.py` | Replaces ERPNext's default "New Customer" quick-entry dialog with a guided form that creates Customer + Contact + Address atomically, with ZIP-based city/state autofill. |
| **Branded print formats** | `ns_app/print_formats/` (shipped as fixtures) | Double-window envelope layouts for Invoice, Sales Order, Quotation, and Dunning. |
---
## 2. Component map
```
ns_app/
├── hooks.py # App manifest: JS injection points + fixtures
├── api/
│ ├── payments.py # Payment gateway integration (NMI)
│ └── customer.py # Atomic customer creation endpoint
├── public/js/
│ ├── customer_quick_entry.js # Overrides CustomerQuickEntryForm (global)
│ ├── custom.js # Legacy/alt quick-entry enhancer
│ └── sales_invoice.js # Payment UI on the Sales Invoice form
└── print_formats/print_formats/ # HTML/Jinja print templates (fixtures)
```
### Injection points (`hooks.py`)
- `app_include_js``customer_quick_entry.js` loads on **every** desk page
(needed because Customer quick-entry can be triggered from many forms).
- `doctype_js["Sales Invoice"]``sales_invoice.js` loads only on the Sales
Invoice form.
- `fixtures` → three Print Formats (`NS Invoice`, `NS Sales Order`,
`NS Quotation`) are version-controlled and synced on migrate.
> Note: `custom.js` is **not** referenced in `hooks.py`. It is an earlier
> iteration of the quick-entry enhancement, superseded by
> `customer_quick_entry.js`. See [Known issues](#8-known-issues--tech-debt).
---
## 3. Payment flow (end-to-end)
```
┌─ Sales Invoice form (submitted, outstanding > 0) ─────────────────────────┐
│ "Run Payment" button → check_autopay(customer) │
└────────────┬──────────────────────────────────────────────────────────────┘
┌───────┴────────┐
│ AutoPay on? │
└───┬────────┬───┘
yes no
│ │
▼ ▼
run_autopay_ open_manual_payment_form() ── Collect.js renders NMI-hosted
payment() │ iframe fields (PCI out of scope)
│ │
│ user enters card → CollectJS.startPaymentRequest() → payment_token
│ │
│ ▼
│ run_token_payment(invoice, token, invoice_names[], …)
│ │
▼ ▼
call_payment POST https://secure.nmi.com/api/transact.php (type=sale)
_api() │
│ response=1 ? ── no ──► error surfaced to UI, no Payment Entry
│ │
│ yes
└───────┬───────┘
create_payment_entry(invoices[], transaction_id, mode_of_payment)
│ (dedup on reference_no == transaction_id)
Payment Entry inserted + submitted → invoice outstanding updates
▼ (only if "Save for Auto Pay" checked AND enable_autopay_signup=1)
customer_vault=add_customer → vault_id stored on Customer
```
### The three server entry points to a charge
1. **`run_token_payment`** — interactive, one-time or multi-invoice card
charge from the manual dialog. Optionally vaults the card.
2. **`run_autopay_payment`** → **`call_payment_api`** — charges a previously
vaulted card (`customer_vault_id`) with no card entry.
3. **`crystalclear_webhook`** (`allow_guest=True`) — gateway-initiated
confirmation that creates a Payment Entry for the referenced invoice.
All three converge on **`create_payment_entry`**, which is idempotent on the
transaction ID (`reference_no`).
---
## 4. Data model (custom fields)
The app relies on custom fields on **Customer** (created outside this repo —
via ERPNext Customize Form / Custom Field, not shipped as fixtures here):
| Field | Type | Purpose |
|-------|------|---------|
| `custom_auto_pay_status` | Check | Whether AutoPay is enabled |
| `custom_auto_pay_id` | Data | NMI Customer Vault ID |
| `custom_auto_pay_first_name` | Data | Cardholder first name (vault) |
| `custom_auto_pay_last_name` | Data | Cardholder last name (vault) |
| `custom_auto_pay_company` | Data | Company on the vault record |
| `custom_auto_pay_zip` | Data | Billing ZIP on the vault record |
| `custom_send_via` | Select | Preferred delivery method (mail/email/fax) |
> ⚠️ These fields are a **required dependency** that is not tracked in this
> repository. See [Known issues](#8-known-issues--tech-debt).
---
## 5. Customer Quick Entry
`customer_quick_entry.js` subclasses `frappe.ui.form.CustomerQuickEntryForm`
and overrides `render_dialog()` to present a custom dialog instead of
ERPNext's. Key design points:
- **Preserves `this.after_insert`** — the originating link-field callback — so
that after creation the new customer is written back into the field that
triggered quick entry (e.g. Customer on a Sales Order).
- **Polls** for `frappe.ui.form.make_quick_entry` to exist, then patches it to
re-assert the override on every `Customer` invocation (defends against bundle
load-order races).
- Submits to **`ns_app.api.customer.create_customer_full`**, which creates
**Customer + Contact + Address** inside one DB transaction
(`begin`/`commit`/`rollback`).
- **ZIP autofill** via the public `api.zippopotam.us` service populates
city/state/country.
---
## 6. External dependencies
| Dependency | Used for | Notes |
|------------|----------|-------|
| **NMI Gateway** (`secure.nmi.com/api/transact.php`) | Sale + vault transactions | Requires `nmi_security_key` in `site_config.json` |
| **NMI Collect.js** (`secure.nmi.com/token/Collect.js`) | Client-side card tokenization | Tokenization key is currently hard-coded in `sales_invoice.js` |
| **api.zippopotam.us** | ZIP → city/state/country autofill | Public, unauthenticated, US only |
---
## 7. Configuration
`site_config.json`:
```json
{
"nmi_security_key": "your_nmi_security_key",
"enable_autopay_signup": 0
}
```
- `nmi_security_key`**required** for all charge and vault calls.
- `enable_autopay_signup` — feature flag. When falsy, the "Save for Auto Pay"
checkbox is ignored server-side and no vault entry is created, even if the
user checks the box.
Hard-coded values worth noting:
- `paid_to` account for card/ACH payments: **`"ENB Bank Account - NIL"`**
(company-abbreviation specific — see `create_payment_entry`).
- Collect.js tokenization key in `sales_invoice.js`.
---
## 8. Known issues / tech debt
- **Undeclared custom-field dependency.** `custom_auto_pay_*` and
`custom_send_via` on Customer are required but not shipped as fixtures.
A fresh install will fail until they are created manually.
- **Duplicate quick-entry logic.** `custom.js` and `customer_quick_entry.js`
both override `make_quick_entry`; only the latter is wired in `hooks.py`.
`custom.js` appears to be dead code.
- **Hard-coded account & keys.** `"ENB Bank Account - NIL"` and the Collect.js
tokenization key are not configurable.
- **Version drift.** `setup.py` declares `0.0.1` while `__init__.py` declares
`0.1.0`.
- **Verbose payment logging.** `payments.py` writes request/response snippets
via `frappe.log_error` as a debug channel; ensure no PII/PAN leakage and
consider a proper logger + log level.
- **Webhook trust.** `crystalclear_webhook` is `allow_guest=True` and does not
verify a signature/shared secret before creating Payment Entries.
---
## 9. Security model
- Card numbers and CVV are entered into **NMI-hosted iframes** (Collect.js) and
never touch ERPNext's DOM or backend — only a single-use `payment_token`
does. This keeps PCI scope minimal.
- ERPNext stores only the **vault ID**, never card data.
- The `nmi_security_key` lives in `site_config.json` (server-side only).
- All gateway calls are HTTPS.
See [SECURITY_NOTES.md](./SECURITY_NOTES.md) for hardening recommendations.

217
docs/DESIGN_SPEC.md Normal file
View File

@@ -0,0 +1,217 @@
# NS App — Design Specification
**Status:** Living document · reverse-engineered from the current
implementation (branch `main`, commit `4e0acde`).
**Owner:** NS Innovations Engineering
**Applies to:** `ns_app` Frappe/ERPNext custom application.
This spec describes the intended behavior, contracts, and design constraints of
NS App so the system can be maintained, extended, and re-implemented
consistently. For a component tour see [ARCHITECTURE.md](./ARCHITECTURE.md).
---
## 1. Goals & non-goals
### Goals
- Let staff take a card payment **without leaving the Sales Invoice**, and have
the ledger (Payment Entry) update automatically and correctly.
- Support **saved cards (AutoPay)** for frictionless repeat/recurring billing.
- Allow one card charge to settle **multiple outstanding invoices**.
- Keep the app **PCI-light**: card data never transits ERPNext.
- Speed up **customer onboarding** with a single guided dialog that produces a
complete, linked Customer/Contact/Address.
### Non-goals
- The app is **not** a general payment-gateway abstraction — it targets NMI
specifically.
- It does **not** manage subscriptions/scheduling itself; "AutoPay" here means a
vaulted card that can be charged on demand or via webhook, not a scheduler.
- It does **not** own the custom-field schema on Customer (assumed present).
---
## 2. Personas & primary use cases
| Persona | Use case |
|---------|----------|
| **AR / billing clerk** | Opens an unpaid invoice, clicks *Run Payment*, keys the customer's card, optionally saves it for AutoPay. |
| **Clerk (repeat customer)** | Opens an unpaid invoice for a customer with AutoPay; confirms a one-click charge of the saved card. |
| **Clerk (bulk settle)** | Charges one card for several of a customer's open invoices at once. |
| **Sales user** | Creates a new customer from any link field via the guided Quick Entry dialog. |
| **Payment gateway (system)** | Posts an async confirmation to the webhook, which reconciles a Payment Entry. |
---
## 3. Functional requirements
### 3.1 Payment button visibility
- **Shown** only when: `docstatus == 1` (submitted) **and** a `customer` is set
**and** `outstanding_amount > 0`.
- When `outstanding_amount <= 0`: show a green **Paid** dashboard indicator, no
button.
- Otherwise: show a red **Unpaid** indicator plus **Run Payment** under
*Actions*.
### 3.2 AutoPay-vs-manual branching
- On *Run Payment*, call `check_autopay(customer)`.
- If `autopay_enabled` and `autopay_id` present → confirm dialog → charge the
vaulted card (`run_autopay_payment`).
- Else → open the manual card-entry dialog.
### 3.3 Manual payment dialog
- Collects: first name, last name, company (optional), billing ZIP.
- Renders **Collect.js inline fields** for card number / expiry / CVV.
- Prefills name/company/ZIP from the invoice's customer where possible.
- Optional **Save for Auto Pay** checkbox.
- Optional **Pay Additional Invoices** toggle → loads the customer's other open
invoices (`get_unpaid_invoices`) into a selectable table with a running
selected-total and a select-all control.
- **Pay** button label reflects the current selected total.
### 3.4 Charge semantics (server)
`run_token_payment` must:
1. Resolve `invoice_names` (JSON string → list; fall back to `[invoice]`).
2. Force `save_autopay = 0` when `enable_autopay_signup` is falsy.
3. Load every selected invoice; **reject** if any is not submitted or already
fully paid.
4. Sum `outstanding_amount` across selected invoices as the charge amount.
5. Generate a unique `orderid` (`<invoice-label>-<hash>`).
6. POST a `type=sale` transaction to NMI with the `payment_token`.
7. On `response == "1"`:
- **Dedup**: if a Payment Entry already exists with
`reference_no == transactionid`, return `duplicate: true` and do nothing.
- Else create **one** Payment Entry allocated across all selected invoices.
- If vaulting requested and a `vault_id` returned, persist AutoPay fields on
the Customer.
8. On failure: return `{success: False, error}` and **create no Payment Entry**.
### 3.5 AutoPay charge (server)
`run_autopay_payment``call_payment_api`:
- Requires `custom_auto_pay_status` and `custom_auto_pay_id`.
- POSTs `type=sale` with `customer_vault_id` (no token, no card entry).
- Derives `mode_of_payment` from the response `type` (`check` → ACH, else Credit
Card).
- Same dedup + `create_payment_entry` path.
### 3.6 Payment Entry creation (invariant)
`create_payment_entry(invoices[], transaction_id, mode_of_payment)`:
- Idempotent on `reference_no == transaction_id`.
- `paid_to` = `"ENB Bank Account - NIL"` for ACH/Credit Card, else the company's
default cash account; throw if none resolved.
- `payment_type = Receive`, party = customer of the first invoice.
- One `references` row per invoice, `allocated_amount = outstanding_amount`.
- Insert **and submit** with `ignore_permissions=True`.
### 3.7 Webhook
`crystalclear_webhook` (`allow_guest=True`):
- Ignore unless `response == "1"`.
- Map `orderid` → Sales Invoice, create a Payment Entry via the shared path.
- Always return a short string ack.
### 3.8 Customer Quick Entry
- Override ERPNext's Customer quick-entry dialog globally, preserving the
originating `after_insert` link-field callback.
- `create_customer_full` requires: `customer_name`, `customer_type`,
`customer_group`, `mobile_no`, `address_line1`, `pincode`, `country`.
- Enforce: if `custom_auto_pay_enabled` then `custom_auto_pay_id` required.
- Create Customer + Contact + Address in a single transaction; rollback on any
error and log.
- Return the new customer name; caller writes it back into the triggering field
via `after_insert({ name })`.
---
## 4. Interface contracts (server API)
All are `@frappe.whitelist()` unless noted. Return values are dicts consumed by
`frappe.call` on the client.
| Method | Args | Returns |
|--------|------|---------|
| `check_autopay` | `customer` | `{autopay_enabled: bool, autopay_id: str\|None}` |
| `get_unpaid_invoices` | `customer` | `[{name, posting_date, customer_name, outstanding_amount}]` |
| `run_token_payment` | `invoice, token, invoice_names?, first_name?, last_name?, company?, billing_zip?, save_autopay?` | `{success, transaction_id?, vault_id?, duplicate?, error?}` |
| `run_autopay_payment` | `invoice` | `{success, message, transaction_id}` or throws |
| `save_to_autopay` | `customer, token, first_name?, last_name?, company?, billing_zip?` | `{success, vault_id?}` / `{success:False, error}` |
| `crystalclear_webhook` | form dict (guest) | `"ok"` / `"ignored"` |
| `create_customer_full` | `**data` (see 3.8) | new customer `name` (str) or throws |
### Error conventions
- **Validation / preconditions** → `frappe.throw` (surfaces as a msgprint).
- **Gateway / recoverable** → `{success: False, error: <message>}`.
- **Post-charge Payment Entry failures** are caught and logged (the charge
already succeeded) — they must **not** raise to the client.
---
## 5. Design constraints & rationale
| Constraint | Rationale |
|------------|-----------|
| Card fields via Collect.js iframes only | Keep PAN/CVV out of ERPNext → minimal PCI scope. |
| Dedup on gateway `transactionid` | The charge is the source of truth; retries/webhook races must not double-post to the ledger. |
| Charge-then-record ordering, with PE errors logged not raised | Never lose money already captured at the gateway; reconcile a missing PE manually rather than re-charging. |
| `enable_autopay_signup` feature flag | Ship vaulting code dark; enable per-site only after testing. |
| Preserve `after_insert` in quick entry | Only correct way to resume the originating document flow without racing the create transaction. |
| Single DB transaction in `create_customer_full` | Never leave an orphan Customer without Contact/Address. |
---
## 6. Idempotency, concurrency & failure modes
- **Double-click / double-submit:** client guards with
`window.ns_payment_processing`; server guards with the `reference_no` dedup.
- **Charge succeeds, PE fails:** logged under
`PAYMENT ENTRY FAILURE AFTER SUCCESSFUL CHARGE`; invoice stays unpaid in
ERPNext until reconciled. **Recovery:** re-run `create_payment_entry` using
the logged `transaction_id` (idempotent).
- **Webhook after interactive PE:** dedup prevents a second PE.
- **Gateway unreachable:** returns a generic error; no PE created.
- **Multi-invoice partial validity:** if any selected invoice is unpaid-invalid
(not submitted / zero balance) the whole request is rejected **before**
charging.
---
## 7. Security requirements
- `nmi_security_key` server-side only (`site_config.json`); never sent to the
client.
- HTTPS for every outbound gateway call.
- Store only vault IDs, never card data.
- **Recommended hardening (not yet implemented):**
- Authenticate the webhook (shared secret / signature) before creating PEs.
- Move the Collect.js tokenization key and `paid_to` account into config.
- Scrub gateway response logging to guarantee no PAN/PII is written.
- Rate-limit / permission-check payment endpoints.
---
## 8. Open questions / future work
- Should AutoPay include a **scheduler** (true recurring billing) rather than
on-demand vault charges only?
- Ship the Customer `custom_*` fields as **fixtures** so installs are
self-contained.
- Consolidate `custom.js` and `customer_quick_entry.js`.
- Make `paid_to` company-aware instead of the hard-coded ENB account.
- Add automated tests around `create_payment_entry` idempotency and
multi-invoice allocation.
---
## 9. Acceptance criteria (smoke test)
1. Submitted unpaid invoice shows **Run Payment**; paid invoice shows **Paid**.
2. Manual charge with a test card creates exactly one submitted Payment Entry;
invoice outstanding goes to 0.
3. Repeating the same gateway transaction (webhook replay) creates **no** second
PE.
4. AutoPay customer: *Run Payment* → confirm → one-click charge succeeds.
5. Multi-invoice: selecting N invoices produces one PE with N allocations
summing to the charged amount.
6. Quick Entry creates a linked Customer/Contact/Address and populates the
originating link field.
7. With `enable_autopay_signup = 0`, checking *Save for Auto Pay* creates no
vault entry.

30
docs/README.md Normal file
View File

@@ -0,0 +1,30 @@
# NS App — Documentation
Reference documentation for the **NS App** ERPNext/Frappe custom application
(NS Innovations). This app extends the Sales workflow with embedded card
payments, AutoPay vaulting, multi-invoice settlement, guided customer
onboarding, and branded print formats.
## Contents
| Document | What's in it |
|----------|--------------|
| [ARCHITECTURE.md](./ARCHITECTURE.md) | What the app does, component map, payment flow diagram, data model, external dependencies, configuration, and known tech debt. |
| [DESIGN_SPEC.md](./DESIGN_SPEC.md) | Goals/non-goals, personas, functional requirements, server API contracts, design constraints, failure modes, and acceptance criteria. |
| [SECURITY_NOTES.md](./SECURITY_NOTES.md) | Current security posture and a prioritized hardening list. |
## Quick orientation
- **Backend:** `ns_app/api/payments.py` (NMI gateway), `ns_app/api/customer.py`
(atomic customer creation).
- **Frontend:** `ns_app/public/js/sales_invoice.js` (payment UI),
`ns_app/public/js/customer_quick_entry.js` (customer quick entry).
- **Wiring:** `ns_app/hooks.py`.
- **Print formats:** `ns_app/print_formats/` (shipped as fixtures).
## Related
- Root [README.md](../README.md) — install & configuration guide.
- `ns_app/api/payment_flow_documentation.md` — original narrative walkthrough of
the payment flow (kept for historical context; superseded by
[ARCHITECTURE.md](./ARCHITECTURE.md) §3).

33
docs/SECURITY_NOTES.md Normal file
View File

@@ -0,0 +1,33 @@
# NS App — Security Notes & Hardening
Companion to [ARCHITECTURE.md](./ARCHITECTURE.md) §9 and
[DESIGN_SPEC.md](./DESIGN_SPEC.md) §7.
## Current posture (as implemented)
- **PCI scope minimized.** Card number, expiry, and CVV are entered into
NMI-hosted Collect.js iframes. ERPNext receives only a single-use
`payment_token`. Raw card data never reaches the browser JS context or the
server.
- **No card storage.** Only the NMI Customer Vault ID is persisted on the
Customer (`custom_auto_pay_id`).
- **Secret handling.** `nmi_security_key` is read from `site_config.json`
(server-side) and sent only in server→NMI requests.
- **Transport.** All gateway calls use HTTPS to `secure.nmi.com`.
## Gaps & recommended hardening
| # | Issue | Recommendation |
|---|-------|----------------|
| 1 | **Unauthenticated webhook.** `crystalclear_webhook` is `allow_guest=True` and creates Payment Entries from any POST whose `orderid` matches an invoice. | Require a shared secret / HMAC signature; verify before writing. Optionally allow-list source IPs. |
| 2 | **Debug logging of gateway payloads.** `payments.py` writes request/response snippets via `frappe.log_error`. | Confirm no PAN/PII is ever logged; use a dedicated logger at an appropriate level; consider truncation/redaction and log retention limits. |
| 3 | **Hard-coded tokenization key** in `sales_invoice.js`. | Move to a server-provided value / site config; makes key rotation and per-environment keys possible. |
| 4 | **Hard-coded ledger account** `"ENB Bank Account - NIL"`. | Make company-aware via config or a Company-level custom field. |
| 5 | **Permissions.** Payment endpoints are whitelisted to any logged-in user. | Add role checks (as `create_customer_full` does with `frappe.only_for`) and/or rate limiting. |
| 6 | **`ignore_permissions=True`** on Payment Entry and Customer writes. | Acceptable for a system flow, but document the trust boundary and ensure the whitelisted entry points are themselves access-controlled. |
## Operational reminders
- Keep `enable_autopay_signup = 0` in production until vaulting is fully tested.
- Never commit `site_config.json` or the NMI security key to version control.
- Rotate the NMI security key and Collect.js key on any suspected exposure.