# Import Validation and LLM Mapping

## Scope
FieldOps Cloud now has a guided import foundation for:

- Customers
- Jobs
- Quotes
- Invoices
- Team members

The import workflow is validation-first: the customer selects the data type before upload, then FieldOps validates mandatory headers, maps columns, reviews row-level issues, and only then imports records.

## Import Protocol
- Use RFC 4180-style CSV with first-row headers, comma-separated fields, double quote escaping, and consistent line endings.
- Use UTF-8 text and reject binary content, executable macros, scripts, and unsupported MIME types.
- Validate file extension, MIME type, file size, row count, required headers, and row-level values before writing records.
- Use dry-run validation first; never silently import partial or invalid data.
- Store tenant id, user id, import type, header map, validation summary, idempotency key, and row-level error report.
- Treat cells beginning with `=`, `+`, `-`, or `@` as unsafe for later CSV export and require review.

## Mandatory Fields
- Customers: `name`
- Jobs: `title`, `client_name`, `status`
- Quotes: `quote_number`, `client_name`, `status`, `subtotal`, `total`
- Invoices: `invoice_number`, `client_name`, `status`, `total`, `due_date`
- Team: `name`, `email`, `role`

## LLM API Foundation
The platform now has provider metadata, settings UI, Gate 6 readiness snapshots, and provider contract coverage for:

- OpenAI
- Azure OpenAI
- Anthropic
- Google Gemini
- Custom compatible HTTPS endpoints

The current implementation does not store API keys or send imports to a model. Production must encrypt keys, restrict access to tenant admins, and audit all LLM usage.

## AI Assistant Runtime
The operational AI Assistant at `/ai-assistant` is currently a deterministic local assistant backed by `AiAssistantService`.

- `POST /ai-assistant/generate` accepts CSRF-protected form data: `mode=quote|schedule|general`, optional `job_title`, and optional `details`.
- The endpoint requires `use_ai_assistant` RBAC permission.
- The assistant uses local FieldOps job context and does not call external LLM providers.
- The assistant never mutates jobs, quotes, invoices, schedules, or customer records.
- User-provided details are normalised and redact API-key, token, password, secret, and long-card-number-like values before the response is built.
- Generated guidance includes a local-only safety note and must be reviewed by a person before customer use.

## AI Import Guardrails
- Do not send full imported files to an LLM by default.
- Use headers and a small redacted sample for mapping suggestions only after user approval.
- Gate 6 redaction removes emails, phones, addresses, and secret-like fields before any future provider call.
- Never include API keys, passwords, payment card data, or private notes in prompts.
- Store prompt and response hashes for audit instead of raw customer data.
- Require human review before applying model-suggested mappings or row fixes.

## Database Contracts
- `llm_connections`: tenant-scoped LLM provider, model, endpoint, encrypted API key placeholder, status, and verification metadata.
- `llm_usage_logs`: non-sensitive LLM usage audit with prompt/response hashes.
- `import_mapping_profiles`: saved tenant import mappings by import type.
- `import_validation_reports`: dry-run validation summary, missing fields, warnings, row counts, idempotency key, and bounded short-lived normalized replay payload metadata for valid previews.

## Next Build Steps
1. Add hosted import worker runner evidence and hosted large-file performance evidence for production-scale imports. Durable validation report, import job, undo, and export job persistence now have Gate 18 local coverage, bounded retry-state transitions now have Gate 19 local coverage, and local payload replay queue/claim/complete/failure lifecycle now has Gate 21 local coverage.
2. Add CSV parser service using standards-compliant quoting and escaping.
3. Add row-level validators per import type.
4. Add encrypted API key storage using an environment-managed app key.
5. Add LLM mapping endpoint that sends headers and redacted samples only after explicit approval.
6. Add review/commit step with idempotent import writes and audit logs.