# SMS Admin ES - Full Project Summary (In-Depth)

## 1) Executive Summary

This repository is a **large Laravel 11 + Filament 3 admin platform** for managing subscription-based digital products and operational analytics. It combines:

- Multi-tenant admin operations (tenant isolated by `app_id`)
- Customer and subscription lifecycle management
- Billing, support desk, tax administration, and reports
- Dashboard and analytics modules with high-volume read endpoints
- Elasticsearch-backed query paths for performance
- AI-driven churn prediction pipeline (Python + LightGBM + SHAP)
- AI chat analyst for natural-language data insights

The project is mature and heavily modularized. It exposes both:

- UI routes (Filament admin panel)
- JSON/AJAX APIs for widgets, listing tables, exports, and external/bearer use cases

Current route map includes **285 routes** (`php artisan route:list`, checked on 2026-05-12).

---

## 2) Tech Stack and Runtime

### Backend

- PHP `^8.2` (Docker currently uses PHP 8.3 image)
- Laravel `^11.31`
- Filament `^3.2`
- Livewire `^3.5`
- Sanctum `^4.2` (personal access token support)
- Octane available (`laravel/octane`)
- DomPDF (`barryvdh/laravel-dompdf`) for PDF/report use

### Frontend Tooling

- Vite 6
- Tailwind CSS
- Chart.js
- Axios

### Data and Search

- PostgreSQL support configured and heavily used in code
- MySQL/MariaDB/sqlite config also available
- Elasticsearch integration (`elasticsearch/elasticsearch`) with dedicated services per module

### ML/AI

- Python scripts in `ml/`
- LightGBM churn models (global + app-specific)
- SHAP explainability for top factors

---

## 3) High-Level Architecture

The project follows a module-based Laravel structure, with key layers:

1. **Filament resources/pages/widgets** for admin UI
2. **Controllers** for AJAX/API endpoints
3. **Services** for business logic and query orchestration
4. **Models** with app-level scoping
5. **Middleware** for auth mode enforcement, resilience, and caching headers
6. **Python ML pipeline** for churn features, training, and scoring

Core directories:

- `app/Filament` (admin pages/resources/widgets)
- `app/Http/Controllers` (feature APIs)
- `app/Services` (business logic + ES adapters)
- `app/Models` (domain entities)
- `resources/views/filament` (custom UI rendering/hooks)
- `ml/` (churn AI pipeline)
- `docs/` (extensive feature + query documentation)

---

## 4) Admin Panel Composition (Filament)

The admin panel provider (`App\Providers\Filament\AdminPanelProvider`) configures:

- Panel path: `/admin`
- Custom login page
- Dynamic navigation groups loaded from DB `modules` table and cached
- Per-user/module access filtering using module IDs
- Render hooks for:
  - font behavior
  - sidebar script behavior
  - global alerts
  - API bearer token exposure in head

Navigation groups default to **Core / Operations / Analytics / Reports**, then extended dynamically from DB.

---

## 5) Multi-Tenant Isolation Strategy

Tenant isolation is centered around `app_id` with multiple safeguards:

- Global model scope: `App\Models\Scopes\AppIdScope`
- Additional trait-based filtering where direct `app_id` is absent (`FilteredByAppId`)
- Helpers (`DashboardHelper`, support services) normalizing app context from authenticated user/session
- Cache key partitioning with app-specific keys (`app_key`) to avoid cross-tenant cache leaks

Important implementation note:

- App ID source generally prioritizes **authenticated user app_id** over session to reduce stale-session cross-user contamination.

---

## 6) Authentication and API Access Modes

The project uses **two important API access patterns**:

1. **Session-authenticated web/admin AJAX**
2. **Bearer-token-only APIs** (Sanctum PAT enforced)

Custom middleware `RequireBearerToken` strictly validates:

- `Authorization: Bearer ...`
- Sanctum token existence and ownership
- user hydration from token

Token lifecycle endpoints are exposed under `/admin/api/tokens/*` via `ApiTokenController`:

- generate token
- get current token
- list/revoke/revoke-all tokens

`EnsureApiToken` middleware auto-generates/maintains session token for authenticated users where needed.

---

## 7) Major Product Modules

### A) Dashboard

- Rich widget APIs under `/admin/api/dashboard/widgets/*`
- Supports KPI cards, conversion/cohort/timeline/payment charts
- At-risk/churned customer lists with pagination
- Caching + CDN-friendly responses + filter hashing
- Performance profiling/logging per section (query count and timing)

### B) Subscriber 360

- JSON endpoints for cards, charts, table, and per-customer related listings
- Works with both DB and Elasticsearch paths
- Includes orders/invoices/payments/devices/tickets/coupon consumption endpoints

### C) Business Analytics

- KPI and trend endpoints (`MRR`, `ARR`, churn, LTV, activations, revenue)
- Segment APIs (list/KPIs/details)
- Support service layer for date/filter resolution and query logging

### D) Billing + Dunning

- Billing widgets and exports
- Payment trends, gateway distribution, invoice/subscription lists
- Dunning campaigns CRUD/toggle/detail paths

### E) Back Office

- Email/SMS templates
- Bulk SMS/email dispatch flows
- Coupon management and coupon analytics

### F) Product + Package Management

- Product action API (`insert|update|delete` style endpoint)
- Package CRUD via AJAX
- Upgrade subscription flows from UI and API

### G) Tax Administration

- Regions/tax groups/business units/compliance reports APIs
- Mapping and compliance dashboard endpoints

### H) Reports

- Report filter options
- Report-specific data and export endpoints

### I) Global Search and Risk/AI Pages

- Global search module
- Risk Management page
- AI churn management and AI chat agent page

---

## 8) Service Layer Patterns

`app/Services` contains core logic and adapters, including:

- `DashboardMetricsService`, snapshot services, query debug service
- `BusinessAnalyticsService` + support service
- `SubscriptionUpgradeService` (idempotent, proration/effective modes, deferred execution)
- `NavigationMenuService`, `PlanDistributionService`
- `Tax/ComplianceDashboardService`
- `Sms/*` bulk dispatch abstractions
- `Elasticsearch/*` specialized services per module

A recurring pattern is:

- Try Elasticsearch path when enabled/configured
- Fall back to SQL path when ES unavailable

This pattern appears in multiple analytics services and helps balance speed + resilience.

---

## 9) Elasticsearch Integration

Dedicated ES service classes exist for major modules:

- Dashboard
- Billing
- Business Analytics
- Business Units
- Campaign Segment
- Global Search
- Product Admin
- Reports
- Subscriber360
- BackOffice coupon analytics

Configuration in `config/elasticsearch.php` includes:

- `ES_URL` / host / port / auth / API key
- index name default: `customer_subscription_v1`
- data-source mode (e.g., dashboard)

The `docs/elastic-search/` folder is extensive and includes:

- query references
- field subsets
- cURL examples
- service maps

---

## 10) AI Churn Pipeline (ML)

The `ml/` directory contains an orchestrated churn pipeline:

- `feature_engineering.py` (build/refresh feature store)
- `generate_labels.py` (training labels)
- `train_model.py` (hybrid training)
- `predict.py` (batch scoring + SHAP explanations)
- `config.py` (DB config + feature list + model naming)

Model strategy:

- **Global fallback model** (`lgbm_global.pkl`)
- **App-specific model** per `app_id` if sample threshold met
- Registry tracking in `churn_model_registry`

Prediction output:

- Upserts into `churn_predictions`
- Risk bands (`low`, `medium`, `high`, `critical`)
- Top risk factors + recommended actions JSON

Laravel command orchestrator:

- `php artisan churn:pipeline:run` (`RunAiChurnPipeline`)

Operational shell wrappers:

- `churn_first_run.sh`
- `churn_nightly_update.sh`
- `churn_monthly_retrain.sh`

Scheduler (`routes/console.php`) currently includes:

- daily churn nightly script
- monthly retrain script

---

## 11) AI Chat Data Analyst

`AiChatApiController` + `AiDataAgentService` provide a conversational data-analyst flow:

- Streamed responses (`askStream`) over event stream
- Prompt-injection phrase stripping and input sanitization
- SQL generation + execution + summary behavior in service
- Sensitive field redaction and internal schema name sanitization
- Retry behavior for failed generated queries

This module is intended for safe natural-language querying while reducing risk of exposing raw internal structures.

---

## 12) Database Artifacts and Data Bootstrapping

Current state in repository:

- `database/migrations/` is empty in this workspace snapshot
- Schema artifact exists at `database/schemas/schema.sql`
- Multiple seeders available for master/type/reference data

Also present:

- `mtv_sms_dump.sql`
- `table_schemas.json`

This implies schema evolution may be managed externally or via SQL/schema snapshots rather than fully tracked Laravel migrations in this branch snapshot.

---

## 13) Middleware and Resilience Features

Notable custom middleware:

- `RequireBearerToken` (strict PAT enforcement)
- `EnsureApiToken` (auto-session token management)
- `HandleDatabaseErrors` (graceful login/admin behavior on DB outages)
- `CdnCacheHeaders` (cache headers + ETag for key GET API paths)
- `TrackNavigation`, `QueryDebugMiddleware`

Bootstrap configuration (`bootstrap/app.php`) includes:

- trusted proxy headers for cloud environments
- CSRF exceptions for selected API endpoints
- custom exception rendering for database connectivity problems

---

## 14) Performance and Caching Strategy

Observed performance patterns:

- Aggressive module-specific response caching
- app-aware cache keys + filter hash normalization
- CDN-friendly response wrappers for widget APIs
- optional query debug capture and grouping (`QueryDebugService`)
- dashboard section profiling logs (latency + query stats)
- ES offloading for expensive analytics workloads

---

## 15) Deployment and Infrastructure

### Docker

- Apache + PHP image
- Composer install during build
- PHP extensions include DB drivers (`pdo_mysql`, `pdo_pgsql`, ldap, etc.)

### Cloud Build

`cloudbuild.yaml` pipeline:

1. Docker build tagged by `${BUILD_ID}`
2. Push to Artifact Registry
3. Deploy to Cloud Run (`asia-south1`)
4. Route traffic to latest revision

---

## 16) Testing and Quality Signals

- PHPUnit configured (`phpunit.xml`)
- Current visible tests are limited (few unit/feature tests)
- Large query-heavy domain relies heavily on runtime safeguards, logging, and docs

Potential quality gap:

- Test coverage appears small relative to system size

---

## 17) Documentation Quality in Repo

The repo has a large `docs/` corpus with deep coverage of:

- SQL inventories
- data mapping
- feature flows (Dashboard, Subscriber360, Support, Business Analytics)
- Elasticsearch query references
- API-specific docs
- troubleshooting and smoke test checklists

This is a strong operational asset and suggests ongoing production support usage.

---

## 18) Strengths

- Strong modular decomposition for a large admin suite
- Good tenant-isolation awareness in many layers
- Hybrid SQL + Elasticsearch strategy for scalability
- AI churn pipeline integrated into app operations
- Rich operational docs and query inventories
- Defensive auth patterns (bearer-only endpoints + token management)

---

## 19) Risks / Improvement Opportunities

1. **Migration tracking gap**
- Empty `database/migrations` in this snapshot can complicate reproducibility.

2. **Token lifecycle controls**
- Some tokens are no-expiry by default; consider policy hardening per environment.

3. **Coverage gap**
- Automated test suite appears small for a 285-route system.

4. **Mixed legacy/new paths**
- Presence of old/duplicate controllers/resources (`OLD` names, duplicate namespaces) may increase maintenance complexity.

5. **Large controller/service files**
- Some files are very large and can become brittle without additional modular extraction.

---

## 20) Quick Operational Reference

### Local app run

- `composer install`
- `npm install`
- `php artisan serve`
- `npm run dev`

or use:

- `composer dev`

### Useful commands

- `php artisan route:list`
- `php artisan churn:pipeline:run --all --train`
- `php artisan churn:pipeline:run --all`
- `php artisan churn:cache-refresh --app-id=0 --validate --alert`

### Key config files

- `config/database.php`
- `config/elasticsearch.php`
- `config/ai_churn.php`
- `config/query-debug.php`

---

## 21) Final Assessment

This is a **production-oriented, enterprise-style admin platform** with broad operational surface area (billing, support, tax, analytics, product lifecycle, reporting, AI). The architecture is not minimal, but it is practical: feature modules + service layer + ES adapters + operational scripts.

The most important strategic strengths are:

- tenant-aware design,
- performance-minded analytics APIs,
- integrated churn intelligence,
- deep internal documentation.

The highest-value next engineering investments would be:

- stronger automated test coverage,
- migration/version reproducibility hardening,
- continued refactoring of very large files into smaller domain components.

---

## 22) Analysis Scope Note

This summary is based on static code/config/document inspection in this workspace on **2026-05-12**, including route registry output and key service/controller/ML files. It does not include live external dependency health checks (DB/ES runtime telemetry) beyond what is encoded in source and scripts.