# Operations Runbook — Fon2Fix POS

This document covers the preflight, rotation, deployment verification, and recurring checkup procedures introduced in the security hardening pass. Keep it next to the codebase.

---

## 1. Preflight (run BEFORE deploying the patched files to production)

Run each step on the live server. Do not proceed if any step fails.

```bash
# 1.1 Take a full backup via the existing UI button OR:
php /home/USER/public_html/my_pos/backup.php > /dev/null   # only if you have CLI bootstrapping
# Otherwise: click "BACKUP" on the dashboard and download the ZIP.

# 1.2 Snapshot every file we will touch (also done automatically; this is a
#     backup-of-backup):
cd /home/USER/public_html/my_pos
mkdir -p backups/pre_patch/$(date +%Y%m%d)
cp db.php index.php api.php backup.php uptime_monitor.php bug_scanner.php \
   optimize_database.php sync_to_backup.php php.ini config.php helpers.php \
   dashboard.php invoice.php receipt.php README.md \
   backups/pre_patch/$(date +%Y%m%d)/

# 1.3 Verify HTTPS cert validity (>= 14 days remaining):
echo | openssl s_client -servername system.fon2fix.com.my \
     -connect system.fon2fix.com.my:443 2>/dev/null | openssl x509 -noout -dates

# 1.4 Confirm config.php is readable and locked down:
ls -l config.php   # expected: -rw------- 1 user user ... (mode 600)
chmod 600 config.php

# 1.5 List every existing cron entry (you will need this BEFORE the cutover
#     because uptime_monitor.php now requires ?access_token=...):
crontab -l > backups/pre_patch/$(date +%Y%m%d)/crontab.txt
```

---

## 2. Secret Rotation (do these one at a time, smoke-test after each)

After deploying the patched files, rotate every value that was previously committed in source.

### 2.1 Database password

1. Generate a new strong password.
2. In cPanel → MySQL Databases → change password for `fonfixco_leng`.
3. Edit `config.php` and update `database.password`.
4. Smoke test: load `index.php`, log in, navigate to dashboard. If the login fails, restore `config.php` from `backups/pre_patch/*/config.php`.
5. Then update the password in your backup-server `config.php` so `sync_to_backup.php` continues to work.

### 2.2 Backup sync `secret_key`

1. Generate a long random string: `openssl rand -base64 48 | tr -d '/+=' | head -c 64`.
2. Update `backup_sync.secret_key` in BOTH `config.php` (this server) AND the backup server's `restore_from_sync.php` configuration.
3. Smoke test: `curl "https://yoursite/sync_to_backup.php?cron=1&cron_key=…"` and confirm a 200 in `backups/sync_log.txt`.

### 2.3 Backup `cron_key` and uptime `monitor_token`

1. Generate two new random strings.
2. Update `backup_sync.cron_key` and `backup_sync.monitor_token` in `config.php`.
3. Update every cron entry that calls these scripts. **The uptime monitor will silently 403 if you skip this step.**

### 2.4 Telegram bot token

1. Open Telegram → BotFather → `/revoke` for the old token → save the new token.
2. Update `telegram.bot_token` in `config.php`.
3. Smoke test: trigger `uptime_monitor.php?access_token=…` (CLI is fine) and confirm a synthetic alert lands in the chat.

---

## 3. Cron Installation

Replace any prior fonfix cron entries with the block below. Substitute `{MONITOR_TOKEN}` and `{CRON_KEY}` with the values from `config.php`. Stagger minute offsets to avoid load spikes.

```cron
# every 5 minutes - uptime
*/5 * * * * curl -fsS "https://system.fon2fix.com.my/uptime_monitor.php?access_token={MONITOR_TOKEN}" > /dev/null 2>&1

# hourly - health check
15 * * * * curl -fsS "https://system.fon2fix.com.my/health_check.php?cron=1&token={CRON_KEY}&format=json" > /dev/null 2>&1

# daily 02:25 - off-site backup
25 2 * * * curl -fsS "https://system.fon2fix.com.my/sync_to_backup.php?cron=1&cron_key={CRON_KEY}" > /dev/null 2>&1

# daily 03:30 - rotate logs
30 3 * * * curl -fsS "https://system.fon2fix.com.my/log_rotate.php?token={CRON_KEY}" > /dev/null 2>&1

# weekly Sun 01:45 - security audit
45 1 * * 0 curl -fsS "https://system.fon2fix.com.my/security_audit.php?cron=1&token={CRON_KEY}" > /dev/null 2>&1
```

Alternative for environments with one cron slot only:

```cron
*/15 * * * * curl -fsS "https://system.fon2fix.com.my/cron_runner.php?token={CRON_KEY}" > /dev/null 2>&1
```

---

## 4. Post-deployment verification

Run every check; the deployment is not complete until all pass.

```bash
# 4.1 Sensitive files are denied
curl -i https://system.fon2fix.com.my/config.php       # expect 403
curl -i https://system.fon2fix.com.my/db.php           # expect 403
curl -i https://system.fon2fix.com.my/php.ini          # expect 403

# 4.2 Bot UA is blocked
curl -i -A "" https://system.fon2fix.com.my/api.php    # expect 403

# 4.3 Login still works (interactive — open browser)

# 4.4 Health check returns green for an authenticated Master/Endie user
#     (open https://.../health_check.php in browser logged in as endie)

# 4.5 Backup sync cron succeeds
curl -fsS "https://system.fon2fix.com.my/sync_to_backup.php?cron=1&cron_key={CRON_KEY}"
# Then look at backups/sync_last_status.json - "ok": true

# 4.6 Uptime cron rejects unauth, accepts with token
curl -i https://system.fon2fix.com.my/uptime_monitor.php                  # 403
curl -i "https://system.fon2fix.com.my/uptime_monitor.php?access_token={MONITOR_TOKEN}"   # 200

# 4.7 Browser smoke test — open dashboard:
#  - QR scan in Stock module works            (html5-qrcode pinned)
#  - Charts render on Report module           (chart.js pinned)
#  - PDF export from receipt page works       (html2pdf.js pinned)
#  - No SRI rejection errors in DevTools console

# 4.8 bug_scanner.php still clean
#  - Visit https://.../bug_scanner.php as endie - issues count unchanged from baseline.
```

---

## 5. Rollback

Every patched file has a snapshot in `backups/pre_patch/{YYYYMMDD}/`. To revert any single file:

```bash
cd /home/USER/public_html/my_pos
DATE=20260525   # the snapshot folder for the deploy you're rolling back
cp backups/pre_patch/$DATE/<filename>.php ./
```

To revert the `.htaccess`:

```bash
rm .htaccess   # there was no prior .htaccess to restore
```

To revert cron changes:

```bash
crontab backups/pre_patch/$DATE/crontab.txt
```

To revert a secret rotation: swap `config.php` back from the snapshot. Then change the live secret back via the appropriate provider (DB, Telegram BotFather, etc.).

---

## 6. Optional follow-ups (not in this pass)

- Migrate the 585 inline `onclick=`/`style=` attributes out of `dashboard.php` so CSP can be enforced (not just report-only).
- Move `config.php` outside the document root entirely (add `realpath('../config.php')` lookup in `db.php`).
- Snapshot CDN files into a local `vendor/` folder to eliminate the supply-chain link entirely.
- Enable the force-HTTPS block at the bottom of `.htaccess` once the cert lifecycle is solid.
- Bump `Strict-Transport-Security` `max-age` from 300 to 31536000 after a week of trouble-free HTTPS operation.
