init

DEVELOPMENT.md

@@ -0,0 +1,168 @@
+# Development
+
+## Prerequisites
+
+```bash
+sudo apt install -y qemu-system-x86 libvirt-daemon-system libvirt-clients \
+    genisoimage ovmf screen
+sudo usermod -aG libvirt $(whoami)
+sudo usermod -aG kvm $(whoami)
+pipenv install --dev
+pipenv run ansible-galaxy collection install -r ansible/requirements.yml
+```
+
+## Repository layout
+
+```
+servers/
+├── ansible/
+│   ├── inventories/
+│   │   ├── debug/          # Local VMs (vm-proxy, vm-idp)
+│   │   └── prod/           # Production VPS hosts
+│   ├── playbooks/
+│   │   ├── setup.yml       # Main deployment playbook
+│   │   ├── backup.yml      # Backup playbook
+│   │   └── restore.yml     # Restore playbook
+│   ├── roles/              # All roles (formerly a separate collection)
+│   └── requirements.yml    # Ansible Galaxy dependencies
+├── scripts/
+│   ├── provision.sh        # One-time: create VMs + full playbook + snapshot
+│   ├── deploy.sh           # Iterate: restore + deploy a single service
+│   ├── reset.sh            # Restore VMs to a snapshot
+│   ├── prod-deploy.sh      # Deploy to production
+│   └── vm/                 # Low-level VM lifecycle (wrapping qemu-vm)
+├── letsencrypt/            # TLS certificates (not committed)
+├── Pipfile                 # Python dependencies
+└── ansible.cfg             # Ansible config (roles_path, vault)
+```
+
+## Development workflow
+
+The workflow is: develop and test locally against VMs, then deploy to
+production once changes are idempotent and working.
+
+All scripts are meant to be run inside `pipenv shell` or via `pipenv run`.
+
+### 1. Provision (once)
+
+Creates VMs, runs the full ansible playbook, and snapshots the result as
+`provisioned`.  Prompts for sudo to set up the bridge network:
+
+```bash
+pipenv run scripts/provision.sh
+```
+
+This gives you two VMs:
+- **vm-proxy** at `10.10.0.2` (screen session `vm-proxy`)
+- **vm-idp** at `10.10.0.3` (screen session `vm-idp`)
+
+Attach to a VM's serial console with `screen -r vm-proxy`.
+
+### 2. Iterate on a service
+
+Deploy a single service to the local VMs:
+
+```bash
+# Deploy bugzilla on top of current VM state
+pipenv run scripts/deploy.sh bugzilla
+
+# Skip dependencies you know haven't changed
+pipenv run scripts/deploy.sh bugzilla --skip docker,apache
+
+# Restore to clean state first, then deploy
+pipenv run scripts/deploy.sh prosody --restore
+
+# Restore to a specific snapshot first
+pipenv run scripts/deploy.sh authentik --restore initialized
+```
+
+### 3. Reset
+
+Go back to a known state without deploying anything:
+
+```bash
+# Reset to bare VMs (pre-ansible)
+pipenv run scripts/reset.sh
+
+# Reset to fully provisioned state
+pipenv run scripts/reset.sh provisioned
+```
+
+### 4. Custom snapshots
+
+Save and restore intermediate states during development:
+
+```bash
+# Save current state
+pipenv run scripts/vm/snapshot.sh after-bugzilla
+
+# Restore it later
+pipenv run scripts/reset.sh after-bugzilla
+```
+
+### 5. Destroy and recreate
+
+Start completely from scratch:
+
+```bash
+pipenv run scripts/vm/destroy.sh
+pipenv run scripts/provision.sh
+```
+
+### 6. Deploy to production
+
+Once your changes work locally and are idempotent:
+
+```bash
+# Full deployment
+pipenv run scripts/prod-deploy.sh
+
+# Targeted deployment
+pipenv run scripts/prod-deploy.sh prosody
+
+# Skip dependencies
+pipenv run scripts/prod-deploy.sh bugzilla --skip docker,apache
+```
+
+## Low-level VM management
+
+The `scripts/vm/` directory contains building-block scripts for direct
+VM control:
+
+| Script | Purpose |
+|--------|---------|
+| `vm/start.sh` | Start VMs in screen sessions |
+| `vm/stop.sh` | Stop VMs gracefully |
+| `vm/status.sh` | Show instances, volumes, screen sessions |
+| `vm/snapshot.sh <label>` | Stop + snapshot both volumes |
+| `vm/restore.sh <label>` | Restore + start both VMs |
+| `vm/create.sh` | First-time VM creation + cloud-init |
+| `vm/destroy.sh` | Delete VMs and volumes |
+| `vm/setup-network.sh` | Create bridge network (requires root) |
+
+## Working with roles
+
+Roles live directly in `ansible/roles/`. No collection build/install step is
+needed — Ansible finds them via `roles_path` in `ansible.cfg`.
+
+Each role is referenced by its directory name (e.g., `include_role: { name: prosody }`).
+Cross-role dependencies use the same short names.
+
+## Working with vault
+
+```bash
+# Edit encrypted variables
+pipenv run ansible-vault edit ansible/inventories/prod/group_vars/all/vault.yml
+
+# Encrypt/decrypt files
+ansible-vault-dir encrypt <file>
+ansible-vault-dir decrypt <file>
+```
+
+## YAML formatting
+
+The project uses `yamlfix` for consistent YAML formatting:
+
+```bash
+pipenv run yamlfix ansible/playbooks/setup.yml
+```