168 lines
4.6 KiB
Markdown
168 lines
4.6 KiB
Markdown
# Development
|
|
|
|
## Prerequisites
|
|
|
|
```bash
|
|
sudo apt install -y qemu-system-x86 libvirt-daemon-system libvirt-clients \
|
|
genisoimage ovmf screen
|
|
sudo usermod -aG libvirt $(whoami)
|
|
sudo usermod -aG kvm $(whoami)
|
|
pipenv install --dev
|
|
pipenv run ansible-galaxy collection install -r ansible/requirements.yml
|
|
```
|
|
|
|
## Repository layout
|
|
|
|
```
|
|
servers/
|
|
├── ansible/
|
|
│ ├── inventories/
|
|
│ │ ├── debug/ # Local VMs (vm-proxy, vm-idp)
|
|
│ │ └── prod/ # Production VPS hosts
|
|
│ ├── playbooks/
|
|
│ │ ├── setup.yml # Main deployment playbook
|
|
│ │ ├── backup.yml # Backup playbook
|
|
│ │ └── restore.yml # Restore playbook
|
|
│ ├── roles/ # All roles (formerly a separate collection)
|
|
│ └── requirements.yml # Ansible Galaxy dependencies
|
|
├── scripts/
|
|
│ ├── provision.sh # One-time: create VMs + full playbook + snapshot
|
|
│ ├── deploy.sh # Iterate: restore + deploy a single service
|
|
│ ├── reset.sh # Restore VMs to a snapshot
|
|
│ ├── prod-deploy.sh # Deploy to production
|
|
│ └── vm/ # Low-level VM lifecycle (wrapping qemu-vm)
|
|
├── letsencrypt/ # TLS certificates (not committed)
|
|
├── Pipfile # Python dependencies
|
|
└── ansible.cfg # Ansible config (roles_path, vault)
|
|
```
|
|
|
|
## Development workflow
|
|
|
|
The workflow is: develop and test locally against VMs, then deploy to
|
|
production once changes are idempotent and working.
|
|
|
|
All scripts are meant to be run inside `pipenv shell` or via `pipenv run`.
|
|
|
|
### 1. Provision (once)
|
|
|
|
Creates VMs, runs the full ansible playbook, and snapshots the result as
|
|
`provisioned`. Prompts for sudo to set up the bridge network:
|
|
|
|
```bash
|
|
pipenv run scripts/provision.sh
|
|
```
|
|
|
|
This gives you two VMs:
|
|
- **vm-proxy** at `10.10.0.2` (screen session `vm-proxy`)
|
|
- **vm-idp** at `10.10.0.3` (screen session `vm-idp`)
|
|
|
|
Attach to a VM's serial console with `screen -r vm-proxy`.
|
|
|
|
### 2. Iterate on a service
|
|
|
|
Deploy a single service to the local VMs:
|
|
|
|
```bash
|
|
# Deploy bugzilla on top of current VM state
|
|
pipenv run scripts/deploy.sh bugzilla
|
|
|
|
# Skip dependencies you know haven't changed
|
|
pipenv run scripts/deploy.sh bugzilla --skip docker,apache
|
|
|
|
# Restore to clean state first, then deploy
|
|
pipenv run scripts/deploy.sh prosody --restore
|
|
|
|
# Restore to a specific snapshot first
|
|
pipenv run scripts/deploy.sh authentik --restore initialized
|
|
```
|
|
|
|
### 3. Reset
|
|
|
|
Go back to a known state without deploying anything:
|
|
|
|
```bash
|
|
# Reset to bare VMs (pre-ansible)
|
|
pipenv run scripts/reset.sh
|
|
|
|
# Reset to fully provisioned state
|
|
pipenv run scripts/reset.sh provisioned
|
|
```
|
|
|
|
### 4. Custom snapshots
|
|
|
|
Save and restore intermediate states during development:
|
|
|
|
```bash
|
|
# Save current state
|
|
pipenv run scripts/vm/snapshot.sh after-bugzilla
|
|
|
|
# Restore it later
|
|
pipenv run scripts/reset.sh after-bugzilla
|
|
```
|
|
|
|
### 5. Destroy and recreate
|
|
|
|
Start completely from scratch:
|
|
|
|
```bash
|
|
pipenv run scripts/vm/destroy.sh
|
|
pipenv run scripts/provision.sh
|
|
```
|
|
|
|
### 6. Deploy to production
|
|
|
|
Once your changes work locally and are idempotent:
|
|
|
|
```bash
|
|
# Full deployment
|
|
pipenv run scripts/prod-deploy.sh
|
|
|
|
# Targeted deployment
|
|
pipenv run scripts/prod-deploy.sh prosody
|
|
|
|
# Skip dependencies
|
|
pipenv run scripts/prod-deploy.sh bugzilla --skip docker,apache
|
|
```
|
|
|
|
## Low-level VM management
|
|
|
|
The `scripts/vm/` directory contains building-block scripts for direct
|
|
VM control:
|
|
|
|
| Script | Purpose |
|
|
|--------|---------|
|
|
| `vm/start.sh` | Start VMs in screen sessions |
|
|
| `vm/stop.sh` | Stop VMs gracefully |
|
|
| `vm/status.sh` | Show instances, volumes, screen sessions |
|
|
| `vm/snapshot.sh <label>` | Stop + snapshot both volumes |
|
|
| `vm/restore.sh <label>` | Restore + start both VMs |
|
|
| `vm/create.sh` | First-time VM creation + cloud-init |
|
|
| `vm/destroy.sh` | Delete VMs and volumes |
|
|
| `vm/setup-network.sh` | Create bridge network (requires root) |
|
|
|
|
## Working with roles
|
|
|
|
Roles live directly in `ansible/roles/`. No collection build/install step is
|
|
needed — Ansible finds them via `roles_path` in `ansible.cfg`.
|
|
|
|
Each role is referenced by its directory name (e.g., `include_role: { name: prosody }`).
|
|
Cross-role dependencies use the same short names.
|
|
|
|
## Working with vault
|
|
|
|
```bash
|
|
# Edit encrypted variables
|
|
pipenv run ansible-vault edit ansible/inventories/prod/group_vars/all/vault.yml
|
|
|
|
# Encrypt/decrypt files
|
|
ansible-vault-dir encrypt <file>
|
|
ansible-vault-dir decrypt <file>
|
|
```
|
|
|
|
## YAML formatting
|
|
|
|
The project uses `yamlfix` for consistent YAML formatting:
|
|
|
|
```bash
|
|
pipenv run yamlfix ansible/playbooks/setup.yml
|
|
```
|