Troubleshooting Configuration Not Applied

This guide helps you diagnose and resolve issues when Arctic configuration changes are not being applied to the underlying services (Pegasus, Tempest, NFTables).

Understanding Configuration Flow

When you create or modify services and routes:

Changes are stored in the Arctic database
Subsystem managers detect the changes
Configuration files are regenerated
External services (Pegasus, Tempest) reload their config
NFTables rules are updated

Symptoms

Created a service but traffic is not being routed
Updated routes but old routing still applies
Bandwidth limits not taking effect
MACVLAN interface not created

Diagnosis Steps

1. Force Configuration Sync

First, try triggering a manual sync:

arctic cluster sync

Wait 10-30 seconds for configuration to propagate.

2. Check Subsystem Status

View agent logs for subsystem activity:

journalctl -u arctic-agent | grep -E "(netmgr|fwmgr|tproxymgr|iptunmgr)"

Look for errors or warnings from each manager.

3. Verify Generated Configurations

Check the configuration files were generated:

# TProxy (Pegasus) config
cat /opt/tillered/pegasus/config.json

# IP Tunnel (Tempest) config
cat /opt/tillered/tempest/config.yaml

# NFTables rules
cat /etc/nftables.d/arctic.nft
# or
nft list ruleset | grep -A 20 "table inet arctic"

4. Verify Services Are Running

Check that Pegasus and Tempest are running:

ps aux | grep pegasus
ps aux | grep tempest

If not running, check their logs:

journalctl | grep pegasus
journalctl | grep tempest

Common Issues

Config File Not Updated

Symptoms: Config file has old content or missing entries.

Resolution:

Check agent logs for write errors
Verify disk space: df -h /opt/tillered
Check file permissions: ls -la /opt/tillered/

NFTables Rules Not Applied

Symptoms: nft list ruleset does not show expected rules.

Resolution:

Check if NFTables service is running:
```
systemctl status nftables
```
Manually reload rules:
```
nft -f /etc/nftables.d/arctic.nft
```
Check for syntax errors:
```
nft -c -f /etc/nftables.d/arctic.nft
```

Pegasus Not Reloading

Symptoms: Pegasus config updated but old tunnels still active.

Resolution:

Check Pegasus is running:
```
ps aux | grep pegasus
```
Send reload signal:
```
pkill -HUP pegasus
```

Restart if needed:

# Arctic manages Pegasus lifecycle
systemctl restart arctic-agent

Tempest Not Reloading

Symptoms: Tempest config updated but tunnels not established.

Resolution:

Check Tempest status
Verify WireGuard interface exists:
```
ip link show type wireguard
```
Check Tempest logs for errors

MACVLAN Interface Missing

Symptoms: Service requires interface but it was not created.

Resolution:

Check Network Manager logs:

journalctl -u arctic-agent | grep netmgr

Verify parent interface exists
Check for name conflicts with existing interfaces

Database vs Config Mismatch

Sometimes the database has correct data but config generation fails.

Check Database State

# Services in database
arctic services list -j

# Compare with generated config
cat /opt/tillered/pegasus/config.json | jq

Force Regeneration

Restart the agent to force full config regeneration:

systemctl restart arctic-agent

Timing Issues

Configuration changes may take up to 60 seconds to apply automatically. The subsystem managers run on periodic intervals:

Network Manager: Every 30 seconds
Firewall Manager: Every 30 seconds
TProxy Manager: Every 30 seconds
IP Tunnel Manager: Every 30 seconds

Use arctic cluster sync to trigger immediate processing.

Collecting Debug Information

When reporting issues:

# Agent logs
journalctl -u arctic-agent --since "10 minutes ago"

# Current state
arctic services list -j
arctic routes list --service SERVICE_ID -j

# Generated configs
cat /opt/tillered/pegasus/config.json
cat /opt/tillered/tempest/config.yaml
nft list ruleset

# Process status
ps aux | grep -E "(arctic|pegasus|tempest)"

Configuration Not Applied

On this page