Tillered Arctic

Routing

Understanding traffic routing in Arctic

Routing

This article explains how Arctic routes traffic between peers, the role of TProxy and IP tunnels, and how routing rules are evaluated.

Traffic Flow Overview

When traffic matches an Arctic route, it is directed based on the target peer and protocol:

Traffic destined for different peers is routed through separate tunnels, ensuring proper isolation and routing.

TCP Traffic (TProxy)

TCP traffic uses transparent proxying via Pegasus:

TProxy Benefits

  • Transparency: Original source IP can be preserved (with Transparent Mode)
  • Application agnostic: No client modifications needed
  • Connection awareness: Full TCP state tracking

TProxy Limitations

  • TCP only (UDP, ICMP, etc. not supported)
  • Requires kernel TProxy support
  • Higher latency than IP tunnel for small packets

Transparent Mode

By default, TProxy preserves the source IP only to the exit node. The destination sees the exit node's IP address. When Transparent Mode is enabled on a service, the original source IP is preserved all the way to the destination.

See Transparent Mode for detailed information about how it works, when to use it, and how to configure it.

Non-TCP Traffic (IP Tunnel)

Non-TCP traffic (UDP, ICMP, etc.) uses encrypted tunnels via Tempest:

IP Tunnel Benefits

  • All protocols: UDP, ICMP, and any IP traffic
  • Encryption: Strong encryption for all tunneled traffic
  • Performance: Efficient for bulk traffic

IP Tunnel Limitations

  • Does not preserve source IP (masquerade)
  • Per-peer routing (not per-connection)

Route Evaluation

Routes are evaluated in order of specificity:

Specificity Rules

  1. MACVLAN interface matches first (most specific)
  2. Source + Destination CIDR matches next
  3. Source-only CIDR matches
  4. Destination-only CIDR matches last

Priority Within Category

When specificity is equal, higher priority value wins:

Priority 200: 10.1.0.0/16 -> 192.168.100.0/24  (wins)
Priority 100: 10.0.0.0/8  -> 192.168.100.0/24

Example Evaluation

Given routes:

PrioritySourceDestService
10010.0.0.0/8192.168.0.0/16SVC-A
20010.1.0.0/16192.168.100.0/24SVC-B
100-172.16.0.0/12SVC-C

Traffic from 10.1.2.3 to 192.168.100.50:

  1. Matches both route 1 and route 2 by source
  2. Route 2 is more specific (10.1.0.0/16 vs 10.0.0.0/8)
  3. Route 2 wins -> SVC-B

Traffic from 10.2.3.4 to 192.168.200.50:

  1. Matches route 1 by source
  2. No better match exists
  3. Route 1 wins -> SVC-A

Transport Selection

Traffic is routed based on protocol:

ProtocolHandlerNotes
TCPPegasus (TProxy)Source IP preserved
UDPTempest (IP Tunnel)Encrypted tunnel
ICMPTempest (IP Tunnel)Encrypted tunnel
OtherTempest (IP Tunnel)Any IP protocol

Bandwidth Limiting

Services can have bandwidth limits (QoS):

arctic services create --target-peer PEER_ID --bandwidth-limit 1000

Bandwidth limiting is implemented in Pegasus using token bucket:

  • Applies to TCP traffic through TProxy
  • Measured in Mbps (megabits per second)
  • Fair sharing between multiple connections

MACVLAN Interfaces

When --requires-interface is set:

  1. Arctic creates a MACVLAN interface
  2. Interface name = first 15 chars of service ID
  3. Traffic through this interface is captured
  4. Provides network isolation per service

Use Cases

  • Bind applications to specific service interfaces
  • Isolate traffic by service
  • Assign dedicated IPs per service

Troubleshooting Routing

Traffic Not Being Routed

# Check firewall rules are applied
nft list table inet arctic

# Check Pegasus tunnels
cat /opt/tillered/pegasus/config.json | jq '.tunnels'

# Verify route exists
arctic routes list --service SERVICE_ID

Wrong Destination

  1. Check route priority ordering
  2. Verify CIDR specificity
  3. Use --trace to see route evaluation

Performance Issues

  1. Check bandwidth limits: arctic services get SERVICE_ID
  2. Consider KCP transport for high latency
  3. Monitor Pegasus/Tempest resource usage

See Also