Operations

After enrollment, the agent maintains an outbound tunnel to the regional gateway and forwards relayed HTTP to your backends. This page covers runtime behavior operators and administrators should know.

Tunnel and forwarding

• The gateway sends HTTP requests over the tunnel; the agent forwards them to the backend base URL configured on each service.
• By default the agent handles up to 128 concurrent backend requests per tunnel. Override with:

export XG3_AGENT_MAX_CONCURRENT_BACKEND_REQUESTS=256

Raise this only when your backends can absorb additional parallel load.

• The agent strips X-Xg3-Request-Id before forwarding to your backend (that header is for gateway/client tracing).
• If the agent cannot accept another concurrent request, or a backend forward fails unexpectedly, the gateway may receive a 502 response with Content-Type: application/vnd.xg3.error+json.

Reconnection

If the tunnel drops (network blip, gateway restart, etc.) and the process is not shutting down, the agent:

1. Waits with exponential backoff (starting at 1 second, capped at 120 seconds). After a connected tunnel drops, the next wait starts again at 1 second; failed connect attempts keep escalating.
2. Reloads the certificate from disk if present
3. Opens a new tunnel

The gateway detects abrupt network loss (for example a cable unplugged) within tens of seconds and releases the old tunnel, so reconnect after a brief outage is more reliable than waiting for a long idle timeout. No operator action is required.

Cooperative shutdown (Ctrl+C or Windows service stop) exits cleanly without endless reconnect loops.

Upgrading from the console

When a newer agent release is available, operators with the appropriate permission can upgrade from the agent detail page while the agent is online. See also Agents and services — Upgrade an agent.

What happens

1. Open the agent in the console (tunnel must be connected).
2. Click Upgrade to vX.X.X when shown, then confirm.
3. The agent downloads the new binary from your regional gateway (https://{region}.xg3.io), checks its integrity, and runs a short health canary against the gateway.
4. If the canary fails, the current agent stays online and continues serving traffic. Retry later with Upgrade again while online.
5. If the canary succeeds, the agent briefly drains in-flight backend requests, swaps in the new binary, and restarts the Windows service.
6. After reconnect, the Agent version on the detail page updates. Event Viewer records Agent Upgrade Requested, then Agent Upgraded or Agent Upgrade Failed.

Requirements:

• Outbound HTTPS from the agent host to the regional gateway only (same egress as normal operation — no extra release CDN)
• Agent running as a Windows service (recommended)
• Brief service restart only after a successful canary

The attempt is one-shot. If it does not complete, check Event Viewer and {data-dir}/logs/ on the host, then re-click Upgrade while the tunnel is connected.

Previous binary retained

After a successful swap, the agent keeps the prior executable next to the running binary as xg3agent.previous.exe (for the default install: C:\Program Files\xgress3\xg3agent.previous.exe). That file remains until the next successful upgrade overwrites it.

Roll back to the previous version

Use this if you need to return to the binary that was running before the last successful seamless upgrade (for example after a regression). This is a host-side restore; there is no console “downgrade” button.

1. On the agent host, open an elevated PowerShell or Command Prompt.
2. Stop the service:

Stop-Service Xg3Agent

3. Confirm the previous binary exists (default install path shown):

Test-Path "C:\Program Files\xgress3\xg3agent.previous.exe"

If this returns $false, there is no retained previous binary (for example the agent was never seamlessly upgraded on this host, or the install directory was cleaned). Reinstall a known-good release from Install and run instead.

4. Replace the current binary with the previous one. Optionally keep a copy of the version you are leaving:

$install = "C:\Program Files\xgress3"
Copy-Item "$install\xg3agent.exe" "$install\xg3agent.rolled-forward.exe" -Force
Copy-Item "$install\xg3agent.previous.exe" "$install\xg3agent.exe" -Force

5. Start the service:

Start-Service Xg3Agent

6. In the console, open the agent detail page and confirm the tunnel reconnects and Agent version shows the restored build. Event Viewer may show tunnel disconnect/reconnect events around the restart.

If the service fails to start after restore, check Event Viewer on Windows and {data-dir}/logs/, and verify the service ImagePath still points at xg3agent.exe under your install directory (see Install and run).

TIP

Rolling back does not remove enrollment or certificates under the data directory. Only the executable is replaced.

Certificate renewal

While the tunnel runs, the agent periodically checks certificate expiry. When within three days of expiry it requests renewal over the tunnel, persists the new certificate, and opens a new tunnel before closing the old one (make-before-break).

Ensure the agent host clock is accurate (NTP). Severely skewed clocks can affect renewal timing.

Exit codes

Code	Meaning	Action
0	Clean shutdown	Safe to restart
1	Unexpected error	Inspect {data-dir}/logs/ and RUST_LOG; fix and restart
2	Stand down (renewal limit exceeded)	Do not restart automatically — contact support

FAQ

Which hostname and port does the agent use?

The agent uses one gateway base URL for both enrollment REST and the tunnel:

• Production: https://{region}.xg3.io unless XG3_GATEWAY is set
• Connections use the host and port from that URL (https defaults to port 443)

TLS SNI uses the hostname from the URL, not the port.

What is the reconnect strategy?

Described above: exponential backoff up to 120 seconds, resetting to 1 second after a drop that followed a successful connection, reload cert, retry tunnel. Enrollment and initial CSR use separate retry logic inside the agent.

Does the agent validate JWT or service keys?

No. Caller authentication happens at the regional gateway before traffic is relayed. The agent only forwards authorized requests.

Can one agent serve multiple services?

Yes. Create multiple services in the console that point at the same agent with different service IDs and backend URLs.

When does ingress fail even though the agent is running?

Ingress requires an active tunnel. Check the agent detail page in the console for connection status. Also verify the service is enabled and the backend URL is reachable from the agent host. If the gateway returns InternalBackendAccessDenied, enable Allow internal backend targets on the agent or use a public backend URL — see Deny codes.

Logging

The agent writes logs to {data-dir}/logs/ (see Install and run — Log files). Set verbosity with RUST_LOG.

# Default (info) — logs only in {data-dir}/logs/
xg3agent --region au

# Tunnel and enrollment diagnostics
$env:RUST_LOG = "xg3_agent=debug"
xg3agent --region au

On interactive Windows CLI, log output does not appear on the console; use the log files (or run with --no-interactive if you need warnings on stderr). Windows services should set RUST_LOG in the service environment; logs remain under the service --data-dir.

When investigating exit code 1, start with today's log file under logs/ before increasing RUST_LOG.

Related documentation

• Transport and limits — parallel downloads, agent concurrency, ingress errors
• Install and run
• Agents and services — Upgrade an agent
• Event Viewer — upgrade requested / upgraded / failed
• Ingress authentication
• Troubleshooting
• Deny codes — ingress 502 errors including internal backend denies