Troubleshooting Cloud Firewall

Modified: 26 Sep 2019 21:20 UTC

This document provides basic instruction on the process of troubleshooting the Cloud Firewall (FWAPI) in Triton.

Firewall API architecture and troubleshooting

This document is intended to give an overview of the Firewall API architecture: its various pieces, and how to interact and troubleshoot them.

Components

These are the components that propagate changes to firewalls in Triton:

On the compute node

Interaction with ipfilter

fwadm controls its own ipf instance, which is separate from the one visible in the zone. The zone can't see or modify the GZ-controlled ipf rules. When filtering, the rulesets are evaluated like so:

Add / update / delete rule workflow

Modifying a firewall rule goes to the Firewall API directly, which pushes the change to the firewaller agents on the compute nodes:

Provision / update / destroy instance workflow

Modifying any of the instance parameters that can affect firewalls (tags and NICs) causes an update to be posted to FWAPI (in the fwapi.update workflow task), which is then pushed to the firewaller agents on the compute nodes:

Troubleshooting

Is the firewall enabled? Is firewall_enabled set (VMAPI)?

Via VMAPI:

$ sdc-vmapi /vms/<uuid> | json -Ha firewall_enabled
true

On the compute node:

# vmadm get <uuid> | json firewall_enabled
true

# fwadm status f7dc262f-dd0b-475f-9461-aefb0b6cb2b1
running

What instances does a rule apply to?

Via FWAPI (in the FWAPI zone):

# fwapi vms <uuid> | json -a uuid
ae513916-4ea3-cece-dfc0-e28b51cb74fa

On the compute node:

# fwadm vms <uuid>
ae513916-4ea3-cece-dfc0-e28b51cb74fa

What rules apply to an instance?

Via FWAPI:

# fwapi rules <uuid>
[
  {
    "enabled": true,
    "owner_uuid": "e6fcbc64-3f32-11e2-a144-bf78292e9628",
    "rule": "FROM any TO vm ae513916-4ea3-cece-dfc0-e28b51cb74fa ALLOW udp PORT 54",
    "uuid": "c21912b0-e2b7-471d-aee2-a024d82621ba",
    "version": "1386926482712.093012"
  },
  {
    "enabled": true,
    "owner_uuid": "e6fcbc64-3f32-11e2-a144-bf78292e9628",
    "rule": "FROM any TO vm ae513916-4ea3-cece-dfc0-e28b51cb74fa ALLOW tcp PORT 80",
    "uuid": "e449bf0a-a3f6-4b2f-a426-73bd16b8219a",
    "version": "1386926716940.093012"
  },
  {
    "enabled": true,
    "global": true,
    "rule": "FROM any TO all vms ALLOW icmp TYPE 8 CODE 0",
    "uuid": "27775f65-d377-4979-9c7c-63c9d4f98525",
    "version": "1386743867795.003240"
  }
]

On the compute node:

# fwadm rules <uuid>
UUID                                 ENABLED RULE
27775f65-d377-4979-9c7c-63c9d4f98525 true    FROM any TO all vms ALLOW icmp TYPE 8 CODE 0
e449bf0a-a3f6-4b2f-a426-73bd16b8219a true    FROM any TO vm ae513916-4ea3-cece-dfc0-e28b51cb74fa ALLOW udp PORT 54
c21912b0-e2b7-471d-aee2-a024d82621ba true    FROM any TO vm ae513916-4ea3-cece-dfc0-e28b51cb74fa ALLOW tcp PORT 80

What rules are being hit on a instance?

On the compute node:

# fwadm stats <uuid>
0 pass out quick proto tcp from any to any flags S/SA keep state
0 pass out proto tcp from any to any
0 pass out proto udp from any to any keep state
0 pass out quick proto icmp from any to any keep state
0 pass out proto icmp from any to any
0 pass in quick proto icmp from any to any icmp-type routerad
0 pass in quick proto tcp from any to any port = smtp
0 pass in quick proto icmp from any to any icmp-type echo code 0
130 block in all

What remote instances are stored on a compute node?

On the compute node:

# fwadm list-rvms | json -a uuid
6b1db4be-63bb-11e3-bf59-db53ba2a8cef

Something has gone wrong - what information should I gather?

Logs
Rule and remote instance data

The output of:

Other data to gather

If appropriate: