ZeroTier Flow Rules

JaredBusch

Is anyone using ZeroTier flow rules yet?

I tried to set up a rule to allow ping, ssh, and RDP but broke ZeroTier.

I'm sure this is a stupid user error, but I'm tired and gave up for now.

This is the default rule set.

The manual: https://www.zerotier.com/manual/#3

black3dynamite

Here's what I have so far.

# Whitelist only IPv4 (/ARP) and IPv6 traffic and allow only ZeroTier-assigned IP addresses
drop                      # drop cannot be overridden by capabilities
  not ethertype ipv4      # frame is not ipv4
  and not ethertype arp   # AND is not ARP
  and not ethertype ipv6  # AND is not ipv6
#  or not chr ipauth      # OR IP addresses are not authenticated (1.2.0+ only!)
;

# Allow SSH and RDP by allowing all TCP packets (including SYN/!ACK) to these ports
accept
  ipprotocol tcp
  and dport 22 or dport 3389
;

# Drop TCP SYN,!ACK packets (new connections) not explicitly whitelisted above
break                     # break can be overridden by a capability
  chr tcp_syn             # TCP SYN (TCP flags will never match non-TCP packets)
  and not chr tcp_ack     # AND not TCP ACK
;

# Accept other packets
accept;

black3dynamite

Add "and" before "not ethertype arp" and "not ethertype ipv6"

JaredBusch

@black3dynamite said in ZeroTier Flow Rules:

Add "and" before "not ethertype arp" and "not ethertype ipv6"

Yes, I have that as it is the default rule set. The above was a typo when I redid a the default after breaking things..

That is not my question though.

FYI, this is the full default rule set with comments on a new ZT network.

#
# This is a default rule set that allows IPv4 and IPv6 traffic but otherwise
# behaves like a standard Ethernet switch.
#
# Please keep in mind that ZeroTier versions prior to 1.2.0 do NOT support advanced
# network rules.
#
# Since both senders and receivers enforce rules, you will get the following
# behavior in a network with both old and new versions:
#
# (old: 1.1.14 and older, new: 1.2.0 and newer)
#
# old <--> old: No rules are honored.
# old <--> new: Rules work but are only enforced by new side. Tags will NOT work, and
#               capabilities will only work if assigned to the new side.
# new <--> new: Full rules engine support including tags and capabilities.
#
# We recommend upgrading all your devices to 1.2.0 as soon as convenient. Version
# 1.2.0 also includes a significantly improved software update mechanism that is
# turned on by default on Mac and Windows. (Linux and mobile are typically kept up
# to date using package/app management.)
#

#
# Allow only IPv4, IPv4 ARP, and IPv6 Ethernet frames.
#
drop
	not ethertype ipv4
	and not ethertype arp
	and not ethertype ipv6
;

#
# Uncomment to drop non-ZeroTier issued and managed IP addresses.
#
# This prevents IP spoofing but also blocks manual IP management at the OS level and
# bridging unless special rules to exempt certain hosts or traffic are added before
# this rule.
#
#drop
#	not chr ipauth
#;

# Accept anything else. This is required since default is 'drop'.
accept;

manxam

This is strictly a guess by looking through their documentation as I do not have a ZT node here to test.

drop                      # drop cannot be overridden by capabilities
  not ethertype ipv4      # frame is not ipv4
  and not ethertype arp   # AND is not ARP
  and not ethertype ipv6  # AND is not ipv6

accept			  # but accept
  ipprotocol rdp	  # RDP (not sure if this is both TCP AND UDP)

accept			  # and accept
  ipprotocol icmp     	  # ICMP

accept;			  # This is required since default is 'drop'.

black3dynamite

@manxam said in ZeroTier Flow Rules:

This is strictly a guess by looking through their documentation as I do not have a ZT node here to test.

drop                      # drop cannot be overridden by capabilities
  not ethertype ipv4      # frame is not ipv4
  and not ethertype arp   # AND is not ARP
  and not ethertype ipv6  # AND is not ipv6

accept			  # but accept
  ipprotocol rdp	  # RDP (not sure if this is both TCP AND UDP)

accept			  # and accept
  ipprotocol icmp     	  # ICMP

accept;			  # This is required since default is 'drop'.

Has soon as you add accept; those other accept rules isn't necessary.

JaredBusch

@black3dynamite said in ZeroTier Flow Rules:

@manxam said in ZeroTier Flow Rules:
This is strictly a guess by looking through their documentation as I do not have a ZT node here to test.
drop                      # drop cannot be overridden by capabilities
  not ethertype ipv4      # frame is not ipv4
  and not ethertype arp   # AND is not ARP
  and not ethertype ipv6  # AND is not ipv6

accept			  # but accept
  ipprotocol rdp	  # RDP (not sure if this is both TCP AND UDP)

accept			  # and accept
  ipprotocol icmp     	  # ICMP

accept;			  # This is required since default is 'drop'.
Has soon as you add accept; those other accept rules isn't necessary.

Most examples have a break rule before the final accept.

manxam

That runs counterintuitive to their site and confused me as well.
They have a sample showing basic layout with the BLOCK at first, ACCEPT after and explain what they're allowing, and then at the end they have ACCEPT;

They then go on to say that this blocks X, but allows Y. When, with that final ACCEPT, you'd think it would also allow Z.

I dunno..

EDIT : maybe I missed a "break"?

manxam

I am curious to see what works for @JaredBusch as I could see this coming in handy very soon...

black3dynamite

Here's what I have so far.

# Whitelist only IPv4 (/ARP) and IPv6 traffic and allow only ZeroTier-assigned IP addresses
drop                      # drop cannot be overridden by capabilities
  not ethertype ipv4      # frame is not ipv4
  and not ethertype arp   # AND is not ARP
  and not ethertype ipv6  # AND is not ipv6
#  or not chr ipauth      # OR IP addresses are not authenticated (1.2.0+ only!)
;

# Allow SSH and RDP by allowing all TCP packets (including SYN/!ACK) to these ports
accept
  ipprotocol tcp
  and dport 22 or dport 3389
;

# Drop TCP SYN,!ACK packets (new connections) not explicitly whitelisted above
break                     # break can be overridden by a capability
  chr tcp_syn             # TCP SYN (TCP flags will never match non-TCP packets)
  and not chr tcp_ack     # AND not TCP ACK
;

# Accept other packets
accept;

JaredBusch

@black3dynamite need ICMP also. I thought that was

accept 
  icmp 4 -1
;

but it did not work.
That or I broke something else at the time. Iw ill be back on this shortly myself.

black3dynamite

@JaredBusch said in ZeroTier Flow Rules:

@black3dynamite need ICMP also. I thought that was
accept 
  icmp 4 -1
;
but it did not work.
That or I broke something else at the time. Iw ill be back on this shortly myself.

I was still able to ping without adding icmp.

black3dynamite

https://www.ibm.com/support/knowledgecenter/en/SS42VS_7.3.1/com.ibm.qradar.doc/c_DefAppCfg_guide_ICMP_intro.html

icmp 0 -1 and icmp 8 -1

JaredBusch

@black3dynamite said in ZeroTier Flow Rules:

https://www.ibm.com/support/knowledgecenter/en/SS42VS_7.3.1/com.ibm.qradar.doc/c_DefAppCfg_guide_ICMP_intro.html

icmp 0 -1 and icmp 8 -1

ok I have RDP but no ping to a desktop.
but I can ssh and ping a server.
so likely my lack of ping is the windows firewall.

So all working.
without the icmp rule.

this looks all but identical to what I setup last night, but could not get working.
so I'm going with typo or something that was in the rules parser, but not right.

JaredBusch

Here is my updated rule set that I use on my personal ZT network.

# Whitelist only IPv4 (/ARP) and IPv6 traffic and allow only ZeroTier-assigned IP addresses
drop                      # drop cannot be overridden by capabilities
  not ethertype ipv4      # frame is not ipv4
  and not ethertype arp   # AND is not ARP
  and not ethertype ipv6  # AND is not ipv6
#  or not chr ipauth      # OR IP addresses are not authenticated (1.2.0+ only!)
;

# Allow SSH, SMTP, HTTP, HTTPS, and Cockpit by allowing all TCP packets (including SYN/!ACK) to these ports
accept
  ipprotocol tcp
  and dport 22 or dport 25 or dport 80 or dport 443 or dport 9090
;

# Drop TCP SYN,!ACK packets (new connections) not explicitly whitelisted above
break                     # break can be overridden by a capability
  chr tcp_syn             # TCP SYN (TCP flags will never match non-TCP packets)
  and not chr tcp_ack     # AND not TCP ACK
;

# Accept other packets
accept;

ICantIT

Sorry about dragging this old topic back but, it is probably the most relevant to what I'm looking for.

I have been trying to get the ZeroTier FlowRules to work but must be doing something wrong. My ruleset is very close to what @JaredBusch has but, the ZeroTier nodes don't work as expected.

When I leave the final accept statement, ZeroTier passes all traffic. When I comment out that last accept all traffic stops.

# Allow only IPv4, IPv4 ARP
#
drop
	not ethertype ipv4
	and not ethertype arp
# Drop IPv6 Ethernet frames.
#	and not ethertype ipv6
;
#
#
# Uncomment to drop non-ZeroTier issued and managed IP addresses.
#
# This prevents IP spoofing but also blocks manual IP management at the OS level and
# bridging unless special rules to exempt certain hosts or traffic are added before
# this rule.
#
#drop
#	not chr ipauth
#;
accept
	ipprotocol tcp
		and dport 80
;
# Accept anything else. This is required since default is 'drop'.
accept;

Any help on what I'm doing wrong will be greatly appreciated.