IPv6 for SmartOS guest VMs on Hetzner hosts

This Hetzner wiki page covers how to configure a SmartOS installation at Hetzner including how to route an IPv4 subnet to guests via the global zone.

Matters are a little bit more involved for IPv6, because

• vmadm and friends don't support IPv6, so you have to configure guests manually
• Hetzner network won't talk to the virtual MAC addresses generated by SmartOS

I ended up doing something similar to the IPv4 setup, but all inside the one /64 IPv6 subnet that came with the server. The global zone creates a bridge for the IPv6 traffic, and guests route via that instead of the upstream gateway at Hetzner. Hetzner provide guidance for a similar approach for Linux VM servers.

# setup IPv6 on the physical nic
ifconfig rge0 inet6 plumb 
ifconfig rge0 inet6 addif ${IP6SUBNET}::1/${IP6PREFIX} up
route add -inet6 ${IP6GATEWAY} ${IP6SUBNET}::1 -interface
route add -inet6 default ${IP6GATEWAY}

# create a bridge and an etherstub
dladm create-bridge six
dladm create-etherstub stub2
dladm set-linkprop -p mtu=1500 stub2

# bridge the physical nic and the stub
dladm add-bridge -l stub2 -l rge0 six

# add a vnic to the stub in the global zone with an adjacent address
dladm create-vnic -l stub2 vnic2
ifconfig vnic2 inet6 plumb
ifconfig vnic2 inet6 addif ${IP6SUBNET}::2/${IP6PREFIX} up

# forward IPv6 packets to/from outside world
svcadm enable ipv6-forwarding

• Guests set their gateway to the vnic2 address.
• FreeBSD guests (seemingly others too) need to disable Duplicate Address Detection with a judicious net.inet6.ip6.dad_count=0 sysctl tweak.
• I wrapped the script up as a SMF service based on this Gist.
• I am unsure why the vnic is necessary, but the IPv6 alias on the physical NIC wasn't visible on the bridge to guests.
• This page was a useful start for dladm bridging.
• My first time out with dladm - disclaimers apply.

Update (20140814)

To use IPv6 in the guest OS it's important that SmartOS knows to put the guest's NIC in the bridge. This is done through the nic_tag attribute, which needs to match the etherstub configured in the global zone. Further, allow_ip_spoofing must also be set so that SmartOS will deliver traffic outside of any IPv4 configuration.

# vmadm get xxxx-yyyy-zzzz | json nics
[
      {
    "interface": "net0",
    ...
    "nic_tag": "stub2",
    "allow_ip_spoofing": true,
    ...
  }
]

The SmartOS wiki has some good information about setting up IPv6 in a SmartOS guest OS.