Select Page

Sonos across subnets on Juniper EX2200

Since I have a Juniper EX2200 performing my layer 3 routing for internal traffic and I have Sonos on a separate subnet than some Sonos clients, I needed to allow multicast across subnets. By creating a loopback interface and using it as my PIM rendezvous point, I was able to get my Windows desktop on 10.1.20.0/24 (Wired) find my Sonos speaker on 10.1.50.0/24 (WiFi).

This is pretty much taken directly from the guide here.

First, I leave the default igmp-snooping configuration alone. If yours doesn’t look like this, then you have modified the default configuration:

Then I created a loopback interface to use for multicast:

Next, I added the new loopback interface as passive in my OSPF configuration. The export and the other interfaces were already there for something unrelated to this multicast configuration:

Now I define the IP address of that loopback interface as the rendezvous point and add the necessary VLAN interfaces in sparse mode:

I have subnet 10.1.20.0/24 on vlan.20 for trusted wired, 10.1.40.0/24 on vlan.40 for guest wifi, and 10.1.50.0/24 on vlan.50 for trusted wifi. Since Sonos is on vlan.50, I want all three VLANs to share multicast. You can block multicast from guest wifi to trusted wired via a firewall filter.

Note that because I did not specify a multicast range, the entire 224.0.0.0/4 range is allowed. Sonos only uses SSDP, or 239.255.255.250, so if you want to block all other multicasts, you can limit the range with the following:

I hope this helps!

Ubuntu 18.04 LXD/LXC, ZFS, Docker, and Advanced Networking

Note on 2018.5.20: This is a first draft. This will undergo many revisions.

I used many, many other tutorials, blogs, and Q&A posts to assemble this. Relevant links are sprinkled throughout the guide.

 

This is a complete, step by step tutorial on configuring the following:

  • Ubuntu 18.04 install on a server with two NICs
  • One NIC for host traffic
  • Other NIC for LXC/Docker traffic
  • Plex, Sonarr, Radarr, Jackett in Docker on host
  • rTorrent, ruTorrent, Flood, and OpenVPN nested in Docker in LXC container on host

Topology from a visual perspective:

Ubuntu Bionic Container Topology

Topology from a CLI perspective:

INSTALL UBUNTU

At Filesystem Setup for 120GB SSD:
1. Leave bootloader partition alone
2. I gave 20GB to / partition
3. I gave 60GB to /home partition
4. I left the rest as free space so it can be used later in this guide for our ZFS pool

CREATE USER, ADD TO SUDO GROUP, SWITCH TO USER

adduser will
usermod -aG sudo will
su will

CONFIGURE HOST NETWORK INTERFACE

Get names of network interfaces
ip a

enp1s0f0 is for my host network
enp1s0f1 is for my container network

Edit the existing YAML file
sudo vim /etc/netplan/50-cloud-init.yaml

Apply changes
sudo netplan apply

Schedule script every boot to set the physical interface used for containers to be UP with PROMISC ON. This is necessary right now on Ubuntu 18.04 due to a bug documented here.
mkdir ~/scripts
echo "ip link set enp1s0f1 up && ip link set enp1s0f1 promisc on" > ~/scripts/enp1s0f1.sh
chmod +x ~/scripts/enp1s0f1.sh

The following command will write to crontab so that your script runs as root at boot.
This did not work with only the first sudo, so I threw a bunch of extra sudos in there to make it work. I don’t know if they are all necessary.
sudo crontab -u root -l | { sudo cat; sudo echo "@reboot /home/will/scripts/enp1s0f1.sh"; } | sudo crontab -

 

SET UNUSED INTERFACES DOWN

I have extra physical interfaces that I’m not using right now, so I’m shutting them down.
sudo ip link set en131s0f0 down
sudo ip link set en131s0f1 down

 

INTERFACE TROUBLESHOOTING

You can do the following to remove an IP from an interface. For example, I accidentally assigned 10.1.20.24/24 to enp1s0f1 but I want that IP on enp1s0f0 instead:
sudo ip address del 10.1.20.24/24 dev enp1s0f1

You can do the following to restart an interface:
sudo ip link set enp1s0f1 down && sudo ip link set enp1s0f1 up

If you need to fix 127.0.0.53 being in resolv.conf (this happened to me):
sudo rm -f /etc/resolv.conf

 

CREATE YOUR HOST UPGRADE SCRIPT

Create update script:
cd ~
vim update.sh

Make update script executable:
chmod +x update.sh

Run script to update. You can run this whenever you want to update.
./update.sh

Install whatever common packages you use:
sudo apt install tree unrar ncdu -y

 

INSTALL ZFS AND IMPORT ZFS POOL FROM HBA CONTROLLER

sudo apt install zfsutils-linux -y
zpool import
zpool import tank

Import zpools at boot. Not necessary if you reference the disk by-id when doing initial import as discussed here.
sudo systemctl enable zfs-import-cache

 

INSTALL AND CONFIGURE SAMBA

This section is entirely optional. I like being able to access my entire media ZFS pool from Windows.

Install Samba as described here.
sudo apt install -y samba
sudo nano /etc/samba/smb.conf

Turn SMB sharing on:
sudo zfs set sharesmb=on tank

Create Samba user and define password
sudo pdbedit -a will

Find out who owns the media folder
ls -l /tank/media

I don’t care about UID 1420, but I do want to name the GID 1420 group “media”
sudo groupadd media
sudo groupmod -g 1420 media

Observe changes:

Add myself to the media group and restart Samba
sudo adduser will media
sudo systemctl restart smbd nmbd

 

CONFIGURE LXC PROFILE TO USE MACVLAN

Run lxc without typing sudo every time:
sudo setfacl -m u:will:rwx ~/.config/lxc

Create and edit the YAML file:
lxc profile create lxcnet
lxc profile edit lxcnet

 

CREATE ZFS PARTITION ON LOCAL SSD

Find disk/partition to be used
sudo fdisk -l

Check lxc version. On 18.04 it’s 3.0.0 right now.
lxc info

Start configuration of LXC
sudo lxd init

The outcome of these steps is that the network configuration from the lxcnet profile is copied to the default profile, and the default profile is populated with the ZFS pool information.

You can see this with the following:
lxc profile show default

 

INCREASE FILE AND INODE LIMITS

sudo vim /etc/sysctl.conf

sudo vim /etc/security/limits.conf

Now reboot:
sudo reboot

 

VERIFY HOST NETWORK, LXD NETWORK, AND ZFS

Observe your primary network interface matches what you set in /etc/netplan/00-netcfg.yaml
ip a

Note that a pool has been created with datasets:
sudo zfs list

 

CREATE AN LXC CONTAINER FOR TORRENT/VPN

Note, this is an empty Ubuntu 16.04 container. I’m naming it ‘torrent’
lxc launch ubuntu:xenial torrent

See that the container has started:
lxc list

Look for the MAC address in the container:
lxc config show --expanded torrent

In my case, I see the following:
volatile.eth0.hwaddr: 00:16:3e:e1:65:36

On my DHCP server, I create a new MAC reservation:
Name: torrent
IP: 10.1.20.12
MAC: 00:16:3e:e1:65:36

Enter the torrent container:
lxc exec torrent bash

Remove the dynamic IP so you can get the static one assigned
ip addr flush dev eth0
rm /var/lib/dhcp/dhclient.eth0.leases
dhclient -r; dhclient

You should see the DHCP-assigned static IP address:
ip a

Now exit the container
exit

 

MOUNT HOST DIRECTORY INTO LXC CONTAINER

Stop it if it’s running:
lxc stop torrent

Make it privileged to avoid file ownership issues as noted here:
lxc config set torrent security.privileged true

Mount /tank/downloads to /downloads:
lxc config device add torrent downloads disk source=/tank/downloads path=/downloads

Allow Docker inside LXD container:
lxc config set torrent security.nesting true

 

CREATE NON-ROOT USER AND ASSIGN PRIVILEGES

Start the container and enter it:
lxc start torrent
lxc exec torrent bash

Create user in the container and assign permissions:
adduser will
usermod -aG sudo will
groupadd media
adduser will media
usermod -u 1420 will
groupmod -g 1420 media

Log into user and create command so you can run sudo as documented here:
su will
vim ~/.bashrc

Install the .bashrc:
source ~/.bashrc

 

INSTALL DOCKER IN LXC CONTAINER

Instructions pulled from here.
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt update
sudo apt-get install -y docker-ce

Start it and enable it to start at boot:
sudo systemctl start docker
sudo systemctl enable docker

Let user do docker things without typing sudo:
sudo gpasswd -a will docker
sudo service docker restart
sudo systemctl enable docker

Leave and come back:
exit
su will

 

INSTALL PIA VPN AND TORRENT DOCKER IN LXC

Enter container if you aren’t already in it:
lxc exec torrent bash

Prepare host (LXC container) and create torrent config directory:
mkdir -p ~/torrent/config/openvpn
mkdir ~/torrent/openvpn_all
cd ~/torrent/openvpn_all
wget https://www.privateinternetaccess.com/openvpn/openvpn.zip
sudo apt install unzip
unzip openvpn.zip

Copy only the key, cert, and ovpn file you want to use:
cp *.crt ~/torrent/config/openvpn
cp *.pem ~/torrent/config/openvpn
cp US\ Midwest.ovpn ~/torrent/config/openvpn

Create and run torrent docker:

If you need to add a flag on the fly to a running container, here’s an example:
docker update --restart=always torrent

Troubleshooting Docker:
docker events&
docker start sonarr
docker logs 345fceb9d7589a51c6b2d40c4b84c2e7b4e23463a363a24d7bd47fffd3dec013

Enter a Docker container for troubleshooting:
docker exec -it torrent /bin/bash

 

INSTALL AND SET UP DOCKER ON HOST

Install Docker in Ubuntu 18.04:
curl -fsSL test.docker.com | sh

Create macvlan for hosts
docker network create -d macvlan \
--subnet=10.1.20.0/24 \
--gateway=10.1.20.1 \
-o parent=enp1s0f1 mvdock0

Create and run Plex container (source):
docker run -id \
--name plex \
--network=mvdock0 \
--ip=10.1.20.11 \
-h plex \
-e VERSION=latest \
-e TZ="America/Chicago" \
-e ADVERTISE_IP="http://plex.paw.blue:32400/" \
-e PLEX_UID=1420 -e PLEX_GID=1420 \
-v /tank/plexdata/config:/config \
-v /tank/media/tv:/tv \
-v /tank/media/movies:/movies \
-v /tank/media/education:/education \
-v /tank/transcode:/transcode \
plexinc/pms-docker

Sonarr:
docker run -d \
--name sonarr \
--network=mvdock0 \
--ip=10.1.20.15 \
-p 8989:8989 \
-e NAME_SERVERS=10.1.20.254,10.1.20.253,208.67.222.222,1.1.1.1 \
-e PUID=1420 -e PGID=1420 \
-e TZ=America/Chicago \
-e DEBUG=false \
-v /etc/localtime:/etc/localtime:ro \
-v /home/will/sonarr/config:/config \
-v /tank/downloads:/downloads \
-v /tank/media/tv:/tv \
--restart=always \
linuxserver/sonarr

Radarr:
docker run -d \
--name=radarr \
--network=mvdock0 \
--ip=10.1.20.16 \
-p 7878:7878 \
-e NAME_SERVERS=10.1.20.254,10.1.20.253,208.67.222.222,1.1.1.1 \
-e PGID=1420 -e PUID=1420 \
-e TZ=America/Chicago \
-v /etc/localtime:/etc/localtime:ro \
-v /home/will/radarr/config:/config \
-v /tank/downloads:/downloads \
-v /tank/media/movies:/movies \
--restart=always \
linuxserver/radarr

Jackett:
docker run -d \
--name=jackett \
--network=mvdock0 \
--ip=10.1.20.17 \
-p 9117:9117 \
-e NAME_SERVERS=10.1.20.254,10.1.20.253,208.67.222.222,1.1.1.1 \
-v /home/will/jackett/config:/config \
-v /tank/downloads:/downloads \
-e PGID=1420 -e PUID=1420 \
-e TZ=America/Chicago \
-v /etc/localtime:/etc/localtime:ro \
--restart=always \
linuxserver/jackett

 

DOCKER EXPERIMENTAL THINGS

If you want to use an ipvlan instead of a macvlan in Ubuntu 18.04, you will have to start docker in experimental mode.

Enabling experimental mode:
dockerd --experimental=true

Examples of layer 2 and layer 3 ipvlan networks:

 

EXPLANATIONS ARE IN ORDER

Q: Why don’t you just run the vpn/torrent docker container on the host?
A: I tried to do that with the network=mvdock0 and ip flags, but it wasn’t working. I think it has something to do with the way the VPN is influencing the network connection. By making the vpn/torrent docker container use the “host” network and having the “host” actually be an LXC container, I can still ensure this traffic passes through enp1s0f1 on the physical host.

Q: Why not do all LXC or all Docker? Why mix and match?
A: First, so I could learn both. Second, Docker is so easy to get my applications up and running.
Third, my inspiration for this project was this amazing post by Jason Bayton,  and I loved the idea of hosting LXC containers in ZFS. As it turns out, I only hosted one. But perhaps more soon!

 

SUMMARY

This took me many hours to assemble, as I had very little LXC or Docker experience before setting out on this journey. I’m sure people will point out many, many flaws in this tutorial. Please comment so that I can fix them!

If this guide helped you, please consider a small crypto donation!
BTC: 3LypUnvR1RktXqF4FAA7WT5tCZXxjSDH1f
ETH: 0xc23D1cb22324873Bf7dc4e3FFaD81621Ce50F8EC
LTC: MJb1YqnFSdPkM6vaX1ApnGz96donGZfAU3

FortiGate 90D-POE with FortiAP on 5.4.1

Okay, I feel the need to write this because I just upgraded from FortiOS 5.4.0 to FortiOS 5.4.1 on my FortiGate 90D-POE, and my FortiAP couldn’t handle it. I actually had to completely reconfigure my FortiGate from scratch, as I lost everything but console access whenever I would attempt to import my configuration.

Before we begin:

  • You have two SSID types you can configure
  • I chose “Bridge to FortiAP’s local interface” for my main WiFi network for two reasons:
    • This Fortinet doc says “Bridge mode is more efficient than Tunnel mode, as it uses the CAPWAP tunnel for authentication only”
    • A post in this Reddit thread suggests that tunneled APs use more system resources than a bridged AP
  • I chose “Tunnel to wireless controller” for my guest network because I want to apply different levels of UTM to guests than I do to my own WiFi traffic, and if I bridge both SSIDs to the FortiAP interface, I can’t apply separate UTM policies.

Step 1: Set up your POE interface

Note: I use the term “bridged wireless clients” here for settings that specifically apply to devices connected to my private WiFi network that will have access to my wired devices through a firewall policy. If you create a guest network (tunnel to wireless controller), it will have its own subnet and DHCP server as explained in a later step. Even without a tunneled SSID, you still need a DHCP server on your POE interface for your FortiAP to receive an IP address.

  1. Network > Interfaces
    1. Edit the POE interface where you connect your FortiAP.
    2. Select LAN role, Manual addressing mode, type an IP/Netmask that will act as a gateway for your bridged wireless clients (I chose 172.17.100.1/255.255.255.0), select only CAPWAP for administrative access, and create a DHCP server for your bridged wireless clients.

Step 2: Create SSID(s)

  1. WiFi & Switch Controller > SSID
    1. Create New > SSID
    2. Assign a name for the interface (never visible to public), type WiFi SSID, traffic mode “Local bridge with FortiAP’s Interface”, SSID name (visible to public by default but can be made private), security mode, security mode options, and click OK.
    3. If you wish to create a guest WiFi network, create a new SSID, choose traffic mode “Tunnel to Wireless Controller,” and create a unique IP/Netmask for this subnet, a DHCP server, and finally name your SSID and configure security before clicking OK.

Step 3: Create FortiAP Profile

  1. WiFi & Switch Controller > FortiAP Profiles
    1. Create New, assign a name, **select your model of FortiAP next to platform (DO NOT SKIP THIS STEP)**, choose your radio settings, choose “Select SSIDs,” and select both SSIDs you created in step 2.
    2. You are welcome to limit one or more SSIDs to specific bands if you wish.

Step 4: Assign FortiAP Profile to FortiAP

  1. WiFi & Switch Controller > Managed FortiAPs
    1. By now, your FortiAP should have received an IP address from the DHCP server on the POE interface you configured in step 1. If it still does not have an IP address, wait. Periodically, click Refresh. Eventually, it will get an IP. This should not take more than 5 minutes, but the time can vary by model.
    2. Double click your FortiAP.
    3. Assign a name (optional), Authorize the AP, assign the FortiAP Profile you configured in step 2, and configure any override settings as you wish.
    4. With my FortiGate 90D-POE on firmware v5.4.1-build1064, a Fortinet support representative had me upgrade my FortiAP OS version to FP321C-v5.4-build0339.
    5. Click OK to finish.

Step 5: Create addresses (subnets) to be used with firewall policies

  1. Policy & Objects > Addresses
    1. Create New > Address.
    2. Create a name for your bridged (private) WLAN, put in the same subnet you created in step 1-1-B, and assign it to your POE interface. Click OK.
    3. Create New > Address.
    4. Create a name for your tunneled (guest) WLAN, put in the same subnet you created in step 2-1-C, and assign it to your guest SSID. Click OK.
    5. Create an address for your internal hardware switch if you don’t already have one!

Step 6: Create Firewall policy

  1. Policy & Objects > IPv4 Policy
    1. Create New, assign a name, POE interface as incoming interface, internal hardware switch as outgoing interface, address you created in step 4-1-B as source, address you created in step 4-1-E as destination, service ALL, uncheck NAT if selected, ensure “Enable this policy” is checked and click OK.

      What this policy does: Allows devices on your private, bridged WiFi network to communicate with devices on your internal hardware switch.
  2. Policy & Objects > IPv4 Policy
    1. Create another policy for the reverse direction (internal to WiFi). See screenshot.

Step 7: Allow WiFi subnets access to the internet using firewall policy

  1. WiFi & Switch Controller > SSID
    1. Because you already have a firewall policy that allows devices physically connected to your internal hardware switch access to the internet, you can simply add your POE interface and guest SSID to this policy. Personally, I created a separate policy for my guest WiFi so I can apply more granular control in the future.

      Shown above is my policy for all traffic from my wired devices and private WiFi clients to the internet
      Shown above is my policy for all traffic from my guest WiFi clients to the internet

That’s it! If this helped you, please consider a donation of any amount at all via the PayPal or Bitcoin buttons on the left side of the page. Comments and criticisms are welcome in the comment section.

Google Fiber with Fortigate 90D

Goal:Replace Google Fiber Network Box with your own FortiGate router

Note: I do not know how to get TV working if you use that service. This tutorial covers internet only.

In this tutorial, I am using the following hardware:

  • Google Fiber fiber jack
  • Fortinet FortiGate 90D-POE Firewall

Before we begin:

  • I am using the Web-based Manager in FortiExplorer version 2.6.1083 while connected via the USB management port
  • My internal LAN is already set up to use the default hardware switch utilizing all ten LAN ports and custom DHCP settings for my personal network
  • I configured the internalA interface for my FortiAP 321C and disabled internalB, C, and D.
  • I disabled wan2
  • I have successfully tested the following configuration under firmwares 5.4.0-build1011 and 5.4.1-build1064.

Step 1: Set up your wan1 subinterface

  1. Network > Interfaces
    1. Create New > Interfaces
    2. Assign a name, type VLAN, interface wan1, VLAN ID 2, role WAN, addressing mode DHCP, click OK. I called mine GFIBER.

Step 2: Create IGMP service and set up your firewall policies

    1. Policy & Objects > Services
      1. Create New > Service
      2. Assign IGMP as the name, protocol type IP, Protocol Number 2, click OK.
    2. Policy & Objects > IPv4 Policy
      1. Create New.
      2. Assign a name (I chose “QoS DHCP”), select your internal subnet(s) for incoming interface, select the VLAN interface you created in last step for outgoing interface, source all, destination all, schedule always, service DHCP, action accept, NAT enabled, do not assign any security policies, click OK.
    3. Policy & Objects > IPv4 Policy
      1. Create New.
      2. Assign a name (I chose “QoS IGMP”), select your internal subnet(s) for incoming interface, select the VLAN interface you created in last step for outgoing interface, source all, destination all, schedule always, service IGMP, action accept, NAT enabled, do not assign any security policies, click OK.
    4. Policy & Objects > IPv4 Policy
      1. Create New.
      2. Assign a name (I chose “QoS All Others”), select your internal subnet(s) for incoming interface, select the VLAN interface you created in last step for outgoing interface, source all, destination all, schedule always, service ALL, action accept, NAT enabled, select the security policies you wish to use on the traffic between your LAN and the WAN, click OK.
  1. Make sure “QoS All Others” comes after the first two policies in the list of policies. I put them in the order of IGMP>DHCP>ALL. I’m not sure if the order of the first two matters, but both of them must be processed before the third policy, as you want IGMP and DHCP services to match in their own respective policies before the third policy, which includes them, has a chance to match them. This is because each policy will have a unique QoS bit.

Step 3: Use CLI to assign QoS bits to your three QoS policies

Note: The QoS bits come from here. Each policy has a unique number assigned to it, and they may be different from mine. My IGMP policy is 9, DHCP is 8, and All Others is 2.

  1. Login to the CLI of your FortiGate
    1. Enter the following commands using your own policy numbers as determined by the “show” command after “config firewall policy”:
      config firewall policy
      show
      edit 9
      set vlan-cos-fwd 6
      end
      config firewall policy
      edit 8
      set vlan-cos-fwd 2
      end
      config firewall policy
      edit 2
      set vlan-cos-fwd 3
      end

That’s it! With these firewall rules in place, I get over 900 Mbps down/up. Please leave a comment if this worked for you!