~ read.

pfsense on bhyve

One of my goals with my shiny new server is to run bhyve powered vms. Specifically pfsense with a ethernet adapter passthru.

I followed the excellent instructions on how to set up these on pr1ntf.xyz and got it up and running without passthru via iohyve quickly. When doing the passthru however it failed. I'm running 11-Current which has a new version of bhyve requiring bhyve to wire the guest memory to allow for pci passthru.
I added this functionality to iohyve in the #86 PR in the iohyve repository.

Setup

Follow the instructions on pr1ntf.xyz to setup and do the initial configuration of iohyve/bhyve

Download pfsense iso

Download and decompress the image then renaming it to reflect that it is uncompressed

iohyve fetch http://files.ie.pfsense.org/mirror/downloads/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz  
gunzip /iohyve/ISO/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz  
mv /iohyve/ISO/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso /iohyve/ISO/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz  
iohyve isorename pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz pfSense-LiveCD-2.2.6-RELEASE-amd64.iso  

Configure pfsense virtualhost

See the wiki entry for pci passthru

get Pci device numbering for em0 ethernetcard

pciconf -lv | grep em  
em0@pci0:10:0:0:        class=0x020000 card=0x704b103c chip=0x10bc8086 rev=0x06 hdr=0x00  
em1@pci0:10:0:1:        class=0x020000 card=0x704b103c chip=0x10bc8086 rev=0x06 hdr=0x00  
em2@pci0:9:0:0: class=0x020000 card=0x704b103c chip=0x10bc8086 rev=0x06 hdr=0x00  
em3@pci0:9:0:1: class=0x020000 card=0x704b103c chip=0x10bc8086 rev=0x06 hdr=0x00  

Set up /boot/loader.conf to allow these to passthru

echo "pptdevs=\"10/0/0 10/0/1 9/0/0 9/0/1\"" >> /boot/loader.conf  
reboot  

Create and install the pfsense virtual image

iohyve create pfsense 8G  
iohyve set pfsense ram=2048mb  
iohyve set pfsense cpu=2  
iohyve set pfsense pcidev:7=passthru,10/0/0  
iohyve set pfsense pcidev:8=passthru,10/0/1  
iohyve set pfsense pcidev:9=passthru,9/0/0  
iohyve set pfsense pcidev:1=passthru,9/0/1  
iohyve set pfsense os=pfsense  
iohyve set pfsense bargs="-S -A -H -P"  
iohyve install pfsense pfSense-LiveCD-2.2.6-RELEASE-amd64.iso  

Connect to the serial console and follow the installation instructions. (Preferably in a another shell session)

iohyve console pfsense  

After the "reboot" which isn't a real reboot start the image

iohyve start pfsense  

Then configure as ordinary. Im using the TAP interface as lan.

Gotchas

Passthru all interfaces on the sam physical pci-lane, the host will panic otherwise (eg 9/0/0 and 9/0/1 is needed, only doing 9/0/1 will not work)

bhyve

bhyve is a lightweight hypervisor. It leverages the virtualization extensions of modern cpus. and is BSD-licensed from the ground up.
And of course it's included as standard in freebsd.

iohyve

iohyve is a management tool for bhyve much like iocage is for jails.

comments powered by Disqus