One of my goals with my shiny new server is to run bhyve powered vms. Specifically pfsense with a ethernet adapter passthru.

I followed the excellent instructions on how to set up these on pr1ntf.xyz and got it up and running without passthru via iohyve quickly. When doing the passthru however it failed. I’m running 11-Current which has a new version of bhyve requiring bhyve to wire the guest memory to allow for pci passthru. I added this functionality to iohyve in the #86 PR in the iohyve repository.

#Setup Follow the instructions on pr1ntf.xyz to setup and do the initial configuration of iohyve/bhyve

##Download pfsense iso Download and decompress the image then renaming it to reflect that it is uncompressed

iohyve fetch http://files.ie.pfsense.org/mirror/downloads/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz
gunzip /iohyve/ISO/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz
mv /iohyve/ISO/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso /iohyve/ISO/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz/pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz
iohyve isorename pfSense-LiveCD-2.2.6-RELEASE-amd64.iso.gz pfSense-LiveCD-2.2.6-RELEASE-amd64.iso

##Configure pfsense virtualhost See the wiki entry for pci passthru

get Pci device numbering for em0 ethernetcard

pciconf -lv | grep em
em0@pci0:10:0:0:        class=0x020000 card=0x704b103c chip=0x10bc8086 rev=0x06 hdr=0x00
em1@pci0:10:0:1:        class=0x020000 card=0x704b103c chip=0x10bc8086 rev=0x06 hdr=0x00
em2@pci0:9:0:0: class=0x020000 card=0x704b103c chip=0x10bc8086 rev=0x06 hdr=0x00
em3@pci0:9:0:1: class=0x020000 card=0x704b103c chip=0x10bc8086 rev=0x06 hdr=0x00

Set up /boot/loader.conf to allow these to passthru

echo "pptdevs=\"10/0/0 10/0/1 9/0/0 9/0/1\"" >> /boot/loader.conf
reboot

Create and install the pfsense virtual image

iohyve create pfsense 8G
iohyve set pfsense ram=2048mb
iohyve set pfsense cpu=2
iohyve set pfsense pcidev:7=passthru,10/0/0
iohyve set pfsense pcidev:8=passthru,10/0/1
iohyve set pfsense pcidev:9=passthru,9/0/0
iohyve set pfsense pcidev:1=passthru,9/0/1
iohyve set pfsense os=pfsense
iohyve set pfsense bargs="-S -A -H -P"
iohyve install pfsense pfSense-LiveCD-2.2.6-RELEASE-amd64.iso

Connect to the serial console and follow the installation instructions. (Preferably in a another shell session)

iohyve console pfsense

After the “reboot” which isn’t a real reboot start the image

iohyve start pfsense

Then configure as ordinary. Im using the TAP interface as lan.

##Gotchas Passthru all interfaces on the sam physical pci-lane, the host will panic otherwise (eg 9/0/0 and 9/0/1 is needed, only doing 9/0/1 will not work)

#bhyve bhyve is a lightweight hypervisor. It leverages the virtualization extensions of modern cpus. and is BSD-licensed from the ground up. And of course it’s included as standard in freebsd.

#iohyve iohyve is a management tool for bhyve much like iocage is for jails.