Homelab 2021

Over the last 12 months I have been working almost exclusively on Palo Alto firewalls concentrating on a hybrid cloud WAN topology. Initially we had a lab not far up the motorway from my house where all the physical kit was stacked and arranged into hub and spoke sites. We even emulated AWS Direct Connects so that we could extend the design into the cloud. All very nice, but eventually this hardware needed to be moved to their various sites and the lab was disbanded.

The network design however did not remain static and went through a re-imagining we also had instances where we need to ship complete configs to remote sites on other continents. These requirements led me to EVE-NG which has been pivotal in providing some great results. The machine I ran these labs on was equipped with Devils Canyon i5-4690k, and although only possessing 4 threads it behaved admirably. That said ,larger labs needed to be started incrementally to help it cope.

So around Christmas time I began thinking about buying some huge multi core PC to run my labs on. Luckily this coincided with the release of the AMD 5900x/ 5950x processors. I waited patiently for the release date, read the forum posts about UK stock levels, received the stock alerts, managed to miss the in stock items and then shortly saw them all on ebay at incredibly inflated prices. Not to be deterred I started looking at AMD Epyc single socket processors and patiently waited for the April release of the Milan units. That release date slipped, but as luck would have it stocks of 5950x appeared so I jumped on the ‘Buy’ button as quickly as I could.

A couple days passed and finally all the parts arrived in the post:

  • AMD Ryzen 9 5950x
  • MSI MAG X570 Tomahawk
  • Crucial Balistix 32GB

This was the first ATX build I’ve done in several years meaning that all my cases are m-ITX. Luckily I had an old Antec Three Hundred case available housing my old Core2Quad Q9650, the build could begin.

I had read some threads warning about he size of NH-D15 especially that it may foul the RAM DIMMS and that units of a particular height should be purchased. I duly ignored the warnings only to find that the second CPU FAN did in fact come into contact with the RAM. Thankfully the fan can be mounted higher up the heat sink and despite being proud of the top, the side panel manages to close on top of it without issue!

Virtualisation

In the past I have run EVE-NG using vmware Player under Linux and this has worked well, however this has a limitation of provisioning a maximum of 16 vCPUs to a VM, half of what I have available. I took a look at ESXi (having last used a v4.1 deployment!) but the free license has the same limitation. That said the Professional version of EVE-NG now supports clustering, so you could run two parallel EVE-NG instances to get around this limitation but there will be two underlying OS’s running which strikes me a wasteful.

After this cursory evaluation of the vmware product line I have settled on using Proxmox for running my EVE-NG instance and with now license nonsense I should be able to get the CPU up to temperature quite quickly!

Only one step into the install and it grinds to a halt with some strange graphical artifact drawn across the top of the screen. Editing the grub boot loader, and adding nomodeset to kernel boot options got me a little further.

A few more seconds elapse and now encounter another problem which I can’t easily mitigate. The Realtek 8125B onboard Ethernet card is not supported out of the box with Proxmox and it looks like the installer will not continue.

A few days elapse and a ‘new’ NIC turns up, a Sun 375-348101 Quad Ethernet PCIe. I don’t run a 2.5Gbps network at home so I am not too concerned about not using the Realtek port.

This time the install process now picks up an DHCP lease but then stalls again. A bit of googling suggests an Xorg issue. In my case it appears the Xorg nouveau driver is being loaded which doesn’t support the RTX card I have installed. Instead the ‘fbdev’ driver must be used. Switch to tty3, create a new xorg.conf file, replace the line ‘Driver: "nouveau" ‘ with ‘Driver: "fbdev" ‘.

# xorg -configure
# mv /root/xorg.conf/new /etc/X11/xorg.conf
# vi /etc/X11/xorg.conf
# startx

The install now completes and I can boot into the new Proxmox environment, just remembering to add the ‘nomodeset’ parameter at the boot loader. Once booted edit /etc/default/grub and configure the line:

GRUB_CMDLINE_LINUX_DEFAULT="quiet nomodeset"

Then run update-grub to permanently change the grub.cfg file.

Finally access the webgui via the URL mentioned on the CLI:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.

Up ↑

%d bloggers like this: