10Gb Networking, Unraid, and Improved Workflow
/Previously, we were using a peer-to-peer setup to achieve 10Gig transfers. Now, we have added a UniFi US-16-XG 10Gb Switch into the mix so that more clients can now take advantage of the extra available bandwidth. It’s been an entire year since we last tried to achieve 10Gig speeds and now we are finally here. Let’s take a dive into the entirety of the setup.
List of Components
I can’t think of a better way to start this outline without first listing all of the various devices and parts that are being used to make this happen. We will get into greater detail as you scroll/skim downwards.
Aqtion AQN-107 (client)
CalDigit 10GbE Thunderbolt 3 adapter (client)
Intel X540 T2 10GbE (server)
The Setup
Here is a diagram roughly depicting the network and systems layout of SPX Labs. This does not include items like VLAN(s) and such. The purpose of this diagram is to simply paint an image of the lab and how it is laid out.
The blue line indicates 1Gb/s while the red line indicates 10Gb/s.
Detailed Setup and Configurations
US-16-150 and US-16-XG Linking
Quick list of personally verified SFP(+) modules I know that work with both the Ubiquiti 1G and 10Gb switch.
Now let’s dive a bit further into how to get the 1Gb 16-150 switch to communicate with the XG switch, well, this is what I had to do anyway.
Connect both switches together via ethernet
Adopt the US-16-XG
Insert an SFP or SFP+ module (must be the same on both ends) into the appropriate ports
Manually configure the interconnecting ports to 1000Mbp/s. Look for an option to change the default Auto Link Negotiation. This must be done on both switches.
???
Profit
Storage Configuration
This is arguably the most important item to note for all of this to function correctly. All of the systems connected to the 16-XG use NVME M.2 drives either as main storage or as cache. Such is the case for my Unraid server, it uses dual NVME drives as cache in a RAID 1 configuration. If you would like to know more about how this is achieved, you can read about it here.
Unraid Configuration
Unraid version 6.6.6
My Unraid server is using an Intel X540 T2 10Gb Ethernet NIC to connect to the ethernet ports on the 16-XG via CAT6a cables. Unraid should automatically be able to use this NIC without any effort from the user. I literally just slapped it in my server and I was good to go.
How to configure Unraid to support 10 gigabit speeds? You must set the MTU / Jumbo Frames to 9000. In my case I set it to 9014 because in Windows Server, there is a drop down selection that explicitly let’s you select 9014 for the X540 T2. 9000 could very well be the appropriate setting and does work fine, however, I have zero information to prove why it should be 9000 instead of 9014. Hopefully some kind soul will tell me in the comments or directly one day.
Windows 10 Configuration
Windows 10 1708 - 1903
The only thing I did on the Windows side of the house was simply add the Aqtion AQN-107 NIC into one of the available PCI-E slots in my gaming systems, aka “The Truncator”. From there, I changed out the CAT5e cable for a CAT6a cable. Now, I’ve read and seen other people saying they needed to set the MTU/Jumbo Frames to 9014, I however did not need to do anything beyond the aforementioned details.
MacBook Pro Configuration
2018 MacBook Pro High Sierra +
Here I am using the CalDigit Thunderbolt 3 10Gb Ethernet adapter. Plug it into the Thunderbolt 3(TB3) port on the MacBook Pro and you are good to go. It should be noted that this device does not work with USB-C ports. It will only work with TB3 ports. Also for some weird reason I feel obligated to point out that I am using a CAT6a cable with this adapter as well.
If AirPlay is something you use regularly, then you will love to hear that once you use the ethernet adapter, you will lose AirPlay support. Hmmmm, or maybe that is just in my setup. It was really hard to confirm this one from other sources but I definitely do not see my MBP in AirPlay when connected via ethernet.
Workflow
So how does all of this improve workflow? Time. The amount of time that I would spend waiting for file transfers has be drastically reduced. If I record some game play on my gaming rig and the video is hours long it can create some really big files. I can now transfer those videos over to the Unraid server in a matter of seconds. Need some archived footage from a previous video? Boom there it is, ready to be copied over in a few seconds.
One of the things I use to do in the past after I finished creating a video, was to take that video and transfer it to my Unraid server and then upload it to YouTube, when my MacBook Pro was on WiFi only. I did this because there a ton of WiFi devices on my network and more often than not someone is streaming or gaming on WiFi. While it doesn’t have a huge impact on me because my UniFi AP Pro is resilient AF, sometimes I just straight prefer the more consistent upload speed of copper based connections. Secondly, after I’m done making a video. I typically keep an copy on my Unraid server as a backup or for use later and I can retrieve that content a bit more speedily.
So while this may not technically be workflow related, it is most definitely a workflow performance improvement more than anything.
Still Here?
Here is some of this information in video format, if you prefer.
Why did I buy the Ubiquiti UniFi US-16-XG switch?
Short answer: I felt like it had an overall good value position for the price compared to some other products.
Long answer: While I found many switches on the cheap, most where limited to 2 Ethernet ports or 4 SFP+ ports. I have exactly 4 ethernet based devices so I was kind of in a bind. Now, I could have bought some Mellanox SFP+ NIC’s on the cheap and a MikroTik SFP+ 4 port switch probably cheaper than the XG switch itself but then that would have left me with extra hardware laying around. I already have a bunch of components sitting around collecting dust and that need to be recycled or sold, the last thing I want to do is harbour more stuff. Also what would I use my Aqtion 10GbE NIC and Intel X540 NIC for? Display units? Naw, I ain’t about that life.
Secondly, I’m a complete network newbie. If you have ever seen any of my content you will know that I have no idea of what I am doing. Most of the time I just ramble along. With that in mind and because I already have other Ubiquiti UniFi products, it gave the XG some much needed sex appeal. Oh la la. I really really enjoy the walled garden I’m setting myself up in and I really appreciate the ease of use of the UniFi Controller AND the plethora of information/support that is available from community members and staff.
Thirdly, I got a pretty good deal on it. I only had to fork over $494 dollarydoos. While that still sounds like a bunch of money consider the fact that it comes with 12 SFP+ ports and 4 10Gb ethernet ports. While the newer UniFi 6 XG PoE switch only has 4 ethernet ports and 2 SFP+ ports and is not rack mountable out of the box. One less thing to worry about. I don’t know what the future holds for SPX Labs but so long as I have the 16-XG, that future is not nearly as bleak!
Okay yeah but why did you but it?
I convinced myself that I needed it because my impatience was ever growing while transferring larger and larger video files around. Yeah, I could have just waited till I went to bed but believe me, I don’t turn off too well. Also, believe it or not thanks to a growing number of individuals support, SPX Labs now has a decent revenue stream and was able to provide it for itself and my labour. So thank you for your purchases and donations! Boom roasted.
Can you combine two the SFP ports on the US-16-150?
Yes, you can. This is called Link Aggregation and can be done in the UniFi Controller application. You will gain a bit of bandwidth between switches and maybe a little resiliency.
Does the US-16-XG support SFP modules?
Yes, just make sure you use the same module on both ends and in my case configure the Auto Link Negotiation to 1000Mbp/s on both switches.
Does the US-16-150 support SFP+ modules?
Yes, refer to the previous question on how to make it work.
Can I use an ethernet cable to provide internet and network access between both UniFi switches?
You bet your tail end you can!
Why do you sometimes talk and type English words?
Because I’m a chuffed individual and the Brits are bloody brilliant folks.
How loud is the US-XG-16?
It’s between 42db and 45db. I used my phone to capture the decibel level and I’m pretty sure it is more quiet but my phone was picking up noise from the outside AC unit. So take this all with a grain of salt.
Is the US-XG-16 quiet?
Yes, it is amazingly quiet. It was so quiet in fact I completely forgot to mention it in my video. It is also hella light weight.