Start of my proper server and network rack

[axipher]

I've been running off a little StarTech.com 12U self-standing rack for a while now with my UPS just on the floor so I figured it was about time to upgrade that rack and move my gaming PC in to the same rack to clear up some office space.

 

New 18U Enclosed Rack: StarTech.com RK1833BKM

 

Power:

• Eaton 5PX3000RT2U

• Eaton 5PXEBM72RT2U

• Pyle PDBC50 Power Strip

 

Network Gear:

• Unifi USG-3P - Security Router/Gateway
• Unifi US-48-G1 - 48 Port Switch with 2 SFP and 2 SFP+
• Unifi US-8 - 8 Port Switch
• Unifi UAP-AC-Lite - Wireless Access Point
• Phillips Hue Bridge

 

Server PC:

• Rosewill RSV-R4000
• SeaSonic Focus Plus gold 650 W
• Asus Prime X570-Pro
• AMD Ryzen 3600X (Will probably upgrade to a 3900XT at some point)
• 4x 8 GB Corsair 3200 MHz RAM
• 1x Intel 660p 1 TB SSD
• 2x ADATA SX8200 Pro 512 GB SSD
• Asus Hyper M.2 Gen 4 Expansion card (4x M.2 NVME SSD)
• 3x 6 TB IronWolf HDD
• 3x 4 TB IronWolf HDD
• Zotac GTX 1050 Ti (For Plex transcoding)
• Mellanox Connect X-3 10 Gig Network Card
• TC-RAIL-26 Rails
• Mediasonic USB 3.1 4 Bay 3.5” SATA Hard Drive Enclosure – (HF7-SU31C) - For Cold Backups

 

Gaming PC:

• Chenbro RM42300-F
• SeaSonic Focus Plus gold 750 W
• Asus Prime X470-Pro
• AMD Ryzen 2600x
• 2x 8 GB Corsair 3200 MHz RAM
• Sapphire Navi RX 5700 Pulse
• ADATA SX8200 Pro 1 TB SSD
• Mellanox Connect X-3 10 Gig Network Card
• Elgato Game Capture 4K60 Pro MK.2
• TC-RAIL-26 Rails
• Fractal Design Celsius 360 AIO

 

Misc Cables to make it all work nicely:

• Cable Matters 2-Pack USB 3.0 Extension Cable
• Cable Matters 25 foot Unidirectional Active DisplayPort 1.4 Cable.
• Takya USB 3.0 Switch Selector

 

Current status photo:

Edited September 24, 2020 by axipher
fixing image

[axipher]

Reserved TOC

[axipher]

Unboxing:

Doesn't look as big compared to the bike:

 

Included half shelf:

 

Some of the Accessories:

 

Ground Strap for the back door:

 

Full welded frame:

 

Top and bottom 120 mm fan mounts and fully disassembled so I can carry it up the stairs:

 

Removed from the shipping pallet:

 

Little tight at the top of the stairs:

 

UPS temporarily in the new rack, top rack almost ready to migrate to the new rack:

Edited September 18, 2020 by axipher

[Laithan]

Subbed

Nice Rack! ; ) Looks like, has some nice depth to it (full size it looks like). Good choice for sure. Mine only accepts a 19" depth (cannot use rails) so I've been restricted in quite a few ways. Looks like a nice project for the weekend, look forward to some updates. 

 Is it on wheels?

 

[axipher]

It is on wheels with leveling pegs and it is 25" Depth from Rail to Rail.  I need to figure out a material to use as filter for the side and doors to help with dust and noise levels, if anyone has recommendations before I go scour Reddit, let me know please

 

I just spent the night with my new Chenbro case that just arrived, not really worth a review and not going to recommend it unless you just want a barebones case that can support a gaming rig.  It does fit a 360mm as most 4U cases would but I ned to come up with a proper mounting bracket for it and some sort of mesh front.

[Bastiaan_NL]

Subbed
Looks like that rack is pretty decent. Quality welds and all, I like it!
Would be nice to put my folding rigs in there and use it as a mobile heating solution
I'm using some sort of dustfilter material I had to cut to size on one of my systems. It is a restriction but it does work.

[edit] "Premium Ultra Thin 0.17mm PVC Case/Fan Mesh Dust Filter"

Edited September 19, 2020 by Bastiaan_NL

[ENTERPRISE]

10 hours ago, axipher said:

It is on wheels with leveling pegs and it is 25" Depth from Rail to Rail.  I need to figure out a material to use as filter for the side and doors to help with dust and noise levels, if anyone has recommendations before I go scour Reddit, let me know please

 

I just spent the night with my new Chenbro case that just arrived, not really worth a review and not going to recommend it unless you just want a barebones case that can support a gaming rig.  It does fit a 360mm as most 4U cases would but I ned to come up with a proper mounting bracket for it and some sort of mesh front.

That is a nice rack system I have to say. I did not realize you even keep your gaming rig in a rack mount lol, that is great, a little restrictive in some use case purposes I suppose but not by much as that is a fair bit of space you have to work with.

[axipher]

4 hours ago, Bastiaan_NL said:

Subbed
Looks like that rack is pretty decent. Quality welds and all, I like it!
Would be nice to put my folding rigs in there and use it as a mobile heating solution
I'm using some sort of dustfilter material I had to cut to size on one of my systems. It is a restriction but it does work.

[edit] "Premium Ultra Thin 0.17mm PVC Case/Fan Mesh Dust Filter"

Thanks for that, I will check that out for sure.

 

My first idea was to source some large pieces of foam similar to that Fractal puts in the front of the Meshify-C or some really large commercial furnace filters (the wire mesh ones) and use magnets to secure them form the inside of the case and add some large 200+ mm fans if required for airflow.  This would be on the front door at least.

The side panels could probably make due with just some old black pillow cases as well as the rear then use those top two 120mm fans with my two remaining Gentle Typhoons as exhaust fans.

 

2 hours ago, ENTERPRISE said:

That is a nice rack system I have to say. I did not realize you even keep your gaming rig in a rack mount lol, that is great, a little restrictive in some use case purposes I suppose but not by much as that is a fair bit of space you have to work with.

Yeah, in my case with all quiet fans installed, its perfectly fine in my home office beside my desk.  For some people, that could be a problem, but then I would like do that Linus Sebastian did in this video with Long Thunderbolt cables through the wall to the adjacent room:

 

[ENTERPRISE]

2 hours ago, axipher said:

Thanks for that, I will check that out for sure.

 

My first idea was to source some large pieces of foam similar to that Fractal puts in the front of the Meshify-C or some really large commercial furnace filters (the wire mesh ones) and use magnets to secure them form the inside of the case and add some large 200+ mm fans if required for airflow.  This would be on the front door at least.

The side panels could probably make due with just some old black pillow cases as well as the rear then use those top two 120mm fans with my two remaining Gentle Typhoons as exhaust fans.

 

Yeah, in my case with all quiet fans installed, its perfectly fine in my home office beside my desk.  For some people, that could be a problem, but then I would like do that Linus Sebastian did in this video with Long Thunderbolt cables through the wall to the adjacent room:

 

Yeah that is an awesome setup from LTT. However..I actually like having my PC next to me, but all comes down to that preference thing.

[Laithan]

 You mentioned this rack is right on the side of your desk in your office. I wanted to make some comments/suggestions for your consideration.

 You have quite a few options with rack airflow. As the rack is configured right now, air can come in and out from any side it is basically an open design like leaving the side panels off your case. In this configuration creating negative pressure/directing air intake/exhaust isn't really possible (but is it even needed?). You may not need to do anything and just let the individual components cool themselves. I guess filters could help in this use case there is really no negative/positive pressure created (how effective could they be?). I honesly wouldn't bother with filters and I would do everything possible to keep it as "open" as possible otherwise you will start to trap heat "enclose" and then you'll need to make more noise (fans) just to get rid of it. 

 Personally I have found that dust filters will certainly filter some dust, but even in the most ideal configurations they still end up providing more air restriction than anything else (even when clean). I think it is just par for the course and a foregone conclusion to have to take your components outside routinely and fire up the air compressor to clean out all the dust (once a year at least). There's just no way to keep all dust out so I don't bother fighting that fight anymore because cleaning your components is still needed eventually filter or no filter. 

 What were your plans with the 4 x 120mm intake/ehaust ports? It is pretty standard to have 120mm "muffin fans", 2 at the bottom and 2 at the top of a rack to constantly move air up and out but they are noisy by nature since they move a lot of air (wind noise, hard to avoid). These 120mm muffin rack fans run on 120v. You could technically use 120mm PC fans but they would be far weaker and would require some kind of AC/DC adapter but easiliy doable (can also use a fan controller). 

 Over time, I filled my rack.. I cannot fit anything else and the point is that I now have a lot more heat to get rid of than I originally did. I was forced to install high-rpm 120mm muffin Fans to get the heat out of the rack as I was seeing temps of 90F-114F which is not good. It would be impossible for me to use quiet fans at this point. Thankfully my rack is in my basement where the noise is not a concern. I also get far less dust down there. 

Looking forward to the results

 

[axipher]

Small Update:

Managed to get all the gear moved in to the full case today.  I have power coming in the bottom cable hold and all data cables coming out the top cut-out, picture to follow.

 

 

On 9/19/2020 at 12:46 PM, Laithan said:

 You mentioned this rack is right on the side of your desk in your office. I wanted to make some comments/suggestions for your consideration.

 You have quite a few options with rack airflow. As the rack is configured right now, air can come in and out from any side it is basically an open design like leaving the side panels off your case. In this configuration creating negative pressure/directing air intake/exhaust isn't really possible (but is it even needed?). You may not need to do anything and just let the individual components cool themselves. I guess filters could help in this use case there is really no negative/positive pressure created (how effective could they be?). I honesly wouldn't bother with filters and I would do everything possible to keep it as "open" as possible otherwise you will start to trap heat "enclose" and then you'll need to make more noise (fans) just to get rid of it. 

 Personally I have found that dust filters will certainly filter some dust, but even in the most ideal configurations they still end up providing more air restriction than anything else (even when clean). I think it is just par for the course and a foregone conclusion to have to take your components outside routinely and fire up the air compressor to clean out all the dust (once a year at least). There's just no way to keep all dust out so I don't bother fighting that fight anymore because cleaning your components is still needed eventually filter or no filter. 

 What were your plans with the 4 x 120mm intake/ehaust ports? It is pretty standard to have 120mm "muffin fans", 2 at the bottom and 2 at the top of a rack to constantly move air up and out but they are noisy by nature since they move a lot of air (wind noise, hard to avoid). These 120mm muffin rack fans run on 120v. You could technically use 120mm PC fans but they would be far weaker and would require some kind of AC/DC adapter but easiliy doable (can also use a fan controller). 

 Over time, I filled my rack.. I cannot fit anything else and the point is that I now have a lot more heat to get rid of than I originally did. I was forced to install high-rpm 120mm muffin Fans to get the heat out of the rack as I was seeing temps of 90F-114F which is not good. It would be impossible for me to use quiet fans at this point. Thankfully my rack is in my basement where the noise is not a concern. I also get far less dust down there. 

Looking forward to the results

 

 

Thanks for the feedback.  I was planning on covering up with some low restriction foam to more help with noise but you bring up some good points.  I was going to attempt to somewhat seal the rails to the side of the case and put in blank plates in all the unused rack spaces so that the equipment all creates a negative pressure in the front space which could help pull in air through a loose filter.

 

That will come down to some testing.  I also ordered some Arctic 80mm fans for the UPS to quiet them down along with the extended battery module for that UPS that adds another 12 cells, just a casual 109 kg added to this thing...  Battery life will not be an issue in an outage anymore.

 

Edited September 24, 2020 by axipher

[axipher]

My Eaton EBM arrived, another 72V/18Ah of capacity:

 

Edited September 24, 2020 by axipher

[ENTERPRISE]

That is a beefy looking UPS. Not to pry but how much do those usually set someone back ?

[axipher]

1 minute ago, ENTERPRISE said:

That is a beefy looking UPS. Not to pry but how much do those usually set someone back ?

I got the UPS from my work as they were upgrading all the server racks to be fed from a central UPS so I got it through a work auction for around $500 with old batteries.  I just bought the Extended Battery Module new though and got a deal on it through my work's commercial account.

 

Prices new are as follows:

Eaton 5PX3000RT2U: $1500 CAD

Eaton 5PXEBM72RT2U: $900 CAD

[Laithan]

Those steel toed boots are going to come in handy

Congrats! Nice UPS

Edited September 23, 2020 by Laithan

[axipher]

10 hours ago, Laithan said:

Those steel toed boots are going to come in handy

Congrats! Nice UPS

Yep, luckily Eaton's site was wrong and the extended battery module is only 109 lbs, not 109 kg.  So it's completely manageable on my own with proper gloves and steel toe boots as you mentioned.

 

I still need to swap out the noisy Sunon 3600 RPM fans our for some quieter 80mm Arctic's that play nicely with the RPM sense line of the UPS, but that means shutting down my entire rack including network.  I really should have invested in a UPS Bypass switch, but since the UPS as three distinct outlet groups, I would need at least 2 of the bypass switches to make it worthwhile.

 

I think I can afford a couple hours of downtime later today to swap out those fans, if not tomorrow night for sure.  I also need to power cycle the UPS to plug in the extended battery module anyway so that it recognizes the additional batteries on start-up and does a full battery test.  So I can do the fan swap at the same time.

[axipher]

So as a little bit of historical tracking, the following thread helped me with the fan replacement in my Eaton UPS as the stock Sunon 3600 RPM fans (ME80251VX-0000-G99) were a little loud and definitely designed for a worst case of the UPS being installed in a non-cooled rack.

 

Original HardForum Thread:

Quote

 

Arctic Replacement fans I used: Arctic F8 TC - Temperature-Controlled 80 mm Case Fan

 

 

New fans are significantly quieter and I opted to place the temperature sensors of the fans on the actual fin stacks of the inverter section of the UPS.  The main thing to note is that in this UPS case, the Black and Red are the Ground and 12V lines for the fan and the Yellow wire is the Locked Rotor feedback that instead is connected to the RPM feedback line.

 

 

I also managed to get the Extended Battery Module installed, it was at about 72% charge when I installed it and it quickly charged up to 90% then takes about 12 hours to fill the last 10% to help keep the batteries lifespan.  Even at only 96% charge right now, I went from around 40 minutes of runtime at 300W of load to over 3 hours of runtime.  I feel much more comfortable running my gaming PC for Folding@Home off the UPS now as I also have email notifications when the UPS goes in to battery mode (notification from my UnRaid server with USB connection to server) and I also have Pulseway setup on the UnRaid server itself so that I can get an email notification from Pulseway's server if I lose internet access.

 

 

I also installed two Gentle Typhoon fans in to the top two 120mm slots on the server rack and connected those down to my UnRaid server on a spare fan header so that I can control the fan speeds instead of just having the fans run 100%.  I figure that since the server is the one thing that should always be running, it's a safe bet to control the rack fans from.

 

 

 

 

 

Next on the list for this server will be:

• Unifi UDM-Pro to replace the Unifi USG-3P just sitting on the shelf
• 1U Cable organizers and/or 1U cable brushes and/or 1U keystone panel
• 1U Blank plates in the rest

Edited September 24, 2020 by axipher
fixing image

[Laithan]

I really like those temperature controlled fans... I didn't know they were a thing.

 

PS. The really long link isn't working for me.. Not sure what it is.. A picture? I don't see a pic in the message and when I click the link it looks like it is trying to display an image. Just me?

 

 

 

 

Edited September 24, 2020 by Laithan

[Laithan]

[axipher]

1 hour ago, Laithan said:

 

Okay, should be fixed now.

 

It looks like when I was doing "Right-Click > Copy Image" in Chrome from my Google Photos, it was actually hard-linking to the image URL on Google's servers, so unless you had a Google Account logged in, you couldn't see those images.  Fixed them all by actually uploading images to EHW directly instead.

[tictoc]

10 hours ago, axipher said:

So as a little bit of historical tracking, the following thread helped me with the fan replacement in my Eaton UPS as the stock Sunon 3600 RPM fans (ME80251VX-0000-G99) were a little loud and definitely designed for a worst case of the UPS being installed in a non-cooled rack.

 

Original HardForum Thread:

 

Arctic Replacement fans I used: Arctic F8 TC - Temperature-Controlled 80 mm Case Fan

 

 

New fans are significantly quieter and I opted to place the temperature sensors of the fans on the actual fin stacks of the inverter section of the UPS.  The main thing to note is that in this UPS case, the Black and Red are the Ground and 12V lines for the fan and the Yellow wire is the Locked Rotor feedback that instead is connected to the RPM feedback line.

 

 

I also managed to get the Extended Battery Module installed, it was at about 72% charge when I installed it and it quickly charged up to 90% then takes about 12 hours to fill the last 10% to help keep the batteries lifespan.  Even at only 96% charge right now, I went from around 40 minutes of runtime at 300W of load to over 3 hours of runtime.  I feel much more comfortable running my gaming PC for Folding@Home off the UPS now as I also have email notifications when the UPS goes in to battery mode (notification from my UnRaid server with USB connection to server) and I also have Pulseway setup on the UnRaid server itself so that I can get an email notification from Pulseway's server if I lose internet access.

 

 

I also installed two Gentle Typhoon fans in to the top two 120mm slots on the server rack and connected those down to my UnRaid server on a spare fan header so that I can control the fan speeds instead of just having the fans run 100%.  I figure that since the server is the one thing that should always be running, it's a safe bet to control the rack fans from.

 

Next on the list for this server will be:

• Unifi UDM-Pro to replace the Unifi USG-3P just sitting on the shelf
• 1U Cable organizers and/or 1U cable brushes and/or 1U keystone panel
• 1U Blank plates in the rest

Good call on replacing the UPS fans.  I swapped out mine for some Noctuas, and it is much quieter now.

I think that UnRaid uses NUT for UPS monitoring.   If that's the case, you can pretty easily script custom actions based on the different states of the UPS.  For example, with cron or systemd timers + ssh you could setup scripts to do things like pause folding after a certain amount of time, extending the uptime of your server and/or network appliances, during an extended outage.    

[Laithan]

3 hours ago, axipher said:

 

Okay, should be fixed now.

 

It looks like when I was doing "Right-Click > Copy Image" in Chrome from my Google Photos, it was actually hard-linking to the image URL on Google's servers, so unless you had a Google Account logged in, you couldn't see those images.  Fixed them all by actually uploading images to EHW directly instead.

Much better, now I can see all the pics I was missing

Great job, looking nice!

[axipher]

1 hour ago, tictoc said:

Good call on replacing the UPS fans.  I swapped out mine for some Noctuas, and it is much quieter now.

I think that UnRaid uses NUT for UPS monitoring.   If that's the case, you can pretty easily script custom actions based on the different states of the UPS.  For example, with cron or systemd timers + ssh you could setup scripts to do things like pause folding after a certain amount of time, extending the uptime of your server and/or network appliances, during an extended outage.    

 

Yeah, I couldn't find many people that had done the swap on this particular model as the Sunon Fans that Eaton selected actaully use something called "Locked Rotor Detection" on the sense line and I guess some fans have too low or high resistance on their sense line that causes fan alarms, but these Arctic's seemed to have worked for other people.  I didn't spend much time looking in to the technical details of it too much as these were cheap alternatives and my rack is in a climatized environment anyway so the high-speed fans probably aren't needed.  I'm assuming that Eaton put in the fans they did to account for being installed in a non-airconditioned industrial telecom rack somewhere so the added airflow would be important.

 

The default UPS plugin wouldn't recognize my Eaton UPS, so I had to use NUT-Settings Plugin that allows me to access all the values from the UPS including all 3 outlet groups, input voltage, input frequency etc.  I'm using a User Script to feed those to InfluxDB then to a Grafana Dashboard per the below screenshots. 

 

As for shutdown of my gaming PC, I currently just have a Powershell script that runs on my gaming PC that polls the InfluxDB every 5 minutes.  If it detects that the UPS is on Battery mode for 15 minutes, then it issues a safe shutdown command.  The UPS itself is configured to hard power off Outlet Group 2 if the battery gets down to 50% as an added backup to give my network gear and UnRaid server more time.  The UnRaid server will shut itself off at 20% battery then ideally I would still have another couple hours of internet as long as the ISP doesn't cut the signal.

 

 

 

 

30 minutes ago, Laithan said:

Much better, now I can see all the pics I was missing

Great job, looking nice!

 

Thanks mate, really need to do all those little final clean up steps along with build a front filter panel the the Chenbo Case.

[ENTERPRISE]

7 hours ago, axipher said:

 

Yeah, I couldn't find many people that had done the swap on this particular model as the Sunon Fans that Eaton selected actaully use something called "Locked Rotor Detection" on the sense line and I guess some fans have too low or high resistance on their sense line that causes fan alarms, but these Arctic's seemed to have worked for other people.  I didn't spend much time looking in to the technical details of it too much as these were cheap alternatives and my rack is in a climatized environment anyway so the high-speed fans probably aren't needed.  I'm assuming that Eaton put in the fans they did to account for being installed in a non-airconditioned industrial telecom rack somewhere so the added airflow would be important.

 

The default UPS plugin wouldn't recognize my Eaton UPS, so I had to use NUT-Settings Plugin that allows me to access all the values from the UPS including all 3 outlet groups, input voltage, input frequency etc.  I'm using a User Script to feed those to InfluxDB then to a Grafana Dashboard per the below screenshots. 

 

As for shutdown of my gaming PC, I currently just have a Powershell script that runs on my gaming PC that polls the InfluxDB every 5 minutes.  If it detects that the UPS is on Battery mode for 15 minutes, then it issues a safe shutdown command.  The UPS itself is configured to hard power off Outlet Group 2 if the battery gets down to 50% as an added backup to give my network gear and UnRaid server more time.  The UnRaid server will shut itself off at 20% battery then ideally I would still have another couple hours of internet as long as the ISP doesn't cut the signal.

 

 

 

 

 

Thanks mate, really need to do all those little final clean up steps along with build a front filter panel the the Chenbo Case.

 

I have been reading through this thread, that rack setup is certainly coming on and I was just looking at the image of your Unraid CP, that looks so nice bud !

[axipher]

3 hours ago, ENTERPRISE said:

 

I have been reading through this thread, that rack setup is certainly coming on and I was just looking at the image of your Unraid CP, that looks so nice bud !

 

Once I got InfluxDB and Grafana setup, I kind of just went overboard with the stats I was throwing in and making dashboards, most just end up looking pretty and not having any real troubleshooting stats on them, but sometimes it can help identify trends.