Laithan

The 140mm fan isn't doing a good enough job for the controllers so I am going to add this Delta 60mm fan directly on the spot. LSI 9271 cards (most LSI cards) are really undercooled...they will overheat without some kind of cooling.. but they perform excellent so as long as you're aware they are a great choice. I'll post some benchmarks once I get Winderz installed.

https://www.delta-fan.com/AFC0612D-SP07.html

​

I guess I never really showed the controller so here's a couple pics with the battery backup and the cachecade 2.0 chip installed.

http://www.ssdworks.com/MegaRAID-SAS-9271-8i.asp

If you have the room, this is a great fan (I use them on all my LSI cards) and is the "proper" one

https://www.ebay.com/itm/New-Cooling...YAAOSwxu5ZOB81

Intel 10GB

https://www.silicom-usa.com/pr/serve...erver-adapter/

I cleaned up some of the wires as best as I could and bending some things uncomfortably

I've also got both RAID controller batteries installed. The fan for the cards are just an insurance policy as the 10GB nic and especially the LSI controllers get quite toasty.

That is a heavy boy, but to be expected, not surprised you have to reinforce the rack to accommodate, crumbs you would not want that falling to a heap on the floor !

Nope, that would not be a good day... I weighed it with the drives and NIC installed and it is just about 60lbs total. Of course as soon as I installed the fan for the cards I remember that I didn't plug in the HDD led for the controller lol. The array is initializing now..

All that remains for the build itself is a single fan, should be in Monday or Tuesday. I managed to brute force the chrome fan guard on to the CPU fan because there is just something about chrome and beige lol.

Comparison of the old fans and the replacement A9

That is looking sexy, glad that you got it up and running, that first boot is always nerve wracking haha.

Ty. It sure is nerve racking More so than ever this time, if I had to troubleshoot, both drive bays would have to come out (they cannot just slide forward) and getting the 10 sata cables plugged in when they are on the bottom of the enclosures wasn't joyful the first time.. I am not looking to ever revisit that again. <No Jynx!>

 

 

Almost at the finish line, very nice presentation ! btw, how close will you be sitting to this system (...wondering about the Delta fans as I have 'experience' with them) ?

 

Ty. Oh I'm not sitting anywhere near this thing . I have a small rack in my basement with my network/server gear (You can see some pics here). These Delta fans wouldn't be the loudest fans I have lol. The rack fans (two on top and two on bottom) are beasts and are running 100% 24/7. I have the Deltas using Q-Fan (which I don't like) so they are not running at full load all the time and should only ramp up when temps get higher. I have to do some experimenting with this however I understand that once in the O/S (windows) I should be able to define a custom fan curve. One of those features I just never paid any attention to but have had it on dozens of boards.

One of the interesting things I didn't anticipate is the weight of this server. Without any drives installed in the bays it is easily 40-50lbs (I should actually weight it somehow). The entire case is flexing considerably if I attempt to pick it up. I would think that 60Lbs would be a low estimate when fully loaded. Due to this reason I had to install some additional support on the rack to hold the extra weight lol.

EDIT: I weighed it and with only 2 drives installed it is 46Lbs

Very close now, just have to replace the CPU cooler fans and a few more items. This was a puzzle for sure as everything had a specific order and so many things barely fit! The good news is that the system booted right up and everything was detected. Woo-hoo.

No show-stoppers yet however I now have an army of cables to attempt to put somewhere... lol

So for my home network, I'm using some Mellanox Connect-X3 cards with SFP+ ports and picked up a pair of them alongside some 15 feet Direct Attach Copper Cables for only $170 on eBay.

Hmm that's not bad at all. So you're not going through a fiber switch?

 

Very nice build-up layout (& motherboard )! ...I am looking forward to the rest of the build-log.

 

Per other threads @ EHW, I plan to try out that win-raid modded bios you had linked for the X79-E WS (NVME; bifurcation) and then see if the MSI Aero card below works. It's PCIe 3, but afaik, PCIe 4 M.2s drives should work, just at PCIe 3 speeds (thinking about future upgrades for these drives into a PCIe 4 environment) ?

 

Thank you.

I'm very much looking forward to your results. The card looks amazing. I wonder if you can boot from it also?

You were not lying about a tight fit were you lol, jeez there really is not much in it, but good that it fits in otherwise that would have been a facepalm moment. At least you have plenty of room for the GPU though.

 

Fingers crossed that you can give it a moderate OC and the cooler will hold up, as you say though, replacing the fans will help you on that front and there are plenty to choose from. If noise is not an issue then you have it made and you could get some really powerful units on that heatsink.

Come to find out, things are getting even tighter, you'll see in the next update. I was glad that I got that Zotac GPU because just about any other GPU would not have fit. The drive cage fans are massive about double the thickness of a standard fan so I have only a little bit past the PCIe slots.

Usually I will run my O/C at full boost all the time (all cores) and temps are managed based upon workload but this motherboard has an EPU feature that I'm going to look into and also see if I get the CPU stable with speedstep and power management enabled since it will be running 24x7. It makes the O/C a bit more challenging but I think I'll be fine since I'm targeting a modest 4Ghz.

https://www.asus.com/support/FAQ/1036401

So much storage, love it.

 

 

I love my current Plex Server in a Rosewill RSV4000 with up to 12 3.5" bays and M.2 as Cache drives, but I think my next build I might actually want to paly with the idea of networked storage via a 10 or 25 gigabit link then just have a separate box of purely drives and redundant power supply.

 

That way I don't need to take my main array down if I am upgrading storage and the big network storage block would be for the less important bigger things like backups, all the media, etc. and reserve the main array for files, web server, VM's, etc.

I really like that case also and I had considered it. It reminds me a lot of a dell rack mount chassis and has a nice clean look to it. Rosewill is hit or miss but usually when they hit, they knock it out of the park.

I haven't really experienced storage over 10GB+ in my home lab, only 1GB (NFS) where it honestly wasn't very impressive. I am curious how well it would actually perform with benchmarks like ATTO over the wire. A lot of the used equipment like the EMC NS-120s are dog slow but tick all the checkboxes on paper. Good 10GB NICs are still $150.. We seem to be waaaay overdue for an industry standard bump to 10GB, it's still treated like it was just invented yesterday..

EDIT: I looked it up, the standard for 10GB ethernet was defined in 2002... 18 years ago... and only now are motherboards starting to ship with them standard. That's kind of interesting how network performance stayed stagnent for so long.

 

Thanks. I also just saw @Laithan 's link above for the X79-E WS win-raid updated bios for NVME & bifurcation ...now I can plug in that 4x M.2 MSI Xpander Aero card that came with the X399 Creation and see what's what. Whether Plex, NAS or (2nd) retro gamer with EVGA Classies, this combo offers a lot of versatility.

I'm also curious how that works out

I've got more drives than I need but it is good to have spares on hand.. I think I'll go with (10) 4TB 7,200 RPM Spinners and (4) of the SSD's. That would give me around 36TB of usable storage. I was thinking of (1) SSD for the cachecade and the other (3) for a RAID 5 O/S & apps drive. I ran into some issues with the LSI 9271-8i card it may be defective so I ordered another... <sigh>

Good news! It looks like the drive bays will fit over the motherboard. It was another really tight fit. I need to install the BluRay drive and reverse the fans but wanted to get a quick test fit.

Nice, glad it arrived ! I would recommend changing out the CMOS battery for a new one as well due to the age of the board. Plus it means if you have setting retaining issues you can remove the CMOS battery as a factor.

Agree, good advice. It actually didn't even come with a battery, I guess maybe an oversees shipping thing.. So I put a brand new battery in before I fired it up as I have a bunch in stock. The issue is odd, I will set "Advanced mode" in the BIOS and then I find that "Ez mode" is enabled again...usually after power was disconnected. The time and date settings stuck and fan settings stuck but many other settings were reset. I would see a CSM error every time I booted but there were no boot devices attached. Once I attached a drive I didn't see the error so I think it may just be due to not having a boot device to select. It may just be a quirk that will be resolved once I get things assembled and select the boot device. I loaded BIOS defaults etc. also.

I also noticed that my U9DX-i4 came with NF-B9 fans (discontinued, replaced with redux) so I am upgrading them to NF-A9 PWM as they are the successor and have a higher RPM (1600 vs 2000). I really didn't feel like the stock fans were pushing enough air. I like the Noctua fans but sometimes they are just too weak. I suspect the cooler would do pretty well for a moderate overclock but not without the better fans. I'm hoping to have it hit 4.0Ghz on all cores (this Xeon is unlocked).

The motherboard has finally arrived and passed initial testing! I am noticing a similar issue where the BIOS loses partial settings after a power off but I just flashed it with the lastest 1704 BIOS + latest 42E microcode and I'll see if it resolves the issue. I can confirm that the latest microcode 42E has been included in the MOD BIOS link I posted a few posts ago at the bottom. I am using BIOS flashback. I immediately noticed about 5C lower idle CPU temps after flashing the new BIOS but it could be specific to this Xeon E5-1680v2 (since it is a very special CPU).

Tomorrow we see if things fit... <this is going to be a nail biter!>

Intel has been making motherboards for years but I never actually used one for personal use until just a few years ago (my first ESXi build). I suspect all the thoughts of the SuperMicro boards also apply to these . It does get the job done indeed and is stable... but features are pretty much minimal/stripped.

 

Yeah it came in, all installed and working as I would expect a motherboard too....remember your BIOS settings lol. Kinda basic. Great board though! Good to have the server running again.

Great to hear! I'm STILL waiting on my replacement board... and who knows if this one will work. I wonder how to properly benchmark a Plex server.. hmmm

Did the new board come in yet?

 

...I just kept this one since new, even after adding other more modern systems. The Gigabyte Z170 SOC Force and Asus X99-E WS are in that same 'special reserve'...when the X399 Creation workstation will be updated, there's a seat on that table for it, too ...unlike later Intel gens, the X79-E WS was as fast if not faster than its Rampage Extr counterparts, especially on m-GPU.The only limitations are:

 

a.) Asus has still not gotten around (at last check) to update the bios re. side-channel stuff even though they did for a lesser version of this board, and

b.) the bios does not do memory settings over 2400 MHz (unlike the Rampage), even if the CPU does, yet will go much higher via FSB tuning. It also takes decent amounts of memory tightening on secondary timings

 

This one also belongs in that club

https://www.asus.com/us/Motherboards/Z9PED8_WS/

I downloaded this BIOS, it looks like it has the updated microcode also. The ZIP itself has been updated.

https://www.win-raid.com/t4040f44-OF...-Bios-MOD.html

If you find yourself needing mare RAM, shoot me a message.

I have a bunch of 8GB ECC RDIMMS. https://www.samsung.com/semiconductor/dram/module/M393B1K70DH0-YH9/

 

The sticks are 1333 rated dimms, but I ran them at 1866Mhz with a bump in voltage, on my old 2P ASRock server.

Thanks, will let you know but I am most likely limited to low profile DIMMs due to the size of the drive encosures. Once I get the motherboard I can test fit everything and I'll know (may not even fit with the low profile DIMMs). I actually have 32GB of ram now so I should be good but never know if it will like this board.

The board I am using is hard to find because of the number of PCI-e slots (and dual PLX chips). I will share some pics hopefully soon. Thanks guys

https://www.asus.com/us/Motherboards/P9X79E_WS/

(Estimated)

SLOT 1 (and covers 2) = GPU

SLOT 3 (and covers 4) = LSI 9271-8i

SLOT 5 = Intel X550 10GB

SLOT 6 = SAS expander

SLOT 7 = 2nd LSI 9271-8i

Fingers crossed dude. As it happens, I have given up on my Asrock Phantom Gaming ITX board, there is definitely a hardware fault with it. Diagnosed it with Asrock to see if they could come up with a solution, but nothing can be done. So will be returning that for a refund. Seems to be a common issue with that board so wont risk a replacement. ASUS ROG Strix I-Gaming here we come lol.

It is never as easy as it is supposed to be I guess... Hope you have better luck with the new one. I always stick with ASUS boards.

I ordered another from China... <sigh> The long wait begins all over again.

Yeah that motherboard was a real bummer man. Some sellers are unbelievable ! Hope you get your replacement soon.

 

Ha, yeah you are at that stage where you are hoping everything fits ok, been there plenty of times !

Yup! Should be arriving today. Praying to the motherboard gods that it works and it fits.. otherwise it will be a nuke in this build since I have embraced "overkill" lol

It arrived...and they shipped the wrong ^$(!) motherboard...

Nice choice on the Blu-ray drive. I have one too, though an external one. I must admit I barely use it as I keep all my files digital as opposed to on disk, but its nice to have the option.

 

Yeah working in the BIOS issue with Asrock. They have released a new one which I will test, but only after giving them my diagnosis results from using th current BIOS so they have more to go on if this new BIOS is still poo lol.

 

Looks like your build is coming together nicely !

I'm at a stand-still until the replacement motherboard comes in... then I can see if the drive cages are going to fit... This is going to be tight, cages are pretty large..

Hopefully next week sometime.

Also posted up my build here: https://extremehw.net/forum/full-pc-builds/server/7175-my-plex-server-build-rough-guide#post7175

It looks amazing! You got that together pretty quickly also. Hope you get the BIOS issue resolved.

 

I imagined that was what you were doing. That is fair enough, probably a little overkill but hey ...we chase performance right ! I have put my transcode folder on my standard SSD, should still suffice nicely for transcode temp files.

Overkill.. has such a nice ring to it.. love that word! After all, this isn't BarelyAdequateHardware.net

I figured I would put the slim drive space to some good use so I picked up this LG BU40N and flashed the modified firmware to allow it to read my UHDs. I needed the sata slim to regular sata adapter also. https://www.lg.com/us/burners-drives...ray-dvd-writer

Looks great! Nice job on the build. I love the case especially the transparent top. Nice selection of gear also this ought to be perfect!