Laithan

Great to see you making progress and working through the PLex Gotcha's.

 

I definitely think they need to spend some more time on the "auto" settings on clients or give better controls for selecting quality for the average user. Right now they give you options on android and web apps, but casting apps and smart TV's are stuck basically auto adjusting on the fly which can lead to changing quality for the first 5 minutes of a show/movie until it settles on a good setting.

Ty. Agree, there are definitely some additional controls needed. Just because I have a 4K TV doesn't always mean that I want to view a video in full 4K resolution. I would have thought that these simple controls would exist (or do they and I just need education?).

I'm loading the rest of my media today. The server had a taste of thumbnail creation yesterday, tonight it will have a buffet . I noticed that it isn't storing the thumbnails etc in the app folder instead it is using C:\Users\profilename\AppData\Local\Plex Media Server.. I don't really want this. This is the solution: https://support.plex.tv/articles/201...rver-settings/ . You will want to shutdown your plex server first. You can enable hidden settings. In this case I enabled the setting to change the location of what was being stored in AppData to my application folder on another partition. I won't have to worry about filling my O/S drive now. I am attempting to move all the data over now as it lost all my settings when I made the change because Plex was running when I edited the registry (oops). It all came back when I copied the contents of APPDATA over.

I am sure you know this but I figured I would share for anyone that did not. I just discovered that when playing on a PC it is better to use Plex Media Player vs using a web browser. The biggest reason is that the Plex Media Player will allow direct play where the browser will transcode.

This is going to be real interesting for sure!

Grab some popcorn folks

:eat_popcom:

I was starting to get alarmed because the entire RAMDISK was filling up..

but after some reading it appears that Plex will basically fill up whatever size you provide it and it will perform cleanup automatically as the drive gets down to 100MB free..

We shall see

EDIT: I didn't seem to have any problems... when I stopped the movie it cleaned it all up. I read that at least 4GB was recommended for each simultanoues transcode.

Yes, audio transcoding is still a single core task. Being at 100% will happen on 1080p or 4k at various numbers of audio channels, it will just be able to transcode longer buffers and keep processing audio. It's normal and doesn't necessarily lock that core fully to harm other tasks on that thread.

 

I can't see a 3+ GHz core having issues with the audio processing unless the source is like 7.2 or Atmos or something and trying to transcode it to 2.1 or just stereo.

 

Well, ya it was lol . Source was a 4K UHD movie.

I had multiple transcodes from 4k to 1080p running at the same time and I really didn't have any actual playback troubles.. I think I was alarmed about the core being maxxed out more than anything. My laptop was choking a bit because it only has 2 real cores (4 threads) but that wasn't the server's fault...

I also ran into some bandwidth issues with my TVs that are 4K capable. Plex wants to direct play them and it is too much for wireless. I managed to get it to transcode to 1080p on my TCL but the Samsung doesn't want to play.. again not the server's fault so overall I think I may actually be OK. It is good to know others see that high single core workload.

 

Yeah do not blame you for just using it as is, you can always look into it later if you run into issues, but for testing and assessing performance over time, you might as well leave it as is. By the way, do not forget to add this in Rig Creator, we are interested in the full specs of this beast.

Thanks! I do keep forgetting to to that

I did get the Plex media server installed and configured and did some preliminary testing. I still have some tweaking to do as @J7SC_Orion said below. I noticed a little difficulty with 4K transcoding. I think this is the same issue just about everyone has and the only bottleneck that I am seeing is with a single core on the CPU.

I assume this is the AUDIO transcoding that is causing me issues... (have to look into workarounds perhaps)

 

 

With a sophisticated build and all its components like this, there will be lots and lots of options to fine tune for months to coem. BTW, I have used and just love Asus RAM disk, one reason why I tend to run larger amounts of RAM in my builds. Asus RamDisk even works on some non-Asus boards if they are of similar vintage. That said, RamDisk obviously might not be such a good idea on a Plex media server, but it can make mincemeat out of the fastest nvme drives out there.

Agree. This is my 2nd build during the pandemic (built a PC for my son also) so it was a good project and something to help keep me busy for a while. Thankfully in this build I didn't go in completely blind, the only component I didn't have experience with was the SAS expander and cachecade 2.0. Turns out it is just plug N play so I didn't even have to configure anything. I've been testing things and it has really gone very smooth for the most part. I ripped a 4K Ultra movie last night and everything is working as expected there. The last issue is the one above, if I am dealing with single core performance then I may need to dive into overclocking sooner than expected.

 

Nice, looking good.

 

Another program that may be of interest to you is Primo Cache. I use it for writing my data to my RAM first when it comes to files etc and then it will do a deferred write to my SSD's. The nice thing is that aside from the speed, as it defers the writes back to the SSD in bulk, it helps reduce the wear on your drives which you would get from multiple writes. I have used it for a good while now, super reliable and never caused me any instability.

Thank you

I will definitely take a look that looks very impressive. The server is on battery backup so I would be OK with trying it. I actually saw that ASUS offers something like that also call ed ASUS RamCache. Initially I confused it for the ASUS RamDisk because I didn't know that it existed. It actually sounds very much like Cachecache except taking it to the next level using system RAM and SSDs instead of SSDs and HDDs. I'd have to look and see if it is recommended for use with HDDs. I think at this point I'm just going to try everything as-is first and see how things perform. I am a little worried about the CPU since it does not have Quicksync but I'm feeling confident that the Turing NVENC/NVDEC will do the job. Fingers crossed!

Digital Extortion indeed... sadly where it really originates is from the agreements accepted within the terms and conditions set by digital storefronts. They surely had the lawyers comb over ever word to ensure that there is no possible way to bypass the digital storefront's "cut". I really don't see how EPIC can win if they accepted these terms and conditions up front (regardless if they are fair, right or wrong). I commend them for fighting this because let's be honest, 30% is waaay too much... Digital distribution should earn a slice and be very profitable.. what they do is allow the download...manage some keys and certainly provide discovery of the software. IMO 30% (x%) is just not proportional since they were not responsible for ANY development costs at all...

Agree, finally something useful for those wasted 1x PCI-e slots

I've got the RAMDISK installed (ASUS Ramdisk 2.03.00). We'll see how reliable it is but so far so good.

I've got 32GB total which the Plex server definitely won't need so why not.. It will save writes on my SSD array and the performance speaks for itself. This is only on 1600Mhz ram and nothing is overclocked yet so this is just baseline speed.

I was also testing network performance and I'm incredibly happy to see much better transfer rates vs running in a virtual machine. The bottleneck is the slowest drive involved, not the wire The screenshot below is from Windows 7 x64 connected to the Plex server via administrative share. SMB is a bit older on 7 although it doesn't really appear to be slowing down much if at all. Will have to test Windows 10 -> Windows 10.

I've pre-staged all currentl data (less than 3TB) and I'm pretty much ready to turn it into a Plex server now.

 

...all TR CPUs are unlocked, afaik (don't know for sure about the new 3995X TR Pro line = 64c/128t, 8 channel Ram). However, with latest TR gens in particular, heavy cooling and 'PBO' setting usually gets you better every-day results than old-school multiplier / FSB all-core oc

 

You have to love that! AMD is doing great things

 

It does take a little more time than an all core OC or just leaving it to PBO. However once you get there, it is rewarding and kind of a fun process.

 

Exactly, I love that. I never actually heard of PBO until you mentioned it. Quite interesting indeed.

...Glad you got your custom fan-curve and SSD cache issues settled. As to the power consumption, I'm jelly...my main workstation can easily peak past 1070W (2x 2080 Ti @ 380W max EACH on stock bios, 1x oc'ed TR, various peripherals). 450W + - sounds like a dream

lol Well done! That's the way to do it!

That is a monster system!

<drool>

I'd love to build a TR system one day as I've never built or used one yet... How do you like it? When it comes time to refresh my ESXi server I suspect that's the platform I'll end up on and by then the prices should be more reasonable. Are any of the TR CPUs unlocked?

The CPU is pretty old at this point but it is worth mentioning that the highest core/thread count consumer CPU on the X79 platform was 6 cores/12 threads. All of the unlocked processors were either 4/8 or 6/12. There is a fun fact here at the end.

The next "sorta" secret on this platform is that you can use Xeons which are unlocked. The E5-1650v2 and E5-1660v2 are actually unlocked processors and overclock extremely well. A lot of people knew this but the average person didn't.

Then we have a very special CPU that very few knew about (at the time).. the E5-1680v2. This is the ONLY 8 core/16 thread UNLOCKED processor on the entire platform (there was also a 10-core server CPU but it wasn't unlocked). There is also a very large 25MB L3 cache. This is the highest amount of cache per core on the entire platform if I am not mistaken. This processor was never sold on retail shelves. This processor was not available for OEMs. This processor was never sold in any Windows based workstations. This processor wasn't listed in the promotional material for the E5-16xxV2 series. Intel was getting ready to promote X99 where they would introduce desktop/workstation 8 core/16 threads (also with 25MB cache, so this CPU is basically an early Haswell). The ONLY way to obtain this E5-1680V2 processor when it launched was through a purchase of a 2013 Intel based MAC PRO. It is clear that Intel was sort of "hiding" this CPU as they wanted everyone to focus on the upcoming Haswell.

Wow, congrats ! ​

That X79-E WS starts to look smaller with all those other components in there - good thing it as 72 instead the usual 40 PCIe lanes for that gen. BTW, do you know what the full power draw at the wall is when under full storage i/o load ?

Thank you, I am really happy with how it came out. I was able to get the fan situation sorted last night also by using a certain version of ASUS AI Suite II that I found. I have fan curves configured for all fans now .

I also did some burn-in testing. While running IOMeter on the HHD array I fired up the AIDA64 Burn-in test to stress the CPU/MEMORY/CACHE and I was able to pull about 450W from the wall. This test does NOT use the GPU so it could increase theoretically. I doubt transcoding is going to use a ton of power but I would say the server is going to use between 300W and 400W under typical "real world" load. I will certainly report actual transcoding speeds when I get to that point (last step).

Fantastic build completion ! Nice work, looks like a great rig. As for EPU, disable that puppy as that relates to power savings I do believe. If power saving is a goal then leave it as it is, but if you are looking at maximum performance I tend to disable it.

Thank you very much. Yeah power savings is kind of one of my goals since this will be running 24x7... I donate enough to Mr. Electric already lol.. I am not expecting magic but it appears to be a feature specific to ASUS that is "above and beyond" your typical power savings. I am assuming that would mean same performance, less power but I haven't really looked into EPU that much yet other than a thought of it possibly saving me money. I'm going to hold off until everything is fully up and running before I do any testing as I might need some real world tasks to stress things. If you're saying that what it really does is reduce performance in favor of power savings then yeah, I don't want any of that

​

​

​

​

The HDDs when not cached should have performed better methinks... I wonder if the overall performance is limited to the speed of the SSD in the CacheCade.... Hmmm.. The performance looks about what I would expect from a single fast SSD.

Quoting myself like a champ

My spidey senses were correct. The performance was WAY lower than it should have been. It was indeed was bottlenecked by the single SSD. I wiped out the array and this time left the SSD out and did not enable cachecade. Notice the performance difference from the old scores above and now. In addition, since I now had an extra SSD installed I made an Acronis image of the O/S array, wiped that out also and added the 4th SSD. I got a little boost in performance there also (array was still initializing).

Note: Cache size below refers to CONTROLLER cache.

The downside of cachecade 2.0 (that I've now discovered):

(1) If the sequential performance of the SSD(s) being used is less than the total performance of the HDD array, then the SSD will become the bottleneck.

(2) If you lose the cache'd drive then you cannot access the rest of the array (I also did not know this). This means that a RAID 1 mirror would be a minimum for redundancy but also in order to exceed the performance of (10) 7,200rpm enterprise class drives (256MB cache each) I would need to use at least 4 or more likely 5 SSDs in RAID 5.

Note: Notice the (10) drives actually write faster than the (4) SSDs? Only with sequential of course..

Note2: I also realize that I may have not even been able to take full advantage of cachecade 2.0 since I'm not running a database, I did it because "I could" and to live up to my middle-name (overkill) lol.

Note3: You can now see why I am such a big fan of these LSI/AVAGO 9271-8i controllers. (10) HDD drives @ 150MB/s each = 1.5GB/s and I'm reaching that maxium threshold on the writes. This is excellent efficiency and usage of those drives either solid state or spinning. (4) SSD drives @ 550MB/s each = 2.2GB/s and again we can see the performance reaching just below that mark. If you are curious yourself, just don't forget the active cooling for them

Some info:

(1) The reason AU Suite III would not install is because they release these apps programmed for specific motherboard sensors. You MUST install an older one that supports the sensors in your board. In my case I needed to install an older ASUS AI Suite II which did not work well so I ended up with Argus monitoring software instead and it works great so far.

(2) @J7SC_OrionJ7SC_Orion With stock clocks I'm @ roughly 310W at the desktop idle.. When I fired up I/O meter and some other background copy jobs the most I saw was around 350W. If the CPU/GPU was being pushed at the same time it would be higher. I am curious how much power draw I'll see with Plex transcoding using both the cpu and gpu for different things.

I will experiement with EPU also if it doesn't give me too much trouble.

Wow, congrats !

That X79-E WS starts to look smaller with all those other components in there - good thing it as 72 instead the usual 40 PCIe lanes for that gen. BTW, do you know what the full power draw at the wall is when under full storage i/o load ?

I'm going to further investigate a few things related to storage and while I am at it I will test this and report back.

I decided to step back and leave everything stock at this time. I want to see how it performs before I go through the trouble of overclocking and dealing with additional heat. I already have an idea of the power with and without the overclock and a broad estimate would be roughly 50W-100W additional with the overclock applied.

In other news, I'm having an issue installing ASUS AI Suite III for Windows 10 x64... Apparently the latest updates keep breaking it. I found the newest (??) .60 version and it also refused to install. This is a problem because I was depending on using that application to manage my fan curves. I tried SpeedFan and it doesn't detect 2 of my fans . I just found FanExpert 4 standalone install so fingers crossed. If anyone has ever worked with PWM fans you'll know right away why a controller of some sort is needed, the constant reving up and down would drive anyone crazy

Here are some quick benchmarks on the storage.

C: = LSI 9271-8i / (3) SAMSUNG 850 PRO RAID 5

D: = LSI 9271-8i / (10) SEAGATE 7200 RPM RAID 5 + (1) SAMSUNG 850 PRO CACHECADE 2.0

Benchmarks performed with a test file size both smaller and larger than the controller cache size. This will show maximum performance within both of these scenarios.

The HDDs when not cached should have performed better methinks... I wonder if the overall performance is limited to the speed of the SSD in the CacheCade.... Hmmm.. The performance looks about what I would expect from a single fast SSD.

All done!

Final build pics

(all done with the hardware anyway, I still have heat testing, overclocking, Plex performance and likely some BlueIris testing.)

The new shorter SFF-8087 cables were perfect! All that clutter gone! The 60mm Delta fan was also perfect. I am using PWM on all but one fan (it supports PWM but I just have it maxxed).

The Delta fans were a good choice because there is a lot of heat to remove from this case but they are doing well. The power is averaging just above 400W @ idle (slight overclock) so when I am finished overclocking it will likely be more (if the cooler is strong enough).

PS. For the curious, no the HD bay fan doesn't keep the controllers cool. The LSI 9271's need direct cooling but they are nice and cool now

 

I've been thinking of home surveillance options and would be interested if you went down this route in seeing what you do.

I will continue to update this thread with my testing

Oh nice....but now this gets me thinking...Stop giving me ideas that involve money haha.

ROFL

No better time to have a robust security system

Just be glad you don't have the "overkill" bug

I've been thinking about which additional roles this "server" could play... I could of course use it as a file server which I will definitely do to some extent but I'm also thinking about possibly using it to control my security cameras using Blue Iris. I understand BI can use the GPU. Just an idea at this point but I figured I would share to see if anyone else has built thier own NVR. My current NVR is not PC based.

Psh, flight sim 2020 flight schmim. You're a retro junkie now.

 

Where I come from, we play Wing Commander. !!!!

 

Hehehe

 

M DISCS are supposed to last 1000 years, be sure to burn your memoirs and bury it for some future person to find, include plenty of pictures of our modern stone age computers (and a plain text explanation to read the data in your picture format of choice ((and a hard copy explaining binary to ascii encoding)) in case of nuclear war)

Wing Commander... oh those were the days! :yessir:

Long live Windows 7!

Ordered some shorter SFF-8087 cables because there is just way too much cable to put somewhere... These are 9" long so hopefully just right.

Ended up using the used Delta, it was practically brand new. Armor installed.

 

Been here before when it comes to Windows and drivers, it is one of the smallest but most annoying snags as it can stop you in your tracks ! Complete plain. That being said I would imagine you can achieve what you need to under client Windows 10 ? That or use Windows 10 to test and bench the system and then you can look into getting server 2019 now...or later when price is more agreeable.

 

What I also find very interesting is that Windows 7 (released in 2009) drivers are provided for Turing... but there are no compatible drivers for Server 2016... yet apparently Pascal and below will work just fine in Server 2016...

Well, this is an interesting one I didn't see coming...

After reviewing driver/hardware compatibility I decided to go with Windows Server 2016 Standard. It is based on Windows 10 and the license was cheap. I loaded the ISO up as a bootable USB drive and away I went.. I got almost all the drivers installed, some disk benchmarks, everything was going great until it came time to install the video drivers.. The drivers were not having it.. I tried standard vs DCM drivers, I tried studio drivers, I tried older drivers... until I finally came a across a post that seemed to say that TURING GPUs require a higher build than 1607... and since we're talking about the Windows 10 "server" flavor, that turned out to translate into "I must have Server 2019".... so in server land the only way to get the latest updates is to pay 2x where on the desktop the same updates are free... gotcha...

<FACEPALM>

So, it looks like I'll need to go ahead and load up Windows 10 instead. There I go wanting to make a server a server lol...

EDIT: Winderz 10 is installed/updated (and shutup) and all drivers are installed. I've also got the new CPU fan installed. I can start doing some overclocking now

Side Note: I am pretty impressed with the rebuild time (initialization). It is a RAID 5 array and the rebuild is just about done, 26-27 hours total... which if you think about it for 10x4TB spinners that's not too bad..

 

LSI controllers...or any controller for that fact that is dealing with the traffic of multiple devices tend to run on the hotter side. It still surprised me that we do not see these more actively cooled, or at the very leas a more capable passive cooling solution.

Exactly... I mean it is not like they MIGHT overheat, they WILL...