HP Z840 36C/72T Workstation Build

[Storm-Chaser]

2 minutes ago, Avacado said:

There really isn't one. I had literally no idea that a socket could have sub variants. Given i'm at work and piecing together research last minute on a hunch. I have never encountered that before 

Guess you are right and you cant just change out the mounting mechanism to make it work. Same thing happens with LGA 2011. The different, later variants such as 2011-1 and LGA 2011-2 were not backwards compatible. 

 

I've got a spare z820 if you want to tinker with that. It has two rare 2673 v2 eight core processors and 64GB of DDR3 1600MHz RAM. 

 

Pay me for shipping and it's all yours (just FYI its so heavy, it could be used as a boat anchor). Only issue with the rig is it has one bad memory slot on CPU #1. I have not troubleshot that much so it may be a fixable, really don't know short of reseating and swapping ram, but the ram is confirmed good.

 

You are going to have to provide GPU and SSD. 

[Avacado]

20 minutes ago, Storm-Chaser said:

Guess you are right and you cant just change out the mounting mechanism to make it work. Same thing happens with LGA 2011. The different, later variants such as 2011-1 and LGA 2011-2 were not backwards compatible. 

 

I've got a spare z820 if you want to tinker with that. It has two rare 2673 v2 eight core processors and 64GB of DDR3 1600MHz RAM. 

 

Pay me for shipping and it's all yours (just FYI its so heavy, it could be used as a boat anchor). Only issue with the rig is it has one bad memory slot on CPU #1. I have not troubleshot that much so it may be a fixable, really don't know short of reseating and swapping ram, but the ram is confirmed good.

 

You are going to have to provide GPU and SSD. 

I really appreciate that. But I'm going to pass. They need to more clearly label the products being sold. If it's LGA 3647 or LGA 3647-1, that needs to be evident. I know I can get a new intel 3647 for around $400 from ebay, but I want dual socket. If I can find a dual socket model, I might keep everything and wait a few years to hop on it. But I don't think a dual socket board exists for the Phi. Hey, at least I learned a bunch from this. 

Edited April 22, 2022 by Avacado

[tictoc]

3 hours ago, Avacado said:

I really appreciate that. But I'm going to pass. They need to more clearly label the products being sold. If it's LGA 3647 or LGA 3647-1, that needs to be evident. I know I can get a new intel 3647 for around $400 from ebay, but I want dual socket. If I can find a dual socket model, I might keep everything and wait a few years to hop on it. But I don't think a dual socket board exists for the Phi. Hey, at least I learned a bunch from this. 

 

I don't belive any dual socket boards ever hit the consumer market.  Most of the high density Phi clusters were/are running on blades, so 4 boards/processors per 2U rack space.

[Avacado]

33 minutes ago, tictoc said:

 

I don't belive any dual socket boards ever hit the consumer market.  Most of the high density Phi clusters were/are running on blades, so 4 boards/processors per 2U rack space.

Im down to buy a blade.

[Fallenzoul]

@Storm-Chaser   hey FYI and for reference if someone else has a z840 avoid Updating the BIOS to M60_0259(Original was M60_0252), it will brick the Motherboard, i had a dead Z840, so i just brought a new/Used Z840 Motherboard from ebay, everything was working all the memory channels, both cpu sockets, but i was planning on using the E5-26xxV4 so i thought lets update the BIOS just to be safe and get all the new patches.

 

 

 

Both Motherboards, the Upper one was the one i brought from Ebay to replace the Original Bottom one, that was dead

 

 

 

 

This is the version of the BIOS that bricked the New motherboard

 

Bad idea, it worked fine until i turned off the machine, after that i cannot make the computer POST, i tried everything on the HPE Support page, using the failbios set up, with the jumpers and a Recovery USB and nothing works, so i contacted HP(I know this Products are already EoL but i still tried) had 3 chats with different Support agens, all confirmed that its a known issue, but they still doesnt care to update the BIOS Driver page, or the HP Assitant to use a different rom than the M60_0259, since is EoL product i guess they dont support/care for it anyways.

 

 

 

So please im looking for an older BIOS, i just brought a BIOS Programmer and i hope i can bring the Motherboard back to Life

 

 

@Storm-Chaser i just saw that you are currently using a M60_0256 BIOS, could you please back it up and send it to me? it will be of great help, since i had a rought 2 days trying to find an older version, but everywhere theres only the one that bricked mine(Already try it, and still it does not work).

 

[Storm-Chaser]

11 hours ago, Fallenzoul said:

@Storm-Chaser   hey FYI and for reference if someone else has a z840 avoid Updating the BIOS to M60_0259(Original was M60_0252), it will brick the Motherboard, i had a dead Z840, so i just brought a new/Used Z840 Motherboard from ebay, everything was working all the memory channels, both cpu sockets, but i was planning on using the E5-26xxV4 so i thought lets update the BIOS just to be safe and get all the new patches.

 

 

 

Both Motherboards, the Upper one was the one i brought from Ebay to replace the Original Bottom one, that was dead

 

 

 

 

This is the version of the BIOS that bricked the New motherboard

 

Bad idea, it worked fine until i turned off the machine, after that i cannot make the computer POST, i tried everything on the HPE Support page, using the failbios set up, with the jumpers and a Recovery USB and nothing works, so i contacted HP(I know this Products are already EoL but i still tried) had 3 chats with different Support agens, all confirmed that its a known issue, but they still doesnt care to update the BIOS Driver page, or the HP Assitant to use a different rom than the M60_0259, since is EoL product i guess they dont support/care for it anyways.

 

 

 

So please im looking for an older BIOS, i just brought a BIOS Programmer and i hope i can bring the Motherboard back to Life

 

 

@Storm-Chaser i just saw that you are currently using a M60_0256 BIOS, could you please back it up and send it to me? it will be of great help, since i had a rought 2 days trying to find an older version, but everywhere theres only the one that bricked mine(Already try it, and still it does not work).

 

So I went back to my build log and it looks like I updated from v2.56 to v2.58 about a week after I posted the picture above. Current configuration:

I have the v2.58 bios download which should still work for your setup. I'll see if I can attach it to this post in just a minute.
 

[Storm-Chaser]

Okay I zipped the BIOS file and just for reference this is version 2.58 which is fine and did not brick my z840.

 

Should be attached to this post. 

HP Z440Z640Z840 Workstation System BIOS - sp137086.zip

[Fallenzoul]

1 hour ago, Storm-Chaser said:

Okay I zipped the BIOS file and just for reference this is version 2.58 which is fine and did not brick my z840.

 

Should be attached to this post. 

HP Z440Z640Z840 Workstation System BIOS - sp137086.zip 21.3 MB · 0 downloads

Hey thank you so much, I will try to use it with the usb recovery tool, if that does not work I will wait for the reprogrammer.

 

Still thanks for the BIOS file.

[ENTERPRISE]

Honestly, you people have me looking into this for a FreeNAS project. SIGH. All your faults lol.

[ENTERPRISE]

Edit*** 

 

So I have been bitten by the bug. Looking to get one of these to consolidate my NAS & Plex requirements into a single machine, plus any other services I feel like.

 

I am looking at this configurator here:  https://www.bargainhardware.co.uk/hp-z840-tower-a-configure-to-order

 

Looks like I can get a good system .

 

I have configured one as per the below

 

1 x HP Z840 (Grade A+) - Windows 10 Pro
2 x Intel Xeon E5-2696 V4 - 22-Core 44-Threads 2.20GHz (3.70GHz Boost, 55MB Cache, 150W TDP)
2 x HP Z840 Heatsink
1 x HP Z-Series 2nd CPU Fan
8 x 8GB - DDR4 2666MHz (PC4-21300, 2Rx8) - TBD
1 x 512GB - NVMe SSD - Major Brand
1 x HP Z Turbo Drive G2 - NVMe Adapter (New)
4 x HP Z Series LFF HDD Caddy
1 x HP Z840 1125W PSU

 

However I am not great at RAM and the RANKS side of things. I am going to opt for 64GB as I do not see the benefit for my use case to go any higher. The RAM configuration options are below, what would be the best performance wise ?

 

 

Also, What is the latest BIOS for this system that will not kill it ? I saw mentions of some hubbub with a BIOS bricking these units, for obvious reasons I do not want to go there haha.

 

Thanks,

E

 

 

[Storm-Chaser]

On 14/05/2022 at 23:18, Fallenzoul said:

Bad idea, it worked fine until i turned off the machine, after that i cannot make the computer POST

This is a long shot, but if all else fails you could try doing a cold boot and letting it sit for a few minutes in hopes that it POSTS eventually. Reason I am recommending this is because some changes in the BIOS will cause my z840 rig to take over a minute just to post the first time they are loaded upon reboot. 

[Storm-Chaser]

On 17/05/2022 at 05:17, ENTERPRISE said:

Edit*** 

 

So I have been bitten by the bug. Looking to get one of these to consolidate my NAS & Plex requirements into a single machine, plus any other services I feel like.

 

I am looking at this configurator here:  https://www.bargainhardware.co.uk/hp-z840-tower-a-configure-to-order

 

Looks like I can get a good system .

 

I have configured one as per the below

 

1 x HP Z840 (Grade A+) - Windows 10 Pro
2 x Intel Xeon E5-2696 V4 - 22-Core 44-Threads 2.20GHz (3.70GHz Boost, 55MB Cache, 150W TDP)
2 x HP Z840 Heatsink
1 x HP Z-Series 2nd CPU Fan
8 x 8GB - DDR4 2666MHz (PC4-21300, 2Rx8) - TBD
1 x 512GB - NVMe SSD - Major Brand
1 x HP Z Turbo Drive G2 - NVMe Adapter (New)
4 x HP Z Series LFF HDD Caddy
1 x HP Z840 1125W PSU

 

However I am not great at RAM and the RANKS side of things. I am going to opt for 64GB as I do not see the benefit for my use case to go any higher. The RAM configuration options are below, what would be the best performance wise ?

 

 

Also, What is the latest BIOS for this system that will not kill it ? I saw mentions of some hubbub with a BIOS bricking these units, for obvious reasons I do not want to go there haha.

 

Thanks,

E

 

 

 

That's a great price for a z840! Good find. Really interesting idea. Looking forward to see how it works out.

 

IIRC the V4 chips have a maximum supported memory speed of 2400MHz. So your 2666 kit will likely default back to this speed.

I think dual rank memory will give you about a 3-5% increase in memory bandwidth when compared to single rank. But it's more expensive. 

In terms of memory configuration, I would recommend populating all 16 slots. This will allow for maximum performance as it provides you with 8 channel effective memory. 

The minimum supported DIMM size is 4GB, so I run 64GB in total in this same configuration. Since the 4GB modules are cheaper, this is a good way to go about it. 

 

Right on point with processor selection. Thats best in class for the v4 series. I also have a 145W xeon and I noticed upgrading to liquid metal helped with temps a bit, the cooling system is more than capable of 300W but it's always nice to have lower temps in general. 

 

 

EDIT: I would go for the water cooling if that is an option. Very rare to see factory liquid cooling. 

 

 

 

 

 

Edited May 20, 2022 by Storm-Chaser

[Storm-Chaser]

On 17/05/2022 at 05:17, ENTERPRISE said:

Also, What is the latest BIOS for this system that will not kill it ? I saw mentions of some hubbub with a BIOS bricking these units,

Most recent safe BIOS is v2.58, I zipped it up and uploaded to this thread, it's just a few posts up. 

[ENTERPRISE]

3 hours ago, Storm-Chaser said:

 

That's a great price for a z840! Good find. Really interesting idea. Looking forward to see how it works out.

 

IIRC the V4 chips have a maximum supported memory speed of 2400MHz. So your 2666 kit will likely default back to this speed.

I think dual rank memory will give you about a 3-5% increase in memory bandwidth when compared to single rank. But it's more expensive. 

In terms of memory configuration, I would recommend populating all 16 slots. This will allow for maximum performance as it provides you with 8 channel effective memory. 

The minimum supported DIMM size is 4GB, so I run 64GB in total in this same configuration. Since the 4GB modules are cheaper, this is a good way to go about it. 

 

Right on point with processor selection. Thats best in class for the v4 series. I also have a 145W xeon and I noticed upgrading to liquid metal helped with temps a bit, the cooling system is more than capable of 300W but it's always nice to have lower temps in general. 

 

 

EDIT: I would go for the water cooling if that is an option. Very rare to see factory liquid cooling. 

 

 

 

 

 

Thanks for your help bud, 

 

I agree with what you are saying, however it looks like I can only get 16 modules of the 4GB 2133Mhz, I guess I hesitate as it is the slowest of the bunch, I have no idea if have it in 8 Channel effective would offset the fact it is slower ram ?

[Storm-Chaser]

1 hour ago, ENTERPRISE said:

Thanks for your help bud, 

 

I agree with what you are saying, however it looks like I can only get 16 modules of the 4GB 2133Mhz, I guess I hesitate as it is the slowest of the bunch, I have no idea if have it in 8 Channel effective would offset the fact it is slower ram ?

You are looking at about 125GB/s read speeds with 2133MHz memory, if that's enough than just go with it. Another option would be to get it loaded with just one or two memory modules, sell them and buy another 16 module kit online thats going to run at the full 2400MHz speed. 

 

With the 2400MHz memory you are probably looking at 5-10% bandwidth improvement, which works out to like 130GB/s or so. So you have some options. 

[bonami2]

On 17/05/2022 at 05:17, ENTERPRISE said:

Edit*** 

 

So I have been bitten by the bug. Looking to get one of these to consolidate my NAS & Plex requirements into a single machine, plus any other services I feel like.

 

I am looking at this configurator here:  https://www.bargainhardware.co.uk/hp-z840-tower-a-configure-to-order

 

Looks like I can get a good system .

 

I have configured one as per the below

 

1 x HP Z840 (Grade A+) - Windows 10 Pro
2 x Intel Xeon E5-2696 V4 - 22-Core 44-Threads 2.20GHz (3.70GHz Boost, 55MB Cache, 150W TDP)
2 x HP Z840 Heatsink
1 x HP Z-Series 2nd CPU Fan
8 x 8GB - DDR4 2666MHz (PC4-21300, 2Rx8) - TBD
1 x 512GB - NVMe SSD - Major Brand
1 x HP Z Turbo Drive G2 - NVMe Adapter (New)
4 x HP Z Series LFF HDD Caddy
1 x HP Z840 1125W PSU

 

However I am not great at RAM and the RANKS side of things. I am going to opt for 64GB as I do not see the benefit for my use case to go any higher. The RAM configuration options are below, what would be the best performance wise ?

 

 

Also, What is the latest BIOS for this system that will not kill it ? I saw mentions of some hubbub with a BIOS bricking these units, for obvious reasons I do not want to go there haha.

 

Thanks,

E

 

 

 

 

If you want to go custom aliexpress got some dual socket x99 with ram as combo too

C$ 441.25  JGINYUE X99 D8 Dual CPU Motherboard LGA 2011-3 Kit Set With 2*E5 2650 V3 CPU 32GB=4x8GB DDR4 ECC Memory Support Eight Channels
https://a.aliexpress.com/_mLVaygI

[Storm-Chaser]

9 hours ago, bonami2 said:

 

 

If you want to go custom aliexpress got some dual socket x99 with ram as combo too

C$ 441.25  JGINYUE X99 D8 Dual CPU Motherboard LGA 2011-3 Kit Set With 2*E5 2650 V3 CPU 32GB=4x8GB DDR4 ECC Memory Support Eight Channels
https://a.aliexpress.com/_mLVaygI

But it's never going to be the same experience you will get from a z840. Plus it has less memory slots. 

[ENTERPRISE]

Thanks for the advice guys, I can confirm I purchased the below spec. 

 

 

I wanted all DIMM slots populated, but this was not possible with 4GB modules to get 64GB total. Actually that is a lie, I could have, but meant using slower 2133Mhz modules. However they had 8GB modules at 2400Mhz. So Alas my config is a total of 128GB lol. I will use of it for caching that is for sure. 

 

This machine will enable me to go from 3 machines to 1. But at a little bit of a cost :/.

[Storm-Chaser]

6 minutes ago, ENTERPRISE said:

Thanks for the advice guys, I can confirm I purchased the below spec. 

 

 

I wanted all DIMM slots populated, but this was not possible with 4GB modules to get 64GB total. Actually that is a lie, I could have, but meant using slower 2133Mhz modules. However they had 8GB modules at 2400Mhz. So Alas my config is a total of 128GB lol. I will use of it for caching that is for sure. 

 

This machine will enable me to go from 3 machines to 1. But at a little bit of a cost :/.

Looking good my friend! The z840 does not disappoint. 

[bonami2]

40 minutes ago, Storm-Chaser said:

But it's never going to be the same experience you will get from a z840. Plus it has less memory slots. 

Am just not a fan of hp haha. Those setup are great.

 

For some reason i like those unbranded fire risk setup . That one look cool but i would be afraid to receive an empty box or a fire hazard.

 

Ordered some x79 without issue but driver where a nightmare for usb and audio.

 

 

Anyone here with 128gb can try rosetta@home with python work unit?  For some reason i can run 20 vm on 64gb but cpu usage stay low so i wonder how those xeon would perform.

 

[Storm-Chaser]

7 minutes ago, bonami2 said:

Am just not a fan of hp haha. Those setup are great.

Yeah some of the lower level products from HP are questionable but their high end workstations (like this one) are very well designed and purpose built to handle anything you can throw at it. Active cooling on the ram modules as well. 

[bonami2]

10 hours ago, Storm-Chaser said:

Yeah some of the lower level products from HP are questionable but their high end workstations (like this one) are very well designed and purpose built to handle anything you can throw at it. Active cooling on the ram modules as well. 

I wonder why you are all going with those build? I sold my x79 stuff because my 12700k as 10 time the horsepower.

I would love a mini itx x99 or x299 

[Storm-Chaser]

15 minutes ago, bonami2 said:

wonder why you are all going with those build? I sold my x79 stuff because my 12700k as 10 time the horsepower.

I would love a mini itx x99 or x299 

Your 12700K has 10 times the HP of this?

[bonami2]

4 minutes ago, Storm-Chaser said:

Your 12700K has 10 times the HP of this?

 

 

Nah but x79.  Ram bandwith is insane on server setup

But cpu z is not really a real life comparison. If you look at passmark i do as much on 8 ht core and 4 non ht core as a single cpu

Task need to be multithreaded most software arent.

 

I saw cinebench benchmark tho and those dual cpu are insane! Not like x79

 

 

 

 

 

[Storm-Chaser]

due to the obscurity of this build I am doing quite well with hwbot submissions.