Glad to hear you are looking to configure a more reliable set-up.
"I am about to buy 2 new servers with much higher RAM frequency, better CPU, all nVME SSDs etc."
Please ensure all relevant components (vSAN controller, Cache-tier and Capacity-tier devices) are listed on the vSAN HCL and for the intended purpose (e.g. not all NVMe/SSDs are certified for All-Flash Cache-tier), a colleague of mine created this handy tool for checking:
Otherwise you can reference the current HCL listings:
"So my question is instead of adding 2 new servers to an old vSAN setup and create bottlenecks, wouldn't it be better to create two "two node vSAN cluster" and transfer my VMs that I want to have greater speed to the new vSAN environment?"
Due to vSAN keeping data in sync between data-replicas on different nodes, having some slow nodes and some fast nodes may result in poor or inconsistent performance - at the least it won't perform as well as it's fastest nodes.
What you could do here is have the 2 new nodes run as a DirectConnect (e.g. no switch between them) configuration and then use one of the older servers to either directly use as a physical Witness or just put VMFS on it and run the Witness as an appliance VM on this host. Witnesses licenses are free and the host they are running on doesn't need to be licensed for vSAN nor have vSAN HCL certified components
"Also if something happens I retrieve from a NAS back up to any of two!"
Yes, having backups of your data is important and I would advise configuring a solution for this purpose - A secondary VMFS back-up location could also be configured on the server being used as Witness/running the Witness VM
I am going to have
2x Dell 1.6TB, NVMe, Mixed Use Express Flash, 2.5 SFF Drive, U.2, P4610
1x Dell Express Flash NVMe P4800X SSDPE21K375GAT 375 GB, 2.5” U.2
With an AMD Epic, I believe I am fine. I can't wait to see the speed of those little monsters
Isn't the idea of two vSAN running side by side a good idea? The old server had two CPUs.
The two Servers are going to have one CPU each...
With regard to HCL, should be fine, though please ensure the variant of P4610 is on the vSAN HCL - this can be clarified from the part number and/or product ID of the specific device in question e.g.:
"Isn't the idea of two vSAN running side by side a good idea?"
Unsure what you mean - do you mean running new cluster as 2-node (+ Witness) cluster or running 2 vSAN clusters (new one + old one)?
"The old server had two CPUs.
The two Servers are going to have one CPU each..."
If they are relatively old servers/CPUs then decent AMD Epycs will likely make up for this - note as well that as vSAN is licensed per socket that this may be more cost-effective.
I am thinking of converting the three node vSAN setup that I currently have (old servers) in a two node vSAN setup.
The two new servers when they arrive I am going to make a two node vSAN setup again.
So I am going to have two vSAN envionments, one new (better) and one old. Each one will observe the other (I do not know if it is possible)
I am going to distribute VMs according to needs to two vSAN setups. They are going to have a shared NAS backing up the VMs. If something of one of the two setups goes wrong I am retrieving from NAS to a working environment.
Does this make sense?
"Each one will observe the other (I do not know if it is possible)"
Unfortunately it is unsupported to have two 2-node clusters and have the Witnesses for each on the other cluster - I will be honest though and say on the scale of unsupported things, you could configure things a lot worse than doing this.
A supported alternative would be to fix the 3rd host from the original cluster and use that to host the Witness appliances from both clusters.
"I am going to distribute VMs according to needs to two vSAN setups."
Yes, this is one of the main points of being able to have multiple smaller vSAN clusters with different hardware capabilities and thus you can tier them and put less storage-intensive workloads on the lower spec cluster.
By reading this article, I think I am able to install vSAN Witness Appliance in an ESXi by iteself.
So suppose I have:
Old servers: Server 1, Server 2 -> One vSAN Disk group, one disk for logging and running vSAN Witness Appliance for new servers
New servers: Server 3, Server 4 -> One vSAN Disk group, one disk for logging and running vSAN Witness Appliance for old servers
Wouldn't this work? Or it is not official supported?! It isn't that I do not want to power up the broken server, but let's try to be more friendly to the environment