VMware Horizon Community
Matrix__1970
Enthusiast
Enthusiast
Jump to solution

Storage groups cannot copy file

Hi all,

I have 2 sites with AppVolumes 2.14. We have on these two vSAN datastores. We have created another NFS volumes that on the first site and a storage group with vSAN_site_1 and NFS_Original. This second NFS volumes is not attachable and it is mirrored on the second site on a volumes in Read Only mode (named NFS_Mirrored_Copy).

In the second site, we have an analog storage group with vSAN_Site_2 and NFS_Mirrored_Copy.

During replication on second site, we have errors like this in  AppVolumes:

[2019-01-05 12:33:47 UTC] P1008DJ89466 ERROR   RvSphere: vSphere operation failed: Cannot complete file creation operation.

[2019-01-05 12:33:47 UTC] P1008DJ89466  INFO   RvSphere: Failed to copy "[vSAN_Site_1] cloudvolumes/apps/Microsoft!20!Project!20!2013!20!32bit!20!ital.vmdk" to "[NFS_Mirrored_Copy] cloudvolumes/apps/Microsoft!20!Project!20!2013!20!32bit!20!ital.vmdk"

Is this correct? Only the vSAN DS will attach appstacks. The NFS_Mirrored_Copy must be in R/W mode? And... the new AppStacks are always on NFS_Mirrored_Copy: this should acts as source for copy or I'm wrong?

Thanks to all!

Matrix

Reply
0 Kudos
1 Solution

Accepted Solutions
hockeyguyin714
VMware Employee
VMware Employee
Jump to solution

So in your design you are assuming that the source never changes. AppStacks metadata file has to be updated as it tracks items like number of attachments, OS versions etc. The Data in the VMDK file does not change as it is attached as independent non persistent.  Your NFS will not be able to replicate that change because it is read only. It will update on the VSAN side but fail to update or replicate on the NETAPP side.  App Volumes does not have a source so it believes all storage luns are equal depending on which one has the latest data that is what it determines to overwrite.  The typical design recommendations for storage group is a single NFS shared between two sites so App Volumes would replicate via a Storage Group or multiple Storage Groups.  You are introducing something the product was not technically designed to account for.  I would recommend reviewing -  VMware App Volumes Deployment Considerations: VMware App Volumes 2.12 and Later

Another option you could consider is manual replication after creating the AppStack on site B storage group to avoid the errors being ongoing.

View solution in original post

Reply
0 Kudos
7 Replies
sjesse
Leadership
Leadership
Jump to solution

How are your storage groups set up in app volumes in both sites? In your second site do you have the nfs datastore selected in the correct location where its not trying to copy the appstacks to instead of being a source of where to copy the appstack from?

Reply
0 Kudos
hockeyguyin714
VMware Employee
VMware Employee
Jump to solution

Both NFS stores should be set to non attachable in App Volumes - Storage.   The storage groups should be set to VSANSiteA+NFSSiteA in site A.  Site B should VSANSiteB+NFSSiteB in site B.  The replication sounds like you are doing this in some other function from NSFA to NSFB outside of App Volumes. If it is read only any updates will fail to NSF that is in read only mode.  Which is expected behavior as you have blocked writes so we won't even be able to modify the metadata file which we use for tracking and updates.  I would consider changing this setting of read only due to some issues you may have on SiteB on updating files.   The storage group can be setup to auto replicate from VSANSiteA to NFSSiteA.  The same could be done on SiteB as well. 

Reply
0 Kudos
Matrix__1970
Enthusiast
Enthusiast
Jump to solution

Thank you for your answer and to the others who have helped me.

The behaviour of this scenario is that in Site B we wouldn't create AppStacks, but use that in SiteA. The idea is that we copy from SiteA (NFS on Site A) these AppStacks on NFSSIteB and, with the replication of Storage groups, use these AppStacks that will be copied in vSANSiteB. We don't know why vSANB try to copy AppStacks on NFSSiteB. This is our goal. for this we think that NFSSiteB can be in read only, because it acts only as source of these copies.

On SiteA the replication is automatic. The copy from NFSSiteA to NFSSiteB is via Netappa Mirror. Storage group replication can be forced in only one direction, for example? Why vSAN try to write to NFS that could be the source?

Thank you very much to all. Please, let me know.

Matrix

Reply
0 Kudos
Matrix__1970
Enthusiast
Enthusiast
Jump to solution

Hello,

what do you mean when you say "In your second site do you have the nfs datastore selected in the correct location where its not trying to copy the appstacks to instead of being a source of where to copy the appstack from?"?

I have just added the two datastore on the sotrage group and marked the NFS as not attachable.

Thank you

Matrix

Reply
0 Kudos
hockeyguyin714
VMware Employee
VMware Employee
Jump to solution

So in your design you are assuming that the source never changes. AppStacks metadata file has to be updated as it tracks items like number of attachments, OS versions etc. The Data in the VMDK file does not change as it is attached as independent non persistent.  Your NFS will not be able to replicate that change because it is read only. It will update on the VSAN side but fail to update or replicate on the NETAPP side.  App Volumes does not have a source so it believes all storage luns are equal depending on which one has the latest data that is what it determines to overwrite.  The typical design recommendations for storage group is a single NFS shared between two sites so App Volumes would replicate via a Storage Group or multiple Storage Groups.  You are introducing something the product was not technically designed to account for.  I would recommend reviewing -  VMware App Volumes Deployment Considerations: VMware App Volumes 2.12 and Later

Another option you could consider is manual replication after creating the AppStack on site B storage group to avoid the errors being ongoing.

Reply
0 Kudos
Matrix__1970
Enthusiast
Enthusiast
Jump to solution

Hello,

thank you for your answer. I think you mean that, even if we add (via copy/mirror) the NFS volume on Site1 to Site 2 (mirrored copy in read only), we must set in R/W the NFS on Site B because the replication process ivolves always write operations on all volumes in storage group. Correct? it's strange because logs (and vCenter) says me that vmdk files are unable to be  written on NFS (not only .metadata). But these messages can be generic.

However, I have the same error if I do manual replication via Storage Group.

Have a good day... if you wnat to explain me these doubts, I'll appreciate it.

Matrix

Reply
0 Kudos
hockeyguyin714
VMware Employee
VMware Employee
Jump to solution

Correct. So App Volumes Manager will consider the entire AppStack including the metadata as part of the AppStack.  The Metadata changing is typically what causes the replication to trigger in both directions for the Flat, Descriptor and Metatdata files.  The copy will include all 3 pieces not just 1.

Reply
0 Kudos