VMware {code} Community
Ratndeeep
Contributor
Contributor

When 2 vmdk files uploaded using vim25 package by yavijava, 2nd vmdk gets corrupted

Problem statement:

When 2 vmdk files uploaded using HTTP connection URL with the help of vim25 package bundled in yavijava using automation tool, 2nd vmdk last partition gets corrupted and hence VM OS didn't come up successfully and stuck at below message.

Assuming drive cache: space through

Assuming drive cache: space through

This behavior is not consistently happening and out of 5 attempts we re getting failure for one attempt .

When same vmdk files which are part of OVA is installed using directly Esxi host manually, VM gets successfully installed and coming up and functional.

Code flow details.

1. Automation tools used for VM deployment uses https://github.com/yavijava/yavijava/tree/6.0.02 java library and this library internally using https://github.com/yavijava/yavijava/tree/6.0.02/src/main/java/com/vmware/vim25 package.

2. CreateVM.java which is implemented in automation tool which has a core logic to upload VMDFK files and create VM by making use of vim25 stub classes which are avilable in package(yavijava/src/main/java/com/vmware/vim25) from yavijava library as mentioned in #1

3. Please refer java class CreateVM.java

public void createVMAndUploadVmdk() throws Exception

which internally calling method

previousFileWritten = uploadVmdkFile(ovfFileItem.isCreate(), absoluteFile, urlToPost, bytesAlreadyWritten, totalBytes);

4. Please refer below uploadVmdkFile method:

method arguments details:

CHUCK_LEN = 64 * 1024;

put = false (so this is POST request call)

diskFilePath = fully quialfied vmdk file path
e.g.
/swlibrary/CM-Simplex-010.1.0.0.974-e70-0_EXTRACTd34e049e-a36c-43d9-9def-383e7895ce30/CM-Simplex-010.1.0.0.974-e70-0d34e049e-a36c-43d9-9def-383e7895ce30_EXTRACT/system.vmdk
/swlibrary/CM-Simplex-010.1.0.0.974-e70-0_EXTRACTd34e049e-a36c-43d9-9def-383e7895ce30/CM-Simplex-010.1.0.0.974-e70-0d34e049e-a36c-43d9-9def-383e7895ce30_EXTRACT/Var_Disk.vmdk

urlStr = prepared URL for vmdk file upload.
e.g.
https://tma182.avaya.com/ha-nfc/522b983c-093c-8f1c-1102-648465bb5136/cm-115-0.vmdk
https://tma182.avaya.com/ha-nfc/522b983c-093c-8f1c-1102-648465bb5136/cm-115-1.vmdk

bytesAlreadyWritten = 0

totalBytes = addition of both vmdk files in Bytes e.g. 2136300544

private long uploadVmdkFile(boolean put, String diskFilePath, String urlStr, long bytesAlreadyWritten,
long totalBytes) throws Exception {
debugLogger.info("Disk File Path" + diskFilePath);
debugLogger.info("Url to POST - "+ urlStr);
HttpsURLConnection conn = (HttpsURLConnection) new URL(urlStr).openConnection();
conn.setDoOutput(true);
conn.setUseCaches(false);
conn.setChunkedStreamingMode(CHUCK_LEN);
conn.setRequestMethod(put ? "PUT" : "POST"); // Use a post method to write the file.
conn.setRequestProperty("Connection", "Keep-Alive");
if (put) {
conn.setRequestProperty("Overwrite", "t");
}
conn.setRequestProperty("Content-Type", "application/x-vnd.vmware-streamVmdk");
conn.setRequestProperty("Content-Length", Long.toString(new File(diskFilePath).length()));

BufferedOutputStream bos = new BufferedOutputStream(conn.getOutputStream());

BufferedInputStream diskis = new BufferedInputStream(new FileInputStream(diskFilePath));
int bytesAvailable = diskis.available();
int bufferSize = Math.min(bytesAvailable, CHUCK_LEN);
byte[] buffer = new byte[bufferSize];

long totalBytesWritten = 0;
try {
while (true) {
int bytesRead = diskis.read(buffer, 0, bufferSize);
/*
* debugLogger.info("byterAlreadyWritte " + bytesAlreadyWritten); debugLogger.info("totalbytewritten " +
* totalBytesWritten); debugLogger.info("total Byes " + totalBytes); debugLogger.info("bytes Read " +
* bytesRead);
*/
if (bytesRead == -1) {
debugLogger.info("Exiting lease updater...");
break;
}
totalBytesWritten += bytesRead;
bos.write(buffer, 0, bufferSize);
bos.flush();
int progressPercent = (int) (((bytesAlreadyWritten + totalBytesWritten) * 100) / totalBytes);
// debugLogger.info(" Percentage completed - " + progressPercent);
leaseUpdater.setPercent(progressPercent);
}
// leaseUpdater.setPercent(100);
}catch(Exception e){
debugLogger.error("Exception while vmdk upload ",e);
throw e;
} finally {
diskis.close();
bos.flush();
bos.close();
conn.disconnect();
}
return totalBytesWritten;
}

 

Reply
0 Kudos
2 Replies
AndyBWL
Contributor
Contributor

I have no solution to this, but only to add:

 I have observed the same behaviour usnig the .NET VIM25 service but the corruption happens on the 1st VMDK file uploaded and is, so far, 100% repeatable.

Booting the affected VM into a WinPE environment and checking the deployed filesystem shows it as "raw" and dir &c fail with an unrecognised file syatem as expected.

As well as the web inteface working 100% reliably the PowerCLI 6.5.0 shell using 'Import-Vapp' works 100% reliably also.

AFAICT there no exceptions in the actual underlying upload code.

This is against an HPe specialised image for 7.0.3 running on an HPe DL380 Gen10

Reply
0 Kudos
ViniciusRabello
Contributor
Contributor

Hi Ratndeeep

Have you ever found the solution for this issue? I think I have a quite similar problem, and it would be great to get some direction.

Kind Regards

Vinicius

Reply
0 Kudos