I would like to delete my old VMFS3 to re-create them to take advantage of VMFS5. When I try to delete the volume I have a error like the ressource 'VMFS1' is in use.
I migrated all VMs to another VMFS and the datastore is empty. I don't know why I can't delete it.
Could you please help me?
Check out http://kb.vmware.com/kb/2011506 for the correct procedure to remove the datastores.
You may try to use e.g. RVTools (http://www.robware.net/) to see whether there's a reference to this datastore by e.g. a mapped ISO file. Other reasons could be an active snapshots which has initially been created for a VM on this datastore.
Btw. instead of deleting and re-creating the datastore, you can also live upgrade it to VMFS-5. When you highlight the datastore in Configuration -> Storage, you should see an Upgrade link.
Check if the datastore is part of datastore cluster or used by vSphere HA, if you have an option to reboot one of your host you could try that.
I never heard about of loosing disk space. Take a look at http://blogs.vmware.com/vsphere/2011/07/new-vsphere-50-storage-features-part-1-vmfs-5.html to see what the differences are between an upgraded and a new VMFS-5 datastore.