Hi,
If I have a VMFS3 datastore with 1MB block size limiting max file size to 256GB and I upgrade this datastore to VMFS5, will it remove this limit?
yes, it will grow automatically only after ui exceed 2 TB limit
Also you cannot modify the block size in VMFS5, it supported only 1MB
refer http://blogs.vmware.com/vsphere/2011/07/new-vsphere-50-storage-features-part-1-vmfs-5.html