I have vSphere 4.1 and VM View 5.0
I wanted to upgrade to vSphere 5.1 and at some later date upgrade to View 5.2.
If I upgrade to vSphere 5.1 from 4.1 will VM View stop working in any way or any view connections/security server settings I'll need to remake, or is it hassle free.
View 5.0 does not support vSphere 5.1 so, yes, you may run into issues. You should instead upgrade the version of View first - View 5.2 supports both vSphere 41.u3 and 5.1u1. Please refer to the upgrade guides for both View and vSphere for further details.
In addition to paying attention to the VMware Product Interoperability Matrixes, you should also reference Update sequence for vSphere 5.1 Update 1 and its compatible VMware products (2037630) to determine the proper upgrade order.