1 2 Previous Next 15 Replies Latest reply on Mar 24, 2016 9:20 PM by UlyssesOfEpirus

    Why can't a linux live CD be modified to act as the host?

    UlyssesOfEpirus Hot Shot

      Just read the following and was wondering if linux live CD's (or freeBSD live CD's) have been fully explored as a host for workstation. Maybe workarounds can be found in these open-source operating systems to make it possible for workstation to work when the entire host o/s is loaded into ram?

       

      Time to wear a dream to its grave

        • 1. Re: Why can't a linux live CD be modified to act as the host?
          UlyssesOfEpirus Hot Shot

          Here's an idea: nested virtualization where the host closest to the hardware has an old version of workstation installed that is happy with the entire o/s in ram, and a VM running on this host has the latest version of vmware workstation installed in it.

           

          The second host o/s then thinks it is accessing a hard disk but in fact it is accessing the first host's ram where the vmdk resides.

           

          Maybe performance is reduced a little because cpu support for virtualisation is not used with nested virtualization, but the benefit from the host o/s being loaded to ram makes up for it.

          • 2. Re: Why can't a linux live CD be modified to act as the host?
            continuum Guru
            Community WarriorsvExpert

            FYI - I use LiveCDs with Workstation since Workstation 4.5.3
            Upto version 6.5 I used Windows PE as Windows LiveCDs with WS had a few advantages compared with Linux and were able to be radically faster than Linux.
            Nowadays Linux LiveCDs with WS are the superior platform and as I have to build them anyway for my VMFS-recovery Cds I regularly build updated versions.
            During 3 years I used a Windows based LiveCD with WS 6.5.4 for all my computer work. No regular disk-based HostOS was required. The LiveCD run completely in RAM so every reboot of the host cleaned up any malware.
            Very nice to work with such a system - it did not require any maintenance.
            5 years ago I got a job in IT and since then I have to use regular systems again - but I still miss those times with WS running in RAM.
            Several years I had a Windows-PE based Construction kit on my site that created a PE with WS 6.5.
            Distributing a LinuxLiveCD with WS is much trickier - I dont see a legal way to do so.
            Also I lost a lot of the original enthusiasm over the years.
            For me this 3 years with such a LiveCD instead  of a Windows-host OS were so convincing - I assumed lots of folks would want to use such a system too. It really changes the way you use a home computer.
            I still wonder why the idea appears to be so obscure to almost everybody - cant explain the lack of interest.
            I stumbled across LiveCDs and Workstation at almost the same month in 2002 and immediatly got the idea: this has to be combined

             

             

            I maintain the PE based construction kit as long as folks ask for 32bit versions. I think the links should still work.
            If you are really interested in this stuff feel free to call me - I always enjoy a chat about my MOA babies ;-)
            Ulli

            • 3. Re: Why can't a linux live CD be modified to act as the host?
              wila Guru
              vExpertCommunity WarriorsUser Moderators

              Ulli,

               

              I still wonder why the idea appears to be so obscure to almost everybody - cant explain the lack of interest.

              I stumbled across LiveCDs and Workstation at almost the same month in 2002 and immediatly got the idea: this has to be combined

              Timing...

               

              If you look at Qubes OS then you'll see that there is more interest for such a thing nowadays as back then.

               

              It's a bit different, but the ideas are similar. As a user you use throw away OS's, Joanna and her team are working on making the workflow for that idea more easy to use.

              I played a bit with it (even have it installed here on a host atm) but for experimenting you can actually run it in a VM.

              Just set the guest OS type to "ESX".

               

              --

              Wil

              | Author of Vimalin. The virtual machine Backup app for VMware Desktop Products
              | Vimalin : Automated backups for VMware Fusion and VMware Workstation Professional
              | More info at https://www.vimalin.com
              | Twitter @wilva
              | VMware Wiki at http://www.vi-toolkit.com
              • 4. Re: Why can't a linux live CD be modified to act as the host?
                UlyssesOfEpirus Hot Shot

                I am a big fan of non-persistency, all my internet-facing VM's have non-persistent system partitions and my host has Shadow Defender installed which is akin to a live cd minus the load-to-ram performance.

                 

                If your MOA live cd wasn't limited to overpriced back then windows server 2003, I'd have been your number one follower.

                 

                After my OP here I noticed the team that developed WS has been dismantled. What are we to make of this security-wise? That WS will not be updated any more and therefore WS will eventually become a security risk? And we better migrate to virtualbox now?

                 

                Have you published instructions somewhere for making a linux live CD with WS?

                • 5. Re: Why can't a linux live CD be modified to act as the host?
                  wila Guru
                  User ModeratorsCommunity WarriorsvExpert

                  Hi,

                   

                  Not Ulli, but answering your questions anyways ... I'm sure Ulli will correct me if I'm wrong.

                   

                  Ulli's MOA is free and can be found here: sanbarrow.com

                   

                  You are correct that the hosted UI team has been let go off and -apparently- a new team will be set up (maybe has been setup by now) in China.

                  The Workstation and Fusion products had been partly hosted UI team and part other teams.

                  For example the hypervisor engine itself is for quite big parts shared by vSphere and the vSphere team is still there.

                  There are some other developers left that also work on parts of Workstation/Fusion.

                  Even read the other day on twitter that -at least one- of the former (G)hosted UI team has been rehired, but in another desktop area.

                  So his knowledge is at least available again.

                   

                  What the exact effect for the product will be is hard to say, but it certainly didn't make me cheer for VMware after hearing that news last month.

                   

                  --

                  Wil

                  | Author of Vimalin. The virtual machine Backup app for VMware Desktop Products
                  | Vimalin : Automated backups for VMware Fusion and VMware Workstation Professional
                  | More info at https://www.vimalin.com
                  | Twitter @wilva
                  | VMware Wiki at http://www.vi-toolkit.com
                  • 6. Re: Why can't a linux live CD be modified to act as the host?
                    continuum Guru
                    vExpertCommunity Warriors

                    > If your MOA live cd wasn't limited to overpriced back then windows server 2003, I'd have been your number one follower.
                    The first BartPE-versions included MS-files. That was noticed by MS and Bart Lagerwij was contacted by lawyers who kindly asked him to remove the downloads.
                    Later versions were accepted by MS. To build a PE no serial numbers were required - not surprising in a "Pre installation Environment"
                    The 2003 setup.iso is available without any necessary registration.
                    It probably was a grey area - but we all assumed that using 2003 as source for a PE does not require licence fees.
                    The construction kit I offered downloaded the 2003 source iso automatically - and I never received any mail from MS-lawyers.
                    I tried to find a contact at VMware to discuss a possible redistribution of the WS-files - of course I would not include a serial number - but even with the help of a VMware Partner company I never found a person who could give at least relevant advice.
                    So all isos I post in public do not include any VMware binaries - I  just include all the dependancies so that WS or VMplayer or the VDDK-kit can be installed on the fly.
                    Building Linux LiveCDs is quite easy - I use a tool named Remastersys for all my Ubuntu based LiveCDs.
                    Remastersys does not officially support recent Ubuntus and the author stopped developing it.
                    But I was able to modify it for my special needs and so I can still use it with good results.
                    So basically you simply setup a VM with Ubuntu - install everything you want via apt-get.
                    Once you have everything - you remove all docs and other unnecessary files and run the remastersys script.
                    There are lots of other tools to create Ubuntu-LiveCDs so it is quite easy to create your own build.
                    The only problem is the cleaning up of unnecessary files - so that you get a build that is small enough to load into RAM completly.
                    By the way - check your emails

                     

                    Ulli

                    • 7. Re: Why can't a linux live CD be modified to act as the host?
                      UlyssesOfEpirus Hot Shot

                      Is anyone who wants to build their own linux live CD likely to need your modifications to remastersys? What were they in a nutshell?

                       

                      When you say cleaning up unnecessary files is a problem, do you mean some deletions are risky and could break some stuff much later on?

                      • 8. Re: Why can't a linux live CD be modified to act as the host?
                        continuum Guru
                        Community WarriorsvExpert

                        > Is anyone who wants to build their own linux live CD likely to need your modifications to remastersys? What were they in a nutshell?
                        Nope - if you start building LinuxLiveCDs nowadays there are other options available.
                        I just removed functions that I never use and which would fail on newer Ubuntus as the dependancies are missing.
                        > When you say cleaning up unnecessary files is a problem, do you mean some deletions are risky and could break some stuff much later on?
                        Exactly - just have a look at WS itself. It comes with documentation and lots of binaries you will never need in a LiveCD. If you remove those files you have to be careful not to remove anything essential.
                        Examples for files you will not need: vmware-vmx-stat, vmware-vmx-debug, ovf-tool ...
                        I usually list all files by size and start looking at the biggest ones first.
                        /usr/share has lots of stuff that can be removed - but finding out which stuff must not be removed will probably produce several bad builds

                        • 9. Re: Why can't a linux live CD be modified to act as the host?
                          UlyssesOfEpirus Hot Shot

                          Someone must have thought of this and implemented it: how about putting just the obviously necessary files in the initial ramdisk (initrd) and the remaining files in the regular CD filesystem so everything is available?  Isn't that close to what your big windows MOA live DVD's did?

                          • 10. Re: Why can't a linux live CD be modified to act as the host?
                            continuum Guru
                            vExpertCommunity Warriors

                            To modify the content of initrd you need a level of Linux skills that is beyond my skills.
                            But for about 2 years I used a similar approach: I created a small core system about 100 MB and loaded that into RAM. Everything else was stored on a USB-drive which was then loaded as an overlay (persistence mode)
                            After a while I noticed that this way was not stable enough - shutting down the system by a hard power off often resulted in a corrupt overlay. So I stopped doing it.
                            I also noticed that the theoretical performance gain you would expect with running the system completely in RAM was not as significant as expected.
                            With 2003 PE it made a big difference - but with Linux the difference was much smaller.
                            My last PE builds with WS 6.5.4 (my all time favorite version) were highly optimized and I only needed about 250 MB of files for the Windows OS plus Workstation.
                            With the current versions - using Linux-  you could get the size of the CD down to maybe 450 - 500 MB - that should be small enough for the computers you find nowadays.
                            During last years my focus has changed - my first PE LiveCDs with Workstation were built to replace a regular host OS.
                            So it made sense to optimise them for best Workstation performance.
                            Nowadays I use Workstation on LiveCDs as a tool among others for my ESXi-recovery work. So performance is less important. More important is having all the dependancies included to be able to add missing tools on the fly.

                            • 11. Re: Why can't a linux live CD be modified to act as the host?
                              UlyssesOfEpirus Hot Shot

                              What sort of disaster scenaria do you prefer to use the live CD in in order to recover VM's rather than use ESXi?  Can't you just copy the vmdk's to a working ESXi server and recover data from there?

                              • 12. Re: Why can't a linux live CD be modified to act as the host?
                                continuum Guru
                                Community WarriorsvExpert

                                Good joke

                                I see you dont have long term experience with ESXi.
                                ESXi is stubborn: if he thinks a VMFS looks a bit suspicious he refuses to display files, directories or even complete volumes.
                                This behaviour seems to be necessary to support access to a volume by several hosts at the same time.
                                Troubleshooting these heartbeat-errors is very time consuming and completely undocumented. So I learned from experience that I get more reliable results if I do not use ESXi at all when I inspect the corruption.
                                Instead I use a Linux-system and inspect the ESXi filesystem indirectly via sshfs in readonly mode. When I started with VMFS recovery jobs I used a Linux-LiveCD to boot the ESXi host itself.
                                Then I used to inspect the VMFS-volumes with the Linux vmfs-tools. Problem with this approach was that all VMs have to be stopped - this is often impossible in a production environment.
                                Nowadays I create a Linux-VM and start it into http://sanbarrow.com/livecds/moa64-nogui/MOA64-nogui-incl-src-111014-efi.iso

                                This is a commandline LiveCD which has all the tools I need in 95% of my cases.
                                When I have to work with snapshot problems I sometimes use LiveCds with GUI and Workstation. As WS can use native ESXi vmdks without modifications you get a view useful additional options.
                                To make it short... the use of a LinuxLiveCD that I build myself has so many advantages that I only rarely work with the ESXi system directly nowadays
                                - known environment - means that all tools will behave as expected
                                - no problems with remote access : teamviewer or similar to connect to the admin-windows of my customer and putty from there to the MOA-system
                                - production use of VMs does not have to be interupted
                                - the indirect readonly access allows reading files that are locked if you look from ESXi, it often even allows reading files that appear corrupt with I/O errors when using an ESXi
                                - several MOAs can be used in parallel to save time while scanning large datastores

                                I try to get the work done with the commandline CD whenever possible because the download is smaller and because the chance that it boots almost anywhere is higher. The CD with GUI sometimes fails on systems with strange hardware.
                                If you are looking for more exciting use-cases for a LiveCD with WS consider the use in forensics.Such a LiveCD can be used to boot a suspicious, maybe booby-trapped Windows-machine. After full boot you create a VM, assign the phsical disks to it and immediatly create a snapshot.
                                Then you can patch the registry so that the physical system can start as a VM. Now you can let it boot and you can watch how the injected malware destroys the system. Thanks to the snapshot you can revert to the unbooted state. Nothing on the actual physical disks will be changed.
                                Have you seen the-as-if-stunt ?
                                Another video that shows MOA booting "Kiosk Mode
                                MOA-kiosk-mode

                                The Workstation program at that time was so small that I could install it while booting the PE and was able to reduce the required files so much that WS could be used from a 11 mb large wim archive.  Those days VMware was pure fun

                                • 13. Re: Why can't a linux live CD be modified to act as the host?
                                  UlyssesOfEpirus Hot Shot

                                  Fascinating. Someone ought to do an interview with you for a tech magazine. The McGyver of virtualization. It is said the most important difference between a genius and everyone else is that the former perseveres with problems.

                                   

                                  Btw I have noticed you're in Germany but for some reason I thought you were in Austria before. Did you move to Germany? And set up your freelance recovery business there because it was easier than in Austria plus all the big potential customers were there?

                                   

                                  It's many people's dream job to have some fun doing what they do for a living and even do it from home occasionally, with clients all over the world. But then it's hard to imagine how clients would trust their business data to someone they've met over skype, and there's office politics too getting in the way, not to mention managers like to have people in physical proximity so they feel they have them "underneath them".

                                  • 14. Re: Why can't a linux live CD be modified to act as the host?
                                    continuum Guru
                                    Community WarriorsvExpert

                                    > Btw I have noticed you're in Germany but for some reason I thought you were in Austria before. Did you move to Germany? And set up your freelance recovery business there because it was easier than in Austria plus all the big potential customers were there?
                                    Hope you are not disapointed ....
                                    I never moved to Germany - I was born here. Wonder what made you assume I am from Austria ?
                                    I am no Freelancer - not at all. I had a very boring factory job (every 15 minutes I had to count the edges of hex-nuts for the automobile industry - id they had more or less corners then 6 I had to call for help)
                                    During the economy crisis 5 years ago I was fired and since then I had a job in VMware-support.
                                    Next month I will be unemployed again - the company I worked for prefers to work for big german customers and does not have enough use for my special skills.
                                    > It's many people's dream job to have some fun doing what they do for a living.
                                    I had the most fun with VMware while I had that dull factory job - publishing the first working Workstation running on BartPE after a year of fighting with the registry was a lot of fun.
                                    Especially when all the smart folks in the PEbuilder-scene told you again and again that it is impossible

                                    > But then it's hard to imagine how clients would trust their business data to someone they've met over skype ....
                                    I was surprised about that too - but I seem to have quite a good reputation in VMTN and the german forum - I dont remember any negative feedback in all the years I am active in the forums.
                                    And probably more important : in  many VMFS recovery cases the customers already tried everything they could think off, VMware support most of the times does not help and Ontrack is too expensive.
                                    So for many customers I am the last hope - folks in despair tend to be flexible about the rules they made up themselves.

                                    1 2 Previous Next