Managing Your Scripts - Directory Structure

Script management can be a daunting task after some time, even for a small environment.  One of my initial responsibilities in my new role is to assist in automation and scripting in one of the largest environments I'm likely to ever encounter.  We're talking 1000+ hosts and thousands of VMs.  An environment of that size demands some serious scripting and I'm extremely excited about the opportunity to both flex my PowerCLI muscles as well as strengthen my chops tremendously.  On top of that I was going to be working with a very talented guy who already had his previous work.  This might pose some headache.

To the Expert!

I was fortunate enough to have a conversation with Hal Rottenberg recently about our challenges and he provided quite a few suggestions that we're looking to implement.  I will only elaborate on the things as we implement but here are the tips he gave me.

Tips from Hal Rottenberg

  1. Common Naming - Name using      PowerShell Verb-Noun nomenclature
  2. Centralize Functions and      Snippets - Keep functions in files on central location so you can quickly      reference them from your scripts without having to include them in the      script itself.
  3. Get-Help aware - make      functions so that you can access help from the Get-Help command.
    1. http://blogs.technet.com/b/heyscriptingguy/archive/2010/01/07/hey-scripting-guy-january-7-2010.aspx
    1. http://powershell.com/cs/media/p/9345.aspx
  4. Try Version control -      Subversion
    1. http://tortoisesvn.net/
    1. http://subversion.tigris.org/

Thanks Hal!
Twitter: @halr9000

Blog: http://halr9000.com
Buy his book!  (Amazon | Sapien)

Over the coming months/years I hope to share as much of the work that I've been doing with my coworker Jade Lester as we are able.  I want it to be known that much of what I post is work that we have worked on together with varying levels of contribution from each of us per script.  Now that that is out of the way let's get back to work.

Our Challenges and Goals

The first challenge I saw was how to combine the efforts that Jade and I had up to the point where we started working together.  Being PowerCLI guys it was easy to follow what the other was doing but our organization was a bit different.  He organized based on target (VM, VC, Host) while I organized based on function (Reporting, Maintenance, Implementation).  Both methods have strong arguments and it's impossible to say who is right or wrong.  What to do then?

Some of Our Goals

  1. Create central repository for      scripts
  2. Create guidelines for script      names using Target-Verb-Noun structure (i.e. vCenter-Get-Hostlist,      ESX-Get-ActiveHBAs)
  1. Create consistent structure      for scripts [headers, formatting, variables, functions]
  1. Create central repository of      functions and snippets for rapid scripting. 
  1. Identify most common scripts      and convert to new structure.       Isolate possible functions and snippets and make available      accordingly.

Our Results - So Far

This is a work in progress but here's what we came up with so far.  Comments and suggestions welcome:

===========================================================

Two primary parent directories

  1. /Scripts_CRs  | Location for scripts used for specific      Change Requests.  Subdirectories      should be CR#.  This location      intended for all users based on their CRs.
  2. /Scripts_PowerCLI | Default      Script repository.  This location      intended for script developers.

----Subdirectories for Z:\Private\Scripts_CRs

  • \CR#######
  • \CR#######

----Subdirectories for Z:\Scripts_PowerCLI

  • \CRs
    • Location of scripts in       development for CRs.  All files       should include CR name.  Any files       not including CR will be deleted periodically.

  • \Downloads
    • Repository of scripts found       online - Add note on location script came from but do not change original       script

  • \Functions
    • Repository for functions to       be used by other scripts

  • \Implementation
    • Assist in implementation/Provisioning       tasks

  • \Import
    • Location of CSV, TXT, and       Credential files to be imported into scripts

  • \Maintenance
    • Perform Maintenance tasks -       Update tools, config changes, change resource reservations, file/config       backups

  • \Reporting
    • Reports on environment -       Connected devices, Resource reservations, WWNs

  • \Snippets
    • Location for code snippets       and reusable code that does not work as functions

  • \Team
    • Scratch Directory for       scripts in progress.  May want use       usernames as subdirectory names.
    • \Team\Jade
    • \Team\Josh

  • \Troubleshooting
    • Scripts for identifying and       resolving issues.

=============================================================

We both liked being able to separate the scripts based on their target so several of the directories will include subdirectories.

  • \Host_Specific
    • Scripts that target ESX(i)       hosts
    • Ex. Network Configs, NTP
  • \Reports
    • Placed here so that any       reporting can be placed quickly in similar working directory
  • \vCenter_Specific
    • Scripts that target vCenter       Server or containers
    • Ex. Cluster or Resource Pool       info
  • \VM_Specific
    • Scripts that target Virtual       Machines
    • Ex. Update-Tools,       Reservations

==============================================================

Make it Happen, Cap'n!

Right now we have code all over the place.  We each have scripts on our laptops.  I have scripts on flash drives.  He has scripts on jump servers.  I have scripts on jump servers.  He has scripts in his head and I'm wondering how he can run those.

We both recognize that having stuff local makes sense for offline work, rapid prototyping, and testing things out in a safe environment.  The difference is now as we start needing code it is time to polish it up and place it in the right directory.  This is similar to the method my wife intends on cleaning up our kitchen drawers.  Put everything in a big box.  Pull it out as you need it and after 3-6 months you're likely using everything you'll need.  It may take a while to find it but you KNOW it's in the box if it's not in the drawer it's supposed to be in.

Work in this directory structure has already begun and is proving fruitful.  I have already started gathering various code Snippets that we use regularly and placed them in dedicated files.  Just 20 minutes after adding a couple I was asked for a snippet.  I quickly pulled up the code from the snippets directory and was able to give him what he needed right away. We knew then we were off to a good start.

I hope you have found this useful.  Feedback from your experience and methods are definitely appreciated.