4 Replies Latest reply on Aug 23, 2011 9:25 PM by AndreTheGiant

    FAQ for Unsuccessful VCDX Candidates


      The pass-rate goal of the VCDX program is that every candidate who is accepted to defend passes.  We have not yet achieved this goal.  In my time as VCDX Program Manager, I have worked with a number of unsuccessful candidates. I have found that they tend to express similar concerns. This document collects these concerns, together with responses. I hope that it will help past candidates make constructive use of their experiences, and that it will help future candidates, including those reapplying, to succeed.


      Panelists asked me questions that are directly answered in my submitted design. Clearly, they did not read it.
      Panelists are indeed trained to ask, among other kinds of questions, those that are  answered in your submitted design. Your ability to answer these questions impromptu confirms that you are the author of the parts of the design you claim. I am sure nobody reading this would ever consider submitting someone else's work as their own, but it has happened before, and the program must guard against it.
      Now let's set aside all concerns about plagiarism. To achieve VCDX, you must display full command and knowledge of your design, including fictitious parts and parts done by others. (If the program did not have this requirement, candidates would be able to evade questioning on weaker parts of their design by claiming they were others' work.) Verifying this full mastery is another reason why panelists include questions answered in your design.
      Beyond that: your performance at defense trumps your performance in the submitted design. In other words: if your defense improves on your submission--for example, because you successfully explained discrepancies or apparent design errors--your score will be improved.  Conversely: if, in your defense, you poorly cover a section that you covered well in the written submission, your score will be reduced.
      My panel only asked me configuration questions! I didn't know VCDX was just VCAP-DCA warmed over.
      My panel only asked me logical design questions! I didn't know VCDX was just VCAP-DCD warmed over.
      Before candidates defend, panelists review their submissions and record a preliminary score that is based only on the submitted design.  In the defense, panelists focus their questions on areas where candidates need to improve their scores in order to pass.
      So, if your panel mostly asked you questions about the logical-design portion, they probably thought you did fairly well in the physical-design portion; and vice versa.
      VCDX is unique among VMware certifications in that it is a hybrid credential. To achieve it, you must show a certain level of logical-design or enterprise-design skills AND show a certain level of physical-design or engineering-spec skills AND display both in a standard format. VMware recognizes this combination of skills because it believes the combination is a strong predictor of success.  We do not claim it is the sole predictor of success, but it is the one the VCDX program recognizes.
      A panelist interrupted me! I felt disrespected.
      Chances are that the panelist was trying to do you a favor.
      Recall that panelists are trained to focus on areas where candidates need to score points in order to pass. Panelists can see the scoring rubric, and candidates cannot. So a panelist knows when a candidate is doing himself or herself no favors: either because the candidate has already demonstrated competency in a given area, or else because it is time to cut losses and move on to another area.
      Panelists are also trained to treat candidates respectfully: for example, by not interrupting. Sometimes these goals conflict. Panelists do the best they can to strike a balance.
      I submitted a really excellent design, and yet I didn't achieve VCDX.
      As I mentioned earlier, your performance at defense trumps your performance in the submitted design.
      VCDX also requires that you perform well in the standardized format of the defense.  It's not everyone's style. Sometimes people ask, "Why doesn't VMware do away with the defense? Instead, it should just grant VCDX based on a review of the candidate's portfolio, references from clients and peers, and resume." In fact this architecture was considered; it is perfectly viable. But it rewards a different set of candidates. The existing VCDX program rewards people who can think on their feet, tend towards conciseness, and have an organized habit of thought. A credential based on a portfolio review would instead reward people with written presentation skills and human networking skills.  That would be meaningful too, and we might introduce such a credential someday. But it's not VCDX4.
      I spent tons of time on my fictitious design, and yet I didn't achieve VCDX.
      When we wrote the VCDX4 rules explicitly permitting fictitious designs, we knew it would be a tough road for candidates. As a matter of fact, it's proved to be even tougher than we thought. Success rates of candidates submitting fictitious designs are worse than rates of candidates submitting actual designs they worked on.
      I will be posting a separate writeup on the pitfalls of fictitious designs and how to avoid them.
      I tied everything I did to published best practices, and yet I didn't achieve VCDX.
      "Best practices" is a VCAP concept. Conversations at the VCDX level use "best practices" as a starting point. To succeed in VCDX, you should know why a published best practice is a best practice, and you should also know the kinds of customer requirements or engineering constraints that would cause you to ignore them and do something else.
      I think a particular panelist was determined to fail me.
      It would be very difficult for an evil panelist to plan ahead of time to wreck a candidacy. The process by which panelists are assigned to candidates is, at various points, blinded and randomized.
      But let's suppose a particular panelist simply decided, in your panel's opening moments, that he or she just didn't like you. In order to arbitrarily fail you, the panelist would have a lot of work to do. Panelists must evaluate each candidate based on a standard rubric. In essence, they must determine the candidate's performance level (from amazing to unqualified) on each part of the VCDX4 blueprint. So the evil panelist would have to make specific lies in order to fail a qualified candidate.
      Even so, the plan would probably fail. Panels always consist of an odd number of panelists. If the candidate is qualified, the evil panelist will be outvoted. (And the VCDX Program Manager will notice the peculiar scoring pattern.)
      An evil panelist could not even influence the other panelists, because panelists are specifically forbidden from discussing candidates' applications or defense performance until all results are finalized.
      A panelist was not a native speaker of English, and this posed a serious problem for me.
      All panelists, at the present time, speak English as part of their jobs. A panelist who can hold down an IT/virtualization job in English is presumptively a good enough speaker of English to serve on a panel.
      Nevertheless: yes, it's a little harder to deal with someone who is not a native speaker of your language. With a major exception (in the next paragraph), all candidates have equal odds of having such a judge on their panels. This mirrors real life: sometimes our design clients are also not native speakers of our own languages, but we must work with them anyway, sometimes in tense situations.
      Don't forget that many brave souls who themselves are not native speakers of English have attempted VCDX, and many have succeeded. Because all panels are presently in English, these candidates faced an even greater burden: most or all of their panelists were not native speakers of their language.
      The feedback I got in my result letter was totally inadequate for my needs.
      Don't forget that the session itself is a feedback forum.  The areas in which the panelists quizzed you the most are those areas to focus on.  In most panels, specific shortcomings of the submitted design are revealed, as are skills the candidate would benefit from developing.
      But let's talk about result letters.  These letters contain feedback that follows the structure of the VCDX4 blueprint. The feedback lists those blueprint sections that, based on the panelists' scoresheets, are the biggest opportunities for improvement.  Next to each section, one or more suggestions for improvement are listed.  (People who applied unsuccessfully in the VCDX3 era will recognize that this is an enhancement. VCDX3 letters were sparser.)
      Some unsuccessful candidates have demanded a lot more. In effect, they want a personalized narrative, in the words of the panelists, documenting their shortcomings with specific instances of error. Seems reasonable; why doesn't VMware do this?
      First and foremost: free-form feedback is exceptionally difficult to deliver fairly. Inevitably, some candidates get feedback that is much more prescriptive than what others get.  They are at an advantage when they reattempt.
      Second: it is very costly. VCDX panelists' time is expensive; many are billable personnel with high hourly rates. Compared to other IT vendors' capstone credentials, VCDX is a bargain, and we hope to keep it that way.
      A few unsuccessful candidates have demanded, in effect, the right to confront their accusers: to have their panel reconvened and their result re-argued. We do not entertain this, not only because of the enormous expense, but also because it defeats the purpose of the standardized process. VCDX cannot be a uniform and fair process if unsuccessful candidates get a second forum, this one on their own terms.
      I don't understand the feedback I got at all. It claims I need to improve in areas I know I'm very good at.
      There are two possibilities. First, as the moderator's standard introduction points out, there is a chance for lower scores if the panel doesn't get a chance to cover all scoring areas in the allotted time. Yes, this is an artifact of the defense format, but it applies equally to all candidates.  You may need to improve your oral communication skills, focusing especially on crisp, relevant answers.
      Second, consider this. A panel of VMware experts, without comparing notes with one another, found some areas for improvement in your performance. Maybe they saw something useful. After all, if the people reviewing me only find areas for improvement I already know about, what use are they?
      Perhaps you genuinely have a weakness you didn't know about. It happens. In VCDX panels, I have watched otherwise impressive candidates reveal very important holes in their knowledge. Or perhaps you need to improve the way you communicate, or to get more practice thinking on your feet. Any of these will further your career.
      My result letter contained a sentence like this: "The panelists recommended that you gain more career experience before reapplying." This is just boilerplate, right?
      No. Not all candidates get this sentence in their result letters. The sentence only appears when panelists unanimously agreed upon it. In effect, they agreed  that a candidate's career advancement would be better served by getting more on-the-job project experience than by attempting VCDX again right away.
      If I failed at the defense section, why was my application accepted to defend in the first place?
      VCDX applications advance to the defense stage if they seem to show a "material chance" of success at defense.  What does a "material chance" mean?  A reviewer scores the application in much the same way panelists do at the defense stage; candidates are invited to defend if this pre-score exceeds the ordinary passing score, OR if it's below the ordinary passing score but within a certain leeway. This reflects the fact that the defense is an opportunity for candidates to raise their scores.
      I am very discouraged by my failure to achieve VCDX, because the VCDX blueprint is basically my job description.
      Please don't make this mistake. There are a lot of people out there whose jobs require them to display virtualization design skills and engineering skills. Not many, however, are required to integrate those skills all at once, without prompting, in front of unfriendly audiences. VCDX is intended to be an aspirational credential: it recognizes achievement of a superior skill level, not just competence.
      I think VCDX is totally subjective.
      VCDX is not as objective as, for example, a boxing match, in which each judge carefully counts punches. There's simply no way to map that to something as complex as a virtualization design. (I hope I am not offending boxing fans by saying that.) So, instead, we build measures into VCDX that limit and mitigate subjectivity.  Examples:
      • Blinded and randomized selection of panelists
      • A standardized defense format
      • Mandatory training for panelists
      • A scoring rubric with detailed definitions of each performance level in each scoring area
      • Majority voting by panelists (that is, unanimity among panelists is not required)
      We're not perfect, and we don't claim to be. We do claim to be improving continuously, so that each VCDX-N is better than N-1.
      Well, okay, here I am with a negative VCDX result. What do I do next?
      Particularly if you got feedback you did not understand, I recommend an anonymous "360 review." Make a survey (using Surveymonkey.com or the like) and send it to your clients, your peers, and your boss, and ask for anonymous feedback. They might corroborate your VCDX feedback, or they might contradict it. Every time I have done a 360 review, I have learned things about myself, some of it painful, all of it useful.
      What if they contradict your VCDX feedback? If so, consider getting experience thinking on your feet in front of groups. Organize a mock panel with
      your peers, asking them to throw rocks at your work. Make sure that these forums are time-limited, and try to answer every question concisely. Maintain a brisk pace.
      Most importantly: try to answer your questioners in their own terms, not yours. If you are uncomfortable answering questions without scrolling to the relevant PowerPoint slide (or a few slides before that, to set the context), this is a predictor of failure.
      If your 360 review corroborates your VCDX feedback, your corrective actions should be clearer.
      Brian, have you ever seen a successful VCDX defense?
      What's it like?
      Interestingly enough, all unsuccessful candidates' defense sessions vary, but successful candidates are remarkably similar. The candidate is comfortable answering all questions, no matter whether they're about the customer requirements, the logical design, or the physical design, jumping freely among them as needed. The candidate speaks crisply, without any unnecessary words. When the candidate doesn't know the answer to a question, he or she says immediately, "I don't know.  But here's where I'd look to find the answer, and the consequences of that answer on my design would be X."
        • 1. Re: FAQ for Unsuccessful VCDX Candidates
          mcowger Champion

          "Interestingly enough, all unsuccessful candidates' defense sessions  vary, but successful candidates are remarkably similar. The candidate is  comfortable answering all questions, no matter whether they're about  the customer requirements, the logical design, or the physical design,  jumping freely among them as needed. The candidate speaks crisply,  without any unnecessary words. When the candidate doesn't know the  answer to a question, he or she says immediately, "I don't know.  But  here's where I'd look to find the answer, and the consequences of that  answer on my design would be X.""



          ^^^ That right there is the most important thing to keep in mind.

          • 2. Re: FAQ for Unsuccessful VCDX Candidates
            jasonboche Champion

            This is fantastic Brian. Thank you to yourself and VMware for putting it together.


            Not to turn this into yet another scattered source of tips, but I'll add on one thing.  A common theme can be categorized as "I think I am good enough". For all of the hard work that goes into the process, it's definitely a fair self assessment.  However, this is where the 360 feedback is key. I performed several mock examinations prior to my defense to help prepare myself.  The mock defenses don't necessarily need to be "in person" panelist style but it helps.  I felt that I didn't really have anyone locally that I could lean on for mock exams so I conducted all of mine through email.  Gracious friends stepped forward, took all the time to go through my design, and then drill me with questions. Again, I feel the results were very beneficial.  Questions were asked which I wasn't quite prepared for.  Great questions.  An untold number of hours are spent preparing the design and reviewing it.  But a valid assessment can't be completed in a vacuum.  The exponentially increasing numbness to your own design yields a false sense of security.  Peer review is an excellent reality check.



            • 3. Re: FAQ for Unsuccessful VCDX Candidates

              I'm glad y'all think this writeup is helpful.  Here's another excellent question I got from a candidate.


              The scenario I got for my design exercise was based on EMC storage. I am not an EMC guy, so this put me at a disadvantage.



              All candidates have equal odds of getting scenarios involving network and storage gear from any major vendor.  In spite of EMC's part-ownership of VMware, we are not biased in favor of EMC.  (I should also mention that Cisco's and Intel's ownership stakes in VMware also don't affect VCDX scenarios.)  In the past, I have seen scenarios involving IBM, HP, and NetApp storage too.


              We have considered making all scenarios involve fictitious hardware, so that every candidate would be at an equal disadvantage. We might say "This customer uses Frammistan 2000 storage, and Nestle-brand routers and switches!"  But frankly, the benefit of this fictionalization would be small.  When in the real world you're confronted with storage you don't know, what questions do you ask?  You ask whether it's iSCSI, FCoE, FC over fiber, or NAS; whether it's a high-end or low-end model; whether it's active-passive or active-active or supports ALUA; what it's capacity is; what kind of IOPS it can sustain; and so forth.  You can do this quickly and efficiently in a VCDX defense session as well, and it doesn't count against you.


              I should caution candidates that, even though it's not in the scoring rubric, most folks assume that VCDXes have an airline-magazine-level understanding of the storage and network markets.  People assume that VCDXes know that NetApp loves NFS, that HP's high-end storage is called XP (and licensed from Hitachi), that EMC's big gun is the Symmetrix VMAX, and so forth, and that routers and switches are probably Cisco unless you hear otherwise.  I should emphasize that none of that stuff is scored for.  But having this kind of market awareness will make you more confident during your defense.

              • 4. Re: FAQ for Unsuccessful VCDX Candidates
                AndreTheGiant Guru
                User ModeratorsvExpert

                I can understant the except. But I think that a VCDX candidate must know (not necessary in the detail) most common networking and storage solutions.

                I also got in my desing a specific hardware solution... but you do not need to be a specifical to understand how you can use it during the defense.

                And the VCAP4-DCD (at least in the beta version) incuded some vendor specific questions.