Automation & Test Cases
I’ve come to realize that my answers to life and the universe may vary from others. Sometimes I fall in love with my answers, but I’m nearly always quite open to ideas that differ from mine. In fact, I love the conflict in thought because it makes me re-think the original problem.
One such example is in my mind. Slightly mentioned in HWTSAM and described in detail in my chapter of Beautiful Testing is my approach to automation. When I write automation, I don’t think of test cases first. I may use a spec, notes from a spec, or just lot’s of notes in general to help me design test automation. I explore the component I’m testing– either manually, or in the debugger and create the tests as I develop the automation (skipping lot’s of design stuff because that’s not the point of this post). When I’m done, I prefer an approach where I extract the test cases from the test automation (either via c# attributes or export names or some other form of tagging meta data) and import theses test cases into the test case management system.
An alternate approach is to write the test cases first, import them into the TCM, then write the automation (optionally creating stubs automatically from the test cases). At first, I thought I didn’t like the idea at all, but I suppose it does force you to think about the details of testing up front (if that’s a good thing). One con that I haven’t resolved is what to do when the automation changes. With my approach, you just re-import the metadata. If you change the automation using the alternate approach, you probably need to update the TCM manually – but I suppose you could find a way to make that easy.
How about you – do you prefer one approach over the other or have an alternate approach?
Hi Alan,
My approach is somewhat model-based.
1. Take business logic flow
2. Explore it extensively, around, above, and beyond.
3. Build a test logic tree (good to document with mindmapping tools or IDEF0 blocks)
{Finding new defects and issues at stages 2 and 3 is typical}
4. Decompose the test flow
5. Implement test components of the flow, assembly the flow
6. Create or import business logic test data. Create test control data (my framework is hybrid keyword/data driven)
7. Attach exit points for negative test cases, supply additional business and test control data for those TCs.
8. If company uses traceability matrices, build coverage documents
None of the steps above require coding.
I use abstract functional modeling language to write instructions; XML format is a container for both execution instructions and execution logs produced.
I used to work in environments with a highly centralized TCM system, and environments where all TCs where distributed across the teams Excel/Word documents, and in both ways I’m not comfortable with a high duplication rate of test steps in traditional Test Case documentation approach. Personally I see TCs as a dynamic multi-branched flow, and this is how I automate them.
Thank you,
Albert Gareev
I love the no TCM approach. This is where you comment your automation really well, demo it, get it reviewed, create a collaborative descriptive wiki that is easy to update if it changes, and then you do your testing.
I have yet to see a TCM really add true value. I do realize there are policies and how things are done, but I think overall there may be less TCM and not more. In some places they are considered retro.
I think it should really be a combination approach. We have had TCM first approach while we were sorting out the resources and tools to use for the web automation, and when we did start the automaiton design we stumbled road blocks in terms of mapping the test cases to the automation design and vice-versa. So we had to really combine test cases or split test cases to accomodate the design. And this became a further level of complication since we also used a tool to read the test cases off of TCM and log the results back to it.
What we realized was a combined approach of having the scenarios nailed down and start the automation design would have helped to add test cases as we go along with automation design in parallel.
But having the scenarios would be really helpful before designing both applicaiton code and the automation in my observation.
-Ram