Quick Testing Challenge
Some updates and clarification below.
And a NEW HINT below too
Recently, I was discussing the diagnostic, debugging, and troubleshooting aspects of software testing, and this morning, while playing with Visual Studio 2012, I created a quick little exercise where the solution relies on these skills.
This zip file contains an app (theapp.exe), and the the vc 2012 runtime dll (both for 32-bit Windows) (note, this should be all that is needed, but if you get a load error, let me know in the comments or on twitter).
But even with the runtime, the app won’t launch. I’m sure that many of you will be able to figure out why –but I’m curious to know how you find the answer. When faced with problems like this, I have a favorite method – but I’m curious what other people do.
Two things to add:
- This application, does indeed launch. When it runs, you’ll have a generic windows application, and a message box indicating success.
- Some of you are probably thinking, “If the app doesn’t even launch, I’ll talk to the developer, since he’ll know what changed.”. But consider this scenario (which is the exact scenario that led me to create this app). You are the tester for Application Awesome. Your application is…Awesome. However, a user tells you that after installing Application Awesome, another app they love (TheApp.exe) fails to launch. WTF happened?
- HINTS: Think for a moment about what sorts of things an application may check on launch. Could one of them be failing? How would you (could you) know?
What if the main function just calls exit?
What’s the definition of “launch”?
I tried it with debugger and I did not see any exception.
From the last callstack, the exit is called by the application. For me, it seems existing is the intentional behavior.
It should show a UI (and perhaps a ‘success!’ message.
What else could cause an application to not launch?
Hi Alan,
I tried the following to figure why the application won’t start –
1. Double click, “TheApp.exe” – doesn’t start.
2. Right and run as administrator – doesn’t start.
3. Access Event Viewer logs and see if the application is logging any error logs. – there were no logs available.
4. Copy the “TheApp.exe” into a new folder without the dll file – doesn’t start.
5. Try to run “TheApp.exe” in different compatibility modes – doesn’t start.
My next steps would then be finding out if the application requires dll or runtime software for in order for me to run it. If not, I would then go consult with the developer.
Not sure if this what you were looking for but I would be interested in how you would approach this exercise.
Thanks,
Michael
In this scenario, you don’t know the developer (your app broke “TheApp”).
So, what else can you do?
Hi Alan,
Thanks for the reply.
Okay, so no contact with the developers – check.
My steps would be to ensure is whether the app is compatible with the Operated System that its been run from. I re-read the blog post and you mentioned 32 bit. My work machine is running windows 7 64 bit, so my “assumption” (now only if I had 32-bit VM lying around so I could test my theory) would be the application won’t start due to incompatibility.
Thanks,
Michael
Well couldn’t get it working yet…
So far:
Run app
check event logs
run as admin
run app
check logs
run as xp sp3 and a few others
check logs
An error, wonderful!
Event ID 1001
Fault bucket , type 0
Event Name: PCA2
Response: Not available
Cab Id: 0
Found this error to be kind of common with number of games, without a real common solution for it on the internet.
Before I go further am I on the right track?
No :}
But I will add another hint to the challenge.
Is it about UI/Locale/IME resources?
nope
LoadString fails.
Did you miss some resource?
Since this is not a trick game, with my CSS background, the 1st thing I would do is to use process monitor to monitor the run.
From process monitor, there was no failure on file or registry loading. The last registry place before exit in resource related entries. The last API is LoadString.
With this information, I launched the application inside debugger and monitored the LoadString call.
The LoadString was called twice before exit. I changed the error handling behavior for the 2nd call of the LoadString, and the application did not exit any more.
Based on above, I would suggest the “customer” to double check the resources. I would give the resource ID to the customer as a clue. (something like 0x67? I closed my debugger when writing this)
It took me about 15 mins total time. ProcessMonitor is always the best tool to start with for such kind of problem.
Good investigation. The actual root cause is a bit simpler (I bet that LoadString is used internally by the actual failure).
Hi Alan!
My report, with solution, can be found here: http://testing.gershon.info/rapidreporter/pairingrr/AngryWeaselTheApp/
I lost a lot of time hunting some ghosts, but inevitably turned to ProcMon for answers :).
You can skip the ghost hunting part, but I think it is interesting too!