My talk at the Metro Apps Seminar today. It was to an adventurous bunch of non-developers with no prior experience for the most part in building their own mobile applications from scratch.
I only have a brief experience to share, a few weeks back I approached a group of start-up gaming companies in Ultimo, the start-ups are at a similar stage to yourself, testing their beta applications, before they port them over to iPhone, iPad, Android and web platforms. So I offered them some Uni Grads to test their applications for free, as the grads needed real world experience.
I found it interesting that because of the internet, they run their business today essentially the same as the application consumer market. And what I took away from that experience is the value they got from complete end to end user experience testing.
So what that is – it's when you do usability testing not just for the functionality or the quality of the product, but for the complete user experience.
As a tester you are looking through the eyes of someone who is seeing a website for the very first time; or your app on an app store, or an ad with a link to Facebook, Linkedin..whatever.. What is that users experience in total – from the first point that the user interacts with the web, downloads, installs, logins, the first look, the first impressions, using it, to exiting and then possibly sharing it with their friends. The complete end to end – entire experience - that you will depend on to generate revenue stream for your business.
At any one of those points if the user is dissatisfied, they will walk away and you will lose them forever. So that is what you need to test for – at what point is there friction – at what point does the user disengage with your app - that is going to cost you this potential relationship. I think you really need to look for testers that have had no prior knowledge of your product – similar to the profile of a potential user – And testers who are able to give you deadset honest feedback - who are respectful but totally unsparing of your feelings; not there to make friends, but give you the touch-points on what is good user experience for your product.
'Can you be trusted?'
Honestly I don't think so. I think you guys would find it hard to test your own product from a new user perspective. We found that the developers are just too close to their apps, they tended to hover over us, checking to see that we were doing it 'right', trying to tell us to 'not look' at that part of the app..look 'over here' instead. Generally hovering over us like anxious parents dropping off their children on their first day at school. Traditionally speaking, in my experience that was true to form; creators tend to think in terms of wanting to test to prove their stuff works, not where it fails or misses the mark. I think too, getting some distance from your work is a very difficult thing if you have an intimate knowledge of the inner workings of your app, even if you're striving to be completely objective and rational.
'At what point do I lose the user, my potential customer? Who I depend on to generate revenue stream for my business?'
I made a basic template, which we found useful particularly if you are going to beta test forums online, it focuses feedback around what you are looking for and keeps it simple;
*At what point do they lose me as a user?
*What are the touch-points for a good user experience that should be here, but aren't?
*What do I think about the usability of the game?
*What’s my entire experience of the product?
*Do I feel like buying it? If so, at what point was that?
What you find out from this type of testing helps determine your priorities on what to work on next. (End.)