Heck Yeah, We Test Ugly Prototypes
Nima is a big fan of “let’s test it.” We want to see how our product works in the hands of “real” people. By “real,” we mean people who really need what we’re building and who don’t work for us.
At the end of last year, one thing we really wanted to understand was our form factor. We made some material changes to the design and knew it’d be important to see how folks reacted. We also had a litany of specifics: Where do people carry the device? How easy is it to carry? How do they hold it? Now that they have it in hand, what are the foods they would want to test? How would they use something that gets brought out in social situations? How much food would people really want to test? What do people call each component?
Hardware is a little harder to iterate on that software, in that you have to have enough of a decent product to test before you just go out there. The question is how decent does the product need to be? Then, who do you get to help you test it?
The latter question being somewhat easier, we had a range from 6 year olds with hockey practice to moms who homeschool to young guys in college to women building careers – all of whom follow a gluten-free diet.
The question of how good/real the product had to be dominated our internal conversation for a day or two. Some folks on the team expressed concerns about testing something that incomplete and not-quite-beautiful. Others worried whether folks would get it if it didn’t have any working parts. The research and opinions in the wider world vary in their thoughts. It’s been my experience that what we were seeking answers to would be fine with even non-working versions.
The advent of 3D printers means that it’s easy to print versions of your product — even if they aren’t perfect, they still provide an idea of the general weight and the broad sketches of the design. We’re in an era where 3D printing is not completely widespread and so this ended up being really fun for our early testers.
We printed up versions (and no they were not pretty) of the product. Since part of the product involves a disposable pod, we had just a slug that we would use as a proxy for what that pod would do. We packaged these up and shipped them off to various places across the US. The nice thing about this is that it was immediately obvious we weren’t asking anyone about the colors or materials. They were going to need to use their imagination.
Each one was shipped with a quick page of “instructions” and the various pieces. We asked people to carry it with them for at least a week. We followed up with them via text and email. We did a short survey and phone calls to debrief at the end.
Guess what? We got awesome information from our early testers. We answered all the questions we set out to test and then some. We made changes to the form factor based on feedback. We also heard things that we hadn’t heard before we got started (we’ll share in a later post). We got a sense of the vocabulary that people would use for the sensor and the capsules. We also got a sense of how people felt using the device. This is really important to know – where we going to be able to solve an emotional need as well as a physical one? All signs point to yes.