10 Lessons We Learned During “The Good, the Bad, and the Buggy” This Year
Test and Monitor | Posted December 18, 2018

This year we launched the first SmartBear podcast, “The Good, the Bad, and the Buggy.” The podcast explores different cases of testing, development, and design to examine how technology shapes the way you think of different companies.

We’re thrilled to have reached Episode 14, and we’ve enjoyed the support of our fans and episode guests along the way:

In the spirit of the New Year, we wanted to look back on what we’ve talked about on the podcast and learned in our research during 2018. Whether you’ve tuned to every episode bi-weekly (like our parents), listened to a few here and there, or have never heard of us before, here’s a breakdown of our top takeaways from the first 10 episodes:

  1. Ensure your app is easy for first-time users to navigate...or they may not come back - Our adventure with MoviePass is what inspired us to start this podcast, and it’s something we’ve talked about a lot (just ask our co-workers, or read the blog we wrote about it). As first-time users of the service that, at the time, offered unlimited movies in theaters for $10 a month, it would be an understatement to say we were excited to try out MoviePass and go to our first show. We were able to check in to the movie early since we were inside the parameters set by the geolocation API. However, this meant we got locked out of buying our ticket when we got actually to the theater because of the 30-minute window for checkin. After much frusutration and a few conversations with the in-app support admist panicking that we would miss our movie, we were finally able to get our tickets. Unfortunately, the damage was already done, and this left a negative impression of MoviePass on us. Though we eventually familiarized ourselves with the process, it raised our intrigue to the negative affect a bad user experience can have on customers.
  2. Having intoxicated user tests your app just might provide the brutally honest feedback you need - Drunk user testing is, in fact, a real method of testing that revolves around the idea that “your application should be so easy to use a drunk person could do it”. We experienced this first-hand when we went to Appcues’ Drunk User Testing event and then talked to a professional, Richard Littauer of TheUserIsDrunk.com. While your first instinct may be to write off the idea of drunk testing for a number of reasons, we found that it’s actually a great way to get honest user feedback whether you’re coming out with a new app, updating your UI, or adding a new featuring.
  3. Dark patterns may give you short term success at the risk of compromising your reputation - A dark pattern is a UX device on an app or website used to trick people into completing an action -- whether it be clicking something, signing up for something, or buying something -- that they didn’t intend. Dark patterns have become really common, but they’re not always something we recognize, and they could be costing you your users’ trust. When looking for dark patterns, we’ve found that some of them have become industry standard, such as when you sign up for a free trial and your credit card is automatically charged after, while others are sneakier. As software professionals, it’s crucial that we look out for these patterns so we can make the effort to avoid them and focus on user-friendly design instead.
  4. Don't forget about load testing and monitoring -- especially on high traffic days - If you’re one of Amazon’s many customers, you’re probably aware of Amazon Prime day. Unfortunately, Amazon didn’t quite realize how popular it was, and Amazon’s website experienced serious downtime on their biggest day of the year -- losing about $1 million every minute. While Amazon alleviated some disappointment with 404 pages that featured adorable dogs, some were still unsatisfied that they couldn’t access the day’s deals. We chatted with Director of Product Management for Alertsite, Denis Goodwin, to find out more and learned how important it is to test the entire user journey -- not just the homepage of your site -- and to set up monitoring so you can know as soon as there’s downtime.
  5. Diversity in technology is important for more than just hiring statistics - We hear all the time about the lack of diversity in technology, but we don’t often talk about how it affects the products you release. In fact, when diverse data is not considered during testing and development, it may not work at all for certain people. You can read Angie Jone’s blog, which includes various examples of what can go wrong when this happens. While sometimes these issues can be mildly inconvenient or annoying, other times they may be downright dangerous. As technology evolves and we rely more than ever on digital devices, we need to be careful that we consider diversity in our data.
  6. Robots aren't replacing our jobs, and automation isn't replacing testers - When we heard there was a new restaurant where robots cook your food, we had to try it out. We just couldn’t help comparing the automated kitchen to automated testing, so we invited Paul Grizzaffi to join us and talk about the state of manual vs automated testing. While automation made the restaurants process more efficient, we noted that there were still many humans behind the process between the engineers, the chefs, and the workers topping off your bowl. Similarly, in testing, we recognize that automation has made our lives easier in many ways, but it’s a tool to complement human testers, not replace them.
  7. Making your application accessible doesn't have to mean compromising design - In an effort to understand accessibility better, we downloaded the plugin “See” and experienced what it’s like to be colorblind for the day. What we found is that many websites use the colors red and green to initiate certain actions like stop and go, which can make usability difficult for people that can’t differentiate between these colors. As we consider the different abilities of people who interact with our applications every day, it’s important to also give more thought to designing and testing for accessibility. While some may have the idea that this puts certain limitations on how you build out an app, we alternatively found that it has often helped improve overall usability and impacted innovation throughout history -- from the typewriter to your smartphone.
  8. Meet your users where they are on the devices they use - The fitness industry used to be all about boutique, in-person workout classes. With new technologies, however, more fitness companies are creating experiences for users to access excersise classes when they want, where they want, and how they want. We talked to a Software Engineer at Aaptiv, an audio fitness app, to learn a little bit more about what it takes to appeal to users that want more flexible fitness options and are using technology to work around their schedules.
  9. Bugs will costs you - How much will a software bug cost you? It depends, but speaking about some of the worst software bugs of all time, we wanted highlight just how important testing is -- not only for your reputation, but also for your bottom line. In 2017, software failures cost the US economy over $1.7 trillion, which really put things into perspective. We shouldn’t be expecting to release 100% bug free software, but by testing earlier and more often in the software development life cycle, we can be more confident that a critical bug won’t slip through and make us go bankrupt.
  10. Securing customer data should be a top priority - Any company who has experienced a hack will tell you how detrimental it is to your reputation (we’re looking at you Equifax, Home Depot, and Facebook). We learned how to keep your information safe and your customers’ information safe from McAfee’s Chief Consume Security Evangelist, Gary Davis. Plus we discovered what penetration testing is and discussed why you might actually want to hire a hacker.

We presented this live episode for the attendess as SmartBear Connect in September, but these takeaways just skim the surface of what we learned in our first ten episodes. Since then we’ve talked about topics from smart homes to artifical intelligence, and we have more planned in the coming months.

We hope you can take some of these lessons into 2019 with you and join us in the episodes to come! Check out the full live episode from SmartBear Connect below, or visit the podcast homepage (we're also on iTunes, Spotifiy, Stitcher, and Google Play!) to see all episodes.