David Hamill
The Good, the Bad, and the Ugly of Remote Research
In this straight-talking interview, David Hamill discusses the ups and downs and side-to-sides of UX research, including why it's okay to make mistakes when moderating and why unmoderated research can be dangerous.
Highlights include:
- How can UX researchers best help product managers?
- Is UX research under threat by automation?
- What on Earth is a usability test ‘punty’?
Who is David Hamill?
David Hamill is a UX consultant, coach and strategist, with 20+ years of experience in usability and user research.
Joining Skyscanner when it was an early-stage startup, David embedded UX research as a fundamental part of the company’s product practice.
Now running his own UX consultancy, David is a sought-after advisor and coach for high-growth companies who want to run effective user research and up-skill their product teams on the latest techniques.
Transcript
- Brendan Jarvis:
- Hello and welcome to another episode of Brave UX. I'm Brendan Jarvis, Managing Founder of The Space InBetween and it's my job to help you to put the pieces of the product puzzle together. I do that by interviewing world-class experts in UX research and product management and they share their stories, learnings, and advice with us. My guest today is David Hamel. David is a UX research consultant coach and strategist hailing from Edinburgh, Scotland. He has 17 years of experience helping companies of all sizes and stages to make smarter product and design decisions. David received his first taste of UX in 2003 while working to improve the usability of the Royal Bank of Scotland's internet. After leaving the bank, he worked as a usability consultant at the well respected user vision where he led projects for the bbc, the Royal Navy, glass Direct, and many more, not one to sit still.
- David established his own UX research practice in 2008 where he consulted with a range of clients, including the then early stage company Skyscanner. In 2013, Skyscanner made David an offer he couldn't refuse and David joined the company as a full-time senior user researcher. While there, David introduced human-centered design practices and established user research as a fundamental part of product development. Leaving Skyscanner in early 2019, David launched up UX where he is a sought after consultant and coach for high growth companies wanting to run effective user research and to upskill their product teams and active contributed to our field. David has spoken at events such as Northern UX and Product Tank. He also authors a blog on Medium where he is had his post featured in many popular channels including UX Collective, UX Planet, and the startup. David has also recently launched a YouTube channel where he shares insightful and practical UX tips. David, welcome to the show.
- David Hamill:
- Thanks Brendan. Its good to be on. I'm quite impressed by that. You mentioned things that I'd forgotten. I sorry, my coffee machines just about to turn off, so you're gonna have to Yeah, no worries.
- Brendan Jarvis:
- It's early morning for you. I'm glad you getting fueled up on coffee. We've got a big sort of list of topics ahead of us that I'm sure our audience will be really keen to hear from you about. But before we get into anything that's too heavy well this could actually go the other way. And while this isn't a show about football, I have to ask, is it Cel Rangers?
- David Hamill:
- Actually. So Cel Rangers are Glasgow teams and well, most of the people in Scotland either support Cel Rangers because they're the only real good teams but I'm a Harpo and that's club for those who have no idea who harps are.
- Brendan Jarvis:
- I'm glad that we won't be offending half of Scotland from this point on.
- David Hamill:
- Oh give. Oh, [laugh] [laugh].
- Brendan Jarvis:
- Hey. So unlike some of the other guests that I've had at the show on the show who have come to UX from HCI or from psychology I understand that you are a reformed marketer. What's the story there?
- David Hamill:
- I started off doing a degree in economics and then realized there was actually quite a lot of hard work and back then I was a bit of a slacker. I guess I maybe still am now then. So I transferred to marketing cuz it just seemed more fun, more interesting. But then when it came to work, I didn't really enjoy marketing as a profession. I mean I don't want to disrespect any marketers, but I just didn't enjoy it. No,
- Brendan Jarvis:
- Listen to anyway,
- David Hamill:
- [laugh]. So I ended up sort of teaching myself web design because back then that's the way people learned. Web designers, they taught themselves to, there were no degree courses or anything on these things. And then that sort of progressed me towards working at RBS and then at RBS that's where I or Royal Bank of Scotland, for people who don't know what RBS is and picked up user experience, user center design, whatever you want to call it.
- Brendan Jarvis:
- Yeah, I understand that you had a bit of a reputation for asking annoying questions. How do we know any of this stuff is going to work?
- David Hamill:
- Yeah, so I used to ask that and then of realized there was a job and so my boss at the time, Melanie Thompson, she was a great boss and she actually only gave me a job because I made her laugh in the interview, which is [laugh] a bit of a lucky term on my part. But I guess she recognized that sort of quality and in me she knew Kate Cox, who, who's still at RBS in fact, and essentially steered me her way in order to learn more about it. And that's the way my career went in that direction after that.
- Brendan Jarvis:
- Yeah. So it sounds like Kate was quite important in your sort of entry into the field.
- David Hamill:
- Yeah, she was. I think she taught me and in the same way that I've done, I've done numerous people now is sort of showed me how to do usability testing and then again it gave me the confidence that I was actually quite good at it. So the way that I do this is with the people is often based on the way that she did with me was run a couple of tests with the person watching. So with me watching and then get me to do the rest of them with her watching and then with the view that she was going to give pointers about things to improve, et cetera. And after the first one she was like you pretty much nailed that. You're a bit of a natural at it. And yeah, so I just seemed to have the aptitude for the discipline quite early on.
- Brendan Jarvis:
- And I'll also understand that you might know my previous guest on the show, Dr. David Travis.
- David Hamill:
- And we were on the lookout for somebody to help with essentially the refactor of the entire internet and David won that contract. And so I just, [laugh] sort of shadowed him, didn't leave him alone. [laugh] when was wherever he was doing that, I just sort of spectated and et cetera. And he's been quite I David's very generous with his time and so all the way through my career if I've had a question about something, generally he's been the person that I go to about it and very normally it's talking to him the same date that I asked the question. So yeah, he's been very generous with this time. I'd say I've had to say I had a mentor, I'd say David was that person.
- Brendan Jarvis:
- Yeah, he's a good man. He's made a big impact on the field. Yeah,
- David Hamill:
- Absolutely.
- Brendan Jarvis:
- I also understand that we might have something coming up that we're gonna talk about that you might disagree on, so we won't give that away just yet, but hopefully it's not an O one can nobi sort of Vata
- David Hamill:
- [laugh].
- Brendan Jarvis:
- No,
- David Hamill:
- No. We've had these discussions, we've had the discussions in the pub already, so we're aware of different views on things
- Brendan Jarvis:
- Where in safe territory then that That's good to hear. Hey, so you, you've been working as an employed consultant as well as being self-employed Now what was it that initially inspired you to go out on into your own consultancy that first time?
- David Hamill:
- So when I first went into consultancy was really, it was to learn. I thought that was the best pre for me to learn more. And that turned out to be true cause you just get a massive range of a massive range of different projects to work on. It's a great way of getting diversity of experience. But the, there's something that there's a chap who, and he would know at Amazon Miles Hunter who used to work for David as a sort of consultant for David. And he told me when I started there, he don't stay 18 months and then go freelance and I never really sort acknowledge that at the time, but [laugh] about 18 months then I was like, I found out that I like to have control of the design of the research of try and get the best, do the best work I can for whatever I'm working on.
- And often and often being in an agency can get in the way of that because you know, want to hit sales targets or whatnot. Or actually it's largely to do with the way that they have to pitch for work. So essentially have to tell them what work they're going to do before they get the work. And so 100% of the time when you go on to do that work you learn stuff that would would then, if you'd known it, you'd done the work differently. So I just wanted to do better work and essentially have more control over that. And so being freelance was, was the way to do that. And another time, and David Travis gave me a big confidence boost at the time won't tell you what he did or said, but he's given a big confidence boost at the time where I thought, well if he believes in me to that extent, then why on earth I believe in myself to that extent. So then I just, alright, I'm off. And
- Brendan Jarvis:
- There's always people in the journey that are really important and helping you take the next step.
- David Hamill:
- So I try now to be, cuz I'm getting on it, but I try to be that person to other people. [laugh] tell one
- Brendan Jarvis:
- I've got more gray hair than you do. I dunno what you're talking about.
- David Hamill:
- No actually it's just got more hair than me. [laugh].
- Brendan Jarvis:
- Hey, hey look, turning our attention to something that's probably a bit more of a serious topic. I mean we're obviously living in this world with Covid 19 at the moment, [affirmative] and it's having devastating effects around most of the globe. From your observations and experiences since March when it all sort took off, how has it impacted UX research in your practice?
- David Hamill:
- The biggest impact that I guess it had is what I used to have was live observation from a client's office. So for example, quite a lot of the research I do is on Zoom, pardon me but I don't wanna have a zoom call with 25 people on it and one participant that's just not on for the participant. So what I would do was have I'd book a meeting room at the client's office, have a zoom call from there, it was muted, no camera, et cetera. And it was just the meeting room and really just, that would be the only observer in the session. And so that means a lot of people could live observe without the person feeling that there was a ton of people on the call. Now because more often than not the clients are at home, then I'll say, well you can have one person observe and live, everybody else is watching the video type of thing. Maybe the bigger impact that I've had,
- Brendan Jarvis:
- David Travis shared with me some concern about the use of unmoderated usability studies [affirmative] because he felt that the overuse of those studies took away from that observed experience because people were unlikely to sit through six or seven sessions in their own time and sort replay in full what it was that the person was doing. What has been your experience with our moderated usability studies and how have you found them in your mix of research methods?
- David Hamill:
- Okay, so my biggest use of unmoderated of essentially usability testing, I guess it was mostly was during my time at Sky Scan. And so I was watching your interview with Elizabeth Allen and her time at Prosper, I think it was, it just ticked to all the boxes of my, I was like, yep, yep. Just nodding away. And part of that is that you're more in demand. You're in demand and you basically have to choose the most important things to work on. But if you wanted people to still be engaged in doing user research, they're either gonna go ahead without you or you can give them something to help. And so unmoderated usability testing helped to fill that gap cause I could help with that. I could help with that without it taking up being the thing that I'm working on that week. If I'm running moderated user research or for somebody, I'm not juggling a number of projects at the same time because the context switching that's required to do that means you're gonna do a rubbish job at it.
- So my head would be in one place at a time and unmoderated testing helped drastically with that. So a lot cause a lot of people, so when you've got a tech company that you've got load of product teams and a sort of joke that when you go and work for a get these people who are get to the top of their game and then they go and work. Sorry, I realize you've probably watched some of my talks already. So you must be used to seeing these tangents happen that I go off on one. So when [laugh], as long as
- Brendan Jarvis:
- You don't do any haing and let's tell everyone what Havening
- David Hamill:
- Yeah, havening, yeah. So yeah, we'll come back to Havening. But yeah, this is a, that's happening. The people get to the top of their game say design, and they go and work at Google or Facebook or whatnot and then they're working on a settings menu in an app and they're trying to affect something in some way is sort of my idea of hell. But the research that's required to do that is fairly low touch. Can people change the sort of privacy setting in this menus? That's unmoderated usability testing fodder if ever I saw it. And so it allows an overstretched UX researcher to satisfy lower level importance projects without them doing nothing and just doing it wrong or making a big path. I'm not really
- Brendan Jarvis:
- Sure you and David Travis do disagree. I was hoping for a bit of BFO tonight.
- David Hamill:
- No, no, no, no. So with, David gave a good talk at Northern UX as well on this subject, which I'm sure you've watched now that I've seen how much you research you clearly go into before you speak to somebody and that's what he was essentially talking about. But I think we agree it is, it's just, it's just the fact that he had reservations about it and the message that I get from David is don't do it. I have reservations about it, but my message is do it [laugh] but don't rely on it alone. And so one thing that I pick up from a lot of more experienced UXers is this view that they're under threat. Like the disciplines, their work is under threat. The value of their work is under threat. Cause they are these cheap rubbish methods that are threatening them. If they really are cheap and rubbish, then they're no threat because in my experience I often end up working for a company that initially thought they had no need for somebody like me because they hadn't nailed, oh no, we've hired a UX designer, were good. And then you
- Brendan Jarvis:
- 'em laugh on the interview again.
- David Hamill:
- Sorry,
- Brendan Jarvis:
- Make them laugh on the interview again.
- David Hamill:
- [laugh]. Yeah, so people realize when the results, well the companies that I'd like to work for realize when they're not getting the results that they're asked. There are other companies that do UXs theater, they're always gonna do the same thing, but UX is part of the theater that gives them all the sense of security that they're doing it well and they then release stuff that's not that good. But other companies realize that they're not getting the results that they're after and they're wondering why that is. And often it's because that they're not doing it very well and then you need somebody to help them. So if there is truly value in the work that you do, then it shouldn't go out a demand because a bad approach exists. The other thing that I'd say about, I'm not sure, well I'm not sure how well I articulated that, but the other thing about that is people, practically every client that I work with, if they've been using unmoderated testing, I sort of ask to watch some of the videos because it's generally the way that they've approached it, generally quite low quality and they're all, they're often making a lot of the same mistakes. What
- Brendan Jarvis:
- Would some of those mistakes be?
- David Hamill:
- So [laugh], I knew that was gonna be the question [laugh] steal for a blog post [laugh]. But since you asked [laugh] the first and most widely spread one in Unmoderated is either no screener questions or bad screener questions. And so they've just used the demographic filters and then they're asking somebody to use an accounting platform or something like that. And so Moira from Moira from Bos who worked in the sweet shop is trying to use [laugh], an accounting, a bit of accounting software and well that's not your target user is it? Unless Moira from the Sweet Shop bonus owns the sweet shop and has accounting software than shes, but so no screen of questions, but mostly it's bad screen of questions. So say for example, you run a website that sells cars. I was, and so you think, well we'd like to talk to people who have done this before or actually who are in the market for a car just now.
- And so they add a screener question, say, are you looking to buy a new car? Yes or no? And really when you understand how these platforms work, generally most of them are big panels and quite a lot of the protestors as people call 'em, they've got five, six accounts under different email addresses where they're in one time they're an account executive for a larger organization and another one they're a student. So they get invites to all the questions regardless of the demographic filters that you ask for. And so if you ask a screener question that's got a yes no and then the person who's desperate to get on that test who's gonna be the quickest to respond to the screener, is know exactly how to answer that in order to get on it. So then you've got somebody blagging through a session rather than somebody that's no. Then the other things is asking whether people would use it, would you use this? Which that's just not unmoderated moderated bad research. That's just bad research and there's nothing about it being unmoderated that makes that a bad question to ask. And there's no difference if you ask that question in person, it doesn't make that question any better. It's still a sucky question. David are
- Brendan Jarvis:
- You saying that you actually need to know what you're doing before running
- David Hamill:
- Research? Exactly. Go. And so the problem isn't bad unmoderated research, bad research and unmoderated just makes it easier to access for people. And so they're often doing bad research because they're trying to do it as easy as possible. And so the tool's not really the problem, but there is a problem with the fact that it's so wide that it's making the amount of research is done. So widespreadly bad is as an issue that I recognize and of agree with but I'm quite a fan of using, using those tools. But it doesn't really replace in person tests. But every approach you take has benefits and drawbacks. And so all the people, for example, all the people lamenting and ask, you asked me earlier and I, I've said that actually it hasn't changed their research that I do that much. One of the reasons I like doing research remotely, which I used to actually hate I [laugh] because back then the technology was so flaky that it pretty much meant that you were letting down your client and it was making note your work look terrible because the call wouldn't happen.
- But now I sort of favorite because it's the quality of participant that you can get. If you've got a lesser geographic sort of constraint, then you can in a shorter period of time you can access those people. And obviously you say, oh well you know, should be going to visit people in their office for the best reception. That's fine. I agree for that one bit of research. Most of the time you'll be doing it better if you go and visit people, but that the time footprint in that research will be exponentially bigger than if you were doing it remotely. And so if you are in a sort fairly fast moving environment, you're work working in an agile environment like everybody is, well not everybody but a lot of people are then your learning is way further up the road because you've done four bits of research in the time somebody's planned and organized one and fair enough that one bit of reception did, they got some nuances that they would never have known if they hadn't gone in and visit. But that person's done four bits and so they've learned a whole lot more that you haven't. And so it's just how are you going to balance those things out? How important is one thing over another? And then you have to make a judgment call on those things.
- Brendan Jarvis:
- So one of the things that comes up regularly in my conversations with other researchers is the role of culture and enabling great research to happen. And this is something that I'm quite personally invested in. How did you go about building a culture that supported research at Skyscanner? Cause I understood you joined the company when there was only 20 employees or you first started working for them.
- David Hamill:
- Yeah, so I guess I wouldn't claim to have built the culture from that point. At that point I was in the office every so often doing a bit of work for them. It was really essentially when I joined as an employee, I tried to make it UX research a more standard practice in the work that we did. And really the way I did that was tried to do that was through proof. Just you know, couldn't convince everybody that they needed to stop the way that they were doing it and listen to this new guy that's come in because the day I started there was 14 other people starting that [laugh] day and imagine we all had to, and then the next week there was another 14 and et cetera, et cetera. And so you can't make the entire company stop and listen to every new person that starts a fast growing company.
- But I just started with proof. So I would stick my nose in the places where I think I would have the most impact and then try and prove that this was the approach that worked. And then so when you've got back then I say a dozen product managers and they're wanting to know why this product manager X is getting good results and they said, well one thing we're doing is research. And so the thing is I can't really claim that I went and I was some sort of apostle that came in and [laugh] sort told them all about UX research because essentially it was a fast growing company. So they were hiring product managers who were used to UX research. So they were people coming and going, so where's your UX research team? And they're like, you mean David [laugh]? So every so often I would get somebody appealing and email and then Slack when it came along.
- Brendan Jarvis:
- So what you sounds like what you would do is you would do poor quality and moderated remote usability studies for the product managers you didn't like and that you did look really good.
- David Hamill:
- No, they were doing nothing. They were doing opinion based stuff and when I joined there was still a lot of opinion and actually it's strategy wise, I guess I shouldn't get too much into that, but I still feel that we could could have been done from an evidence perspective than that. So a product team that we're working on something that had no was all based on assumption and didn't use any sort of usability testing to help improve as it went on either. There'd be lots of projects that would hit blank results and people would want to know, oh well what were we doing wrong? And then there was other projects where people were getting success and there was a correlation there that happened that they're either I was doing research for the more I was helping the product managers could quite a lot of the time the product managers were like some of them were so into it, they wanted to be the ones doing it and that's fine, I just helped them do that.
- And persuade with proof is I sort try and still take that view now. So when I'm working with a client it's easier for me to just get a bit of trust from them. We're gonna do this bit of work and then I can try and sell you, but if you trust me and we do this, you're gonna see for yourself that it's worthwhile. And if you don't then if you're not convinced, I don't really want to do the work anyway. So it's win-win from that. So yeah, persuading with demonstrations is sort of my go-to approach.
- Brendan Jarvis:
- I mean there seems to be a bit of a culture, particularly when it comes to the management levels of organizations to rely more heavily in their decision making on quantifiable data. So you talked about opinions and I'm sure that surveys played a bit of a role in things there. What is it that you believe that, I shouldn't say that some people when it comes to making decisions sort of devalue or don't value as to equal footing qualitative research?
- David Hamill:
- Partly I think it's because every individual people have got in their heads when they work in a company they've, whether they vocalize it or not, they've probably got in their heads what their target user looks like not physically looks like. And whenever they see a of user research session because that person doesn't actually exist whenever they see a user research session, they see somebody who's maybe understands a bit less or has different domain knowledge entirely or has different mental models and it's easy for them to dismiss that because it doesn't match that sort of made up person in their head. But then the numbers are the numbers and so they're sort of irrefutable. I do think from UX research point of view is that from a sort of business point of view, businesses are over relying on numbers that they don't know. They don't understand anything behind why those numbers are the way that they are.
- But from a UX research point of view, it's too far. The other side, they way too, as a discipline we're way too reliant on qualitative techniques and mistaking that for reality. So for example, if you run a, let's say you run a website what was the example I used last time? I can't remember what it was now. But let's say you're selling shoes. So you run some usability tests of people using your shoe selling website and you see certain behaviors if you record that screen and just take the audio away from it and see how the person's going through the screen and say for instance you've got screen record session recording software that what runs on your website, if you compare the screen movement on a usability test to actual behavior, you find it's markedly different behavior. And so UX research is a habit of thinking that what they're observing is the reality of what happens rather than an indication of some of the things that are going on behind the reality of what happened. So I've answered a different question, I'm sorry, but [laugh], yeah, what
- Brendan Jarvis:
- You were describing, people watching say screen recordings through Hotjar or something similar and not having that behavioral input of the person's face as doing that at task.
- David Hamill:
- Yeah, so I mean there are clients that I've worked for, they've had Hot Jar, full story or whatever good. And what they see is reality on the session recording because that it actually happened. But then they invent a rationale behind it and then they mistake that rationale for the reality because the session recording was real, but the explanation is theirs. So their made up explanation becomes fact because it's based on a fact, it is an assumption based on a fact. And then that becomes a fact. And really I'm just a big fan of triangulation of stuff and qualitative research is a massive part of that. But if you're not used to using whatever method that you can to find the evidence that you need then you're leaving Behar in my view, you're leaving behind the big part of your job. But that maybe we're then touching on the fact that actually as a discipline, UX research is becoming more about the research than the UX. What's
- Brendan Jarvis:
- Tell me about that.
- David Hamill:
- So now we've got UX researchers who do any sort of method that exists and their focus is on methodology of research design facilitation and being a good research is just about good moderation or whatnot. And really UX research is just a means to an end and the end is making good decisions. And so you can do amazing research but if the good decisions don't happen then your work is pointless. And so I think as a discipline we just dwell too much on cognitive biases and et cetera, et cetera. Trying to sound pseudoscientific, not pseudoscientific but you know what I mean, but to try and elevate our status but not actually helping people make decisions. Cause you could have done an amazing research and if it does, but if it doesn't, but nobody makes the decision whether it's like a tree falling over in the wood when nobody was there.
- Brendan Jarvis:
- You talked about helping product managers with research at Skyscanner. So it sounds like what you're talking about here is making sure that the outcomes of research get implemented. What are some of the things that as a researcher you can do to assist the product manager or whoever is in charge of the design decision to make the right decisions?
- David Hamill:
- So I think to begin with it, you need to begin with by trying to track what you found, what was done and then the outcome rather than do the research walk away because you can never help anybody with anything if that's all you ever do because all you do is research you didn't track the decisions that were made you didn't track the impact of those decisions on the sort of business outcomes. So you can't actually give any advice unless you doing that. The product manager has a better view on that than you because they [laugh] had to make decisions like that and they've seen the outcome of decisions like that. And so you're not really in a position to advise that product manager. If all you're doing is research and then chucking it over the wall or actually allowing them to analyze it themselves and then walking away and doing the next one, then all you are good for is research is facilitating research. And if that's the sort of job that you want to do, then that's fair enough. But at some point you're gonna come across somebody who needs a better help. And so going for tracking, the reason for following the decisions that were made, what the outcomes were then allows you to give better advice.
- Brendan Jarvis:
- In your last blog post, you spoke about company leadership spending less time in front of you as they grow as the company grows and people put in layers of management in between the founders and the user customer. If they're successful, if the founders or the management are successful and the company is growing and making money by them not spending time in front of users, why should they?
- David Hamill:
- Because there's always somebody there else coming along that's trying to eat your dinner or breakfast or whatever they call it. Yeah. So you become the incumbent and then you're like how are these other companies able to come up with this sort better way of this, that and the next things? Well, because they're the version of you 10 years ago, [laugh] is essentially the answer there because the layers of management is one thing that inhibits it because every layer of management wants to have an input on something. So people talk about autonomy of product teams and it doesn't exist as much as people pretend that it does.
- And so every layer of management wants to have their input of what should be done and before it. People who are nothing like your users and have actually no contact with your users are the ones that are calling the bigger shots rather than the people that are spending time trying to learn about it are the ones that aren't really getting listened to anymore. Whereas a founder will generally be all over understanding what user needs and behavior are but after a while they just get too busy. And then it's what happens after that that can dictate how user centered the company remains. I'm sure there are some big companies that are still really sort really user centered. I just haven't come across them.
- Brendan Jarvis:
- And I think it's probably fair to say that there's quite a few founders out there that would've been doing their own user research, possibly guerrilla user testing at the local coffee shop. That's fast and it's cheap isn't to about that method
- David Hamill:
- A few things, but for us, for a startup founder is usually, more often than not your products new, it's a new concept that doesn't exist or hadn't exist until you came up with it. So also there's an incumbent approach of doing things that that's not the way that your product does it. And so you can see how so approachable the usability of what you've got is to the average coffee shop dweller but that's only our mindia of the user research that you need to do. And if you're making all your decisions based on talking to people in coffee shops, then you're not talking to the, so you're not talking to the people who are next most likely to adopt your product and the adoption of these new things. A great example is software like Zero or free agent like accounting software still got companies out there using the old ones. Not everybody still everybody uses zero, but lots of people use Zero now as accounting software. But if you went back sort of five years, I dunno how actually Old Zero is, it's certainly more than five years old.
- The type of people that use using Zero back then are different than the ones that have adopted it since. And so in a sense what I'm referring to is this tech adoption life cycle that you'll read about in books. There's a book Bridging the Gap that talk talks about this tech adoption life cycle. And I sort of simplify that to the fact that your early users are the ones that are most motivated and able to adopt it because they need it and they're more likely the type of people that go looking for new ways of doing things. So that person with the vanilla latte is not that person in that coffee shop. Well it might be, but these people are sort of in the minority. So if you're going throw a stone in a coffee shop, you're unlikely to hit somebody like that. Early days of Sky Scan when I was recruiting for research participants, I had to say books, flights online has to had to be one of those considerations has booked a flight online in the last six months because without it, back then people would say, oh I'll just phone, I'd phone the I'D phone the airline.
- And behaviors change and you actually search sometimes a bit sort blamed to the fact that can happen.
- Brendan Jarvis:
- So I've heard you speak about the importance of being human when carrying out user research in particular usability testing [affirmative], what do you mean about that?
- David Hamill:
- At the time I gave that talk, we've probably moved on a little bit that it's probably less needed and it's a less needed thing to say. But previously there was a very sort almost like a laboratory approach to usability testing in that you were trying not to influence the participant in any way. So essentially you were being completely cold to the fact that they existed or that they knew you were there [laugh] and like so the fact that you're trying your hardest to make nothing that you say influence their behavior leads to an influence on their behavior. [laugh] because they're this robot sitting next to [laugh] that in a [laugh]
- Brendan Jarvis:
- Lab, it's
- David Hamill:
- Not back to the being human thing, is that in the quest for actually not making mistakes, people can end up just being so petrified of seeing anything that was anything other than what was scripted that it just leads to a very sort of dry and uninformed session. Whereas if you relax and make mistakes because so long as you, you're aware of the mistakes that sometimes you say something like, oh I put it like that or whatever, just relax and make them because you'll get a much better session from the participant by just being a human that they can speak to than being this person that's like nervous, a nervous wreck that doesn't wanna make any mistakes. And so what
- Brendan Jarvis:
- Are some tips you've got for people that may be nervous about running a or moderating a session? What sort of things can they do to be more human
- David Hamill:
- Moving away from everything? Being rigid to everything, being sort of flexible but just having direction. So to begin with, for example, you might need an introduction that that's word for word cause you've not got it in your head to say it. So for your purposes you need to just read it out but as a matter of course trying to move away from that. So then the next thing you can do is is write yourself a list of prompts. Make sure you cover these things so you can say it in a more human way rather than reading it off a off a script. But you've got a we reminder checklist and then you can move away away from that. So as my introduction, I do from memory but that doesn't mean it's great because I, oh yeah, I forgot to say this and I'm much happier saying, oh I forgot to say this than give myself distraction.
- Cause I want to focus on the person. So if I can remove anything that's a distraction from that I do. And so I try to have as little stuff scripted as possible. So for example, if you're doing a user interview don't have a script. You can have some questions written down but don't ask them. And that sounds really stupid, like writing questions down that you don't ask. But if you can jot yourself down some themes that you wanna make sure you've covered and then write questions that you've not, so then when you say, when you hit a dead spot or something in the conversation you can go, is there anything I've not asked? And then you can see the question go, oh wait a minute, it was not covered that. And then you can ask it. But if you write a script of questions, then you're just gonna ask a question, get an answer, ask a question, [affirmative]. Yeah, and the level of insight you get into that sessions way lower than when you've got a good idea of what you want to try and find out and you find that out with a conversation.
- Brendan Jarvis:
- And I find that myself actually interviewing people for the show is that a series of topics and questions, but often the discussion goes in very different directions and you have to be able to, I suppose, follow it along to catch people where they are and go from one thing to another. So we were talking a little bit before just about the mindset of the moderator and the usability testing environment and how they can become a bit more relaxed and be a bit more human. What about the mindset that the product team or the people that are observing the session have? What expectations should they have of what they're about to see, particularly when they've been involved in the design?
- David Hamill:
- Anybody that's making any decisions on part of the thing that we're doing is as they are watching as many of the sessions as possible, they don't have, well, I mean it's hard now they're all at home because when I've got an observation room set up, it is good to, it's hard to tell clients, give clients rules but say, look, if possible, no phones, no laptops because they're just distractions. And then a bit of guidance about try not to think about solutions when you're watching because people always try and relate what they're seeing back to something that we're doing or an idea that, or the way the product currently exists is trying to forget all of that and just focus on that person. And if that person was the only user you had, then just listen to them. And hard, I think the hardest thing for people observing is maintaining concentration on what's being said.
- Cuz frankly, quite a lot of, lot of what they're listening to can at that point be boring to them. It can be boring, it's not boring to me but it can be boring to them and they can, so if they quickly get bored and start looking around the room et cetera, they missed the thing that they said in the midst of all the boarding stuff. And then the thing we said later on, when those two things get tied together, they have a big impact and they just miss it all because they get a bit bored. And so I try as much as I can to manage that. So for example, when I first started doing usability testing for Sky Scan, we did very short sessions. So 45 minutes tops, 15 minutes in between sessions and trying, so you've a bit boring, the next person's kind coming along so you just listen. So you're asking you're more variety happening there are more frequent breaks et cetera to try and keep people engaged in the sessions. Hopefully
- Brendan Jarvis:
- This will be a fun question to answer. What's a pti?
- David Hamill:
- So a Ponte. Ah, okay. So Aon is actually a pontis one of these words in Scotland it's not even universal in Scotland. So if I drive 20 miles that way, the word for it will be different probably. But pun is when you need if you're a kid and you're trying to get over a wall and the wall's too big
- Brendan Jarvis:
- When you're running from the place or something, well
- David Hamill:
- Look, you can use that example [laugh]. And what one of you does is you stand with your back to the wall with your hands like that and then your friend puts his on your hand and then you roll at that and so that gets them over the wall. That's where I grew up. That was called a panty.
- Brendan Jarvis:
- What is a panty in the context of a usability test?
- David Hamill:
- Yeah, so essentially any US big usability issues in website or software block you from finding out other ones. So they dominate the session to the extent this is why you should do it iteratively you know, fix those problems and then you find the next ones. But in a session, once something's an issue, you can either sit there and watch them stew in it for ages and be true to the not influence the session thing or you can give them a pun and get 'em over them over the walls. So you basically just get them beyond that bit. And again, it's less controversial to suggest that now. And I'd suggest that that's probably how we, as a discipline, we've got a bit better at usability testing.
- But a while back if I suggested doing that, people would be like, oh no, you don't do that cuz you don't get involved in the session. You're sort of contaminating the session. It's like, well yeah, but then I've got one finding to get [laugh]. There was a big brick wall at the start of the session and we saw nothing after that because I didn't get involved. Or you can give them a ponty over that wall and find more stuff behind it. And so that analogy was used as advice to once something's something's an issue and you can be confident that they're pretty much stuck there, then get them beyond it so you can find more things to fix because some people will get stuck at that wall, but usually other people get over that wall and so you want to find out things that are behind that wall that are a problem as well.
- Brendan Jarvis:
- I'm just conscious of time, so I might give us both a panty into the closing part of the interview.
- David Hamill:
- Okay.
- Brendan Jarvis:
- David, what is it that has kept you fighting the user's corner for the past 17 years?
- David Hamill:
- I find that I just find it interesting I guess I find it investigative element of it interesting and trying to decipher what's going on out there. It is a bit investigative and I really enjoy that bit of it, but I also I'm easily frustrated by things I like things that I have low patience for things that don't work. And so it just feels like I'm making it the world a bit better and making fewer of those things in the world. And also I can then turn that frustration into sort of benefit because I can decipher why that thing's happening the way it is. I just turn that frustration about things into my job. So it sort of eases the frustration knowing that I'm putting it to some sort of use that's about the best explanation that I have for why I keep going.
- Brendan Jarvis:
- The job's largely about observing behavior and you've observed quite a bit of behavior over the years. What has been the single most funny thing you've observed as a researcher?
- David Hamill:
- One that really got people in stitches and when I came back into an observation room after people had been watching the session and people and the people were on the floor, [laugh] sort of laughing was the lady who the lady pronounced Pacific as specific. It is usually done another way around. So when somebody says what's the specific thing that sometimes people say, what's the specific thing that you're talking about? Which is a, I've never really seen it the other way around [laugh]. And when you have a sort of desire to fly with Cathy specific. So she kept saying Cathy specific and I think they started calling her Cathy specific because she just kept saying Cathy specific I said of Cathy Pacific is as this airline. And I was just like straight faced all the way through it and I came into the observation room and everybody was just on the floor because people, when you say it once I was like, oh, that was quite funny. But they just, when it keeps going and it keeps going, it's quite
- Brendan Jarvis:
- I think you've seen a couple of the other interviews, so this is probably not unfamiliar to you, but we call this game, what's the first word that comes to mind?
- David Hamill:
- Yeah, I saw that with David.
- Brendan Jarvis:
- So we're gonna have three words. I'm gonna say the word and then you are gonna tell me the first word that comes to mind. We'll go through them one at a time. Make sense?
- David Hamill:
- Yep.
- Brendan Jarvis:
- All right. So I'll say a word, you tell me the first thing that comes to mind, here we go. Empathy
- David Hamill:
- I [laugh]. So the word [laugh] that came to mind w was not about empathy, it was about you and it wasn't flatter, it wasn't flattering because I've just pieced a, I've pieced together in my head the fact that how well you've done your homework [laugh]. So the word that I had in my head was, was crude term for a sort of fatherless child, shall we say?
- Brendan Jarvis:
- We'll, strategic research focus
- David Hamill:
- [sigh] words beginning with B [laugh] and that one and that one's like bs. And not to say that that strategic strategy doesn't exist, but it's often sort of a smoke screen for bullshit, basically.
- Brendan Jarvis:
- It's a ways word. Yeah,
- David Hamill:
- I mean strategies, essentially things that are a decision where, whether, where the impact is beyond the current decision. That's the way I see is that simple, but it's used by seniority to create a mystique for their opinions quite all the time.
- Brendan Jarvis:
- Right. Last word, are you ready?
- David Hamill:
- Okay.
- Brendan Jarvis:
- Marketing
- David Hamill:
- M a word that still hasn't come in, just selling I guess is essentially what comes into my head just now is just selling
- Brendan Jarvis:
- For everyone that's listening. David has an honest degree in marketing and so clearly it didn't make much of an impact.
- David Hamill:
- Yeah, you're assuming a certain level of knowledge becomes to anybody that's want to through a university degree, choose a marketing one. Just read the tech. That's really easy to get a degree in marketing. Don't even have to go to the lecture anyway. Sorry.
- Brendan Jarvis:
- What's your greatest hope for UX research once we're over COVID 19?
- David Hamill:
- That people realize that getting good UX researchers is really hard. There's a lot of people on the market that are UX researchers but the really good ones are really hard to find. And so if making location a dependency is basically just doing you a disservice I guess is what I would say, find good UX practitioners wherever they are. Yeah, I think the dependence on location, I guess is what I'd hope improves.
- Brendan Jarvis:
- Look, David, this has been a really great conversation. It has been a great pleasure having you on the show and I really do appreciate you so generously sharing your knowledge and your contribution to UX research over the years.
- David Hamill:
- Thanks very much for having me. I enjoy these chats as you can probably tell. So I, I've enjoyed it greatly. So thanks for asking me on Brendan.
- Brendan Jarvis:
- Yeah, my pleasure. And look, you've shared a lot of insights with us today and I'm sure that people are gonna get a lot out of it and it's gonna lead to creating even better products. What's some of the best ways that people can contact you if they're interested in following you or talking to you about research? How can they get in touch?
- David Hamill:
- Okay, so you did a good sales job on my [laugh] UX Tips YouTube channel that I, I've just sort of started so I think I'm under 40 subscribers just now. So [laugh] I think you
- Brendan Jarvis:
- Probably need of me. So I think you're doing
- David Hamill:
- As continu continuation will be dependent on people actually watching them. The best way to get in contact me would be either LinkedIn's, a really good way of getting in contact with me and on my website is up your p UX Bez. And what I'm started to do now from the last couple of weeks is my, I'm, I'm not meeting, I'm working at home, not meeting people about, is that if you need a 30 minute chat with me about something sort of you extra related, if it's about your career, about your business, your ideas, et cetera you can just book a Zoom call with me from my website. So you can just go up my website and book some time on my calendar. From there you can also email me, which is it, which is this latest technological email, which you can do from our website as well.
- Brendan Jarvis:
- Fantastic. That's a great idea. Having people being able to book time with you, David, that's really cool. Hey, so thank you to everyone who's tuned in. Everything that David and I have covered today. I'll be posting in the show notes including all the places that you can find, David, his website that he is mentioned, the blog YouTube channel, and his LinkedIn plus all of the other resources that we may have spoken about today. If you enjoy the show and you wanna hear me do more of these great conversations, don't forget to the video, make a comment, ask a question, anything that comes to mind that you might wanna know more about, and of course subscribe to the channel. And until next time, keep being brave.