Jason Buhle

Episode 78
Brave UX

Travelling from Academia to Industry and Thriving

In this episode of Brave UX, Jason Buhle discusses what’s important when moving from academia to industry, UX research as a career path, and why usability research matters.

Highlights include:

Why is Neil Gaiman’s Sandman your LinkedIn banner image?
What did you learn when you moved from academic research to UX research?
Have you hired someone for a UXR role from a UX bootcamp background?
Why is there less rigorous usability research being conducted as UX research grows?
Do academic-origin UX researchers see usability research as beneath them?

June 21, 202201:13:52

Listen on:

Who is Jason Buhle, PhD?

Jason is a Managing Director at AnswerLab, the world’s largest independent consultancy, exclusively focused on UX Research.

At AnswerLab, Jason leads a team of a dozen UX Strategists, responsible for designing research programmes for some of the world’s most important and innovative companies, such as Google, Meta, Amazon, eBay, FedEx and Wells Fargo.

In 2012, Jason was awarded a PhD in Cognitive Neuroscience from Columbia University, for his research into cognitive and emotional control.

His related academic work has been cited over 5,000 times and features in 16 peer-reviewed publications, including top journals such as Psychological Science, Cerebral Cortex, and Nature Reviews Neuroscience.

Transcript

Brendan Jarvis:
Hello and welcome to another episode of Brave UX. I'm Brendan Jarvis, Managing Founder of The Space InBetween, the home of New Zealand's only specialist evaluative UX research practice and world-class UX lab, enabling brave teams across the globe to de-risk product design and equally brave leaders to shape and scale design culture. Here on Brave UX though, it's my job to help you to put the pieces of the product puzzle together. I do that by unpacking the stories, learnings, and expert advice of world-class UX, design and product management professionals. My guest today is Dr. Jason Buhle. Jason is a Managing Director at AnswerLab, the world's largest independent consultancy that's exclusively focused on UX research. At AnswerLab, Jason leads a team of a dozen UX strategists responsible for designing research programs for some of the world's most important and innovative companies. You may have heard of some of these before, companies such as Google Meta, Amazon, eBay, FedEx, and Wells Fargo.
In 2012, Jason was awarded a PhD in Cognitive Neuroscience from Columbia University, for his research into cognitive and emotional control. His related academic work has been cited over 5,000 times and featured in 16 peer-reviewed publications, including top journals such as Psychological Science, Cerebral Cortex, and Nature Reviews Neuroscience. Someone who believes in the importance and transformative power of education, Jason has maintained his connection to academia. He currently lectures Masters of Science and Applied Psychology students in UX Research at the University of Southern California. I've followed Jason's thoughtful contributions to the field for some time now, and I can tell you that I have been very much looking forward to this conversation with him on Brave UX today. Jason, welcome to the show.
Jason Buhle:
Oh, thank you so much for having me Brendan. Thank you for that warm introduction. I don't get called world-class every day.
Brendan Jarvis:
Well, you certainly are, sir. And it is good to have you here and people won't have realized is that I actually was on mute the first time that I was trying to record that introduction and we almost made it through to the end there, but I'm glad the second time around we got there. [laugh] glad didn't hear it. Hopefully the rest of the conversation will stay on record and we won't have to redo the whole thing. Hey Jason, I was, as I do, I do some research as you would probably know into the people that I have on the show. And one of the things that really captured my attention with you, of which there were a number, but the first thing I wanted to start with was on your LinkedIn profile there's a photo, a banner behind your profile image and it has what looks like it's a gothic person that has sand or something running through their fingers and beneath it there's a caption, which is the famous Martin Luther King. I have a dream.dot. What is the story of that photo? What was the point you were trying to make? Who were you speaking to? Give us some of the coloring context there.
Jason Buhle:
Yeah, for sure. So the event was an event for researchers here in southern California. It was hosted by ADP in Pasadena, put together by Nate Bolt, who's the founder of Ethnio. The topic that Nate selected for that event was comparison of UX research and market research, and he knows that I teach in the Masters of Applied Psychology at USC where we do have a lot of folks going into market research as well as UX research. So anybody made the other folks to speak the image that was on the screen. That's from Neil Gaiman's Sandman. So that's the
Brendan Jarvis:
Yeah, Neil Gaiman very, very famous author.
Jason Buhle:
You identified the sand accurately and the Sandman is the god of the dream world in the series. So I wanted to make the point that I had a dream and my dream was UX research, market research in other areas of applied insights coming together and working together cohesively without the friction that I think is so, or competition that is so common. And I thought it would be a little rude, honestly to put Martin Luther King up there for that particular degree. So that is why I chose the image of the Sandman.
Brendan Jarvis:
You, if you haven't already should have a conversation with Michaela Mora. she too also believes in bringing the discipline of market research and UX research these two together and removing some of that friction. I spoke with her a few weeks back on the podcast and I think the episode's out now. So yeah, probably some mutually beneficial territory to mul over there. So you had this dream of removing some of the friction between the disciplines together, these applied insights disciplines. What was your idea? What was the big idea there? How can we do that and what are some of the, I suppose, the practical takeaways that you shared with that audience when you were speaking with them?
Jason Buhle:
Yeah, well I think the good news is it's happening anyway. There's clearly been a trend in recent years. Probably think on both sides of the aisle. Certainly in the UX research space, people are embracing some of these traditional market research techniques. So you're seeing some of the quantitative techniques conjoint or Max for example, brought over, seen some folks using some of the techniques from areas like mystery shopping. I think there's a lot for us to learn. It's important to remember that market research has been around for many, many decades and they've helped a lot of techniques. They have a lot of great resources available. At the same time, I think that market research is increasingly aware of UX research. Digital of course, is becoming more important for many of the companies that traditionally had strong market research departments and now have to make sure to also bring in UX research and well, I think there is some pressure happening there.
Folks are noticing that there are a lot of great job opportunities in UX research. So I think there's some people who are really trying to cover a wider range of topics and methods and I think it's great. So there's a lot of cross-learning. What I would advocate for, and I know it's very hard to do, is for companies that do have robust departments in both areas to try to look for opportunities for integration, remove that competition. I've definitely heard from many folks, particularly at more traditional companies, so not necessarily the tech companies, but companies that have been around long a lot longer than UX research where the lines are quite established and held often by the market research folks have been around for a long time and don't want UFS research encroaching on their area. I think trying to break that down maybe have unified leadership across applied insights is a great opportunity for some of those companies to explore.
Brendan Jarvis:
I really like that idea of unified leadership. You mentioned market research in some traditional and established companies potentially being a little bit wary of UX research. It sounded like a bit of patch protection perhaps going on there. This is not uncommon across many disciplines. Actually UX can sometimes feel like that about product management, getting more involved in research and in other areas. What's behind this? What is the fear that we have or what is it that we think we're trying to protect when we do get a bit in the way of people who are trying to integrate some perspectives with an established discipline?
Jason Buhle:
I think if you ask people probably often they would say they're trying to protect research quality, integrity of their discipline, and I think that is true. Oh, of course. I think also people are worried about losing power influence potentially in some cases worried about losing budget or headcount or jobs. And I think those are all legitimate concerns, but partnership is such a strong way to bring the best insights, the best skills together, especially I think within the research domain. In my opinion, research is research and a great researcher actually can operate very successfully across some of these domains. I think we have so much to learn from each other, so I see tremendous benefit there, certainly within the sphere of research as much as possible.
Brendan Jarvis:
It sounds almost like it's status that's at stake across the board. And if we are to truly get the benefits of merging these fields, or at least temporarily bringing these perspectives together, we need to find a way to do that that is mutually beneficial to the status of the various disciplines in which we're trying to integrate
Jason Buhle:
For sure. I think that's a great point. The more we can make it a win-win, the more these things can happen. It's funny for me, I spent a long time in academia where sure, yeah, all kinds of competition but you don't have these large highly integrated organizations that are modern, large companies. Universities can be quite large, but they're really more like a franchise model in many ways. Individual researchers are kind of off doing their own thing to a large degree, but in companies there's so much integration. I think that's what breeds actually that kind of competition. Cause you really are competing at some level for the same resources, for the same impact. So I think it, it's harder to bring folks together and you really have to structure that win-win quality.
Brendan Jarvis:
Yeah, that's a key point. I'm glad you mentioned academia and education specifically. I had a look at your personal website, which I realize is maybe a little outta date yet, but you did have a really interesting section on there, which was talking about your teaching philosophy and I think it was the fourth principle where you say, and I'll quote you now, I want students to enjoy learning and thinking in my class. Most importantly, I have fun when I teach. I believe enthusiasm is contagious and I want my students to see how much I love what I do. Why is it that enthusiasm and that love for what you do so important to you?
Jason Buhle:
Yeah, that's a great question. I think it's just how I'm built. I like to go hard on things and I always have especially, I love thinking, I love analyzing. That's why I love being a researcher. It's actually why I love teaching as well because you're constantly asked difficult questions. You have to take that step back and think from a different perspective, think of a different way to connect to all the different learners in your classroom. And I just really enjoy it. I genuinely really enjoy it. Oh, I want students to be touched by that. I think it makes learning a lot more fun and I want them to enjoy grappling with problems. And I think we all, of course easy answers. There's something really satisfying, but knowing the right answer but really satisfying is to definitely work on something over time to get to an answer and to recognize that it doesn't have to be scary or terrible or super stressful to go through that process to get to an answer. I think that's a key part of the mindset of a successful
Brendan Jarvis:
Researcher. Now that's interesting because you talked about learning being enjoyable and being fun, but you also talked about it being difficult and that it's almost sounded like you were suggesting that things are more fun or more enjoyable when you have to work to get to the insight or the outcome.
Jason Buhle:
Yeah, well think about games like video games. Usually it's all getting the right level of challenge. If they're too easy, they're not fun. If they're impossible, they're not typically very fun very much fun either. And so I think that's when you're hunting for insights as a researcher, I think, oh, that right amount of challenge really kind of makes it into a game. And of course ultimately there are going to be tough questions. There are going to be tough questions that you do have to really grapple with and struggle with. Learning to not be afraid of that to dive into it to take the time is key for being a researcher and also a
Brendan Jarvis:
Student. Was there a professor or perhaps somebody else in your life that was responsible for passing on that enthusiasm to you?
Jason Buhle:
So many. I was really lucky. I went to a fantastic high school with really small classes where I was really taught to think and I think there was a genuine enthusiasm for thinking in my high school, Trinity School in New York City for any New Yorkers out there. And then really blessed to go to also a fantastic college. I went to Pomona College, which I chose in part cause of the small classes that focus on rigorous thinking and writing. But I would have to say most of all if I had to credit someone, I would've to credit my PhD advisor to Waer. And really everybody who influenced me significantly during my PhD process. Also Kevin Oxner and Ed Smith were two other mentors for me. And I think that's where I really learned that process of diving in engaging, sticking with something, recognizing that it can really take time and that I don't have to be afraid of that process.
Brendan Jarvis:
Yeah, that's such a key gift to have been given, or at least to have teased out from within you and your education. Clearly it's something you take really seriously and you've obviously been fortunate and no doubt worked really hard to study at the institutions that you have studied at. I noticed that while you were studying Columbia towards your PhD, you received a scholarship from the Ruth Bit Fund and you're now a member of the board there. What is the Ruth Bit Fund and who's it for?
Jason Buhle:
Yeah, thank you for asking. The Ruth per fund is a scholarship organization. It's really pretty unique in that we give scholarships to folks who are motivated spiritually and that can be, that's very broadly defined by us and so people define it in many ways, but we give scholarships to folks to pursue educational goals who have this spiritual motivation. And then I think what's different from a lot of organizations, a lot of scholarship organizations out there is that we focus not just on supporting them financially, but on creating an experience of fellowship for them. So we have programming that is some which is required that they participate in to really get to know the other fellows and to also get to know the fellows who are off grant. So you're considered a fellow for life once you've received a scholarship from the Ruth Burt. And actually it's really amazing, A lot of our events folks come back who got the scholarship even decades ago, and some people come back almost every time to our flagship event, which occurred twice a year. So yeah, that's the Ruth Bird. And I have been on the board for, well, I think about a decade. I'm actually stepping down next year. But yeah, fantastic organization, if that fits anyone out there, definitely encourage them to take a look at the website and apply. And we do fund, actually a lot of organizations only fund undergraduate students or doctoral students. We fund folks in those categories, but also master students.
Brendan Jarvis:
Now what people who are listening to the audio vision of this won't see clearly cuz they'll be listening, not looking Is the statues of Buddha that you have in your background there, and this is sort of making sense for to me now because I asked you about these statues before we hit record and you'd mentioned that after high school you took some time off and out from education as such, and you worked on your meditation practice from what it sounded like was that the spirit, the width of the spiritual lens that the Ruth Bit Foundation is interested in, is that encompassing of multiple faiths or multiple belief systems?
Jason Buhle:
Yeah, absolutely. In fact that strongly encouraged by the fund. When we select folks for the class every year we do try to bring in a range of perspectives and it's really a requirement that you're open to other perspectives. So sometimes these people who look fantastic in many ways, but they believe there is one truth and they're not typically going to be a good fit for the Ruth for
Brendan Jarvis:
[laugh].
Jason Buhle:
So we try to bring and people who have that more ecumenical perspective, including we actually have folks who are agnostic or even atheist, but still see themselves and can articulate why they see themselves as motivated by spiritual values and their educational pursuits.
Brendan Jarvis:
And was your time developing your Zen practice, was that the standout or one of the standout features that the Ruth Bit Foundation L your fund looked at when you were applying?
Jason Buhle:
Oh, I think so. I wasn't part of the conversations, but yeah, that I believe was how I explained my motivation to pursue science at the time they funded me while I was working on my PhD at Columbia. Mm-hmm.
Brendan Jarvis:
Tell me a little bit about Albert and Tony. Ruth bit.
Jason Buhle:
Yeah, so those are the founders of the Ruth Burke fund. Well, I never got to meet either of them. They both passed away before I became involved with the fund, but as I understand it, they really wanted to support spiritual growth. They felt that's what was really necessary in the world having seen the Holocaust and the devastation of World War ii. And so they started doing so, oh, I think about 70 years ago now. Also, they were really interested in teaching and really wanted to support people who wanted to become teachers and they were really interested in social justice. And so that was also a big part of their early vision to support people who had a spiritual motivated focus. It was bringing them to either teaching career or a social justice career
Brendan Jarvis:
Given their experience in escaping Nazi Germany and subsequent career in life and opportunity that they're able to seize upon in America. What do you think they would say if they were here with you and I now about things that we've recently observed in our democracies, things like populist politicians, the quiet right outrage at the way African Americans have been treated in America, the war in Ukraine. What do you suspect they would share with us if they were here with us now?
Jason Buhle:
No, I have to imagine they'd be quite surprised to see where history is trending right now. And I have to imagine they'd be quite disturbed. I do believe, based on what I know from their writings and from the many folks I've talked to who knew them, that they would believe part of the solution actually was spirituality and was for all of us exploring our spirituality and exploring spirituality broadly and really bringing people together. And that's why they made fellowships such a big part of the fund. As I understand it Tony, she lived in New York City and a big part of what she did is she just kept a Rolodex of everybody who was in the fund and she was always connecting the dots. So whenever she, someone knew came into the fund or she was having a conversation, she learned about an interests or something that they were thinking about as a career she would say, oh, I know who you should talk to. And then she would make the connection, make the phone call bring people together for lunch. So I think that informal fellowship, all the different ways of bringing together people together was really important to them. And you have to wonder if maybe our world doesn't need more of that now especially here in this country. I think in many countries right now where there seems to be increasing polarization.
Brendan Jarvis:
Yeah, I think we need to be very aware and cautious when it comes to binary perspectives. And it seems to me at least that not a lot of great things happen when we live in a world of absolutes and black and whites. And it's interesting listening to you describe earlier on the challenge of integrating disciplines and the perspectives around disciplines and then us talking now about some of the broader challenges in the world. They at least seem in part attributable to our failure to see each other and integrate each other's perspectives
Jason Buhle:
Or at least talk. I think speaking respectfully to each other, we don't always have to agree, maybe it's not possible to integrate our perspectives all the time, but at least if we can speak politely, try to learn from one another, try to see the perspective, maybe then we can find common ground or at least avoid the worst othering and the terrible outcomes that come from
Brendan Jarvis:
That. So your PhD, it was no lightweight effort as far as I can tell, and I'm not an academic, so this is just my perspective looking at what it is that you did. And clearly Columbia University is an excellent university, so I can't imagine it was an easy place to get into or to be awarded a PhD from. And I mentioned in your introduction that you were looking at how we control our cognitions and our emotions and we've been in a particularly emotional state, the world as of late and we still are. So I was curious just about this work that you did, and I'm imagining it's looking at the individual or a population of individuals, just how good are we at controlling our cognitions and our emotions?
Jason Buhle:
Yeah, that's an interesting question and it's a hard one to answer, answer cuz what's the baseline? What is good or bad? I mean, I think we're incredible, honestly. I think we're really good. I have two young children. I have a son who's three and a daughter who actually turned five today.
Brendan Jarvis:
Oh, happy birthday.
Jason Buhle:
Oh thank you. Well let me tell you they're not so great at controlling their emotions. We're all world, pretty much all of us adults are world class experts and of course there are many interesting pathologies. You can see certain kinds of brain damage that lead to very sudden, profound loss of ability to control our behavior and our emotions. So I think our abilities are astounding and profound and yet there's more opportunity, clearly for a lot of us being.
Brendan Jarvis:
So what advice or what did you learn from your PhD in subsequent work on this that the next time, for example, there's a stakeholder challenging the findings of a research report and our listeners can feel their blood start to boil. What should they do in that moment? Is there any hope for them or are they too far gone by that
Jason Buhle:
Point? Yeah, I think there's often hope. Well, let's see. I would say two things, and I'm not a clinician, I'm sure a clinician could give better advice, but I would say one that there are many different techniques for regulating one's emotions. And so it's good to have a few different tools in the tool bag and then use the one that's the right fit for the moment and that might just be the one that works or feels like it's going to work for you at the moment. I would also say I think one that is really powerful that people don't always think about is distancing and specifically temporal distancing. This one helps me a lot try to imagine what are you going to think about this in a day, five days a year? You know, may have to pick the appropriate timeframe, but sometimes as soon as you remove yourself from the moment just mentally you can, you'll realize that actually you're not going to care that much and some time in the future and that can help.
Brendan Jarvis:
So you should get that sort of glassy eyed look in that meeting and stare out the window for a moment and just detach. Okay,
Jason Buhle:
Your blood is really boiling, that might be the best choice.
Brendan Jarvis:
Yeah, definitely. Hey you, you've said in the past, and I'm just going to paraphrase you now, that you didn't consider a career pathway outside of academia until you finished your PhD. And you also said in what I was listening to, you said that you didn't feel like you set yourself up for success as well as you could have. Now I thought this was really interesting because looking at you on paper, looking at your LinkedIn profile, this is coming from someone who's now a managing director of Answer lab and as I mentioned, that's the largest UX research consultancy in the world. You're leading a team of a dozen strategists and you worked on some of the most impressive businesses that are available to work on. It seems to have worked out for you. What is it that you felt that you were missing or could have done better?
Jason Buhle:
Yeah, well and I think going back to our earlier conversation, I feel blessed to have the education that I've had because I think that's what allowed me to pivot and adapt so well multiple times in my life, in my career, in fact, including as I move from academic research through product development into UX research that said, yeah, what do I think I missed? First of all, I had never heard of UX research and I was really not aware of alternatives. I think things have changed a lot and I think this is a great thing, but when I was working with my PhD, I heard very little conversation about alternative routes. They were generally considered to be a route of failure, it was a failure if you did, didn't go on in academia and there just wasn't a lot of awareness or visibility as an earlier era technologically.
So LinkedIn existed, but I don't know, I don't recall anyone really being on it or using it to learn about alternatives. Now I remember meeting a few alums actually from the psychology department at Columbia at an event and several of them were in market research and I was like, oh, how interesting. But I had never really never heard about it. I never heard about it in any other capacity. And certainly as I said, UX research was just not on the radar at all. And I don't think it was for anyone else. And just that I was naive. Now I think it's really quite different. I believe almost every student in a psychology department and probably also in anthropology and a variety of other disciplines is aware of UX research as an option. And I think students, they are much savvier about a alternatives. I mean, for example, I had never heard of anyone doing an internship while they're working on their PhD.
And I know a lot of people now who have done that or are doing that. So yeah, I think it's a better world for those PhD students to have those opportunities. And then I also have to admit that I think I also just, this is probably my intensity I was just really laser focused on the academic work I was doing and probably should have paid a little more attention earlier to some alternatives. But I just was really passionate about it. I really didn't didn't imagine I was going to want to go in another direction until really one day I was like, oh wait a second, maybe this isn't a great fit for all of my goals and what I wanna do overall with my life.
Brendan Jarvis:
Well I want to go into that with you. And you mentioned that there was potentially a view in academia that if you didn't continue with academia as a career, that it was viewed as failure. Did you fail?
Jason Buhle:
I don't see it as a failure. I think an academic life is a great life and there's a lot that's really wonderful about that life. My wife is an academic here at ucla and there's ups and downs of course, but mostly I think she loves what she does and I think overall it fits really well into her life. I think there are always parts of me that miss academia and think about some of the things that I would've loved to have done, but in the end I think there were some things about it that just weren't a fit for me personally. And you know, have to look at the whole package. Certainly being able to make more money outside of academia has been really meaningful and important for me and my family. And there are other great things there are I, there's so many great things about working in industry, working on products that impact a huge number of people soon. I would say also the speed actually is really satisfying for me. Academic work is great, but it can just take a long time to do that really, really careful work. So yeah, I certainly some days have some regrets overall see a lot of positive for me in a transition and totally support people who stay in academia. I think there's many a Twitter battle where it becomes quite black or white. And I think actually there are pluses and minuses to both
Brendan Jarvis:
Paths. I'm not sure if this saying travels well outside of New Zealand, but there are horses for courses.
Jason Buhle:
I dunno exactly what does that means? Horse for course. It's different courses, different horses. Is that,
Brendan Jarvis:
That's all right. Yeah. So you gotta know. Gotcha. I, you gotta know what kind of horse you had or what kind of course is best for you. One of the things that you mentioned just a moment ago actually was money. And money is a area of our culture, which has some taboo related to it. Usually it is applied to the personal context. I would never sit here and ask you for example, what your salary is cuz that would be quite rude. But I feel like as a profession it's important for us and UX to talk about money and its role in enabling other things in our lives. And you said that it's been quite a beneficial thing for your family, that decision for you to move into the private sector. You have said, and I'll just you again here, I was really naive about money in many ways. I was unaware of how expensive life is. I underestimated my needs tremendously. It sounded like there might have been a moment of truth for you there somewhere. Yeah,
Jason Buhle:
Yeah.
Brendan Jarvis:
What
Jason Buhle:
Was that for sure. Oh, I guess, I don't know if it was really one specific moment, but my wife and I were both students at Columbia in New York City. It's very expensive in New York City. And although I think we had I remember when I was trying to pick which program I wanted to go to, I looked at the stipends and you get paid. Typically in this country when you work on a PhD you get paid a stipend every year. And I remember I was looking at the stipends and Columbia offered the highest stipend, but when you put into a cost of living calculator, it was like the lowest or one of the lowest. So there were some difficult years there. And I think I remember, maybe this is a moment, I remember when we decided we wanted to buy a couch, when we moved in together, we wanted to buy a couch.
And I was like, oh man, couches are really expensive. This is the kind of stuff I think I just never really thought about the stuff that's around the house, but it really adds up. I remember we bought the least expensive couch that we found to be tolerably comfortable, which was from ikea. Oh, we didn't realize something about the lighting. I thought Ikea, we didn't realize that it was eggplant color. We got it. Oh we thought it was, I think we thought it was black or some darker color and we got it home and it was just like this kinda weird eggplant colored couch that we had for [laugh] for many years living in New York. But must
Brendan Jarvis:
Have been satisfying to sell that one.
Jason Buhle:
Yeah, I don't think that one we could sell in the end. I think that one, that one was a dumpster. But yeah, I think that that was a moment. I think actually when we got that couch shopping for that couch and I was like, oh man, things really are expensive. And of course some of it I think it's just so interesting. I've seen a lot of people my generation, I don't know how this plays out across the world, but here in the US I think there are many folks like me who felt like, hey of course you need some money but maybe I don't need to have the fanciest house or the two houses or this or that. I really didn't want a lot of those things. I still don't. But what's so tricky is how expensive the price of housing has just gone up so many fold over the last few decades.
So I think what made sense when I was, I don't know, a teenager and thinking about my life turned out to not make any sense 15 years later price of education is another example. Price of childcare, price of healthcare, they've all just gone up far past inflation. And so I think it's many folks, myself included, probably made choices that made sense based on what those were a few decades ago. And then as things changed, we had to adapt to make sure we could have the families that we wanted. I always wanted to have a few kids. I have two got another one coming in August and I couldn't have done that. I think if I hadn't made the switch to industry and critically when I did needed to make the switch to make the money to get things in a place where I could have those kids.
Brendan Jarvis:
That's a very understandable and very pragmatic approach and I have taken a similar approach in my life. What is it that you want people in UX and potentially product that are listening to this to know about money as it relates to their careers?
Jason Buhle:
Well maybe first I'll speak to folks who are in academia and I think it's pretty clear now, but I definitely would encourage folks, especially younger folks. I think actually it's almost the folks before they go and before they start an academic path, it's actually like the college students who maybe need to think about this the most or people who are research assistants in a lab preparing for a PhD application process. I would say just really take a look at what those salaries look like and a lot of 'em are publicly available in the US we typically publish compensation for public universities. So you can at least get a sense and of course there's results of other sources of information and compare those to what you can make in industry. And if you can do well with that academic salary and some of the other limitations in academic life, that's great.
But I think there are a lot of people who it's really not a fit when they actually sit down and look at the numbers. And I never did that when I was 18, when I was 22, whatever. I never really sat down and actually looked closely at the numbers. So I would definitely say that for folks who are earlier in their career, I think earlier you were talking about that we do consider it impolite to ask people what their compensation is and there are a number of reasons it can be problematic to discuss that. I really love the efforts that enabled by technology. We're seeing more and more of where people are anonymously sharing salary and even better full compensation information. And in our field really appreciate the very successful effort by Amy Sani last year to encourage lots of people to enter their information, people across the UX space in a Google sheet.
It's a great resource. Encourage people to jump on and take a look. And I know she is close to releasing another one for this year that has been improved in a number of ways. So should be a really fantastic source of information. And it's great cause you don't just get the averages, you can actually see specific compensation packages, details, people give advice about how they negotiated those, which is really interesting. Some characteristics about themselves. You can get a sense of what to expect if you have a certain level of experience live in a certain place. I think that's fantastic and I think the more we can have of that, the better.
Brendan Jarvis:
Amy is fantastic and I will link to that particular resource and the new ones out by the time I publish this in the show notes. Really useful and really important for people to know what they're worth. And one of the ways that you can do that is by getting, I'll try to get an objective read on what the market is paying really, really critical. Yeah. I was also listening to you Jason, reflect on your experience of entering business, going from the academic world into the business world. And I'll quote you again now, you said I had heard when I was in academia that academia was this old person's network all about who and that business is so much more meritocratic. That is absolutely wrong. That is the opposite of truth in my experience. So what was your experience? What was the experience that made you realize that the opposite of what you had thought was true?
Jason Buhle:
Yeah, well first getting the job that my first job at Answer Lab was through someone I knew, someone from my academic days, we had met each other at a summer, institutes a thing, academics do. We come together typically the summer and have an intense period of learning. So I met her, she had left academia a few years before and eventually found her way to answer Lab at the time she was running the New York office. And I connected to her and she was very helpful and explained to me what the opportunities were for me at Answer Lab and facilitating becoming to answer lab. So I had that on my personal experience. But what's really made that clear to me is seeing my students get jobs over the years. And for sure now many dozens enter this field, other fields as well. Oh my gosh. The ones who invest the most in networking are the most successful.
The ones who blindly apply without knowing folks. Eventually they get jobs too, but they have to put in a lot more applications for sure. So I think knowing people, getting the referral of course can be very helpful. But also it's really the insider knowledge and the coaching and the support that people get that makes a huge difference. And I think that's something that is surprising to a lot of folks from academia cuz we talk about academia is having a problem in that way. But what's fantastic about academia is that most of your work is public. So the whole goal is to publish things, put them out into the public sphere and you know, wanna see how good this person is, you can go read their papers and there is also a big focus on building your CV and recording all of your accomplishments and getting grants and so on. So you really have a very clear track record that someone can use to evaluate you. And that doesn't really exist in the private sector. Very rarely can things be publicly shared. Certainly if they're shared, they're very rarely published and accessible to anyone.
Brendan Jarvis:
What role has LinkedIn played in your own career success and that of what you've observed in your students in more recent years?
Jason Buhle:
I don't know that LinkedIn has played a huge role in my own career success. I think I learn a lot through LinkedIn. I think it's really evolved from being a helpful Rolodex essentially to being more of a true social media platform. And I do see a lot of really great conversations. I think it's actually really helpful that you can put more content into your posts. It's so challenging on Twitter channel the small tweets and linking them and so on. So I see just a lot of thoughtful posts on LinkedIn and I think also because it's so closely connected to your name and deployment, I think people behave better on LinkedIn and have better conversations. So I think it's a fantastic learning resource and I do encourage something that also I find when I'm mentoring folks, they're sometimes surprised to learn how valuable LinkedIn is in that way.
And so I always encourage folks, go on and just follow people. You don't have to know them or connect them, just follow a bunch of people. I think it's a great resource in that way. And so it's benefited me personally in that way. And I guess meeting all kinds of folks and including folks like you. So I guess it has benefited my career at least a little bit as well for my students. I see it's absolutely huge. It's such a great way to look at different career paths. It's such a great way to actually track all the alums. That's a huge challenge for a lot of academic programs and made much easier and it's democratized by having so much of the information available. Pretty much everyone's on LinkedIn so it's really useful for them in that way. And also, I don't know if this is true in every field, but I would say in UX research people are generally pretty open to talking and providing mentorship. And so my students do reach out a lot to folks on LinkedIn and do get a lot of great replies. Maybe not everyone I'd say. I think generally they say it's somewhere between one and five and one and 10 maybe respond. But it's an opportunity to not just speak to a UX researcher, but to find someone who's doing a more niche kind of research that might particularly appeal to you. I think that's a real special benefit of LinkedIn.
Brendan Jarvis:
You know are someone who is an educator still in your LinkedIn profile, somewhat jokingly, you make yourself sound a little bit like Batman by day, your Bruce Swain at AnswerLab and by night you are the Batman at USC when it comes to the dreams that we sell students with education. And I know that you have a particular university styled lens here. There's been some criticism from some corners of the industry that the quality of the education that students are receiving for the money that they're paying may not line up. Now you are also a hiring manager. So I was curious to ask, have you ever hired someone from a bootcamp or that didn't have traditional academic training and research for a UXR position?
Jason Buhle:
Yeah, I should clarify that when I'm hiring, I am hiring for these strategy roles, which are kind of a complicated, require a complicated overlap of skills. So a little different from hire than hiring for a UX researcher. That said, certainly I have hired folks whose educational background or Jermaine educational background was from a bootcamp they had to have other strengths. Other strengths for sure. But that could be the final piece that convinces us to bring someone in for an interview for one of our strategy roles. And more broadly at Answer lab. I know we've had some really successful researchers whose UX research background was from a boot camp. So I think it definitely can work my feeling. And I think many people in the field feel that on average of boot camps out there aren't doing enough and are not a replacement for a more rigorous master's degree or a more advanced degree. So I think the people who are really successful coming with a boot camp background have a lot of other strengths as well.
Brendan Jarvis:
What are those strengths? I mean what comes to mind, the people that you've met in the past few years that have had that background? What strengths do they possess that enable them to shine brighter than perhaps their academic qualification?
Jason Buhle:
Yeah, I would say ultimately it means the skills that make you a great researcher. The ability to figure out what the question is, you have to ask good questions. I think the ability to pay really close attention and dive in and look for more and more detail is a critical researcher skill. And then I would say communication. And really what I mean there I think is the thinking that goes behind putting together an argument and analysis, putting together insights be able to present them in a way that other people can understand whether that's written or really probably typically need some of both. So I think those core skills are really critical. And if, well going back to when I was reflecting on my own experience, I think folks who have a really strong educational background develop those skills in another domain. Sometimes a bootcamp just gives them the focus that they need and to our world and then they can take that and run with that. I don't think you're typically getting as much out of those programs from what I've seen and heard as you do from a more rigorous and presumably just longer program that would describe most master's programs.
Brendan Jarvis:
That's interesting. It almost sounded like you were suggesting that four people with Master's degrees or above that a bootcamp might be a suitable bridging mechanism into industry but not something that people should rely on necessarily entirely on its own.
Jason Buhle:
And again, I know it's worked for folks so I don't wanna discount that possibility but I do think that it's more limited. I think probably sometimes a lot of those bootcamps they are, they're trying to make money and so they're selling a vision which maybe doesn't always come to pass. And I don't know how easy it is actually to do, but I think that's a place where I really try to look the alums and see how much success they've had and you know, can't just look at, of course with the successful ones you have to have some broad view and certainly enabled by LinkedIn. That's one of the things I'm really proud of. Our program at USC, which is way more expensive than a bootcamp I believe it in All In is going to cost you something like $70,000. And so it's really, it's a significant outlay as well as of course the opportunity cost of that loss period of time. But
Brendan Jarvis:
Yes, it's an important thing. Yeah,
Jason Buhle:
You definitely have to factor into the equation. That's it. I think go on LinkedIn, see where our alums are and you can just see tremendous success. You're not going to see a lot of people who didn't get dramatic value add. I think out of the
Brendan Jarvis:
Degree something that has come into focus for the field as a result of recent events and possibly have always been a focus is needing to make research practices more inclusive. And that's both with regards to who's doing the research but also who's participating in the research. What is it that you have been doing and the team at Answer Lab have been doing to address both sides of that challenge?
Jason Buhle:
Yeah, thanks for asking about that. It's been a huge focus for us here at AnswerLab really for last several years. And I think we've made some great progress. I can speak more to the latter cause I've been more personally involved in that. So maybe I'll start there and I'll say one thing we start to ask ourselves is, well actually who are we doing this research with? And we realized that we didn't have any way of looking across studies to see, well even a simple question, how many of our participants identify as black or African American. We really didn't have a way of knowing that. And of course we wanna make sure that we're getting a PR representation that more or less matches population numbers and not way under shooting for key demographics like that. So it actually took a lot of infrastructure building to create a system that would integrate all the different kinds of studies that we're doing with all the different kinds of questions and to also trying to make sure that the key areas that we wanted to focus on were being asked as much as possible. Sometimes it's not possible, client won't permit it, for example, it's not an appropriate fit for that research for some reason. But what we started doing is broadly acquiring that information, integrating it, and then looking at it to make sure that we're not falling way short in any area that we wanna prioritize. And well the truth is it's actually quite hard cause there are some areas that we discovered we're falling short in. And so working hard now to try to bring up those numbers and it's a real challenge. It's a real challenge to do
Brendan Jarvis:
That. What impact, if any, have you observed? And it could just be anecdotally on research as a result of being more intentional about how inclusive it is of those demographics that you've spoken about.
Jason Buhle:
Yeah, I think it's a great question and I think that there are a number of possible benefits and so many things in life, you know, don't know if you don't go out and do the research. No, certainly been lots of times when folks have pointed out something as potentially offensive or uninclusive that one could easily miss. Whether it's the photographs on a website or the language used in a signup form. And I think these are areas where we can develop best practices over time and we don't need to bring in someone cause we have the best practices that tell us how we wanna set these things up, but there's always new areas that we might otherwise miss especially as technology grows and new kinds of domains voice experiences and so on. I think also a lot of research traditionally has been done on folks who are more like the folks who are conducting the research more representative of the folks who are conducting the research than the population as a whole. And specifically that means younger and more urban and more well educated.
Brendan Jarvis:
You see that at university with academic research as well. The student population is often the first.
Jason Buhle:
Yeah, absolutely. Yeah. And that's been well studied challenge and it turns out sometimes it doesn't matter at all if you're studying a lot of topics in vision science, actually it doesn't matter too much. It might for some, but for many it doesn't. So sometimes you know have to figure out what matters for your research in some areas it makes a huge difference. And I think especially,
Brendan Jarvis:
I was just going to say I spoke with Katie Swindler maybe six or seven weeks ago and she'd just written a book on designing for the human stress response and part of her research for writing the book, she discovered that a lot of the research that had been done on the human stress response was done through the US military. And of course there's a certain type of person that is more likely to serve in the US military and be available to be studied under high stress situations. So there's a bit of a call in that particular sphere to look more broadly at stress response and other people. And I know in medicine also, it's often had a bias towards men, for example, over woman. And so some of the conclusions that have been drawn out necessarily as watertight as they may have been if they'd drawn from a broader sample of
Jason Buhle:
People. Yeah, absolutely. And with really tragic results. I mean literally people have died as a result of those blinders. No, we may not have the same stakes with a lot of the research that we're doing but I do think that there is a lot of opportunity as we broaden that lens to actually get better findings, get more insights, sometimes bringing in folks who are older, for example, and I can say this now, the years are ticking and I will say I don't have the same ability to throw myself into two new technologies that I did when I was younger. I definitely see that's one, I know the defining moment there. It was when I first tried Snapchat and I was like, oh my gosh, it was harder for me to pick up than it clearly was for people who are younger. So I think expanding into older demographic and also all those other demographics that I mentioned is a way of stress testing our products to some degree and something I would encourage everyone just from a strict of value to usability perspective to do just so you can push your product teams to make more usable product.
Brendan Jarvis:
Well let's talk about that. We'll start somewhere else and we'll get into your paper the best of times for UX research, the worst of times for usability research. I just wanna talk to you about that though, because in that paper you suggested that recently UX research was ranked 39th best job in America by CNN and PayScale. What needs to happen for it to be ranked number one?
Jason Buhle:
Oh geez. Yeah, I don't know what's number one. So I don't know how Skip the competition is there. Well
Brendan Jarvis:
Hopefully weren't just 39 careers looked at on the list.
Jason Buhle:
Yeah, yeah. I think 39 was an excellent ranking. I think it's a great career. I think it's a great career. You're the kind of person who likes to ask questions, who likes to wrestle with data qualitative and quantitative? Not for everyone, but it's super fun for people who love to do that, who want to use their minds and to have impact. And it also happens to be a really well paying job, which is fantastic. Will it always be? I don't know, but I think right now it really is in a fantastic sweet spot and demand is quite high right now. I think globally, certainly here in the US we're just seeing tremendous demand. So it's a very exciting field to get into. And I think probably, again, I dunno what the competition is, but I think it is an even better field to get into than I think a few years back when they did that ranking
Brendan Jarvis:
And you highlighted, and I think it was in that same paper, it might have been something else that I was listening or reading of yours, that the career ladder that is now available to researchers has changed even in the sort of seven or eight years since you entered into the private sector. And you've suggested that we are now seeing more people in director, even maybe up to C-suite level in terms of chief research officers appearing, but also that the starting point for the industry or the career is more accessible now with associate roles becoming available and more internships and these sorts of things. My observation, again, it's anecdotal is that often though the most senior research leaders still reports into someone more senior from either a design org background or the product org. If that's something that you agree with, I'd be interested in your perspective on why that may be. And if it's not something that you agree with or it hasn't been your observation, I'd be interested to hear what your observation is.
Jason Buhle:
Yeah, definitely agree and I think the reasons are probably, I think there's probably two big reasons. One is those professions and most companies, those orgs are just much larger. And also they're older, they've been around longer. So I think it's going to be a long time before a UX research can report all the way up. And you have folks in a C-suite with a researcher title consistently if we ever get there. But
Brendan Jarvis:
Do we need that?
Jason Buhle:
Which is exciting.
Brendan Jarvis:
Yeah, it is exciting. It's great to see more director roles, senior director, maybe even the odd vp. Do we need SVP and C-suite research leaders? I
Jason Buhle:
Don't think need, I don't know, maybe it's a nice to have for our field just so there are more opportunities and so there's more penetration to the highest levels of an organization. I I, I believe in the value of research and I think the higher we can bring the conversation and the larger we can make the organizations with reason, the more benefits there are for really many companies. I think most companies out there need more research, not less and more need to pay more attention at all levels to research than they currently are. So I do think there's a lot of opportunity there and I think we'll see the ladder continue to grow and the orgs continue to grow. I I paid attention to it in part because it's something I remember someone warned me when I was thinking about fully committing to this field and leaving the startup that I had been working with.
And I said, but there's not a lot of space to grow. How high can you go? You can become a senior researcher. There's just not much beyond that. And clearly in the last seven, eight years, whatever it was, since someone told me that, we now do see that there are a lot more levels that people are reaching. So I think that warning, maybe it's still there, but less relevant for a lot of folks. So much more growth potential. And what's critical too is we're seeing the growth in the other end of the ladder and very sensitive to this. Now, this of my students and other folks that I mentor, so daunting to enter this field when every posting out there says we need three to five years. And there a few years ago, there were very, very few internships, very few university grad positions, very few associate entry early career, early career type positions. And there still aren't enough and still very challenging for folks, but there are so many more than there once were. So it's easier to find the first wrong or two of the latter by a good margin then really even just two years ago.
Brendan Jarvis:
Yeah, I think your paper suggested that the UXR part of the UX field itself is actually growing at a rate of around 25% a year. And firstly, that's wonderful news. Really, really truly great news that we are getting more insight into informing the design of the products that are touching many, many people's lives. But you also suggested in the paper that you'd written that the benefits or the spoils haven't been shared evenly. And that in particular, you said, and I'll quote you again here, it's my observation that many companies are pulling back from conducting rigor, rigorous usability testing, even as they are investing more in UX research overall. So what are the factors that are driving this behavior? We've got more UX research going on, but we've got less happening when it comes to usability research. Is this a symptom of that broader movement in UX circles to democratize design? What's really going on here?
Jason Buhle:
Yeah, I think there are a few factors, and it's worth noting that because the field is growing so much and there are so many more UX researchers I'm sure that there is more usability research being done now than in the past. And yet I think the focus on usability research has waned a bit. I think the value people place in it may have waned a bit. I think the challenge that people associate with it has definitely waned. I think those are reasons for potential concerns, certainly something we should be paying attention to. Sandy, it's funny, you can look, there's longer data sets in the form of Google Trends and the Google and Graham viewer, I'm sure others as well. But I've enjoyed playing around with those a bit and looking at how some of these concepts have shifted in their importance and importance of those measures, at the very least in the public mind.
But you can see is that usability has declined over the years in terms of how much people are searching for it or writing about it. At the same time UX research has really surged as a phrase that's searched for and written about. So there does appear to be some divergence there in those data. And I think that really, you can see it pretty clearly if you talk to folks definitely over and over again, you hear UX researchers say, what I wanna do is more foundational research, more generative research. And I totally agree. We need a lot more of that. I think still the problem is often companies not doing enough of that and making these big bets on a designer or product owner or VP's, Whis. We should be grounding that process more in research for sure. And yet I think what's happening is and I've heard from quite a few research leaders that they've been directed to move the usability research off of their researchers, off of their researchers towards contractors tend to be junior folks.
It's essentially an entry level job, entry level researcher job at a lot of tech companies, sending it out to agencies. And we love doing usability research at an answer lab. We do tons of it. I think we're really good at it. We also, we do tons of foundational research, so it's kind of nice for us when people wanna send us all that work, but not nice for us if they're doing it because they think it's less important for any reason. And then I think as you already alluded to, I think the democratization movement is also part of the equation here. People saying, Hey, we can just give you some guidelines, a few tools, and you can do research as just as good as a professional researcher. Well, then you're probably going to have fewer researchers who want to be doing usability research and fewer leaders who want to be hiring researchers to do usability research. And then I'll throw in, I think another factor that's driving some of this is also the increased availability of remote unmoderated platforms. And I have seen some product teams who rely exclusively on that kind of research, maybe with a contractor reviewing the videos and writing up the reports. So kind of a combination of ways of really taking out of the hands of a highly trained and skilled researcher you would've seen in previous eras, I think in technology.
Brendan Jarvis:
I was thinking about this topic quite a lot in advance of our conversation. I was thinking about your paper in particular and some of the views that you put forward in that. And clearly, I run an evaluative UX research practice, so it's quite close to my heart as well, the importance of usability research. So I'm not coming at this from necessarily from a 30,000 foot view. I'm in the trenches here, and with the greatest respect to Steve Kru, who's been a phenomenal contributor to the field for a very, very long time and has done many great things to make usability research more prominent in industry and more accessible for people who want to be doing it to understand how it's done. But I couldn't help but wonder, has a book like Rocket Surgery made Easy Done More Harm than Good overall as it relates to how usability research has seen and the esteem to which it's held within the organization?
Jason Buhle:
Yeah, I think it's definitely had both positive and negative impacts, not so many things from life. I think similarly, the now widespread belief that you only need to do research with six participants. And in fact, what was great about Steve's book, what was great about the efforts by Jacob Nielsen, others to call out how much value you can get from a really small sample doing this kind of research, is that it got a lot more people doing research. The problem back in the eighties and the nineties is that people weren't doing research on many experiences. And those of us were alive back then, remember what digital experiences were like, can attest to that. They were much less usable. How research is much more commonly done throughout the product lifecycle. Certainly usability research is much more commonly done. So it's great. I think those battles were won by those folks.
And debt gratitude for sure. They popularized research and they showed that it didn't have to be the super heavy, weighty, massive and costly effort that would slow down your product development. And to a degree that it was really unacceptable to a lot of folks. So they won. But now today, it's a different situation. We have much better design standards, and at the same time, we're facing new challenges as technology in particular, as penetrating all different aspects of our lives and evolving rapidly. And also we wanna bring these processes outside of the technology sphere. So now I think we need to challenge ourselves and say, okay, well, the challenge isn't to get people to do research, it's to get more insights. And the truth is, it's actually not that hard to get a few good insights, especially if your product isn't that good. It's hard to, there
Brendan Jarvis:
Are still plenty out there that need work.
Jason Buhle:
It's hard to get, get the insights that are more subtle, to find those more subtle usability problems, to find the ones that aren't a problem for the young, highly educated folks on average, but are potentially for other people, the things that come off the edge cases the things that cause a little bit of friction, maybe don't lead to a complete failure, but cause a little bit of friction. And I think as a feel, what we need to do is push ourselves, let's make ourselves more valuable by finding more true usability problems, by finding those more subtle, those hard to find usability problems. And it's not even just about usability, but by chasing those more subtle and hard to find insights more broadly.
Brendan Jarvis:
You touched on this in your paper, I believe, and it's something that Dr. David Travis, who's a UK-based user researcher and has been in the field since I think around 1989. He's got a PhD in psychology as well. He always advocates recruiting on the curve, at least some users that are less competent than the mean obviously. And you would know this, and I believe you talked about it in your paper, that you are more likely to surface some of those more potentially subtle issues that you might not get if you're always recruiting in a particular demographic that is savvy in that way.
Jason Buhle:
Yeah, strong agree. Yeah, actually, I don't see a lot of teams doing that intentionally. I think there's more interest in inclusivity. I don't think there's, I'm not seeing a ton of awareness of how bringing in folks who are further along in the curve can increase the success of our evaluative research in that way. I do wanna call out though Nadine Levin when she was at Meta and Facebook did some really great research in this area, I think promoted it throughout the org with a lot of success and did publish some stuff so you can find some of her work and spoke about it as well. Can you find some of that online? So encourage anyone out there to take a look at some of her work in this area as well.
Brendan Jarvis:
There seems to be the school of thought and the sort of trends you've observed around enterprise, moving their usability research further down the experience scale and out to agencies that anyone that can string a few words together can moderate a usability test. And you've got even gone so far, and I'll quote you again now, you've said, I see that professional UX researchers often fail to rigorously adhere to best practices in moderation. So clearly even people who have experience and not above beyond reproach here in terms of the techniques that they're using. And I was really curious because of your academic grounding and of course your expertise, and as a practicing researcher previously, and now as a person who manages UX research strategy, what are some of those best practices? What are the things that researchers are doing that they shouldn't be doing when they're moderating those sessions?
Jason Buhle:
Yeah. Well, it should be sad. Of course, everybody makes mistakes. Probably even the most experienced researchers make some mistakes in every interview. But I have been surprised over the years, and I've had the opportunity to work with just many excellent, brilliant researchers at many top companies. And yet, you know, do see, I think a, I've been surprised at the lack of adherence to some of theses best practices. Just simple stuff. Not asking leading questions, not asking questions about texts that is on the screen. [laugh], that certainly changes the nature of the task for the participant in a pretty significant way. So I think there's opportunities even with pretty basic things to ensure that there's greater adherence. And I think what's interesting is I think very rarely in a professional setting is one's performance evaluated and scored and is one received feedback?
Brendan Jarvis:
Huge. Yeah,
Jason Buhle:
I've heard of some efforts to do that. We have some programs that support that at Answer Lab, but doing at scale I think is so
Brendan Jarvis:
Time intensive. I mean, for someone to review a session that someone else is moderating, they either have to be there and watch it in real time, of which case there's a huge opportunity cost, or they have to watch it at one and a half times or something afterwards in the recording to get a bit more time efficiency. But it doesn't scale particularly well.
Jason Buhle:
Yeah, I don't think it'd be possible without some technology solution to do it for all sessions. But I think what's striking is it seems like it's not really done much at all. I haven't heard many people talk about having good feedback systems in their organizations. And it's not just within companies, it's also within the training programs. I think there are very few training programs that provide that kind of detailed feedback to help someone become a really strong, moderate
Brendan Jarvis:
Do. Just listening to you talk about that, I couldn't help but think about if people are making mistakes, basic mistakes, like leading questions in a usability session context. And we are not setting up those feedback mechanisms to catch those and coach people on how to develop better skills over time and muddy the findings less over time. Like you said, perfect isn't the goal here. You are always going to make mistakes, but the objective should be to get better with the passage of time. And there's an increasing focus on foundational or generative research going on. Just how much of this poor technique, and now this is an unanswerable question, but it seems to me at least there's a risk there that the bad hygiene that is evidenced in watching someone moderate a usability session may be leaking into some of the insights that are being driven outta that foundational and generational gener generative research, which has a higher cost of failure in insights not being accurate at that level.
Jason Buhle:
Yeah, for sure. I think the challenge when you move to the foundational generative side is it's much harder to figure out what your outcome metric is. It's easier in the usability space to just quantify those insights and to compare them. But nonetheless, I think the same principles must apply. And as you're pointing out, the cost can be a lot greater in the foundational space. Yeah, and I think the other thing that comes to mind for me that I think is really critical in this conversation is also to recognize the limits of what we know. We do have best practices. There's been some great research showing that skilled researchers do things in a lot of different ways. There are different opinions. I mean, we've all heard lots of different opinions on some key matters. And as a field, we need to figure out how can we support research to try to improve those best practices, to evaluate them, to turn our field into a field that is truly, maybe it's a cart before the horse, honestly, think about how we really rigorously improve practice in education and in industry when we don't even necessarily know truly what best practices are.
I think we need to do it all at the same time, if that's the best
Brendan Jarvis:
Solution. Yeah, it's a delicate juggling act. Yeah. Hey, look, just before I bring the show down to a close, I'm just mindful of time and how much of this focus on foundational research, and like I said, the sort of deprioritization of usability research actually. How much of this blame may fall at the feet of academics who have become practicing UX researchers and potentially look at usability researchers, something that's beneath them?
Jason Buhle:
Yeah. Yeah. I think that is definitely a phenomenon and it's a concern. Yeah, it's surprising to me. It's surprised to me how many people have come from academia and not brought the rigor of academia to their own education. One of the things that I actually really loved about being an academic I lot when I was in grad school is how much time you have to spend is learning. You're constantly moving into a new area and you're constantly you submit a paper and then someone challenges you. So you have to go learn a ton about some new areas so you can figure out how it fits with everything that you've been doing. So you become very good at finding quality information and putting it together quickly, and knowing how much you need to know how deeply you have to go. And I don't see that, don't see people bringing that to the field.
I hope it's coming. I think that, as I mentioned when I was working on my PhD, you really didn't hear anyone talking about UX research. Now it's common, clearly the flow from academic research to us, research has increased dramatically. So my hope is more folks will start to bring that desire to delve into the literature that we do have. There is a literature out there. It may be somewhat nascent, and there may be still a lot of opportunity to improve it, but love to see more engagement with that literature and less blind acceptance of things like you should, five or six participants is sufficient for usability research and that that's not what the literature says. So I hope that will bring that focus. And then in turn, people will start to realize how much opportunity there is to do usability research better and deliver more insights. And that to do it really well is not just challenging and a skill that want you be proud of, but also that it's a story we're still writing and that we can continue to improve this field rather than just leave it behind and focus on foundational research, which everyone loves is super fun. It's super important a yes. And let's do both.
Brendan Jarvis:
Well, I think that's a really great place to leave things today. Jason, you are clearly someone who is highly skilled. You set the bar high, and you've given a great deal of thought as to how this field can improve itself. Thank you for such a thought-provoking conversation today. I really do appreciate you sharing your stories and insights with
Jason Buhle:
Me. Oh, thank you so much for having me, Brendan. Really enjoyed
Brendan Jarvis:
It. You're most welcome. It's been my pleasure. And Jason, if people want to find out more about you, what's the best way for them to do that?
Jason Buhle:
Oh I would say follow me on LinkedIn, follow me on Twitter. That's where I tend to be most active in that order.
Brendan Jarvis:
Great. Thanks Jason. And to everyone else who's tuned in, it's been great having you here as well. Everything that we've covered will be in the show notes, including where you can find Jason and all of the great stuff that we've spoken about. If you've enjoyed the show and you want to hear more great conversations like this with world class leaders in UX, design and product management, don't forget to leave a review and subscribe to the podcast. And also, if you feel that someone that you know would get value from these kinds of deep dive conversations into our field, share the podcast with them. If you wanna reach out to me, you can find my LinkedIn profile link at the bottom of the show notes, or you can just search for me, Brendan Jarvis on LinkedIn. Or you can head on over to the website for the space in between, which is thespaceinbetween.co.nz, that's thespaceinbetween.co.nz. And until next time, keep being brave.

Jason Buhle

Travelling from Academia to Industry and Thriving

Subscribe

Who is Jason Buhle, PhD?

Transcript