Lawton Pybus, PhD
The State and Trajectory of the UX Job Market
In this episode of Brave UX, Lawton Pybus shares research-driven insights into the state and trajectory of the UX job market 💡, calls for more critical and independent thought in UX research 🤔, and explores a practical framework for ethical AI use in UX research 🤖.
Highlights include:
- What is the PAIRED framework for AI in UX research?
- Balancing candid views with business interests
- Surprising findings from new UX job market research
- Where should the broader UX community focus its efforts?
- Pursuing independent thought as a UX researcher
Who is Lawton Pybus, PhD?
Lawton is the Co-Founder and Principal of Drill Bit Labs, a consultancy dedicated to advancing the practice of user experience research and its impact on the organizations it serves. At Drill Bit Labs, Lawton and his team believe in the power of data-driven design to unlock the true potential of products and services 🔍.
Previously, Lawton was a UX Research Manager at UserTesting, where he led a team delivering comprehensive research services for strategic customers. He has also worked as a Senior UX Researcher at Charles Schwab and as a Lead UX Researcher at MeasuringU 💼.
With a PhD in Human Factors and Applied Cognition, his extensive background in psychology 🧠 has been instrumental in helping organizations make informed decisions and enhance their users’ experiences.
Lawton is becoming increasingly well-known and well-regarded in the UX community, having written extensively about UX research as the author of the popular Substack “The ¼″ Hole,” which has recently evolved into “Depth by Drill Bit Labs” ✍️.
Transcript
- Lawton Pybus:
- We presented some research where we had analysed 1400 UX researcher job description, and these are collected over about nine months. So we've developed a process where we can essentially comb the entire UX researcher job market inventory every month from an API, and we're hoping that now that we've kind of done this proof of concept, we can do the same with design roles, even product manager roles, so that we can get a more holistic sense of what these proportions look like in real time.
- Brendan Jarvis:
- Hello and welcome to another episode of Brave UX. I'm Brendan Jarvis, managing founder of The Space InBetween, the behavior-based UX research partners for enterprise leaders who need an independent perspective to align hearts and minds, and also the home of New Zealand's first and only world-class human-centered research and innovation lab. You can find out more about me and what we do at thespaceinbetween.co.nz. Here on Brave UX though, it's my job to help you to keep on top of the latest thinking and important issues affecting the field of design. I do that by unpacking the stories, learnings, and expert advice of a diverse range of world-class leaders.
- My guest today is Lawton Pybus. Lawton is the co-founder and principal of Drill Bit Labs, a consultancy dedicated to advancing the practise of user experience research and its impact on the it serves at Drill Bit Labs. Lawton and his team believe in the power of data-driven design to unlock the true potential of products and services.
- Previously, Lawton was a UX research manager at User Testing where he led a team delivering comprehensive research services for strategic customers. He has also worked as a senior UX researcher at Charles Schwab and as a lead UX researcher at Measuring-U.
- With a PhD in human factors and applied cognition, his extensive background in psychology has been instrumental in helping organisations to make informed decisions and to enhance their users' experiences.
- Lawton is becoming increasingly well-known and well-regarded in the UX community, having written extensively about UX research as the author of the popular Substack, the Quarter Inch Hole, which has recently evolved into depth by Drill Bit Labs.
- But it's not all work for Lawton. Word has it that when he's not doing or writing about UX research, he can be found dodging rattlesnakes in the Colorado Hills. But now he's here with me for this conversation on Brave UX. Lawton, a very warm welcome to the show.
- Lawton Pybus:
- Thanks so much, Brendan. It's an honour to be here truly.
- Brendan Jarvis:
- It is wonderful to have you here, Lawton. And I was being a little tongue in cheek in my introduction about the rattlesnakes, but seriously, you do have, from what I gather, quite an active outdoor lifestyle. How, if any, have you actually run into when you were out there?
- Lawton Pybus:
- I've seen a few while running, but I actually have, and maybe this is a story for another time, I have been bitten by a venomous snake. That was one of my few stays in the hospital was,
- Brendan Jarvis:
- Oh, we definitely need to go into this right now. Tell me what's the story? How did that happen?
- Lawton Pybus:
- So about 10, 12 years ago, I was living with my parents and I had a little garden in their backyard and they had a little spigot in the ground where you could turn on a faucet and water the plants. So I always checked it every time I reached down in there because I thought there might be spiders and I have a little bit of a fear of spiders, but unknown to me. There's kind of a feeder pipe leading into that spigot that had just enough room for a snake to hide away in there. And sure enough, there was one hanging in there and I got bit, next thing you know, I'm in the hospital and getting the antivenin and all that stuff. So it was fun story. In hindsight, it was some of the worst pain I've ever experienced. I
- Brendan Jarvis:
- Can imagine. Did you find out what kind of snake it was?
- Lawton Pybus:
- Never got eyes on it. They said that it was probably not a rattlesnake, actually probably was something like a cotton mouth or a copperhead snake. But fortunately they have sort of anti vinn that covers everything and that was what they tried to administer to me.
- Brendan Jarvis:
- Yeah, I didn't actually realise that either. But we just got back from Australia and I learned while I was there that there is a blanket anti venom, right. You don't need to have put eyes on the snake. They'll give you something and it basically covers everything.
- Lawton Pybus:
- Yeah, they give it to you slowly saying sometimes people are allergic to it, which of course I was. So that was the next exciting thing. So I didn't end up really getting any anti veon, but it kind of fades over time and obviously I'm here to tell the tale.
- Brendan Jarvis:
- And so where does your heart or your mind go to when you've seen those rattlesnakes out and about now? Do you start running in a different direction or have you made your peace with them?
- Lawton Pybus:
- Some people have a very deep fear of snakes, and I've never been one of those, and even to this day, I'm not one of the, like I said, the spiders were the things that always kind of got under my skin. But if anything, I might be a little more fascinated by snakes. I feel like I have a certain level of intimacy with them at this point. I mean, not like I want to get bit again, but I guess I'm less afraid of them, if that makes any sense.
- Brendan Jarvis:
- I'm picking up that snakes on a plane is higher up your list of movies to watch than arachnophobia.
- Lawton Pybus:
- There you go.
- Brendan Jarvis:
- Something else I was curious about and that is your last name, and why am I talking about this on the podcast? Heaven knows, but it's one of the things that I was interested in and I took the liberty of doing what I have learned is called Mastics, a bit of amateur mastics, and I had a look at the meaning origin of your surname, and I learned that it has English nor Viking origins. It even has its own coat of arms, which I thought was really impressive. Are you familiar with its meaning and origin?
- Lawton Pybus:
- You are probably more familiar than I am at this point. I mean, what I know of my family history is that I come from the part of the world where all the pale people are. Right. So as you said, kind of English, Scandinavian, Germanic, it's all in the mix there, but no, tell me what you found out.
- Brendan Jarvis:
- So ninth century, so it's very old as far as surnames are concerned, and that's because surnames really only came about when they introduced individual taxation. So before then, I presume that people didn't really use surnames, they just had first names. Anyway, it means the bushy hill. And there's also some line of thought there that it could mean pike, bush, or spiky bush, which could be goss if you're thinking about say Scotland and the GOs bushes there. So anyway, I thought that was an interesting segue when I was researching, I'd never come across a pbu before. What was it that brought your family to the states? Has it been a longstanding United States family or was it recently that they'd immigrated from Western Europe?
- Lawton Pybus:
- That's a really great question. We went down the ancestry.com kind of rabbit hole a few years ago. I gave my mom a licence for Christmas and she tried to track all this stuff down, but my father's lineage stopped very dramatically at his grandfather and there was no getting past that. We couldn't find his father or anyone past that. And as we dug a little deeper, I mean already I've told you about the snake bite. This is going to sound kind of larger than life, but it's true. My father's grandfather, my great-grandfather was a bank robber in the old West. So I mean, there was a period of time where he was wanted by the law and he kind of died in early death. So he may have changed his name, he may have taken that on. We're just not sure. It's a dead end.
- Brendan Jarvis:
- That is fascinating, especially considering a few years ago I played Red Dead redemption too, and I'm getting flashbacks from that game. What a fascinating story. Wow. I certainly wasn't
- Lawton Pybus:
- Expecting it today there's, there's a real newspaper clipping, I could send this to you where I talked about the bank robbery and they blew off the vault and the town came out in the middle of the night and people were shooting back at him. I mean, really it was something out of a movie.
- Brendan Jarvis:
- Wow. I dunno how we're going to segue from bank robbers into human factors, but we'll make it happen. I think I mentioned in your introduction that you've got a PhD in human factors and what's called applied cognition, which I understand is a, sounds like a branch of psychology. What was it that got you into human factors? I mean, why pursue human factors and not bank robbing lorton? That's what I'm really asking here.
- Lawton Pybus:
- I think we pie this men try to earn an honest living and my parents instilled in me to work hard and study hard. So I, like a lot of people kind of chose a passion major in university. So I was interested in people and what makes people tick, how do they think? So I got into psychology now the unusual thing about that is that most folks who take on a psychology degree want to become a therapist of some kind. And that was never really interesting to me. I didn't see myself doing that. So I was just kind of planning to find my way as I studied. And the classes that resonated the most with me that I found most interesting were the required courses that most psych majors would kind of roll their eyes and take begrudgingly. So experimental design, research methods, cognitive psychology, so kind of the neuroscience as well. And even statistics. When I took my first statistics course, it was a revelation to me that we could use numbers in a way to find meaning in the world. So those are the things that were most interesting to me. And you can't do much with a bachelor's degree in psychology. I was looking for ways to apply those specific things and learned about human factors. One thing led to another, and that's how I ended up in a human factors programme.
- Brendan Jarvis:
- And that experience that you had in human factors, you obviously took that to the PhD level. How has that shaped your approach that you've taken and applied research with UX research? How have you brought that through into your UX research practise?
- Lawton Pybus:
- So one thing that's a little different in a human factories PhD programme, or at least the one that I was in, is that unlike many doctoral programmes where folks are generally on a academic path, they are publishing, they are looking towards a postdoc and then kind of a tenure track position. After that, I'd say nine out of 10 of the students in my programme were specifically gearing up for user experience research roles. So that was kind of the precedent that was set. And it may sound a little unusual, but when you actually look at the history of the field, right, user experience did not emerge out of a vacuum. It had these other academic disciplines that were feeding into it. And some of those psychology and HCI, A lot of people are familiar with Human Factors was another one of those where it had its origins in applying psychology to various systems, whether that was combat aeroplanes or the cockpit, the cabin of a vehicle. And so it kind of gradually evolved to apply even just the digital systems and interfaces like that. So I never saw it as really that I led off with human factors and took on a completely different thing. I just saw it as a natural continuation.
- Brendan Jarvis:
- Speaking of natural continuations, you recently launched Drill Bit Labs with your colleague from user testing, former colleague, I should say Dr. Thomas Stokes. And from what I can see, Gilbert Labs as a UX R consultancy. And you've launched it, I think it was around about six months ago or so. What was it that made you feel like it was a good time to do this as opposed to go and find another job in the field?
- Lawton Pybus:
- Thomas and I have, we actually met in that grad programme, the Human Factors programme, and we've both worked a variety of research roles in our career. So whether that's in-house, like you mentioned, I worked at Charles Schwab, had an internship with Cabela's early on or kind of on the consulting side. So measuring you for me was one of those formative experiences and people can be a bit opinionated about which type of role they prefer. I know I've spoken with people who would never want to do any kind of consulting. They really like being in-house and going deep with a product team and learning a space really, really well. I've always gravitated towards the consulting side, and as you mentioned at the top of the hour, our mission, our guiding light is to advance the practise of UX research. And I think we find ourselves in this moment right now where we've gone through years of layoffs in the tech space, and I mean you still see it whenever you scroll LinkedIn or go to a local meetup, you hear people saying, what is our purpose?
- What are we here for? How are we supposed to interact in this new reality? Are we still valued? And I think when you zoom out and think about that kind of human factors background and where we've come from and where we're going to, there's been a need for this for a long, long time. And I'm a believer in UX research. I think that companies need something like this. It may go by a different name, it may take on a slightly different form, but UX research in general as a concept is going to continue on. And I hope that part of what we're doing with Drill Bit Labs is bringing the mojo back to the field.
- Brendan Jarvis:
- I spoke with Dr. Jacob Nielsen a couple of months ago now, and he had just launched or recently launched UX Tigers, which is basically his platform for putting out his views on the field of UX more broadly. And we had an interesting conversation about how he had felt while he was hitting the Nielsen Norman group, he was unable to as freely express his thoughts on the field as he has been able to do since he's left through UX Tigers. How have you thought about, if at all, the line that you may need to walk between expressing your candid opinions about the field and what is required to further it and what is best for your business that you're running through the consulting side of Drill Bit Labs?
- Lawton Pybus:
- I would be curious if we took some time to go through Jacob Nielsen's work, if we could find a specific point in time where there was that shift in his thought leadership. I mean, Jacob Nielsen Giant in the field, the heuristic evaluation is 30 years old this year. And I mean still to this day will Google a topic relevant to UX research and often one of the first things will be something that he wrote on the Nielsen Norman website that's maybe 20 years old, but it's Jacob's words. So I wonder if there was a point in time where Nielsen Norman, the company grew to a certain size and they had a certain mission, they were evolving, right? And he felt that his role as a thought leader needed to change in response to that. I don't know if that's something that you talked about, but to answer your question, I guess I feel like Drill Bit Labs is it's early days, it's months old, not years old.
- The lab is me and my partner Thomas, and we've been, as you said, building some momentum with our newsletter and putting our thought leadership out there. But I think that at this point in time, what attracts potential clients our way is that perspective and saying, okay, I know how Lawton feels about this, how he feels about UX research as a field and how he feels about using these methodologies to enhance our relevance as strategic partners within an organisation that's attractive to me. I want to work with them. And that's kind of how we're thinking about it right now. But as I said, maybe as the organisation evolves over time, that'll also change.
- Brendan Jarvis:
- I want to take you back now to something that you said in 2022, which was when you were interviewed on Design Leader Insights by Fuego UX, and it's something that you said, which was, I was surprised when I came into the field. How much of the job is actually just giving research a voice in an organisation? You're working with people who might have no concept of what UX is, they really just want some insights and to get on with their day. So thinking about what you were saying then, and what I understand is the mission of Drill Bit Labs now and what you've just talked about through putting your thought leadership out there, how closely do those early in the trenches experiences that you had, sort of the harsh realities of how research was being received in the organisation relate to what you are now trying to achieve through jaw built labs?
- Lawton Pybus:
- So when we talk about advancing the practise of UX research, there are a lot of different ways that can look right. So let's say you're at a startup and maybe you don't have a dedicated UX professional where we have employees in the number of teams or something like that, that's going to look different than an enterprise organisation that's had a UX research team for 25 years. And I think the reality looks different for both of those. And likewise, it looks different when we are consulting with those groups. So on the startup end, it may be helping them to prove the value of what user experience can do. It may be helping a very small UX research team, maybe just one person to amplify their impact and build upon what they've already started. But on the other end of that, again, a well established maturity UX research team, maybe they're wanting just to reach the highest levels of maturity and it's coming up with a game plan to do that. So I guess it really depends on the reality that we're entering into.
- Brendan Jarvis:
- Thinking about what you've just done, as in starting your own consultancy, you previously worked for measuring you a consultancy. You've seen the inside of larger organisations as well, being a research manager at user testing, do you see the door that you've just walked through as being a one-way door?
- Lawton Pybus:
- I like that phrase one-way door. And that's something that Thomas and I often use when we're trying to make decisions about the future of Drill Bit Labs. Is this the decision that we can't reverse or is a decision that we could easily back out of? And starting a company, starting a consultancy of your own, as you well know, is no small thing, right? There's a lot of planning, a lot of legal logistics, setting up an entity, setting up a bank account, getting insurance, all that kind of stuff. And you have to have planned for it and maybe given yourself a safe buffer to float yourself during the lean periods and things like that. It's not a small decision, but is it a one-way decision? I don't think so. I think if anything, let's say worst case scenario, and I have no plans of this being the case, but God forbid, drill it, labs fails horrifically. I have experiences that I would've never gotten on the inside. I am now familiar with sales and HR in a way that I never would've been before. I've also had to do things like sell the value of UX research to potentially hesitant stakeholders in a way that I never really had to do before. So I think it is really just amplified and extended my personal value so that if I needed to, I could fail upwards, so to speak, and end up in a better spot.
- Brendan Jarvis:
- I understand. Yeah, I was curious about your thoughts on that because I've always, like you felt that if things didn't work out with the space in between, which is now I think 14 years old, that I wouldn't write off going to work for somebody else, and I'd certainly be able to bring a bunch of experiences forward into that role, whatever that would be. But my heart is really by the stage being so far into it quite firmly rooted in being the independent. And so I was curious to hear how you were thinking about it with the decision you've made, especially because you said earlier that you'd always leaned more to the consulting side of the work rather than the in-house side. Thinking about the in-house side though, one of the things that you spoke about, and it was at the end of an article you'd written called Building and Evaluating UX Research Portfolios, there's a section at the end of that article called Another Thought, which I quite like.
- It feels like you're getting a little bit of extra value there, and it was quite a valuable little piece of writing that you put in there, which it was titled, should I Take That Management Role? And there were three questions in there that you posed to help researchers think more critically about the decision to leave the IC track and get into management. And one of them was, and I'll quote you now, are you motivated to learn a new skillset? And you said, because in your own words, research competencies have little overlap with people management. And I was like, there's never a true word spoken here. How obvious was that to you going into your first management role?
- Lawton Pybus:
- Not obvious at all. I mean, a bit like becoming a parent or even starting a business, there are a lot of things that you didn't know that you didn't know, and you kind of learn as you go. And part of the reason I wrote that was I wish someone had said that to me that I think this is a common experience for people promoted to management. You were probably a really good researcher, I was a pretty good researcher, and maybe you had some relevant skills to management. Maybe you mentored someone on your team or maybe you just had really good people skills, and so you were kind of selected on the basis of I'm good at research and I have these other qualities that might indicate I'm going to be a good manager. But for the most part, managers don't go to management school and get any kind of formal training on this. So yeah, it's not just that you're learning a new skillset, it's that you're not even really told that it's there, that it's something that you're going to have to grow into.
- Brendan Jarvis:
- You also shared, which I thought was a really interesting, intimate insight to how you were potentially feeling back at the time that you first became a manager. You shared be prepared to go from feeling like a rockstar at your job to a total beginner. And that can be, I suppose, for most people, that can be, for some it can be exciting, but it can also be a bit uncomfortable regardless of how you look at it, and a tricky feeling to navigate your way through, particularly if you don't have any experience in doing the thing that you've now been tasked to do. How did you go about building your skills as a manager, building that confidence that is required in order to lead other people?
- Lawton Pybus:
- I guess if I could add a little bit of colour to that, maybe I didn't write. It's that you're not just starting as a total beginner in a new skillset. It's that the development path looks a lot different. So when you're learning your research skills, generally you have a mentor who's helping you along the way, giving you positive feedback, you're doing a great job. That doesn't happen nearly as much as a manager because most of your interactions will be with people who are reporting to you and the feedback you get from them is not necessarily reliable, right? There's a power dynamic there, and so they may be wanting to please you, et cetera. So the kind of feedback that you're used to getting as an individual contributor, you're not going to get as a manager. I am a believer in coaching. You mentioned my outdoors life.
- I'm a big runner. I ran my first marathon a few years ago and I hired a coach to help me do it. I would highly encourage anyone who wants to do that, hire a coach, they will help you to reach your goal. I also hired a coach, a leadership coach. When I took on my first management role, it was incredibly helpful to me to have specific situations and scenarios that I didn't know how to feel about and to talk to someone who isn't in my organisation, who isn't beholden to the politics of the organisation or again, that power dynamic relationship with me to get that impartial and objective perspective. And even if I didn't get clear direction on what I should have done or how I would've done it differently, it was just incredibly valuable to bounce that off of another person. So that's one area where I would tell people.
- The other thing that I would say is that there are more resources out there than people are typically aware of. So for example, most organisations, HR teams will put together materials that are kind of like a quote management school, but they generally don't require you to do it, so you may not know about it. So it's worth asking around. They also will provide professional development budgets. So if there's a course that you have your eye on, even if it's something through Coursera or something like that, those are good resources. And I definitely took advantage of some of those. And then the other thing that was super useful for me, and we may talk more about this later, is original research on managers. So I did a lot of research with Thomas on the interview cycle and what that typically looks like, what pain points candidates and hiring managers have with UX researchers. And that taught me a lot personally. It was almost kind of a therapy for me to understand that some of these things that I was experiencing, I'm not alone and this is how other managers have dealt with it.
- Brendan Jarvis:
- You mentioned pain points, and I suspect we might come back to more of the specifics around managing others later in the conversation, but if I could just pick up on pain points, it's no surprise that there's been a bit of pain in our field lately, and you've certainly picked up on this and some of the research that you've conducted and the findings that you've shared with the community in particular regarding layoffs. In November, 2022, you said, and I'll quote you again now, I found some evidence that suggested UX and UX research in particular may have been disproportionately affected by layoffs. And then a year later, so talking about November of last year, 2023, now you updated that analysis that you'd done the year prior with a different and more up-to-date set of data. What was the TLDR if you like, of what you learned?
- Lawton Pybus:
- Yeah, so maybe to back up a little bit, after a layoff, you've probably seen these where someone will develop a spreadsheet where people put in their contact information, and this is the kind of role that I had in the past and what I'm looking for. So they were mutual support directories that laid off employees would put together to help each other get a new job after a layoff. And these have become less and less popular over time. I think as the layoffs became more consistent and more widespread, it probably seemed a little less useful to even try. But early on, there were a lot of these. And so the first part of that analysis in 2022 was really looking at those titles that were in these, again, voluntary spreadsheets that people put their names in and just to see if there were any patterns. And as you said, the pattern that we observed was there are these benchmark ratios of how many engineers to product managers, to designers, to researchers are in a typical organisation.
- And these have been vetted out over time by measuring you by Nielsen Norman. So they're fairly reliable statistics. And so the question was based on what we're seeing here in these spreadsheets, is that ratio more or less than expected? And I don't remember the numbers offhand. I think it was maybe like we typically would expect one researcher for every five designers, but instead we saw something like two researchers for every five. So I mean that's twice as many as we would've expected. And again, those numbers might be off, but it was something like that. We validated that again in 2023, looking at worn data, which is actually legally mandated. So in the US, companies have to kind of give notice to the government that they're planning to lay off X number of employees in such and such a location. And in California specifically, it gets even more granular where it's by department and organisations.
- So you can do a much better analysis rather than using self-report data of how many specific UX researchers to designers were laid. So we validated that again and saw that that pattern held up, and that's the finding. I was asked again and again, what do you make of this? What's the explanation for this? Why would this be the case? And the truth at that point is that we're just conjecturing. Companies don't release detailed explanations of why they decided to lay off such and such a proportion of people from this department versus this department. And I laid out some possible explanations. I think that the discourse online has gravitated towards kind of we did this to ourselves. We didn't provide enough value. We should have been doing more strategic research. And you hear different variations on this argument. I don't have evidence to support that, again because they don't release that information. But that is an explanation that people have thrown out there.
- Brendan Jarvis:
- I'm going to warn people now that the following hypothesis may be disturbing. You've previously said, unfortunately, you UX may simply be a luxury in leaner times in the absence of data. The simplest answer is usually the best. Now, most UXs, and of course I'm being a little tongue in cheek here, it's a very serious topic, but I mean a bit tongue in cheek in terms of it may disturb people. Most of us would consider what we do to be essential. And it's probably doesn't take a rocket scientist to figure out why we would consider our own livelihood to be essential. But what do you think the source of the disconnect, if you could deal in conjecture a little bit more here, what do you think the source of the disconnect may be between how businesses have been looking at the UX teams or the UX function and how we as UXs look at our own value?
- Lawton Pybus:
- Well, I think the way I put it was you got to keep the lights on first, right? And that's more or less the job of the engineering team, the development team is to make sure that it's functional as far as usability and delight and the kind of things that a UX team can add. Those are great. Those are extremely valuable. But are they table stakes? Necessities? I don't know. I agree with you. I would like to say, of course you should hire us. The ROI speaks for itself, but we're in a larger economic environment and people have dissected this to death. But for a long time, interest rates were low, companies could make longer term investments and wait until later to see the return on those. And as inflation has gone up, as interest rates have gone up, it's been harder to justify some of those things. We also are in a world where investors are calling a lot of shots and they like to see a return on their money pretty much immediately. So is it the wisest decision? Is it rational? I'm not sure that I would say that, but I can see the chain of reasoning and why it might have turned out this way.
- Brendan Jarvis:
- Thinking about the chain of reasoning, and I know you don't necessarily have any insight information here, but one of the graphs that you'd included in your post was to do in the post in case people are interested in finding it is called UX and Flux. What led to the layoffs and what Lies ahead was a graph that showed the headcount growth between 20 10, 20 20 and then 2022 across Microsoft Alphabet, apple, and Meta, as well as their respective layoffs. Those companies had conducted between 2022 and 2023. And one of the things that was really interesting to me looking at that graph was that it didn't look like Apple had conducted any large scale layoffs. It also, I should add that it looked like they grew their headcount at a slower rate proportionately to the other companies mentioned. But what do you suspect makes Apple an outlier, aside from being a phenomenally successful company, what makes them an outlier in this layoff scenario that we've seen play out over the last couple of years?
- Lawton Pybus:
- Yeah, I think you hit on too, I mean, they are phenomenally successful, so they're worth trillions of dollars. And I think that when you're that big, when you're that valuable, some of the rules maybe don't apply to you, but as you mentioned, their headcount grew a little bit more slowly, so that indicates that they were probably more conservative in growth. And being a little more conservative can be annoying when let's say you're working at Apple and you're seeing your colleagues at other teams like, oh yeah, we just added 10 researchers to our team. And you're thinking, well, we could use 10 more. Why haven't we done that? It could be annoying during those kind of boom times, but in the leaner years when you see all these layoffs going on, you might be a little more grateful that, okay, we've grown a little bit more responsibly and we feel really strongly confident that we can justify that this is the team we need. We really don't want to get any leaner than this. So it's almost a philosophical strategic difference between different companies.
- Brendan Jarvis:
- I just want to give people some insight into the proportionality. Hopefully that's the correct way of framing this, that you discovered in your last round of research between the open roles that were available across the various disciplines, you found that there are only about 70% as many open research roles as might be expected for the equivalent open design roles. And I think the more challenging finding here was that there were only 33% as many open design roles as you would expect from the open engineering roles. And I think this ties back into what you were saying earlier about the functional precedence that engineering takes in terms of keeping the lights on and the organisation, whether it's right or wrong, as far as we, well each individual perceives it to be in terms of the decisions that are being made. It certainly seems to be reflected in the way in which roles are being advertised. Now, I know that this data, if I'm remembering correctly, was current as of April, 2023. So that's a little over a year ago now. Have you been conducting, or do you intend to conduct any more recent research into the current state of the market?
- Lawton Pybus:
- So I want to respond to the first thing you said about the proportionality first. And I like that theory the best because it tells a nice story to me. And basically the theory is that there's an order of operations. Now, if you think about it this way, let's say you've got a hundred person startup, you've got an engineering team because you're building whatever the thing is, you start to invest in user experience. Who's your first hire going to be? Probably a designer. That tends to be where most organisations make their first investment. It's not unheard of for them to go with research first, but designers have some basis in research generally. They may not be able to do all the complex conno and max diff and really beefy mixed methods type studies, but they can do a usability test or a user interview. And you can get a lot just out of those two methods.
- So it makes sense to start with a designer and to grow from there. And maybe you have a researcher after you've gotten five to 10 designers. And that's where these kind of ratios start to come into play. And then as research teams mature, we start to see even more specialisation where you might have mixed methods specialists, you might even have a quant UX researcher. You may start to develop dedicated research operations roles. So as the research function develops even, it starts to specialise. So it tells a nice story, at least to me, that if we were going to have layoffs and start rebuilding, it would kind of go in reverse order and then come back sort of in the way that it originally grew, right? Maybe that's just the natural path of growth. And again, I don't have data to back up that story, but it makes sense to me.
- Brendan Jarvis:
- Well, speaking of data, are you going to be conducting any more research into the state of the job market?
- Lawton Pybus:
- We are going to continue that line of research. It will look a little different. So you might've noticed there's a historical graph showing UX researcher headcount specifically. Well, my data source on that basically got compromised, so I can't continue in that exact vein. But what we did do was find a better data source. So Thomas and I just went to UX VA in Fort Lauderdale. We presented some research where we had analysed 1400 UX researcher job descriptions, and these are collected over about nine months. So we've developed a process where we can essentially comb the entire UX researcher job market inventory every month from an API. We're hoping that now that we've done this proof of concept, we can do the same with design roles, even product manager roles, so that we can get a more holistic sense of what these proportions look like in real time.
- Brendan Jarvis:
- Can you share any insight into anything that you learned that surprised you based off the previous base of insight that you developed?
- Lawton Pybus:
- Well, it's a little early to compare things like inventory and how many roles are being offered with the historical data that I had recorded, or we don't even really have a point to compare it against designers and things like that. But what we did do with those 1400 job descriptions was try to make sense of career progression. So we had everything from internships to senior lead principal and staff positions and trying to understand why do seniors slot in where they do and what makes them eligible for the next phase? What is the next phase? Is it lead? Is it principle? So we kind of created a data-driven career ladder, if you will, that's company agnostic. So if your company doesn't have a career ladder, you can dive into this and assess yourself where you should be and how you could maybe prepare for the next role that you want to take on.
- And there were some, as you said, some things that surprised us out of this level of analysis that we maybe wouldn't have picked up if we were doing a more qualitative analysis of job descriptions. So one example I'll give you is years of experience. So we charted out the years of experience required for each of these different levels of seniority, and it follows a really nice linear pattern on one level of analysis. Well, if you actually look at most job descriptions, they will give you a low end and a high end. So we want three to five years of experience as a UX researcher for this senior level role or mid-level role. So we broke it out. What was the low end and what was the high end? Well, again, we saw kind of a nice linear growth on the high end, but on the low end, we saw almost like a U-shaped function where it would go from one year, three year, five year, seven year, and then down to five or three for the most senior levels.
- And so we asked ourselves what could be explaining this? That doesn't make any sense. Well, as you progress in your career, you're not just doing research. In fact, you're often spending a lot more time evangelising research and setting up operations, giving research, more impact throughout the organisation. So it's maybe a little less emphasis on the tactical doing of research and more in these facilitative aspects, and especially in the highest tiers where a lot of jobs, we're looking for a second skillset. So we want eight to 10 years of experience as a researcher and maybe two to four years leading a team or two to four years mentoring junior researchers. So it's that second level that is actually dropping that years of experience finding down. So there were a lot of things like that that again, you would not have found just from combing through a few job descriptions. It was from extracting a bunch of data and calculating means and looking at graphs and being like, wait a minute, where did this come from?
- Brendan Jarvis:
- It almost suggests that there's some sort of second mini career. It's like a minor alongside the major perhaps that if you can start to see what those are and where the demand for those miners are and when they become relevant to organisations, that could help you as a practitioner to think ahead about what type of minor I might want to pick up alongside my UX research that may see me into that management track or help me further my IC career.
- Lawton Pybus:
- And going back to what we said earlier about, congratulations, you're a manager, now you have to learn this new skillset. Giving people that heads up of, okay, this is expected for a principal level role. I don't have experiences doing that. Maybe I should start trying to work that in somehow.
- Brendan Jarvis:
- Hey, just coming back briefly before we move on to something else, coming back to the broader conversation about the layoffs and where we currently find ourselves. Now, I know that's a little murky because the latest data is about a year old at this stage, but you had a bit of a silver lining in there, and sometimes I understand if you're listening to this and you've just lost your role or you've been through more than one round of redundancy, it can seem like maybe this is a bit in danger of bright siding. It's definitely not intended that way, but I did want to pick up on this. I think it is important to share the data in a way that people can start to see the present or hopefully the future more clearly. I'll get to the point now. So what you said was that there were some encouraging signs in there that there was potential recovery for tech jobs, and also that there was some signs that even though we've been through all these layoffs, that the number of open roles that are currently or were currently at the time being advertised for, was greater than the number of open roles that were advertised for in 2020 or prior to 2020.
- So picking up on that, and please go with us wherever you'd like, just how encouraged or otherwise are you about the prospects over say the next 12 months for people in the field?
- Lawton Pybus:
- I'd say the trajectory right now is a positive one. I think that human psychology is a little finicky and strange, as we all know as researchers, and we don't necessarily adjust to new realities as quickly as they happen. So for example, if you zoom back on the graph to that kind of high point where you saw maybe a thousand new UX researcher roles per month, people were riding high and saying that essentially this is a cushy job. It's always going to be this good. If you're not liking where you're working, just quit and find a new job. There's going to be one there for you. So people's attitudes were super different. And even as the layoff started, I distinctly remember there being a lot of denial that this was going to last very long. Like, oh, this is temporary, that there's not going to be any long-term consequences or recession here.
- And now as it's set in, we're having another reaction to that which may not be rational. And as you said, I do want to be sensitive. There are a lot of people out there who have been searching for a long time and people who are despairing honestly, who need to pay their bills. And believe me, I have a tonne of sympathy for that. But at the same time, I think that the general tenor is that, okay, things are bad and they're always going to be bad and it's never going to get good again. And I just don't think that's true. At least the data's not showing that. It's showing that we're starting to bounce back. And as I said, long-term zoom out, UX has grown leaps and bounds over the past 20 years. And I think that that one step that you called out that we're really at a higher floor now than we were pre covid, should stand out to people. Now, that doesn't mean that there isn't still a lot of competition. There's probably more competition for those roles than there was pre covid. But it does mean that the longer term trajectory, not just the short term one, but the longer term trajectory is going to be a more positive one for the field.
- Brendan Jarvis:
- I get the sense from what you've written previously in hearing you talk about this today, that there's this underlying, or maybe not even underlying, there's this evident, empathetic frustration that you seem to feel for the discourse that's going on there at the moment about the field. And you've talked about layoffs, how we feel about those as one example. There's others that you've shared in your writings, and I'm thinking about the conversations that keep going on about methods or the tension that appears to be there between experienced seasoned practitioners and the influx of juniors that have come into the field. There's all these kind of pockets of disgruntledness that seem to be out there in the community, and of course, algorithms love this kind of stuff. It's kind of interesting for people to get involved in, whether it's productive is another matter, but I get the sense that you are somewhat frustrated with where we're currently expending our energy and our time. Where do you feel Forest start actually? Is that accurate? And if it is, where do you feel we would be better served as a wider UX community to invest our time and energy?
- Lawton Pybus:
- I'd say you said my position really, really well, actually, and I don't blame the community for this. I think what you said is spot on. It's the algorithms that kind of zoom in on, ah, this is controversial. This makes people stop dead in their tracks on their feet and they engage with it. So yeah, a lot of what I write, what I try to put out there is to try to counterbalance that. Just give people something in their feed that's a refreshing breath of fresh air. That's what I try to do. And the second part of your question is a good searching one, what should we be focusing on? There are a lot of big problems in our space that are real problems, right? Artificial intelligence is a big one right now. And large language models, chat, GPT, how do we integrate these tools into our work in a responsible way that makes things more efficient, that maybe adds value and finds things that maybe we wouldn't have found otherwise, but reduces false positives, doesn't displace human jobs, doesn't lead us into mistaken conclusions, getting us to the wrong answer faster, if you will.
- But that's a big problem and it's one that people need to be nuanced about and really kind of get into the weeds of that discussion. And I mean, it wouldn't be accurate to say that we're not talking about it as a field. We are, but are we getting into the weeds? I don't think so. We tend to take very simplistic 100% pro or 100% against kind of positions that they're just not going to hold up in the long run. This is a powerful tool. We haven't learned the boundaries of where it's most useful and there are people doing some really thoughtful work here, but by and large, it is not what you read about on social media. So that's a great example of a huge unsolved problem that I think that all of us could be doing our part to solve. Right now,
- Brendan Jarvis:
- What does paired stand for? And you may have touched on this earlier, but what was it at that point in time that you felt the community needed a framework like this to start having a conversation about AI tools in research?
- Lawton Pybus:
- So the paired framework really talks about a set of questions to ask yourself for any given use case of an artificial intelligence tool in your process. And you want to make sure that it's principled, that your team has established principles for this is when we will and when we won't use it, what we'll use it for, et cetera. Accountability is the A, so that's you as the researcher, are ultimately accountable for the outcome of this usage. If you put an AI generated finding in your report, that's on you, if it's inaccurate, that it's initiated by you, the researcher. So that's the I initiated, and that just means that we're not having AI go off and do studies on its own, which wasn't really plausible or even possible when we wrote that in February of 2023. But now we're starting to get into a world of autonomous agents, and there are even demonstrations where someone gives a prompt like on chat, GPT, build me a website that takes payments for this and it'll go off and do it.
- We don't necessarily support that type of research being done for anything serious, maybe for an exploratory use that it's reviewed by a human, which is the R, and then E is for enabled. So making sure that the team has everything that they need to do this properly and safely, and that it's all documented. That's the final D. So all of the kind decision making process that your team has made around these things, let other people get up to speed and check your work. And I don't think that anything there is controversial. I'm glad you brought it up. I feel like it's aged fairly well, even though AI as a field has developed a lot pretty rapidly, and none of those were original ideas, we just kind of packaged them into a framework that people could use. I think the timing aspect was really the interesting part there, right?
- February, 2023, so chat, GPT had kind of been announced to the world. I think it was late November, early December of 2022. And still, I think to that point it was mostly nerds checking it out and getting into it. So there were a lot of people who still hadn't even tried it. We were just trying to get ahead of what ended up happening indeed, which was all these sort of viral posts on social media saying like, ah, here's 10 prompts for you to have a UX researcher in chat, GPT. I think that all of those were generally violations of the paired framework, and we just really wanted to give people, again, a principled way of thinking about these things.
- Brendan Jarvis:
- And it's the principles that, from my way of looking at the framework, the most important part of it, it's almost as if you don't have those defined and thought out in a really meaningful way. You don't have really anything else for the rest of it to stand on. And this is no criticism, it's more a curiosity, but I noticed that you have highlighted the importance of principles in the application of AI to research, but you stopped short of actually defining any specific principles yourselves, and I was curious about that. Was that an intentional decision? Was it that things was so murky at the time? You couldn't quite see what they were, what was behind not actually putting out the provocation more specifically,
- Lawton Pybus:
- Jacob Nelson put together his heuristics. And if you go back into the history of that, there were all of these different findings and recommendations in the early days of usability testing and interface design that people had so many sort of tips and if you want to call them principles of design, that it became almost too much. And so he used some fancy statistical techniques to simplify those and kind of rename them and came up with the 10, and that's what made them sticky. And I think that ultimately we'd love to do something like that in the artificial intelligence space. But it's still early days in a sense. I mean, artificial intelligence isn't new, but kind of this generation is. And so the way that it's interacting with the world, it's still a little bit to be determined. The other reason why we didn't want to develop anything too prescriptive early on was I think that this is going to be different from team to team depending on what your team is working on, and maybe it's particular risk tolerance or maybe how far it's working out At Drill Bit, we work with teams that are almost like the in-house futurists and they're working five to 10 years out.
- So the principles for a team like that might be very different from, let's say the team I used to work with at Charles Schwab, which it's a financial institution. People have their life savings in it, they're going to be much more risk averse. So I don't know that there is necessarily a one size fits all approach just yet. And partially that's again, it'll be different from team to team. And because we're still figuring it out,
- Brendan Jarvis:
- I'm thinking about a conversation I had with Peter Morville a few years back around how we classify people in census data and how at the moment we don't have a very good way of capturing the broad spectrum of how people see themselves. For example, and I dunno why I'm thinking about that when I'm listening to you talk about principles, but I suspect it's because defining something too early where it has multiple different applications depending on the team, the organisation, how far out they're looking, all those factors. It seems to come back to why perhaps you started with questions or I suppose a more open provocation. On the other hand, I do wonder if more broadly our field suffers more recently from a lack of universality in some of the ways in which we think about the work that we do, even if it's from the perspective of if we had something, even if it was wrong, at least we could point to it and then have informed conversations about why we disagree. And so this is more of a thought than a question, but I do wonder if there is room in the near future to do what, say people like Jacob Nelson did do, which is define things that then people could argue about as a way of actually furthering a meaningful conversation about what ethical usage of AI looks like in the field.
- Lawton Pybus:
- I would say 100%, yes, I agree. I think we as a field have become a little bit too comfortable with being the in-house research department. And what I mean by that is I come from an academic background. Traditionally academic research is all publicly published out there so that people can go out and replicate and build upon previous research. And there are good reasons why we don't do that in industry for the companies that we serve, that we're often doing confidential work that maybe gives our company a competitive advantage. But as you said, there are principles, there are overarching guidelines that maybe do come out of some of these studies that we do that would be beneficial and wouldn't harm our organisations if we shared them. And I've been on the record as saying it should be everyone's duty to try to contribute to that public discourse. I've talked about some of the research that we've been doing and publishing and talking about at conferences and even some of the ideas that we have, like, oh yeah, this is something that we want to look into too. But I mean, there's only so many hours in a day and it is just me and Thomas. So I think other people need to pick up the torch, which isn't to say that no one has, but I feel like more people should feel empowered to be a part of that conversation.
- Brendan Jarvis:
- And I wonder how much of that ties back to the connection or lack of connection people feel to what I would say are the governing bodies, if you like, or at least they govern some of our body of UX, like UX PA and other organisations. I know there's been some trouble out there recently with some of the oldest ones falling over, not being able to sustain themselves and sort of symptomatic of that disconnect. But just coming back briefly, I know we need to wrap things up shortly, this conversation, and I wish we had more time to talk about this in more depth, but what you've been talking about there with picking up the torch feels to me like the thrust of the article you wrote, which was called You and your UX research. And it's a wonderful reflective piece that I think that everyone listening or watching this should go and check out.
- And the TLDR for people is that you'd picked up on a, it was inspired by the late Richard Mings, provocative talk called you and your research. And in there there's many great things, and definitely people should go and check it out. But in there, there's an argument to pursue work of significance, to seek important problems. But there's also something that you talked about regarding the importance of an independent thinking and the courage that goes behind that. And you said, and I'll quote you now, you said hemming champions the courage to pursue independent thought, even in the face of conventional taboo or fear of failure. By daring to ask impossible questions, researchers can unlock breakthroughs where others have faltered. Now, where I sat with that is I love it. It's courageous rights, right? In alignment with the purpose of brave UX and these types of conversations.
- So I'm a hundred percent aligned with the key message there. It sounds really noble as well. And I quite like that pursuit, that conviction that has to go behind independent thinking, and I feel that we need more of that. And then I came to a place where I was thinking in my little corner of the world and the type of work I do as a researcher, what does that look like? What does that practically look like in the type of applied research that we're doing? And not the, I suppose it was applied research that hamming was doing, but from the perspective of UX researcher, what does that independent thought look like?
- Lawton Pybus:
- I think it'll look different for everybody, but I can maybe add some colour to what it has looked like for me, and maybe there are some lessons to draw from that. So I enjoy writing, right? And I enjoy conferences. So I mean, these are things that I'm kind of intrinsically motivated to pursue, and I think it's worth, again, a lot of people with academic backgrounds might resonate with that. If that resonates with you, then I think it's worth talking with maybe your manager and saying like, Hey, this is something that I'm thinking about. How could we possibly construct a study where I could carve out a piece of it that some of it might be public facing? And simply starting the conversation and seeing what that process looks like can open some doors. Now, the question about, well, what would I spend my time trying to investigate for me in my work with clients or even just things that I have been, we talked about the management research and how that was partially therapy for me.
- Those were questions that came up in my day-to-day about what are the best practises here? Well, I'm a researcher, so naturally I gravitate towards using research methods to solving that. And I think the same thing when I'm working with clients and a particular issue comes up several times, I'm starting to sense a pattern. Maybe we've been using a stock methodology and trying to do it in a certain way, and I'm sensing some shortcomings. Well, maybe you can construct a study where you compare it against something else that you've been developing or something that you've been trying. And if you can put that out there, if you can anonymize your client data or whatever, that could be valuable to the field at large. So it's partially just solving problems that are interesting to you that you think might be interesting to others. But part of it's just starting to explore what would it look like? And for a lot of people, it may not be writing, it may not be going to conferences, but there are other ways of doing it. It may be a podcast. So what does that look like for me? And what might I look into?
- Brendan Jarvis:
- Most importantly, do something about it, right?
- Lawton Pybus:
- Start to come up with a plan, right? Yeah, don't just daydream about it, but look for opportunities. And I think this was throughout that piece, you and your UX research, look for the opportunity. If you start to think about these things and articulate them, often those opportunities will emerge naturally as you start to go about your work. But if you don't think about it, then it's not going to show up. It doesn't just magically appear on your doorstep.
- Brendan Jarvis:
- I 100% agree. Lawton, for my final question, I'd like to ask you about something that you've previously said. Funny that, and depending on who you're listening to and how often this might sound to the people listening, like a bit of a fringe viewpoint, particularly in the current moment, and what you said was far from being a fragile discipline. UX practitioners should take pride in the robustness of their profession and look forward to even better years ahead. Do you stand by that statement? And if you do, what gives you the confidence to do so?
- Lawton Pybus:
- The layoffs had been starting, but it wasn't the felt reality that we're experiencing right now where it's kind of been dragging on for a long time. And all of us know someone if we haven't been personally affected. The trouble is zooming out from current situation to big picture is challenging for personal reasons, right? It's like we talked about earlier. You've got bills to pay, you need to have a job. And if UX research isn't working out right now, it might make sense to pursue other opportunities. Maybe look for a market research role. Maybe you just do something else to bide your time. But again, thinking bigger than that, thinking bigger than maybe the next year or even five years, what will UX look like in 10 years? Will it be as important as it is now? Will it be less important, will be more important?
- And I'm of the opinion that it'll be more important, perhaps a lot more important. I think that we have had a really beautiful organic growth trajectory over the last 30 years of the field. We've really dug our way into different product teams. And again, it may look different in the years to come, the responsibilities we take on the exact methodologies, but I think that the need is going to remain. You can develop the most flawless system in the world. If people don't know how to use it, then it's for nothing. I mean, you even kind of see that with some of these AI tools, right? Like mid Journey and chat GPT, the interface is very rudimentary, and if you don't know kind of prompt engineering techniques and maybe some of the special parameters that people put into a mid journey prompt to get a really good image, you're not going to get really good results from that. So even with these cutting edge technologies, you're seeing the need for an emphasis on the user experience. And that's where our job security long-term comes from. Short-term economic trends be damned so to speak.
- Brendan Jarvis:
- That's an important place to end our conversation today, Lawton. This has been a really insightful and well measured conversation. Thank you for sharing your insights and stories with me today.
- Lawton Pybus:
- Thank you, Brendan. It's been a lot of fun, and I really appreciate the time and care that you've put into this podcast. You're a really good interviewer and very well prepared, and I know that you're listeners really appreciate that.
- Brendan Jarvis:
- Lawton, you're most welcome, Lawton. If people want to connect with you, they want to find out more about all the wonderful contributions you've been making to the field, what's the best way for them to do that?
- Lawton Pybus:
- I would say go now to drillbit labs.com. Click on depth, and that's our newsletter. We put out content every couple of weeks so you can follow that and get updates right to your inbox. I like to push people there. I know we all have a lot of newsletters that we subscribe to, but those algorithms can be very fickle if you want to make sure you get it, that's the best way. And other than that, follow me on LinkedIn. I publish new stuff pretty regularly. If you look up lot and PBIS on LinkedIn, I'm probably the only one you'll find. If not, there might be some impersonators, but those would be the two main channels.
- Brendan Jarvis:
- And you can verify if they're an impersonator or not. If you ask them about some snakes and spiders questions, see if they're the real deal.
- Lawton Pybus:
- Thanks. Snakes and bank robbers. That's right.
- Brendan Jarvis:
- Yeah, snakes and bank robbers. I love it. I've not had that before and I probably will never have that again. So anyway, thank you Lawton, to everyone who's tuned in, it's been great having you here with us as well. Everything that we have covered will be in the show notes, so don't forget to check those out.
- If you've enjoyed the show and you want to hear more conversations like this with world-class leaders in UX research, product management and design, don't forget to leave a review, subscribe so it turns up every two weeks in your feed, and tell someone else about the show. Maybe there was something in here that you heard today that you thought that one other person would get some value from. Please share it with them.
- If you want to reach out to me, you can find me on LinkedIn. I'm Brendan Jarvis. Funnily enough, search for me there. Or you can head on over to my website, which is thespaceinbetween.co.nz. That's thespaceinbetween.co.nz. And until next time, keep being brave.