Stephanie Marsh

Episode 18
Brave UX

Research Ops for Global Teams

In this episode of Brave UX, Stephanie Marsh, PhD unpacks a range of important UX research and research ops topics, as well as shares her experience of being bullied in the work place (now that’s brave!).

Highlights include:

Why did you move from UX Research and into Research Ops?
What different models are there for running UX Research?
How do you decide what to focus on in Research Ops?
Why is it important to review the data you already have?
How do you know if you're being bullied at work?

April 14, 202101:02:07

Listen on:

Who is Stephanie Marsh, PhD?

Stephanie is the UX Research Operations Lead at Springer Nature Group - the 175 year old publisher of the world-renowned Nature journal, as well as Scientific American.

Before joining Springer, Stephanie was the Head of User Research and Analysis at the United Kingdom’s Government Digital Service.

She is the author of “User Research: A Practical Guide to Designer Better Products & Services”, an incredibly useful reference for researchers.

Transcript

Brendan Jarvis:
Hello and welcome to another episode of Brave UX. I'm Brendan Jarvis, Managing Founder of The Space InBetween and it's my job to help you to put the pieces of the product puzzle together. I do that by unpacking the stories, learnings, and expert advice of world class UX design and product management professionals. My guest today is Dr. Stephanie Marsh. Stephanie is the UX Research Operations Manager at Springer Nature Group, a 175 year old publisher that publishes the world renowned nature journal and Scientific American amongst others. Before joining Springer Nature, Stephanie was the head of user research and analysis at the United Kingdom's Government Digital Service or GDS, the department of the UK Government that's responsible for digital transformation. Stephanie has also held senior positions at the UK Ministry of Defense, where she was the head of digital strategy and at HSBC Bank in London where she was the UX manager. She's the author of User Research, A Practical Guide to Designing Better Products and Services, and incredibly useful reference for researchers that covers all of the key research methods and gives expert insight into the nuances, advantages, and disadvantages of each published by Kogan Page. The second edition is due out in February, 2022. Stephanie holds a PhD in information science from City University, but right now she's holding the virtual line and about to join me for a wide ranging conversation about all things UX research and research ops. Stephanie, welcome to the show.
Stephanie Marsh:
Thank you. Very happy to be here.
Brendan Jarvis:
Yeah, it's great to have you here. And it was fascinating looking at your background as part of preparation for this conversation is any good research you would do you want to come into these things well informed and I discovered that you came to UX via science and I mean by that actual science your PhD was about using HCI techniques and geo visualization. I'm not even gonna pretend to know what that is. So tell us, what was that about and how did it lead to you becoming a UXer?
Stephanie Marsh:
So yeah, my PhD is actually where I taught myself to do user research. So yeah, HCI and human computer interaction, which is sort what collectively UX was in the olden days. So I finished my PhD in 2007, so it is quite old now. So yeah, I was essentially looking at how highly interactive tools that were for visualizing geographic information could be used in education and learning and of understanding what usability use research methods could use to evaluate them. And actually looking at, I mean user research was fairly young still in that academic field when I was there and looking at what technics would be previously used and why weren't getting the results they expected. And it may sound obvious now, but kind of using your target users is obviously essential, but often they was using master students as proxies for professors or something like that and mm-hmm [affirmative], just understanding sort of what the right thing to do was at what time, depending on the context. So I feel like I've come full circle kind of being research operations and supporting people doing research to that. That's a lot of what I talk about and it's like what's the right method? What questions are we answering of What's the context? Yeah,
Brendan Jarvis:
[affirmative] and we'll definitely come to your move into research operations. I really wanna explore that with you. It's fascinated to understand with your post PhD move instead of staying in academia, you moved into I think it was the private sector at least initially into consulting. There's a number of PhD or academic researchers that have contacted me recently asking about how do they best make the move into private or even public sector user research. You seem to do that quite early on. What was the decision that you'd made to leave academia and go into the private sector? What was that about?
Stephanie Marsh:
The honest answer was there were no academic jobs [laugh] that I wanted to do. And at the beginning, yeah, I was doing a lot of job interviews while I was writing up my PhD. Cause obviously you kind of literally anything as you're doing so you can do other things. And I was just like, yeah, it's great. You can have a PhD but you don't have any experience. I was like, yeah, [laugh], I had that a lot. So I went and did an internship essentially and [affirmative], I worked for about nine [email protected] which was a travel booking site and experience booking site and essentially did two days a week for nine months to get that commercial experience. And it was a combination in the end of having the PhD and that experience that someone at the HSBC basically gave me a chance and said okay, yeah, yeah, I think it was that, yeah, combination of that in depth knowledge, but knowing that I'd been able to apply what I'd been doing in a business was kind of what pushed it over the edge.
Brendan Jarvis:
[affirmative]. Yeah. Maybe not what the university have sold you on at the end was the reality of the requirement that a lot of companies have of actual actual real world experience, which is always a bit of a chicken or the egg kind of situation for people I feel.
Stephanie Marsh:
Yeah. And it's partly just finding people who are willing to take a chance I think some time. But yeah, I definitely recommend I mean get people contacting me in LinkedIn for example, saying I'm working in this field, how do I make the move into UX and should I take a masters? And there are some great courses out there, but it's still young enough field that it's not a requirement to have a formal education. And I usually recommend doing some kind of internship or finding a small local charity that probably has a website but they probably don't have a digital team or a UX person and just saying, can I help you? And just getting a hands on experience to see if it's for you and having that on your CV I think often helps.
Brendan Jarvis:
Yeah, I think that's a really important point that it is still quite a young field and it's interesting talking to people in product management as well. Everybody comes from such a wide and very background and that's definitely the case in UX two. Definitely. So if we thought fast forward a decade or so after you finished your PhD, you wrote a book called User Research, A Practical Guide to Designing Better Products and Services. And that came out in 2018. How did that opportunity come about?
Stephanie Marsh:
It was a colleague at the Ministry of Defense was writing a book for Cogen page as well and on an entire different subject and just suggested that I talk to them because yeah, it was like they don't have any books on news research. I'm still the only, it's a business [affirmative], a business books publisher essentially. So yeah, they have a lot of market research books, but I'm the only user research. So I did a proposal and they liked it. They saw a gap in the market. So I was very happy to do that. Very cathartic experience of writing. Well, I loved it. It's not an easy thing to do, but just like being able to all these things you've got in your head and putting them on the page is really great experience.
Brendan Jarvis:
Yeah, I was gonna ask you about that and it's interesting that you say that you loved it, although you did say it was hard as well, cuz most of the authors that I speak to it's kind of a love hate relationship that they have with their work.
Stephanie Marsh:
Yeah, very much.
Brendan Jarvis:
But you obviously loved it enough that you were gonna write a second edition, which I understand is your recent blog post is coming soon.
Stephanie Marsh:
Yeah. Yes, I'm working on that now. It's interesting. Yeah, cause I've thought about it and I was hoping at some point I'd get a chance to do it cuz you never stop learning. So there's things that I've learned since it was published like, oh I wish the book included this. And it's great to get an opportunity to do that and yeah. Yeah, I've always having a new perspective of what could be useful to people
Brendan Jarvis:
And what is coming in that second edition.
Stephanie Marsh:
So obviously research operations has exploded since published it. I mean, yeah, super young filled. So I've, I'm gonna put some stuff about basically setting up for success, assuming that it isn't, everyone's got a super mature research operations practice in our organization. So what other of foundational things you can do that are gonna help you in the future if you keep doing research just to make it easier and ease the burden of the admin or user research. And obviously GDPR is also much more prevalent now. So very much more into depth about data protection, [affirmative], legal and ethical side of things. And originally in the book I did a little bit on analysis, but I've realized over the last few years, just analysis is it's a real skill. I mean the whole research, each phase is a skill, but I've seen time and time again research, the analysis side of it is what people can struggle with the most and it takes time to build up those skills. So I'm going in depth in how to do robust analysis as well.
Brendan Jarvis:
Those are some really important topics, particularly the analysis part. When I was reading through your blog and how you are articulating that, it really hit home for me it is often not necessarily undervalued, but maybe overlooked when compared to the actual data gathering or collection type methods that we tend to focus on.
Stephanie Marsh:
Yes. Yeah. It's like that's the visible part of research and of analysis is almost the invisible bit that happens in the black box of magic. Yeah,
Brendan Jarvis:
I've actually got some analysis to do after we conclude our conversation today. And I have to say I actually prefer the collection over the analysis, but that's just, it's part of the job, isn't it? Yeah. I also was reading in one of your recent blog posts a post that mentioned that at gds, which you recently left I think at the end of 2019 to join Springer Nature, that at the time you left you mentioned that there were service design, graphic design, interaction design, content design and technical writing heads at the GDS all separate from one another. Earlier on you mentioned to me that the field is quite young. Is this sort of, I suppose lack of rationalization in terms of how we structure the UX or UX research parts of the organization symptomatic of that youngness of our field? Or is this just bureaucracy that's gone mad in the case of public institutions?
Stephanie Marsh:
I think it's a conversation that's going on everywhere in terms of how do we best structure ourselves to be effective? And I think not just in government, but I see that in terms of what I've experienced and talking to fellow practitioners because of the youngest of the field, it feels like it's still subservient to more established more established disciplines. [affirmative]. There's also something in the fact that if you don't not familiar with these research you often hear like, oh it is talking to people. We can all put people. Whereas, and I think it's the same for context, nice, oh we can all write not realizing the skills that go in there, but something like being a developer and coding, we know that not everybody can code. So there's a power in that. I think UCD people don't necessarily feel they have, cuz there's something not quite as tangible's not the right word, but it's just not as obvious what those hardcore skills are, even though they are there, they very much exist, but because it looks like oh it's talking to people or it's just writing, there's values. So you don't necessarily get people at the really senior level representing UCD for example. So it's often they're more at the top, they're more kind of tech oriented or product oriented people and we don't get the same level of seniority yet. It's moving. I think there are organizations that are starting to get those people that they have equal footing with the others, but I think slow progress. Does that make sense?
Brendan Jarvis:
Yeah, it does make sense. And we were speaking via email before this conversation and we were talking about the topic of bullying in the workplace and you mentioned to me in there that sometimes I think the way you framed it was that user researchers can be perceived as being a bit of a thorn in the side of other SMEs or other people that we are working with. You've also openly written about aspects of your own experience with being bullied in the workplace, [affirmative], and this is such an important topic to speak about. I wondered if you would mind sharing with me what that experience was like for you.
Stephanie Marsh:
Yeah, I think the fall in the side is an interesting one cause particularly if you're in an agile environment, you're working at pace, then you've got pressure and you've got deadlines. A research saying, hold on, we need to, are we asking, building the right thing? We need to check that this works with the users. And unfortunately it's certain kinds of research aren't easy to do within one sprint and I think can often be seen as a bottleneck or a barrier to delivering something. And I know many researchers that have felt that sort of resentment or pressure that we need to get this done and saying we're trying to get it done the first time cuz we, we've validated things with users for example. Or we're not building things just on assumptions. And that can be a tricky balance I think to negotiate you it all.
Yeah, human interactions isn't it? And trying to negoti the right thing is considering all the pressures. And I guess I think some of the, cuz I've had quite a lot of what is bullying, what is harassment, discrimination training in government. They're pretty good at making people understand what those things look like. And I think in terms of what I experienced, it's often quite subtle and you're like, you even question yourself is that bullying? But if you, it's how it makes you feel. Whatever the intention of the person was, they may not realize how they're making you feel. But I think yeah.
Brendan Jarvis:
And what were those subtle behaviors that you experienced that maybe others out there are again, like you mentioned, questioning themselves, hang on, is this sort of normal and acceptable behavior or is this actually crossing that line into what could be considered bullying?
Stephanie Marsh:
Yeah, so I think most recently a few years ago now it was having my work consistently blocked and being told, oh use research. It's not real data and stuff like that. And it's one thing when that's said to your Facebook, it's another thing when you're trying to work in a team and that's being said in front of your team and that's constantly undermining the value they see I'm giving or any authority that I might have in the team. And then that sort of starts eroding trust and stuff like that. So yeah, it's kind of those things and I've seen it quite a lot in toxic, toxic masculine environments. It's like, well you just don't get my sense of humor. It's like you can't hide behind that as an excuse. This behavior's not appropriate, your demanding me with it, my position in the team. And yeah, I think particularly why I wrote that post is you just see so many people suffer in silence like oh I can't say anything cause I'm not entirely sure what it is that's happening to me or if I say something it will make it worse.
For example. And as you know, when I was the head of visa research and nurses, obviously there's a lot of HR involved in that and supporting people who felt bullied and there is rightly a lot of process in place so that people aren't wrongly accused and it can be properly investigated. So it can take a long time. So there can be a perception of if I say something, nothing will happen, it'll just make my life worse. And I think it's a scary thing to say something's happening and I don't think it's appropriate and I want help to change the situation. It's not an easy thing too. It does take a lot of bravery I would say to do that. But I think I wrote that to encourage people to do that because if people don't speak up then we will just stay in toxic environments and ultimately good people leave and you just left with the ones that happily sw around in a toxic soup I guess.
Brendan Jarvis:
Yeah, yeah, it makes a lot of sense. I mean you obviously took action. What was the impact or that moment where you realized that this behavior wasn't okay and what did you actually do about it to address it?
Stephanie Marsh:
I, I think I realized that it was just an accumulation, this keeps happening. This isn't about ban or humor, this is systematic undermining of me. And I spoke to my manager and they said, if you feel comfortable then talk to the person directly. Cause they may not just realize. So I did that and it didn't do anything. And
Brendan Jarvis:
I think, oh that's brave doing. That's brave actually going to the person that has been systematically undermining you and addressing it with them.
Stephanie Marsh:
And
Brendan Jarvis:
I mean really brave.
Stephanie Marsh:
It's certainly not for everyone and certainly as a manager myself, I would also, when people are having intention with a colleague say feel comfortable, talk to them. If not, I can be with you there and talk to why you talk to them or I can talk to them on your behalf and it's, there's a different path for everyone and what they feel comfortable with. And I think I personally felt comfortable enough in that because I felt it was very much directed towards my job and although I would say the person without really knowing was much harsher on women than men, it wasn't about me as a person. I think if I had been experience in real misogyny or if it was about sexual orientation or anything like that, I think I probably wouldn't have felt so comfortable. But because it was literally I'm a user researcher and that was sort another, it felt like it was a doable thing for me to confront them.
Brendan Jarvis:
So it sounds like you weighed up how safe you felt in that situation and you felt safe enough to address it. How did the person respond when you raise this with him?
Stephanie Marsh:
It was very much, oh, you just don't understand my sense of humor. Even though I like the shy introvert I am, I felt very, felt very carefully about what I was gonna say before I said it and had concrete examples but everything was dismissed. So it went back to my manager, went to their manager and it was in the end wasn't able to get anything done formally. But what I realized and the benefit of being people who study other people, you can see how others work and what will influence them and what won't. So I realized to, we weren't going be friends [affirmative], but we needed to be functioning work colleagues and essentially I just created a strategy of when I knew I needed to get something done, for example, evidence used to do a thing, I would share it with the people that he respected and was influenced by and then got them to influence him to then do the work. So obviously I would much flirt it if we could have worked together to use it, do the thing, but that wasn't gonna happen. So I had to find other ways. It's not really solving the bullying in the end it was sort of how can I get the work done?
Brendan Jarvis:
It's very pragmatic and it's great to hear that you were able to take power back into the way that you were interacting with that person. It's just a shame that it wasn't perhaps better orchestrated by the organization to enable that to be a more sort of usual type of relationship. [affirmative], if someone is listening to this episode and is being bullied or suspects that they might be, what would you recommend to them that they do?
Stephanie Marsh:
If some interaction you're having is making you feel bad in some way, then I would say yes, you're probably being bullied and just go with that instinct and find someone at work, hopefully at work that you feel comfortable talking to, that you feel safe talking to. And depending on, obviously startups is very different to government department, but if you have a structure where you have someone in HR or someone who deals with that kind of thing, they're very much there to listen to you and to support to you. So yeah, I would say find someone to talk to or to friends first and just got sound out a strategy that you are comfortable with. But yeah, I would say just try not to suffer in silence. Find a way to talk to someone and try and get something done about it. Because often you're not the only one, they're probably bullying other people and not just you as well. So there's just many people suffering in the end.
Brendan Jarvis:
I was talking with a leader in product management last week, a gentleman by the name of Marty Kagan, and we were talking about this topic of toxic behavior and toxic individuals and basically the outcome from that conversation was that there should really be a zero tolerance for those type of individuals because they, as the older old sort of analogy goes, one bad apple spoiled the entire barrel. It's so true. And he actually referenced the New Zealand All Blacks as a example of that. They have a no assholes rule in the team and it applies to coaching as well. And that's one of the reasons he believes anyway why they've been so successful on the sporting field. So yes, talk to someone, don't suffer in silence. It's a really important point and thank you Stephanie for sharing that with us today. Before we get into some practical UX research topics, I'm really curious about why you made the move from research into research ops.
Stephanie Marsh:
From my many years of experience, cuz I use a researcher, it just made so much sense what was being talked about and developed and I could see as a head of a community of about 30, 35 people that it was so we didn't have that and could just see the impact it was having to not have that infrastructure in place. And unfortunately at the time, gds, the way budgets were structured or siloed as a, just wasn't able to get budget to invest in it. So it started doing as much as we could in a structured constructive way as a community to do it for ourselves. And that was great for many reasons, but it's very much dependent on what people's workloads are like [affirmative]. So you definitely feel very busy then obviously you're gonna focus on the day job and not the side projects. But that made a big difference and I like helping people and I think I saw there was lots of interesting stuff going on and there was the aspect of helping people and it just felt like the right thing to do. Yeah.
Brendan Jarvis:
You mentioned at GDS in one of the conference presentations I was reviewing before today you sort of broke down the different models that research is often running in within organizations. Can you tell us about what you realized about the model that GDS was running, what you were attempting to move to, and perhaps where Springer Nature is at the moment in terms of the model that you are implementing or have implemented already there?
Stephanie Marsh:
So I would say that GDS at the time was very much distributed. Everything was done in programs and we did community activities together, we learned things together, we discuss things together. But tools and budgets and planning of work very much happened within the program and there wasn't necessarily a lot of crossover [affirmative]. Whereas at spring and nature, obviously you still have people in product teams in programs of work, but we have a centralized team as well. So within the central team there is myself and my colleague as research operations, but we also have researchers that are doing cross we call 'em domains, gds called them programs. But essentially research that's at that high level that's beneficial for everybody. [affirmative] and obviously we are doing, no matter what team you're in, we as research ops are there to support you and the budgets for tools are centralized and things like that. And we've set up guidance and governance infrastructure that is for everybody. And that is much more effective rather than everyone creating their own thing and having lots of redundancy in the,
Brendan Jarvis:
And you've got the added complexity at Springer from what I gather, that you've got teams distributed across London, bill and Puna and India. How has that been in terms of a challenge to manage the rollout of a research ops framework and agenda
Stephanie Marsh:
[laugh]? And we also have teams in Shanghai now, so I think Oh
Brendan Jarvis:
Right,
Stephanie Marsh:
It's say that particularly because yeah, so both research people were both based in London, so obviously time is a factor. So in Puna they're like four hours ahead so it's not too bad, but our colleague in Shanghai, they're eight hours ahead. So as we are starting our day, they're finishing theirs. So
Brendan Jarvis:
[affirmative],
Stephanie Marsh:
There's a scheduling that has to be quite creative sometimes or doing things multiple times for multiple time zones just to make sure everybody's included or finding ways to do things asynchronously. Cause we want everybody to feel equally supported and not to think like, oh cuz they're in the uk, they're gonna support UK people more. We want everybody to be equal. I wouldn't say it's challenge, but it's extremely interesting to learn about different cultures and what's appropriate in different cultures in terms of how we interact with our users who are UX is the researchers, but also when doing international research cuz most about our research is international, sort of how different cultures react to being a participant or the way we ask questions, things like that. That's very interesting thing to consider as well.
Brendan Jarvis:
So your researchers you're supporting and outside of the uk, are they largely researchers from other cultures?
Stephanie Marsh:
Yes. Yeah.
Brendan Jarvis:
So what does that learning loop look like between what they understand about the local context and what you need to understand about that in order to enable the research ops to support them and the way that's gonna be effective for them?
Stephanie Marsh:
So particularly for us, we invite feedback on a very regular basis. We always saying to people, if you don't tell us when we don't know if we're doing something wrong or if we're not supporting you in some way. So it is just being very open to conversation and constructive feedback. But also, I mean as a department, just giving space to talk about those things and acknowledging people's expertise. We've been doing a lot of research in China and learning the different cultural things like how to talk to people and most Chinese people don't like small talks, so there's not necessarily a preamble at the beginning of a session. And just understanding those things so you can slightly adjust your approach has been really useful. And our intention is to document some of these things so that any new researcher coming along can go read up before they do their first session. For example.
Brendan Jarvis:
So you are 18 months or so in now to that role of running research ops at spring and nature. What ways are you measuring the effectiveness of UX research ops?
Stephanie Marsh:
Yes we do. We have some metrics. So we track in terms of what GDPR issues we have identified and resolved [affirmative] and that kind of thing. And particularly for the year first year, we was tracking all the different kinds of questions that we were being asked in when people were asking for support to try and identify is there something we need to provide in that area? Got it. And part of being international is obviously we are there to help, but if you are in a different time zone, we want there to be documentation, guidance, et cetera that you can use if we are not there to answer the question immediately. So thinking about, okay, maybe we need a guide on this because we keep getting asked questions about it [affirmative] and we do a lot of research as well. So the work we do is informed by researching with those users. Even though it's a group of 20, it's pretty stable who those 20 people are, but every project will have an element of doing research to know if we're doing the right thing.
Brendan Jarvis:
Time is always a constraint and almost every line of work of heard you say before that there's never enough time to do everything that needs to be done in research ops. In your experience to date, what are the most important things for someone that is perhaps a new research ops leader to focus on?
Stephanie Marsh:
I would say it's different for every organization and the first thing to do is work out what is key to focus on in your organization. So within three months of the team forming we did a series of workshops with all the UX people to identify what all the pain points were and then we could prioritize what stuff was coming up the most. So I would say if you have to start somewhere, probably start with data protection [laugh] in terms of being legally and ethically important. But I think having those conversations to understand where the biggest gaps are, where the biggest pain points are, [affirmative] is the best way to go.
Brendan Jarvis:
And what is it about that? That is one of the best ways to go.
Stephanie Marsh:
So I, I guess I take that approach and still use a researcher at heart and just [affirmative], particularly if you're new to an organization you can make some assumptions of what might be the right thing to do, but understanding the research process that people are using and how you can fit into that and where you can support, I think it's just useful to really get to know that before you start, particularly part of our jobs is change management and even when we're changing things for the better, it's still a change that people have gotta get used to. So starting to build those relationships so that you can understand how best to implement change.
Brendan Jarvis:
Yeah, that's a really valuable point and it's often underestimated just how important the people call them. The soft factors are in leading any kind of change, but ultimately it's reliant on those factors being well looked after in order to be successful.
Stephanie Marsh:
Yeah, I think there's again, people feeling listened to. It's a research, it's job, it's an amazing job, but it can be quite tough. So being able to just say, I find this really hard or this is working well and oh I can't see this. I think that helps as well to start that engagement process.
Brendan Jarvis:
You mentioned the importance of that new leader to research ops, getting things in terms of data compliance, gdpr from a legal and a risk and an ethical point of view sorted as one of the first things that they do. GDPR has been in effect for three or so years now and it really has forced companies to implement better ethics around how they collect and store and manage people's data. What impact have you observed that GDPR has had on the way that UX research has conducted?
Stephanie Marsh:
I think it forces is not the right word, but yeah, I'm gonna say it forces us to be much more organized about particularly data storage and where things are stored and how long for, and also in, I think it's a really good thing in terms of questioning what is the data that we really need rather than we'll just gather everything and see what comes, but really focusing in on what data do we need to answer the question that we have. And I think that's a really good thing. I know when GDPR first came out, people were quite scared. It's like, oh well we weren't done some research anymore. But it often is perceived as a like, oh GDPR because of this, it is gonna take longer. But I don't think that's true. I think it's complex and if you're not thinking about it every day, I mean we think about every day it can seem a bit scary, but there will always someone to ask. So just don't be afraid to ask a question. And if you're not sure if what we're doing is complying with data protection regulations ask rather than just hoping it's okay,
Brendan Jarvis:
What are some practical considerations that researchers that may not have a research ops function or even research ops people need to consider when planning and running research to be compliant with gdpr?
Stephanie Marsh:
So I think informed consent is one of the key things. And having a consent form that is signed off by data protection signed off by legal, everyone's happy so that it covers you, but also it protects the participants as well. And they're very clear on what data we're gathering and how we're gonna use it and all that kind of thing. I think that's really key and if you can make it accessible, then even better as well. So anybody can understand it, anybody can read it. I think that's a really important thing. And the other sort of things we spend a lot of time on is making sure that the tools that we use are GDPR compliant
Brendan Jarvis:
[affirmative],
Stephanie Marsh:
And particularly as we are, yeah, we're recording people potentially sharing personal information as they do the research. So just making sure that it's really secure and we are storing the data in the right place in the right way is really key.
Brendan Jarvis:
Sounds like a bit of collaboration with a few other experts in the organization are required. Yeah,
Stephanie Marsh:
Yeah. I think particularly for us is also obviously being a big organization, there's lots of teams that have customer contacts and working with them to work out what is the data compliant way to potentially tap into their lists of people. Can we do that? We sort of understanding what people are signed up to. I think, yeah, it's a lot of collaborating with other people to make sure that it works.
Brendan Jarvis:
Luckily. So one of the strong points of research is generally [laugh]. So we are talking about data collection. There would be remiss of us not to have a conversation about quantitative and qualitative data. And there's often a bit of tension between the worlds of those that favor quantitative UX research over qualitative UX research. People seem to be geared one way or another. There are the odd ones out there that sort of get both of those worlds, but generally I find that they're divided into two camps. What has been your experience and if that's been a similar experience, why do you think that is?
Stephanie Marsh:
I think it's partly it's two different skill sets and usually stronger on one than the other. So yeah, usually a qualitative researcher, a quantitative but in terms of I guess your stakeholders, it's often they wanna see the numbers [laugh]
A lot recently in terms of again, we're always asking what method do we need to answer this question or what method? Yes, what data is gonna speak to the stakeholders? And it's often combining the two. So you get the why with the qualitative and then you get the scale with the quantitative about it in meetings today. And just talking about framing it in sort of each method has its own, what it's really good at and what its limitations are. So if you're combining different methods of quantitative and qualitative, you are reducing potential for error. You're reducing potential for biased and potential for, and those limitations if they compliment each other. And I think reducing risk is ultimately what we are here to do. Yeah, I I mean find myself recommending a lot, do both if you can, but also acknowledging that it is a different skill set. So if you're a qualitative researcher, you might need some more support on the quantitative side because statistical significance can be a scary thing if you're not familiar with it. And just appreciating that it may take more effort or more time to do both types of research. Well if you are more versed in one than the other, it's also perception again, it's like yeah we want numbers but it's also the perception that quant is quicker and cheaper.
But I always try to go back to what question and we trying to answer and what method is gonna get us the best answer that we can. I can't always use exactly the ideal combination of methods that we'd want to use. There's always time pressure and budget pressure, et cetera. But what is gonna be good enough? Yeah,
Brendan Jarvis:
[affirmative]. So I really like the way that you came back to what is the research question, what are we really trying to learn? And then letting that guide what the method might be [affirmative] rather than defaulting to what you might be most comfortable with having been one way or another. And that's a really valuable piece of insight there. So what's your talk at UX Istanbul in 2020? And you spoke about combining qualitative and quantitative decision data and decision making we've just talked about here as a way of triangulating or confirming your hypothesis about a research issue. You said it was important in that talk to take some time to assist the data that you already have. Why is it important to take that breath and do that before you run into data collection?
Stephanie Marsh:
You may already have the data that you're looking for. A big organizations, someone might be collecting that data and you don't know about it. So it's partly we might not need to spend this money to collect this data cause we've already got it. But also it's about if we know a certain amount about what we're looking for, we can then really focus in on the specifics. We know where the gaps are. So I think it's rather than just, it's partly redundancy, you don't wanna do the same thing again, again. But it's also can we get more focused on what it is that, what question we're answering, what we do know, what we don't know. And again, it comes back to GDP as well. It's like do we need to collect this data or not? Yeah.
Brendan Jarvis:
And you've written about the storage of research data in terms of what I believe you've called it insight library perhaps other terms people might be familiar with. There may be some distinction here and please make the distinction if there is, they might be familiar with the term research repository. What are the factors that before you, first of all I suppose what is a insight library? And secondly, what are the factors that you need to consider as an organization or as a UX research ops team before you go and try and implement one of these libraries into your organization?
Stephanie Marsh:
That's a good question. So I would say an insight library is a knowledge management tool. So it's very much you are extracting the insights from universal truths. We could probably call them from the research and it's sort of context agnostic. You're not about a specific product, it's probably sort of more about a user's context, the way they behave. It's been shown true from multiple methods. So there's many data points. So you can say, yeah we noticing about our users in this environment, this is how they behave, this is their context [affirmative]. And I think if you're gonna have an insight library, you really need a librarian that is a full-time job to maintain that insight to make sure that you're, because that's quite, that's a huge additional job to do as a researcher when you're trying to deliver things for your team to then go and put the insights into a library when you're trying to plan the next project.
So curating it and keeping it up to date and making sure that those universal truths are still true. Has there been a major shift in the market or in your organization's direction, you'll have to go back and review them. Are these still valid? So it, it's a lot of work and I think often those things fail because people are doing them as a side job rather than having someone to dedicate it [affirmative]. But there are still useful things that you can do before then. I do make a distinction between a research repository and an insight library. So research repository is more like you can go somewhere, you can go and find, it might be all the templates we have or the reports we have. So you haven't extracted the meaning from the report, but you can go and find it and read it for yourself and decide if it's relevant or
Brendan Jarvis:
Not. Got it. And is this so you can avoid repeating research that may have already been run?
Stephanie Marsh:
Yeah, yeah, exactly. Yeah. So you can avoid that redundancy or it's still a lot about making sort of connections. So I've read this report that my colleague did, I then go and talk to them about it and maybe get some inspiration for what I'm doing. Yeah,
Brendan Jarvis:
It makes a lot of sense. This is actually quite a good point I feel to segway into some practical scenario based questions. Are you ready for some of those? Yes. Okay. Here's the first one. Your team recently ran a study only to find out that almost the same study had been run by another team a few months before this frustrated you and you want to avoid it in the future. Where do you start?
Stephanie Marsh:
So good question. I think I would first check, is it really just repeating the same thing or are there different focuses or is it a valid thing to do it twice? But I think it's often about establishing practices that so that you are, as a team sharing, we are planning this, we're planning this. So what other people are planning to do, what's on the roadmap? And having those tools, those places to that people can go and find what's been done before so that you're not repeating. Yeah, I think it's partly tools, partly infrastructure, partly all again sort of networking and just connecting with people. Part of the usefulness of having a centralized team where I sit is that because we're talking to everybody, when people say, oh, I plan this research, I can say, oh yes, I spoke to another team three months ago, they did something very similar. Maybe you should talk to them before you start doing this. So if you've got people who have that sort of overview, that bird's eye view of the bigger picture, that can be very useful as well.
Brendan Jarvis:
Yeah, that sounds really useful. You mentioned tools in there as a way of helping to alleviate that issue. What tools have you ever used in the past or are currently using to make it obvious what research is being planned or is recently being run, planned or run?
Stephanie Marsh:
So we currently use Trello and not everybody loves it, but it works fairly well, particularly for what's being planned at the moment. Not so well for finding old research and reading that. Yeah, I've been thinking a lot about this cause we're gonna be working on it this year and sometimes you need to start way before the tour and think about do we have good naming conventions so that you can search? Cause we also have a lot of our stuff on Google Drive. So can you search Google Drive and find a report? Cause it's got a sensible name that I can understand. And those sorts of things are, are they all stored in the right place? So I know I can go and look in this place and I'll find what I'm looking for. And so part of it is governance and part of it is tools.
Brendan Jarvis:
Got it. That's really practical, really helpful advice. Are you ready for another one?
Stephanie Marsh:
Yeah.
Brendan Jarvis:
Okay, here we go. You're a junior researcher and you've been given the opportunity to present the findings and recommendations of your team's recent study to a group of senior stakeholders. How do you present these effectively?
Stephanie Marsh:
I would say practice fast, [laugh], [affirmative], talk to you. Yeah. If you are working with a more senior person, talk to them about it. Get an understanding of what the priorities of the stakeholders are is what do we lead with essentially If we're there gonna take away one message, what do we want it to be and of front load to your presentation. But
Brendan Jarvis:
Yeah, that's good advice right there. Everybody,
Stephanie Marsh:
[laugh], [laugh],
Brendan Jarvis:
Do your research on your stakeholders.
Stephanie Marsh:
[laugh]. Yeah, I think it's knowing what's important to them so you can talk to them about that first. And I mean particularly for me when I was more junior, being able to practice before going into very senior space, it really helps as well. And then you can sort of hone and refine the way Yeah, you're telling the story.
Brendan Jarvis:
Got it. Yeah. Really good, really good. One more?
Stephanie Marsh:
Yep.
Brendan Jarvis:
[laugh]. Okay. This one is, you've taken on the challenge of establishing research operations at your company. How do you decide what to focus your limited time and resources on?
Stephanie Marsh:
So I would look at what's already been done, [affirmative] and yeah, what is it that absolutely has to be done? We talk about data protection, so make sure things are secure and legal and protected. First I would say, from what I know of research, recruitment, participant recruitment is quite a good place to start. Particularly if you haven't got someone that's doing a recruitment for everybody. Quite difficult labor intensive part of the research process. So there is probably things you can do to support them there and it will make the people you're supporting. Very happy [laugh]
Brendan Jarvis:
Yet to find anybody that loves doing the recruitment part of UX research. But I'm sure there are some. There's someone out there somewhere.
Stephanie Marsh:
Yeah. So yeah, us as a team of two, we provide support for recruitment, but we don't do the recruitment ourselves and they wish we would.
Brendan Jarvis:
So why is research up so important?
Stephanie Marsh:
I think it enables high quality research, impactful research to be done. And I think it's for the researcher, it's taking some of the burden away. So when they are under pressure and they're trying to plan and do a project, they don't have to think about we have templates for the interview guide, for their plan and for their reports so they don't have to think about those other things. They can really focus on doing the right, researching, doing a good thing. And I just think it helps with consistency as well as quality and just, I think it's a key part of a maturing research practice to have that support in place. Yeah,
Brendan Jarvis:
[affirmative]. Yeah. If anyone listening to this is either worked in an organization where research ops is in place they will agree with what you've said. It makes a huge difference. It takes a lot of load off of the researchers and enables them, like you've said, to just focus on doing the research that they already do. Well, even better.
Stephanie Marsh:
Yeah, exactly. Yeah, that's a great way to put it.
Brendan Jarvis:
How do you feel about playing a quick game?
Stephanie Marsh:
Okay, [laugh],
Brendan Jarvis:
Nothing to worry about, I promise. It's a really easy one. It's called What's the first thing that comes to mind? Okay. So what I have is I've got three in this case, three words or phrases. I'm gonna say one of them at a time and then you just let me know what comes to mind.
Stephanie Marsh:
Okay.
Brendan Jarvis:
Are you ready? Yep. Okay. The first one is research ops.
Stephanie Marsh:
Love it. [laugh].
Brendan Jarvis:
[laugh]. I love that. The second one is gdpr.
Stephanie Marsh:
Essential.
Brendan Jarvis:
And the final one is writing
Stephanie Marsh:
Harder than you think. [laugh].
Brendan Jarvis:
Yes, I've triangulated that data with me in author. I would agree to just the conclusion that most people draw [laugh]. So Stephanie, thinking about UX research and that I'm imagining encompasses in this case, research ops, where it's at now, what's your greatest hope for where the profession might go over the coming years?
Stephanie Marsh:
I think it goes back to what we was talking about earlier in terms of getting that representation at the really senior levels [affirmative]. So that I think that would really help to really integrate it into more into the culture of an organization and just everyday practice. I think that would help. Yeah, it's really a combination of marrying sort of culture and skill together to have really impactful research.
Brendan Jarvis:
That's a great thought to finish on Stephanie. It's been a really interesting and insightful conversation today. Thank you for so generously sharing your knowledge and experiences with me. I'm sure that the people listening to this episode will get a huge amount of value out of what they've heard today.
Stephanie Marsh:
Great, thank you. I hope it is helpful. Yeah.
Brendan Jarvis:
If people wanna find out more about you, Stephanie, what you do, your books your writing, what's the best way for them to do that?
Stephanie Marsh:
So I use Twitter only for research and research operations and sort of ECD stuff. So you can find me on Twitter if only I can remember my handle or yeah, I blog mostly on Medium and even if I blog on other platforms, I tend to then also put it in my medium so it's all in one place. So I would say the best places to go.
Brendan Jarvis:
Great. Thanks Stephanie. I'll be sure to put links through to your Twitter and also to your medium and the show notes to everyone who's tuned in or listening to this episode. It's been great having you do that. Everything that we've covered and the conversation today will be in the show notes on YouTube, including where to find Stephanie and her book User Research plus any of the other resources that we've mentioned. If you've enjoyed the show today and you wanna hear more of these great conversations with world class leaders in design, UX, and product, don't forget to leave us a comment and subscribe to the podcast. And until next time, keep being brave.

Stephanie Marsh

Research Ops for Global Teams

Subscribe

Who is Stephanie Marsh, PhD?

Transcript