Clara Kliman-Silver
Orchestrating Effective Research & Design
In this episode of Brave UX, Clara Kliman-Silver shares how she brings together humour 🤭, people and cognitive psychology for research impact 🚀, and how teams can make the most of their design ecosystems 🗺️.
Highlights include:
- How do you use cognitive psychology to support your findings?
- What is a design ecosystem and why is it a helpful framing for our work?
- How have you used humour when engaging with stakeholders?
- What is the right balance between AI and human agency within design tools?
- How much does research impact rely upon how it's orchestrated?
Who is Clara Kliman-Silver?
Clara is a Staff UX Researcher at Google, where she manages a team of UX researchers, and leads company-wide strategic research projects 🦉. She also works on the Material Design team and has previously worked as a Senior UX Researcher on design tools and systems.
Before joining Google, Clara was a UX Designer at Bonsai, an artificial intelligence platform startup 🤖, where she designed workflows and facilitated a range of research projects.
In 2014, Clara founded the Boston Chapter of Ladies that UX, an international organisation that connects women in the field of user experience 🙋♀️. An initiative that’s close to her heart, Clara is now the organiser of the San Francisco chapter, and a mentor to several other chapter leaders.
She has spoken at events across the globe, including at the UXPA International Conference, Rosenfeld Media’s Enterprise UX, the CHI Conference, SXSW and at UX New Zealand 🎤.
Transcript
- Clara Kliman-Silver:
- There's this notion of AI is kind of in this box where you have a bunch of stuff that goes in and then something happens and then stuff that comes out, you don't really know how it got there.
- Brendan Jarvis:
- Hello and welcome to another episode of Brave UX. I'm Brendan Jarvis, managing founder of The Space in Between, the behavior-based UX research partner for enterprise leaders who need an independent perspective to align hearts and minds, and also the home of New Zealand's first and only world-class human-centered research and innovation lab. You can find out more about what we do at thespaceinbetween.co nz.
- Here on Brave UX though, it's my job to help you to keep on top of the latest thinking and important issues affecting the fields of UX research, product management, and design. I do that by unpacking the stories, learnings, and expert advice of a diverse range of world-class leaders in those fields.
- My guest today is Clara Kliman-Silver. Clara is a staff UX researcher at Google where she manages a team of UX researchers and leads company-wide strategic research projects. Now that sounds just a little bit top secret!
- Clara also works on the material design team and has previously worked as a senior UX researcher leading research on design tools and systems.
- Before joining Google, Clara was a UX designer at Bonsai, an artificial intelligence platform startup where she designed workflows and facilitated a range of research projects.
- In 2014, Clara founded the Boston chapter of Ladies that UX, an international organization that connects women in the field of user experience clearly an initiative that is close to her heart. Clara is now the organizer of the San Francisco chapter and a mentor to several other chapter leads in North America.
- She has spoken at events around the globe, including at the UXPA International Conference, Rosenfeld Media's Enterprise UX, the CHI Conference, South by Southwest and at UX New Zealand, which is where we first met. And now several months later, it's my pleasure to have Clara here with me for this conversation on Brave UX.
- Clara, hello and a very warm welcome to the show.
- Clara Kliman-Silver:
- Thank you so much. It's such an honor to be here and great to chat with you again.
- Brendan Jarvis:
- Yes, I've really enjoyed having this in my diary to look forward to. It's been in there for several months and the time is finally here and I've learned a few things about you that I didn't know when we were talking back at UX nz. And one of those things is that rumor has it that you're occasionally a standup comic.
- Clara Kliman-Silver:
- Yes, occasionally I have been in standup shows. I started doing this when I was living in Boston after undergrad and then have picked it up out here a few times.
- Brendan Jarvis:
- Interesting. And what is it that you've learned about yourself through standup?
- Clara Kliman-Silver:
- I think that I've always enjoyed telling stories and sort of making fun of myself and stand up with you permission to do that. My first standup teacher said, set up you have two choices. You make fun of yourself or you make fun of other people and I feel like it's bit nicer for me to make fun of myself. So that's how we do it.
- Brendan Jarvis:
- Yeah, I think that's probably a way of making fun that wins you more friends than makes you enemies, that's for sure. Thinking about the work environment, generally speaking, across your career, do you feel that this sort of levity that comes out through standup, that sort of making fun of yourself, having a laugh with other people, is that something that you've often felt is permissible at work?
- Clara Kliman-Silver:
- I think so. I've been very fortunate to work in environments that have felt very friendly, have been welcoming of different kinds of people, and I think humor has been a good way to connect with new stakeholders and new teammates and also break down some of the barriers that I might expect, especially when I was starting a new role or eventually bringing new people onto my team. So I think it's definitely permitted, and it's one thing I really enjoy about being a researcher. When I conduct interviews that can be a little bit funny, get to know my participants and connect with 'em beyond the formal script. So I think it's a good skill to have. I know at least out here a lot of UX professionals do improv or there's a tradition of what can you learn from improv and moderating research study or what can you learn about how to take it? Just capitalize on situations that you're not expecting.
- Brendan Jarvis:
- Yeah. And are there any specific go-tos, for example, in the context of a research session with a participant? Do you have any sort of favored ways of breaking the ice or funny jokes that you tend to always make? Or is it really in the spirit of true improv where you just, like you said, you're laid back and you're just going with whatever's unfolding and seeing where it takes you?
- Clara Kliman-Silver:
- Yeah, I mean I obviously want to make people comfortable, so I don't want to say anything that might be too strange, but I think a lot of it is just listening to what they bring up. I try to keep things fairly loose when I moderate. Obviously it depends on the type of study that you're doing, but in general, I just want to see what they say. Some people just you ask them a question and they talk for five minutes and then that elucidates all kinds rather all kinds of interesting things that you can pull onto and refer back to later, sort of a callback as you might call it, and instead up. And that I think that allows me to connect with people to sort of show that I'm listening and to just try to weave together a story of what they're saying for a richer conversation. If I know something about what they're doing, if it turns out early on we have something in common that that can be a point that we connect about, I probably wouldn't make a joke about it, but it's definitely just trying to be friendly and trying to help them, especially if they've never participated in research before, understand what this conversation is and also what it isn't.
- Brendan Jarvis:
- I think you mentioned earlier on about using humor as a way of connecting with stakeholders and I was curious if you cast your mind back to perhaps the last time that you used humor when you were presenting some findings or whatever the context was with stakeholders, what did that look like? How did you introduce humor as a way of building a better connection when you were presenting to stakeholders?
- Clara Kliman-Silver:
- So some of it's intentional, some of it probably isn't. I think one of my presentation tendencies, whether or not this is a good thing or a bad thing, is to sort of just make puns and not really realize it and then comment on it. So I'll often just end up saying something that I don't know, maybe it's stupid, but I'll notice it. And maybe other people noticed it but didn't know if they should laugh. But then when I say, oh, look at the silly thing I did, then they laugh too and that breaks the ice. I think one thing that I try to do in most of my presentations is create something gripping on each slide. And sometimes that will be an image that could be entertaining. Sometimes it's a quote. I still want to center the research or the insight or the recommendation that I don't want to detract from that.
- But in my first role, I had the fortune of working with someone who would come out of consulting for many, many years, and I remember she told us, this is when I was a very junior researcher, imagine that you are getting a certain amount of time with someone very senior at the company and you need them to know what to act on. You need them to know what your recommendation is, what they should take away. If you have tons of stuff on your slide, they're going to zone out. So have a single thing that is interesting that provokes a conversation that makes 'em want to engage more. And I think for me, that's generally being a really provocative quote or something, an interesting graph. So it's not necessarily humor, maybe it's more humor and delivery, but I think when I've picked things that I pick something that maybe is very opinionated.
- I do a lot of research with developers and I think at least in some spheres, developers are famously blunt in their feedback. So sometimes I will have a quote or I'll have something that they say, and I'll put that up there and on the slide and really have that be the focus. And it's not something to laugh at, but it is something to engage with. So I try to center the research and have it be a conversation, but I think that it's fine to talk about it in a way that allows us to feel lighter about it while unpacking what it means.
- Brendan Jarvis:
- Yeah, it's interesting you talk about including things like provocative quotes or imagery, whether it be with a humorous touch, not just something to capture people's attention is this taps into, I suppose our part of our psychology, the things that stand out. We tend to notice the things that blend in. We tend not to, and I suppose that plays into what we pay attention to as someone who in your undergrad studied cognitive psychology. So this understanding of what's actually going on for us is something that you are quite aware of. And I understand that your first job was, or one of your first jobs was as a lab assistant in a child language lab. Yeah. What did that job involve for people that don't understand what that would be and what did it teach or inspire in you?
- Clara Kliman-Silver:
- It was a really cool job. So I got very lucky. I found out about a professor at the university that I ended up attending who had a lab that was focused on how children learn language. So we say children, we're looking at children who are around two, two and a half years, which is a critical moment in language development. And I think if I remember correctly, some of the work was on language acquisition in general. Some of it was looking at children who are growing up in bilingual environments, generally Portuguese English or Spanish English. So they're maybe hearing two different languages at home or one at home and one in social situations outside of the home. Maybe some relatives speak one and others speak a different one. So anyway, it was a really interesting job in high school I found out about the field of linguistics and immediately decided I wanted to do that.
- And then somehow when I started an undergrad, ended up in this lab that was doing what we call, so the study of how children or how people learn language, how people process language, things like that. And I was a lab assistant, so some of my work was fairly mundane. People come in to the lab, actually they bring toddlers with them because you're doing research with the toddlers. So receiving them, talking to them, entertaining the children toys, taking notes, setting up the experiment rooms. But I also got to run some experiments and I think at some point I got to work on experiments and design experiments. I was only working in this lab for a year, so trying to remember exactly what I did. But a few things that stick out. So without boring you, with all the underpinnings of linguistics, a couple of things that are interesting about language acquisition, at least in English, is English is famous for lots of irregular verbs.
- And as children, first you hear a bunch of words and you sort of pick up on patterns and figure out what you're supposed to say. And then you might learn that when you are talking about something in the past tense, most of the time you add the ed sound or I walked or I, no, everything I'm thinking of right now is an irregular verb. I steeped the T, I dunno why that's what I went for. But anyway, you get what I'm saying. So you have these past S and then there's also things like when you make something plural, you add an S typically, but not always. So children will figure these things out and then they might understand there's a rule, and then at some point they also learn that there's exceptions to the rules and the sounds that you actually produce will depend on things like does the word end in a vowel?
- Does it end in consonant? What kind of consonant? And all these kinds of things. So we were basically coming up with what are called NASS words, so nonsense your made up words that represented a whole slew of linguistic features. And we would put them in front of children basically. We were giving them a bunch of different stimulators to sort of understand how do they pluralize things, how do they put things in past tense, what are they thinking is going on? And it was a really good preparation for a career in research, which I didn't know at the time. I didn't find out about UX until much later, but it made me actually switch my major, which is what we call it here in the US from linguistics to cognitive science because I realized that cognitive science, at least at the place I went to undergrad, was very much a laboratory based degree. So instead of studying theory, we would spend a lot of time in labs studying up and designing experiments. And that was really appealing to me. So I ended up pursuing that. I worked in labs for all four years of undergrad and actually I think I worked in four different ones, some of which were language focused and some of which were professional perception focused.
- Brendan Jarvis:
- Now I imagine that working in child learning labs would've also been alongside your time, and standup would've been brilliant for developing patience and the ability to just go with the flow because if anyone listening to this has ever had much to do with a two and a half year old, it can be very challenging to arrive at the place where you want to get to with them. You have to be very patient and have to just, as I said, you have to go where things take you. I wanted to touch on the communicating what matters because something else that I understand that you've done in the past, and I'll quote you now, you've said sometimes when I'm trying to explain why I think we should do something or not do something, I'll tie it back to a psychology principle that people understand. So there you're directly leveraging your previous experience in the labs in your undergrad, and what is it about doing this, about tying it back to a principal that you feel is beneficial for supporting your position. And if I can add to this, which is probably a bit of a naughty thing to do, to ask you one question and then add another one to it, but we can start wherever you like. How do you do that without it being overtly condescending or so that you can land that well that reference back to psychology? Well,
- Clara Kliman-Silver:
- That's a really good question. I'll say what I try to do and hopefully it is successful. So I guess first of all, it depends on the context. I think one of the things that I love about this field, as I said earlier, is that it is inherently interdisciplinary. And everywhere I've worked, I've worked with people who have backgrounds like mine, people who are much more steeped in knowledge about psychology. I'm very fortunate right now to have multiple people around me who have PhDs in computer science and psychology and human computer interaction and know many, many things that I don't. I've also worked with people who are computer scientists who are visual designer or visual artists. So I think one of the things that I try to do, especially when I'm trying to communicate a research finding or maybe especially if it's something contentious or complex, if try to figure out what is something that we both understand, they probably understand better than I do, but we can use as an anchor to talk about this and what it means in UX, at least in my experience.
- Many people do have at least some foundation in psychology, whether it's truly studying psychology or they read some books, they took some courses in a bootcamp and they learned about the principles of psychology that are most relevant in say, visual design. And I think that can be another good anchor point. Many of us know at least something about visual hierarchy, about directionality that people read depending on what languages they're accustomed to or the cultures that they're experiencing media in. And that can be a really good starting point. I think one thing I also try to do, and some of this is probably because some of my career or my path into UX is being a bit unorthodox, is just admitting what I don't know and say, this is what I observed and this is why I think it matters, and here's my data or here's the thing that I'm trying to tie it back to.
- But ultimately this is a conversation I really love. And again, I've been very fortunate to have these jobs, but I really love that as a researcher, it's never felt like I'm sitting on one side of the wall doing my thing and the rest of my team is somewhere else or behind their own walls. But rather I see research as a group activity facilitated by a researcher, but something where I really do want everybody on the team who cares about it, which is ideally the whole team to have a part in understanding what we want to learn, how we learn it, and then what it means. And sometimes, again, that does mean drawing those connections to things that we individually understand together.
- Brendan Jarvis:
- And to what end is that everyone playing a part that team sport.
- Clara Kliman-Silver:
- Yeah, so again, this is my experience. I think a lot of this was influenced by the first place I worked and the way that they did research there. But generally sometimes I'll propose my own research and I'll say, I think this is really interesting. We should do research on this. And then I lead that conversation. A lot of the time there's a mutual agreement that we need to understand something better and then it becomes a conversation between myself and the rest of the team about what do we want to know? And it could be, should we think about doing this thing? Could be does this design work? It could also be one of my favorite topics. How do we know that we're successful? What does success look like in this particular context? And especially because I spent most of my career working on technical products and my entire career working in enterprise, I am never the most knowledgeable person in the room about whatever we're building, partly because I'm not an engineer, partly because when I worked in healthcare, I didn't work in healthcare.
- I'm not a doctor and not an AI engineer. So I'm always relying on other people who have the technical expertise or the policy expertise or some other kind of domain knowledge to help me understand what's most important. So I absolutely want to get those perspectives and capture them. And then of course, designers have their own questions and writers have their questions and product managers and engineers do too. So I lead the creation of the research plan, but I do want input from everybody about what is it that we want to know and what are we going to do with the insights when we do this research, what would change as a result of our findings? And then throughout the study, this obviously depends on what we're doing. A set of interviews or a contextual inquiry or some kind of workshop is very different than doing a survey.
- But I'll bring in other people partly as note takers, partly as collaborators. At one point in my career, I was working on a series of really technical, a really technical product that was focused on something to do with machine learning models. And I didn't understand very much about it. So I remember bringing in my product manager and my engineering lead and having them co-facilitate sessions with me, and I designed the protocol and I led the session, but I'd frequently say, Hey, so-and-so do you think here, or I'm going to ask my product manager counterpart to lead this exercise. And then that would allow that person to bring in their expertise and I could sort of monitor and make sure we weren't asking leading questions and things like that. But what I liked about that is that it allowed us to elicit more, I think a more informed discussion, but also it gave my teammates ownership over the data and the research in a way that I think allowed them to feel more connected, more invested, and ultimately more incentivized to address whatever it was that we were learning. And I approached analysis in a similar way as a researcher, obviously it is my job to analyze the data, make recommendations and such, but I do want to bring in other people to debrief the session. We have multiple discussions as time permits after sessions or after the study wraps up about what means and what we're going to do about it. And I found that really successful in terms of getting research buy-in and research uptake.
- Brendan Jarvis:
- There's a distinct element of humility that you acknowledged earlier on there when you were speaking about knowing that you don't know as much as some of your other colleagues do about certain areas which necessitates you involving them in the process. Just listening to what you were talking about there, how much of your ability to influence outcomes to affect change is reliant upon your orchestration of the process that you're taking, your cross-functional peers and other colleagues through?
- Clara Kliman-Silver:
- That's a really interesting question that I hadn't thought about. I think that I've encountered, I've encountered a couple of different scenarios throughout my career. So I think in every role I've had, every UX role I've had, I've worked on at least one team that never had research support before. Sometimes I didn't have any kind of UX before, and part of my job is to educate people on what I can bring, but sometimes you don't want to walk in. I mean, maybe you do, but I don't like walking into a room and saying, everyone, listen to me. I'm going to change these things for you and it's going to be great. I want them to understand and to be bought in, and ultimately, not everybody wants to work in the same way that I do, and that's a conversation we need to figure out together. So I think part of it is just establishing that I am here and here's how I believe I can be helpful, but ultimately I want to help you, so let me understand how you work, what is useful to you, and how we can do this together.
- I've also, I've encountered this in a number of situations. Sometimes it's just people have never worked with research before. They've never worked with UX before. Sometimes it's that they have, but they are 10 times zones away, and we're very rarely going to be able to be on a call at the same time to make decisions. So we just have to influence your processes and figure out something new that works anyway. And sometimes people are interested, but in the past, research has to be more siloed so they don't feel like they can ask to be part of the study or they don't know exactly how things work. So understanding where people fall in that and how invested they are is usually a good starting point and then we can figure out something together.
- Brendan Jarvis:
- That makes a lot of sense. It sounds like a very practical and sensible and intune way of figuring out where you can play a role and to the best effect. Before we get into something that I know that you are quite well versed in, which is design tools and the processes that sit around those tools, I wanted to ask you about Ladies that UX, which is, as I mentioned in your introduction, this is something that you've been involved in for nearly a decade. First as the founder of the Boston chapter and now as the organizer, sorry, of the San Francisco chapter, what is it that you value about ladies, that UX that's kept you engaged with the organization at the level that you have been for so long?
- Clara Kliman-Silver:
- I think one of the other things that I really enjoyed about it was that I got to meet people all over the world. So Ladies UX is a global organization. I think now they're about 80 chapters, and even when I joined there was somewhere between 15 and 20 I think. So at the time we would also have all of these calls where every month we would meet with other leaders who are in our geographic location. And I would check in with the people in the UK maybe every other week for my first couple of months running the Boston chapter and just got to learn about what UX was like around the world, what was different in different places, which domains or enterprise industries are big there. I had thought on and off about moving to the UK where my dad is from. And so it was really nice to be able to meet UXers in London and other cities in the UK and just sort of understand what it might be like to build a career there.
- And then later on I wanted to move to California, and as we talked about, I think before we started recording, the US is really big and it's a six hour, six and a half hour plane ride from the east coast to the west coast where San Francisco is. So I didn't have an easy way to network and I just needed to start building connections and getting introductions. So Ladies UX was actually one of the groups that helped me meet people out there when I was trying to find an opportunity. And I met with the then organizers of the San Francisco chapter. They introduced me to people, they gave me feedback on how to interview and how it was different in San Francisco than where it was in Boston. And when I moved out here, finally I felt like I had a community of people that I knew already, and moving cities can be really hard, especially when you're 3000 miles from home, so 5,000 kilometers.
- So it was definitely very helpful. And then at some point I actually became the director of North America because I was meeting other cities and basically connecting with people who wanted to start their own chapters. And I remember having a lot of conversations with the folks over Rania and APAC about what was different in different parts of North America and what might work in all of these regions are huge, but what might work in one part of the world might not in another. So that was very meaningful to me and I wanted to try to make it better and help people build and establish their own communities. But yeah, mean, I think some of my happiest memories of legacy UX have been opportunities I've had to connect with UXers in other parts of the country or in other parts of the globe because UX is so similar in so many places, but also so different. And right now I'm very privileged to work in global context, some design systems that are used all over, but at other points in my career I haven't been, so I really had no minimal exposure to international markets and what's different and certainly have very limited experience on consumer products. So I've really seen it as a way to be more educated about design and research globally.
- Brendan Jarvis:
- Well, let's talk about design and research globally and in particular, let's talk about design tools, which are often very exciting. They're always changing. They're things that help us to do our jobs, whether it's in research and or design and or engineering and product. They kind of cross many boundaries or they're starting to cross many boundaries. You've previously said about tools, and I'll quote you now, we should be talking less about tools and more about process or how people work together. So it sounds like you feel that we've got our emphasis wrong.
- Clara Kliman-Silver:
- Yeah, I mean, I don't want to downplay the importance of tools. I think tools are really important and I think that the leaps and bounds we've seen in design tooling and developer tooling over the last decade have really advanced the field and have allowed of people to do a lot of very interesting, exciting things that they weren't able to do before, maybe weren't accessible to everyone before. But I think that especially when you're working on a team, and especially when your team involves a lot of people in a lot of places, then you really need a good process. And I've seen this time and time again, both in my personal experience as a professional as well as in research I've done with developer teams, with design teams, with cross-functional teams. A lot of people, they get married to a tool or they get very fixated on introducing something or a suite of things.
- And if it improves their work, that's great, but if they can't hand something off to another role or they don't have a good way to share their work or communicate things or give feedback or whatever it is that you need to do, then it doesn't really matter what tool you're using because you're still going to have a problem. So I think yes, we should pick tools that work for you, that work for your team. There's probably no gold standard for everyone because everyone has different requirements. But ultimately you want to ask questions, how do I know what we're working on? Or how do I give feedback? Is it all written comments? Is it an email? Do you have meetings? One thing that I remember learning early in my time at Google when I was doing research with design teams was I would ask people's question, where's the final design?
- And I would ask some cases, I would ask several people who were working on the same project as question and I would be told a whole bunch of different things. Sometimes it was in a file, sometimes it was in a whiteboard, sometimes it wasn't a file or in a screenshot that had been exported. But then also there is this addendum from a product manager about these things need to change sometimes didn't exist or in someone's head. So I think that that's, to me, that's an indicator of a process issue rather than a tooling issue because you could have that problem with any number of tools. You could also probably have that problem if everybody was in the same tool. So I think we need to be thinking about the two of them in parallel.
- Brendan Jarvis:
- And you've described this interplay between tools, people and systems as something more, it's a phrase that you've called a design ecosystem. So what is it about that particular framing of what's going on here for people at work that you feel is more useful than looking at perhaps the component parts?
- Clara Kliman-Silver:
- Yeah, so I think when I started talking about design ecosystems, this actually was maybe a little bit humorous. I was working on a team at the time that was a design system team material design. And most of the work that I was seeing around me was about the design system itself. And this was sort of the context. I saw it as the context around it. So you're working with the design system to design products, to build things, to implement things, but then at the same time, you are there, your teammates are there, you're working together to actually make these things happen, and you're using tools as a conduit to actually create whatever it's that you're creating. So I was thinking about what is the context around the design system itself and how do those things influence how we do or do not get things done?
- So my point there was more that when we think about building products, and I'm not an expert on organizational psychology, but from what I do know about that field, I think you see a lot of the same things there. People are, they're not just the tools they use, they're not just the things they do, but ultimately you're doing that in a context that is influenced by what other people are doing, what they're using, and then sort of processes the underlying piece of that. So I don't know if I would say this definitively, but one thing I've observed in Silicon Valley over the years is that there is this tendency to jump and make a new product or make a new tool or whatever, an app that solves something. And there are a lot of amazing things to get made as a part of that and as a part of that tendency, but there are also a lot of things that don't really solve the problem or they don't solve a problem that people have, which again, is a reason to do research. But I think that ultimately it's really important to interrogate what is the issue and then what is a potential solution. And again, my favorite question, what does success look like? How do we know that we've solved this problem? And I think if you do that a few times, then you'll start to notice that yes, sometimes it is a failure of a tool, but often it's a tool in concert with something procedural.
- Brendan Jarvis:
- It's often also interesting to explore how do we know whether this is a problem at all, which is to what you were talking about there, it sounded like almost many things in Silicon Valley get created, most of which sounded like their vitamins rather than painkillers. And there are some good outcomes from that because you get a lot of activity happening, and then obviously some really great things bubble up to the top there that really do solve problems. But there's possibly a lot of wastage along the way coming back down to design ecosystems. Now I understand that you have designed a workshop that you use to help teams understand exactly what that ecosystem looks like for them, and you like to start it with a simple prompt. What is that prompt and where do you go from there?
- Clara Kliman-Silver:
- Yeah, so this was something that I developed with some of my teammates that initially was actually a research activity. We wanted to understand how teams were building products, but then we realized after the first session, or actually after several sessions, that people were commenting that this activity was actually very therapeutic for them, and it was allowing them to recognize what wasn't working and help them figure out for themselves what might make it better. So that's when we started talking about using it as a tool to improve processes in addition to a research activity. But to your question, the prompt is or is generally think about a recent project. What did you do first? What did you do last and what happened in between? Who is involved and what tools to you use? So again, sort of those pieces of the ecosystem. So the way that the workshop has generally gone is we'll ask people to think about that.
- I like to have people draw pictures because I a bad artist, but I think, well, when I doodle, so I often do that also just to get people kind of in the mindset of thinking about things piece wise and being willing to just put stuff out there and see what happens. So we'll have them map out these steps for themselves and then have them come together as a group and talk about what does the flow of whatever activity or project that we are working on look like. So usually that's when teams, if you're talking about designer and developer teams or cross-functional teams, they'll say, okay, well at some point we decide what we're doing. And that's actually an interesting question within itself because where does that come from? Is it a mandate handed down from somebody? Do you come up with it as a group?
- Does someone on the team say, Hey, I really want to do this, and everyone else is like, cool, let's do it. Or is there some other version of things going on? But everybody will map out their process. Then the team's process when you write out your tasks and then you look at what did you do, what did you use to do that task? So that's a tool question. Then how painful were certain aspects of this process for you? And basically then look at those pain points and have conversations about how you want to improve them.
- Brendan Jarvis:
- You said before, well, I dunno if these are your exact words, but it sounded like it was fairly cathartic for teams to go through this. What is the pain, if you like, that prompts people to want to do a workshop like this? Or what are some of the common pains that are experienced in teams and what is the solution or the value or the outcome that they get from mapping these things out together?
- Clara Kliman-Silver:
- Yeah, so I'll say sometimes it's a hard sell. When we were doing this as a research project, we had told teams that we had recruited that we wanted to do this for three hours, and that was a lot of time. And people said, okay, well can we really do it for three hours? Is this really necessary? So that in itself was difficult. But then I think once people saw the conversations going and they realized that in a way it's like a retro, if you're following, I mean, this again is more of a engineering thing, but other industries have their own versions of this. Following a scrum type model, you probably have some kind of retrospective at the end of a sprint or end of a cycle development cycle. And that can be a moment to say, here's what worked, here's what we want to fix, sort of start, stop, continue type frameworks.
- So for groups that have those kinds of processes in play, they may be doing this already, but again, that's very isolated and you sort of have to think in the minute about what was frustrating you this time or what was the specific thing that stood out. Whereas a lot of the time, what I've found when we've done these activities, and again, I'm looking at this more from the lens of cross-functional product development teams, but I know people who have taken this method and applied it in other industries, and a lot of the time what I've heard is that it's not so much a specific tool that's causing the problem or maybe a specific activity that's causing a problem, but often a lack of awareness. One role is doing one thing and another role is doing something else at the same time, but because they're not aligned, that creates these three extra steps down the road.
- And that's an issue. So a commonplace where you see this is designer developer handoff. So what happens after a designer designs something and then sends it to development for implementation? A lot of the time there are breakdowns there, but sometimes there's breakdowns earlier on. How do you decide what you're doing? Is this a beginning? Does everyone have the same interpretation of what needs to be done, why it matters, what success looks like? What is the minimum viable version of this feature? Something like that. So we've seen a lot of things sort of fall between the cracks of steps or tools, and many of these things are process issues or communication issues rather than tooling problems in their own right.
- Brendan Jarvis:
- And by that, it sounds like there's actually a lot more hard or perhaps more complicated or complex work that needs to go into resolving them than just swapping out a tool.
- Clara Kliman-Silver:
- And I think yes and no. I think it's a different kind of work. Swapping out a tool is often not trivial either, especially when you think about all the different things that might be contingent on that tool and the ramp up and ramp down of transitioning files or work or whatever, integrations on and off that tool. But I think when you do this activity and what teams have found cathartic is looking at it and say, oh, actually maybe we have been aggressively going against changing the tools or the resources we're using for something, but actually it's because these two people just don't get enough time to talk to each other, or these two things are happening in isolation and we don't know what the status of either one is. Sometimes
- Brendan Jarvis:
- It's an aha moment as a result of the workshop, people come in with like, oh, do I really need to even be here? And it sounds like what they walk away with is like, oh gosh, now I truly understand why we keep having this problem.
- Clara Kliman-Silver:
- Yeah, I think that is something that surprised me about it in a very positive way that people were getting that out of it and saying, oh, this is an issue on our team. I remember we did one of these research activities, and the team wanted to document everything for themselves because they could take it back with their larger team and use it in their next retrospective and say, this is what we learned about our process when we did this random workshop that some research group recruited us to do.
- Brendan Jarvis:
- How do you set up the conversation for the people participating in the workshop so that they can be candid about the issues, but focus on the issues and not the people so that people don't get their noses at a joint when perhaps a controversial or a sensitive area, maybe it's the way decisions get made or how things get handed over, how do you grease those wheels?
- Clara Kliman-Silver:
- Yeah, so again, I'll speak to this from the research perspective because I haven't really been present for a lot of these as a reflection activity. So I can't say what has happened in those rooms. But what I've tried to do, I think by being a researcher and making it very clear that I'm here to learn about your process because we want to study this thing or we're doing this because we want to understand how you do this thing or how you engage with this tool or whatever, I think that removes some of the tendency for people to blame each other. I think also because I'm there and I'm an outsider and I am not there to mediate, I think people also tend to, I've not experienced people yelling at each other or anything like that. One thing I try to motivate is that this activity is not a retro in the same way that something at the end of a development cycle might be, but this is more to understand all the things that are happening and where there might be gaps and it's more seeking a solution that might be an immediate thing, but it might also be a longitudinal thing sometimes.
- There was one that we ran where somebody learned, one team realized that they just didn't have a key role. There were people who were doing this thing as they could and it was rotating around the team, but ultimately part of the problem was that there was no one who was constantly responsible for this activity, and that's something we needed to fix. So I can't say that we do it perfectly, but I think that when you treat it as a research activity that helps. And for retros, looking at this is an opportunity to see what we might be missing when we reflect consciously on our work. That can be another way to talk about it.
- Brendan Jarvis:
- I'm getting images here of like a post match review of game footage in a sporting context where people are, we're all on the same team, but we're all reflecting back on our performance on the game that we've just played. And you are talking there about setting the context is an important, it sounds like setting the context is really important in helping people to feel like they can participate in that conversation more openly. I want to talk to you about now a couple of surveys that I understand that you've done on tooling before we come to something different. I think you did the first one in 2019 and you had around 600 respondents, people who worked in UX specifically, and you ran another one in the middle of the pandemic in 2021 and pre pandemic when the first survey was done, you found that 50% of the respondents were using a collaborative tool when designing, and perhaps unsurprisingly, most participants came back during the 2021 survey and were working in a real time collaborative sense with their colleagues. But that is, I said, is probably to be expected. I think the interesting thing that I wanted to ask you about here was that you also found out that amidst all of the remote collaboration that was going on, that people still wanted to do things by themselves and their own private spaces. How strong was that desire and where do you think that desire originates from?
- Clara Kliman-Silver:
- Great question. So I don't know that I measured specifically how strong it was, but I do think that design adjacent things can definitely be group activities. And I think collaborative tooling has made big headways there because it allows you to participate in real time, but also even if you're not looking at the same screen at the same moment, you can see what someone did three hours ago, or you can see that you're seeing the same thing that everybody else is seeing at some point in time, and you don't have to worry about is this the latest version of the thing? But I think that there's still, there are many people who for good reasons, want to do things on their own. And sometimes you just need that space to iterate and you want to try out 10 different things, and you want that to be your activity, and it's okay if other people can go and look at what you just did, but you want that space to be creative. And some people are creative in groups, and some people are creative on their own, and I think we still need to honor that as we're thinking about how people will build things in the future.
- Brendan Jarvis:
- Well, speaking of the future and how we'll build and perhaps design things in the future, Phil, this is a really good point to talk about something else that you've done, which is a few years ago you wrote a paper where I believe you tried to classify human perception of AI driven experiences of which, and feel free to talk to these. There were three dimensions that you came up with. But one of the dimensions that I wanted to focus in on here is it sounds like what you were talking about there with the online collaboration that happens with our design tooling as there's almost like a will in some people to exert more agency over whatever it is they believe that they're doing or participating in and having some of that, this is me projecting here, but having some control over a personal space to create in is exercising that agency. And something that you identified in designers in this paper, this paper about AI and designer's relationship to AI is that there's a tension that exists between how efficient AI infused tooling can make us and saves us time doing manual tasks, things that we'd prefer probably not to do, but there's also this tension between that efficiency and also our agency as creative human beings. So is there a sweet spot as far as AI's influence in our tooling goes like a balance to be struck here?
- Clara Kliman-Silver:
- Yeah, I mean I think that's absolutely the question of the moment. A lot of, as we've seen new advances in AI powered tools and both in design and development, but also beyond, I think that there's this question of what is the right amount of intervention and what is the right amount of you let the tool do everything versus you let the human do everything. I think one thing that we've seen in a lot of our work and our colleagues in academia and other people who are doing this research and investigations also observed is that I think it's important to make sure that users have a certain amount of agency and control and they feel like they're in the driver's seat. One thing that we've observed is that when people don't know what is driving a decision, they don't understand when something's happening, then that reduces trust.
- And this is something that I talked about it at UX n z and tooling, but I think also this has been a thing for a long time that one of the early things that I remember hearing about AI when I started working in the field in 2016 was that there's this notion of AI is kind of in this box where you have a bunch of stuff that goes in and then something happens and then stuff that comes out, you don't really know how it got there. And one thing that we heard repeatedly is that when people don't know what happened in the box and it makes it harder for them to trust what's going on, and obviously it's more complicated than that, and there's a lot of stuff that is happening in the box that you may look at it you still don't get. And that's, that's just how it is.
- But I think that ultimately giving users agency and allowing them to be able to approve or define parameters or sign off on certain decisions is really key. And a lot of the work that we've looked at around tooling, people have said, well, I want to be able to sign off on a design decision that an AI powered design tool made because it will give me confidence that whatever we collectively as in myself and the AI are creating is the right thing. I think that that's something that even when you look, and this is again is an area where I'm not an expert, but when you look towards some of the consumer products, don't always, you can have AI powered settings and consumer products where they make guesses about what you might want based on prior behavior or things like that, but you as a user can generally tune these and override things because you know yourself better than the AI does.
- Brendan Jarvis:
- So in the context of designers working with these AI infused tools, or perhaps generative AI more broadly where certain aspects of design work that people wanted the right of veto over their AI for, and were there other aspects where people were more relaxed or was it uniformly this desire to exercise control at all levels over what was going on in that black box?
- Clara Kliman-Silver:
- Yeah, this is a nascent field. I think that it's something that we're starting to see more work on. We're starting to read more papers about, but it's something that is relatively new in the design space specifically. I think that what I have noticed from the speculative work that we did and from some of what I've read from others mostly in the academic world who are doing research in this space, is that generally when things are less, it's great to have lots of explorations. So I think a lot of us do this when you're designing something, if you maybe make a bunch of different versions of something and just see what sticks, and that's a place where generative AI could be very useful. But ultimately when you're about to ship something, you probably want to have a look at it first and make sure that whatever gets produced meets your standards and passes your bar.
- Brendan Jarvis:
- And that sounds like that would be something that before ai we would've done anyway. We're applying a similar procedural approach to how we would manage our creative work or our cross-collaborative work.
- Clara Kliman-Silver:
- That's a really good point. As we were
- Brendan Jarvis:
- Before. I'm curious about this. I think you summed it up really well. People see it as a black box and therefore it's difficult to trust. And the other side of that coin is that it's magic whatever gets produced, and I feel like as a field, we are getting more comfortable, this is just a feeling here, more comfortable with understanding how it works and how we might be at risk and how we might also see it as an opportunity to evolve our work. And it also seems to me that on the other hand, we've got a coin and a hand here in this little analogy, but on the other hand, we've also seeded a lot of our control over to computers as is already not really, I couldn't profess to really understand all the inner workings of what's going on in some of the work that I've done.
- And for some reason, I think perhaps because of the mystery that still exists around AI and some of the hype and the science fiction that inspires a bit of fear. We're still a little bit reticent with it and possibly rightly so. There are some ethical considerations. I understand that you've also looked into previously that may impact design. And again, I know you've said that you're not a specific expert in this area, but what are some of the things that as designers and people in product that may be working on these tools can be mindful of when we are creating them, that they don't cross over from this sort of AI is cool territory into AI is now creepy. What are the things that we can be doing to prevent that from happening?
- Clara Kliman-Silver:
- Yeah, so as you acknowledged, I think when I was at the startup that I was at before I came into my current role, this is something I'm thinking about, but this is now like five years out of date. But I do think some of these things still likely apply. There's a lot of discourse and a lot of really great AI researchers out there who have talked about the need for diverse data sets, making sure that you're really inspecting and evaluating training data to make sure that it's representative looking for biases, for issues with your models. I think, again, not an expert in this space, but one thing I've noticed casually browsing things over the last couple of years is that it seems that there are more and more companies and research labs that have really invested in the AI ethics world and have created tools to allow you to inspect models and to, or frameworks for thinking about these things.
- So I'm really happy to see that there has been a lot of attention given to that. I think a lot of it though is also the same as we think about in product research or user research for things that are not necessarily AI powered test with your users and make sure, as we always try to do that, you're looking at a wide range of use cases and a wide range of people. One of the beautiful things about product design, although this is obviously risky, is that people often use things for things that were not the, for situations that are not the intended purpose. So you want to make sure that you're not just asking, can people use this, but how do they use this and how don't they use this? Because sometimes there's a mismatch in use cases between what you intend and what people end up actually doing. So I think those are two things that are really key.
- Brendan Jarvis:
- You've previously referenced Google's people plus AI guidebook. It's commonly known as peer, P A I R as a resource where people can learn a little bit more about the best practices and methods as it relates to applying or designing for AI or with ai. Is that still your go-to and your recommendation for people to have a look at?
- Clara Kliman-Silver:
- Funnily enough, someone asked me this yesterday. Yes, it actually, it is. I mean, so this guidebook was written by a wide range of experts from across Google and people who are studying ai, people who are working in UX and many, many other roles. And I really like the chapter in there on explainability and trust, which is basically what we're talking about. How does understanding what a model is doing, at least at some level, how does that engender trust and how does that influence trust? And how do you help people feel comfortable with an experience if they don't totally understand what's going on? So I think that reading resources like the para guidebook to look at what people are saying, how things have been applied, what are common missteps that can be really valuable. Cool.
- Brendan Jarvis:
- So I have one final question for you today, which is the designers out there that are currently struggling with their design ecosystem, whether it be because of too many tools or issues with decision-making or difficulty collaborating with their cross-functional peers, what do you suggest they do first to improve that situation?
- Clara Kliman-Silver:
- I would say begin by interrogating what you consider your ecosystem. So what are the tools you're using? Who are you working with? Who aren't you working with that maybe you want to be working with? What resources are you using? Looking at some of the, I guess, exogenous factors, so things like if you're on a distributed team, which many of us are these days, are you in the same place as most of us are not working in the same place as our collaborators? Maybe you're not in the same time zone, which is something that I think a lot of people are dealing with in this day and age. Maybe you don't have the same devices or the same resources or things like that. So sort of trying to understand what are some of these extenuating circumstances that are actually really important and see if some of the problem lies there, and then what you might be able to do about it versus what happens just fixating on a specific tool or a specific activity that you perform. Those things may absolutely be part of the issue. They may be at fault, but it's not impossible that there's something else going on too.
- Brendan Jarvis:
- I think that's some really great advice and a great place to finish our conversation today. Clara, thank you. It's been such a great experience speaking with you about your perspectives on design ecosystems and also touching on AI at the end there. Thank you for so generously sharing your stories and insights with me today.
- Clara Kliman-Silver:
- Thank you so much for having me. This was a joy.
- Brendan Jarvis:
- This might be my pleasure. Clara, if people want to keep in touch with you or want to keep up to date with the things that you are doing in the field, the type of work and talks that you are giving, what's the best way for them to do that?
- Clara Kliman-Silver:
- Feel free to reach out to me on LinkedIn, find my name. I assume it's going to be spelled somewhere on your podcast because it's long and hard to spell, but yeah, feel free to reach out to me on LinkedIn. I also have a website, which is my surname.com, no hyphen, and I try to post things there. You can contact me that way too.
- Brendan Jarvis:
- Thanks, Clara, and to everyone that's tuned in, it's been great having you here as well. Everything we've covered will be in the show notes, including where you can find Clara, including her website and LinkedIn, and anywhere else that I can link to where she's been publishing things on the internet.
- If you've enjoyed the show and you want to hear more great conversations like this with world-class leaders in UX research, product management and design, don't forget to leave a review. Subscribe to the podcast as well, so it turns up every two weeks and tell someone else about the show if you feel that they would get value from these conversations at depth.
- If you want to reach out to me, you can find me on LinkedIn, just search for Brendan Jarvis. There's also a link to my profile at the bottom of the show notes, or you can head on over to thespaceinbetween.co.nz. That's thespaceinbetween.co.nz, and until next time, keep being brave.