038 - (Special Co-Hosted Episode) Brian and Mark Bailey Discuss 10 New Design and UX Considerations for Creating ML a...

Mark Bailey is a leading UX researcher and designer, and host of the Design for AI podcast — a

program which, similar to Experiencing Data, explores the strategies and considerations around designing data-driven human-centered applications built with machine learning and AI.

In this episode of Experiencing Data — co-released with the podcast Design for AI — Brian and Mark share the host and guest role, and discuss 10 different UX concepts teams may need to consider when approaching ML-driven data products and AI applications. A great discussion on design and #MLUX ensued, covering:

Recognizing the barrier of trust and adoption that exists with ML, particularly at non-digital native companies, and how to address it when designing solutions.

Why designers need to dig beyond surface level knowledge of ML, and develop a comprehensive understanding of the space

How companies attempt to “separate reality from the movies,” with AI and ML, deploying creative strategies to build trust with end users (with specific examples from Apple and Tesla)

Designing for “undesirable results” (how to gracefully handle the UX when a model produces unexpected predictions)

The ongoing dance of balancing UX with organizational goals and engineering milestones

What designers and solution creators need to be planning for and anticipating with AI products and applications

Accessibility considerations with AI products and applications - and how itcan be improved

Mark’s approach to ethics and community as part of the design process.

The importance of systems design thinking when collecting data and designing models

The different model types and deployment considerations that affect a solution’s UX -- and what solution designers need to know to stay ahead

Collaborating, and visualizing — or storyboarding — with developers, to help understand data transformation and improve model design

The role that designers can play in developing model transparency (i.e. interpretability and explainable AI)

Thinking about pain points or problems that can be outfitted with decision support or intelligence to make an experience better


Resources and Links:
Designing for AI Podcast

Designing for AI

Experiencing Data - Episode 35

Designing for Analytics Seminar

Seeing Theory

Measuring U

Contact Brian

@DesignforAI
Quotes from Today’s Episode
“There's not always going to be a software application that is the output of a machine learning model or something like that. So, to me, designers need to be thinking about decision support as being the desired outcome, whatever that may be.” - Brian

“... There are [about] 30 to 40 different types of machine learning models that are the most popular ones right now. Knowing what each one of them is good for, as the designer, really helps to conform the machine learning to the problem instead of vice versa.” - Mark

“You can be technically right and effectively wrong. All the math part [may be] right, but it can be ineffective if the human adoption piece wasn't really factored into the solution  from the start.” - Brian

“I think it's very interesting to see what some of the big companies have done, such as Apple. They won't use the term AI, or machine learning in any of their products. You'll see their chips, they call them neural engines instead have anything to do with AI. I mean, so building the trust, part of it is trying to separate out reality from movies.” - Mark

“Trust and adoption is really important because of the probabilistic nature of these solutions. They're not always going to spit out the same thing all the time. We don't manually design every single experience anymore. We don't always know what's going to happen, and so it's a system that we need to design for.” - Brian

“[Thinking about] a small piece of intelligence that adds some type of value for the customer, that can also be part of the role of the designer.” - Brian

“For a lot of us that have worked in the software industry, our power trio has been product management, software engineering lead, and some type of design lead. And then, I always talk about these rings, like, that's the close circle. And then, the next ring out, you might have some domain experts, and some front end developer, or prototyper, a researcher, but at its core, there were these three functions there. So, with AI, is it necessary, now, that we add a fourth function to that, especially if our product was very centered around this? That's the role of the data scientist. And so, it's no longer a trio anymore.” - Brian
Transcript
Brian: Hello, everyone. Welcome back to a special edition of Experiencing Data that's actually not just Experiencing Data today. Mark, why don't you introduce yourself and tell them what we're doing today?

Mark: Yeah, sure. Hi, I'm Mark Bailey. I'm actually the podcast host for Design for AI. It's another podcast, it's in the same space.

Brian: Yes, excellent. So, just for listeners of Experiencing Data, you know what my typical guests are like, and I talked to Mark, and we both felt like there was a lot of synergy between our shows. And while, typically, I don't have designers on the show, I thought Mark had some really good tactical and strategic advice for people looking at User Experience and Human-Centered Design on a wide variety of different artificial intelligence angles. And so, we felt it would be great to just record a conversation that's less host and guest-oriented and more just us having a conversation about this space. And so, we're both going to be releasing this episode. This will be a Design for AI episode as well as an Experiencing Data episode.

Mark: Yep. And so, if you're listening to this on the Design for AI side, I would definitely recommend listening to Brian's podcast, Experiencing Data.

Brian: Likewise, I recommend for all Experiencing Data listeners out here, too, the Design for AI podcast has some really great stuff, and I've been enjoying actually reading a lot of the transcripts is what I've been doing lately, just to, kind of, dig into some of those topics. So, check that out at designforai.com, Mark, I took some notes down just to—I just had these ideas in my head about things that I think are important for—I don't feel like the product design community is necessarily thinking about these things, I don't know about you. So, I thought I would just rattle off my 10 things, we can go one by one if you want. I'd love to just get your reactions to some of these concepts and see if you agree with me or challenge me on them. What do you think?

Mark: Yeah, sure. No, that sounds like a great idea.

Brian: Cool. I'm going to try to give a quick overview of just all 10 of them, and then we can run through them. The first one here is that—and again, I'm thinking about these in terms of designers, particularly people working on software products, because I'm guessing that a lot of what the audience is on the Design for AI side, we assume there's some kind of application that's going to be the output of the work, or that is the product.

So, the first one of those is, we still need to be problem-first and application- or products-second. So, when we're talking about AI, and particularly machine learning, the point here being that there's not always going to be a software application that is the output of a machine learning model or something like that. So, to me, designers need to be thinking about decision support as being the desired outcome, whatever that may be. So, even if you're spitting out an Excel spreadsheet for somebody, or whatever that output necessarily is, that's really what we're trying to do with some of this.

The second thing is trust and adoption is still a challenge, particularly at non-digital-native companies. So, getting employees, and internal users, and partners that are supposed to be taking a new version of something that might have been an offline process, and then integrating an AI-driven solution. We really need to be thinking about these UX principles of adoption and trust of the solution. And I think that's a—designers get that. But that's really something that needs to be factored into the solution space. You know, actually, Mark, let's go—I’m not going to go through all 10 here. They're too long, and I want you to be able to chime in.

Mark: Okay.

Brian: So, let's take those first two right there. Do you agree? I've seen some mixed stuff online about this, like, do we still need to be problem-first oriented with data, with AI-based design? Do you agree with that? Or do you think—

Mark: Oh, yeah, no, definitely. So, actually I think this is one of the most important things for designers that are involved in machine learning, that they really need to learn the space because a lot of the different machine learning models out there have different, kind of, side effects to them. So, I could give a really technical answer or just an overview.

Brian: Yeah, I think keeping at a high level I'm not sure how much—one thing that's tough, that's ironically about our podcasts, right, is we don't always know exactly who our audience is unless they're writing into us. So, I took the assumption, none of our audience needs a basic overview of what AI is. They all probably get that. They probably know what machine learning is, but maybe they haven't worked on a project before. But this problem first thing, it still is important. And it's not, how can we use ML for X? That’s a question that I think constantly needs to be challenged. It's not about can we use this hammer? It’s, what is the need? What is the gap? What's the better future we want? And then, is ML the right thing for that? Sounds like we're in agreement on that.

Mark: Oh, yeah. No, totally. So, anyway, so what I was going to say is that, so once you've defined the problem, and you know what the problem is, that really dictates if you're—so, the joke is always if you even need to use machine learning. That’s always the first question and then the second one is, you have to work with the developers and of course they're going to be recommending their own machine learning model that they like the best, or the one that they have the most experience with. And so, that’s—because you know what the problem is, you need to know what the side effects are for that machine learning model. And so, because there are, I would say there's good 30 to 40 different types of machine learning models that are the most popular ones right now and knowing what each one of them is good for, as the designer, it really helps to conform the machine learning to the problem instead of vice versa.

Brian: Right. Without going into all of them, can you give two examples of that to a designer so they conceptually can understand why do you need to know about this?

Mark: Well, sure. So, an example would be—so the hot thing right now is, AGAN. So, it's a machine learning model that generates images, or sound, or whatever you want. Now, you could use this model, and it's very broadly based. Or if you had—so a different thing would be a Transformer model, and something like the one that just got really popular, GPT-2 is the one that can create text, and it sounds like a human is talking, but it's very specific. The way that it works is it looks at the next word, and it just tries to predict words, so it spits out a string of text. And so, it's very narrowly focused. So, depending on what your problem is, one might be a really good solution, or the other one might be. So, that would be a good example of that.

Brian: Got it, got it. So, moving on to the second one, and we'll try to get through all these 10, then maybe we can reflect on some of the hot ones if we want. Again this idea of trust and adoption, when we're starting to use these types of solutions that can be perceived as black-box—we'll get into model interpretability down the road, I think that's an area as well—but do you agree that this empathy for the customer, it doesn't go away. We still need to understand their context for use, what are their risks and concerns such that the design of the solution factors in the human engagement as part of the solution. So I always talk about, you can be technically correct and effectively incorrect, right? All the math part is right, but it's ineffective because the human adoption piece wasn't really understood. So, talk to me a little bit about what you think about that with the trust and adoption angle here.

Mark: Sure. So, now, are you talking about it from getting the end-user to trust the data or how to get the data so that the user can trust it?

Brian: I'm talking about simply in the last mile where someone is going to sit down and interact with whatever the output of the, quote, “data science project” is, am I willing to use this? And again, I tend to think more in terms of enterprise data products, things like this. Am I willing to use this? Do I understand it and trust it enough versus the old way I do things, or whatever came before?

Mark: Got it. Yeah, so when you're designing stuff—and I mean, AI right now, as soon as you say AI to a customer, the first thing they think of is Terminator, and so the trust level goes down. I think it's very interesting to see what some of the big companies have done, say, like Apple. They won't use the term AI, they won't use the term machine learning in any of their products. You'll see their chips, they call them neural engines instead have anything to do with AI. I mean, so building the trust, part of it is trying to separate out reality from movies. So, if you can just provide assistance to whatever the user is doing, you don't have to call out that this is machine learning or anything like that, and that's a very good first step as far as building the trust. And then, as they accept that, then you can—it's very much a staged process. So, you just add more and more features as the trust builds.

Mark: Sure.

Mark: And I think a really good example of this is Tesla, with their self-driving cars. They have rolled it out very, very slowly, just because that's a very big trust.

Brian: Right. Yep. No, I agree. I think it's funny in the enterprise space, and even the startup, I think—one of my listeners was telling me how 80 to 90 percent of all funded startups right now have AI or machine learning as deeply—a theme or it's literally in the title of the company, right now. Business users, I see it, when they're buying software, it's like everyone's jamming it into their marketing stuff. The consumer end, though, the end-users, I'm not sure, as you said, you can see the opposite end of that, right, with Apple really avoiding some of these kinds of words. So, ultimately, I think, when the smoke clears a little bit from all this stuff, it's still going to come back down to those design things we all know about, right? It's not about, we need a social version 2.0, you know, the web 2.0 or whatever. All that hype is going to blow over at some point. And it's going to be back to does this thing, this solution help me do the work that I need to do? Does it make my life better? It's going to come back to that stuff that designers are focused on all the time. But I really think that trust and adoption thing is really important because of the probabilistic nature of these solutions. They're not always going to spit out the same thing all the time. We don't manually design every single experience anymore. We don't always know what's going to happen, and so it's a system that we need to design for.

Mark: No. And so, that's one of the big things is consistency. You're going to build trust, the more you can build consistency. And one of the things—when I'm designing a product when I talk to the developers, I always say, “Okay, so I want you to predict, before you build the model, what's a really good answer that you think is going to come out, and then what's a—say that the machine learning model goes in the completely the wrong direction, and it spits out a really bad thing.” And then, when I do user testing on both of those, I can actually see how the user responds. And if they're not—you can see the trust take a hit when they get the bad answer. And so, you need to be able to design the experience so that when that bad answer comes up, that the flow of the models, it allows the user to go back and change their mind or you know, they—

Brian: [crosstalk] recovery of some kind—

Mark: Exactly, yeah.

Brian: —built-in.

Mark: They don't have the problem, they're not stuck with it.

Brian: Right. Right. I’m curious: when you do those tests—I had talked about this on another episode of my show asking some fairly well-known data scientists whether they ever prototype without building out these solutions because, when we're doing predictions, we can come up with mockups and prototypes that don't need any data science behind them, we can predict outlier responses to test reactions. Is that how you do it? Do you come up with some wild outlier answers, some, kind of, in the ballpark answers and then you look at those, how people are reacting to it and then adjust accordingly?

Mark: So, you're talking about doing a—how to do prototyping and stuff?

Brian: Yeah. Like, being able to move quickly. How can we learn quickly, and you're saying show me what a really awesome answer, a high-value answer would look like versus a complete failure, right? Or even probably something that could be really risky looking. How do you—

Mark: So, it depends on the situation, of course. How hard it is for them to build the model. Of course, as soon as they build a model, they're going to want to ship it. And so, that's a whole ‘nother problem, moving from the prototype model to a production model. So, if they can predict—it really depends. If they're just doing a new version of the model, it's pretty easy to predict on what the accuracy that they're shooting for. But if they can't predict it, and they have to build a model, what I've found that works for me is that you have them choose one really narrow persona type, and to tweak the model only for that one persona. And then, when you do the user testing, it's real easy to bring in 20 something that does whatever job and it likes whatever things so that all of the recommendations can be in this very narrow range. And then that is able to test—and that way you don't have to worry about building the entire model. And so, it's a lot easier to get the accuracy if you can tweak down the persona range so that it's very narrow.

Brian: Got it, got it. Yeah, that's a great idea. So, we sort of touched on this one, but the, I think, designers have a role in addressing the, how can we use ML to do X question. So, this gets back to our leads, our design leads really trying to go up the ladder into the problem-finding space. So, one thing I was going to ask you what you thought about this. I have an acronym that I like to think about with this which is [00:15:55 PICAA]. And it's these different verbs to help teams think about how they might use machine learning particularly. And it's Predict, Classify, Augment, and Automate. And it's looking at the human’s work, or the jobs, or the things they're doing and figuring out, are there places we can use these verbs to help their experience and then the I in PICAA is Ignore, which reminds us that sometimes we need to ignore machine learning entirely and not use it because it's not the right hammer for the nail. Is that how you like to think about it as well when you—I don't know if you've been in this situation where teams feel the itch. Like, there's something to do here that machine learning would be valuable for, but we need that objective lens. And so, how do you go through that?

Mark: So, yeah, no, I've come across this. A lot of the time there's a stakeholder that’s saying, we're collecting all this data, it's costing us money, I want to be able to use it. And so, I usually have a little bit simpler—so they have the list of all of the things that they're collecting, so all the different classifications of things that they're collecting, as far as data goes, and then what I usually do is I take a list of all the problems that their users are reporting. So, anything that their customers have a problem with, we're trying to track that stuff anyways, so we can pull that, and then it's literally just trying to draw lines between the two. And if it's a prioritized list of the problems then it does a really good job. Because, I mean, there's usually a lot of parallels, but it's usually fixing problems that are really low on the problem priority list. And so, that does a really good job of putting those—they get a lot lower rating as far as in the scrum meetings, I should say.

Brian: Right, right. Yeah. And I think this also gets back to things that hopefully the product design leaders are doing, which is also little looking at organization goals, quarterly goals, things like this, and making sure that the work here—particularly on projects that can be months to even years-long—we're not just going through a tech rehearsal, where we're trying to use this technology, and at best, we learn something about how to how to build a pipeline and deploy a model, but deploying a model isn't a solution for a person. That's an engineering milestone, it's not an outcome. And so, I think designers typically understand this, that we need to look at our business objectives as well as what the customers want to do. But in general, this problem finding thing it's understanding what the tech can do, but keeping it back to, as you said, what are some of the known issues that exists now, and providing that leadership there, so we don't approach it as a data problem. I feel like that’s—with the data science persona, it's a data problem a lot of the time. And I'm always like, no, it's not. It's a human problem, which may have a data problem, but that's a technical issue. That’s its core when we're using these technologies, but that's not really what it's about. I know do you agree with that?

Mark: Oh, yeah. No, it's like you're saying, it's the hammer and nail, everything's a nail kind of problem. Yeah, no, when you do get one of those where they are collecting data, and they do match it up to a problem that is high on the list, those are real happy moments, because everyone's happy. You're solving a bad problem for the customer and it's for free because they already have the data.

Brian: Right? Exactly, exactly. The fourth one I wrote down was another place that designers can be helpful here is, is getting the Star Trek experiences. One of my guests talked about it. I call it envisioning the grandchild. So, it's like, what's that two-generations-out version of your product? What might it look like? What is that futuristic thing? But also starting to think about that you need data to do a lot of these things, we will need data. And I think when designers can be thinking ahead to that, and not just creating fictitious-type, way outside visions of the future, having some kind of context about we will need data for this, and where might that come? And what concerns might arise when I do show a prototype of a grandchild of our current products? What are some of the downsides to that as well that we need to think about? How do you think about future prototypes and things like that?

Mark: Oh, yeah, it's very much a product map. You start with the blue sky, if I could create a perfect experience for the user, what would that be? Okay, then you break it down. Well, what data do we need to be able to create that experience? Well, most likely, you're not collecting half that data right now. So, the only way that you're going to be able to start collecting that data is if you give the users something. So, what are you collecting now that you could start—that you would make it so that the customer would want to use it? So, the MVP, or whatever. And then, how can you slowly expand that to where you're collecting the data that you need to improve your product to eventually be able to get to that blue sky thing, where you're able to collect all the data you need to give the ultimate experience?

Brian: Mm-hm. And do you think it is that designers, I mean, I'm sure you've had this thing, right, like, should designers code? I do not want to have that conversation with you here. My short answer is no, you don't need to code. You should know conceptually how to code, and you should know what an API is, and you should know what these languages can do, even if it's not your job to do it because it helps you understand the medium you're working in, and it helps you work better with technical people. And overall, it's just like we're asking our stakeholders to give a shit about usability, and testing, and user experience, and research, and all this stuff, it works both ways. That's our toolset, their toolset are models and data pipelines and all this other stuff. So, how much is it the designer’s responsibility to think about data collection in the context of AI?

Mark: I mean, it's, the more tools you have, the better the job you can do. So, I'm very open to it. And I mean, one of the things that I feel is very important as a designer, is that you definitely want to train your developers and your data scientists on the best practices of UX. And it seems kind of selfish, that if I'm going to be training them on how to do my job, that the vice versa isn't also true. So, definitely, the more you know—and so like I was saying earlier, the more different machine learning models you know, you might not know how to implement them. But if you know, what they are, what data they need, I mean, that's going to help you with the product map, too.

Brian: Yeah, I agree. And I think if you're just getting started with this, you'll probably get that exposure over time. And I think it can be hard until you've gone through a process to contextualize some of these things. So, it's not like you can't do any work until you've learned all of this pre-work. It's always a journey, right? And we're all starting at different places, so you can still get in there. But I think being open to the fact that this data stuff, it's like pixels, it's like the new pixel, and we need to think about that just as much as we think about screen layout, and—I think you had commented in one of your episodes about, we don't need to recreate another table GUI. For the rest of our lives, I'm going to keep coming up with tables. We don't need to spend—there's a better use of designer’s time, right? There's bigger problems to work on than that. And as much as I love beautiful renderings and really getting the pixels right, and the rounding of the corners, and which radius, and the fonts, and all this kind of stuff. You know, a lot of that is not—those aren't problems anymore.

Mark: Yeah. Well, but I mean, that's the point that we are in the grand scheme of things is that this is—machine learning is still very much a research endeavor. Everything is a custom solution right now, and definitely that's one of the things that I do have for my podcast is I don't want it—I mean, it can't survive if it stays everything is a custom solution. So, I was thinking that, because this is all research, I would say that not even the developers are experts. Not everybody can know everything, because this is such a new thing. That's one of the reasons that I was attracted to this space, is because I found that everyone's really humble because there is so much unknowns. There's a whole lot of stuff that when I talked to the developers, and I ask them, “Why is it working a certain way?” And they don't know, it's just, it does. So, yeah, they're working in the same space as us. You know, we don't know everything. Neither do they.

Brian: Right, right. Let's move on to the next one. The next one I wrote down is—there’s an overall topic of ethics and community as I like to think of it. So, second-order consequences to the products and services we build, and particularly edge cases, right? And I forget who said it, you probably know, but someone said, “Edge cases define the edges of what you actually care about,” or something like that. There was some great quote, I don't remember where I saw it, but I was like, wow, that really sticks, right? Like when you say that's an edge case, then that's the edge of what you're saying you care about. You've talked about—I've seen in your episodes about accessibility issues with this, where models round stuff out and they don't factor in what could be perceived as outliers when they're not necessarily outliers. So, can you talk a little bit about second-order consequences, ethics, accessibility? What what's your take on this?

Mark: So, I mean, when I've done an accessibility design, that's always something that, like you were saying, a lot of stakeholders like to focus on the core user base. And so, that's usually something that I have to upsell, is if we think about accessibility, this is going to expand our market by 10 to 15 percent. And that's usually a pretty good, convincing factor. The problem, like I've said before, is that machine learning, it averages everything. And so, accessibility can be left by the wayside. So, to be able to include accessibility—and I mean, as machine learning gets into the enterprise space, this is definitely something that's going to have to be covered better than right now. You know, for the consumer space, you can throw machine learning models out there all day long, and just say, “Oh, well, that's not part of our customer base.” That doesn't really work for enterprise or government solutions. So, one of the hot topics right now is called an ensemble model, where you take a bunch of little models and you put them together. This does a really good fix for fixing for accessibility because you can just create a specific model that is only for low-vision users or speech-impaired users, depending on whatever your solution is, and include that. And so that, if someone has that accessibility need, that there's already a model there for them to use.

Brian: Mm-hm. Got it, got it. And talk to me about your thoughts on, you know, ethics gets thrown out there. It's kind of this amorphous thing. I actually just interviewed Cennydd Bowles on my show, the author of Future Ethics, who I think is a great voice in this space about what designers need to be thinking about here. How do you approach this? Do you use some of his toolsets? He talks about the front-page test, which I love, where it's like, would you like to see this on the front page of the news if it went south? Would you be okay with your family and your friends knowing that you participated in building a solution that showed up with this article on the front page of the New York Times, or whatever? I think that's a great way of thinking about it. The designated dissenter role, where your job is to be the naysayer and to think about how could this go wrong. What are some of the ways you think about ethics and community?

Mark: Yeah, definitely, it's one of those best practices kind of things of how you can alter the process more than—this is one of—it's definitely one of the things that you have to get in at the very beginning as opposed to at the very end. Who you're impacting? Who is this going to be touching? That kind of stuff. But one of the ways that I found it, people are motivated by money. So, if you can work this into a performance review, so how can they measure how they're affecting the users. And I found when that is part of their performance review, people, they really start to care about it. So, that helps.

Announcer: Here's what people have been saying about the designing human-centered data products seminar.

Participant: A pretty quick and accessible snapshot of how to think about user design for data solutions products. I mean, you can go super deep on UX design, you can talk about data solutions, but how do you combine the two in a very digestible way? I think it's a very, very solid framework to get started.

Brian: Interested in joining me and a group of fellow students the next cohort? Visit designingforanalytics.com/theseminar for details and to learn how to get notified when pre-registration begins. I hope to see you there, and now back to the episode.

Brian: A sixth one I talked about—or had in my ideas to talk to you about, understanding the full system. And this starts out at the point of data collection, transformations of the data, what training data is being used, testing data, all this kind of stuff. Even the experience of, I think, the Google PAIR researchers, their toolkit talks about factoring in even the design of the rater’s experience. So, if you need to manually collect data, like is this a car or not? The example I give is, well, if you show them a picture of an SUV or a truck, is that a car? If the answer is, is it a car or not? Well, it's a vehicle, right? Some people might say, “Yeah, it's a car because it's not a tree.” And so, right at that point, you've introduced bias into the data. And so, part of the, to me, the UX thing is also looking at all these different points along the line that feed into the overall last mile, the GUI that finally ends at the very end of the process. Do you agree with that, or do you think that's not our job—

Mark: Oh, yeah.

Brian:—that’s the data scientist’s job?

Mark: Well, so it depends on how you view your job. You know, if you're looking at the whole user experience or even one step bigger is the whole customer experience, so from the first touch of the brand all the way through the whole lifecycle. The intent of why they were collecting the data is so important, just because—the one that I've always heard is a veterinarian collecting data about dogs is going to have different intent on what data is collected as opposed to a breeder. And so, the data that's collected—if the source of that data is going to affect the accuracy of your model in certain ways based on, say, if it's about a dog's health. You know, you're going to get a whole lot more—the intent is going to be more health-based if it was a veterinarian that was collecting that data. So, that's something that you need to look at as the UX person. But also, how that data is transformed as it goes through the system. So, you can't—I’m not expecting everyone to be able to read through the code. But if you can storyboard the model with the developers, and I usually try and do this as just some kind of a design session with the developers themselves. And a lot of the times I will find holes, that they hadn't thought of, in their model that need to be caught. And suddenly those become bugs that they need to be fixed anyway.

Brian: You talk about, kind of, doing this in a design jam format. Can you give listeners, especially designers who tend to be visual, what's an example of one of those bugs that you might have discovered recently, that became something that needed to be paid attention to?

Mark: Yeah, it's really, it's when you're creating the model, you have to make sure that the data gets massaged in the right way. And you have to catch a lot—so you have to catch the zeros, and you have to have all the different groups represented. And so, each group has a different model, and then you have one big model that chooses which model to use in different circumstances. And so, it's really a matter of if the person came in through this way, then we know this about them, so they are most likely to get this model. But there's also this other circumstance that can affect it. I don't know how to really make it universal. I mean, think of it like mapping out the user journey. And it’s, pretty much, it's just that you're doing, but it's within the model itself.

Brian: Right, right. No, that's helpful, for sure. The next one I wrote down here was tuning. So, probabilistic applications don't spit out the same thing every time. So, we sort of touched on this earlier, but do you agree that there's this tuning phase from a UX perspective, too, when we see in test, an unexpected answer or an unexpected reaction to a prediction that's come out? How is that handled in the software? Is there recovery? Is there a way to—I was playing around, is there a way for me to undo the playing around so I don't start getting these strange recommend—like I never listen to rap, and now I just tried it out with rap, and then it's like, now I'm getting all these rap recommendations, and it's like, oh, how do I get back?

Mark: Yeah, that's all you get.

Brian: What do you think about tuning?

Mark: Yeah. So, recovery is definitely important. So, I mean, it really depends on what is on the line by the machine learning making this recommendation. And one thing that I've found that helped recently was that not only was it supposed to show the degree of accuracy that it believed that the machine learning model was recommending the correct thing. But then it also showed a different percentage of the degree of accuracy that it thought it was representing the wrong thing. And so—but this was for a very important choice. So, you wanted the user to be able to either take or deny that recommendation with as much information as possible. And so, the first one with the degree of accuracy, that's pretty much—any machine learning model does that, but the degree of accuracy that the machine learning model thinks that it's wrong, that's not normally the way the machine learning models are built. So, including that information required the developers to build a whole ‘nother model that was listening to the first model. But given those two numbers, that really helped with the user journey, and that the user was able to make a more intelligent decision with that data.

Brian: Mm-hm. Yeah, I think, you know, for, I'm guessing, some of the designers is maybe getting a little heavier technical, so I think it's worth stating that if you're new to some of these technologies, especially with when we talk about a machine learning model, it's not going to come up with a complete wild surprise. It can't factor in a huge change that has nothing to do with the world that that model was focused on. It's really good at the one thing it was trained to do. And you've been talking about how you chain these together with other models that might predict the different thing, and then eventually there's some type of output that the user is going to be presented with. So, there's this general intelligence thing is not what we're talking about here, it's not going to come up with a surprise. That's not how it works. And so, designers need to realize that it's not just, throw all this data at it, and then magically, somehow data scientists get the wand out, and then it's going to come up with these cool ideas. They're, kind of, dumb in a way. It's very statistics-driven, and very specific and narrow. And so, that's why we're talking about multiple models and this kind of thing.

Mark: So, yeah, no, that's a very good point. You know, this isn't like the movies. Being in the design side of it, you're constantly amazed at how dumb computers can be. [laughs] Yeah, so always remember that any machine learning model is never going to make a recommendation for something that it already hasn't seen. So, you can look through all data and that'll give you all of the edge cases right there. And if your data set is too small, then that might be limiting what the recommendations are possible that your machine learning model can actually give.

Brian: Mm-hm. This gets in the next one, number eight for me, which was model transparency, or sometimes this is called model interpretability or explainable AI. And this is how well our interfaces can actually explain why did it say 82 percent was the number? Why did it say turn right? What factors went into that recommendation? This doesn't always matter, like, there's certain times where I don't care why it’s—like, why should I watch this movie instead of that movie? Do I care? I mean, Netflix, it doesn't give me a justification for every movie that it thinks I want to watch. But should you get this loan or not? If you're in enterprise, like at a bank, “Sorry, you don't get the loan.” That's the answer from the loan officer. No explanation, it's just we fed your information into a computer, and it said, “No.” So, to me, this is very much where designers have a role, right? It's like what type of transparency is needed, not just for risk and compliance reasons, which is a good reason—and I think GDPR is definitely a factor here—but what do you think about that. I mean, talk to me about model transparency here, and interpretability, and the role of designers?

Mark: Oh, yeah. And I think this goes right back to the very first thing that we were talking about where you have to solve for the problem first. How much of a problem is this for the user? What are the consequences for the user, based on what your recommendation is? Like you were saying, if this is a movie choice, eh, that's fine, they can just ignore it. But if you're telling somebody that they don't get a loan, there could be a lawsuit involved. And so, there's one machine learning model where it's completely transparent as far as all the decisions it makes. The problem is is that it can't make as good recommendations as the ones that are black boxes. And so, trying to balance between those two extremes, that is something that—it depends on the laws that are involved, and what the customer needs, and the budget of the product, and what data you have. And so, there's tons of variables that go into every part of it.

Brian: Sure. I’ll also add to, I mean, I had a whole episode on my show about this where I think the title was, like, “When is 60 percent better than 85 percent accurate?” and the context for this was that for certain situations, 60 percent accurate model in a solution, or an application, whatever it is, that gets deployed and put into the business or into the hands of a user might be better than waiting six months to get a 85 percent accurate model when that decision doesn't need to be that informed. It could be something where you're transitioning from wild-ass guessing, where people have no idea, they just currently—either that or they have to do a tremendous amount of mental math and looking at analytics and all this kind of stuff to inform a decision. Now they can get a 60 percent accurate decision. That may be enough in some cases to move the business forward or move the user experience forward. So, I think that's something that designers should be looking out for, particularly—and you tell me if you agree—with certain data scientists, this is a very academic discipline, I think a lot of them want to create very accurate models and accuracy is seen as this thing that you strive to have a—write a paper about it or whatever, when it's a business context—

Mark: [00:41:47 crosstalk] the holy grail.

Brian: Yeah, it's not the holy grail for the customer necessarily or the business. So, we have to think about our MVP mentality, shipping fast when it's appropriate, moving things along, and learning, and getting that feedback early to justify it; is it worth spending three months to get from 82 to 87 percent accurate? What does that mean, for everybody, for the business, for the customer? Is it worth that investment and, this much compute resources, all this extra data we need to get or whatever it may be?

Mark: So, definitely. That's something that you want to do at the very beginning. When you're doing the initial design of the product, you want to figure out what the metrics are that you're going to be measuring success by, because, like you were saying, users aren't going to notice a 3 percent better accuracy. You need to focus on the whole journey. Mobile's done a really good job of this, where it used to be that they wanted all the data upfront. And now, most mobile apps change it so that they only ask for data when they need it. And so, it makes it so much better journey, but they don't know as much about you upfront. And so, it's the same thing.

Brian: Right, right. Cool. A couple more here, in terms of like, how might I go out and move for, we talked about how do we find problems and use cases to use this? Or I should say, that since that's not really the goal, how do we manage that conversation and have that conversation? So, one of these is looking at your product and thinking about the concept of injecting intelligence. So, small prototypes of little features. So, if you think about, you know, when I'm typing in Google Docs, and, like, I’m writing my notes here, and it's predicting the words that I should write, and I can just hit tab and it will accept the recommendation that comes out. So, that might be something where it's like, well, we have a lot of typing. There's a lot of data entry required in our service. So, thinking about that as a small piece of intelligence that adds some type of value for the customer, that can also be part of the role of the designer. So, it's not like let's build the AI version of our product. That's not the thing. It's where's there a pain or a problem that we might be able to use some decision support or intelligence to make that experience better? So, I don’t know what do you think about that, injecting intelligence in little bites and pieces?

Mark: Oh, yeah, no. So, like we were saying before, when you have the ultimate product map, so perfect example is the text recommendations. So, every time you either accept or deny the recommended text, it's collecting the data on whether that was accurate or not. And so, it's learning how good the recommendations are in real-time. So, it's collecting some very good data for that. Now, based on being able to collect that data, could you include some new feature that would expand even further down the road, to give the user more what they want? It really depends on what data you need. And so, that's why you need to map this stuff out so much in the very beginning.

Brian: Mm-hm. Cool, good advice. The last one I wrote down here, and this is—I was particularly thinking about tech companies, but this could go beyond that, too, is that for a lot of us that have worked in the software industry, our power trio has been product management, software engineering lead, and some type of design lead. And then, I always talk about these rings, like, that's the close circle. And then, the next ring out, you might have some domain experts, and some front end developer, or prototyper, a researcher, but at its core, there were these three functions there. So, with AI, is it necessary, now, that we add a fourth function to that, especially if our product was very centered around this, and that's the role of the data scientist. And so, it's no longer a triad anymore. Is that how you think of it, that now there's really needs to be four roles here that are really kind of critical to this?

Mark: Yeah, I mean, I guess I see it more as jobs that need to be done. And everybody can do so much. So, it depends on what you're trying to produce, and you have so many tasks. Because this is such an early—products are so early, there's been so much crossover. I’ve found myself doing a lot of PM work, and a lot of UX, and research, and so, I've had to wear a lot of hats, but you're just going down the list of things that need to be done, you can start checking things off. And I found, I even, when I talked to developers, they split up the data collection, and the model creation, model optimization, and the model serving. There is a bunch of different jobs that they have to do. And so, a lot of it is, do you know how to do it? Can you do it effectively, if someone's just looking over your shoulder to make sure you're doing things right. That's one thing that I found for this is that the titles start to matter less because there's a whole lot more blending, and I think that things will start to get firmed up a little bit more in the future, but right now it's pretty gray area as far as what exactly your title is.

Brian: So, let me rephrase this. I'm not talking about that there's necessarily four bodies in the room. I'm saying that these responsibilities—there's a hub and spoke to me, this kind of—product management is the center and you have all these spokes where product may or may not be a commercial software product, it could be an application, or whatever the value is in terms of the business, the stakeholder, the customer, and there's these spokes and roles that need to be factored in, particularly at the birthing of this, the strategic part upstream. It's not to say data engineering’s not important and thinking about all the infrastructure that needs to be there, but do you agree that that, at least in my experience, this trio, I see it constantly with tech teams, it was product engineering and user experience, whether or not it was three bodies or not, but I feel like you can’t—even for the designers listening, data scientists, a lot of them don't call themselves software engineers, and I imagine some designers probably lump them together with engineers because it sounds technical. And they don't think the same way. People with data science backgrounds have much more focus on math, and statistics and they're thinking about things differently than maybe a software engineer is, but they all can contribute different things to the strategy about the solution that's being created. So, I don't know, that's more what I was thinking about.

Mark: Yeah, no, that makes sense. Definitely to create a good product it's going to require a lot of data. And so, having somebody—or, really thinking about how you're going to massage that data and clean the data, that needs to be done. So, having somebody that can do that, it's something that is going to be a requirement that isn't normal for any other type of software being built.

Brian: Mm-hm. I’d also say, too, I'm going to—I always feel like is it bad to generalize? But I'm going to generalize for the design audience here and state that in my experience, speaking at data science conferences and analytics things, this audience tends to be, as a class of people, a lot of them tend to be very STEM-oriented people. They're very technical. A lot of them like to get very focused on the technical problems. It's kind of similar to engineering in that, and the human part here is the part that I think designers can really help with: the empathy and connecting their technical work back to how it is experienced by someone who's going to receive it. They need that help, they often want that help, and we can learn a lot—we, quote, “the designers,” can learn a lot from them. They can help us understand things like bias in ways that we may not be thinking about it. And so, together it's a good soup when we're all in the kettle together, the pot, we can make a much better soup together. So, that’s, I don’t know, that's my take. And it's not to say everyone’s STEM, and they're all nerds and all that. I've met so many cool, quirky, awesome, artsy weird data people and even I—one of the guys I met at one of these conferences is telling me, he's like, “No, data people and engineers are not the same. They're a completely different breed.” And I can kind of see it now after having focused in this area. Super sharp, really interesting. It's a fun—I am totally stereotyping about a huge class of people here, but I think there's something there, where there's a complimentary skill set, if we really generalize the designer and the data scientist together, I think we can do better things.

Mark: Definitely. There's definitely this wide range of people. But I think part of the domain is that there is a whole lot of math involved. Being able to build a machine learning model, it's all math, it's all statistics. And so, somebody that understands that, they're going to at least be able to shift their mind into that mode at some point to be able to think, statistically, how to do the recommendations or whatever. So, it is something that you do need to complement. When you're designing, you got to think a lot bigger picture than more than just the model itself.

Brian: Mm-hm. Do you think all designers need to take statistics to work in this field going forward, or do we not? Is it like—yes, no, maybe?

Mark: Well, I suck at it.

Brian: Me too.

Mark: But, I think that—so all the developers that I've talked to, their description is basically, the human brain is not built for statistics. So, it's not something that you have to feel bad that you're bad at. Basically, anyone that's human, isn't. And it's a matter of memorization. So, again it's one of those tools, I am desperately trying to get better at statistics. I know all the tools but I forget them, and I have to look them up on how to do the math every time I need to do something.

Brian: Well, I'm glad you're doing it, because I'm certainly not doing the math part. I will interject something really quick for designers that are curious about this—maybe you have a favorite link—the Seeing Theory project from Brown. So, the URL is seeing-theory.brown.edu. It's a visual introduction to probability and statistics. So, if you just want a quick overview with pictures, which I know designers love, and even if you're not, even if you are technical, this can be a great refresher for a business person who's working with data science teams to see these concepts. So, I don't know if you have a favorite that you like.

Mark: So, I would recommend measuringu.com. So, it's somebody named Jeff Soros. He’s, kind of, an expert at doing the statistics for UX. And it does a really good explanation of different things that you need to know. And it has a really good just webpage. So, you can punch in numbers and get back the percentages and, kind of, help you if you're trying to do the planning for a lot of this stuff.

Brian: Mm-hm. Got it. [00:54:03 So, I'll put those, seeing-theory.brown.edu and measuring, and then the letter u.com are those URLs. We'll put those in the show notes]. The last thing—so those are really the 10 things that I had. There's one other thing here. And that’s, what if you're a designer—what do you think—you’re a designer and you know there's initiatives to use AI in the business—and by the way, it's probably coming, even if you're not at a tech company, this is something that boards are asking CEOs and leadership, what is our AI strategy? And most of them don't know what it means, they just know, their competitors are probably doing something with it. And they want to know what they're doing. And here's a pile of money, go hire some PhDs and do something. At some point, there's going to be an expectation of some results from that, and I think designers can help with this. But let's say, Mark, if you're a designer, working at a company where that mission is on the books, this is a goal, how do you get involved if you don't feel like—no one's asking for the designer to come and participate in these activities, but they're trying to maybe transform a back-office whole process or something, and it's got all these touchpoints, and they're going to start using machine learning to take care of a bunch of this work that used to be done by three or four people, and you're the designer, you can see how maybe it's not going to work, or how it could be made really efficient. How do you interject yourself into that? What do you think about how to get that trust or participation?

Mark: The seat at the table?

Brian: Yeah.

Mark: So, you basically, you show what you can do for them. It can be something small, and you got to build up trust, and you're doing a sales job as far as what UX can do for the developers and for the PMs. Or if you're working from the PM side, you got to show the bigger picture, and this is really easy when you're doing user testing. You know, people swearing at screens are very persuasive.

Brian: [laughs].

Mark: So, you do a best-of—you know, a bunch of clips, and I found that to be very persuasive as far as showing that there's a problem.

Brian: And did you predict the particular swears? Like, before you tested, or—

Mark: [laughs].

Brian: That would be fun. But yes, I mean, we're back to our old stuff, right? Get out the B-roll footage, not even the A-roll footage, and show the pain, show the gap. But, my suggestion, it's similar there. It's if you can change the conversation from how are we using AI to figuring out what is the desired value or outcome—not the output, the model, or the applications, the output—but what is the outcome that we want that thing to provide to the customer, or to the business, if you can talk about it in terms of that, and say, “Look, we can get the model part right, and we can fail at the outcome part.” I can help make sure that the outcomes are actually achieved, and work with the team building the outputs. And I think if you can talk about it in that language, you can get that seat at the table if that is something that you're having a challenge with.

Mark: All right.

Brian: Yeah. Well, anything else? This has been a fun conversation. I've never done an episode like this. [laughs].

Mark: Yeah, no it has. Yeah, no, I actually, I need to drop off.

Brian: Yeah, this is a good stopping point for me too. So—

Mark: Okay.

Brian: But yeah, this was great talking to you, and I look forward to staying in touch with you. So, feel free to contact both of us. My email address is brian@designingforanalytics.com. Mark, how do people get in touch with you?

Mark: Probably the easiest way is on Twitter. I'm just on twitter @DesignforAI.

Brian: Awesome, awesome. Cool. Well, follow him there, and for the Experiencing Data listeners, thanks for chiming in, and we'll be back in a couple weeks.