JJ Rorie
- 1 min read

Deep Dive into AI / ML Product Management

Episode 037

In this episode, with guest Prerna Kaul, Senior Technical Product Manager at Amazon, we dive deep into product management for Artificial Intelligence / Machine Learning products, including:

Overview of AI / ML products
Building a strategy for AI / ML products
Designing AI / ML products
Iterating and improving AI / ML products
Root cause analysis for AI / ML products
Getting into AI / ML product management as a career
Resources to keep learning

Episode Transcript

SUMMARY KEYWORDS

ml, product, customer, problem, ai, building, machine learning, solve, model, space, learning, standpoint, pms, data, factors, solution, metrics, question, deeper, people

Intro 00:03

Welcome to Product voices, a podcast where we share valuable insights and useful resources to help us all be great in product management. Visit the show's website to access the resources discussed on the show, find more information on our fabulous guests or to submit your product management question to be answered on our special q&a episodes. That's all at product voices.com. And be sure to subscribe to the podcast on your favorite platform. Now, here's our host, JJ Rorie, CEO of great product management.

JJ 00:34

Hello, and welcome to product voices. I'm excited about this episode, this is our first deep dive episode. So we tend to spend about, you know, 20 3040 minutes at most with our guests talking about a topic and you know, digging as deep as we can. But typically, it's a little bit of a high level view of topic area. And today, I really want to dig deep into this. And so I've got this amazing guest with me who's actually been on the podcast before. And I'll link to that episode as well. But we're gonna dig really deep into AI and ML product management today. And so with me today is Prerna Cole, Senior Technical Product Manager at Amazon. She's built AI and ML products and payments, healthcare and Alexa domains. She's also worked with Deutsche Bank, Citi Group and Walmart Labs, where she built industry leading product innovation. So no one better to have this conversation with Prerna. Thank you so much for joining me.

Prerna 01:33

Thank you so much for having me, TJ. It's an honor to be back.

JJ 01:38

So before we jump into a deeper look, let's start with an overview of AI ml products. I know that's a broad area, but give us a bit of background and foundational viewpoint.

Prerna 01:53

Absolutely. So I think the right approach to take care would be to look at the problem statement, as any product manager would you have a business goal that you're trying to achieve a customer goal that you're trying to achieve. And once you've identified it, you want to figure out the right solution, what would be the fastest path to get you to your goal. So you explore multiple solutions, you may explore purely technical solve, you may want to go with something that is completely untechnical. I'll give you an example. So at Walmart Labs, I was solving for the problem where people didn't want to wait in line to check out. And they wanted to use a mobile app to essentially complete the process of scanning all their items, paying for it and then leaving the store. Now we could solve for that by providing associates on the ground at every step and having them actually processed manually. Or we could solve for it using the technical solution. Or we could apply some form of machine learning to say, you know, what are the key things that this person buys on a repeat basis and adding them to the cart automatically. So as you think more about your customer, you realize that there are more delightful ways to solve for their problems. And that's really where AI ml comes into play in the product world. So there are typically four sets of problems that you can solve using machine learning. There are, of course, other classes. But to simplify, one set of problems, typically is recommendations. So when you go to amazon.com, what you see on the website recommendations based on your previous purchase, and search history would be one form of that. There's optimization. So looking at regression analysis across credit scores, or analyzing how much mortgage interest rate a customer would need to pay based on their purchase history and their repayment history would be a second. Third would be extracting meaning from data. So identifying hidden patterns. So that's the identification portion. And that really looks at how do you derive meaningful insights using big blobs of data? That mean does not necessarily make sense to the to the human eye? So that would be a third. And finally, I would say the classification space would be a fourth, right? So when you're trying to identify one or zero is this, is this a cat or a dog? Is this a fraudulent transaction? Or is it a valid one? Are there errors in this grammatical sort of sentence or is it completely accurate? up so those are the four categories or classes, which we typically solve for using machine learning? But overall, I hope the thought process makes sense where you're really working backwards from your customer, and trying to look at it as a product problem and solve for it as effectively as you can.

JJ 05:13

Yeah, I love that, that summary. And it's, you know, AI and machine learning products are fascinating, but sometimes can be intimidating for someone who doesn't have that technical of a background or that much, you know, information or experience with them. But I love the way you've, you've just kind of broken it down to the problems, the case studies, I think that's a really great way and certainly will help someone get get their head around it. So I want to ask if, when you're when you're building a product strategy, or maintaining a product strategy for AI ml products, is there a different spin to creating that? Or is it similar to any other type of product?

Prerna 05:55

Yeah, so I think any product strategy, as PMS will know needs to be grounded in reality, what is feasible, and what is impactful? Those are the two key metrics that you would look at from an impact standpoint, where machine learning benefits is that if you have the data to support your use case, you can start deriving the benefits really quickly from machine learning. So you create a machine learning model that is optimized for certain kinds of goal. And you launch that and you immediately start seeing benefits versus doing something that's, you know, pure tech or rule based solution. So from an impact standpoint, that impact is something that you would distinguish or categorize based on the models that you have available. As you think about feasibility, I think the challenges and also the benefits there are largely dependent on the team and their execution strengths, right? So from impact standpoint, we know machine learning can be very powerful from a feasibility standpoint, when when it comes to execution on the ground. If you don't have a team that is well versed in machine learning engineering, so things like feature engineering or feature competition, how do you do online model inference? These are some things that you will learn as you dive deeper into AI ml product. Or you don't have modelers who have the domain experience or the willingness to explore these problem statements in more depth, and then build models really to target and solve the problem. If you don't have those two core pieces, I do feel like although you, you could propose a very powerful product strategy and write eloquently, you, you will still fail on the execution side. So I think that as I think about product strategy, those two factors come into mind. In addition, I'd say that product strategy doesn't necessarily have to be a three to five year vision. You could also propose it in phases, where you have a two month vision, what are you trying to achieve using machine learning in the next two months, your six month vision, and then so forth. So I think that PMS really dive deep into the data and get very, very nuanced in the AI ml space about what they can achieve with machine learning, which I think is much more unique as compared to other spaces.

JJ 08:40

Yeah, that's an interesting point that you bring up, especially at the end with the, you know, a shorter term vision or strategy, if needed, because I could could imagine and, you know, full, full transparency, I am certainly not an AI or ml expert, but I could imagine it being a little difficult to to, you know, ascertain what the markets going to need, or if customers are even going to understand what they need. In the future, like in some kind of a little bit more mature problems or products or technologies, we can we can predict, you know, several years down the line, and maybe we can't hear and so I want to dig a little there. I have kind of a follow up question there, which is, can you talk a little bit more about how you go about laying out long term strategy for these products? And again, as as you said, and which really resonated with me there, there may be some ambiguity around what customers and markets will need in the future. It sounds like that's true, first of all, is is Do you see that where it you know, it's a little bit hard to predict what customers and markets will need in you know, any kind of longer term future is that true?

Prerna 09:56

Yeah, I think from a three to five years standpoint there are several trends that can be solved for using machine learning. Right. So I will definitely start by acknowledging that, whether it be healthcare or retail or real estate, thinking about problems in manufacturing, logistics, and transportation, several other industries, we do clearly see that AI ml would be beneficial and impactful in many ways. And we have problems and trends that we're spotting that will will not necessarily be problems right now, but will be problems, five years down the line that we need to solve for. So that doesn't change. What is difficult to predict, though, is whether machine learning will solve for it effectively. And I think that is where I was going earlier with the execution risk. I think that it is definitely a trial and error process, where you're launching things iteratively, and then learning from the feedback and continuing to improve your model and your engineering processes to be more effective as an AI ml PM. So that is, you know, what's unique about the product strategy space. So within AI ml, where you could test and learn from market experiments. And that's that's one thing that PMS will, in any case, do. But in this case, you're also testing and learning from an engineering standpoint, there are lots of different things that you would have to throw at the problem to get to the right solution.

JJ 11:39

Yeah, that makes a lot of sense. So So tell me a little bit more about how to balance that how is a product manager or someone working in this space, going to balance the kind of ambiguity with, you know, finding things that we can do now and finding things that we can do in the future?

Prerna 11:57

I think that's a great question. Right? So I think about it from a customer standpoint, and I think about what my customer needs right now, what are the low hanging fruits that I can really target. So I'll give you an example from my experience working in, in the payments organization with an Amazon, our customers have this delivery promise that we commit to so if you're within Amazon Fresh the experience, or if you're buying something on Amazon Prime, we have a delivery promise that we commit to so we tell you that, you know, this, this XYZ item, if you're purchasing let's say, a pair of jeans from from, try before you buy, it will be delivered to your house in the next 12 hours if you make a purchase in the next five minutes. So if you think about that experience, that's very specific, right. And the way we achieve it is by looking at the logistics of delivery, and really forecasting, both supply and demand to give you the most delightful experience that we can. So we know that this is a problem today, where you're trying to get your goods on time. And we know we have the data to solve for it, and can effectively deliver on a model. However, there are problems that are coming up in the future too. So as an example, you want the same experience, not just for the things that you buy, but also for, you know, your transportation or your health care. Or as you think about your interactions with the world, there are so many opportunities and trends that are coming up and telling us and really helping us understand what customers want. So as I think about sort of the ambiguity associated with it, I really go back to the customer needs. And I say, Well, what do my users want to do? Can I solve for it using machine learning? If so, what do I need to build? And then what do my users need in the future? And how can I start laying down the path to making sure that I give them what they will need in the future? And then again, I think from a machine learning standpoint, there are certain keywords that we can definitely go into. But let me pause there and see if you have have some questions.

JJ 14:20

Well, what's interesting to me, and I love this, and again, I'm a bit of an outsider, if you will, in this domain. But you know, so often customers, I often say that customers can't necessarily articulate the details of what they might need. They we just need to ascertain their problem and then solve. But I would think in this in this instance, it's I mean, that the customers, you know, to use your example, you know, me me as an Amazon customer buying something to try on or to have for a while. I have no I have no idea what's working in the background. I have no idea how you do it so well, right. And so it's a really quintessential example of, you know, using a technique Unless you're having a solution that is not transparent to the to the customer, but it ultimately makes their experience so much better. Right. And so I think that was just that's an interesting nuance to, you know, being a product manager in in the in the product ecosystem for these types of products is that you're you have to understand very deeply the the problem, as you say, but then also how how, or if, you know, AI or ml can actually be a benefit, which I just think is really interesting. And I think that's, I love the way that you've, you've talked about this, because it's really resonated with me in in how that's just an interesting nuance for these products.

Prerna 15:42

Exactly. And you will notice that you're much more so involved with the science and the technical aspects of building out this product strategy than you would be as a PM working, let's say, on a mobile application. So that is another nuance that hopefully is showing through here.

JJ 16:04

Yeah, absolutely. So tell me a little bit more about where you were going there at the end of into that, that answer the the key terms and that sort of thing to dig a little bit there. And tell me a little bit more what you were thinking there?

Prerna 16:21

Yeah, sure, did you. So I think how we think about problems is, of course, twofold. We think of it from a customer standpoint. But we also think of the feedback loops that we will need to build, and we need to generate to continue improving our modeling solutions. So some words that PMs in the ML space may be familiar with are transfer learning, active learning, and thinking about sort of incorporating different sorts of features in order to improve our prediction capability. So those are some feedback loops that we very diligently think about as PMS do that? How can I start building for this, and incorporating it in my roadmap, and ensure that it gets built so I can accelerate the process of innovation within my organization with my team? Within my company?

JJ 17:22

That's interesting. So So somewhat of a segue to the to my next question, I think. And again, I'm just fascinated by this, this whole topic. So I know that you, you keep coming back and educating me, I love it. Oh, yeah. And by the way, people are listening. But it's all about me. And you teaching me so. Okay, so I want to I want to know about design for these products, again, because it gives you know that it's it's not that it's it's a complete departure from you know, traditional, if you will, product management, but I could I can imagine that there are some nuances in this as well, how do you go about, you know, the design of AI ml products? Can you share some of your, your experiences and knowledge there?

Prerna 18:05

Yeah. So, I think a really good way to think about it is in the form of our success metrics for the product itself. And that will give you a flavor of how we design a IML products to essentially ensure that we're either meeting a positive metric or exceeding it, or or not, incurring some negative sort of friction for a customer, right? So in the retail space, you have your success metrics in terms of engagement and retention. And we would define retention in the form of repeat customers, repeat purchases, and we really think about it very sort of, clearly that, you know, I have my my adoption metrics, I have my engagement and retention metrics. And the experience that I'm building really needs to optimize for those right that that is really clear sort of perspective, as we think about AI ml, specifically, when we're looking at products that are perhaps omni channel, perhaps hardware products, their net new innovation. So within the Alexa organization, you have devices within, let's say, working for Google or Facebook, you have, perhaps their website, the metrics that they're looking at are those traditional metrics of engagement and retention, but they also have a ton of friction metrics that are specific to ml. So they may have user perceived latency as one of their goals or they may be looking at whether a customer who is trying to do a certain thing has achieved their goal or not. So they have very specific things they look at in terms of user perceived you latency within AI ml experiences, they look at whether you've achieved your goal as a user. And they actually optimize for those things, because they understand that if there's some AI ml working in the background, there's no, you know, clear set of rules behind it, there is some some level of ambiguity in terms of how the model is making decisions, that that can be things that go incredibly wrong. And you want to make sure you're not giving your customer a bad experience.

JJ 20:31

So are there any other nuances that the team must be aware of as you're going about the designing of these products?

Prerna 20:41

Yeah, I think that you will definitely see a shift in terms of how ml products are perceived by consumers in the market. You may have heard of this concept of Uncanny Valley. So a lot of people interacting with AI ML Assistants, especially those that are either androids or humanized humanoids in some form, actually have the sense of, you know, being completely dissonant with this interaction. So there's a sense of cognitive dissonance that is called or define this uncanny valley. So in Japan, we there's a problem statement of an aging population that they're trying to solve for, with the assistants that are humanoids. I think the problem could be solved for through that approach. But then people also feel like, the interactions that they're having a knot with real human, and they think a lot about, you know, do I really want this kind of an experience? And am I comfortable with it? Or does it scare me in some way, and that prevents them from necessarily adopting, right. And I think that that is one perspective that comes into the design of your products. So the emotional connection that the customer has with the product becomes super important. Whether you're building a smart assistant, or you're building something on a retail website, or you're building a car, there is definitely the sense and awareness between AI and ML PMS, that there is a component of ethical design, and there is definitely a component of emotional design and empathy for your customer. That is that is unique, because you do need to think about the fears they might have from adopting an AI solution.

JJ 22:42

Oh, wow. Yeah, I that's, that's fascinating. And I can, I can definitely see how that could be an interesting, you know, byproduct, if you will, of the use of these products. That's that's a good example of fascinating, and probably not always easy to, to predict or ascertain without some real, you know, customer validation of at least some some fairly, you know, real prototypes, if you will. That's fascinating. I have a question on the development and the ongoing improvement of these products. So of course, software products, for example. I mean, we iterate on those all of the time, and we iterate on them before we release them. And then after release, we're constantly iterating and improving and optimizing. Can you iterate on these products like you can other types of software products?

Prerna 23:36

Absolutely, absolutely. I think iteration is the name of the game. This this idea that I was referring to earlier of having feedback loops is so important in machine learning, the data and the nature of the business continues to shift. And customers needs continue to shift what they want out of the products that they use on a daily basis keeps changing, which means that your models may have been trained on certain set of data and be optimized for a certain goal. But then that could be completely different three months down the line. So iteration is super important. And there's really two aspects to it. One aspect, of course, is the data drift. And the consumer needs shifting and the modelers need to be aware of how frequently their customers needs shift. And there's there's actually way to predict that and then design for it as well. Right. So make sure that the model continues to be upgraded to meet customer requirements. And there's a second aspect of the the engineering upgrades that need to be made on a consistent basis to ml infrastructure is relatively new to a lot of companies today. So ensuring that you accelerate the lifecycle of launching new models through the process. Since a data collection, model hosting model, refresh feature engineering, monitoring metrics online and actually testing a be testing different models, that is another unique aspect, which perhaps from an infrastructure standpoint, has been solved for in other parts of tech. But it's definitely sort of a newer problem that now organizations are beginning to solve at their level. But of course, you know, bigger org, such as Google, Facebook, Amazon, have ways to solve for it from an overall b2b product standpoint as well.

JJ 25:39

So I'm going to take us into the weeds a bit here, I want to talk to you about root cause analysis in Oh, AI Mo. Yeah, great. This is gonna be fun. And I'm definitely going to learn a lot here. So first of all, just give us an overview of root cause analysis in this space and why it's so important.

Prerna 26:01

Sure, so there's this concept of explainability in machine learning, where I'll give you a simple example, GG. So let's say that you're building an algorithm to predict whether a customer can pay for a certain transaction or not. So what would be the factors you'd consider? Actually, this could be a good, good way to kind of brainstorm here together. So what comes to mind as you think about you building a product in payments, where you're trying to predict whether your customers will pay for something?

JJ 26:34

Well, I would think about the their account, I would think about their credit limit, if you will, I would think about, you know, some of that kind of demographics, if you will, of the customer.

Prerna 26:49

Okay, so these, to me, when you when you see these specific things, they would be rules, it would be part of your data set where you have perhaps clearly defined rules. So perhaps you'd have information on how many times did the make successful purchase in the past, which is what you were referring to in terms of accounts, you may have a consideration around whether the have been reaping the the credit limit on a credit card on a consistent basis. So those would be your factors. And they would be all a set of rules, right? So MasterCard, or visa, banks and network processors in the space will definitely look at that set of rules and come up with some hypotheses. That is an explainable model, where there's a clear set of rules, and you know exactly what decision will be taken. Based on that rules, you have that equation ready to go. with machine learning what's unique is you may not have a completely explainable model, where because you have hundreds of features, sometimes 1000s of features in a single machine learning model, the equation for how you make decisions, inherently becomes more complex. So now imagining, imagine taking, you know, problem statement where you're trying to root cause something with three features, which we were talking about earlier, not boiling that up, or scaling that up rather, to 1000 features. That's really that will really give you a sense of why root cause analysis is important and and also complex in EMS.

JJ 28:29

Wow, yes. That's amazing.

Prerna 28:33

And,yeah, and I think they're definitely different ways to go about RCW in aim, but typically, you would break it down by looking at your market factors and your internal factors. So to give you an example, one of the problems that I worked on in the past is on recruitment. So a company that is trying to ensure, as a consulting firm, they give the best candidates to the firms that that hire them in terms of getting candidates and hiring accountants, right. So I was working within with this firm, and they gave me a data set of 20,000 user resumes and profiles, and they said, Well, can you help us pick out the best candidate from this mix? And we built a model around it and we produced a solution and provided that to the customer. But we had this persistent issue where the customer kept seeing you know, certain subset of resumes from you know, certain schools and with certain certifications, but when those people actually interviewed at the at the firm, they didn't they weren't found to be necessarily good candidates. So we were working with these recruiters and they were the ones who were training our data set. And they would tell us well, you know, when I talk to a candidate, here are the exact features I look at, I look at, you know, whether they have the right qualifications, whether they're certified, how many years of experience do they have? How much time have they spent working for big tech firms are big, big accounting firms. And that's how I think about it. So that's how it should work with the machine learning model. But that is not the case. So then we then start breaking down the problem. First thing we did is to kind of see whether there was some market forces impacting how we should think about the problem. So as an example, in that scenario, what was happening is, we had a growing economy. And there were definitely a plethora of jobs and a plethora of educational opportunities. So a lot of companies were offering educational stipends. And a lot of people who may not have necessarily been qualified, started taking these certifications that were applicable. And one of the requirements for sort of VP level or C suite level roles at the accounting firms that were our customers. And so we found that, you know, a lot of candidates that were not necessarily good fit, would fall into that bucket. So that was an external factor. From an internal factor standpoint, we also then realized that we needed a larger data set to really hone in on how to identify the best candidates. And then we start started thinking about, well, if we don't have enough data right now, how can we source it? What are the factors we could look at? Could we dive deeper on the existing data set that we have by conducting some usability tests and doing interviews? And then finally, you know, working with the modelers to really brainstorm and think deeper is how we approached it in that case, right? So you will see there's a sort of a pattern here of looking at the market factors and talking to your people on the ground, who really understand the problem space, talking to the modelers, and really identifying what's going on inside the model and understanding it as a leader. And then also looking at some other internal factors that could potentially help you get to a faster solution.

JJ 32:25

So tell me a little bit more about some of those other factors that you just mentioned, there may be some other factors, any any examples of some other factors that may be impacting your model?

Prerna 32:35

Absolutely, thank you for diving deeper there. So I was talking to you about some external factors from that specific example. I think that as I scale this up to other organizations that I've worked at, in the AI ml space, I do see patterns from other from other parameters as well, right. So there's definitely this component of privacy and data handling that keeps coming up, regardless of the machine learning problem that you're trying to solve. From both a regulatory regulation and regulatory standpoint. But also from a customer standpoint, if there is a model that is in some way, consuming your data, and then making certain decisions based on it, customers want to know. So building that awareness and creating comfort around you solving their problems, and then showing them that you're really using your data, their data to solve for specific problems and not sharing it with third parties or misusing it in some way becomes super important. And you as the pm are really owning the communication of that and making sure that everyone has a sense of comfort around what you're doing. So that would be the first a second, I'd say is there's this unique aspect of competition, we're now in the AI ml race. There's there's no doubt about it. And I will be I won't be the first to say this. But I agree with the sentiment that AI will eat software, right? So there's a huge sense of competition between AI ml companies, and even between the models that we build. So as you start to root cause you also need to think more deeply about, well, the solution that I'm proposing, will this get me to my goal? And will this get me to my goal faster than, you know, potential other solutions? And will it help me achieve my goal in a consistent way so that I can compete in this in this marketplace as well right and establish a really successful product? And then finally, there's there's definitely factors that you'd need to consider as you're actually developing the solution in terms of how many team members that you have to work with, what expertise do they offer, are they the right set of folks that you brainstorming with. And then how fast can you actually iterate and make the right trade offs and launch a root cause sort of an effective solution for what you use root cause? Right? So I think there's a lot of similarities, to be honest with any other technical pm role. But I've definitely found, these are some unique patterns that keep coming up in the AI ml space.

JJ 35:25

So I have a question here about customer journey or user journey, and, and really even that kind of tool or exercise of customer journey mapping. So I'm just curious, you know, if you think of a typical software product, for example, and using the customer journey map, as an exercise, and you kind of walk through the entire, you know, the entire tire path of a customer through the product, how they experience it, how they engage with it, you know, all of the actions they take and the touch points. Is this something that product managers in a IML? find valuable? Do they use that same kind of tool or exercise? And the reason I ask is because so much of the power of the product is behind the scenes and not transparent to the customer. But I can imagine it impacting the experience. So just curious on your thoughts on customer journey mapping and how that might be used in this type of product?

Prerna 36:25

Yeah, absolutely. I think that is a great question, right? Because we've spent quite a bit of time talking about the backend nature of a IML products and the fact that you're really building things behind the scenes, but what does the customer really see at the end of the day? So I'd like to walk you through a brief example, taking the example of a rideshare apps. So the persona I have in mind is a customer that is familiar with Rideshare apps, and is trying yours for the first time. What is the journey that you want them to have? Where can AI ml help? And then how would you decide, you know, what touch points to add aim and add versus leave out? And then how do you make sure that if there are certain things that you're optimizing for, from metric standpoint, that AI ml can help you achieve it. So I think that would be a good example to take up. So let's say that this customer is beginning account setup on your website, one potential thing that you could do here is to use autofill. So you see that within your iPhone or your Android phone, there is an option for autofill, where your previous information is saved in some way and then gets filled in based on the use case, right. So it actually analyzes the whole form, sees what are the fields sees if it has a similar mapping, and then fills them out to reduce friction reduced time, during the account signup fees. Similarly, within the rideshare app, once you sign up, and you enter your destination address, it is pulling in a lot of data points and making forecasts to decide where the closest driver is, how far are they from your current location? And then how far can they come? You know, how quickly can they come pick you up and then drop you off and then providing you with actual visuals of that time, right? So you really can make an effective decision as a consumer. And then finally, there would be some sort of a learning curve as well or learning process that the next time you make a similar request within let's say, Uber or Lyft application, it wants to ensure it gives you the best possible experience. So it would take multiple factors into account. So for example, if you have concerns regarding safety, if you have certain requirements, how does it factor that in into the entire feature set as it makes the right forecast and right prediction, to give you you know, the power to make a decision as a customer, right? So you will see that the entire journey here in some way, there's a lot of opportunities to use AI ml across the board. And then as you do that, you also realize that there's you know, some potential risk that you're taking, although there's a huge upside, there is some potential risks that you're taking by introducing AI ml. And so you really make those like trade off decisions at each touchpoint within the customer journey and then decide, do I want machine learning here? Can I solve for it using rules? You know, what's the best way to to meet the needs of my customers? So I would agree 100% that empathy mapping is super important. Even if you're building something for an internal customer, I think as a pm you are a business owner and can definitely not shy away from that.

JJ 39:54

Yeah, I love that example and those insights I I agree and I can totally see how that how that would work together. So that's great. Thank you. So so if somebody, let's take it a little bit higher level now. But if you if if somebody wanted to get into product management within a IML, and and you know, not not just, you know, any product management, but specifically AI, ml product management, are there certain skills and competencies that they need to think about building? How would you advise someone who may want to get into product management in this domain,

Prerna 40:32

I think an inherent curiosity is necessary. We all are, in some ways new to AI ml unless, you know, gotten that technical background and have taken the courses in undergrad or in grad school, a lot of us are definitely new to the space, it's a new and evolving domain. And there is definitely a learning curve, even for other technical PMS who are coming from non AI ml domains, there is a learning curve. So without that inherent curiosity, you would really suffer in the role in additional love for data, I think, quite quite obvious, but important to be said that you really need to demonstrate and continue to have a love for understanding data, deriving insights, looking at it in unique ways slicing and dicing it. So that goes unsaid. And then finally, I think the obsession for the customer cannot really run away from that as a as a PM, and that continues to hold in the AI ml space. So in terms of, you know, tangible or perhaps so hard skills, that you could look to develop our familiarity with machine learning familiarity with the metrics that you would need to populate or target and developing closer relationships and empathy for your scientists. In addition to that is another, you know, I would actually call it a hard skill that would be really helpful to develop.

JJ 42:06

That's all really great advice. So what resources do you find valuable as you continue to learn and teach others I know, you teach others a lot of have this, this insight and this knowledge, so what resources do you find valuable about the area?

Prerna 42:19

Yeah, to be honest, the people around me are the my are my best teachers, they're my best resources. But if one were to think of it in terms of books and courses, or even links, I would highly recommend some of the courses from Andrew Inc, who's a Stanford professor, I would recommend taking a Coursera course if there's a specific domain you're interested in. So for example, I developed a love for learning about healthcare and genomics. So I really started diving deeper into AI ml in that space. And there's actually courses for that. So that's really incredible. I would recommend doing that. It shows your innate curiosity. And even if you don't shine in the course, at least, you've acquired a lot of hard knowledge that is differentiated, right. So that will definitely allow you to shine as an AI ml PM. Second thing I would say is to actually kind of talk to if you have scientists on your team or within your organization, talk to them understand what they do on a day in and day out basis, actually shadow them in their work. And that will also teach you a lot. If you're inherently curious person and you ask them a lot of questions, you will realize that they're usually happy to answer them firstly, and secondly, that you learn a lot in a very short amount of time that you will definitely not learn just kind of doing the product thing and not thinking about it from from from the point of view of the people who are developing the solution. So I'd say both, if you can do both to both if you have to do just one, I would say talk to your scientists talk to the people who are developing the solution, ask them questions that will really allow you to shine and excel as an AMPM.

JJ 44:15

Very helpful advice. I've loved this conversation. I've loved having the opportunity to dig a little deeper into this fascinating space protocol, Senior Technical Product Manager at Amazon. Thank you so much for joining me again, for sharing your wisdom and your deep expertise with the audience. I greatly appreciate it. Thanks for joining me. Thank you. Thank you, JJ. And thank you all for joining us on product voices. Hope to see you on the next episode.

Outro 44:43

Thank you for listening to product voices hosted by JJ Rorie. To find more information on our guests resources discussed during the episode or to submit a question for our q&a episodes, visit the show's website product voices.com And be sure to subscribe to the podcast

Resources

Connect with Prerna

Ask a Question

Deep Dive into AI / ML Product Management

Recent Posts