- JJ Rorie
How to Know The Right Next Step When Applying Best Practices
Updated: Jul 28, 2022
Episode 026
Holly Hester-Reilly: "So the first thing that I ask is, do they already deliver software in, you know, agile iterations, whether that's at the end of every two weeks, you know, each sprint, or whether that's actually multiple times a day, you know, very continuous delivery. But I always start with that, like, Are you already at least delivering software might not be the right software, but it's software, and it's getting shipped? And so that's step one, really is just that basic, you know, shift software iteratively. And if they're not yet doing that, I advise they find an Agile coach, and and start with that. But let's assume that, that you've already got that base level down."
Â
Episode Transcript
SUMMARY KEYWORDS
product, people, build, research, organization, question, team, startup, shutterstock, management, discovery, goals, business, markers, validated, improve, resources, software, based, empowered
SPEAKERS
Holly, Intro, JJ
Intro 00:03
Welcome to Product voices, a podcast where we share valuable insights and useful resources to help us all be great in product management. Visit the show's website to access the resources discussed on the show, find more information on our fabulous guests, or to submit your product management question to be answered on our special q&a episodes. That's all at product voices.com. And be sure to subscribe to the podcast on your favorite platform. Now, here's our host, JJ Rorie, CEO of great product management.
JJ 00:37
Hello, and welcome to Product Voices. You know, organizations and product teams are always striving to improve to optimize their operations or their structure, their workflows, the way the teams work, whatever it may be. There are myriad best practices that can be deployed that can improve upon our work. But sometimes it's quite difficult to know, what is the best next change to make? What is that thing that if we improve upon this, it will make the biggest impact in our organization and our product team. That's not always easy to pinpoint.
JJ 01:10
So today's conversation is all about that, how to know what the right next step is when applying best practices when trying to improve within your organization. I have an amazingly special guest today I'm so excited to have Holly Hester-Riley with me. Holly is the founder and CEO of H2R Product Science. She has been a product leader at Media Math and at Shutterstock and is also a former Columbia University Research Scientist. She's also a former competitive figure skater, which is so cool. And I could spend the whole podcast just talking about that. But she also and one of my favorite things that she does, hosts a very popular podcast called Product Science. Holly, thank you so much for joining me.
Holly 01:54
Thank you so much for having me. And yes, I could totally talk about figure skating the whole time, too.
JJ 01:59
Okay, so we'll do that next episode. We'll talk about this, you know, boring stuff this time, and then we'll talk about figure skating next time.
Holly 02:06
Alright, sounds like a plan.
JJ 02:07
I can't even ice skate. So yeah, I mean, I won't be much of a much of a conversation partner with you on that. But okay, so let's, let's focus on product management for now. Tell me a little bit more about your journey I obviously gave gave a quick bio of you. But tell me about your journey in product management and how you ultimately landed where you are today.
Holly 02:29
Sure. So my journey in product management really began in I want to say 2007, when my, my then husband, founded a startup. And I supported him and founding that startup at the time, I was a chemical engineering researcher at Columbia. And I sort of just said, Hey, you know, I really want to help like, I've got all these organizational skills, I've got this like, analytical mind, I'm good at working with people. And like, there's got to be things I can do. And, and so he was very happy to have my help. And I got access to Safari, online bookshelf, and I started just reading all the books on software and product development that I could, that I could fit. And, and that was where I really came across this concept of product management was just through reading, you know, in the late 2000s. And I, I really enjoyed the mix of analytical and social skill that's needed to be a great product manager and sort of started on that path. In the beginning, it was the startup. We were I'm almost embarrassed to say, but I would say I'm recovering embarrassed from the fact that we ran that startup for five years and never had any, like, we had some users, but we never had any paying customers. And in retrospect, like how did it take us five years to shut it down? You know, but it's not actually that. It's more like it took us five years to launch. And, you know, these days, I know much better than to ever do that. But, but I learned a lot of lessons. And then I went to another small startup and that one was bootstrapped. And by that point in time, I had fully transitioned my career from chemical engineering to software.
And and after the second startup is when I went over to MediaMath. And my journey really sort of came into came into a place where it was about doing product management in hive. about startups, and helping them scale. And I really enjoyed adding that extra layer of tools to my toolbox of, you know, how do we do this in a company bigger than 10 people. And that was, that was a great ride. And so I did that for about three years. And then I went to Shutterstock. And one of the cool things that I got to do at Shutterstock was, when I got there, my boss asked me to take a look at a failing business unit and assess whether or not there was a there there. And so I did like a deep dive product discovery. And, you know, talk to both sides of the marketplace, talk to, you know, the people who have been working in the company, talk to people inside Shutterstock, that were not in the internal business unit, and came back and said, You know what, there's opportunity in the market for this business. But we are in no way positioned to capture it, we've made too many wrong decisions along the way. And we would need to scrap it and start from scratch. And I don't think that makes sense for our business. And it was the first time that somebody had managed to actually convince the CEO to shut down this internal startup that had been one of his babies. And so that was a really great experience, went from there to launching Shutterstock editor and Shutterstock editor is a tool for non designers to create professional looking designs that's offered by Shutterstock. And I took that team from beta, or from prototype to public beta, grew the team from about five people to about 40 people, and worked with them for a year. And at one year, we were at 10x, the product usage from where we started. And I basically just like to say like, I applied all the principles that I learned from the leaders in product management, you know, people like Marty Kagan and Teresa Torres. And they work. And so you know, ended up with a product that worked. And from there, I founded my company, and here I am.
JJ 07:01
That's awesome. What a road right? From, from, you know, start up with your spouse to, you know, growing, growing Shutterstock that's amazing. So and by the way, I mean, I don't know anything about the the ex and the relationship there, but at least you got product management out of it, right?
Holly 07:21
Totally. Yeah. No, at least I got this whole awesome career in tech. And exactly, you know, he, we were still together up until two years ago, and he supported me starting my own business. And I know, I got some good things out of that.
JJ 07:36
Yeah, I love it. Fabulous. Okay. So you've worked with lots of companies in your business. And obviously, you've got some experience just back in the corporate world, if you will. And you've helped them optimize their environment. So So tell me how you advise your clients and organizations to know where they are? And how to, let's start with how to take the first step. And then we'll move on to kind of how to take that next best step, how do you advise folks to understand where they are and where to start when they're trying to optimize or transform themselves? A little bit?
Holly 08:10
Yeah, great question. So the first thing that I ask is, do they already deliver software in, you know, agile iterations, whether that's at the end of every two weeks, you know, each sprint, or whether that's actually multiple times a day, you know, very continuous delivery. But I always start with that, like, Are you already at least delivering software might not be the right software, but it's software, and it's getting shipped? And so that's step one, really is just that basic, you know, shift software iteratively. And if they're not yet doing that, I advise they find an Agile coach, and and start with that. But let's assume that, that you've already got that base level down.
Holly 08:52
The next thing that I ask people to think about is, are you validating ideas that you get from, you know, wherever your assignments come from? Are you going and validating those with the market? Are you talking to customers? Are you looking at data from existing products? So you're looking at data for the market as a whole? And are you using that at some point in the process to make decisions? And once again, that's a place where I find most people can say, to some degree that they're doing that, they can at least say yes, we sometimes do that. Whether they do it enough, totally different question, but they usually are at least doing some of that.
Holly 09:36
So okay, once once they're doing that, then the next question is, do you do that at least every sprint, and that's where things fall down for a lot of companies. They, you know, a lot of times, companies are practicing what I think of as project based discovery, which is like we're gonna do product discovery, when we start a new initiative, we're going to do product discovery when we decide to start a new business unit, or launch an entirely new product, but a lot of companies don't regularly do product discovery throughout the process, you know, for the enhancements for the ongoing work. And for companies that are at that stage, I, I call those product discovery practitioners. And I say that the first step that they should take, is they should change the format of their demo. So hopefully, they're doing sprint demos, I've certainly worked with plenty of companies that are not so if you're not doing sprint demos, you have to add those. But the first step is to update that process. So that instead of just being a, what did we build? It's what did we build? What did we learn? And what are we planning, and you want to cover all three of those things in the demo. And what that does is it gives stakeholders exposure to the fact that you're changing. You're changing your plans based on what you learn that you're not just saying, Well, I committed to this roadmap, and therefore I'm going to execute this roadmap for the next six months or the next year. But rather that you're committing to outcomes, and you're learning along the way, how best to achieve those outcomes. And every time you learn new things, there's a chance that you're going to change the solution that you're building, or that's gonna look different. And by bringing that into your sprint process, it's a forcing function that makes you go and do more continuous research, because you want to have something to talk about in the demo. And that is a really great place to start.
JJ 11:52
So I love that advice. Really important. So let's say an organization is, you know, to that point, or they're, they're at a point where they've, they're not starting from scratch, per se, but they need to know what is the next thing to do? What's the next area to improve? Next thing to deploy or implement that's going to help them improve, take some sort of incremental step forward. So are there triggers or markers that help identify what that next best step is for an organization?
Holly 12:25
There's markers, I wouldn't necessarily say triggers. But there are markers in terms of when I look at an organization, I look to see if certain principles are in practice. And the markers would be identifying whether those principles are in practice or not. The principles that I'm looking for are what I call the product science principles. And there are three of them.
Holly 12:51
The first one is evidence based product strategy. And for that one, what I'm looking for is to see whether there is a product strategy in the first place. And what I mean by product strategy is a plan for how you're going to achieve the business outcomes that ties together. What is the problem that we're trying to solve? Who are we trying to solve this problem for? What does a successful outcome look like for that user? And how do we think solving that problem is going to improve our business? So that's the first marker I'm looking for is like, does that kind of plan even exist?
Holly 13:37
And the second marker is, if that plan exists, is it based on evidence? Did it come from doing interviews? Did it come from looking at data? Or was it a me too, you know, was it we looked at our competitors, and they're building x? And so we think we should build next to? Or was it, you know, came out of the CEOs head? Without being validated? Or, you know, that sort of thing? So that's the first marker that I'm looking for is does that exist? The second marker is are they practicing continuous discovery and delivery? And so for that one, you know, like I was saying, it's about both the continuous delivery as well as continuous research and experimentation. And are they doing those things at least every sprint?
Holly 14:27
And then the third one is empowered teams, do they have the right organizational governance model for teams to be able to really build great products? And what that looks like to me is a leadership team that sets clear goals, sets, success metrics for how they're going to measure those goals, and provides all of the context that they can to the teams on the ground, and then the teams on the ground. Take that in Information and go and do the research, do the development, do the execution. And they've they make the decisions day to day based on what information they're getting from the leadership. And then that way, they are empowered to actually make changes to their plans and to make changes based on what they learned throughout the process. And so I look for those three markers. And and I always start with continuous discovery and delivery. So if they're not practicing continuous discovery and delivery, yet, I start with solving that problem. If they are practicing that, but they're not practicing evidence based product strategy, then we'll solve that next. And then finally, we'll solve for empowered teams third.
JJ 15:49
Got it. And do you see any common roadblocks, if you will, or stumbling blocks in that that first one that you you look to identify continuous discovery and delivery? Anything common that you see that makes it hard for folks to embrace that and to employ that in the in the organization?
Holly 16:11
Yeah, the most common one is not having buy in from people that it's necessary. You know, I hear all the time people saying, well, Holly, the the research that you've taught me how to do sounds fantastic, but my boss just wants me to build what they asked for. And I think that's, that's the most common roadblock by far. And what I recommend for that is that you start small. So you can start with, for example, when I was working at Shutterstock, we started with a one question intercept quiz, when we were doing Shutterstock editor, where we wanted to know, what kinds of digital materials people were making with their stock photography. And we decided to start by putting a question on the Shutterstock website, whenever a person was about to download an image, it would ask them, What are you going to use this for, and then it would have multiple choice of like, social media marketing, restaurant menus, print, marketing, you know, things like that. And that was really easy for us to get live, even if even with all the pressure that there was on the team to just develop and work towards the solution that the CEO was envisioning. We were able to get that one question intercept quiz live, and then come back and share those results with the key stakeholders and be able to show them, there's something here that you weren't expecting. And so what actually happened in this case is we were surprised at how high the amount of people downloading an image for social media use was, as I recall, it was somewhere like 45% of the images downloaded were going to be used for social media. And that insight, sort of gave us an in for, hey, maybe we're making assumptions that we shouldn't be making, maybe we need to go and actually assess what's really happening. And, and that was the beginning of, you know, getting the space for the research. There's a couple other ways, you know, in addition to the one question intercept quiz, I also often recommend to people do an informal user interview, you know, maybe you don't have the budget for recruiting interviews, interviewees, and you need permission for that, and maybe you don't have access to the clients because, you know, maybe sales is telling you, you can't or maybe you're in a regulated industry, and it's difficult. Find a local, like, find a find a person who looks like your customers, but isn't your customer and therefore, you don't have those legal barriers or, you know, find a friend or a person in your network that you can just reach out to on LinkedIn and say, Can I have a conversation with you, and start that way. And then another way that I recommend people start small is to gather data from existing sources. So if you're, you know, building something new inside an existing company, or you're adding a feature to an existing product, there's probably data that you can analyze, and you can learn some things from and then you can put forth those learnings. And in all those cases, you're going to bring that data to your stakeholders. And whenever you do, show them something that they weren't expecting, that's going to increase their willingness to spend some time on research.
JJ 19:46
I'm curious, do you see these tips, these kind of small steps that you can take to get started? Do you see those also applying if someone sees that their organization has as a product strategy, but it's not evidence but evidence based. In other words, it did come from the CEOs head or it did come from, you know, just some assumptions being made. Do you? Do you see there are either those that you just mentioned or other tips for someone to, to combat that, because that's one of the hardest things I've seen is that if we've got this product strategy put in place, especially by leadership, and we believe it's, it's just based on assumptions and not and it hasn't been validated, to kind of chip away at that. And and do you have any advice for that? Like, how do people combat that if they've, there's a strategy in place, but they're not convinced that it's actually built on? Real strong evidence?
Holly 20:45
Yeah. So you're absolutely right, I do, I do recommend they do these things where you can start small. There's another element to it, though, if there's a strategy in place, but it's not based on evidence, that means that there is someone in the organization, you know, likely in senior leadership, who has a strong faith in their own ideas. And so one of the things that I tell product managers, if they're in that situation, is, you can't, you can't really, very successfully go to a person like that and say, Hey, I'm not going to start on the work you think we need to do until I do this research? Let me go and do this research first. So I always recommend that you Yes. And that you say, Okay, you want me to build, you know, XYZ? Yes. And I'm going to go do some design research to help me figure out the nuances of how to build it. And in the beginning, with those stakeholders who think they know what they're want built, I will actually really sort of emphasize that is design research, because it gives us a less threatening way to get ourselves into the research mode. And I'll then start incorporating strategy research alongside that design research. So as an example, for that, what I mean is like, you know, maybe I usually find some piece of the product that they want me to build that I think is a pretty reliable piece of it. So for example, maybe we're building a brand new product, and we need to build login and logout. Like, you know, what, if we're going to operate on the assumption that the product is going to be built, we know, no matter what else features it has, that it's going to have a login and a logout, and we need to build that. So okay, great, I'm gonna go ahead and get my engineers started on that base feature that is, you know, pretty sure that you're going to need, and then I'm going to do some design research on what is the best way to do this. And while I'm doing that design research, I'm going to use the same interview sessions, the same, you know, usability tests, to start to ask a few questions that helped me analyze the strategy as well. And that way, I'm able to tell the stakeholder that I am moving forward on their plans. And here's what it looks like. And we're starting with this or, you know, and give them that confidence that I'm listening to them, while also starting to build up a space for research.
JJ 23:20
So let's say that an organization is practicing continuous discovery and delivery, they validated that their product strategy is evidence based, they've gone through all of the steps needed to ensure both of those are active in the organization, and are happening. So now they're ready to say, Okay, let's look at empower teams. How do they start that? Where would you advise them to look at either either kind of sub markers to look for that or other areas to where they can move into more of an empowered team environment?
Holly 23:56
Yeah, great question. So the first place for them to start is to try to assess the current state. So I generally look for a couple of things, I look to see if there is a performance goal setting system, whether that's OKRs, or KPIs or, you know, a different form of setting goals. I'm less concerned with the form, but there needs to be some communication of these are the goals. These are the goals of the company level, these are the goals at the business unit level. These are the goals at the team level. And so the first step is like to those exists, are they known to the people on the team know where to find them? And are they accessible outside of individual teams? So do people on teams know how to find other teams goals? So that's one of the markers that I'm looking for. And then the next marker is. Okay, so let's say they've got goals. The next question is Will, how do people feel about those goals? How are they set? And how often are they updated? does? Does an individual on the team feel like they have the space to negotiate a goal and say, you know, Hey, boss, I don't think that, you know, this goal that you've set is realistic. And here's the reason why can we update it? Is there that security and safety for the team to be able to do that? And then, and then, finally, I also want to assess whether people feel like they have the space to make their own decisions. So are they are those goals actually being used to manage? Or are those goals sort of a, you know, a thing that's built to say we're doing goal based management, but then actually, the bosses are saying, you know, build this screen with this feature. And if that's the case, then the goals are kind of meaningless. Because they're not, they're not being used well. So, so if you know, those things aren't all in place, the place that I usually start is building out that goal system. And with most of my clients, we do OKRs. So that's objectives and key results. And we typically set OKRs at the leadership level, and then sort of the director level and the team level. And we want to make sure those are in a public place that they're accessible, but they're negotiated that they're iterated on. And then we want to make sure from there, that they make their way into a common practice. So for example, does the demo start with the team, just stating their objective every two weeks like and you know, it might sound funny, because you're like, well, it's that it changed, but, but it is, it is forcing the team to remember that that's their ultimate goal, and everything should be measured against that.
JJ 27:21
So final question for you. What resources have you used over your career, as you've learned, and obviously, you you mentioned early in the conversation that you did a lot of self self educating and read a lot of books and that sort of thing. In addition to your own resources, which by the way, listeners, we're going to link to that H2Rproductscience.com, you can learn all about the product science principles - evidence based, product strategy, continuous discovery and delivery, empower teams - you can learn more about Holly's company and those principles there. You'll find that on our website, productvoices.com. And you can also find it at H2Rproduct science.com. So in addition to your own resources, and being a resource for product management, as you are, how do you learn who do you like to follow? What books do you like? What podcasts in addition to yours? Any other kinds of learning? materials do you like? What would you share with with folks out there?
Holly 28:23
Yeah, so I mentioned Marty Cagan and Teresa Torres. They're still staples for me, I pretty much read every, you know, newsletter that they send out. So though, Marty Cagan is that Silicon Valley Product Group, and Teresa Torres runs ProductTalk.org. And so those two are definitely really great resources. I also like to send people to some of them Melissa Perri's resources, like, if someone's early on in their product journey, I recommend the Product Institute, which is online course. And then I like. Also, this one's a little bit outside of your standard, like growth startup, product management, but I also really like Barry O'Reilly, and his work on Unlearn and Lean Enterprises. And so I also recommend that and finally, I like to recommend Jeff Gotthelf and Josh Seiden's work as well.
JJ 29:27
All very amazing resources. And so again, we'll link to all of them. So, so you, you are listening can find those. I'm a big believer that we can all learn from each other. And so I always like to ask everyone who comes on the podcast, who they learn from and what resources they like, and we always link to those. So those are amazing resources. Thank you very much. And again, you yourself are a tremendous resource. So we'll link to that as well. I've loved following you over the years and learning from you as well. So, Holly, this has been a great conversation. Thank you so much for sharing your wisdom and teaching us a little bit about how we can improve every day.
Holly 30:07
Well, thank you so much, JJ. It's been such a pleasure.
JJ 30:10
And thank you all for joining us on product Voices. Hope to see you on the next episode.
Intro 30:16
Thank you for listening to Product Voices hosted by JJ Rorie. To find more information on our guests resources discussed during the episode or to submit a question for our Q&A episodes, visit the show's website productvoices.com And be sure to subscribe to the podcast on your favorite platform.
Resources
Connect with Holly
Ask a Question