Should metrics be on your roadmap? | Pamela Schure
In episode 29 of Talking Roadmaps, Pamela Schure and Justin Woods discuss the crucial role of metrics in product roadmaps. Drawing from her extensive experience, Pamela emphasizes the importance of integrating analytical foundations into roadmap planning and decision-making. She highlights practical techniques for balancing qualitative insights with quantitative data, ensuring product managers can effectively measure success and drive continuous improvement.
Pamela Schure has brought over 50 diverse products and services to market for companies like Apple and is co-author of Product Management for Dummies. She has trained several hundred Product Managers and now consults for large firms bringing Product thinking techniques to Digital Transformations. She combines her BS in Applied Mechanics from the University of California at San Diego and an MBA from Columbia Business School to bring technical insight into solving business problems with analytical foundations. She has extensive global business expertise on three continents in industries as diverse as city transit, vehicles, sawmilling, property management, and training services.
Have a watch and if you enjoy the video don’t forget to subscribe to the channel, like it and maybe sign up to our mailing list!
Here is an audio-only version if that’s your preferred medium - and you can access it through your favourite podcasting platform if you prefer (Apple, Spotify, Amazon).
In the next episode we are talking to Hannah Chaplin, former Principle Product Marketing at Pendo and CEO of Receptive. So watch out for Season 1 - Episode 30!
-
- Hello and welcome to the Talking Roadmaps Channel, the channel where we talk about everything to do with roadmaps from the good, the bad, and the ugly. I'd love to introduce today's guest, Pamela Schure. Pamela, welcome.
- Thank you so much, Justin. It's great to be here.
- We great to have you. I've read through your LinkedIn profile, been really impressed with the experiences that you've have, but maybe for people that aren't aware of you, maybe you could just provide a brief introduction.
- Yes, so I was a product manager for over 20 years, and I've launched, delivered over 50 products, which sort of adds up over time. Yes, and then for the last eight to 10 years, I've been actually training product managers, which gives you a different perspective because you get this huge range of industries all showing up on your doorstep. And finally, now I'm as part of this process, I actually ended up co-authoring "Product Management For Dummies." And for those of you who've never written a book, the interesting bit is you actually have to rethink everything you thought was true about product management so that you can actually write it down coherently. So that was a learning experience on my part.
- I bet, and kind of clarifies your own learning as well. What a great claim to fame and 50 products.
- I know, I know. Every once in a while I do the math and I'm like, "Oof, that's a lot, that's a lot." But you learn a lot.
- Thank you for that great introduction. If you're new to the channel and you've not seen us before, please do engage with us below with comments. If you find value in the conversation that we're having, do please give us a like, and maybe consider subscribing if you find value. So why don't we get into the kind of the meat of this then, which is talking a bit about roadmaps, is it an art, is it a craft? I think we've both seen instances where it's been done well and probably many, many more where it's been done badly. But maybe from looking at it from a top level, what do you think is the, the purpose of a roadmap?
- For me, the roadmap really tells the story that you can't really talk about your product over time without using something like a roadmap. And you as a product manager or whatever your title is, you get to decide what's in and what's out. But fundamentally, if it's not telling your story, then you don't have much, you just have some data on a page, right? I think the other beautiful bit about a roadmap is that it goes in at different levels. So you can have a very high level roadmap, you can have a medium level roadmap, and then you can have like these are the details of what we're building kind of roadmap. That's my view on what is a roadmap and how I've personally used it over the years.
- One of the key takeaways for me there, maybe with the audience as well, is that you mentioned the roadmap at different levels, and I think some people think that the roadmap is a single page, single document type thing. And actually no, no, you know, it's a communication vehicle and there's different audiences and it does need to be different levels as well.
- Yes, I know that there have been times when I put roadmaps up for business planning, for example, and the data that's on that roadmap is vastly different from the data that I'm sharing with my developers.
- You mentioned a business roadmap or things that you share with your developers, in your perspective or your experience. Who have been the audiences of a roadmap or roadmaps in that instance?
- Well, actually I find for me, the developers are probably the least of my problems. They're usually aligned, right? In other words, it's a review. The people that I really need to talk to are the leaders of the company and usually the sales people. Like they're sitting in there going, what am I gonna be able to sell? When am I gonna be able to talk about it? What am I going to be able to say once I can talk about it? What holes does this fill? You know, if I get to time X, what am I gonna be able to do that I'm not able to do now?
- So it's kind of setting expectations as an element of chronology or at least sequence to it. We wanna be careful, we'll talk about that one a bit more.
- A roadmap by almost definition has the bottom, X-axis is time. And so the question is what's in there? I think over time, you know, as I've gone more into the product portfolio, if I actually think back on what's on the roadmap, a lot of it is the entire portfolio, the portfolio that we want to talk about, right? Or the changes to the portfolio. And I think that's part two of it, which is what is changing? A roadmap is should not be static.
- The amount of times I've seen a roadmap, you know, being created and then put on a shelf for a year and then we come back to it.
- No, these are living communication documents. I think the slowest ones I've worked on are maybe quarterly, you know, because that was the cadence of review, but really some of them are much more frequent than that.
- Talking about review, let's talk a little bit about maybe ownership of the roadmap. So who typically owns a roadmap in your perspective.
- If it's about a product or a product portfolio? I would say the top level one is usually the product manager, right? Or maybe if it's a series of products that are coming out from a product group, maybe their group product manager that would be the owner. When you get to lower levels, it'll be an individual product manager. And I think there are cases for roadmaps being owned by the technical people. I think they're also technical roadmaps and we have to give everybody their due. The question really becomes who's allowed to change it Justin.
- You raised a really good point, which is actually that, and one thing that I'm certainly passionate about is that there's roadmaps for the technical team. In fact, architects may well have their own roadmaps, but when you look wider, you know, my wife's a business systems admin and she works on business systems. So they have roadmaps for HR systems, for their CRM systems. And so I'm really passionate about everybody having a roadmap, maybe even a personal roadmap. So you're right that kind of that latitude there.
- You know, you've really inspired me. I think I've gotta come up with my personal roadmap. I might actually involve my husband in that discussion but-
- Just to see if there's alignment, there's certainly opportunities for OKRs within that as well. I mean...
- You did bring up a really good point about OKRs and I think, you know, a lot of times I was looking at the Aha! roadmaps and I know that you work in the area. I think there are times when people say, "Well, this is what I'm going to do." And I think the thing that's missing most of all from roadmaps and the thing you have to put in is always, what do I get? Right, what's the outcome? Like your OKRs will give you, what is the key result you are aiming for as a result of doing this activity? And I think that's the big takeaway over time. And this was the case even many, many years ago when I was putting on roadmaps for product line business planning, and I would just put the revenue by the product. It's like, this is what the outcome is as a result of doing this product. And so that made the whole process of prioritisation a little easier for everybody to get their hands around.
- I'm actually a real big advocate of both because I think it picks up on exactly what you said is that the objectives and the key results or maybe their goals or whatever people want to call them, it provides that quantifiable business success that we are looking after, the success metric that we're going after. But what I don't find it does, is it necessarily tells a story like the roadmap. So I like to almost use both of them. I think if we looked at a bunch of key results and you just said, right, "Pamela, I want you to achieve five of these for the year." It doesn't convey the story or the context so well.
- Well, I don't think it also conveys the sequencing. And so I think actually now that I'm getting more and more inspired, I think I'm gonna drop OKRs into roadmaps from here on in. Cause I think that might help people. But yes, I really do, you know, you need to have a certain sequence of activity, you need to have results that happen as a result of that activity. And roadmaps are a great way to tell that story. I think too many times though, what people do is they find themselves constrained, right? They're like, roadmap, I have to, I have to follow the corporate format, or I have to do this and what they neglect is it's not telling you what you need to do.
- And often burns us, you know, the amount of times I created a well-meaning roadmap at the beginning and it's been more of a delivery plan, which is often the go-to. And actually it's given roadmaps quite a bad reputation. And I've certainly felt that as well. And I think that's healthy.
- I'm just thinking back to roadmap I looked at for a startup, and they sort of had a sequence of things that they were going to build, but the real question was why were they building it? What was the target customer? And I think there's always, there's the core, like this is the sequence and then there's this bit that you need to put in, which is who is it, what's the impact, right? Like all that stuff needs to be added to it. Even if you use a tool like Aha!, you should be cutting and pasting that, you know, and then adding on the other bits, which may not actually be called on.
- No, I totally agree. We've spoken to other guests before about road mapping actually being more of a process and road mapping being more of a document, and being able to pull on these different resources. It shouldn't be a Gantt chart as a single page. It's very much that alignment tool, but it needs to pull on all of these other areas. It comes back to, I think where we left off, which was who maintains the roadmap? So there might be someone who owns it, but from your experience who's kind of been the maintainers of that roadmap? Is it still the person that owns it or maybe a wider audience?
- I don't know if I'm a control freak. I don't think I am. But I in general wanted to own it because I felt like, and maintain it because I felt like if I handed it off to somebody, there was a chance of disconnect. And so I would very much hold onto it. There are very few things which I believe product managers should hold near and dear to their hearts. And I think the roadmap is one of those, and you need to own that.
- It might be not even dedicated product managers, but people somewhere in an organisation, even if it's a very small one, someone who's performing that function. And so that should still be something that they own. Talking about the being more of a document and having vision strategy, and objectives, and things. From your experience, what has been the relationship of a roadmap to those artefacts?
- For me, a vision is so high up there, sometimes it feels like it's very disconnected from anything whatsoever. One of the areas I'm very interested right now, and it's because I've spent a lot of time over the last year working on OKRs, is how does an OKR relate to strategy? How does, 'cause there are different levels of OKRs, right? So I think they should, it should filter down, right? As part of the strategy and part of the initiatives, it should come down and it should kind of reflect what's going on, and your roadmap, you should be able to see traces within the roadmap of what has been agreed at the strategic level. Probably not, like I said, probably not at the vision level. That's probably a little too esoteric. But your strategy, your initiatives, I think are really important that they show up. If you have an OKR, if you've turned that into an OKR and a roadmap, right, then you should be able to again, double click on it and go down to the next level and to say, okay, well we have this OKR, now what is it that we're building that will satisfy this OKR? So again, this is this idea of layering pieces and layering roadmaps together to tell the complete story at the appropriate level of the organisation.
- You know, I definitely see it being that communications tool, but it fits and connects these other artefacts. The roadmap, you should be able to, you know, actually double click into it and be able to go and see that further detail. Maybe that's a delivery plan, it could be a project plan, it could be some metrics that we want to be able to achieve as part of that.
- In that roadmap, if you're smart, you'll put the metrics, you'll figure out a way to put your metrics there, right? Because you want people to kind of focus on what they're going to get as a result of making the investment in the change that's gonna happen as a result of a roadmap.
- But I think that's right and I think that also makes it easier to be able to make trade offs on the roadmap because it's more visual on what some of the impacts if we remove something, may well be.
- Yeah, and I think you touched on it earlier and I really wanted to kind of come back to this, which is your first roadmap is a strawman, right? I don't believe in final roadmaps upfront. It's kind of, you put it up there and everybody goes, "Yeah, I like this and this sucks," you know, or this needs to come in or this is missing, right? So it is also another way of achieving alignment and an input.
- It's got to talk the language of your stakeholders and who you're trying to appeal to. So, you know, one of the key things is just get started, right? And involve your roadmap with your stakeholders so that it shows what they want to see and you can show them what you want to show them on it.
- As a matter of fact, I think there was one company I worked with and we had this mandate, which is the roadmap needed to be updated monthly and it needed to be public within the company. And so people knew, I wanna figure out what's happening next, I can go and find that out. One other benefit of that method, it saves you time.
- Well there sometimes we've spoken to some people where sometimes especially implementing a new tool can be such a step change that the business aren't used to that I've spoken to some product managers that said, well actually we've put it in the location that they typically use just so it's front and centre, rather than getting to search for it or look at a roadmap that's set up differently, they've almost kind of converted it into a format that they're more used to seeing because the organisation has to adjust as well.
- That's true anytime you change a format. So that's an interesting point about roadmaps. People are very addicted to formats.
- You mentioned metrics on a roadmap and that really appeals to me. I like that a lot. What do you believe other than metrics, are some of the key elements or content that you would like to see on the roadmap?
- It really depends how the roadmap manifests itself, what it looks like, okay? I mean, if I can, if for example, a key element on the roadmap is I'm going to change the personas that we're addressing, then I think those should be called out in some way, shape, or form. And I'm not talking about a complete persona description right there, but it should be new persona X, right? You should add those things on if that helps you tell the story. I think there's another part, I'm just, what I'm thinking about this, which is keep it clean. Like I know sometimes we try and jam everything in and it's very difficult to tell, right? And so I like colour coding, which tells me which parts, you know, are going to be important for different things. But again, personas, metrics, changes in direction. So basically I was looking at the Aha! Roadmaps and up at the top I felt like there was a space where you could say, this is part of this initiative, this is part of this initiative. So people can keep it straight. Like why are you doing this, without having to ask you the question all the time.
- Again, I think it's part of that story, part of that narrative, but it's important to be on that. Because I think it's supposed to lead in. I feel certainly that those strategic elements, again, we said that vision is maybe too high, but I want to see on my roadmap that connection with strategy, but also leading off to how we might deliver and it provides that hand in and that handover without necessarily describing too much of it. Talking about kind of tools and some of the roadmaps that you've seen, what's your preferred way to visualise a roadmap or some style? So we talked about colours. I think a roadmap's got to be beautiful, but do you have a preferred way to visualise or style your roadmaps?
- Okay, I guess I'm really old fashioned here. I like PowerPoint or if you're my daughter would say Slides. She much prefer Slides over PowerPoint. But I think those kind of presentation tools are really good. They're very flexible and in fact they're basically good drawing tools and then you can go right to presentation mode. So yes, you can do other things, I go back to PowerPoint.
- I agree for many reasons, even though I'm a tooling implementer myself, I think nothing gives you the latitude to put on that roadmap Slide, whatever you want to show, whatever is meaningful to the business without the constraints of a tool. I think we all know that there can be challenges with keeping that up to date and lots of moving parts, but that's a different part of the challenge. But I think the roadmap in itself, you know, showing the corporate styling, showing the colours, showing it in a way that's meaningful. When I was at Vodafone, I think one team actually created their roadmap as a solar system, and their planets were their milestones. I was like, you know, there's immense latitude to be really creative with these things within the boundaries of obviously not making it overwhelming. But I think that's right. And I think if I ever had an ideal, I don't think any tool is really as flexible as to show the roadmap in the way that the company truly wants to see it. So you might get, use the tool for the source of your data, but maybe not for the presentation of your data.
- Agreed, again, I, while we were talking, I was really thinking about roadmap that a product manager might present and a roadmap that might come out of, you know, the development organisation. And I think those are two different things. And so if I'm part of a development organisation and I'm talking about what I am delivering, then I think the existing underlying tool might provide a perfectly adequate context for delivering a roadmap. I don't think the product managers live in that world, not when they're talking to their stakeholders that really care about what's going to come up and why it's gonna arrive, and when it's gonna arrive, right?
- What do you consider some of the best practises in road mapping? So we mentioned actually kind of like a monthly cadence to roadmap updates. Is that a best practise in your eyes and what else would be?
- In general, I have found that anywhere from one month to three month is about the right cadence for delivering information to the company, to outside stakeholders. Within the organisation, for example, if you have a development team, you're going to be updating it possibly more frequently because you're down at the delivery level, right? But I think one to three months for me is best practise. And it depends again, on the cadence of the product. And I'm actually thinking of someone I trained once, and I was asking her how long between product launches? And she said, "We change our products every 10 years." Obviously there their roadmap is not changing drastically. And so probably they dust one off and update the format, you know, once every five or six years to figure out what's going on. Again, another company, it was seeds and how long is your development cycle, it's 12 years. And even there they had roadmaps. So I developed a specific roadmap for them so that they could explain it to their outside stakeholders.
- It's something I've heard before, certainly in certain methodologies called the rhythm of the business. And I think businesses have a certain cadence, they have a certain rhythm and we need to choose whatever is appropriate for that.
- Sometimes, you know, you have people who come from a different industry or whatever and they wanna force their cadence on the organisation. It's like, you know, it's horses for courses, right? We work at this rhythm, we're going to deliver roadmaps appropriate to it, and by the way, things change. You're gonna change your roadmap pretty quickly. When covid hit, the roadmap changed drastically. The delivery schedule changed drastically too.
- It really did. And so I wonder if that leads into some of the biggest mistakes that you've maybe seen people make in road mapping.
- I joined this company and, who will remain nameless and I was handed this roadmap and my then boss said, "Follow this roadmap because the VP likes it." And I started using it and then I identified an issue in the product line and it had to do with technology, right? Their technology wasn't being followed through. It's kind of a detailed story. And so I developed a different roadmap which had to do with colouring and the technology transfers that we were going from A to B and in fact identified a hole in the product line. Like we just didn't have a product there and we should have, right? And it was very interesting because this gentleman who I was reporting to turned to me at one point, he turned to somebody else actually, and he said, oh, you need to do a roadmap. He goes, "Take Pam's." Right, like he knew that that actually told the story. And so we actually switched as a company. So that's one of those death by template kind of situations. A template, if it's not serving your purpose, ignore it. Make your own, take the heat because you're gonna benefit from it ultimately.
- Yeah, massively. And it's just, you know, it's, I've heard people refer to the roadmap as serving corporate history or being able to preserve corporate history. So you can see decisions that are being made. And I love that, and I think it serves also useful, it's a useful tool to provide continuity. Continuity in terms of when you're new to the company, this is where we are and this is where we're going. But I love the fact that, you know, we have to be realistic that, you know, it's not a delivery plan and if it's held, it's handed over to you as a delivery plan, and things change, then we need to be, feel comfortable to be able to challenge that roadmap. Otherwise the roadmap becomes the, you know, the rope that we use to hang ourselves with and that's not really where we need to be. So there's such a careful balance there I think.
- As a product manager, I have always thought of myself as a change agent. If I just do what came before then I failed. If I make a dramatic change or some kind of change, it should be for the better, and I should leave the organisation in a better shape.
- Yeah, I think that's one of my philosophies for life actually, is always to leave things in a better place than you've found them. What about, so that was one of the, kind of the biggest mistakes you've seen. Are there any antipatterns or bad practises that you regularly encounter in roadmaps, do you think?
- I don't think I'm a really big fan of the Gantt chart roadmaps, I'm not. I think that sometimes they hide more than they tell, right? And it makes everybody feel good. But I don't think it gets into the thinking behind the thinking, like why are we doing this? And so we constantly need to remind ourselves and go back to it and to say, what is it that we'll make the company change in a better way? Is this roadmap telling that story, right? One of your other guests pointed out that the outcomes are critical. I still go back to that. Are you getting the outcomes you want? Now, if you are and all you're doing is tracking progress towards a known outcome, then that's fine. But I think at some point you need to double check and say, did we get there? You and I both know, right? Like many, many companies, we've all written plans and we put things out there in the world and they didn't go to plan. And no one ever comes back and says, "You promised X and we got Y and there's a big delta." No one ever says that. And I think accountability is huge.
- Yeah, me too.
- Well I think a roadmap is also, opens a conversation and there are decision points within a roadmap. So I've been using something called Impact Map, but I hacked it, so I've been using it in a different way, trying to get development organisations to really break down what they're doing and to say, if we have this goal, if we have this metric, which we might have put in a roadmap, then what do we get at the end? Or what do you have to build in order to make this happen? And I think very often in the process of creating roadmap and activities, they put down things which they think are useful, but which might not have an impact, right? And having people revisit their backlogs and say, "Hmm, I thought that this would actually make this change and I can see now that it's not," and changing the roadmap, I think that's a discussion that needs to happen more and more in organisations. Are we actually spending our time on the right things?
- That's right. So many times taking those artefacts as sacrosanct and sitting back and watching people deliver against them is not the right approach. And I think the other, the kind of the pet hate that I tend to have, and this is from my time as a business analyst, is that actually sometimes the delivery of technology is not the right solution. Sometimes it's taking that solution or that product out or removing a feature or implementing a process in a different way. And so I always cringe, you know, when we often talk that's very predominant in product management circles about feature factories and build traps. I'm very passionate about that as a form of BA because sometimes that's not the way we should be doing something.
- Well I think that's a human artefact, right? Which is, we always add, we rarely decide to take things away. And I've heard, so some of the really successful product managers I've talked to, they actually pivoted and they basically said, "We're not gonna add any more features." We're gonna fix the UI, so it's easy to use, right? We're going to remove things, it's fine if it's not being used, it should be removed. But I think there's this push, this human push to build, build, build, build as opposed to, you know, strip away and make it very clear what's what's there and what's valuable.
- And I think we should do the same with our roadmaps, Pamela. We should strip away the noise on them and just deliver what's valuable.
- Right, and I think it also requires a bit of trust because if you do that in a roadmap, if you say, I am doing this, but I'm not gonna give you all the details, you have to develop a management team that trusts you to do that. And so building trust with your executives is really key in this process, that they know that you will deliver without looking at all the nitty-gritty details.
- I'm already learning a lot from speaking with you as well. I'm curious whose advice on road mapping do you listen to? Who do you see out there and you learned little, little, little tricks and tips from?
- It's very interesting cause I was listening to Rich Bernoff talking about sharing roadmaps with salespeople and making sure they're PDFs and that's something I've advocated for 10 years and I just learned that that's it's where it originated. So I was very happy. I, you know what, I hate to say this, but I think sometimes I read a lot, I synthesise a lot. I'm not entirely sure where all of it comes from. The other big teacher is mistakes I have made, and things that I have seen that didn't work.
- And obviously a person with your career, you've had a lot of interactions, 50 products that we've worked on over time. You know, a lot of those mistakes are fantastic learning opportunities as well. And we cultivate our own best practises from avoiding some of the poor practises too.
- There's a risk within a lot of what product managers do of wishful thinking. And if we build it, they will come, those are really big risks. And so those are things that I wish I had a magic wand, I could make sure I never do it again. But it's something I'm much, much more aware of as I go through my career.
- I wish more teams would talk more to customers. I wish they would prototype, and explore, and discover more before committing valuable resource to going down those route.
- There's a certain amount of humility in being a product manager, and I think a lot of people talk about being a product manager is you're the big boss, or you're the boss of your product. And the honest answer is really, really good bosses ask a lot of questions, right? And they don't jump to conclusions. They hold off as long as possible.
- And what about any road mapping resources do you use or typically recommend? Are they again, things you've picked up over time or have you got a few go-tos?
- I honestly do a lot of research no matter what I do. So the moment I have to do something, I'm on Google and I am researching what I think would be the latest cool thing to use. I probably would not be one of these people who would use the solar system cause I think it would get in the way of understanding however cool it would be. I'm really just gonna focus on the basics, get my message across and make sure people understand where we're headed.
- It's a couple of difficult questions now, possibly only because it has to get you thinking about things, but I'm wondering, and this is, I've found this really difficult myself, but after all that we've discussed, if you had to distil your philosophy of road mapping into just a few sentences, how would you describe it to people?
- It's a story, and it's an accountability, a way of communicating your accountability to the organisation and getting to the outcomes that the organisation has agreed are important for it.
- Story telling is such an important part of being a product manager and that needs to come across into your roadmap. Is there anything about roadmap that I should have asked you that I haven't?
- I was actually worried kind of about the constraint of road mapping. I've really think that this has led to a discussion which was quite broad. That there are many components to being a successful product manager. A roadmap is core that we're good at it, that we're good at telling the story via this communication mechanism. And the implications of communicating well or not, could dramatically change how well you're able to function within an organisation.
- Fantastic, well look, we've coming up to where we're gonna wrap today's session up. I've absolutely loved chatting with you. I wanted to give you a chance to just share how people might be able to get in contact with you.
- LinkedIn is awesome and public announcement here, in LinkedIn, you can actually find my email. So you can find my email, I have a very small website which kind of highlight some of the things I've worked on the last few years. And so you're free to go in and and look at that. Yeah, I really am working with, I prefer to work with teams right now, to really say what can I do with a team to make this team successful. Because more and more I think that product managers are, I think the scope of the role is so big that I think if you don't include the other compatriots in this journey, you're not gonna get very far, right. And the product manager can become a bottleneck. So again, anything I can do to help teams get better at producing better products, or faster, or more appropriate products, that's stuff I love working on.
- If anyone in our audience, if that resonates with you and it certainly does with me, and you're looking for a Pamela shape within your business, then please do reach out to her on LinkedIn and get in contact there. Pamela, I wanted to say just a massive thank you, it's been an absolute joy speaking with you today, kind of talking about some different stories, and different experiences there. Folks, if you found value in today, please do consider subscribing. If you haven't already, give us a like, if Pamela or I have mentioned something that resonated with you, drop it down in the comments. And of course if you're listening on the podcast as well, then please go and visit the website and you can get more details there. Pamela, thank you again so much. It's been an absolute joy.
- Thank you.