Ep#115 If Security is Job 0, FinOps is job .5

Host: Jon

Our next guest is Marit Hughes Specialist Master. We're going to be talking about FinOps culture. If security is Job zero, then FinOps is job 0.5. Wait, is that correct? Math? I'm not sure, but we're going to be talking about Marit about it. And she's a self-described AWS billing geek. I don't think it's, I think me is well known in the AWS and outside the community on billing. In fact, to me, she is a specialist at it. Maybe that's why her title is Specialist Master, I'm not sure. But she has 12 years of AWS billing and account management experience across the public sector and commercial and has recently moved to Deloitte to bring thin ops to federal, state, and local governments when not parsing through the curve file. And yes, she enjoys the curve file because she's a billing geek. She spends her time with her animals in Northern Virginia. Please join me in welcoming Marit Hughes to the show. Marit. Thanks for joining me,

Guest: Marit

Jon. Thank you for having me. It's been a journey in the time we've known each other.

Host: Jon

Yes, we met when I was at AWS, then you went to another company and then I contacted you and we reached out there and then you helped me with another thing for billing you're my go-to specialist around AWS billing. I don't reach out to AWS for it because I know you know the answer and can easily explain it to me.

Guest: Marit

I try every once in a while I hit a thing where my answer is, let me back into that for you to get to some of those finer grittier details on the areas I don't live in constantly. But yeah, it's been 12 years back from before any of these resources that we have now existed. It predated the cost and usage report and predated the detailed billing with resources and tags file. It was just the cost allocation report when I started. So it has been a lot of learning along the way

Host: Jon

Back then when you were downloading it into spreadsheets maybe.

Guest: Marit

Yes, I used to. For those of you who have lived in a reselling world or a word world where you wanted to do undo I sharing way back in the day, I did that via hand in Excel. And when some of the tools that are now really popular are given came out, I still had to undo them by hand for I think another year before the tools started to integrate those types of things into their offering.

Host: Jon

So mayor, normally I give everybody a little bit of a chance to give an introduction, but I think we've dived into some of your background and your expertise. I'd like to dive into our topic, what our topic today is, FinOps culture. If security is Job zero, then FinOps is job 0.5. But while that's our topic, I got to read you something and it happens to be here in this Cloud FinOps second edition book. Sorry, I

Guest: Marit

Think you might first page find that quote in there. I think to find that quote in there. I'm

Host: Jon

Exactly. But I think the cool part, first of all, is this is not even, it's like page one, page like pre-Roman NuMo run one, it says Security is job zero, FinOps is job 0.5, just as security is everyone's job. So too is FinOps the second edition of the cloud. FinOps books provide a recipe for FinOps success. It's by Marit Hughes Deloitte. Interestingly, you were, oh, by the way, I have to tell everybody, you're the first one ahead of commerce platforms at Google Cloud. The first one is ahead of the VP core conference at Microsoft and cloud financial management at Citi. So I think your quote has hit home with everybody.

Guest: Marit

I also think that it's the shortest quote. So it made a nice intro and it got even the shortest bio to go behind it. Jr, one of the authors heard me say that as an offhand comment a little bit, and it was about whose responsibility is establishing a pH ope culture and when should that start. And that was really what the conversation was, was when should you start incorporating pH ope? And at the time I said it as a little bit of an aside. It's something that I've said many times before though, which is security is a hundred percent number one, priority, particularly in the government. Depending on the agency, it might have every single person who resides in the US information about their lives, the Social Security Administration, the Census Bureau, whomever that information is there, security has to be job zero.

Guest: Marit

But establishing those components of how you approach solving your technical problems needs to include ops right there behind it. Because a lot of times people use FinOps as a shorthand for cloud financial management and they're cloud financial management are you getting things for the cheapest price possible? Right? Are you buying your ROIs? Are you negotiating your programs with your CSPs? And to me, fops is really from the moment of application ideation, I am going to build a tool that does x. I am going to build a SaaS that does why you need to be putting into that thought process. What are my priorities here on the cost-performance trade-off? Because you might have two very equal capabilities or methodologies you could use to architect a solution and one costs 20% more. Now, it may be that it's 20% more because it is that extra nine worth of reliability instead of four nines, it gives you five nines.

Guest: Marit

Take that moment to go, does this application need five nines worth? We all go to the store and go, oh, I like that shirt, but not for $150. I don't, right? And so it needs to be the same thing for your application design. And maybe you need that five nines for your production environment, but you're testing and staging environment while your staging should match your prod, but your testing environment, maybe you only need one nine. So save that money where you can, particularly if you are working in an environment that's come out of on-premises, right? They're not born in the cloud, really stopping and thinking, is this something we could handle with microservices? Is this something better suited for containers? Is this something that we need to do for legislative region reasons, or regulatory reasons use on-premises models then that's fine, but you had to have taken that moment to thought and consider and put that into your culture so that everyone is asking those questions to be able to really spread it and not just have it be well, the CFO's office says, or the pH OPE team says it needs to be intrinsic.

Host: Jon

All right, let me dive into a little bit about it, you indicated if I go to containers or serverless or however I want to do some things, you're suggesting that FinOps should play a role immediately and okay, I'm going to, this will be a cost saving versus normally it's thought, can I secure this serverless environment? Can I secure my container environment? Yes. Great. How can I do it efficiently and cost-effectively? Or you're saying, let's look at this first from the standpoint that you know where you want to go to the containers for cost-effectiveness. Now how can we secure this?

Guest: Marit

So security job zero, a hundred percent security is there, right? But once you're past the, okay, we understand we've got to have these firewalls, we got to have these not gateways, whatever security mechanisms that you're putting in place, there is still that, for example, how over-provisioned do we need to be? Right? Can we run this on graviton on a 2XL? We're going to our average, we think, right? Cause it’s the ideation stage. We think on a 2 XL with Graviton, we're going to be running like 65, 70% as peak, and two generations ago that is too high. You might run into performance issues, but hey, maybe with modern generations in Graviton, we can do this lower size. Or is your application so critical that you need to be over-provisioned because you can not afford a second of lag caused by CPU overages, right?

Guest: Marit

And having these critical thought discussions at the architecture and design phase. Because if you were just looking at how to save money on the cloud, we have this cloud bill, how do we save money on it? That retrospective culture needs to be from beginning to end. And hey, sometimes you're going to have an engineering team that says, you know what? To do this, I do need the biggest bestest over-provisioned everything. Sometimes your engineering teams are like, Hey, we can do it with less. And somebody who knows what the contract says, says, great, we can do it with less. Will we be able to meet this contractual requirement that's about downtime, about the ability to recover about dr? It's going to be part of how you do your multi-AZ versus global deployment and figure all of those components out. And it's not that the cheapest wins, it is a hundred percent not that the cheapest wins. It's doing that balance of fast, good, or cheap. Pick two. And the answer is not picked two, it's finding the spot in the middle of that triangle that balances all of those needs for what you need to do

Host: Jon

First. I like that you said security job zero, right? And instead of FinOps being one, it's like 0.5. It's very critical to the security part. You take care of the security aspect. And then your next step is talking about FinOps. And also it's not about cost savings. It's actually about what is the business-driven decision for your performance to meet your SLA, to meet your guidelines, to meet all those things. Cost does play a factor in it, but it's not the primary thing. It's really what are some of those guidelines. But one of the things that I find interesting is that it's a culture of practice. It's something that we should have been doing all along, whether it's in the cloud or the business or moving forward to that. But it's only become really huge and visible now that cloud is an on-demand and people are seeing that the bill is now trickling down to the developers, the hands-on engineers that are doing some of the work and provisioning, and then the finance is like, oh my God, my bill is three times as much and I can't forecast this

Guest: Marit

Well. And I think that that's the real distinction. It's not that the billing data has made it to the engineers. It's that for a long time, the cloud bill didn't register on the radar. One of the reasons I learned as much as I did about the true weeds of AWS billing was because I was just obsessed with understanding how they arrived at a hundred dollars and 12 cents. After all, I had to be able to explain it to a government contracting officer who still thought the clouds were the puffy things in the sky. And as I pushed back on AWS a lot back then, I was frequently told, you're the only one complaining. You're the only one complaining this isn't a problem. And I kept saying, well then I'm the only one looking. And two years later, AWS got up on stage at reinvent in the partner summit and went, oh, people complained that our I application method is just willy-nilly, which is a phrase I had used to them to describe it.

Guest: Marit

And what it was is that Bill was hitting more and more people's radars. It was very easy to ignore. Just like your subscription to Netflix, right? It's 1999 on your credit card, you don't even see it. And as cloud bills got larger and hit more companies, all of a sudden CFOs started to go, oh, this $3,000, even $10,000 bill is not a needle mover in my organization. Now that bill's 300,000, it's a needle mover. And then they start to hit 3 million and now it is the needle and find. So finance started to get involved. And I say this as a former finance accounting person now who divides the line between tech and finance and accounting. Is there just how can we make this number more consistent and let me predict it and make it go down, right? They're not concerned with the details of how that happens. And so they're going to push for the cheapest, just bring my bill down.

Guest: Marit

But I think that if we do take the time to establish that culture that says everyone cares and has chosen from the beginning to make sure that we did a proper balance of our needs. Because we all do. We do it when we go to the grocery store. Why can't we do it with our cloud bill to make sure that throughout our culture, this is something that we care about versus, oh, that finance people are over here screaming that we need to bring our bill down? It puts too much blame on finance and not enough on the people who decided to lift and shift and lift and shift is the surest sign of a lack of phen, open culture that I've ever seen in my life. And people continue to do it, even though we're 10 years into lift and shift is not the way to go. And anytime I hear that somebody is doing lift and shift, the first question I ask is, do they understand that it will be more expensive than being on-premises?

Host: Jon

So many topics, and so many comments on lift and shift. I don't know if we have a long enough show for this. I

Guest: Marit

Think I don't think so, and I don't think it's been beaten to death. So I don't know that we need to go through there. But it is part of that culture, what's your cultural practice? And if your cultural practice, the lift, and shift is even a regular option, it is every time you're migrating something to the cloud from on-prem, if lift and shift gets mentioned with a question mark every time you don't have that culture, cause it means that that's a viable option for you. Now, I did hear of a client who had mandatory regulatory compliance to lift and shift a particular application because their contract included certain specs and all of this stuff that could be newer, better, faster, and cheaper, but the contract said X, Y, Z, and they had to do it even though it was going to be more expensive. And that's something we run into a lot in public sector regulatory compliance.

Guest: Marit

Some of them make sense, right? Archive rules, things like that. Others sound good in theory, but no one thought about the consequence. We're just a week ago at the Washington DC Phen Ops Community Day and had a wonderful presentation. It was a panel of government employees who have been working to bring phen ops into federal. And one of them brought up the story of how an executive order came out. So they turned on all of this logging in their w s environment and it sent their bill through the roof because they were logging if it could be locked.

Guest: Marit

They just turned it all. It was locked all on, all of it all the time. And because that agency has put things into place to start alerting when they start seeing things outside of their norm and immediately caught the radar, went back to security. And when we get that there's regulatory compliance here, did you just go all in? Because that was the easiest way to assure ensure compliance because security has to take part in Phen ope, they may be Job zero, but they're not exempt from Phen ope and go back and say, all right, let's

Host: Jon

Needed what's required? Yeah, what is viable that's required out of these logs and what all the logs needed,

Guest: Marit

And do we need them all? Do we need them at this frequency? Does Splunk or Datadog or whoever needs to be accessing all of these logs all of the time? No, no, no, no. Let's turn that off. And because they have established fops in a large portion and they've installed some processes and some alerting, they were able to start pushing that culture backward. But if it had been truly embedded culturally before logging was turned on, there would've been a question asked, how much is this going to cost us? And do the people who are going to get the alert to know about it because sometimes you have to turn it on? That's how regulations work. You got to do it whether you want to or not or whether you can afford to or not. But with that advanced warning, that advanced consideration, you can start making plans of how to get more funding or how to shift funding or whatever it is you need to do to make sure that you can meet your regulatory rules and not go broke in the process.

Host: Jon

So, everybody, we are talking with me Hughes, she's a specialist master, and our topic today, is if security is job zero, then FinOps is 0.5. Merri. Before we dive a little bit deeper into it, we're talking about FinOps and in front of me, I have the Cloud FinOps book. By the way, I'S quoted on the very first page, top line, everybody, you should check it out. Second edition. I'll put a link in the description below, to get the book. There's a lot of information. I think it's two times thicker than the first edition. I didn't measure it exactly. Hey, this looks like a good place to jump in and talk about today's sponsor. Veeam, how would you like to own control and protect your data in the cloud? Are you using Salesforce? Veeam has you cover it with Veeam Backup for Salesforce, backing up your Salesforce data efforts, whether it's on-premise or in the cloud.

Host: Jon

Honestly, why wouldn't you back up your most critical CR M data from loss or corruption? Now imagine your sales team coming in and not being able to recover all their information, their notes, their pipeline because it's the one thing you didn't think you needed to back up. How about doing it Effortly with VE backup for Salesforce while there are nine reasons that you should back up your Salesforce data. How about just two data loss and data corruption? Veeam Backup for Salesforce eliminates the risk of you losing your data and metadata due to human error integration or other Salesforce data law scenarios. Check out Veeam Backup for Salesforce today. Now, how about we get you back to that podcast? I want to jump into a couple of things. We are talking FinOps. Can you distinguish the difference between Cloud FinOps and FinOps? Why is the title different?

Guest: Marit

Honestly, I would say pH ops is a cloud-borne cloud native concept. It's very much analogous to DevOps, which again, kind of in the cloud. And we've taken those lessons learned and moved them into on-premises. The pH ops foundation had one of their women in tech events with the leads from a couple of large commercial entities who talked about it, they started with Fops in their cloud and they're now moving that to their on-premises environments. This what do we need? Because on-premises was very much buy for whatever you think you might need in the next three years, we're going to CapEx that baby, and hey, if we overbuy, that's better than underbuying. And hey, we're already paying for the AC and the utility. Great. Just we're good. We're good. And they're starting to push that, Hey, if you don't need it, turn it off. If just because you can run it on a four xl, but you could also run it on a small, right, why don't you go ahead and downsize that? And so they're starting to take it there.

Guest: Marit

Could they have just called it ops originally? Sure. But when they put out the first edition, nobody knew what pH ops were, whether it was cloud or on-premises, right? So they put the cloud in there to hit the search algorithm and it's stuck, but I think that it doesn't matter. The reality is most of us are using a variation of pH ope in our everyday lives, and we're not applying it to our cloud bill, we're applying it to our monthly budget. We're applying it to our grocery bill, we're applying it to our home remodeling decorating ideas. It's a cost-benefit analysis of, for this amount of money I get X, do I need it? Is it worth it? Can I afford it? Yes or no? The nice thing though about the cloud is it is much easier to purge your cloud environment than it is to purge your house

Host: Jon

Or your data center

Guest: Marit

Or your data center. You can run a few scripts and clean out your cloud environment. But sadly, the house and the data center take moving equipment and recycling trips and dump fronts.

Host: Jon

Oh my God, I think I'm going to quote that Marit. It's going to be in, I can't say my next book because I didn't do a first book, but I think I might have to quote it myself. One of the important things with FinOps is that it's not about cost savings, right? Reduction. It's actually about making money. That's exactly what I've kind of got driven through in a lot of the cultures. It's on the FinOps website most people think FinOps is about reducing cost, right? Sizing, cost optimization. What's your feeling here?

Guest: Marit

I would say that all of those things are components of pH ope because you do want to do those things. But the fundamental truth is all of those things are retrospective. They're cleanup, right? Turn off the things you're not using. Hey, buy your ROIs. Buy your savings plans for spending. You're already doing. I think when something if you're truly culturally engaged in ops and not just ticking the box that says, we have ops check you

Host: Jon

Bought it, we bought it and now we have it. Here's the logo.

Guest: Marit

Yes, exactly. You have taken it to your architecture level, you've taken it to your DES development level. You have kind of checks and balances to make sure that you're doing what you're doing just like you would on your development pipeline for your DevOps and your DevSecOps and your this ops and before your releases. That's all in there. But one thing I wanted to kind of stop you on, and I probably should have started with that, is you had said, this is all about making money. And that's true for commercial SaaS enterprises.

Host: Jon

I like the clarity

Guest: Marit

Because if it's a commercial SaaS enterprise, the application you're building is software as a service that is generating revenue. If you are a retail and your cloud environment is hosting your website, it's making money. If you're a government agency collecting census data, it's not making you money. Very true. If you're a government agency that is processing visa applications that technically have a fee assigned to them, people pay for their fees, their visas, but the amount of money they're taking in, they still took that money in when they were processing actual paper. So those things aren't always actively making money in some even kind of traditional businesses. Cloud is part of their cost of doing business, but it's not in and of itself generating revenue for them. So it's tempting, particularly unit economics is attempting things. What is our cost per client, our cost per customer? Those things are relevant in government too. If you are the VA, if you're the CDC, the cost per individual consumer is a metric, but it's not a revenue-based metric.

Host: Jon

So two things came out of that. I want to understand more about the unit economics, and I'm sure our audience wants to dive into it, but do you feel that cloud pH ops, I'm just going to call it FinOps because I feel like I'm just using new news using another term, but do you feel that the entire methodology and culture and everything that's tied around FinOps applies to the government or now we have a separate variation of it?

Guest: Marit

It a hundred percent all applies. It's just in a slightly different flavor, a slightly different motivator for some of the personas. And we have extra personas, particularly in federal than you would typically find in a commercial enterprise. The drivers of resistance are a little different in government, but very similar to large-scale old school enterprises. And I think the distinction where the distinction needs to lie is less between federal and commercial and more kind of cloud-native industries. The challenges Instacart has with their cloud bill, it's going to be very, very different than some banks that started 200 years ago. It's a lot of just what your overall culture is in your organization and now how that applies to your cloud. But yeah, no, there's the playbooks. The steps are a little different, but the cultural elements of the things you need to consider are all the same, regardless

Host: Jon

How big or how big do you feel that FinOps is in government now and how big do you think it's going to get and the importance of it in the next say? And I know government moves slowly, so we'll just say a year to five years.

Guest: Marit

So what's interesting about how cloud technology is deployed in the government is the vast majority of cloud technology is being deployed in, let's take one agency as an example, agency X. There are not just government employees there, and there is not just one government contractor. There are 10 different government contractors all working on probably a total of 30 different applications, some of which are legacy applications, some of which are net new things being built, and trying to get cohesion across all of those parties while still keeping them in compliance with their contracts, right? Because their contracts contain requirements that may have been written four years ago.

Guest: Marit

They may be supporting an application that was built eight years ago, and it needs to continue to be supported because it's providing needed services to citizens. But also what we run into a lot in the government space is one. And I do not mean Oracle cloud. I mean an organizational conflict of interest. So for example, if I were to get oversight of the entire cloud bill for an agency that I was working for or an agency I was supporting, that would give me insight into what my competitors are doing in their environment. And that in most cases would exclude my firm from bidding on any work related to the cloud for that agency. Now, there are firewalls and stuff you can put up to reduce that, but it's very hard to build a culture with that many disparate parties that have conflicting motivations, and conflicting contracts that don't even align with each other, which don't mean it can't be done.

Guest: Marit

And that's why the work that Melvin and Laura are doing at G GSA and D O E OPMs involved as well is so critical because the government has to be the ones starting to push this culture. As contractors, our responsibility is to support things that are under our control for our applications and make sure our development process includes a fops culture. What's nice here is our pH ops offering has been stood up by our migrations lead. So all of our, just by kind of almost by definition of having the same sponsor, it's already been built in so that everything we're doing on that side from that, no lift and shift are happening unless the contract says it has to, is implementing that throughout the process. And I do not doubt that others are doing it because the government's starting to squeeze down on Bill on how much they'll fund. So you've got to be very aggressive with keeping things tight to make sure you're funded. Because in commercials, they can always shift money around to give you more. And in the government, there's probably a congressional regulation that says somewhere we can't take money that we didn't use to buy tickets to reinvent. You can't use that money to spin up an extra server.

Host: Jon

Now, the more and more that I learned about government, the more I'm educated. I was down at the FinOps roadshow in DC last week and hearing all of our guests, our speakers, and understanding the processes, and just listening that people ask questions, I'm like, wow, this is not something in the commercial aspect. I want to jump

Guest: Marit

Back to, did you feel a little bit like you fell in a suit, a bowl of alphabet soup?

Host: Jon

Oh, the acronyms that were happening in me and throwing around the people, I'm like, wait, what is that? And I had to look up certain lawns and they're like, oh, this one, they do o D to DOJ this. I'm like, what is that? Wait, I've never heard it. I didn't have enough acronyms. I probably had too many acronyms when I left and I didn't understand half of them

Guest: Marit

Get in line. There are a lot of people like that. It was very interesting to me. I ran into several people I knew at that event who were commercial and had come to that event because it was local. And more than one of them admitted that they had not looked at the agenda. So hadn't realized how government heavy it was going to be. And without fail, every single last one of the purely commercial people said, I had no idea this much went into it. And it is a different world because of the regulatory issues mostly, but not entirely strangely. I find those rules. I can't say I find them freeing because that would be fundamentally untrue, but it is nice to have. So in commercials, kind of anyone can make anything happen. You're restricted to the internal politics of your environment, of your company. Internal politics still play a big role in government. Far more at the agency level, day-to-day operations. Internal politics are far more important than national politics. You're arguing with your CIO, it does not matter whether he's a Republican or a Democrat. It's really because you know, just disagree on things.

Guest: Marit

But some of these policies can be very restrictive, but they also give you boundaries. This is where my domain ends, and would I like it to expand further, but at least I know where it ends. So that part can sometimes be a little bit of a stress relief because you're like, I don't have to be responsible for it all. I can only control my area. But it does make implementing pH apps a little bit more difficult because of where those boundaries lie. And that's why it's so important to have government sponsors, government employee sponsors of putting pH pops into their agencies, into their procurement, into how they manage their day-to-day spending.

Host: Jon

Mary, can you help me understand the unit metrics and how it applies to FinOps what are some of the differences between the commercial and then the government for unit metrics? I know you were talking about it, but I'm very intrigued by some of the differences if you're able to help me out.

Guest: Marit

So typically when we talk about unit cost in a commercial world, it is your cost per customer. That's kind of your cloud cost versus your revenue and how you slice and dice what constitutes revenue, what slice constitutes cloud cost. That varies, but the formula is effectively the same in government. It depends on what that agency is doing and what that application is doing. So if it's the D O D, is it cost per soldier? Is it cost per airman? Is it cost for? Is it just for the Marines? Is it just for the air force? Is it for all of them? Is it just for their dependence? So finding that one true metric isn't quite the same because there's no revenue associated with it. So what's going to be your top-line denominator or your numerator? It's very difficult and you certainly can't have one that goes across agencies because how would you compare the Department of Interior or the National Park Service to the Department of Defense?

Guest: Marit

You can't use identical metrics for things like unit cost. You can use them for a whole host of others and have identical metrics for a whole host of other things. What percentage of your environment is tagged? All of that is very static. But to determine that unit cost, you have to understand what the application is doing. And I think we're going to find more and more at the government that it's not going to be a cloud environment. It's going to be a cloud application because one single agency could be doing so many different types of work, and again, not tied to revenue that how you measure how effective something is it will make creating benchmarks across agencies very difficult. When we look at the state of pH, ope, they had all those statistics, where are you at this? Where are you at that are you run, walk, crawl? And I think what we're going to find is there's not enough commonality on some metrics to even do a federal agency against agency benchmark. Other metrics are a hundred percent, you could even put them up against commercials. But those along the lines of unit economics will be very, very difficult because there is no one true numerator and no one true denominator. So it will be difficult to really, and I know I've said this again, but it's going to be very difficult to find this one number that will be, you have reached the nirvana of FinOps,

Host: Jon

But that doesn't apply to me, so I'm a commercial company and I want to do the unit economics around it, kind of thinking about that commercial company as a different agency and I want to apply that to unit economics for this agency, for the Department of Defense or Department of Interior or the parks. Each one will have its separate unit. Economics isn't that acceptable? And we're all doing, I mean I assume they would implement it at their own pace and their level because they're separate groups or separate entities.

Guest: Marit

But I think the difference is Instacart could probably measure up against DoorDash against GrubHub again, and maybe even so there's no comparison in the government, but trying to find who is the competitor to N ih. I mean, I guess maybe C D C, but not really, right? And so trying to determine how efficient you are based on external measurements is I think what's going to be very difficult in government. Not impossible. I think that we're going to find, again, there are some metrics that we will be able to do across government. What percentage of your workload is being run with microservices? What percentage of your workload is tagged? What percentage of your environment is scheduled to turn off? How much, what's your return to your time between failure? There are all sorts of metrics on your environment, but those unit economics ones your cost per customer, those are going to be hard because a lot of agencies are serving multiple types using very different resources. And I think the best option, and this is an off-the-cuff thought that I have not critically thought through, so nobody holds this against me in three years. I think what it's going to be is start within an agency and go, okay, this particular application has a cost per cust per soldier, for example, of $18, but this other application has a cost per soldier of $3.

Guest: Marit

Now what they're doing, why it's so complex, may be valid, or may not be. But I think as we start looking at econ unity economics in the government, it's probably going to be different applications within an agency measured against each other and not an agency-to-agency comparison. And I mean, God, I hope I'm wrong and we can get to the agency to agency, but I don't think that that's anywhere in our near future.

Host: Jon

I thought you were going to say in our lifetime. I'm like, wow.

Guest: Marit

Well, I mean, wait, how healthy are you?

Host: Jon

I'm a healthy male adult, so I should be all right.

Guest: Marit

I am dunno. Let's check with those insurance actuaries. They may have different opinions, but it's hard saying. What's funny is the government gets a reputation for moving slowly, and sometimes they do, but one of the reasons they move slowly is because of regulations that force them to, other times it's just resistance to change, which has plagued commercials as well. I just think even multinational companies that are in multiple areas, Amazon themselves, for example, have a retail business. I got a thing today that they're starting to do telemedicine-type of stuff.

Guest: Marit

I knew they were starting prescriptions, but apparently, I can email them and tell them that I have a painful shoulder and they can give me a prescription. I don't know. Right? As they start to measure their cost per customer between those different business units that are not analog, I'm not going to say that correctly. So I'm going to choose a different world word that is not parallel. So your healthcare, you can't measure your healthcare business cost per customer versus your AWS versus your Amazon retail cost per customer. I think that that's probably a closer comparison to the struggles the government's going to have with unit economics than comparing single industry players. Because even Uber with Uber and Uber Eats is what it still is, car share with one person driving around in their car. It's just sometimes they're driving around food, sometimes they're driving around people. That is a very similar industry versus comparison. If Uber also owned the, I don't even know what business, let's say if Uber also owned urgent care, you can't compare those.

Host: Jon

That would be interesting.

Guest: Marit

They're just acting. You know what they're doing. They're trying. If they do that, it's because they know how expensive ambulances are and they're trying to take in that business. How many people has Uber taken to an actual hospital because ambulances are too expensive? So

Host: Jon

I bet you that there are some stats behind that.

Guest: Marit

I am sure there are, and I'm sure there are some Uber driver horror stories.

Host: Jon

I need to Google that.

Guest: Marit

But I think that it's those real kinds of large-scale enterprises that are in very diverse business areas. I mean, Jeff Bezos owns the Washington Post. How do you compare a newspaper to Amazon? And rumor has it, he's trying to buy the Washington commanders and how do you compare a newspaper to a football team? Those are the problems of the agencies not comparing Nike to Adida's cloud costs.

Host: Jon

Very true, very true. Me. I've got two more questions before we have to wrap it up. One of 'em. All right, one of 'em we talked about, and I want to reiterate the story and use it as an example, but it's going to be retrospect for cost savings and optimization, commercial versus government. So I'm going to give you a little heads-up that we're going to ask that question. Which one do I hope which one we're talking about with the workspaces?

Guest: Marit

I think we'll, do workspaces, okay?

Host: Jon

Yeah, we were having that discussion in person and talking about it. This first one I want to understand. So implementing FinOps, I know it should be done or mainly done upfront, but all these companies exist and now they want to implement it. Can I implement FinOps now and what? What does it look like from actually getting started from the first-time implementation or an existing, one I'm assuming I've got all these applications versus I've got all these new applications and I can implement FinOps from the start?

Guest: Marit

I think when you have an already existing environment that's sizeable, built out, whatever, and now you're starting to say, Hey, we need to take a cultural approach to this and not just negotiate our price down with AWS. It is going to take engaged stakeholders from a bunch of areas. In my opinion, the most important stakeholders to get engaged with are on the technical and engineering side because they're the ones who have the know-how, the ability to determine, and the ability to execute things. Like what if we upgrade to the graviton, oh, we're upgrading to the graviton, but we're sitting on a c4, S and a C seven G mean I could probably downsize two sizes. You don't want that decision in the hands of finance.

Host: Jon

Now finances will say, turn it all off. Or finance. Finance

Guest: Marit

Might say, finance might say turn it all off. Or finance might say, well then just upgrade to Graviton. Well, but is the workload compatible? Maybe the workload needs an intel chip, great needs an intel chip. Could we still go up to a C6 and still save 10%? But going from a T3 large to a T3 medium is only going to save you this much per month. So it might not be worth the engineering time, but you need those engaged stakeholders who want to push efficiency. And that efficiency doesn't just come with a dollar sign. It comes with a little green leaf too, because as you downscale, as you modernize and get things more performant, you are getting more for less using less energy. So if I can go from a C4 four XL to a C7 G xl, I've saved money, I've saved the environment, and hopefully, it comes back to me in my performance review and bonus.

Guest: Marit

So please, if you are doing these large initiatives, be sure to track them. You are not going to get all the savings as a bonus. I wish I just found about $150,000 worth of waste in one category. This is what I thought you were going to ask about. Not EBS I mean not workspaces that you're going to ask about EBS. I will not get a penny of that specific money in my bonus. So track it so that you can say at the end of the year, I saved 1 million. Please do not expect to get a half-a-million dollar bonus as a result, but do track it so you're going to need them. If it is just finance slamming on, save money, save money, save money, you're going to get resentment. So it is building that culture of, hey, we're all trying to achieve this multitude of goals of secure, stable, scalable environment that uses less energy and costs less money. Finance can focus on negotiating EDPs and buying savings plans in our eyes, but they still have to talk to engineering because if engineering's about to resize and move their RDS to the graviton, there are no savings plans for RDS. So please dear finance, do not buy your ROIs without confirming. With engineering, those environments are staying up.

Host: Jon

Hey everybody, Finn ops, financial, and operations working together, team collaboration, and it's getting all the stakeholders, it's coming together now.

Guest: Marit

Yes, exactly. I've long said that the main differences between on-premises and the cloud were on-premises engineering and procurement, finance, whoever would talk every three years, and in the cloud, somebody's like, yeah, they've got to talk yearly. I was like, what are you talking about? They need to talk at least monthly. Daily, yeah. At least monthly. Daily. For most organizations, not all of them should be talking daily, but there should be a pinned chat on your Slack channel. This is a person I talk to often enough, they should get pinned to the top

Host: Jon

At the top of my DMs. I get notified every time they message me just because it's financed or just because it's technical and I lean on them. Well,

Guest: Marit

Just because it's important. So because you do need that support. You don't want to buy the wrong RI cause then you're not saving money and you do need to know what because you're building your budget for next year or your forecast for next year in the government. They have to forecast three years in advance. How many of you can estimate your cloud bill for two years from now? And the government has to because that has to go to

Host: Jon

Congress. I think two months from now is difficult.

Guest: Marit

Yeah, exactly. So building that is even when you're already deployed, you've got to start with building those relationships. I'm in a working group right now for the pH ope foundation on building trust and working together because you do have to build trust because these are not teams that have a history of working together. There are negative stereotypes on both sides. Finance people only care about numbers in black and white and RA and then tech people. They don't have any social skills. And the reality is some of the most eloquent people I've ever talked to are tech workers because are developers in particular. Because developers, problem-solving skills, and the way you have to effectively write your code is a communication method. So they just have to translate it into English which regular people understand. And I say this as one of the regular people too, but they're great communicators. It's just that they use, you don't

Host: Jon

Speak it in JSON, YAML, or Python.

Guest: Marit

I, I'll tell you, I can read JSON, but I cannot write JSON just like a chat. I can read Italian, but I can't speak it, and I can't write Italian. So, it's a challenge, but it can be done and should be done. Don't let the fact you haven't started with PHP stop you from implementing it. Just please don't turn it into a way to beat up engineers. It's not helping anyone.

Host: Jon

Me, my last question is yes, really talking about cost savings and right sizing, you and I were talking about this, and from a commercial perspective, it's very easy for us to do and just turn it off. But from a government perspective, you gave me an insight into why it's not easy. So I'm going to set everybody up. I work example, I work for a commercial company and I have a hundred developers that are using workspaces, but they're only using them for 15 days out of the month. We're going to say a month is 30 days. We'll just narrow down some numbers, right? And we don't need them for the other 15 days or they're not using them for the next six months. Well, we turn them off. We don't need them. Shut 'em down, get rid of 'em and we will. Reprovision is easy with commercials, right? Marit.

Guest: Marit

Correct. Okay. Generally speaking, it's easy with commercials. Yes.

Host: Jon

Well, I'm summarizing, and paraphrasing. I'll turn it off. We'll deploy it. Everything. But you help me understand the government. Will you explain to everybody some of the processes and some of the thoughts and things that go behind the scenes on why we can't turn off terminate or get rid of those workspaces for the next six months and just turn 'em back on or provision them out?

Guest: Marit

So a colleague of mine went through an effort to try and optimize workspaces. They had hundreds of workspaces, some of which had not been logged into. And he couldn't just say, let's turn them off more accurately. He said what we could do is turn them off and that all of these things, whether it's a workspace, whether it's a server, have to go through an approval process. And that the approval process is not email. Susan, that approval process probably include, included four layers of security, a description of what you were going to run on that workspace, and a whole bunch of things. And it probably took you somewhere between two to six months to get permission for that workspace.

Guest: Marit

And it came in while you were out on F M L A. Right? Can't turn it off because when they're ready to use it again, they have to go through that whole process all over again. And if they need it now, they won't be able to deliver it. Right? So because it is not a, yeah, we have a process, but you need an exception, go have your boss talk to Susan. She'll get it launched for you today. It's a, oh, you need it today. I'm so sorry we shut it off because you hadn't used it in three months. Is not something that flies because there is no mechanism to do a fast restore for a lot of those things. And it's something that kind of, the Normies, I don't know, struggle with a lot. Because they're like, well, that just seems like a waste. The government shouldn't be wasting that money.

Guest: Marit

But so many of these rules and regulations were put in to prevent fraud in other areas. And how do you balance all of those things? They had to go through those security checks to make sure that they weren't going to put your social security information on the net right? They have to go through all of these checks and then get permission and have the funding. And that funding is specifically for that type of workload. You can't take it from workspaces and say, I'm going to go store more in S3 or something with it. Sometimes you can, but rarely.

Guest: Marit

And those operational hurdles are non-negotiable. There is no, just have the CEO say yes. And it makes it difficult in some cases to take that, shut it off, shut it off approach. So my colleague went through a whole, what is the usage consumption for monthly versus auto stop hot tip somewhere, depending on your sizing between 80 and 90 hours per month is the breakeven point on monthly versus auto stop. And anybody who was using less than that threshold got converted to auto stop. So at the very least, if they weren't using it, you weren't paying the monthly fee. And, on a different client, I recently used the reverse. They had everything on the auto stop, but a fair number of them were being used 40 hours a week every week. I was like, Hey, you need to go find out what you,

Host: Jon

There were no cost savings there. Yeah,

Guest: Marit

Yeah. You're paying more, right? So go figure out which ones are which, and if they're 40 hours a week every week, convert them to monthly. So it can go both ways, but it can be, sometimes those hurdles in government are frustrating for somebody who's trying to help them as a taxpayer. But as you start to understand those operational hurdles, the regulatory hurdles, just like, Hey, turn on all the logging, please. They are there for a reason. And whether that reason is good, bad, has good intentions, or bad outcomes, it doesn't matter. It it's a regulation. And all we can do is help find the most cost-efficient method to keep everyone compliant.

Host: Jon

Ramira, we, so we got this workspace, and I know we're almost out of time, but I'm very curious. We got this workspace and the developer's not using it, whatever he's not, and I just stop it and they can't submit a ticket to just turn it back on because it's already been provided. I already went through all the approvals, but it's just sitting there stopped. They can't submit a ticket. I can't just go

Guest: Marit

Place start, stop. So auto stop is fine. It's the deleting of them, right? Okay. Because that's what we did. We converted them from monthly to auto stop and then they just sign in and it restarts.

Host: Jon

Yep. Yeah, it takes about 90 seconds to come back online.

Guest: Marit

That is

Host: Jon

Fine. Just suspend is the, okay,

Guest: Marit

You have 300 of them and a hundred of them have not been logged in for six months.

Guest: Marit

You can't just blanket delete those. You can put together a list. You can start approaching people. Are they still here? Do you still need this? And you can go through that much longer process. But in commercials, you can kind of be like the all right, new rule. If your workspace hasn't been used in three months, goodbye went. And in government, you can't be as loose and fast as that because of the hurdles to get it back. So instead you go, all right, well we'll convert you to auto stop for now and then begin the process of saying, Hey, these six instances workspaces belong to people who are no longer here or have changed roles. So don't need them. Let's send them an email to confirm. And then the next 20 are, we got approval to delete. But those things can take weeks and months to work through. But you can get your immediate short-term savings by just saying, okay, well we can't turn them off, but we can convert them to auto stop if they haven't been logged into so that you can try and at least stem that waste. Wow. And again, a little bit greener if they're at least stopped.

Host: Jon

Very true, very true. And auto stop will save some of those as long as it's cost-effective as well. There's always that fine balance. Me. Thank you so much for joining me. This has been insightful. Our topic today has been, if security is Job zero, FinOps is job 0.5. Marit, thank you so much for joining me,

Guest: Marit

Jon. Thank you for having me. And I am sure our paths will continue to cross at conferences if nowhere else.

Host: Jon

Oh yes, and definitely. Marit Hughes, specialist master, and we were talking specifically about cloud, however, you want to do it, and some of the in-depth knowledge that she has, not only around the cost but the government sector, everybody. My name's Jon Myer. You've been watching the Jon Myer podcast. Don't forget to hit that, like subscribe and notify, because guess what, as always, we're out of here.