advice

Questionable Advice: “My boss says we don’t need any engineering managers. Is he right?”

mipsytipsy, , , , , 18 Comments

I recently joined a startup to run an engineering org of about 40 engineers. My title is VP Engineering. However, I have been having lots of ongoing conflict with the CEO (a former engineer) around whether or not I am allowed to have or hire any dedicated engineering managers. Right now, the engineers are clustered into small teams of 3-4, each of which has a lead engineer — someone who leads the group, but whose primary responsibility is still writing code and shipping product.

I have headcount to hire more engineers in the coming year, but no managers. My boss says we are a startup and can’t afford such luxuries. It seems obvious to me that we need engineering managers, but to him, it seems just as obvious that managers are unnecessary overhead and that all hands should be on deck writing code at our stage.

I don’t know how to make that argument. It seems so obvious to me that I actually struggle to put it into words or make the case for why we should hire EMs. Help?

— Unnecessary Overhead(?!?)

Oh boy, there’s a lot to unpack here.

It is unsurprising to me that your CEO does not understand why managers exist, given that he does not seem to understand why organizational structures exist. 🙈 Why is he micromanaging how you are structuring your org or what roles you are allowed to fill? He hired you to do a job, and he’s not letting you do it. He can’t even explain why he isn’t letting you do it. This does not bode well.

But I do think it’s an interesting question. So let’s pretend he isn’t holding your ability to do your damn job hostage until you defend yourself to his satisfaction. 😒

I can think of two ways to make the case for engineering managers: one is rather complicated, from first principles, and the other very simple, but perhaps unsatisfying.

I personally have a … vigorous … knee-jerk response to authority; I hate being told what to do. It’s only recently that I’ve found my way to an understanding of hierarchy that feels healthy and practical, and that was by looking at it through the lens of systems theory.

Why does hierarchy exist in organizations?

It makes sense that hierarchy comes with a lot of baggage. Many of us have had bad experience with managers — indeed, entire organizations — where hierarchy was used as a tool of oppression, where people rose up the leadership ranks by hoarding information and playing dominance games, and decisions got made by pulling rank.

Working at a place like that fucking sucks. Who wants to invest their creativity and life force into a place that feels like a Dilbert cartoon, knowing how little it will be valued or reciprocated, and that it will slowly but surely get crushed out of you?

But hierarchy is not intrinsically authoritarian. Hierarchy did not originate as a political structure that humans invented for controlling and dominating one another, it is in fact a property of self-organizing systems, and it emerges for the benefit of the subsystems. In fact, hierarchy is absolutely critical to the adaptability, resiliency, and scalability of complex systems.

Let’s start with few basic facts about systems, for anyone that may be unfamiliar.

Hierarchy is a property of self-organizing systems

A system is “a network of interdependent components that work together to try to accomplish a common aim” (W. Edward Deming). A pile of sand is not a system, but a car is a system; if you take out its gas tank, the car cannot achieve its aim.

A subsystem is a collection of elements with a smaller aim inside a larger system. There can be many levels of subsystems that operate interdependently. The subsystems always work to support the needs of the larger system; if the subsystem instead optimizes for its own best interests, the whole system can fail (this is where the term “suboptimize” comes from 😄).

A system is self-organizing if it has the ability to make itself more complex, by diversifying, adapting, and improving itself. As systems self-organize and their complexity increases, they tend to generate hierarchy — an arrangement of systems and subsystems. In a stable, resilient and efficient system, subsystems can largely take care of themselves, regulate themselves, and serve the needs of the larger system, while the larger system coordinates between subsystems and helps them perform better.

Hierarchy minimizes the costs of coordination and reduces the amount of information that any given part of the system has to keep track of, preventing information overload. Information transfer and relationships within a subsystem are much more dense and have fewer delays than information transfer or relationships between subsystems.

(This should all sound pretty familiar to any software engineer. Modularization, amirite?? 😍)

Applying this definition, we can say that a manager’s job is to coordinate between teams and help their team perform better.

The false binary of sociotechnical systems

You’ve probably heard this canard: “Engineers do the technical work, managers do the people work.” I hate it. ☺️ I think it misconstrues the fundamental nature of sociotechnical systems. The “socio” and “technical” of sociotechnical systems are not neatly separable, they are interwoven and interdependent. There is actually precious little that is purely technical work or purely people work; there is a metric shitload of glue work that draws upon both skill sets.

Consider a very partial list of tasks done by any functional engineering org, besides writing code:

  • Recruiting, networking, interviewing, training interviewers, synthesizing feedback, writing job descriptions and career ladders
  • Project management for each project or commitment, prioritizing backlog, managing stakeholders and resolving conflicts, estimating size and scope, running retrospectives
  • Running team meetings, having 1x1s, giving continuous growth feedback, writing reviews, representing the team’s needs
  • Architecture, code review, refactoring; capturing DORA and productivity metrics, managing alert volume to prevent burnout

A lot of this work can be done by engineers, and often is. Every company distributes the load somewhat differently. This is a good thing! You don’t WANT an org where this work is only done by managers. You want individual contributors to help co-create the org and have a stake in how it gets run. Almost all of this work would be done more effectively by someone with an engineering background.

So you can understand why someone might hesitate to spend valuable headcount on engineering managers. Why wouldn’t you want everyone in engineering to be writing and shipping code as their primary job? Isn’t that by definition the best way to maximize productivity?

Ehhh… 😉

Engineering managers are a useful abstraction

In theory, you could make a list of all the tasks that need to be done to coordinate with other teams and have each item be picked up by a different person. In practice, this is impractical because then everybody would need to know about everything. One of the primary benefits of hierarchy, remember, is to reduce information overload. Intra-team communication should be high-bandwidth and fast, inter-team communication should be more sparse.

As the company scales, you can’t expect everybody to know everyone else; we need abstractions in order to function. A manager is the point of contact and representative for their team, and they serve as routers for important information.

I sometimes imagine managers as the nervous system of the company body, carrying around messages from one limb to another to coordinate actions. Centralizing many or most of these functions into one person lets you take advantage of specialization, as a manager builds relationships and context and improves at their role, and this massively reduces context switching for everyone else.

Manager calendars vs maker calendars

Engineering labor takes concentration and focus. Context switching is expensive, and too many interrupts can be fatal. Management labor consists of context switching every hour or so, and being available for interruptions throughout the day. These are two very different modes of being, headspaces, and calendar schedules, and do not coexist well.

In general, you want people to be able to spend most of their time working on things that contribute to the success of the outcomes they are directly responsible for. Engineers can only do so much glue work before their calendar turns into Swiss cheese and they can no longer deliver on their commitments. Since managers’ calendars are already Swiss cheese, it’s typically less disruptive for them to take on a larger share of glue labor.

It isn’t up to managers to do all the glue work, but it is a manager’s job to make sure that everything that needs to get done, does gets done. It is a manager’s job to try to line up every engineer with work that is interesting and challenging, but not overwhelming, and to ensure that unpleasant labor gets equitably distributed. It’s also a manager’s job to make sure that if we are asking someone to do a job, they are equipped with the resources they need to succeed at that job. Including time to focus.

Management is a tool for accountability

When you’re an engineer, you are responsible for the software you develop, deploy, and maintain. When you’re a manager, you are responsible for your team and the organization as a whole.

Management is one way of holding people accountable for specific outcomes (building teams with the right skills, relationships, and processes to make good decisions and build value for the company), and equipping them with the resources (budget, tools, headcount) to achieve those outcomes. If you aren’t making building the organization someone’s number one job, it won’t be anyone’s number one job, which means it probably won’t get done very well. And whose responsibility will that be, Mr. CEO?

There’s a real upper limit to what you can reasonably expect tech leads, or engineers, or anyone whose actual job is shipping software to do in their “spare time”. If you’re trying to hold your tech leads responsible for building healthy engineering teams, tools, and processes, you are asking them to do two calendarily incompatible jobs with only one calendar. The likeliest scenario is that they will focus on the outcomes they feel comfortable owning (the technical ones), while you pile up organizational debt in the background.

In natural hierarchies, we look up for purpose and down for function. That, in a nutshell, is the more complicated argument for why we need engineering managers.

Choose Boring technology Culture

The simpler argument is this: most engineering orgs have engineering managers. That’s the default. Lots of people much smarter than you or me have spent lots of time thinking and tinkering with org structures over the years, and this is what we’ve got.

As Dan McKinley famously said, we should “choose boring technology“. Boring doesn’t mean bad, it means the capabilities and failure conditions are well understood. You only ever get a few innovation tokens, so you should spend those wisely on core differentiators that could make or break your business. The same goes for culture. Do you really want to spend one of your tokens on org structure? Why??

For better or for worse, the hierarchical org structure is well understood. There are plenty of people on the job market who are proficient at managing or working with managers, and you can hire them. You can get training, coaching, or read a lot of self-help books. There are various management philosophies you can coalesce around or use to rule people out. On the other hand, the manager-free experiments I’m aware of (e.g. holacracy at Medium and GitHub, or “Choose Your Own Work” at Linden Lab) have all been quietly abandoned or outgrown. Not, in my experience, because leaders went mad for power, but due to chaos, lack of focus, and poor execution.

When there is no explicit structure or hierarchy, the result is not freedom and egalitarianism, it’s “informal, unacknowledged, and unaccountable leadership”, as famously detailed in “The Tyranny of Structureless“. In reality, sadly, these teams tend to be chaotic, fragile, and frustrating. I know! I’m pissed too! 😭

This argument doesn’t necessarily prove your CEO is wrong, but I should think his bar for proof is much higher than yours. “I don’t want any of my engineers to stop writing code” is not an argument. But I’m also feeling like I haven’t quite addressed the core question of productivity, so let’s pick that up again once more.

More lines of code != more productivity

To briefly recap: we were talking about an org with ~40 engineers, broken up into 10 small clusters of 3-4 engineers, each with a tech lead. Your CEO is arguing that you can’t afford to lose any velocity, which he thinks is what would happen if anyone stops writing code full time.

Maybe. But everything I have ever experienced leads me to believe that a fewer number of larger teams, each helmed by an experienced engineering manager, should way outperform this gaggle of tiny groups. It’s not even close. And they can do so in a way that’s more efficient, sustainable, and humane than this scrappy death march.

And systems thinking shows us why! With fewer groups, but larger ones, you have less overall management overhead, and much less of the slow and costly intra-group coordination. You unlock rich, dense knowledge transfer within groups, which gives you more shared coverage of the surface area. With 7-9 engineers per group you can build a real on call rotation, which means fewer heroics and less burnout. The coordination that you do need to do can be more strategic, less tactical, and much more forward-looking.

Would five big teams ship as many lines of code as 10 small teams, even if five engineers become managers and stop writing code? Probably, but who cares? Your customers give zero fucks how many lines of code you write. They care about whether you are building the right things and solving problems that matter to them. What matters is moving the business forward, not churning out code. Don’t forget, the act of churning out code creates costs and externalities in and of itself.

What defines your velocity is that you spend your time on the right things. Learning to make good decisions about what to build is something every organization has to work out for itself, and it is always an ongoing work in progress. Engineering managers don’t do all the work or make all the decisions, but they are absolutely fucking vital, in my experience, to ensuring that work happens and is done well. As I wrote in my last piece, engineering managers are the embodiment of the feedback loops that systems use to learn and improve.

Are managers ever unnecessary overhead?

Sure, absolutely. Management is about coordinating between teams and helping teams run more optimally, so anything that decreases your need for coordination also decreases your need for management. If you are a small company, or if you have really senior folks who are used to working together, you need a lot less coordination. The next most relevant factor is probably the rate of change; if you’re growing fast or have a lot of turnover, or if there’s a lot of time pressure or frequent shifts in strategy, your need for managers goes up. But there are plenty of smaller orgs out there that are doing just fine without a lot of formal management.

Look, I’m not a fan of the word “overhead”, because a) it’s kind of rude and b) people who call managers “overhead” are typically people who disrespect or do not value the craft of management.

But management is, in fact, overhead. 😅 So is a lot of other glue work! By which I mean the work is important, but does not itself move the business forward; we should do as much of it as absolutely necessary and no more. The nature of glue work is such that it too-easily expands to consume all available time and space (and then some). Constraints are good. Feeling a bit underresourced is good, and should be the norm. It is incredibly easy for management to get a bit bloated, and managers can be very loath to acknowledge this, because it’s not like they ever feel any less stressed or stretched.[*]

Management is also very much like operations work in that when it’s being done well, it’s invisible. Evaluating managers can be very hard, especially in the near term, and making decisions about when it’s time to create or pay down organizational debt is a whole nother ball of wax, and way outside the scope of this post.

But yes, managers can absolutely be unnecessary overhead.

However, if you have 40 engineers all reporting to one VP, and nobody else whose number one job is the outcomes related to people, teams and org, I feel pretty safe in saying this is not a risk for you at this time.

<3 charity

[*] In fact, the reverse can be true; bloated management can create MORE work for managers, and they may counterintuitively feel LESS stretched or stressed with a leaner org chart. Bureaucracies do tend to pick up a momentum all their own. Especially when management gets all wrapped up in promotions and egos. Which is yet another good reason to ensure that management is not a promotion or a form of domination.

 

Questionable Advice: “My boss says we don’t need any engineering managers. Is he right?”

Becoming An Engineering Manager Can Make You Better At Life And Relationships

mipsytipsy, , , , , 31 Comments

Original title: “Why Should You (Or Anyone) Become An Engineering Manager?”

The first piece I ever wrote about engineering management, The Engineer/Manager Pendulum, was written as a love letter to a friend of mine who was unhappy at work. He was an engineering director at a large and fast-growing startup, where he had substantially built out the entire infrastructure org, but he really missed being an engineer and building things. He wasn’t getting a lot of satisfaction out of his work, and he felt like there were other people who might relish the challenge and do it better than he could.

At the same time, it felt like a lot to walk away from! He had spent years building up not only the teams, but also his influence and reputation. He had grown accustomed to being in the room where decisions get made, and didn’t want to give that up or take a big step back in his career. He agonized over this for a long time (and I listened over many whiskeys). 🙂

To me it seemed obvious that his power and influence would only increase if he went back to engineering. You bring your credibility and your relationships along with you, and enthusiasm is contagious. So I wrote the piece with him in mind, but it definitely struck a nerve; it is still the most-read piece I have ever written.

That was in 2017. I’ve written a lot over the years since then about teams and management. For a long time, everything I wrote seemed to come out with a pretty noticeable bias against management, towards engineering:

I go back and read some of those pieces now, and the pervasive anti-manager slant actually makes me a bit uncomfortable, because the environment has changed quite a lot since then.

Miserable managers have miserable reports

When I started writing about engineering management, it seemed like there were a lot of unhappy, resentful, poorly trained managers, lots of whom would prefer to be writing code. Most people made the choice to switch to management for reasons that had nothing to do with the work itself.

  • Becoming a manager was seen as a promotion
  • It was the only form of career progression available at many places
  • Managers made a lot more money
  • It was the only way to get a seat at the table, or be in the loop
  • They were tired of taking orders from someone else

But a lot has changed. The emergence of staff+ engineering has been huge (two new books published in the last five years, and at least one conference!). The industry has broadly coalesced around engineering levels and career progression; a parallel technical leadership track is now commonplace.

We’ve become more aware of how fragile command-and-control systems are, and that you want to engage people’s agency and critical thinking skills. You want them to feel ownership over their labor. You can’t build great software on autopilot, or by picking up jira tasks. Our systems are becoming so complex that you need people to be emotionally and mentally engaged, curious, and continuously learning and improving, both as individuals and as teams.

At the same time, our expectations for managers have gone up dramatically. We’ve become more aware of the damage done by shoddy managers, and we increasingly expect managers to be empathetic, supportive, as well as deeply technical. All of this has made the job of engineering manager more challenging.

Ambitious engineers had already begun to drift away from management and towards the role of staff or principal engineer. And then came the pandemic, which caused managers (poorly supported, overwhelmed, squeezed between unrealistic expectations on both sides) to flee the profession in droves.

It’s getting harder to find people who are willing to be managers. On the one hand, it is fucking fantastic that people aren’t being driven into management out of greed, rage, or a lust for power. It is WONDERFUL that people are finding engineering roles where they have autonomy, ownership, and career progression, and where they are recognized and rewarded for their contributions.

On the other hand, engineering managers are incredibly important and we need them. Desperately.

Good engineering managers are force multipliers

A team with a good engineering manager will build circles around a team without one. The larger or more complex the org or the product, and the faster you want to move, the more true this is. Everybody understands the emotional component, that it feels nice to have a competent manager you trust. But these aren’t just squishy feels. This shit translates directly into velocity and quality. The biggest obstacles to engineering productivity are not writing lines of code too slowly or not working long hours, they are:

  • Working on the wrong thing
  • Getting bogged down in arguments, or being endlessly indecisive
  • Waiting on other teams to do their work, waiting on code review
  • Ramping new engineers, or trying to support unfamiliar code
  • When people are upset, distracted, or unmotivated
  • Unfinished migrations, migrations in flight, or having to support multiple systems indefinitely
  • When production systems are poorly understood and opaque, quality suffers, and firefighting skyrockets
  • Terrible processes, tools, or calendars that don’t support focus time
  • People who refuse to talk to each other
  • Letting bad hires and chronic underperformers stick around indefinitely

Engineers are responsible for delivering products and outcomes, but managers are responsible for the systems and structural support that enables this to happen.

Managers don’t make all the decisions, but they do ensure the decisions get made. They make sure that workstreams are are staffed and resourced sufficiently, that engineers are trained and improving at their craft. They pay attention to the contracts and commitments you have made with other teams, companies or orgs. They advocate for your needs at all levels of the organization. They connect dots and nudge and suggest ideas or solutions, they connect strategy with execution.

Breaking down a complex business problem into a software project that involves the collaboration of multiple teams, and ensuring that every single contributor has work to do that is challenging and pushes their boundaries while not being overwhelming or impossible… is really fucking hard. Even the best leaders don’t get it right every time.

In systems theory, hierarchy emerges for the benefit of the subsystems. Hierarchy exists to coordinate between the subsystems and help them improve their function; it is how systems create resiliency to unknown stressors. Which means that managers are, in a very real way, the embodiment of the feedback loops and meta loops that a system depends on to align itself and all of its parts around a goal, and for the system itself to improve over time.

For some people, that is motivation enough to try being a manager. But not for all (and that’s okay!!). What are some other reasons for going into management?

Why should you (or anyone) be a manager?

I can think of a few good reasons off the top of my head, like…

  • It gets you closer to how the business operates, and gives you a view into how and why decisions get made that translate eventually into the work you do as an engineer
  • Which makes the work feel more meaningful and less arbitrary, I think. It connects you to the real value you are creating in the world.
  • Many people reach a point where they feel a gravitational pull towards mentorship. It’s almost like a biological imperative to replicate yourself and pass on what you have learned to the next generation.
  • Many people also get to a point where they develop strong convictions about what not to do as a manager. They may feel compelled to use what they’ve learned to build happy teams and propagate better practices through the industry
  • One way to develop a great staff engineer is to take a great senior engineer and put them through 2-3 years of management experience.

But the main reason I would encourage you to try engineering management is a reason that I’m not sure I’ve ever heard someone articulate up front, which is that…it can make you better at life and relationships, in a huge and meaningful way.

Work is always about two things: what you put out into the world, and who you become while doing it.

I want to stop short of proclaiming that “being a manager will make you a better person!” — because skills are skills, and they can be used for good or ill. But it can.

It’s a lot like choosing to become a parent. You don’t decide to have kids because it sounds like a hoot (I hope); you go into it knowing it will be hard work, but meaningful work. It’s a way of processing and passing on the experiences that have shaped you and who you are. You also take up the mantle understanding that this will change you — it changes who you are as a person, and the relationships you have with others.

From the outside, management looks like making decisions and calling the shots. From the inside, management looks more like becoming intimately acquainted with your own limitations and motivations and those of others, plus a lot of systems thinking.

Yes, management absolutely draws on higher-level skills like strategy and planning, writing reviews, mediating conflict, designing org charts, etc. But being a good manager — showing up for other people and supporting them consistently, day after day — rests on a bedrock of some much more foundational skills.

The kind of skills you learn in therapy, not in classes

Self-regulation. Can you take care of yourself consistently — sleep, eat, leave the house, socialize, balance your moods, moderate your impulses? As an engineer, you can run your tank dry on occasion, but as a manager, that’s malpractice. You always need to have fuel left in the tank, because you don’t know when it will be called upon.

Self-awareness. Identifying your feelings in the heat of the moment, unpacking where they came from, and deciding how to act on them. It’s not about clamping down on your feelings and denying you have them. It’s definitely not about making your feelings into other people’s problems, or letting your reactions create even bigger problems for your future self. It’s about acting in ways that are fueled by your authentic emotional responses, but not ruled by them.

Understanding other people. You learn to read people and their reactions, starting with your direct reports. You build up a mental model for what motivates someone, what moves them, what bothers them, and what will be extremely challenging for them. You must develop a complex topographical map of how much you can trust each person’s judgment, on which aspect, in any given situation.

Setting good boundaries. Where is the line between supporting someone, advocating for them, encouraging them, pushing them … but not propping them up at all costs, or taking responsibility for their success? How is the manager/report relationship different from the coworker relationship, or the friend relationship? How do you navigate the times when you have to hold someone accountable because their work is falling short?

Sensitivity to power dynamics. Do people treat you differently as a manager than they did as a peer? Are there things that used to be okay for you to say or do that now come across as inappropriate or coercive? How does interacting with your reports inform the way you interact with your own manager, or how you understand what they say?

Hard conversations. Telling people things that you know they don’t want to hear, or things that will make them feel afraid, angry, or upset; and then sitting with their reactions, resisting the urge to take it all back and make everything okay.

The art of being on the same side. When you’re giving someone feedback, especially constructive feedback, it’s easy to trigger a defensive response. It’s SO easy for people to feel like you are judging them or criticizing them. But the dynamic you want to foster is one where you are both side by side, shoulder to shoulder, facing the same way, working together. You are giving them feedback because you care; feedback that could help them be even better, if they choose to accept it. You are on their side. They always have agency.

Did you learn these skills growing up? I sure as hell did not.

I grew up in a family that was very nice. It was very kind and loving and peaceful, but we did not tell each other hard things. When I went off to college and started dating, I had no idea how to speak up when something bothered me. There were sentences that lingered on the tip of my tongue for years but were never spoken out loud, over the ebb and fall of entire relationships. And if somebody raises their voice to me in anger, to this day, I crumble.

I also came painfully late to developing a so-called growth mindset. This is super common among “smart kids”, who get so used to perfect scores and praise for high achievements that any feedback or critique feels like failure…and failure feels like the end of the world.

It was only after I became a manager that I began to consciously practice skills like giving feedback, or receiving constructive criticism, or initiating hard conversations. I had to. But once I did, I started getting better.

Turns out, work is actually kind of an ideal sandbox for life skills, because the social contract is more explicit. These are structured relationships, with rules and conventions and expectations, and your purpose for coming together is clear: to succeed at business, to finish a project, to pull a paycheck. Even the element of depersonalization can be useful: it’s not you-you, it’s the professional version of you, performing a professional role. The stakes are lower than they would be with your mother, your partner, or your child.

You don’t have to be a manager to build these skills, of course. But it’s a great opportunity to do so! And there are tools — books and classes, mentors and review cycles. You can ask for feedback from others. Growth and development is expected in this role.

People skills are persistent

The last thing I will say is this — technical skills do decay and become obsolete, particularly language fluency, but people skills do not. Once you have built these muscles, you will carry them with you for life. They will enhance your ability to connect with people and build trust, to listen perceptively and communicate clearly, in both personal and professional relationships.

Whatever you decide to do with your life, these skills increase your optionality and make you more effective.

And that is the reason I think you should consider being an engineering manager. Like I said, I’ve never heard someone cite this as their reason for wanting to become a manager. But if you ask managers why they do it five or ten years later, you hear a version of this over and over again.

One cautionary note

As an engineer, you can work for a company whose leadership team you don’t particularly respect, whose product you don’t especially love, or whose goals you aren’t super aligned with, and it can be “okay.” Not terrific, but not terrible.

As a manager, you can’t. Or you shouldn’t. The conflicts will eat you up inside and/or prevent you from doing excellent work.

Your job consists of representing the leadership team and their decisions, pulling people into alignment with the company’s goals, and thinking about how to better achieve the mission. As far as your team goes, you are the face of The Man. If you can’t do that, you can’t do your job. You don’t get to stand apart from the org and throw rocks, e.g. “they told me I have to tell you this, but I don’t agree with it”. That does nothing but undermine your own position and the company’s. If you’re going to be a manager, choose your company wisely.

charity

P.S. My friend (from the start of the article) went back to being an engineer, despite his trepidation, and never regretted it once. His career has been up and to the right ever since; he went on to start a company. The skills he built as a manager were a huge boost to an already stellar career. 📈

 

Becoming An Engineering Manager Can Make You Better At Life And Relationships

How to Communicate When Trust Is Low (Without Digging Yourself Into A Deeper Hole)

mipsytipsy, , , 28 Comments

This is based on an internal quip doc I wrote up about careful communication in the context of rebuilding trust. I got a couple requests to turn it into a blog post for sharing purposes; here you go.🌈✨🥂

In this doc I mention Christine, my wonderful, brilliant cofounder and CEO, and the time (years ago) when our relationship had broken down completely, forcing us to rebuild our trust from the ground up.

(Cofounder relationships can be hard. They are a lot like marriages; in their difficulty and intensity, yes, but also in that when you’re doing it with the right person, it’s all worth it. 💜)

Tips for Careful Communication

When a relationship has very little trust, you tend to interpret everything someone says in the worst possible light, or you may hear hostility, contempt, or dismissiveness where none exists. On the other side of the exchange, the conversation becomes a minefield, where it feels like everything you say gets misinterpreted or turned against you no matter how careful you are trying to be. This can turn into a death spiral of trust where every interaction ends up with each of you hardening against each other a little more and filing away ever more wounds and slights. 💔

Yet you HAVE to communicate in order to work together! You have to be able to ask for things and give feedback.

The way trust gets rebuilt is by ✨small, positive interactions✨. If you’re in a trust hole, you can’t hear them clearly, and they can’t hear you (or your intent) clearly. So you have to bend over backwards to overcommunicate and overcompensate.

There are lots of books out there on how to talk about hard topics. (We actually include a copy of “Crucial Conversations” in every new employee packet.) They are all pretty darn cheesy, but it’s worth reading at least one of them.

I’m not going to try and cover all of that territory. What follows is a very subjective list of tactics that worked for Christine and me when we were digging our way out of a massive trust deficit. Power dynamics can admittedly make things more difficult, but the mechanics are the same.

Acknowledge it is hard beforehand:

“I want to say something, but I am having a hard time with it.”
“I have something to say, but I don’t know how you’ll take it.”
“I need to tell you something and I am anxious about your reaction.”

What this does: forces you to slow down and be intentional about the words you’re going to use. It gives the other person a heads up that this was hard for you to say. Most of all, it shows that you do care about their feelings, and are trying to do your best for them (even if you whiff the landing).

… or check in afterwards.

“I’m not sure how that came across. Is there a better way I could have phrased it?”
“In my head that sounded like a compliment…how did you hear it?”
“Did that sound overly critical? I’m not trying to dwell on the past, but I could use your help in figuring out a better way.”

It’s okay if it’s minutes or hours or days later; if it’s still eating at you, ✨clear the fucking air.✨

Speak tentatively.

“Speak tentatively” is the exact opposite of the advice that people (especially women) tend to get in business. But it’s actually super helpful when the relationship is frayed because you are explicitly allowing that they may have a different perception, and making it safer to share it.

“From my perspective, it looks like these results might be missing some data… do you see the same thing?” opens the door for a friendly conversation based on concrete outcomes, whereas “You’re missing data” might sound accusatory and trigger fear and defensiveness.

Try to sound friendly.

Say “please” and “thank you” a lot. Add buffer words like “Hey there”, or “Good morning!” or “lol”. Even just using 🌱emojis🍃 will soften your response to an almost unsettling degree. This may seem almost insultingly simple, but it works. When trust is low, the lack of frills can easily be read as brusque or rude.

Take a breath.

If you are experiencing a physical panic response (sweating, heart racing, etc), announce that you need a few minutes before responding. Compose yourself. Firing off a reply while you are in fight-or-flight mode reliably leads to unintentional escalations.

If you need to take a few beats to read and process, take the time. But empty silence can also generate anxiety 🙂 so maybe say something to indicate “I’m listening, but I need a minute to absorb what you said”, or “I’m still processing”. (We often use “whoa…” as shorthand for this.)

(Alternately, if you find yourself really pissed, “whoa” becomes a great placeholder for yourself to get yourself under control 😬 before saying something you’ll regret having to deal with later.)

“The story in my head”.

When you are in a state where you are assuming the worst of someone and reading hostile intent into their words or actions, try to check yourself on those assumptions.

Repeat the words or behaviors back to them along with your interpretation, like: “The story in my head is that you asked me to send that status email because you don’t trust me to have done the work, or maybe even gathering evidence that I am not performing for a PIP.” This gives them the opportunity to reply and clarify what they actually meant.

Engineer positive interactions, even if you have to invent them.

Relationship experts say that there’s a magic ratio for happy, healthy relationships, which is at least five positive interactions for every one negative interaction. If you only interact with the people you have difficult relationships when you have something difficult to say, you are always going to dread it. Forever.

It might seem artificial at first, but look for chances to have any sort of positive interactions, and seize them.

Communicate positive intent.

In a low trust environment, you can assume everything you say will be read with a voice that is menacing, dismissive or sneering. It behooves you to pay extra attention to tone and voice, and to add extra words that overcommunicate your intended meaning. A neutral statement like “That number seems low”, or “Why is that number low?” will come out sounding brusque and accusatory, e.g. why isn’t that number growing? it’s your fault, you should know this, I blame you, you’re bad at your job. Not might: it will. Try to immunize your communication from distortion by saying things like,

“Hey, I know this just got dropped in your lap, but do you have any idea why this number is so low?”
“This number seems lower than usual. I’m wondering if it’s due to this other thing we tried. Do you have any better ideas?”
“I know it isn’t exactly in your wheel house, but can you help me understand this?”
“I’m new to this system and still trying to figure out how it works. Should this number be going down like this?”

It may seem excessive and time consuming, but it will save you time and effort overall because you will have fewer miscommunications to debug. ☺️

Give people the opening to do better.

We tend to make up our minds about people very quickly, and see them through that lens from then on. It takes work to open our selves up again.

“Assume positive intent” is a laudable goal, but in practice falls short. If every word someone says sounds accusatory or patronizing to you, what are you supposed to do with that advice? Just pretend you don’t hear it, or tell yourself they mean well? That’s not sustainable; your anger will only build up.

But if you can hold just enough space for the idea that they might mean well, then you can give them the opportunity to clarify (and hopefully use different words next time). Like,

Person A: “Why is that number low?”
Person B: “I’m not sure.”
(pause)
Person B: “…. Hey, sorry to interrupt, but the story in my head is that you think owning that number is part of my job, and now you’re upset with me, or you think I’m incompetent at my job.”
Person A: “OMG no, not at all. I’m just trying to figure out who understands this part of the system, since it seems like none of us do! 😃 Sorry for stressing you out!”

and maybe next time it will start off like…

Person A: “Hey, do you have any idea why this number is low? It’s a mystery … nobody I’ve talked to yet seems to know.” 🙂

Remember the handicaps, value the effort.

Ever meet someone you didn’t like online, and realize they’re terrific in person? Online communication loses sooooo much in transit. Christine and I know each other extremely well, and still sometimes we realize we’re reading way too much into each other’s written words. That’s when we try to remember to move it to “mouth words”, aka zoom or phone. Not as good as in person, but eons better than text.

Once you’ve met someone in person, it’s usually easier to read their written words in their voice, too.

Some people just aren’t great at written communication. Some people have neurodiversities that make it difficult for them to hear tone. Some people have English as a second language. And so on. Do give points for effort; if they’re trying, obviously, they care about your experience.

To the best of your ability, try to resist reading layers of meaning into textual communication; keep it simple, overcommunicate intent, and ask for clarity. And if someone is asking you for clarity, help them do a better job for you.

 

How to Communicate When Trust Is Low (Without Digging Yourself Into A Deeper Hole)

Questionable Advice: “How can I drive change and influence teams…without power?”

mipsytipsy, , , , , 1 Comment

Last month I got to attend GOTO Chicago and give a talk about continuous deployment and high-performing teams. Honestly I did a terrible job, and I’m not being modest. I had just rolled off a delayed redeye flight; I realized partway through that I had the wrong slides loaded, and my laptop screen was flashing throughout the talk, which was horribly distracting and means I couldn’t read the speaker notes or see which slide was next. 😵 Argh!

Anyway, shit happens. BUT! I got to meet some longstanding online friends and acquaintances (hi JJ, Avdi, Matt!) and got to eat some of Hillel Wayne’s homemade chocolates, and the Q&A session afterwards was actually super fun.

My talk was about what high performing teams look like and why it’s so important to be on one (spoiler: because this is the #1 way to become a radically better engineer!!). Most of the Q&A topics therefore came down to some version of “okay, so how can I help my team get there?” These are GREAT questions, so I thought I’d capture a few of them for posterity.

But first… just a reminder that the actual best way to persuade people to listen to you is to make good decisions and display good judgment. Each of us has an implicit reputation score, which formal power can only overcome to an extent. Even the most junior engineer can work up a respectable reputation over time, and even principal engineers can fritter theirs away by shooting off at the mouth. 🥰

“how can I drive change when I have no power or influence?”

This first question came from someone who had just landed their first real software engineering job (congrats!!!):

“This is my first real job as a software engineer. One other junior person and myself just formed a new team with one super-senior guy who has been there forever. He built the system from scratch and knows everything about it. We keep trying to suggest ideas like the things you talked about in your talk, but he always shoots us down. How can we convince him to give it a shot?”

Well, you probably can’t. ☺️ Which isn’t the end of the world.

If you’re just starting to write software every day, you are facing a healthy learning curve for the next 3-5 years. Your one and only job is to learn and practice as much you possibly can. Pour your heart and soul into basic skills acquisition, because there really are no shortcuts. (Please don’t get hooked on chatGPT!!)

I know that I came down hard in my talk on the idea that great engineers are made by great teams, and that the best thing most people can do for their career is to join a high-performing, fast-moving team. There will come a time where this is true for you too, but by then you will have skills and experience, and it will be much easier for you to find a new job, one with a better culture of learning.

It is hard to land your first job as a software engineer. Few can afford to be picky. But as long as you are a) writing code every day, b) debugging code every day, and c) getting good feedback via code reviews, this job will get you where you need to go. When you’re fluent and starting to mentor others, or getting into higher level architecture work, or when you’re starting to get bored … then it’s time to start looking for roles with better teachers and a more collaborative team, so your growth doesn’t stall. (Please don’t fall into the Trap of the Premature Senior.)

This is an apprenticeship industry. You’re like a med student right now, who is just starting to do rounds under the supervision of an attending physician (your super-senior engineer). You can kinda understand why he isn’t inclined to listen to your opinions on his choice of stethoscope or how he fills out a patient chart. A better teacher would take time to listen and explain, but you already know he isn’t one. 🤷

I only have one piece of advice. If there’s something you want to try, and it involves doing engineering work, consider tinkering around and building it after hours. It’s real hard to say no to someone who cares enough to invest their own time into something.

“how can I drive change when I am a tech lead on a new team?”

“I have the same question! — except I’m a tech lead, so in theory I DO have some power and influence. But I just joined a new team, and I’m wondering what the best way is to introduce changes or roll them out, given that there are soooo many changes I’d like to make.”

(I wrote a somewhat scattered post a few years ago on engineers and influence, or influence without authority, which covers some related territory.)

As a tech lead who is new to a team, busting at the seams with changes I want to make, here’s where I’d start:

  1. Understand why things are the way they are and get to know the personalities on your team a bit before you start pitching changes. (UNLESS they are coming to you with arms outstretched, pleading desperately for changes ~fast~ because everything is on fire and they know they need help. This does happen!)
  2. Spend some time working with the old systems, even if you think you already understand. It’s not enough for you to know; you need to take the team on this journey with you. If you expect your changes to be at all controversial, you need to show that you respect their work and are giving it a chance.
  3. Change one thing at a time, and go for the developer experience wins first. Address things that will visibly pay off for your team in terms of shipping faster, saving time, less frustration. You have no credibility in the beginning, so you want to start racking up wins before you take on the really hard stuff.
  4. Roll up your sleeves. Nothing buys a leader more goodwill than being willing to do the scut work. Got a flaky test suite that everybody has been dreading trying to fix? I smell opportunity…
  5. Pitch it as an experiment. If people aren’t sold on your idea for e.g. code review SLAs, ask if they’d be willing to try it out for three weeks just as an experiment.
  6. Strategically shop it around to the rest of the team, if you sense there will be resistance…

At this point in my answer 👆 I outlined a technique for persuading a team and building support for a plan or an idea, especially when you already know it’s gonna be an uphill battle. Hillel Wayne said I should write it up in a blog post, so here it is! (I’ll do anything for free chocolate 😍)

“How can I get people on board with my controversial plan?”

So you have a great idea, and you’re eager to get started. Awesome!!! You believe it’s going to make people’s lives better, even though you know you are going to have to fight tooth and nail to make it happen.

What NOT to do:

Walk into the team meeting and drop your bomb idea on everyone cold:

“Hey, I think we should stop shipping product changes until we fix our build pipeline to the point where we can auto-deploy each merge set to production, one at a time, in under an hour.” ~ (for example)

…. then spend the rest of the hour grappling with everybody’s thoughts, feelings, and intense emotional reactions, before getting discouraged and slinking away, vowing to never have another idea, ever again.

What to do instead:

Suss out your audience. Who will be there? How are they likely to react? Are any of them likely to feel especially invested in the existing solution, maybe because they built it? Are any of them known for their strong opinions or being combative?

Great!!! Your first move is to have a conversation with each of them. Approach them in the spirit of curiosity, and ask what they think of your idea. Talking with them will also help you hash out the details and figure out if it is actually a good idea or not.

Your goal is to make the rounds, ask for advice, identify any allies, and talk your idea through with anybody who is likely to oppose you…before the meeting where you intend to unveil your plan. So that when that happens, you have:

  1. given people the chance to process their reactions and ask questions in private
  2. ensured that key people will not feel surprised, threatened, or out of the loop
  3. already heard and discussed any objections
  4. ideally, you have earned their support!

Even if you didn’t manage to convince every person, this was still a valuable exercise. By approaching people in advance, you are signaling that you respect them and their voice matters. You are always going to get people’s absolute worst reactions when you spring something on them in a group setting; any anxiety or dismay will be amplified tenfold. By letting them reflect and ask questions in private, you’re giving time for their better selves to emerge.

What to do instead…if you’re a manager:

As an engineer or a tech lead, you sometimes end up out front and visible as the owner of a change you are trying to drive. This is normal. But as a manager, there are far more times when you need to influence the group but not be the leader of the change, or when you need to be wary of sounding like you are telling people what to do. These are just a few of the many reasons it can be highly effective to have other people arguing on your behalf.

In the ideal scenario, particularly on technical topics, you don’t have to push for anything. All you do is pose the question, then sit back and listen as vigorous debate ensues, with key stakeholders and influential engineers arguing for your intended outcome. That’s a good sign that not only are they convinced, they feel ownership over the decision and its execution. This is the goal! 🌈

It’s not just about persuading people to agree with you, either. Instead of having a shitty dynamic where engineers are attached to the old way of doing things and you are “dragging them” into the newer ways against their will, you are inviting them to partner with you. You are offering them the opportunity to lead the team into the brave new world, by getting on board early.

(It probably goes without saying, but always start with the smallest relevant group of stakeholders, and not, say, all of engineering, or a group that has no ownership over the given area. 🙃 And … even this strategy will stop working rather quickly, if your controversial ideas all turn out to be disastrous. 😉)

“How do I know where to even start?!? 😱”

Before I wrap up, I want to circle back to the question from the tech lead about how to drive change on a team when you do have some influence or power. He went on to say (or maybe this was from a third questioner?*):

“There is SO MUCH I’d like to do or change with our culture and our tech stack. Where can I even start??”

Yeah, it can be pretty overwhelming. And there are no universal answers… as you know perfectly well, the answer is always “it depends.” ☺️ But in most cases you can reduce the solution space substantially to one of the two following starting points.

1. Can you understand what’s going on in your systems? If not, start with observability.

It doesn’t have to be elegant or beautiful; grepping through shitty text logs is fine, if it does the trick. But do any of the following make you shudder in recognition?:

  • If I get paged, I might lose the rest of the afternoon trying to figure out what happened
  • Our biggest problem is performance and we don’t know where the time is going
  • We have a lot of flaky, flappy alerts, and unexplained outages that simply resolve themselves without our ever truly understanding what happened.

If you can’t understand what’s going on in your system, you have to start with instrumentation and observability. It’s just too deadly, and too risky, not to. You’re going to waste a ton of time stabbing around in the dark trying to do anything else without visibility. Put your glasses on before you start driving down the freeway, please.

2. Can you build, test and deploy software in under an hour? If not, start with your deploy pipeline.

Specifically, the interval of time between when the code is written and when it’s being used in production. Make it shorter, less flaky, more reliable, more automated. This is the feedback loop at the heart of software engineering, which means that it’s upstream from a whole pile of pathologies and bullshit that creep in as a consequence of long, painful, batched-up deploys.

Here’s a talk I’ve given a few times on why this matters so much:

You pretty much can’t fail with one of those two; your lives will materially improve as you make progress. And the iterative process of doing them will uncover a great deal of shit you should probably know about.

Cheers! 🥂

charity.

* My apologies if I remembered anyone’s question inaccurately!

Questionable Advice: “How can I drive change and influence teams…without power?”

Questionable Advice: “People Used To Take Me Seriously. Then I Became A Software Vendor”

mipsytipsy, , , Leave a comment

I recently got a plaintive text message from my magnificent friend Abby Bangser, asking about a conversation we had several years ago:

“Hey, I’ve got a question for you. A long time ago I remember you talking about what an adjustment it was becoming a vendor, how all of a sudden people would just discard your opinion and your expertise without even listening. And that it was SUPER ANNOYING.

I’m now experiencing something similar. Did you ever find any good reading/listening/watching to help you adjust to being on the vendor side without being either a terrible human or constantly disregarded?”

Oh my.. This brings back memories. ☺️🙈

Like Abby, I’ve spent most of my career as an engineer in the trenches. I have also spent a lot of time cheerfully talking smack about software. I’ve never really had anyone question my experience[1] or my authority as an expert, hardened as I was in the flames of a thousand tire fires.

Then I started a software company. And all of a sudden this bullshit starts popping up. Someone brushing me off because I was “selling something”, or dismissing my work like I was fatally compromised. I shrugged it off, but if I stopped to think, it really bothered me. Sometimes I felt like yelling “HEY FUCKERS, I am one of your kind! I’m trying to HELP YOU. Stop making this so hard!” 😡 (And sometimes I actually did yell, lol.)

That’s what I remember complaining to Abby about, five or six years ago. It was all very fresh and raw at the time.

We’ll get to that. First let’s dial the clock back a few more years, so you can fully appreciate the rich irony of my situation. (Or skip the story and jump straight to “Five easy ways to make yourself a vendor worth listening to“.)

The first time I encountered “software for sale”

My earliest interaction with software vendors was at Linden Lab. Like most infrastructure teams, most of the software we used was open source. But somewhere around 2009? 2010? Linden’s data engineering team began auditioning vendors like Splunk, Greenplum, Vertica[2], etc for our data warehouse solution, and I tagged along as the sysinfra/ops delegate.

For two full days we sat around this enormous table as vendor after vendor came by to demo and plump their wares, then opened the floor for questions.

One of the very first sales guys did something that pissed me off me. I don’t remember exactly what happened — maybe he was ignoring my questions or talking down to me. (I’m certain I didn’t come across like a seasoned engineering professional; in my mid twenties, face buried in my laptop, probably wearing pajamas and/or pigtails.) But I do remember becoming very irritated, then settling in to a stance of, shall we say, oppositional defiance.

I peppered every sales team aggressively with questions about the operational burden of running their software, their architectural decisions, and how canned or cherry-picked their demos were. Any time they let slip a sign of weakness or betrayed uncertainty, I bore down harder and twisted the knife. I was a ✨royal asshole✨. My coworkers on the data team found this extremely entertaining, which only egged me on.

What the fuck?? 🫢😧🫠 I’m not usually an asshole to strangers.. where did that come from?

What open source culture taught me about sales

I came from open source, where contempt for software vendors was apparently de rigueur. (is it still this way?? seems like it might have gotten better? 😦) It is fascinating now to look back and realize how much attitude I soaked up before coming face to face with my first software vendor. According to my worldview at the time,

  1. Vendors are liars
  2. They will say anything to get you to buy
  3. Open source software is always the safest and best code
  4. Software written for profit is inherently inferior, and will ultimately be replaced by the inevitable rise of better, faster, more democratic open source solutions
  5. Sales exists to create needs that ought never to have existed, then take you to the cleaners
  6. Engineers who go work for software vendors have either sold out, or they aren’t good enough to hack it writing real (consumer facing) software.

I’m remembering Richard Stallman trailing around behind me, up and down the rows of vendor booths at USENIX in his St IGNUcious robes, silver disk platter halo atop his head, offering (begging?) to lay his hands on my laptop and bless it, to “free it from the demons of proprietary software.” Huh. (Remember THIS song? 🎶 😱)

Given all that, it’s not hugely surprising that my first encounter with software vendors devolved into hostile questioning.

(It’s fun to speculate on the origin of some of these beliefs. Like, I bet 3) and 4) came from working on databases, particularly Oracle and MySQL/Postgres. As for 5) that sounds an awful lot like the beauty industry and other products sold to women. 🤭)

Behind every software vendor lies a recovering open source zealot(???)

I’ve had many, many experiences since then that slowly helped me dismantle this worldview, brick by brick. Working at Facebook made me realize that open source successes like Apache, Haproxy, Nginx etc are exceptions, not the norm; that this model is only viable for certain types of general-purpose infrastructure software; that governance and roadmaps are a huge issue for open source projects too; and that if steady progress is being made, at the end of the day, somewhere somebody is probably paying those developers.

I learned that the overwhelming majority of production-caliber code is written by somebody who was paid to write it — not by volunteers. I learned about coordination costs and overhead, how expensive it is to organize an army of volunteers, and the pains of decentralized quality control. I learned that you really really want the person who wrote the code to stick around and own it for a long time, and not just on alternate weekends when they don’t have the kids (and/or they happen to feel like it).

I learned about game theory, and I learned that sales is about relationships. Yes, there are unscrupulous sellers out there, just like there are shady developers, but good sales people don’t want you to walk away feeling tricked or disappointed any more than you want to be tricked or disappointed. They want to exceed your expectations and deliver more value than expected, so you’ll keep coming back. In game theory terms, it’s a “repeated game”.

I learned SO MUCH from interviewing sales candidates at Honeycomb.[3] Early on, when nobody knew who we were, I began to notice how much our sales candidates were obsessed with value. They were constantly trying to puzzle out out how much value Honeycomb actually brought to the companies we were selling to. I was not used to talking or thinking about software in terms of “value”, and initially I found this incredibly offputting (can you believe it?? 😳).

Sell unto others as you would have them sell unto you

Ultimately, this was the biggest (if dumbest) lesson of all: I learned that good software has tremendous value. It unlocks value and creates value, it pays enormous ongoing dividends in dollars and productivity, and the people who build it, support it, and bring it to market fully deserve to recoup a slice of the value they created for others.

There was a time when I would have bristled indignantly and said, “we didn’t start honeycomb to make money!” I would have said that the reason we built honeycomb because we knew as engineers what a radical shift it had wrought in how we built and understood software, and we didn’t want to live without it, ever again.

But that’s not quite true. Right from the start, Christine and I were intent on building not just great software, but a great software business. It wasn’t personal wealth we were chasing, it was independence and autonomy — the freedom to build and run a company the way we thought it should be run, building software to radically empower other engineers like ourselves.

Guess what you have to do if you care about freedom and autonomy?

Make money. 🙄☺️

I also realized, belatedly, that most people who start software companies do so for the same damn reasons Christine and I did… to solve hard problems, share solutions, and help other engineers like ourselves. If all you want to do is get rich, this is actually a pretty stupid way to do that. Over 90% of startups fail, and even the so-called “success stories” aren’t as predictably lucrative as RSUs. And then there’s the wear and tear on relationships, the loss of social life, the vicissitudes of the financial system, the ever-looming spectre of failure … 👻☠️🪦 Startups are brutal, my friend.

Karma is a bitch

None of these are particularly novel insights, but there was a time when they were definitely news to me. ☺️ It was a pretty big shock to my system when I first became a software vendor and found myself sitting on the other side of the table, the freshly minted target of hostile questioning.

These days I am far less likely to be cited as an objective expert than I used to be. I see people on Hacker News dismissing me with the same scornful wave of the hand as I used to dismiss other vendors. Karma’s a bitch, as they say. What goes around comes around. 🥰

I used to get very bent out of shape by this. “You act like I only care because I’m trying to sell you something,” I would hotly protest, “but it’s exactly the opposite. I built something because I cared.” That may be true, but it doesn’t change the fact that vested interests can create blind spots, ones I might not even be aware of.

And that’s ok! My arguments/my solutions should be sturdy enough to withstand any disclosure of personal interest. ☺️

Some people are jerks; I can’t control that. But there are a few things I can do to acknowledge my biases up front, play fair, and just generally be the kind of vendor that I personally would be happy to work with.

Five easy ways to make yourself a vendor worth listening to

So I gave Abby a short list of a few things I do to try and signal that I am a trustworthy voice, a vendor worth listening to. (What do you think, did I miss anything?)

🌸 Lead with your bias.🌸
I always try to disclose my own vested interest up front, and sometimes I exaggerate for effect: “As a vendor, I’m contractually obligated to say this”, or “Take it for what you will, obviously I have religious convictions here”. Everyone has biases; I prefer to talk to people who are aware of theirs.

🌸 Avoid cheap shots.🌸
Try to engage with the most powerful arguments for your competitors’ solutions. Don’t waste your time against straw men or slam dunks; go up against whatever ideal scenarios or “steel man” arguments they would muster in their own favor. Comparing your strengths vs  their strengths results in a way more interesting, relevant and USEFUL discussion for all involved.

🌸 Be your own biggest critic.🌸
Be forthcoming about the flaws of your own solution. People love it when you are unafraid to list your own product’s shortcomings or where the competition shines, or describe the scenarios where other tools are genuinely superior or more cost-effective. It makes you look strong and confident, not weak.

What would you say about your own product as an engineer, or a customer? Say that.

🌸 You can still talk shit about software, just not your competitors‘ software. 🌸
I try not to gratuitously snipe at our competitors. It’s fine to speak at length about technical problems, differentiation and tradeoffs, and to address how specifically your product compares with theirs. But confine your shit talking to categories of software where you don’t have a personal conflict of interest.

Like, I’m not going to get on twitter and take a swipe at a monitoring vendor (anymore 😇), but I might say rude things about a language, a framework, or a database I have no stake in, if I’m feeling punchy. ☺️ (This particular gem of advice comes by way of Adam Jacob.)

🌸 Be generous with your expertise.🌸
If you have spent years going deep on one gnarly problem, you might very well know that problem and its solution space more thoroughly than almost anyone else in the world. Do you know how many people you can help with that kind of mastery?! A few minutes from you could potentially spare someone days or weeks of floundering. This is a gift few can give.

It feels good, and it’s a nice break from battering your head against unsolvable problems. Don’t restrict your help to paying customers, and, obviously, don’t give self-serving advice. Maybe they can’t buy/don’t need your solution today, but maybe someday they will.

In conclusion

There’s a time and place for being oppositional. Sometimes a vendor gets all high on their own supply, or starts making claims that aren’t just an “optimistic” spin on the facts but are provably untrue. If any vendor is operating in poor faith they deserve to to be corrected.

But it’s a shitty, self-limiting stance to take as a default. We are all here to build things, not tear things down. No one builds software alone. The code you write that defines your business is just the wee tippy top of a colossal iceberg of code written by other people — device drivers, libraries, databases, graphics cards, routers, emacs. All of this value was created by other people, yet we collectively benefit.

Think of how many gazillion lines of code are required for you to run just one AWS Lambda function! Think of how much cooperation and trust that represents. And think of all the deals that brokered that trust and established that value, compensating the makers and allowing them to keep building and improving the software we all rely on.

We build software together. Vendors exist to help you. We do what we do best, so you can spend your engineering cycles doing what you do best, working on your core product. Good sales deals don’t leave anyone feeling robbed or cheated, they leave both sides feeling happy and excited to collaborate.[4]

🐝💜Charity.

[1] Yes, I know this experience is far from universal; LOTS of people in tech have not felt like their voices are heard or their expertise acknowledged. This happens disproportionately to women and other under-represented groups, but it also happens to plenty of members of the dominant groups. It’s just a really common thing! However that has not really been my experience — or if it has, I haven’t noticed — nor Abby’s, as far as I’m aware.

[2] My first brush with columnar storage systems! Which is what makes Honeycomb possible today.

[3] I have learned SO MUCH from watching the world class sales professionals we have at Honeycomb. Sales is a tough gig, and doing it well involves many disciplines — empathy, creativity, business acumen, technical expertise, and so much more. Selling to software engineers in particular means you are often dealing with cocky little shits who think they could do your job with a few lines of code. On behalf of my fellow little shits engineers, I am sorry. 🙈

[4] Like our sales team says: “Never do a deal unless you’d do both sides of the deal.” I fucking love that.

Questionable Advice: “People Used To Take Me Seriously. Then I Became A Software Vendor”

Giving Good Feedback: Consider the Ratio

mipsytipsy, , , , 2 Comments

Consider the ratio.

You work with someone great. If someone asked, you’d say they are brilliant, inspired and dedicated. They care deeply about their work, they are timely and reliable (for the most part), and their emojis and dry sense of humor brighten your day. Your work depends on theirs, and you are working together on a neat project which is generating lots of excitement at demo days. You would miss them terribly if they left.

But today you are annoyed. They either didn’t hear or forgot your feedback from the last design review, which means you have to redo some components you thought were finished. It’s a considerable amount of work, and this isn’t the first (or second) time, either. You want to tell them so and try to debug this so it doesn’t keep happening.

So far, so good. Giving feedback like this can be hard, especially if they are senior to you. But do they understand the totality of how you see them? Or was the last time they heard from you the last time they fucked up? Out of the last ten times you gave them feedback, how many were complaining or asking for changes? Does that feedback ratio accurately represent your perception of their value?

This doesn’t mean you have to run around saying “you’re amazing!” all the time, but do be mindful of how other people think you perceive them. I can pretty much guarantee that none of the people you love working with realize just how much you value them, but they are acutely aware of all the ways they fall short or fail you. Here are some ways to correct that imbalance a bit

  • Don’t be vague. Do be specific. If you just run around saying “You’re awesome!” to people, they will tune you out. Do try to notice and reflect some of the things that making working with them a joy. Like, “I learned so much about mysql indexes pairing with you today, thank you”, or “Last week in our practice session you suggested approaching it this way, and it was so helpful in my situation”, or “I really admire the way you can talk extemporaneously about $topic, and I LOVE knowing I can rely on you with zero prep”. This is harder, and it absolutely takes more work on your part, but it lands. And sticks.
  • Use the Situation, Behavior, Impact framework…but for praise. The SBI framework is designed for delivering hard feedback, but it works just as well for delivering kudos. Use it to give great praise that isn’t generic and does let people know what they’re doing right/what they mean to you. “In the last team meeting, your overview of the messaging framework was super eye-opening for me. I learned more than ever before about not just our pyramid, but how messaging frameworks in general are used. I understand its impact on my role better now than I have in seven years of product marketing!”
  • Ground critique in your overall reaction. Let’s say someone just presented an idea that you think is super interesting and potentially very high value, but you have questions about its impact on marginalized groups. Do they know you think it is interesting and high value, when you launch into your critique? No they do not. If all they hear is several rounds of criticism, they may very well give up and cancel altogether, thinking everyone hated it. Something as simple as starting with “I LOVE this idea. Have you thought about —”, or “This is really interesting, but I’m curious…” can be enough to convey a less discouraging, more accurate sense of your perspective.
  • Don’t hold out for the “wow” moments. Sometimes even sharing what you see as a neutral description of someone’s work can be mind-blowing and affirming. Most people don’t realize how much they are just noticed, full stop. It is flattering to be noticed or have the things you said remembered. Being seen can be enough. (h/t @eanakashima)
  • Don’t contribute to a pile-on. Feedback is asymmetric — you can only give feedback as one person at a time (you!), but the recipient might be grappling with negative feedback from many, many people. In that context, anything critical you say is likely to feel like one more rock in a public stoning. Or (somewhat less dramatically), if someone asks for feedback and receives a wave of criticism, they may feel deflated and defeated and drop the entire idea. If that isn’t the outcome you want, try to bring some positive balance to the discussion instead of piling on.
  • Give feedback to grow on. Pure positivity can sound cloying and be easy to discount. If you’re just praising me, I’m learning nothing from it. We’re not talking about a shit sandwich here, but the best compliments are the ones you learn something from. “That was GREAT. It might be even better if…” Relatedly, some people find it hard to believe purely positive feedback, but if you give feedback that shows you understand their work and what they did less well, you gain credibility and they will believe the praise. (h/t @inert_wall).

Hard conversations and corrective feedback are absolutely necessary at times. But even poorly-delivered critiques can be dealt with in the context of a good relationship, when the person knows how much you value them, and even the most delicately delivered criticism can be hard to hear from someone when all you ever seem to hear from them is how much you suck.

Engineers can be the worst at this, because we tend to show our interest by eagerly engaging with an idea or piece of work … by picking it apart, and chattering about all the ways it could be better. 🙃 I generally think this is an awesome way to show love, but we could stand to be clearer about the affection part, and not let the perfect be the enemy of the good. So please consider the ratio of critique vs affirmation when giving feedback.

And there’s no reason to save all the nice words and praise and gratitude for someone’s funeral (or when they leave the company ☺️).

Giving Good Feedback: Consider the Ratio

Questionable Advice: Is there a path back from CTO to engineer?

mipsytipsy, , 4 Comments

I received this question in the comments section of my piece on The Twin Anxieties of the Engineer/Manager Pendulum, and figured I might as well answer it. It definitely isn’t a question I’ve spent a lot of time thinking about or anything. 🥰

As a former CTO coming off a sabbatical and wanting to go back to engineering, it’s good to hear that this can be done. Having had coding, architecting, and scaling skills before getting pushed to more lead role and looking to get back to working after the sabbatical, what would the roadmap look like to achieve this? Is it still possible having been away for a few years? What would be a good role to target for re-entry: principal/staff engineer? architect? — Mark

Personally? If I were you, I would return to engineering as a regular old software engineer, writing and shipping code every day in the trenches with (this cannot be emphasized enough) a really, really good team.

Your rustiest skill sets are always going to be the most tactical ones — writing software, reviewing code, reproducing bugs, understanding a new production system.

As a former CTO, you have many other skill sets to pull from — management, strategy, architecture, coaching, staffing, fundraising, etc. These skills are valuable. But they don’t degrade the way hands-on development does. You’ll still remember how to write a performance review two (or twenty) years from now, but writing code is like speaking a language: you use it or lose it. And just like with a language, the best way to freshen up is full immersion.

It’s not just about refreshing your technical chops, it’s also about re-acclimating yourself to the rhythms of hours, days, and weeks spent in focus mode, building and creating.

Think back to the time you first moved from engineering into a management role. Do you remember how exhausting and intrusive it was at first, having meeting after meeting after meeting on your calendar? You had to context switch twenty times a day — you were context switching constantly. You had to walk into room after room after room full of people and their words and emotions. By the end of the day you would be drained dry (and the days felt so long).

As an engineer, you spent your days in stretches of deep focus and concentration, punctuated by the occasional meal, meeting or interruption. But as a manager, your life is nothing but interruptions (and time boxes, and time-boxed interruptions). It took time to for you adjust to manager life and learn where to seek out new dopamine hits. And it’s going to take time for you to adjust back.

How much time? About six months, at least for me. I don’t think it’s being overly dramatic to say that you have to allow enough time to become a different version of yourself. You can’t just change personas and entire ways of being like you change your clothing. The process is more like…a snake shedding its skin, or a caterpillar turning into a butterfly. Don’t rush the process.

And don’t just pick up where you left off as an engineer. This is a beautiful opportunity for you to enjoy the terrible frustration of beginner eyes. ☺️ Learn a new language, learn a new framework, learn a new way of packaging and deploying your code. Freshen up your toolchain. Try a new editor. Catch up on some new testing or validation ideas that have developed or gone mainstream since you were last in the coal mines. (Take modern observability for a test drive? 😉)

Shit moves fast. A lot will have changed.

If you try to pick up where you left off as an expert, it’s really going to suck as you stumble through the motions that used to feel effortless and automatic. But if you start with something new, the friction of learning will feel ordinary and predictable instead. And pretty soon you’ll feel the engine start to kick in: the accelerated learning curve you’ll remember from learning a new technical skill for the 50,000th time.

Because it’s not just about refreshing your technical skills and your daily cadence, either… it’s also about reconnecting with your curiosity, and your attachment to (and love for) technology.

And you better fucking love it, if you plan to inflict the world of agonies that is software development on yourself day after day. 😭 So you have to reconnect with that dopamine drip you get from learning things, fixing shit, and solving problems. And you have to reconnect with the emotional intensity of shipping code that will impact people’s lives — for better or for worse — and of being personally responsible for that code in production. Of knowing viscerally what it’s like to ship a diff that brings production down, or wakes up your coworker in the middle of the night, or corrupts user data.

So yes. It is absolutely possible to return to engineering after a few years away. And yeah, you could come back as a principal or staff engineer. Someone will definitely hire you. However, I suggest otherwise. I suggest you come back as a senior engineer, writing software every day, for a good 6-9 months.

Your grounding in the technical challenges and solution space will be much deeper and richer if you come back hands on than if you came in at a higher level, detached from the rhythms of daily development. Working closer to production and closer to users will give you the chance to rebuild the intense empathy and connectedness to your work and team that tends to seep away the higher you go up the food chain. You’ll be more confident in yourself as a technologist if you honor your need to relearn and rebuild. And you will earn much more respect from your fellow engineers this way. Engineers respect people who respect what they do.

It’s better than jumping straight into the role of a staff+ engineer and trying to refresh your tactical/technical skills on the side. And you’ll be an infinitely more effective staff+ engineer once you’ve done the refreshing.

But if it feels like a demotion, or it’s too hard to swallow, or if the politics of promotions at this company make you leery: compromise by getting yourself hired as a staff or principal engineer, while being clear with your hiring manager that you plan to spend the first 6+ months slinging diffs. They should be fine with it (delighted, really) since a) ANY staff+ hire is an investment for the long run, b) this is a great way to onboard any staff+ engineer, and c) I don’t believe anybody can actually have staff+ level impact during their first 6-12 months at a company, since so much of the job has to do with people, process, technical context, systems history, etc which accrues over time.

Have fun, and do report back! Tell us how it goes!

charity.

P.S.: You don’t say how long it’s been, but I’m operating under the assumption that it’s been 5-10 years since you last worked as an engineer.

P.P.S.: 🚨unsolicited opinion alert🚨 I would personally avoid any role that includes “Architect” in its title (except solutions architects). To me, “software architect” rings of “someone who can no longer write code or perform as a software engineer, who has probably been at the same company for so long that their skills and knowledge now consist entirely of minutiae about that particular company’s technology. likely to be useless and/or helpless at any other company.” I say this with all due apologies to my architect friends, every one of whom is technically dazzling, operationally up-to-date, and has beautiful hair.💆 🥰

 

 

Questionable Advice: Is there a path back from CTO to engineer?

How can you tell if the company you’re interviewing with is rotten on the inside?

mipsytipsy, , , , 15 Comments

How can you tell the companies who are earnestly trying to improve apart from the ones who sound all polished and healthy from the outside, whilst rotting on the inside?

This seems to be on a lot of minds right now, what with the Great Resignation and all. There are no perfect companies, just like there are no perfect relationships; but there are many questions and techniques you can use to increase your confidence that a particular company is decent and self-aware, one whose quirks and foibles you are compatible with.

https://twitter.com/jetpack/status/1486212123525287938

Interviews are designed to make you feel like you are under inspection, like the interviewer holds all the power. This is an illusion. Your labor is valuable — it is vital — and you should be scrutinizing them every bit as closely as they you. In fact, here is Tip #1:

  • If they allow you plenty of time to converse with your interviewers throughout the process, great. If they tack on a cursory “any questions for us?” while wrapping up, they don’t think it matters what you think of them. Pull the ripcord.

Collect and practice good interview questions for you to ask potential employers. Write them down — your mind is likely to go blank under stress, and you don’t want to let them off the hook. There is a LOT of signal to be gained by probing down below the surface answers.

Backchanneling

  1. Whisper networks and backchannels are incredibly important. It can be especially valuable to talk to someone who has recently left the company: why did they leave? Would they go there again?
  2. Alternately, do you know anyone who has worked for or with their leadership, even if not at that company?
  3. If you know any women or under-represented minorities (URM) who work there, buy them lunch and ask for the unvarnished truth. That’s where you usually turn up the real dirt. 🥂

Diversity, equity and inclusion

Just because a company has a diverse workforce doesn’t necessarily mean it is a healthy place to work. (But it’s fair to give some points up front, because that doesn’t usually happen by accident.)

  1. Do they have a diverse leadership team? A diverse board?
  2. Is their company diverse overall, or are minorities concentrated in a few (lower-paying, high-turnover) departments?
  3. You might not want to write off all the companies that don’t meet points one and two, if for no other reason than it dramatically shrinks your available option pool. If they don’t have a particularly diverse team, and this is something that matters to you, that’s your cue to dig deeper:
    • Are they bothered by their lack of diversity? What’s the plan? Do they just feel generically sad about it, or have they set specific goals to improve by specific dates? What investments are they making?
    • Who works on DEI stuff currently? (Answers like “HR and recruiting”, or “we have a woman who’s really good at it” are bad answers.)
    • Who is accountable for making sure those goals are hit? (The only right answer is “our execs”. Having a “chief diversity officer” is an anti-pattern in my book.)
    • If the team is all guys, for example, ask if they’ve ever had any women on the team in the past. Did she/they leave? Do they know why?
  4. This is a GREAT one: “As a white man, I’d ask what they’ve done to find qualified women and minorities for the role I’m interviewing for.” (via David Daly) 🔥🔥

Company stuff

  1. What are their values? Do they feel bloodless and ripped from the pages of HBR, or are they unique, lived-in, and give you a glimpse of what the people there care about? Are they mentioned over the course of your interview?
  2. Ask tough questions about the business and try to ascertain whether they are hitting their quarterly goals, how much funding they have in the bank, what the growth curve looks like, what users really think about their product, and what the biggest obstacles to success are.
    • Companies that are floundering are going to be really stressful places to work, and even if the leadership is decent, they may find themselves backed into making some really tough decisions.
    • You want to work at a company on a strong growth trajectory for lots of reasons, but a big one is your own growth potential. You will learn the most the fastest at places that are growing fast, and have way more openings for promotions and leadership roles than a slower-growing company.
  3. Are people willing to speak freely about things they’ve tried that have failed, and things that don’t work well currently? Being self-aware and comfortable with visible failure are two of the most important self-correcting mechanisms a company can cultivate.
  4. EVERYBODY thinks they value transparency, so I wouldn’t even bother asking. Instead, ask for specific examples of leadership being forthcoming with bad news to the team, and team members delivering hard feedback or bad news to upper leadership. Transparency shouldn’t be something they’re especially proud of, so much as it is taken for granted. It’s in the air that you breathe.

Planning and the unplanned

  1. Ask about how decisions get made. A chestnut is, “how does work end up on my plate?” — meaning is there a business strategy (owned by whom?), a technical strategy, a product strategy, quarterly KPIs, customer requests, manager delegation, JIRA tickets…? (The most important part may be how similar the answers you get are. 🙃)
  2. How often does work get pre-empted and why? It’s a good thing if product development has to get put on ice once in a while so the team can focus on reliability and maintenance work. It’s a bad thing if they’re expected to stuff reliability work in the cracks around their product development, or if they’re incapable of sticking to a plan.
  3. What does “crunch time” look like? Nearly every company has one from time to time (it might even be a bad sign if it never happens), but this is when you find out your leadership’s true colors.
    • Do they praise people or call them out to thank them for pulling all nighters and other extremist behaviors? 🚨BZZT🚨
    • Is it voluntary? Are you trusted to set your own pace, your own limits, or are you pressured to do more? Are people expected to participate to the extent that they are able, and not expected to justify how hard or how much (so long as they communicate their capacity, of course)?
    • How long did it last, and how often does it happen, and why? It should be rare (1-2x/year at most), involve the whole company, and move the business forward meaningfully
    • Did they follow through by making sure people took time off afterwards to recover? Not just give permission, but actually make sure the human beings had a chance to refresh themselves? Did leaders set a good example by taking a breather themselves?

Believe it or not, crunch time done correctly can be an enormously exciting, intense, bonding time for a group of people who love what they do, culminating in a surge of collective triumph and celebration, followed by recovery time. If it was done correctly, and you ask about it afterwards, people will 💡light up💡.

Team stuff

  1. Unfortunately, culture can vary widely from function to function, even from manager to manager. Make sure you get a real interview slot with your actual manager — not just a screener or wrap-up call — and as much of the team as possible, too.
  2. Ask your potential manager about the last person they had to let go. Why? What was the process? What was the impact on the team? How did the person feel afterwards?
  3. Who is on call? How often do people get paged outside of hours, and how frequently do they work an incident? (Do managers track this?) Are you expected to keep shipping product during on call weeks, or devote your time to making the system better?
  4. If you had to ship a single line of code to production using the deploy pipeline, how long would that take? Remember, the lower the deploy interval, the happier and more productive you are likely to be as an engineer there. Under 15 minutes is great. Under an hour is tolerable. More than that, proceed with great caution.

The interview itself

  1. Was your interview well-organized and conducted in a timely fashion? Were you given detailed information about what to expect, and were your interviewers well-prepared, and conversational? Were the questions fair, open-ended, and relevant to the job in question?
  2. If they asked you to perform any kind of take-home labor of more than an hour or so, did they compensate you for your time?
  3. Did they get back to you swiftly at each step of the way to let you know where you stand and what comes next?
  4. Did you find the questions interesting and challenging? Do they have a clear idea of what success looks like for you in this role? Did you leave excited and buzzing with ideas about the work you could do together?
    • This is 👆 definitely more of a “how good is this job” question than “is this a shithole” question, but one of our honeycombers brought it up as an example of how a great interview can make you decide to leave a job , even one you’re perfectly happy with.
  5. The questions they ask you while interviewing you are the questions they ask everyone else. So…did they ask you about your views on diversity and team dynamics while interviewing you? Or is that not part of their filters, only their advertised persona?

Three more

  1. Do their employees seem to speak freely on twitter? If you are an agitator of sorts, are there others who agitate about similar issues — either with company support, or at least lack of censorship?
  2. How does the company respond to criticism and feedback? For that matter, how do they treat their competitors? Being competitive is fine, being mean is not.
  3. Get clear on your own expectations. What’s on your wishlist, and what’s make-or-break for you? If something is very important to you, consider telling the hiring manager up front. For example, “These are my expectations for how women are treated. How do you think your company matches up?” Their answers will speak volumes, and so will their comfort level with the question.

In closing

If you a join a new company, and two or three weeks in you’re just not feeling it, you’re wondering if you made a mistake — leave. You do not owe them a year of your life. Trust your instincts. Just leave it off your resume entirely and roll the dice again.

Employers are all too accustomed to feeling (and acting) like they hold all the power. They do not. Every tech company is a talent business, which rises and falls on the caliber of the people they can convince to stay. They aren’t doing you a favor by employing you; you are doing THEM a favor by lending them your creativity, labor, and a third of the hours in your day.

Do they deserve it? Will their success make the world a better place? If not, stop supporting them with your work and lend your muscle to a company that deserves it.

In the hottest job market of my lifetime, with millions of opportunities newly open to people who live literally anywhere, you owe it to yourself, your future self, and your family to take a good hard look at where you sit. 🍄 Are you happy? 🍄 Are you compensated well, is your time valued? 🍄 Are you still learning new things and improving your skills every day? 🍄 Is your company still on a growth trajectory? 🍄 Do you trust your leadership and your team, 🍄 do still you believe in the mission, and 🍄 do you think your labor contributes meaningfully to making the world a better place?

If not, consider joining the Great Resignation. I hear they have cookies.

Huge thanks to Amy Davis, Phillip Carter, Ian Smith, Sarah Voegeli, Kent Quirk, Liz Fong-Jones, Amanda Shapiro, Nick Rycar, Fred Hebert and David Daly, all of whom contributed to this post!

How can you tell if the company you’re interviewing with is rotten on the inside?

How Much Should My Observability Stack Cost?

mipsytipsy, , , Leave a comment

First posted on 2021-08-18 at https://www.honeycomb.io/blog/how-much-should-my-observability-stack-cost

What should one pay for observability? What should your observability stack cost? What should be in your observability stack?

How much observability is enough? How much is too much, or is there such a thing?

Is it better to pay for one product that claims (dubiously) to do everything, or twenty products that are each optimized to do a different part of the problem super well?

It’s almost enough to make a busy engineer say “Screw it, I’m spinning up Nagios”.

(Hey, I said almost.)

All of these service providers can give you sticker shock when you begin investigating them. The biggest reason is always that we aren’t used to considering the price of our own time.  We act like it’s “free” to just take an hour and spin something up … we don’t count the cost of maintenance, context switching, and opportunity costs of not using the time to build something of business value.  Which is both understandable and forgivable, as a starting point.

Considerably less forgivable is the vagueness–and sometimes outright misdirection and scare tactics–some vendors offer around pricing. It’s not ok for a business to optimize for revenue at the expense of user experience. As users, we have the right to demand transparency and accurate information.  As vendors, we have the responsibility to provide it.  Any pricing scheme that doesn’t align with best practices and users’ interests will be a drag on reputation and growth.

The core question, rarely addressed outright, is: how much should you pay? In this post I’ll talk about what your observability costs include, and in the next post, what you should consider including in your “observability stack”.

But I’ll give you the answer to your question right off the bat: you should probably spend 20-30% of infra costs on observability.

O11y spend should be 20-30% of infra spend

Rule of thumb: your observability spend should come to 20-30% of your infra spend. (I’ve seen 10% a few times from reasonable-seeming shops, but they have been edge cases and outliers. I have also seen 50% or more, but again, outliers.)

Full disclosure: this isn’t based on any particular science.  It’s just based on my experience of 15+ years working in operations engineering, talking to other engineers and managers, and a couple of informal Twitter polls to satisfy my own curiosity.

Nevertheless, it’s a pretty solid rule. There are exceptions, but in general, if you’re spending less than 20%, you’re “saving money” at the expense of engineering time, or being silently dragged underwater by a million little time leaks and quality of service issues — which you could eliminate completely with a bit of investment.

Consider the person who told me proudly that his o11y spend was just 1-3%. (He meant the PagerDuty bill and Pingdom checks, actually.) He wasn’t counting the dedicated hardware for their ELK cluster (80k/month), or the 2-3 extra engineers they had to recruit, train and hire (250-300k/year apiece) to run the many open source tools they got for “free”.

And ultimately, it didn’t meet their needs very well. Few people knew how to use it, so they leaned on the “observability team” to craft custom views, write scripts and ETL one-offs, and serve as the institutional hive mind and software usability tutors.  They could have used better tools, ones under active development by large product teams.  They could have used that headcount to create core business value instead.

Engineers cost money

Engineers are expensive. Recruiting them is hard. The good ones are increasingly unwilling to waste time on unnecessary labor. This manager was “saving” maybe a million dollars a year (he mentioned a vendor quote of less than 100k/month)–but spending a couple million more than that in less-visible ways.

Worse, he was driving his engineering org into the ground by wasting so much of their time and energy on non-mission-critical work, inferior tooling, one-offs, frustrating maintenance work, etc, all of which had nothing to do with their core business value.

If you want to know if an org hires and retains good engineers, you could do worse than to ask the question: “What tools do you use, and why?”

  • Good orgs use good tools. They know engineering cycles are their scarcest and most valuable resource, and they want to train maximum firepower on their core business problems.
  • Mediocre orgs use mediocre tools, have no discipline or consistency around adoption and deprecation, and leak lost engineering cycles everywhere.

So back to our rule of thumb: observability amounting to 20-30% of total spend is where most shops should fall. This refers to cloud-native infrastructure, using third-party services to instrument and monitor code, with the basics covered — resource utilization graphs, end to end checks, paging, etc.

So, what do I need in my “observability stack”?

What are the basics? Well, obviously “it depends”. It depends on your requirements, your components, your commitments, your budget, sunk costs and skill sets, your teams, and most expensive of all — customer expectations and the cost of violating them. You should think carefully about these things and try to draw a straight line from the business case to the money you spend (or don’t spend). And don’t forget to factor in those invisible human costs.

 

How Much Should My Observability Stack Cost?

Questionable Advice: “What Should I Say In My Exit Interview?”

mipsytipsy, , , , 3 Comments

I recently received this gem of a note::

Hi Charity, I really enjoy your writing and a lot of it has directly contributed to me finally deciding to leave a company with a toxic management culture. I’ll also be leaving many great IC friends that will have lost a strong voice.

My exit interview will be next week. Any advice on how honest I should be?

I’ve googled quite a bit but there are only generic “don’t burn bridges” comments. Would love to see something a little more authoritative 🙂

–Anonymous Reader

Ew, fuck that. That’s exactly the kind of quivering, self-serving, ass covering advice you’d get from HR. It’s exactly the kind of advice that good people use to perpetuate harm.

I wouldn’t worry about “too much honesty” or personal repercussions or whatnot. I would worry about just one thing: being effective. This is your last chance to do the people you care about a solid, and you don’t want to waste it.

So … ranting about every awful person, boring project and offensive party theme of your tenure: not effective. Ranting about people who were personally irritating but had very limited power: not effective. Talking only in vague, high level abstractions (“toxic culture”), or about things only engineers understand and are bugged by: not effective.

What is effective? Hm, let’s think on this..

  • Start off with your high level assertion (toxic culture) and methodically assemble a list of stories, incidents, and consequences that support your thesis. Structure-wise, this is a lot like writing a good essay
  • Tie your critiques to the higher ups who enabled or encouraged the bad behavior, not just the flunkies who carried it out.
  • Wherever possible, draw a straight line to material consequences — people quitting, customers leaving, your company’s reputation suffering
  • Keep it crisp. No more than three pages total. Pick your top 1-3 points and drive them home. No detours.
  • This one sucks, but … if someone was perceived as an underperformer or a problem employee, avoid using them as evidence in support of your argument. It won’t help you or them, it will be used as an excuse to discredit you.
  • Keep it mostly professional. I am not saying don’t show any anger or strong emotion; it can be a powerful tool; just be careful with it. Get a proofread from someone with upper management experience, ideally with no connection to your work. (Me, if necessary.)
  • ✨Put it in WRITING!✨ Deliver your feedback in person, but hand over a written copy as well. Written words are harder to ignore or distort.
  • For extra oomph, give a copy to any execs, managers, or high level ICs you trust. Don’t just email it to them, though. Have a face to face conversation where you state your case, and hand them a written copy at the end.

The sad fact is that most exit feedback is dutifully entered in by a low ranking employee who makes a third your salary and has no reason whatsoever to rock the boat, after which it gets tossed in a folder or the trash and is never seen again.

If you want to use your voice on your way out the door, the challenge you face isn’t one of retribution, it’s inertia and apathy. HR doesn’t care about your feedback … but they care if they think their boss saw it and cares about it

And I think you should use your voice! You clearly have some clout, and what’s the point of having power if you won’t extend yourself now and then on behalf of those who don’t?

Good luck!!

charity

Questionable Advice: “What Should I Say In My Exit Interview?”

Know your “One Job”, continued

mipsytipsy, , , , , 4 Comments

Holy macaroni batman, I was not expecting that post to ignite a shitstorm. It felt like a pretty straightforward observation: you were hired to do a job, you should do your best to do that job.

Interestingly, the response was almost universally positive for the first 8 hours or so. But then the tide began to turn as the 🔥hot takes🔥 began rolling in (oh god 🙈) and then began pingponging off each other, competing for and amplifying each other’s outrage. 😕You Had One Job | Know Your Meme

When something touches a nerve like this, it swiftly becomes less about your actual words and more of a public event, or maybe a Teachable Moment. (🤮) Everybody has to weigh in with their commentary, and while it’s not super fun to be on the receiving end, I have mostly learned to distinguish the general pile-on from the few engaging in bad faith. I just keep telling myself that public discourse is how we create shared consensus and shift the Overton window and industry norms. So that’s all fair game, it’s done in good faith and it’s the price of change. I can take a few days of shitty twitter mentions for the team, lol.

The one comment that did worm under my skin was a woman who said she believed my post would give an abusive former boss ammo to use against people like her in the future. And that, more than all the hatertomatoade, is why I wrote this epic followup.

Two responses i’d like to foreground

First, I’d like to link to something Terra said about the importance of ERGs for marginalized workers, especially during times of duress.

Thanks Terra, I stand corrected — I can totally see how that work counts as part of one’s core job, and managers should understand this and define it as such.

I still don’t think it means you promote someone purely on the basis of that work; I don’t see how you can promote someone up an engineering ladder until they can fulfill the technical FAIL Nation - ocd - Vintage FAILs of the Epic Variety - Cheezburgerrequirements of the next level. It could certainly mean expanding the core requirements of your role to include your ERG labor, though perhaps not replacing the technical work entirely (over the long term).

This goes both ways, fwiw. Nobody should be promoted without doing their fair share of the emotional labor and glue work it takes to keep the company going. A successful career in sociotechnical systems means leveling up your social repertoire as well as your technical skills. Your job descriptions and levels must reflect this, spelling out both the social and the technical requirements at each level.

🔥🔥🔥Tip of the Day🔥🔥🔥

If you want to know what your company REALLY cares about, take a management gig for a while and listen closely to the debates over whether or not to promote someone to the post-senior levels, and why or why not. You are LITERALLY witnessing as your organization decides, over and over, what it values the most, what it wants to reward, whose footsteps they want you to follow in, and where to spend its scarcest, most inelastic resource: staff and principal engineering titles. Fascinating shit.

Secondly, please go and read Shelby’s excellent thread, written from the perspective of someone who has been “Susan” for a number of reasons.

https://twitter.com/shelbyspees/status/1368705471532568581

 

These situations are complicated, and managers should always, always try to listen and understand the subtleties of each unique situation before coming to some mutual understanding with their team member about what the core responsibilities of the job are. Jobs are living documents, and it doesn’t hurt to revisit them and clarify from time to time. <3

Objection: “Nobody has Just One Job”

Completely true. I was trying to be funny by referring to the “You had one job!” meme. I apologize. Yes, everybody has a basket of responsibilities. I DO think that a well-written job description and clarification on the priorities of the job is a good thing, and will go a long way towards helping you figure out how to spend your time and how to not get burned out.

Also: am I prone to hyperbole? Yes ma’am, sweeping statements are LITERALLY ALL I DO. (Sorry ☺️)

FTR, I don’t think your core responsibilities should be overwhelming, and there should be plenty of time in your normal 40 hour work week to devote to so-called electives and extracurriculars. I’ve said many times that I don’t believe anyone has more than four hours a day of real, focused, challenging engineering labor in them. So maybe 15-20 hours/week of moving the business forward in your areas of ownership?

Everyone should have cycles free for participating in the “squishy” parts of work. It’s an important part of taking a group of random individuals and connecting them with meaning and a sense of mission. That isn’t HR’s job, or the managers or execs’ jobs, that is everyone’s job, the more the better. Everyone should have flexibility, autonomy, and variety in their schedule, and should feel supported in using work hours to support their peers. Nothing I am saying contradicts any of that. But sometimes something’s gotta give, and usually your core responsbilities are not what you want to sacrifice.

Objection: “Interviewing isn’t optional”

Sure. But you weren’t hired to interview. It’s just part of the basket of secondary responsibilities that come along with being a member of the team. If you’re an engineer, you likely weren’t hired to write blog posts or mentor folks — unless you were! were you?? — which makes them similarly in the bucket marked “valuable, but secondary”. Honestly it could be on purpose though, it could just be words in another language. - Imgflip

We aren’t talking about “steady state”, we’re talking about what to do when you aren’t able to fulfill the core functions of your job. This should be a temporary state of emergency, not the status quo. When you’re overwhelmed, it’s totally normal to ask to be excused from the interviewing rotation for a quarter or so, or any of your other secondary responsibilities.

Objection: “How dare you not promote someone who is getting good peer reviews”

I said she was getting some compliments and rave reviews. If you’re getting compliments from HR, marketing, and other people sprinkled across the company, but you aren’t delivering for your own team; if you’re holding back core initiatives for your closest peers, then you aren’t doing your work in the right order.

I’ve seen this happen when an engineer’s public persona becomes more important to them than their actual work. When they get hooked on the public adulation of giving talks and writing posts and going to conferences, meanwhile their output at work drops off a cliff. This isn’t fair to your coworkers. Maybe you don’t want to do this job anymore, maybe you want a job where your core responsibilities are writing and speaking. I don’t know. All I know is that if I’m the manager of that team, my responsibility and loyalty is to the well-functioning of that team, so we need to have a conversation and clarify what’s happening.

Again, all I am saying is that your commitments to your immediate team come first. Not following through affects way more than just you.

Objection: “You are hating on / devaluing glue work”

I really wish I had thought to link to Tanya Reilly’s invaluable material on glue work in my original post, but I didn’t connect those dots, sadly. Yes, it can be hard for engineering managers to recognize or reward glue work, and yes, glue work is an invaluable form of technical leadership. I don’t have a lot to say about this other than I completely agree, and it is not what I am talking about.

Objection: “All these extra-curriculars should count towards promotions”

Well, yes! Duh! I am all for promoting people not based on raw individual coding output, but on overall impact. People who perform a lot of glue work are invaluable to any high functioning team, and people who run internal ERGs, do lots of DEI work, etc — that SHOULD factor into promotions. In my original post I said that sadly, when someone isn’t performing the key parts of their job, you don’t get credit for these wonderful positives — they are not enough on their own (unless of course you have an understanding with your manager that your core responsibilities have changed), and can even be evidence of time mismanagement or an inability to prioritize.

you had one job Keep trying - Paranoia meme | Make a MemeI cannot honestly understand why this is controversial. If you were hired as a database engineer, and you spent the year doing mostly DEI work, how does it make sense to promote you to the next level as a database engineer? That’s not setting you up for success at the next level at ALL. If this was agreed upon by you and your manager, I can see giving you glowing reviews for the period of time spent on DEI work, as long as your dbeng responsibilities were gracefully handed off to someone else (and not just dropped), but not promoting you for it.

However, if you ARE competently performing your core responsibilities as a database engineer, and are performing them at the next level, then all your additional work for the company — on DEI, ERGs, mentoring, interviewing, etc — adds up to a MASSIVELY compelling body of work, and a powerful argument for promotion. It certainly ought to be enough to push you over the edge if you are on the bubble for the next level..

Objection: “You hate DEI work and demean those who do it”

It does not make me anti-DEI work to point out that you were hired to do a certain job, first and foremost. If you want your first-and-foremost job to be DEI work, that’s great! Go get that job! If you want your first-and-foremost job to be engineering, but then not do that job, I … guess I just fail to see how this is a logically defensible position.

As someone else put it: “Your One Job is the cake, the rest is the icing”.//TODO citation You can often negotiate a job that has a LOT of icing — but you should negotiate, no surprises. DEI work is absolutely valuable to companies, because having a diverse workforce is a competitive advantage and increasingly a hiring advantage. But that work isn’t typically budgeted under engineering, it comes from G&A

I can also understand why you might want to keep the (unfortunately higher) engineering salary and the (unfortunately higher) social status you have with an engineering role, even if engineering no longer feeds your soul the way doing diversity work does. I know people in this situation, and it’s tricky. 😕 There is no single right or wrong answer, just a question of whether you can find an employer who is willing to pay for that configuration. But I should think clarity and straightforwardness is more important than ever when your heart’s desire is unconventional.

Objection: “You’re letting managers off the hook. This is entirely a management failure.”

You might be right. This is often a consequence of negligent, unclear, or biased management. But not always. I’ve also seen it happen — close up — with engaged, empathetic, highly skilled You Only Had One Job on Twitter: "Whoever wrote this, probably never saw a sign like this when he/she was a kid #youonlyhadonejob #youonlyhad1job https://t.co/BGLHCWwtGo"managers who were good at setting structure and boundaries, deeply encouraging, gave tons of chances and great constant feedback, tried every which way to make it work … and the employee just wasn’t interested. They volunteered for every social committee and followed every butterfly that fluttered by. They just weren’t into the work.

YES, managers should be keeping close tabs on their reports and giving constant feedback. YES, it’s on the manager to make sure the role is clearly defined.

YES, managers should be clear with employees on what the promotion path is, and YES, no review should ever be a surprise.

YES, if people all over the org are heaping requests or responsibilities on the Susans of the world, it can be difficult to prioritize and it is not fair to expect them to juggle those requests, the manager should help to shelter them from it. Yes yes yes. We are all on the same page.You had one job! – inkbiotic

But I am writing this post, not to managers, but to engineers, people like me, who are confused about why they aren’t getting promoted and others are. I am writing this post because good managers are in scarce supply, and I don’t want your career to be on hold until you happen to get a good one. I am trying to provide a peek behind the curtain into something that frustrates managers and holds a lot of people back, so that you can take matters into your own hand and try to fix it. If your manager hasn’t been clear with you on what your core job is, they suck and I’m sorry.

Is any of this your fault? Maybe, maybe not. But it is something you have some control over. You can at least open the conversation and ask for some clarity.

You shouldn’t HAVE to do their job for them. But why let a shitty manager hurt your career any more than you must?.

Objection: “This is a gendered thing. ‘Steven’ would have been promoted for this, while ‘Susan’ gets scolded.”

110 You had one job.. ideas | you had one job, one job, jobA surprising number of people thought I was writing this as advice specifically to and for women, and got mad about that. Nope, sorry. It was not a gendered thing at all. I’ve seen this happen pretty much equally with men and women. I had planned on writing three or four anecdotes, using multiple fake names and genders, but the first one turned out long so I stopped.

Confession: I put zero thought into constructing a realistic or plausible list of activities “Susan” has at work. This came back to bite me. A LOT of people were doing a super close read of the situation (e.g. “She’s on three ERGs, so she must be a queer woman of color, which means she’s probably the most senior person at the company along all three dimensions of marginalization…”) — which, AUGH — this isn’t real, y’all —

✨This is Not A Real Situation✨.

— it was MADE UP! Totally! I just listed off the first several things that came to mind that people do at work that aren’t things they specifically got hired to do. That’s it.

I rather regret this. I should have put more time and thought into constructing a scenario Image tagged in you had one job,road stripes,timmy - Imgflipthat was less easy to nitpick. But I didn’t want anyone to see themselves in it, so I didn’t want it to be too recognizable or realistic. I just wanted a placeholder for “Person who does lots of great stuff at work”, and that’s how you got Susan.

TLDR — yep, Susan probably has a tougher go of this than Steven. I would argue that Steven generally wouldn’t be promoted either if he wasn’t adequately performing his core responsibilities, but who knows. This isn’t a real person or scenario. Women and nonbinary folks have it tougher than men. Your point? ¯\_(ツ)_/¯

Objection: “Why do you hate collaboration” and “This doesn’t apply to me because my job isn’t just writing code”

I never meant to imply that your “One Job” was only work you could do heads down on your own. Lots of people’s jobs involve a ton of force multiplying, mentorship, reviewing, writing The best you had one job memes :) Memedroidproposals… typical glue work. If you have a manager that thinks you are only doing your One Job when you are heads down writing code with your headphones on, that’s a Really Bad Manager.

The more senior your role is, the more your One Job involves less “write feature X” and more “use your judgment to help fine tune our sociotechnical systems”. This is natural and good.

It is also MORE of an argument for making sure you are in alignment with others in your org about what is the most important thing for your time and energy to be spent on, not less of one. Communication and persuasion are what upper levels are all about, right?

Objection: // Placeholder

I reserve the right to add to this list of objections after I go catch up on twitter, lol.

In conclusion

Finally, I want to call out a perceptive comment by @codefolio, where he said this:

Ouch. Truth.

This speaks to something I should probably be more explicit about, which is that my writing and my advice generally assumes you are operating in a high-trust environment. That’s the kind of environment I have been tremendously fortunate to operate in for most of my career — an environment where people are flawed but compassionate, skilled, and doing their best for each other, with comparatively low levels of assholery, sociopaths, and other bad actors. If your situation is very unlike mine, I can understand why my advice could seem blinkered, naive, and even harmful. Please use your own judgment.

I only write because I want the best for you — even those of you who very openly and vocally despise me (and yes, there are quite a few of you. Start a club or something). 🥰

All said and done, about 95% of the people who replied said that my post was helpful and clarifying for them, about 3% had very interesting critical takes that I got something valuable from, and only 2% were hurling rotten tomatoes and reading and interpreting my words in ways that felt wildly detached from reality. I try to remember that, when I get discouraged and feel like the whole world hates me and wants me to shut up. 🍅🍅🍅

I am not everyone’s cup of tea, and that’s alright. 💜 My advice is not relevant or helpful to anyone, and that’s okay too. Hopefully I have cleared up enough of the misunderstandings that anyone who is reading my words in good faith now has a clear picture of what I was trying to say. And hopefully it’s helpful to some of you.

PSA: I will be your rubber ducky advice buddy🐥❤️🐣

I have posted this on twitter a few times, but if you are struggling with a career conundrum, sociotechnical growing pains, or management issue, I am generally happy to hop on a phone call over the weekend and talk through it with you. I have benefited SO much from the time and energy of others in the tech industry, it’s nice to give back. (I like giving advice and I think VERY highly of my own opinions, so this also counts as fun for me 😂)

I 💜 talking to women and enbies and queers.🌈 (I love talking to guys too, but if my calendar starts filling up I will rate-limit y’all first to leave space at the front of the line.) If you are a white dude who hasn’t been following me on twitter for at least a year, this offer is not for you, sorry. If you’re a marginalized person in tech, otoh, I don’t care if you use that hellsite^W^Wtwitter or not. And if you hate my blog, you are not gonna like me any better live & unfiltered. 😬

🍃🌸🌼Sign up here🌼🌺🍃
(please read the instructions!)

Things I am generally well-equipped to discuss include startups, observability, databases, leveling and promotion issues, management, the pendulum, senior and staff levels, new manager issues, team dynamics, startup executive teams, and some complex workplace You Had One Job and You So Failed ~ 27 pics | Team Jimmy Joe | Building fails, Construction fails, Architecture failssituations.

Things I am not equipped to help with include how to improve diversity at your company, how to get women to want to work for you, or why only men are applying for your jobs online. I cannot be your personal DEI coach or guide to women in the workplace (there are great consultants out there doing the lord’s work, and you should give them all your money). I can’t find you a new job, help newbies find their first job (sorry 😕), give advice on raising venture money or tell you how to found a company (answer: never found a company, it’s the literal worst). In general I am not very well equipped to discuss early career issues, but if you’re an URM and you’re desperate i’ll give it a shot. I am not a therapist. And if you’re going to ask should you quit your job, I will save you a phone call: yes.

charity.

Know your “One Job”, continued

Know your “One Job” and do it first

mipsytipsy, , , , 13 Comments

Story time.

Susan was hired as a database engineer. Her primary projects, which are supposed to be upgrading/rolling out a major point release and running load tests against various config options and developing a schema management tool, keep slipping. But she is one of HR’s favorite people because she is always available to interview, even at short notice, and gives brilliant, in-depth feedback on candidates.

Susan also runs three employee resource groups, mentors other women in tech both internally and external to the company, and spends a lot of time recruiting candidates to come work here. She never turns down a request to speak at a meetup or conference, and frequently writes blog posts, too. She is extremely responsive on chat, and answers all the questions her coworkers have when they pop into the team slack. Susan has a high profile in the community and her peer reviews are always sprinkled lavishly with compliments and rave reviews from cross-functional coworkers across the company.

Lately, Susan has been getting increasingly exasperated about her level. She is a senior engineer, but the impact of her work is felt all across the company, and many of the things she does are described in higher level brackets. Why doesn’t her manager seem to recognize and acknowledge this?

Actually, Susan’s manager is absolutely right not to promote her. Susan isn’t adequately performing the functions of her job as a database engineer, which is the “One Job” she was hired to do, and which her teammates are all relying on her to do in a timely and high-quality manner.

When someone isn’t meeting the basic expectations for their core responsibilities, it doesn’t matter how many other wonderful things they are doing. In fact, those things can become strikes against them. Why is Susan available for every interview at the drop of a hat? Why is she agreeing to speak at so many meetups and conferences, if she can’t find the time to perform her core responsibilities? Why doesn’t she silence Slack so she can focus for a while? These things that should be wonderful positives are transformed instead into damning evidence of personal time mismanagement or an inability to prioritize.

When you are meeting expectations for your One Job — and you don’t necessarily have to be dazzling, just competent and predictable  — then picking up other work is a sign of initiative and investment. But when you aren’t, you get no credit.

This may sound obvious, but I have seen everyone from super junior to super senior fall into this trap — including myself, at times. When you get overwhelmed, all of your commitments can start to feel like they are of equal weight. But they are not. “You had One Job”, as the kids say, and it comes first.

Extracurriculars can feel like obligations, yet these are qualitatively different from the obligation you have to your core job. If you did only your core responsibilities and none of the extras, your job should not be in any danger. But if you don’t do the core parts, no matter how many extras you do, eventually your job probably will be in danger. Explain to your coworkers that you need to hit the pause button on electives; they’ll understand. They should respect your maturity and foresight.

If you’re feeling underwater, scrutinize what’s on your plate. Which are the parts you were hired to do? the parts that are no one else’s job but yours? Focus on those first. If you need more time, cut down or hit pause on the electives until you’re comfortably on top of things again.

Do you have too many core responsibilities? Those should never add up to 40+ hours of work every week. Everyone needs some flex and variety in their schedule.

If you are having a terrible time summoning the motivation to do the work you were hired to do, or if this is a recurring theme in your life, then maybe you are in the wrong role. Maybe you really want to find a role as a developer advocate instead of a software engineer. Maybe you just aren’t into the work anymore (if you’ve been there a while), or maybe you don’t know how to get started (if you’re new). Maybe it’s even as simple as mentioning it to your manager and reshuffling your responsibilities a bit. But don’t assume the problem will solve itself.

Take these feelings seriously. All of us need to buck up and plow through some work we don’t find engaging from time to time. But it shouldn’t be the norm. In the long run, you’ll be happier and more successful if you are truly engaged by the work you were hired to do, not just by the extracurriculars.

charity.

Know your “One Job” and do it first

Questionable Advice: “How can I sniff out bad managers while interviewing for a job?”

mipsytipsy, , , , Leave a comment

A few weeks ago I got a question from Stephane Bjorne on twitter, about how to screen for bad managers and/or management culture.

This is a great question. I’ve talked a lot about my philosophy for interviews, which boils down to equalizing the power dynamics as much as possible to reflect the reality that the candidate should be interviewing the company just as much as the reverse. You are all highly compensated subject matter experts who can find jobs relatively easily; there’s no reason all the judgment and critique should flow in a single direction.

But HOW? What questions can you ask, to get a feel for whether you will join this team and belatedly discover that you’re expected to be a jira ticket monkey, or that the manager is passive aggressive and won’t advocate for you or take a stand on anything?

Glad you asked.

First of all, backchannel like mad if you can. If you can’t, do ask the same question of multiple people and compare their answers. Pay particular attention to the different answers given by junior vs senior members of the team, and give extra weight to the experiences of any underrepresented minorities

Questions to consider asking the manager:

  • “How did you become a manager? Do you enjoy it?” Trust me, you never want to work for someone who was pushed into management against their will and still seems openly bitter about it.
  • “What do you like about your job?” Red flags include, “I was tired of being out of the loop and left out of decision-making processes.” That could be you next.
  • “Is management a promotion here, or a lateral move? Do people ever go back from managing to engineering? Is that considered a viable career path?”
  • “What kind of training do you offer new managers, and what are they evaluated on? What are YOU evaluated on?”
  • “Do you have a job ladder for individual contributors (ICs)? Can I see it? Do the IC levels track management levels all the way to the top, or top out at some point?”
  • “What does your review and promotion process look like?”
  • “Are your levels public or private? What is the distribution of engineers across levels, roughly?” Here you are looking for how high the IC track goes, and how many engineers exist at the upper bound. Distribution should be scant at the upper levels, roughly an order of magnitude less for each level after “senior engineer”. Do the written level descriptions seem reasonable and appealing to you? Do you see yourself in them, and feel like there is a path for growth?
  • “Do you have any junior or intermediate engineers, and how many? If not, why the fuck not?” Ask.
  • “How often do you have 1x1s with each of your direct reports? How often are the skip levels?”

And then there is the ur-question, which every one of you should ask in every single interview you ever have:

“What is the process by which someone ends up working on a particular project?” In other words, how does work get decided and allocated? Bad signs: they get flustered, don’t have a clear answer, you have no say, there is no product/design process, it’s all done via jira assignments.

Pay just as much attention to their body language and signals here as the answer they give you. Are they telling the truth, or describing their ideal world? Ask what happens when there are problems with the  normal process, or how often it gets circumvented, or how you know if your work aligns with company strategy.

Questions to ask an engineer:

  • “What do you talk about in your 1x1s with your manager?”
  • “How often does your manager have career conversations with you, asking how you want to develop over the next few years?” Ideally at least a couple times a year, but really any answer is fine other than a blank stare.
  • “Does your manager care about you? Has working here moved your career forward? How so?”
  • “Have you ever been surprised by feedback you received in a review from your manager?” You should never be surprised.

Most of all, can you get the manager to tell you “no” on a thing you clearly want during the interview? (Maybe a level, or WFH schedule, or travel, etc.) Or are they squirmy, evasive, or hedging, or make you promises that they’ll look into it later, etc? Anyone who will look you straight in the eye and say “no, and here’s why”, is someone you can probably trust to be straight with you down the line, in good times and bad.

Depending on your level and career goals, it’s a good idea to ask questions to ascertain if the right gap exists for you to fill to reach your goals. Don’t join a team where five other people have the same exact skill sets as you do and are already eyeing the same role you want. (I wrote more on levels here.)

We also got some great contributions from @GergelyOrosz and @JillWohlner:

  • “Ask about specific situations, any half decent manager can BS on hypothetical stuff.”
  • “Who was the last person you promoted? Why/how?”
  • “How do you handle when X complains to you about Y? Can you give an example?”
  • “What was the best team you managed and why?”
  • “What is the biggest challenge the team currently has and why? What are you doing about it? Biggest strength?”
  • “How many people have left the team since you’ve been here?”
  • “Who was the last person to leave your team? Why did they leave?” (These leaving questions are tough to ask but will give you a lot of signal. Especially seeing how the manager frames it — are they playing victim, or owning up to it?)

Anyone who won’t be honest about their real personal weaknesses and struggles, probably isn’t someone you want to report to.

Jill says that she wants to hear from ICs:

  • when their manager has gone to bat for them
  • when their manager gave them hard to hear feedback (if it’s never, it’s a flag — all positive feedback = little growth)
  • when their manager coached them through a tricky situation

and from Managers:

  • when they went to bat for someone on their team
  • a team member’s growth they feel really proud of
  • when they got negative feedback about someone on their team and how it was handled

Obviously, you won’t be able to ask all of these questions in an hourlong slot. So ask a few, and lean in whenever they seem avoidant or uncomfortable — that’s where the juicy stuff comes from. And personally, I wouldn’t consider working somewhere that sees management as a promotion rather than a peer/support role, but that may be a high bar to clear in some parts of the world.

Good luck. Joining a team with a good manager can be one of the best ways to accelerate your career and open the door to opportunities, in part because it ensures healthy team dynamics. Workplace friction, bullying, harassment, passive aggressiveness, etc can be truly terrible, and even in the milder cases it’s still a huge distraction from  your work and a big quality of life issue.

A manager’s One Job is to make sure that shit doesn’t happen. It’s really is worth trying to find a good one you can trust.

Questionable Advice: “How can I sniff out bad managers while interviewing for a job?”