influence

Questionable Advice: “How can I drive change and influence teams…without power?”

mipsytipsy, , , , , , Leave a comment

Last month I got to attend GOTO Chicago and give a talk about continuous deployment and high-performing teams. Honestly I did a terrible job, and I’m not being modest. I had just rolled off a delayed redeye flight; I realized partway through that I had the wrong slides loaded, and my laptop screen was flashing throughout the talk, which was horribly distracting and means I couldn’t read the speaker notes or see which slide was next. đŸ˜” Argh!

Anyway, shit happens. BUT! I got to meet some longstanding online friends and acquaintances (hi JJ, Avdi, Matt!) and got to eat some of Hillel Wayne’s homemade chocolates, and the Q&A session afterwards was actually super fun.

My talk was about what high performing teams look like and why it’s so important to be on one (spoiler: because this is the #1 way to become a radically better engineer!!). Most of the Q&A topics therefore came down to some version of “okay, so how can I help my team get there?” These are GREAT questions, so I thought I’d capture a few of them for posterity.

But first… just a reminder that the actual best way to persuade people to listen to you is to make good decisions and display good judgment. Each of us has an implicit reputation score, which formal power can only overcome to an extent. Even the most junior engineer can work up a respectable reputation over time, and even principal engineers can fritter theirs away by shooting off at the mouth. đŸ„°

“how can I drive change when I have no power or influence?”

This first question came from someone who had just landed their first real software engineering job (congrats!!!):

“This is my first real job as a software engineer. One other junior person and myself just formed a new team with one super-senior guy who has been there forever. He built the system from scratch and knows everything about it. We keep trying to suggest ideas like the things you talked about in your talk, but he always shoots us down. How can we convince him to give it a shot?”

Well, you probably can’t. â˜ș Which isn’t the end of the world.

If you’re just starting to write software every day, you are facing a healthy learning curve for the next 3-5 years. Your one and only job is to learn and practice as much you possibly can. Pour your heart and soul into basic skills acquisition, because there really are no shortcuts. (Please don’t get hooked on chatGPT!!)

I know that I came down hard in my talk on the idea that great engineers are made by great teams, and that the best thing most people can do for their career is to join a high-performing, fast-moving team. There will come a time where this is true for you too, but by then you will have skills and experience, and it will be much easier for you to find a new job, one with a better culture of learning.

It is hard to land your first job as a software engineer. Few can afford to be picky. But as long as you are a) writing code every day, b) debugging code every day, and c) getting good feedback via code reviews, this job will get you where you need to go. When you’re fluent and starting to mentor others, or getting into higher level architecture work, or when you’re starting to get bored … then it’s time to start looking for roles with better teachers and a more collaborative team, so your growth doesn’t stall. (Please don’t fall into the Trap of the Premature Senior.)

This is an apprenticeship industry. You’re like a med student right now, who is just starting to do rounds under the supervision of an attending physician (your super-senior engineer). You can kinda understand why he isn’t inclined to listen to your opinions on his choice of stethoscope or how he fills out a patient chart. A better teacher would take time to listen and explain, but you already know he isn’t one. đŸ€·

I only have one piece of advice. If there’s something you want to try, and it involves doing engineering work, consider tinkering around and building it after hours. It’s real hard to say no to someone who cares enough to invest their own time into something.

“how can I drive change when I am a tech lead on a new team?”

“I have the same question! — except I’m a tech lead, so in theory I DO have some power and influence. But I just joined a new team, and I’m wondering what the best way is to introduce changes or roll them out, given that there are soooo many changes I’d like to make.”

(I wrote a somewhat scattered post a few years ago on engineers and influence, or influence without authority, which covers some related territory.)

As a tech lead who is new to a team, busting at the seams with changes I want to make, here’s where I’d start:

  1. Understand why things are the way they are and get to know the personalities on your team a bit before you start pitching changes. (UNLESS they are coming to you with arms outstretched, pleading desperately for changes ~fast~ because everything is on fire and they know they need help. This does happen!)
  2. Spend some time working with the old systems, even if you think you already understand. It’s not enough for you to know; you need to take the team on this journey with you. If you expect your changes to be at all controversial, you need to show that you respect their work and are giving it a chance.
  3. Change one thing at a time, and go for the developer experience wins first. Address things that will visibly pay off for your team in terms of shipping faster, saving time, less frustration. You have no credibility in the beginning, so you want to start racking up wins before you take on the really hard stuff.
  4. Roll up your sleeves. Nothing buys a leader more goodwill than being willing to do the scut work. Got a flaky test suite that everybody has been dreading trying to fix? I smell opportunity…
  5. Pitch it as an experiment. If people aren’t sold on your idea for e.g. code review SLAs, ask if they’d be willing to try it out for three weeks just as an experiment.
  6. Strategically shop it around to the rest of the team, if you sense there will be resistance…

At this point in my answer 👆 I outlined a technique for persuading a team and building support for a plan or an idea, especially when you already know it’s gonna be an uphill battle. Hillel Wayne said I should write it up in a blog post, so here it is! (I’ll do anything for free chocolate 😍)

“How can I get people on board with my controversial plan?”

So you have a great idea, and you’re eager to get started. Awesome!!! You believe it’s going to make people’s lives better, even though you know you are going to have to fight tooth and nail to make it happen.

What NOT to do:

Walk into the team meeting and drop your bomb idea on everyone cold:

“Hey, I think we should stop shipping product changes until we fix our build pipeline to the point where we can auto-deploy each merge set to production, one at a time, in under an hour.” ~ (for example)

…. then spend the rest of the hour grappling with everybody’s thoughts, feelings, and intense emotional reactions, before getting discouraged and slinking away, vowing to never have another idea, ever again.

What to do instead:

Suss out your audience. Who will be there? How are they likely to react? Are any of them likely to feel especially invested in the existing solution, maybe because they built it? Are any of them known for their strong opinions or being combative?

Great!!! Your first move is to have a conversation with each of them. Approach them in the spirit of curiosity, and ask what they think of your idea. Talking with them will also help you hash out the details and figure out if it is actually a good idea or not.

Your goal is to make the rounds, ask for advice, identify any allies, and talk your idea through with anybody who is likely to oppose you…before the meeting where you intend to unveil your plan. So that when that happens, you have:

  1. given people the chance to process their reactions and ask questions in private
  2. ensured that key people will not feel surprised, threatened, or out of the loop
  3. already heard and discussed any objections
  4. ideally, you have earned their support!

Even if you didn’t manage to convince every person, this was still a valuable exercise. By approaching people in advance, you are signaling that you respect them and their voice matters. You are always going to get people’s absolute worst reactions when you spring something on them in a group setting; any anxiety or dismay will be amplified tenfold. By letting them reflect and ask questions in private, you’re giving time for their better selves to emerge.

What to do instead…if you’re a manager:

As an engineer or a tech lead, you sometimes end up out front and visible as the owner of a change you are trying to drive. This is normal. But as a manager, there are far more times when you need to influence the group but not be the leader of the change, or when you need to be wary of sounding like you are telling people what to do. These are just a few of the many reasons it can be highly effective to have other people arguing on your behalf.

In the ideal scenario, particularly on technical topics, you don’t have to push for anything. All you do is pose the question, then sit back and listen as vigorous debate ensues, with key stakeholders and influential engineers arguing for your intended outcome. That’s a good sign that not only are they convinced, they feel ownership over the decision and its execution. This is the goal! 🌈

It’s not just about persuading people to agree with you, either. Instead of having a shitty dynamic where engineers are attached to the old way of doing things and you are “dragging them” into the newer ways against their will, you are inviting them to partner with you. You are offering them the opportunity to lead the team into the brave new world, by getting on board early.

(It probably goes without saying, but always start with the smallest relevant group of stakeholders, and not, say, all of engineering, or a group that has no ownership over the given area. 🙃 And … even this strategy will stop working rather quickly, if your controversial ideas all turn out to be disastrous. 😉)

“How do I know where to even start?!? đŸ˜±”

Before I wrap up, I want to circle back to the question from the tech lead about how to drive change on a team when you do have some influence or power. He went on to say (or maybe this was from a third questioner?*):

“There is SO MUCH I’d like to do or change with our culture and our tech stack. Where can I even start??”

Yeah, it can be pretty overwhelming. And there are no universal answers… as you know perfectly well, the answer is always “it depends.” â˜ș But in most cases you can reduce the solution space substantially to one of the two following starting points.

1. Can you understand what’s going on in your systems? If not, start with observability.

It doesn’t have to be elegant or beautiful; grepping through shitty text logs is fine, if it does the trick. But do any of the following make you shudder in recognition?:

  • If I get paged, I might lose the rest of the afternoon trying to figure out what happened
  • Our biggest problem is performance and we don’t know where the time is going
  • We have a lot of flaky, flappy alerts, and unexplained outages that simply resolve themselves without our ever truly understanding what happened.

If you can’t understand what’s going on in your system, you have to start with instrumentation and observability. It’s just too deadly, and too risky, not to. You’re going to waste a ton of time stabbing around in the dark trying to do anything else without visibility. Put your glasses on before you start driving down the freeway, please.

2. Can you build, test and deploy software in under an hour? If not, start with your deploy pipeline.

Specifically, the interval of time between when the code is written and when it’s being used in production. Make it shorter, less flaky, more reliable, more automated. This is the feedback loop at the heart of software engineering, which means that it’s upstream from a whole pile of pathologies and bullshit that creep in as a consequence of long, painful, batched-up deploys.

Here’s a talk I’ve given a few times on why this matters so much:

You pretty much can’t fail with one of those two; your lives will materially improve as you make progress. And the iterative process of doing them will uncover a great deal of shit you should probably know about.

Cheers! đŸ„‚

charity.

* My apologies if I remembered anyone’s question inaccurately!

Questionable Advice: “How can I drive change and influence teams…without power?”

Questionable Advice #2: How Do I Get My Team Into Observability?

mipsytipsy, , , , , , , 1 Comment

Welcome to the second installment of my advice column! Last time we talked about the emotional impact of going back to engineering after a stint in management. If you have a question you’d like to ask, please email me or DM it to me on twitter.

Hi Charity! I hope it’s ok to just ask you this… 

I’m trying to get our company more aware of observability and I’m finding it difficult to convince people to look more into it. We currently don’t have the kind of systems that would require it much – but we will in future and I want us to be ahead of the game. 

If you have any tips about how to explain this to developers (who are aware that quality is important but don’t always advocate for it / do it as much as I’d prefer), or have concrete examples of “here’s a situation that we needed observability to solve – and here’s how we solved it”, I’d be super grateful. 

If this is too much to ask, let me know too 🙂 

I’ve been talking to Abby Bangser a lot recently – and I’m “classifying” observability as “exploring in production” in my mental map – if you have philosophical thoughts on that, I’d also love to hear them 🙂

alex_schl

 

Dear Alex,

Everyone’s systems are broken. Not just yours!

Yay, what a GREAT note!  I feel like I get asked some subset or variation of these questions several times a week, and I am delighted for the opportunity to both write up a response for you and post it for others to read.  I bet there are orders of magnitude more people out there with the same questions who *don’t* ask, so I really appreciate those who do. <3

I want to talk about the nuts and bolts of pitching to engineering teams and shepherding technical decisions like this, and I promise I will offer you some links to examples and other materials. But first I want to examine some of the assumptions in your note, because they elegantly illuminate a couple of common myths and misconceptions.

Myth #1: you don’t need observability til you have problems of scale

First of all, there’s this misconception that observability is something you only need when you have really super duper hard problems, or that it’s only justified when you have microservices and large distributed systems or crazy scaling problems.  No, no no nononono. 

There may come a point where you are ABSOLUTELY FUCKED if you don’t have observability, but it is ALWAYS better to develop with it.  It is never not better to be able to see what the fuck you are doing!  The image in my head is of a hiker with one of those little headlamps on that lets them see where they’re putting their feet down.  Most teams are out there shipping opaque, poorly understood code blindly — shipping it out to systems which are themselves crap snowballs of opaque, poorly understood code. This is costly, dangerous, and extremely wasteful of engineering time.


Ever seen an engineering team of 200, and struggled to understand how the product could possibly need more than one or two teams of engineers? They’re all fighting with the crap snowball.

Developing software with observability is better at ANY scale.  It’s better for monoliths, it’s better for tiny one-person teams, it’s better for pre-production services, it’s better for literally everyone always.  The sooner and earlier you adopt it, the more compounding value you will reap over time, and the more of your engineers’ time will be devoted to forward progress and creating value.

Myth #2: observability is harder and more technically advanced than monitoring

Actually, it’s the opposite — it’s much easier.  If you sat a new grad down and asked them to instrument their code and debug a small problem, it would be fairly straightforward with observability. Observability speaks the native language of variables, functions and API endpoints, the mental model maps cleanly to the request path, and you can straightforwardly ask any question you can come up with. (A key tenet of observability is that it gives an engineer the ability to ask any question, without having had to anticipate it in advance.)

With metrics and logging libraries, on the other hand, it’s far more complicated.you have to make a bunch of awkward decisions about where to emit various types of statistics, and it is terrifyingly easy to make poor choices (with terminal performance implications for your code and/or the remote data source).  When asking questions, you are locked in to asking only the questions that you chose to ask a long time ago. You spend a lot of time translating the relationships between code and lowlevel systems resources, and since you can’t break down by users/apps you are blocked from asking the most straightforward and useful questions entirely!  

Doing it the old way Is. Fucking. Hard.  Doing it the newer way is actually much easier, save for the fact that it is, well, newer — and thus harder to google examples for copy-pasta. But if you’re saturated in decades of old school ops tooling, you may have some unlearning to do before observability seems obvious to you.

Myth #3: observability is a purely technical solution

To be clear, you can just add an observability tool to your stack and go on about your business — same old things, same old way, but now with high cardinality!

You can, but you shouldn’t.  

These are sociotechnical systems and they are best improved with sociotechnical solutions.  Tools are an absolutely necessary and inextricable part of it.  But so are on call rotations and the fundamental virtuous feedback loop of you build it, you run it.  So are code reviews, monitoring checks, alerts, escalations, and a blameless culture.  So are managers who allocate enough time away from the product roadmap to truly fix deep technical rifts and explosions, even when it’s inconvenient, so the engineers aren’t in constant monkeypatch mode.

I believe that observability is a prerequisite for any major effort to have saner systems, simply because it’s so powerful being able to see the impact of what you’ve done.  In the hands of a creative, dedicated team, simply wearing a headlamp can be transformational.

Observability is your five senses for production.

You’re right on the money when you ask if it’s about exploring production, but you could also use words that are even more basic, like “understanding” or “inspecting”.  Observability is to software systems as a debugger is to software code.  It shines a light on the black box.  It allows you to move much faster, with more confidence, and catch bugs much sooner in the lifecycle — before users have even noticed.  It rewards you for writing code that is easy to illuminate and understand in production.

So why isn’t everyone already doing it?  Well, making the leap isn’t frictionless.  There’s a minimal amount of instrumentation to learn (easier than people expect, but it’s nonzero) and then you need to learn to see your code through the lens of your own instrumentation.  You might need to refactor your use of older tools, such as metrics libraries, monitoring checks and log lines.  You’ll need to learn another query interface and how it behaves on your systems.  You might find yourself amending your code review and deploy processes a bit.  

Nothing too terrible, but it’s all new.  We hate changing our tool kits until absolutely fucking necessary.  Back at Parse/Facebook, I actually clung to my sed/awk/shell wizardry until I was professionally shamed into learning new ways when others began debugging shit faster than I could.  (I was used to being the debugger of last resort, so this really pissed me off.)  So I super get it!  So let’s talk about how to get your team aligned and hungry for change.

Okay okay okay already, how do I get my team on board?

If we were on the phone right now, I would be peppering you with a bunch of questions about your organization.  Who owns production?  Who is on call?  Who runs the software that devs write?  What is your deploy process, and how often does it get updated, and by who?  Does it have an owner?  What are the personalities of your senior folks, who made the decisions to invest in the current tools (and what are they), what motivates them, who are your most persuasive internal voices?  Etc.  Every team is different.  <3

There’s a virtuous feedback loop you need to hook up and kickstart and tweak here, where the people with the original intent in their heads (software engineers) are also informed and motivated, i.e. empowered to make the changes and personally impacted when things are broken. I recommend starting by putting your software engineers on call for production (if you haven’t).  This has a way of convincing even the toughest cases that they have a strong personal interest in quality and understandability. 

Pay attention to your feedback loop and the alignment of incentives, and make sure your teams are given enough time to actually fix the broken things, and motivation usually isn’t a problem.  (If it is, then perhaps another feedback loop is lacking: your engineers feeling sufficiently aligned with your users and their pain.  But that’s another post.)

Technical ownership over technical outcomes

I appreciate that you want your team to own the technical decisions.  I believe very strongly that this is the right way to go.  But it doesn’t mean you can’t have influence or impact, and particularly in times like this. 

It is literally your job to have your head up, scanning the horizon for opportunities and relevant threats.  It’s their job to be heads down, focusing on creating and delivering excellent work.  So it is absolutely appropriate for you to flag something like observability as both an opportunity and a potential threat, if ignored.

If I were in your situation and wanted my team to check out some technical concept, I might send around a great talk or two and ask folks to watch it, and then maybe schedule a lunchtime discussion.  Or I might invite a tech luminary in to talk with the team, give a presentation and answer their questions.  Or schedule a hack week to apply the concept to a current top problem, or something else of that nature.

But if I really wanted them to take it fucking seriously, I would put my thumb on the scale.  I would find myself a champion, load them up with context, and give them ample time and space to skill up, prototype, and eventually present to the team a set of recommendations.  (And I would stay in close contact with them throughout that period, to make sure they didn’t veer too far off course or lose sight of my goals.)

  1. Get a champion.

    Ideally you want to turn the person who is most invested in the old way of doing things — the person who owns the ELK cluster, say, or who was responsible for selecting the previous monitoring toolkit, or the goto person for ops questions — from your greatest obstacle into your proxy warrior.  This only works if you know that person is open-minded and secure enough to give it a fair shot & publicly change course, has sufficiently good technical judgment to evaluate and project into the future, and has the necessary clout with their peers.  If they don’t, or if they’re too afraid to buck consensus: pick someone else.

  2. Give them context.  

    Take them for a long walk.  Pour your heart and soul out to them.  Tell them what you’ve learned, what you’ve heard, what you hope it can do for you, what you fear will happen if you don’t.  It’s okay to get personal and to admit your uncertainties.  The more context they have, the better the chance they will come out with an outcome you are happy with.  Get them worried about the same things that worry you, get them excited about the same possibilities that excite you.  Give them a sense of the stakes. 

    And don’t forget to tell them why you are picking them — because they are listened to by their peers, because they are already expert in the problem area, because you trust their technical judgment and their ability to evaluate new things — all the reasons for picking them will translate well into the best kind of flattery — the true kind.  

  3. Give them a deadline.

    A week or two should be plenty.  Most likely, the decision is not going to be unilaterally theirs (this also gives you a bit of wiggle room should they come back going “ah no ELK is great forever and ever”), but their recommendations should carry serious weight with the team and technical leadership.  Make it clear what sort of outcome you would be very pleased with (e.g. a trial period for a new service) and what reasons you would find compelling for declining to pursue the project (i.e. your tech is unsupported, cost prohibitive, etc).  Ideally they should use this time to get real production data into the services they are testing out, so they can actually experience and weigh the benefits, not just read the marketing copy.

As a rule of thumb, I always assume that managers can’t convince engineers to do things: only other engineers can.  But what you can do instead is set up an engineer to be your champion.  And then just sit quietly in the corner, nodding, with an interested look on your face.

The nuclear option


if you <3 prod,
prod will <3 you back

You have one final option.  If there is no appropriate champion to be found, or insufficient time, or if you have sufficient trust with the team that you judge it the right thing to do: you can simply order them to do something your way.  This can feel squicky. It’s not a good habit to get into.  It usually results in things being done a bit slower, more reluctantly, more half-assedly. And you sacrifice some of your power every time you lean on your authority to get your team to do something.

But it’s just as bad for a leader to take it off the table entirely.

Sometimes you will see things they can’t.  If you cannot wield your power when circumstances call for it, then you don’t fucking have real power — you have unilaterally disarmed yourself, to the detriment of your org.  You can get away with this maybe twice a year, tops. 

But here’s the thing: if you order something to be done, and it turns out in the end that you were right?  You earn back all the power you expended on it plus interest.  If you were right, unquestionably right in the eyes of the team, they will respect you more for having laid down the law and made sure they did the right thing.

xo

charity

Some useful resources:

 

Questionable Advice #2: How Do I Get My Team Into Observability?

Engineering Management: The Pendulum Or The Ladder

mipsytipsy, , , , , 62 Comments

Last night I was out with a dear friend who has been an engineering manager for a year now, and by two drinks in I was rattling off a long list things I always say to newer engineering managers.

Then I remembered: I should write a post! It’s one of my goals this year to write more long form instead of just twittering off into the abyss.Buffy Jaguar 3.5x5

There’s a piece I wrote two years ago, The Engineer/Manager Pendulum,  which is probably my all time favorite.  It was a love letter to a friend who I desperately wanted to see go back to engineering, for his own happiness and mental health.  Well, this piece is a sequel to that one.

It’s primarily aimed at new managers, who aren’t sure what their career options look like or how to evaluate the opportunities that come their way, or how it may expand or shrink their future opportunities.

The first fork in the manager’s path

Every manager reaches a point where they need to choose: do they want to manage engineers (a “line manager”), or do they want to try to climb the org chart? — manage managers, managers of other managers, even other divisions; while Does Not Kill Us Puppy UPDATEDbeing “promoted” from manager to senior manager, director to senior director, all the way up to VP and so forth.   Almost everyone’s instinct is to say “climb the org chart”, but we’ll talk about why you should be critical of this instinct.

They also face a closely related question: how technical do they wish to stay, and how badly do they care?

Are you an “engineering MANAGER” or an “ENGINEERING manager”?

These are not unlike the decisions every engineer ends up making about whether to go deep or go broad, whether to specialize or be a generalist.  The problem is that both engineers and managers often make these career choices with very little information — or even awareness that they are doing it.

And managers in particular then have a tendency to look up ten years later and realize that those choices, witting or unwitting, have made them a) less employable  and b) deeply unhappy.

Lots of people have the mindset that once they become an engineering manager, they should just go from gig to gig as an engineering manager who manages other engineers: that’s who they are now.  But this is actually a very fragile place to sit long-term, as we’ll discuss further on in this piece.

But let’s start at to the beginning, so I can speak to those of you who are considering management for the very first time.

“So you want to try engineering management.”

COOL! I think lots of senior engineers should try management, maybe even most senior engineers.  It’s so good for you, it makes you better at your job. (If you aren’t a senior engineer, and by that I mean at least 7+ years of engineering experience, be very wary; know this isn’t usually in your best interest.)

Hopefully you have already gathered that management is a career change, not a promotion, and you’re aware that nobody is very good at it when they first start.

That’s okay! It takes a solid year or two to find new rhythms and reward mechanisms before you can even begin to find your own voice or trust your judgment. Management problems look easy, deceptively so.  Reasons this is hard include:

  1. Most tech companies are absolutely abysmal at providing any sort of training or structure to help you learn the ropes and find your feet.
  2. Even if they do, you still have to own your own career development.  If learning to be a good engineer was sort of like getting your bachelor’s, learning to be a good manager is like getting your PhD — much more custom to who you are.
  3. It will exhaust you mentally and emotionally in the weirdest ways for much longer than you think it should.  You’ll be tired a lot, and you’ll miss feeling like you’re good at something (anything).

This is because you need to change your habits and practices, which in turn will actually change who you are.  This takes time.  Which is why …

The minimum tour of duty as a new manager is two years.

If you really want to try being a manager, and the opportunity presents itself, do it!  But only if you are prepared to fully commit to a two year long experiment.

Root Causes DolphinCommit to it like a proper career change. Seek out new peers, find new heroes. Bring fresh eyes and a beginner’s mindset. Ask lots of questions. Re-examine every one of your patterns and habits and priorities: do they still serve you? your team?

Don’t even bother thinking about in terms of whether you “enjoy managing” for a while, or trying to figure out if you are are any good at it. Of course you aren’t any good at it yet.  And even if you are, you don’t know how to recognize when you’ve succeeded at something, and you haven’t yet connected your brain’s reward systems to your successes.  A long stretch of time without satisfying brain drugs is just the price of admission if you want to earn these experiences, sadly.

It takes more than one year to learn management skills and wire up your brain to like it.  If you are waffling over the two year commitment, maybe now is not the time.  Switching managers too frequently is disruptive to the team, and it’s not fair to make them report to someone who would rather be doing something else or isn’t trying their ass off.

It takes about 3-5 years for your skills to deteriorate.

So you’ve been managing a team for a couple years, and it’s starting to feel … comfortable?  Hey, you’re pretty good at this!  Yay!

With a couple of years under your belt as a line manager, you now have TWO powerful skill sets.  You can build things, AND you can organize people into teams to build even bigger things. Right now, both sets are sharp.  You could return to engineering pretty easily, or keep on as a manager — your choice.

But this state of grace doesn’t last very long. Your technical skills stop advancing when you become a manager, and instead begin eroding.  Two years in, you aren’t the effective tech lead you once were; your information is out of date and full of gaps, the hard parts are led by other people these days.

More critically, your patterns of mind and habits shift over time, and become those of a manager, not an engineer.  Consider how excited an engineer becomes at the prospect of a justifiable greenfield project; now compare to her manager’s glum reaction as she instinctively winces at having to plan for something so reprehensibly unpredictable and difficult to estimate.  It takes time to rewire yourself back.

If you like engineering management, your tendency is to go “cool, now I’m a manager”, and move from job to job as an engineering manager, managing team after team of engineers.  But this is a trap.  It is not a sound long term plan.  It leads too many people off to a place they never wanted to end up: technically sidelined.

Sunglasses Tiger Debugger 3.3x5

Why can’t I just make a career out of being a combo tech lead+line manager?

One of the most common paths to management is this: you’re a tech lead, you’re directing ever larger chunks of technical work, doing 1x1s and picking up some of the people stuff, when your boss asks if you’d like to manage the team.  “Sure!”, you say, and voila — you are an engineering manager with deep domain expertise.

But if you are doing your job, you begin the process of divesting yourself of technical leadership responsibilities starting immediately.  Your own technical development should screech to a halt once you become a manager, because you have a whole new career to focus on learning.

Your job is to leverage that technical expertise to grow your engineers into great senior engineers and tech leads themselves.  Your job is not to hog the glory and squat on the hard problems yourself, it’s to empower and challenge and guide your team.  Don’t suck up all the oxygen: you’ll stunt the growth of your team.

But your technical knowledge gets dated, and your skills atrophy..  The longer it’s been since you worked as an engineer, the harder it will be to switch back.  It gets real hard around three years, and five years seems like a tipping point.[1]

And because so much of your credibility and effectiveness as an engineering leader comes from your expertise in the technology that your team uses every day, ultimately you will be no longer capable of technical leadership, only people management.

On being an “engineering manager” who only does people management

I mean, there’s a reason we don’t lure good people managers away from Starbucks to run engineering teams.  It’s the intersection and juxtaposition of skill sets that gives engineering managers such outsize impact.

The great ones can make a large team thrum with energy.  The great ones can break down a massive project into projects that challenge (but do not overwhelm) a dozen or more engineers, from new grads to grizzled veterans, pushing everyone to grow.  The great ones can look ahead and guess which rocks you are going to die on if you don’t work to avoid them right now.

The great ones are a treasure: and they are rare.  And in order to stay great, they regularly need to go back to the well to refresh their own hands-on technical abilities.

Pointless Ice Cream 3x2.5There is an enormous demand for technical engineering leaders — far more demand than supply.  The most common hackaround is to pair a people manager (who can speak the language and knows the concepts, but stopped engineering ages ago) with a tech lead, and make them collaborate to co-lead the team.  This unwieldy setup often works pretty well.

But most of those people managers didn’t want or expect to end up sidelined in this way when they were told to stop engineering.

If you want to be a pure people manager and not do engineering work, and don’t want to climb the ladder or can’t find a ladder to climb, more power to you.  I don’t know that I’ve met many of these people in my life.  I have met a lot of people in this situation by accident, and they are always kinda angsty and unhappy about it.  Don’t let yourself become this person by accident.  Please.

Which brings me to my next point.

You will be advised to stop writing code or engineering.

✹

Fuck

That.

 ✹

Everybody’s favorite hobby is hassling new managers about whether or not they’ve stopped writing code yet, and not letting up until they say that they have.  This is a terrible, horrible, no-good VERY bad idea that seems like it must originally have been a botched repeating of the correct advice, which is:

Stop writing code and engineering

in the critical path

Can you spot the difference?  It’s very subtle.  Let’s run a quick test:

  • Authoring a feature?  ⛔
  • Covering on-call when someone needs a break?  ✅
  • Diving on the biggest project after a post mortem?  ⛔
  • Code reviews?  ✅
  • Picking up a p2 bug that’s annoying but never seems to become top priority?  ✅
  • Insisting that all commits be gated on their approval?  ⛔
  • Cleaning up the monitoring checks and writing a library to generate coverage?  ✅

The more you can keep your hands warm, the more effective you will be as a coach and a leader.  You’ll have a richer instinct for what people need and want from you and each other, which will help you keep a light touch.  You will write better reviews and resolve technical disputes with more authority.  You will also slow the erosion and geriatric creep of your own technical chops.

I firmly believe every line manager should either be in the on call rotation or pinch hit liberally and regularly, but that’s a different post.

Technical Leadership track

If you  love technology and want to remain a subject-matter expert in designing, building and shipping cutting-edge technical products and systems, you cannot afford to let yourself drift too far or too long away from hands-on engineering work.  You need to consciously cultivate your path , probably by practicing some form of the engineer/manager pendulum.

If you love managing engineers — if being a technical leader is a part of your identity that you take great pride in, then you must keep up your technical skills and periodically DIstrust Kittens 2.5x3invest in your practice and renew your education.  Again: this is simply the price of admission.  You need to renew your technical abilities, your habits of mind, and your visceral senses around creating and maintaining systems.  There is no way to do this besides doing it.  If management isn’t a promotion, then returning to hands-on work isn’t a demotion, either.  Right?

One warning: Your company may be great, but it doesn’t exist for your benefit.  You and only you can decide what your needs are and advocate for them.  Remember that next time your boss tries to guilt you into staying on as manager because you’re so badly needed, when you can feel your skills getting rusty and your effectiveness dwindling.  You owe it to yourself to figure out what makes you happy and build a portfolio of experiences that liberate you to do what you love.  Don’t sacrifice your happiness at the altar of any company.  There are always other companies.

Honestly, I would try not to think of yourself as a manager at all: you are an “engineering leader” performing a tour of duty in management.  You’re pursuing a long term strategy towards being a well-respected technologist, someone who can sling code, give informed technical guidance and explain in detail customized for to anyone at any level of sophistication.

Organizational Leadership Track

Most managers assume they want to climb the ladder.  Leveling up feels like an achievement, and that can feel impossible to resist.

Resist it.  Or at least, resist doing it unthinkingly.  Don’t do it because the ladder is there and must be climbed.  Know as much as you can about what you’re in for before you decide it’s what you want.

Here are a few reasons to think critically about climbing the ladder to director and executive roles.

  1. Your choices shrink. There are fewer jobs, with more competition, mostly at bigger companies.  (Do you even like big companies?)
  2. You basically need to do real time at a big company where they teach effective management skills, or you’ll start from a disadvantage.
  3. Bureaucracies are highly idiosyncratic, skills and relationships may or may not transfer with you between companies. As an engineer you could skip every year or two for greener pastures if you landed a crap gig.  An engineer has … about 2-3x more leeway in this regard than an exec does.  A string of short director/exec gigs is a career ender or a coach seat straight to consultant life.
  4. You are going to become less employable overall.  The ever-higher continuous climb almost never happens, usually for reasons you have no control over.  This can be a very bitter pill.
  5. Your employability becomes more about your “likability” and other problematic things.  Your company’s success determines the shape of your career much more than your own performance.  (Actually, this probably begins the day you start managing people.)
  6. Your time is not your own. Your flaws are no longer cute. You will see your worst failings ripple outward and be magnified and reflected.  (Ditto, applies to all leaders but intensifies as you rise.)
  7. You may never feel the dopamine hit of “i learned something, i fixed something, i did something” that comes so freely as an I.C.  Some people learn to feel satisfaction from managery things, others never do.  Most describe it as a very subdued version of the thrill you get from building things.
  8. You will go home tired every night, unable to articulate what you did that day. You cannot compartmentalize or push it aside. If the project failed for reasons outside your control, you will be identified with the failure anyway.
  9. Nobody really thinks of you as a person anymore, you turn into a totem for them to project shit on. (Things will only get worse if you hit back.)  Can you handle that?  Are you sure?
  10. It’s pretty much a one-way trip.

Sure, there are compensating rewards.  Money, power, impact.  But I’m pointing out the negatives because most people don’t stop to consider them when they start saying they want to try managing managers.  Every manager says that.

The mere existence of a ladder compels us all to climb.

I know people who have climbed, gotten stuck, and wished they hadn’t. I know people who never realized how hard it would be for them to go back to something they loved doing after 5+ years climbing the ladder farther and farther away from tech.  I know some who are struggling their way back, others who have no idea how or where to start.  For those who try, it is hard.  

You can’t go back and forth from engineering to executive, or even director to manager, in the way you can traverse freely between management and engineering as a technologist.

I just want more of you entering management with eyes wide open.  That’s all I’m saying.

If you don’t know what you want, act to maximize your options.

Engineering is a creative act. Managing engineers will require your full attentive and authentic self. You will be more successful if you figure out what that self is, and honor its needs.  Try to resist the default narratives about promotions and titles and roles, they have nothing to do with what satisfies your soul.  If you have influence, use it to lean hard against things like paying managers more than ICs of the same level.[2]

It’s totally normal not to know who you want to be, or have some passionate end goal.  It’s great to live your life and work your work and keep an eye out for interesting opportunities, and see what resonates.  It’s awesome when you get asked to step up and opportunistically build on your successes.

If you want a sustainable career in tech, you are going to need to keep learning your whole life. The world is changing much faster than humans evolved to naturally adapt, so you need to stay a little bit restless and unnaturally hungry to succeed in this industry.

The best way to do that is to make sure you a) know yourself and what makes you happy, b) spend your time mostly in alignment with that. Doing things that make you happy give you energy. Doing things that drain you are antithetical to your success. Find out what those things are, and don’t do them. 

Don’t be a martyr, don’t let your spending habits shackle you, and don’t build things that trouble your conscience.

And have fun.

Yours in inverting $(allthehierarchies),
charity.

img_5680

 

[1] Important point: I am not saying you can’t pick up the skills and patience to practice engineering again.  You probably can!  But employers are extremely reluctant to pay you a salary as an engineer if you haven’t been paid to ship code recently.  The tipping point for hireability comes long before the tipping point for learning ability, in my experience.

[2] It is in no one’s best interest for money to factor into the decision of whether to be a manager or not.  Slack pays their managers LESS than engineers of the same level, and I think this is incredibly smart: sends a strong signal of servant leadership.

 

Engineering Management: The Pendulum Or The Ladder

On Engineers and Influence

mipsytipsy, , , , , , 5 Comments

(Based on yesterday’s tweetstorm and the ensuing conversation, https://twitter.com/mipsytipsy/status/1029608573217587201)

Let’s talk about influence. As an engineer, how do you get influence? What does influence look like, what is it rooted in, how do you wield it or lose it? How is it different from the power and influence you might have as a manager?[0]

This often comes up in the context of ICs who desperately want to become managers in order to have more access to information and influence over decisions. This is a bad signal, though it’s sadly very common.

When that happens, you need to do some soul-searching. Does your org make space for senior ICs to lead and own decisions? Do you have an IC track that runs parallel to the manager track at least as high as director level? Are they compensated equally? Do you  have a career ladder? Are your decision-making processes mysterious to anyone who isn’t a manager? Don’t assume what’s obvious to you is obvious to others; you have to ask around.

If so, it’s probably their own personal baggage speaking. Maybe they don’t believe you. Maybe they’ve only worked in orgs where managers had all the power. Maybe they’ve even worked in lots of places that said the exact same things as you are saying about how ICs can have great impact, but it was all a lie and now they’re burned. Maybe they aren’t used to feeling powerful for all kinds of reasons.

Regardless, people who want to be managers in order to perpetuate a bad power structure are the last people you want to be managers.[1]

But what does engineering influence look like?  How do your powers manifest?

I am going to avoid discussing the overlapping and interconnected issues of gender, race and class, let’s just acknowledge that it’s much more structurally difficult for some to wield power than for others, ok?

The power to create

Doing is the engineering superpower. We create things with just a laptop and our brain! It’s incredible! We don’t have to constantly convince and cajole and coerce others into building on our behalf, we can just build.

This may seem basic, but it matters. Creation is the ur-power from which all our forms of power flow. Nothing gets built unless we agree to build it (which makes this an ethical issue, too).

Facebook had a poster that said “CODE WINS ARGUMENTS”. Problematic in many ways, absolutely. But how many times have you seen a technical dispute resolved by who was willing to do the work? Or “resolved” one way.. then reversed by doing? Doing ends debates. Doing proves theories. Doing is powerful. (And “doing” doesn’t only mean “write code”.)

Furthermore, building software is a creative activity, and doing it at scale is an intensely communal one. As a creative act, we are better builders when we are motivated and inspired and passionate about our work (as compared to say, chopping wood). And as a collaborative act, we do better work when we have high trust and social cohesion.

Engineering ability and judgment, autonomy and sense of purpose, social trust and cooperative behaviors: this is the raw stuff of great engineering. Everybody has a mode or two that they feel most comfortable and authoritative operating from: we can group these roughly into archetypes.

(Examples drawn from some of the stupendously awesome senior engineers I’ve gotten to work with over the years, as well as the ways I loved to fling my weight around as an engineer.)

Archetypes of influence

  • “Doing the work that is desperately hard and desperately needed — and often desperately dull.” SOC2 compliance, backups and restores, terrifying refactors, any auth integration ever: if it’s moving the business forward, they don’t give a shit how dull the work is. If you are this engineer, you have a deep well of respect and gratitude.
  • “Debugger of last resort.” Often the engineer who has been there the longest or originally built the system. If you are helpful and cheerful with your history and context, this is a huge asset. (People tend to wildly overestimate this person’s indispensability, actually; please don’t encourage this.)Image result for engineer software meme manager
  • The “expert” archetype is closely related. If you are the deep subject matter expert in some technology component, you have a shit ton of influence over anything that uses or touches that component. (You should stay up on impending changes to retain your edge.)
  • There are people who deliver a bafflingly powerful firehose of sustained output, sometimes making headway on multiple fronts at once. Some work long hours, others just have an unerring instinct for how to maximize impact (this sometimes maps to junior/senior manifestations). Nobody wants to piss off those people. Their consent is critical for … everything. Their participation will often turbo charge a project or pull a foundering effort over the finish line.

Not all influence is rooted in raw technical strength or output.  Just a few of the wide variety of creative/collaborative/interpersonal strengths:

  • Some engineers are infinitely curious, and have a way of consistently sniffing a few steps ahead of the pack. They might seem to be playing around with something pointless, and you want to scold them; then they save your ass from total catastrophe. You learn to value their playing around.
  • Some engineers solve problems socially, by making friends and trading tips and fixes and favors in the industry. Don’t underestimate social debugging, it’s often the quickest path to the right answer.
  • Some are dazzlingly lazy and blow your mind with their elegant shortcuts and corners correctly cut.
  • Some are recruiting magnets, and it’s worth paying their salary just for all the people who want to work with them again.
  • Some are skilled at driving consensus among stakeholders.
  • Some are killer explainers and educators and storytellers.
  • Some are the senior engineer everyone silently wants to grow up to be.
  • Some can tell such an inspiring story of tomorrow that everyone will run off to make it so.
  • Some teach by turning code reviews into a pedagogical art form.
  • Some make everyone around them somehow more productive and effective. Some create relentless forward momentum. Some are good at saying no.

And there are a few special wells of power that bear calling out as such.

  • Engineers who have been managers are worth their weight in gold.  They can translate business goals for junior engineers in their native language with impeccable credibility (something managers never really have, esp in junior engineers’ eyes.). They make strong tech leads, they can carve up projects into components that challenge but do not overwhelm each contributor while hitting deadlines.
  • Some engineers are a royal pain in the ass because they question and challenge every system and hierarchy. But these are sharp, powerful rocks that can polish great teams. Though they do require a strong manager, to channel their energy towards productive dialogue and improvement and keep them from pissing off the whole team.
  • And let’s not forget engineers who are on call. If you have a healthy on call culture, your ownership over production creates a deep, deep well of power and moral authority — to make demands, drive change, to prioritize. On call should not be a shit salad served up to those who can’t refuse, it should be a badge of honor and seriousness shouldered by every engineer who ships code. (And it should not be miserable or regularly life-impacting.)

… I could go on all day. Engineering is such a powerful role and skill set. It’s definitely worth unpacking where your own influence comes from, and understanding how others perceive your strengths.

Most forms of power boil down to “influence, wielded”.

But just banging out code is not enough. You may have credibility, but having it is not the same as using it. To transform influence into power you have to use it.  And the way you use it is by communicating.

What’s locked up in your head has no impact on the rest of us.  You have to get it out.

You can do this in lots of ways: by writing, in 1x1s, conversations with small groups, openly recruiting allies, convincing someone with explicit authority, broadcasting inpublic, etc.

Because engineering is a creative activity, authoritarian power is actually quite brittle and damaging. The only sustainable forms of power are so-called “soft powers” like influencing and inspiring, which is why good managers use their soft power freely and hard power sparingly/with great reluctance. If your leadership invokes authority on the regular, that’s an antipattern.[2]

If you don’t speak up, you don’t have the right to sit and fume over your lack of influence. And speaking up does mean being vulnerable — and sometimes wrong — in front of other people.

This is not a zero-sum game.

Most of you have far more latent power than you realize or are used to wielding, because you don’t feel powerful or don’t recognize what you do in those terms.

Managers may have hard power and authority, but the real meaty decisions about technical delivery and excellence are more properly made by the engineers closest to them. These belong properly to the doers, in large part because they are the ones who have to support the consequences of these decisions.

Power tends to flow towards managers because they are privy to more information. That makes it important to hire managers who are aware of this and lean against it to push power back to others.

In the same way that submissives have ultimate power in healthy BDSM relationships, engineers actually have the ultimate power in healthy teams. You have the ultimate veto: you can refuse to create.  Demand is high for your skills.  You can usually afford to look for better conditions. Many of you probably should.

And when technical and managerial priorities collide, who wins? Ideally you work together to find the best solution for the business and the people. The teams that feel đŸ”„on fiređŸ”„ always have tight alignment between the two.

Pick your battles.

One final thought. You can have a lot of say in what gets built and how it gets built, if you cultivate your influence and spend it wisely. But you can’t have a say in everything. It doesn’t work that way.

Think of it like @mcfunley’s famous “innovation tokens”, but for attention and fucks given.
Image result for engineer software meme
The more you use your influence for good outcomes, the more you build up over time, yes … but it’s a precision tool, not background noise. Imagine someone trying to give you a massage by laying down on your whole back instead of pushing their elbow or hand into knots and trigger points. A too-broad target will diffuse your force and limit your potential impact.

Spend your attention tokens wisely.

And once you have influence, don’t forget to use it on behalf of others. Pay attention to those who aren’t being heard, and amplify their voices. Give your time, lend your patronage and credibility, and most of all teach the skills that have made you powerful to others who need them.

charity

P.S. I owe a huge debt to all the awesome senior engineers i’ve gotten to work with.  Mad love to you all.  <3

  • [0] I successfully answered one (1) of these questions before running out of steam.  Later. 
  • [1] Sheepish confession: this is why I became a manager.
  • [2] It’s also a bad sign if they won’t grant any explicit authority to the people they hold responsible for outcomes. I’m talking about relatively healthy orgs here, not pathological ones where people (often women) are told they don’t need promotions or explicit authority, they should just use their “soft power” — esp when the hard forms of power aligned against with them. That’s setting you up for failure.
  • [3] Some people seem caught off guard by my use of “power” to signal anything other than explicit granted powers by the org. This doesn’t make any sense to me. I find it too depressing and disempowering to think of power as merely granted authority. It doesn’t map to how I experience the world, either. Individual clout is a thing that waxes and wanes and only exists in relation to others’. I’ve seen plenty of weak managers pushed around by strong personalities (which is terrible too).
On Engineers and Influence