On How Long it Takes to Know if a Job is Right for You or Not

A few eagle-eyed readers have noticed that it’s been 4 weeks since my last entry in what I have been thinking of as my “niblet series” — one small piece per week, 1000 words or less, for the next three months.

This is true. However, I did leave myself some wiggle room in my original goal, when I said “weeks when I am not traveling”, knowing I was traveling 6 of the next 7 weeks. I was going to TRY to write something on the weeks I was traveling, but as you can see, I mostly did not succeed. Oh well!

Honestly, I don’t feel bad about it. I’ve written well over 1k words on bsky over the past two weeks in the neverending thread on the costs and tradeoffs of remote work. (A longform piece on the topic is coming soon.) I also wrote a couple of lengthy internal pieces.

This whole experiment was designed to help me unblock my writing process and try out new habits, and I think I’m making progress. I will share what I’m learning at a later date, but for now: onward!

How long does it take to form an impression of a new job?

This week’s niblet was inspired by a conversation I had yesterday with an internet friend. To paraphrase (and lightly anonymize) their question:

“I took a senior management role at this company six months ago. My search for this role was all about values alignment, from company mission to leadership philosophy, and the people here said all the right things in the process. But it’s just not clicking.

It’s only been six months, but it’s starting to feel like it might not work out. How much longer should I give it?”

Zero. You should give it 0 time. You already know, and you’ve known for a long time; it’s not gonna change. I’m sorry. 💔

I’m not saying you should quit tomorrow, a person needs a paycheck, but you should probably start thinking in terms of how to manage the problem and extricate yourself from it, not like you’re waiting to see if it will be a good fit.

Every job I’ve ever had has made a strong first impression

I’ve had…let’s see…about six different employers, over the course of my (post-university) career.

Every job I’ve ever taken, I knew within the first week whether it was right for me or not. That might be overstating things a bit (memory can be like that). But I definitely had a strong visceral reaction to the company within days after starting, and the rest of my tenure played out more or less congruent with that reaction.Progress not Perfection

The first week at EVERY job is a hot mess of anxiety and nerves and second-guessing yourself and those around you. It’s never warm fuzzies. But at the jobs I ended up loving and staying at long term, the anxiety was like “omg these people are so cool and so great and so fucking competent, I hope I can measure up to their expectations.”

And then there were the jobs where the anxiety I felt was more like a sinking sensation of dread, of “oooohhh god I hope this is a one-off and not the kind of thing I will encounter every day.”

🌸 There was the job where they had an incident on my very first day, and by 7 pm I was like “why isn’t someone telling me I should go home?” There was literally nothing I could do to help, I was still setting up my accounts, yet I had the distinct impression I was expected to stay.

This job turned out to be stereotypically Silicon Valley in the worst ways, hiring young, cheap engineers and glorifying coding all night and sleeping under your desks.

🌼 There was the job where they were walking me through a 50-page Microsoft Word doc on how to manage replication between DB nodes, and I laughed a little, and looked for some rueful shared acknowledgement of how shoddy this was…but I was the only one laughing.

That job turned out to be shoddy, ancient, flaky tech all the way down, with comfortable, long-tenured staff who didn’t know (and did NOT want to hear) how out of date their tech had become.

Over time, I learned to trust that intuition

Around the time I became a solidly senior engineer, I began to reflect on how indelible my early impressions of each job had been, and how reliable those impressions had turned out to be.Communicate Positive Intent

To be clear, I don’t regret these jobs. I got to work with some wonderful people, and I got to experience a range of different organizational structures and types. I learned a lot from every single one of my jobs.

Perhaps most of all, I learned how to sniff out particular environments that really do not work for me, and I never made the same mistake twice.

Companies can and do change dramatically. But absent dramatic action, which can be quite painful, they tend to drift along their current trajectory.

This matters even more for managers

This is one of those ways that I think the work of management is different from the work of engineering. As an experienced IC, it’s possible to phone it in and still do a good job. As long as you’re shipping at an acceptable rate, you can check out mentally and emotionally, even work for people or companies you basically despise.

Lots of people do in fact do this. Hell, I’ve done it. You aren’t likely to do the best work of your life under these circumstances, but people have done far worse to put food on the table.

An IC can wall themselves off emotionally and still do acceptable work, but I’m not sure a manager can do the same.

Alignment *is* the job of management

As a manager, you literally represent the company to your team and those around you. You don’t have to agree with every single decision the company makes, but if you find yourself constantly having to explain and justify things the company has done that deeply violate your personal beliefs or ethics, it does you harm.

Some managers respond to a shitty corporate situation by hunkering down and behaving like a shit umbrella; doing whatever they can to protect their people, at the cost ofif it hurts...do it more undermining the company itself. I don’t recommend this, either. It’s not healthy to know you walk around every day fucking over one of your primary stakeholders, whether it’s the company OR  your teammates.

There are also companies that aren’t actually that bad, but you just aren’t aligned with them. That’s fine. Alignment matters a lot more for managers than for ICs, because alignment is the job.

Management is about crafting and tending to complex sociotechnical systems. No manager can do this alone. Having a healthy, happy team of direct reports is only a fraction of the job description. It’s not enough. You can and should expect more.

What can you learn from the experience?

I asked my friend to think back to the interview process. What were the tells? What do they wish they had known to watch out for?

They thought for a moment, then said:

“Maybe the fact that the entire leadership team had been grown or promoted from within. SOME amount of that is terrific, but ALL of it might be a yellow flag. The result seems to be that everyone else thinks and feels the same way…and I think differently.”

This is SO insightful.

It reminds me of all the conversations Emily and I have had over the years, on how to balance developing talent from within vs bringing in fresh perspectives, people who have already seen what good looks like at the next stage of growth, people who can see around corners and challenge us in different ways.

This is a tough thing to suss out from the outside, especially when the employer is saying all the right things. But having an experience like this can inoculate you from an entire family of related mistakes. My friend will pick up on this kind of insularity from miles away, from now on.

Bad jobs happen. Interviews can only predict so much. A person who has never had a job they disliked is a profoundly lucky person. In the end, sometimes all you can take is the lessons you learned and won’t repeat.

The pig is committed

Have you ever heard the metaphor of the chicken vs the pig? The chicken contributes an egg to breakfast, the pig contributes bacon. The punch line goes something like, “the chicken is involved, but the pig is committed!”

It’s vivid and a bit over the top, but I kept thinking about it while writing this piece. The engineer contributes their labor and output to move the company forward, but the manager contributes their emotional and relational selves — their humanity — to serve the cause.

You only get one career. Who are you going to give your bacon to?

On How Long it Takes to Know if a Job is Right for You or Not

On Dropouts and Bootstraps

In my early twenties I had a cohort of friends and coworkers, all Silicon Valley engineers, all quite good at their jobs, all college dropouts. We developed a shared conviction that only losers got computer science degrees. This sounds like a joke, or a self-defense mechanism, but it was neither. We were serious.

We held CS grads in contempt, as a class. We privately mocked them. When interviewing candidates, we considered it a knock against someone if they graduated — not an insuperable one by any means, but certainly a yellow flag, something to be probed in the interview process, to ensure they had good judgment and were capable of learning independently and getting shit done, despite all evidence to the contrary.

We didn’t look down on ALL college graduates (that would be unreasonable). If you went to school to study something like civil engineering, or philosophy, or Russian literature, good for you! But computers? Everything in my experience led me to conclude that sitting in a classroom studying computers was a waste of time and money.

I had evidence! I worked my way through school — as the university sysadmin, at a local startup — and I had always learned soooo much more from my work than my classes. The languages and technologies they taught us were consistently years out of date. Classes were slow and plodding. Our professors lectured on and on about “IN-dustry” in a way that made it abundantly clear that they had no recent, relevant experience.

College dropouts: the original bootstrappers

The difference became especially stark after I spent a year working in Silicon Valley. I then returned to school, fully intending to finish and graduate, but I could not focus; I was bored out of my skull.

How could anyone sit through that amount of garbage? Wouldn’t anyone with an ounce of self-respect and intrinsic motivation have gotten up off their butts and learned what they needed to know much faster on their own? For fuck’s sake! just google it!

My friends and I rolled our eyes at each other and sighed over these so-called software engineers with degrees, who apparently needed their learning doled out in small bites and spoon-fed to them, like a child. Who wanted to work with someone with such a high tolerance for toil and bullshit?

Meanwhile we, the superior creatures, had simply figured out whatever the fuck we needed to learn by reading the source code, reading books and manuals, trying things out. We pulled OUR careers up by our own bootstraps, goddammit. Why couldn’t they? What was WRONG with them??

We knew so many deeply mediocre software engineers who had gotten their bachelor’s degree in computer science, and so many exceptional engineers with arts degrees or no degrees, that it started to feel like a rule or something.

Were they cherrypicked examples? Of course they were. That’s how these things work.

People are really, really good at justifying their status

Ever since then, I’ve met wave after wave of people in this industry who are convinced they know how to sift “good” talent from “bad” via easily detected heuristics. They’re mostly bullshit.

Which is not to say that heuristics are never useful, or that any of us can afford to expend infinite amounts of time sifting through prospects on the off chance that we miss a couple quality candidates. They can be useful, and we cannot.

However, I have retained an abiding skepticism of heuristics that serve to reinforce existing power structures, or ones that just so happen to overlap with the background of the holder of said heuristics.

Those of us who work in tech are fabulously fortunate; in terms of satisfying, remunerative career outcomes, we are easily in the top .0001% of all humans who have ever lived. Maybe this is why so many of us seem to have some deep-seated compulsion to prove that we belong here, no really, people like me deserve to be here.

This calls for some humility

If nothing else, I think it calls for some humility. I don’t feel like I “deserve” to be here. I don’t think any of us do. I think I worked really fucking hard and I got really fucking lucky. Both can be true. Some of the smartest kids I grew up with are now pumping gas or dead. Almost none of the people I grew up with ever reached escape velocity and made it out of our small town.

When I stop to think about it, it scares me how lucky I got. How lucky I am to have grown up when I did, to have entered tech when I did, when the barriers to entry were so low and you really could just learn on the job, if you were willing to work your ass off. I left home when I was 15 to go to college, and put myself through largely on minimum wage jobs. Even five years later, I couldn’t have done that.

There was a window of time in the 2000s when tech was an escalator to the middle class for a whole generation of weirdos, dropouts and liberal arts misfits. That window has been closed for a while now. I understand why the window closed, and why it was inevitable (software isn’t a toy anymore), but it’s still.. bittersweet.

I guess I’m just really grateful to be here.

~charity

Experiment update

As I wrote last week, I’m trying to reset my relationship with writing, by publishing one short blog post per week: under 1000 words, minimal editing. And there marks week 2, 942 words.

See you next week.

Week 1 — “On Writing, Social Media, and Finding the Line of Embarrassment

 

 

 

On Dropouts and Bootstraps

Generative AI is not going to build your engineering team for you

Originally posted on the Stack Overflow blog on June 10th, 2024

When I was 19 years old, I dropped out of college and moved to San Francisco. I had a job offer in hand to be a Unix sysadmin for Taos Consulting. However, before my first day of work I was lured away to a startup in the city, where I worked as a software engineer on mail subsystems.

I never questioned whether or not I could find work. Jobs were plentiful, and more importantly, hiring standards were very low. If you knew how to sling HTML or find your way around a command line, chances were you could find someone to pay you.

Was I some kind of genius, born with my hands on a computer keyboard? Assuredly not. I was homeschooled in the backwoods of Idaho. I didn’t touch a computer until I was sixteen and in college. I escaped to university on a classical performance piano scholarship, which I later traded in for a peripatetic series of nontechnical majors: classical Latin and Greek, musical theory, philosophy. Everything I knew about computers I learned on the job, doing sysadmin work for the university and CS departments.

In retrospect, I was so lucky to enter the industry when I did. It makes me blanch to think of what would have happened if I had come along a few years later. Every one of the ladders my friends and I took into the industry has long since vanished.

The software industry is growing up

To some extent, this is just what happens as an industry matures. The early days of any field are something of a Wild West, where the stakes are low, regulation nonexistent, and standards nascent. If you look at the early history of other industries—medicine, cinema, radio—the similarities are striking.

There is a magical moment with any young technology where the boundaries between roles are porous and opportunity can be seized by anyone who is motivated, curious, and willing to work their asses off.

It never lasts. It can’t; it shouldn’t. The amount of prerequisite knowledge and experience you must have before you can enter the industry swells precipitously. The stakes rise, the magnitude of the mission increases, the cost of mistakes soars. We develop certifications, trainings, standards, legal rites. We wrangle over whether or not software engineers are really engineers.

Software is an apprenticeship industry

Nowadays, you wouldn’t want a teenaged dropout like me to roll out of junior year and onto your pager rotation. The prerequisite knowledge you need to enter the industry has grown, the pace is faster, and the stakes are much higher, so you can no longer learn literally everything on the job, as I once did.

However, it’s not like you can learn everything you need to know at college either. A CS degree typically prepares you better for a life of computing research than life as a workaday software engineer. A more practical path into the industry may be a good coding bootcamp, with its emphasis on problem solving and learning a modern toolkit. In either case, you don’t so much learn “how to do the job” as you do “learn enough of the basics to understand and use the tools you need to use to learn the job.”

Software is an apprenticeship industry. You can’t learn to be a software engineer by reading books. You can only learn by doing…and doing, and doing, and doing some more. No matter what your education consists of, most learning happens on the job—period. And it never ends! Learning and teaching are lifelong practices; they have to be, the industry changes so fast.

It takes a solid seven-plus years to forge a competent software engineer. (Or as most job ladders would call it, a “senior software engineer”.) That’s many years of writing, reviewing, and deploying code every day, on a team alongside more experienced engineers. That’s just how long it seems to take.

What does it mean to be a “senior engineer”?

Here is where I often get some very indignant pushback to my timelines, e.g.:

“Seven years?! Pfft, it took me two years!”

“I was promoted to Senior Software Engineer in less than five years!”

Good for you. True, there is nothing magic about seven years. But it takes time and experience to mature into an experienced engineer, the kind who can anchor a team. More than that, it takes practice.

I think we have come to use “Senior Software Engineer” as shorthand for engineers who can ship code and be a net positive in terms of productivity, and I think that’s a huge mistake. It implies that less senior engineers must be a net negative in terms of productivity, which is untrue. And it elides the real nature of the work of software engineering, of which writing code is only a small part.

To me, being a senior engineer is not primarily a function of your ability to write code. It has far more to do with your ability to understand, maintain, explain, and manage a large body of software in production over time, as well as the ability to translate business needs into technical implementation. So much of the work is around crafting and curating these large, complex sociotechnical systems, and code is just one representation of these systems.

What does it mean to be a senior engineer? It means you have learned how to learn, first and foremost, and how to teach; how to hold these models in your head and reason about them, and how to maintain, extend, and operate these systems over time. It means you have good judgment, and instincts you can trust.

Which brings us to the matter of AI.

We need to stop cannibalizing our own future

It is really, really tough to get your first role as an engineer. I didn’t realize how hard it was until I watched my little sister (new grad, terrific grades, some hands on experience, fiendishly hard worker) struggle for nearly two years to land a real job in her field. That was a few years ago; anecdotally, it seems to have gotten even harder since then.

This past year, I have read a steady drip of articles about entry-level jobs in various industries being replaced by AI. Some of which absolutely have merit. Any job that consists of drudgery such as converting a document from one format to another, reading and summarizing a bunch of text, or replacing one set of icons with another, seems pretty obviously vulnerable. This doesn’t feel all that revolutionary to me, it’s just extending the existing boom in automation to cover textual material as well as mathy stuff.

Recently, however, a number of execs and so-called “thought leaders” in tech seem to have genuinely convinced themselves that generative AI is on the verge of replacing all the work done by junior engineers. I have read so many articles about how junior engineering work is being automated out of existence, or that the need for junior engineers is shriveling up. It has officially driven me bonkers.

All of this bespeaks a deep misunderstanding about what engineers actually do. By not hiring and training up junior engineers, we are cannibalizing our own future. We need to stop doing that.

Writing code is the easy part

People act like writing code is the hard part of software. It is not. It never has been, it never will be. Writing code is the easiest part of software engineering, and it’s getting easier by the day. The hard parts are what you do with that code—operating it, understanding it, extending it, and governing it over its entire lifecycle.

A junior engineer begins by learning how to write and debug lines, functions, and snippets of code. As you practice and progress towards being a senior engineer, you learn to compose systems out of software, and guide systems through waves of change and transformation.

Sociotechnical systems consist of software, tools, and people; understanding them requires familiarity with the interplay between software, users, production, infrastructure, and continuous changes over time. These systems are fantastically complex and subject to chaos, nondeterminism and emergent behaviors. If anyone claims to understand the system they are developing and operating, the system is either exceptionally small or (more likely) they don’t know enough to know what they don’t know. Code is easy, in other words, but systems are hard.

The present wave of generative AI tools has done a lot to help us generate lots of code, very fast. The easy parts are becoming even easier, at a truly remarkable pace. But it has not done a thing to aid in the work of managing, understanding, or operating that code. If anything, it has only made the hard jobs harder.

It’s easy to generate code, and hard to generate good code

If you read a lot of breathless think pieces, you may have a mental image of software engineers merrily crafting prompts for ChatGPT, or using Copilot to generate reams of code, then committing whatever emerges to GitHub and walking away. That does not resemble our reality.

The right way to think about tools like Copilot is more like a really fancy autocomplete or copy-paste function, or maybe like the unholy love child of Stack Overflow search results plus Google’s “I feel lucky”. You roll the dice, every time.

These tools are at their best when there’s already a parallel in the file, and you want to just copy-paste the thing with slight modifications. Or when you’re writing tests and you have a giant block of fairly repetitive YAML, and it repeats the pattern while inserting the right column and field names, like an automatic template.

However, you cannot trust generated code. I can’t emphasize this enough. AI-generated code always looks quite plausible, but even when it kind of “works”, it’s rarely congruent with your wants and needs. It will happily generate code that doesn’t parse or compile. It will make up variables, method names, function calls; it will hallucinate fields that don’t exist. Generated code will not follow your coding practices or conventions. It is not going to refactor or come up with intelligent abstractions for you. The more important, difficult or meaningful a piece of code is, the less likely you are to generate a usable artifact using AI.

You may save time by not having to type the code in from scratch, but you will need to step through the output line by line, revising as you go, before you can commit your code, let alone ship it to production. In many cases this will take as much or more time as it would take to simply write the code—especially these days, now that autocomplete has gotten so clever and sophisticated. It can be a LOT of work to bring AI-generated code into compliance and coherence with the rest of your codebase. It isn’t always worth the effort, quite frankly.

Generating code that can compile, execute, and pass a test suite isn’t especially hard; the hard part is crafting a code base that many people, teams, and successive generations of teams can navigate, mutate, and reason about for years to come.

How working engineers really use generative AI

So that’s the TLDR: you can generate a lot of code, really fast, but you can’t trust what comes out. At all. However, there are some use cases where generative AI consistently shines.

For example, it’s often easier to ask chatGPT to generate example code using unfamiliar APIs than by reading the API docs—the corpus was trained on repositories where the APIs are being used for real life workloads, after all.

Generative AI is also pretty good at producing code that is annoying or tedious to write, yet tightly scoped and easy to explain. The more predictable a scenario is, the better these tools are at writing the code for you. If what you need is effectively copy-paste with a template—any time you could generate the code you want using sed/awk or vi macros—generative AI is quite good at this.

It’s also very good at writing little functions for you to do things in unfamiliar languages or scenarios. If you have a snippet of Python code and you want the same thing in Java, but you don’t know Java, generative AI has got your back.

Again, remember, the odds are 50/50 that the result is completely made up. You always have to assume the results are incorrect until you can verify it by hand. But these tools can absolutely accelerate your work in countless ways.

Generative AI is a little bit like a junior engineer

One of the engineers I work with, Kent Quirk, describes generative AI as “an excitable junior engineer who types really fast”. I love that quote—it leaves an indelible mental image.

Generative AI is like a junior engineer in that you can’t roll their code off into production. You are responsible for it—legally, ethically, and practically. You still have to take the time to understand it, test it, instrument it, retrofit it stylistically and thematically to fit the rest of your code base, and ensure your teammates can understand and maintain it as well.

The analogy is a decent one, actually, but only if your code is disposable and self-contained, i.e. not meant to be integrated into a larger body of work, or to survive and be read or modified by others.

And hey—there are corners of the industry like this, where most of the code is write-only, throwaway code. There are agencies that spin out dozens of disposable apps per year, each written for a particular launch or marketing event and then left to wither on the vine. But that is not most software. Disposable code is rare; code that needs to work over the long term is the norm. Even when we think a piece of code will be disposable, we are often (urf) wrong.

But generative AI is not a member of your team

In that particular sense—generating code that you know is untrustworthy—GenAI is a bit like a junior engineer. But in every other way, the analogy fails. Because adding a person who writes code to your team is nothing like autogenerating code. That code could have come from anywhere—Stack Overflow, Copilot, whatever. You don’t know, and it doesn’t really matter. There’s no feedback loop, no person on the other end trying iteratively to learn and improve, and no impact to your team vibes or culture.

To state the supremely obvious: giving code review feedback to a junior engineer is not like editing generated code. Your effort is worth more when it is invested into someone else’s apprenticeship. It’s an opportunity to pass on the lessons you’ve learned in your own career. Even just the act of framing your feedback to explain and convey your message forces you to think through the problem in a more rigorous way, and has a way of helping you understand the material more deeply.

And adding a junior engineer to your team will immediately change team dynamics. It creates an environment where asking questions is normalized and encouraged, where teaching as well as learning is a constant. We’ll talk more about team dynamics in a moment.

The time you invest into helping a junior engineer level up can pay off remarkably quickly. Time flies. ☺️ When it comes to hiring, we tend to valorize senior engineers almost as much as we underestimate junior engineers. Neither stereotype is helpful.

We underestimate the cost of hiring seniors, and overestimate the cost of hiring juniors

People seem to think that once you hire a senior engineer, you can drop them onto a team and they will be immediately productive, while hiring a junior engineer will be a tax on team performance forever. Neither are true. Honestly, most of the work that most teams have to do is not that difficult, once it’s been broken down into its constituent parts. There’s plenty of room for lower level engineers to execute and flourish.

The grossly simplified perspective of your accountant goes something like this. “Why should we pay $100k for a junior engineer to slow things down, when we could pay $200k for a senior engineer to speed things up?” It makes no sense!

But you know and I know—every engineer who is paying attention should know—that’s not how engineering works. This is an apprenticeship industry, and productivity is defined by the output and carrying capacity of each team, not each person.

There are lots of ways a person can contribute to the overall velocity of a team, just like there are lots of ways a person can sap the energy out of a team or add friction and drag to everyone around them. These do not always correlate with the person’s level (at least not in the direction people tend to assume), and writing code is only one way.

Furthermore, every engineer you hire requires ramp time and investment before they can contribute. Hiring and training new engineers is a costly endeavor, no matter what level they are. It will take any senior engineer time to build up their mental model of the system, familiarize themselves with the tools and technology, and ramp up to speed. How long? It depends on how clean and organized the codebase is, past experience with your tools and technologies, how good you are at onboarding new engineers, and more, but likely around 6-9 months. They probably won’t reach cruising altitude for about a year.

Yes, the ramp will be longer for a junior engineer, and yes, it will require more investment from the team. But it’s not indefinite. Your junior engineer should be a net positive within roughly the same time frame, six months to a year, and they develop far more rapidly than more senior contributors. (Don’t forget, their contributions may vastly exceed the code they personally write.)

You do not have to be a senior engineer to add value

In terms of writing and shipping features, some of the most productive engineers I’ve ever known have been intermediate engineers. Not yet bogged down with all the meetings and curating and mentoring and advising and architecture, their calendars not yet pockmarked with interruptions, they can just build stuff. You see them put their headphones on first thing in the morning, write code all day, and cruise out the door in the evening having made incredible progress.

Intermediate engineers sit in this lovely, temporary state where they have gotten good enough at programming to be very productive, but they are still learning how to build and care for systems. All they do is write code, reams and reams of code.

And they’re energized…engaged. They’re having fun! They aren’t bored with writing a web form or a login page for the 1000th time. Everything is new, interesting, and exciting, which typically means they will do a better job, especially under the light direction of someone more experienced. Having intermediate engineers on a team is amazing. The only way you get them is by hiring junior engineers.

Having junior and intermediate engineers on a team is a shockingly good inoculation against overengineering and premature complexity. They don’t yet know enough about a problem to imagine all the infinite edge cases that need to be solved for. They help keep things simple, which is one of the hardest things to do.

The long term arguments for hiring junior engineers

If you ask, nearly everybody will wholeheartedly agree that hiring junior engineers is a good thing…and someone else should do it. This is because the long-term arguments for hiring junior engineers are compelling and fairly well understood.

  1. We need more senior engineers as an industry
  2. Somebody has to train them
  3. Junior engineers are cheaper
  4. They may add some much-needed diversity
  5. They are often very loyal to companies who invest in training them, and will stick around for years instead of job hopping
  6. Did we already mention that somebody needs to do it?

But long-term thinking is not a thing that companies, or capitalism in general, are typically great at. Framed this way, it makes it sound like you hire junior engineers as a selfless act of public service, at great cost to yourself. Companies are much more likely to want to externalize costs like those, which is how we got to where we are now.

The short term arguments for hiring junior engineers

However, there are at least as many arguments to be made for hiring junior engineers in the short term—selfish, hard-nosed, profitable reasons for why it benefits the team and the company to do so. You just have to shift your perspective slightly, from individuals to teams, to bring them into focus.

Let’s start here: hiring engineers is not a process of “picking the best person for the job”. Hiring engineers is about composing teams. The smallest unit of software ownership is not the individual, it’s the team. Only teams can own, build, and maintain a corpus of software. It is inherently a collaborative, cooperative activity.

If hiring engineers was about picking the “best people”, it would make sense to hire the most senior, experienced individual you can get for the money you have, because we are using “senior” and “experienced” as a proxy for “productivity”. (Questionable, but let’s not nitpick.) But the productivity of each individual is not what we should be optimizing for. The productivity of the team is all that matters.

And the best teams are always the ones with a diversity of strengths, perspectives, and levels of expertise. A monoculture can be spectacularly successful in the short term—it may even outperform a diverse team. But they do not scale well, and they do not adapt to unfamiliar challenges gracefully. The longer you wait to diversify, the harder it will be.

We need to hire junior engineers, and not just once, but consistently. We need to keep feeding the funnel from the bottom up. Junior engineers only stay junior for a couple years, and intermediate engineers turn into senior engineers. Super-senior engineers are not actually the best people to mentor junior engineers; the most effective mentor is usually someone just one level ahead, who vividly remembers what it was like in your shoes.

A healthy, high-performing team has a range of levels

A healthy team is an ecosystem. You wouldn’t staff a product engineering team with six DB experts and one mobile developer. Nor should you staff it with six staff+ engineers and one junior developer. A good team is composed of a range of skills and levels.

Have you ever been on a team packed exclusively with staff or principal engineers? It is not fun. That is not a high-functioning team. There is only so much high-level architecture and planning work to go around, there are only so many big decisions that need to be made. These engineers spend most of their time doing work that feels boring and repetitive, so they tend to over-engineer solutions and/or cut corners—sometimes at the same time. They compete for the “fun” stuff and find reasons to pick technical fights with each other. They chronically under-document and under-invest in the work that makes systems simple and tractable.

Teams that only have intermediate engineers (or beginners, or seniors, or whatever) will have different pathologies, but similar problems with contention and blind spots. The work itself has a wide range in complexity and difficulty—from simple, tightly scoped functions to tough, high-stakes architecture decisions. It makes sense for the people doing the work to occupy a similar range.

The best teams are ones where no one is bored, because every single person is working on something that challenges them and pushes their boundaries. The only way you can get this is by having a range of skill levels on the team.

The bottleneck we face is hiring, not training

The bottleneck we face now is not our ability to train up new junior engineers and give them skills. Nor is it about juniors learning to hustle harder; I see a lot of solidwell-meaning advice on this topic, but it’s not going to solve the problem. The bottleneck is giving them their first jobs. The bottleneck consists of companies who see them as a cost to externalize, not an investment in their—the company’s—future.

After their first job, an engineer can usually find work. But getting that first job, from what I can see, is murder. It is all but impossible—if you didn’t graduate from a top college, and you aren’t entering the feeder system of Big Tech, then it’s a roll of the dice, a question of luck or who has the best connections. It was rough before the chimera of “Generative AI can replace junior engineers” rose up from the swamp. And now…oof.

Where would you be, if you hadn’t gotten into tech when you did?

I know where I would be, and it is not here.

The internet loves to make fun of Boomers, the generation that famously coasted to college, home ownership, and retirement, then pulled the ladder up after them while mocking younger people as snowflakes. “Ok, Boomer” may be here to stay, but can we try to keep “Ok, Staff Engineer” from becoming a thing?

Nobody thinks we need fewer senior engineers

Lots of people seem to think we don’t need junior engineers, but nobody is arguing that we need fewer senior engineers, or will need fewer senior engineers in the foreseeable future.

I think it’s safe to assume that anything deterministic and automatable will eventually be automated. Software engineering is no different—we are ground zero! Of course we’re always looking for ways to automate and improve efficiency, as we should be.

But large software systems are unpredictable and nondeterministic, with emergent behaviors. The mere existence of users injects chaos into the system. Components can be automated, but complexity can only be managed.

Even if systems could be fully automated and managed by AI, the fact that we cannot understand how AI makes decisions is a huge, possibly insurmountable problem. Running your business on a system that humans can’t debug or understand seems like a risk so existential that no security, legal or finance team would ever sign off on it. Maybe some version of this future will come to pass, but it’s hard to see it from here. I would not bet my career or my company on it happening.

In the meantime, we still need more senior engineers. The only way to grow them is by fixing the funnel.

Should every company hire junior engineers?

No. You need to be able to set them up for success. Some factors that disqualify you from hiring junior engineers:

  • You have less than two years of runway
  • Your team is constantly in firefighting mode, or you have no slack in your system
  • You have no experienced managers, or you have bad managers, or no managers at all
  • You have no product roadmap
  • Nobody on your team has any interest in being their mentor or point person

The only thing worse than never hiring any junior engineers is hiring them into an awful experience where they can’t learn anything. (I wouldn’t set the bar quite as high as Cindy does in this article; while I understand where she’s coming from, it is so much easier to land your second job than your first job that I think most junior engineers would frankly choose a crappy first job over none at all.)

Being a fully distributed company isn’t a complete dealbreaker, but it does make things even harder. I would counsel junior engineers to seek out office jobs if at all possible. You learn so much faster when you can soak up casual conversations and technical chatter, and you lose that working from home. If you are a remote employer, know that you will need to work harder to compensate for this. I suggest connecting with others who have done this successfully (they exist!) for advice.

I also advise companies not to start by hiring a single junior engineer. If you’re going to hire one, hire two or three. Give them a cohort of peers, so it’s a little less intimidating and isolating.

Nobody is coming to fix our problems for us

I have come to believe that the only way this will ever change is if engineers and engineering managers across our industry take up this fight and make it personal.

Most of the places I know that do have a program for hiring and training entry level engineers, have it only because an engineer decided to fight for it. Engineers—sometimes engineering managers—were the ones who made the case and pushed for resources, then designed the program, interviewed and hired the junior engineers, and set them up with mentors. This is not an exotic project, it is well within the capabilities of most motivated, experienced engineers (and good for your career as well).

Finance isn’t going to lobby for this. Execs aren’t likely to step in. The more a person’s role inclines them to treat engineers like fungible resources, the less likely they are to understand why this matters.

AI is not coming to solve all our problems and write all our code for us—and even if it wasit wouldn’t matter. Writing code is but a sliver of what professional software engineers do, and arguably the easiest part. Only we have the context and the credibility to drive the changes we know form the bedrock for great teams and engineering excellence..

Great teams are how great engineers get made. Nobody knows this better than engineers and EMs. It’s time for us to make the case, and make it happen.

Generative AI is not going to build your engineering team for you

How can you tell if the company you’re interviewing with is rotten on the inside?

How can you tell the companies who are earnestly trying to improve apart from the ones who sound all polished and healthy from the outside, whilst rotting on the inside?

This seems to be on a lot of minds right now, what with the Great Resignation and all. There are no perfect companies, just like there are no perfect relationships; but there are many questions and techniques you can use to increase your confidence that a particular company is decent and self-aware, one whose quirks and foibles you are compatible with.

https://twitter.com/jetpack/status/1486212123525287938

Interviews are designed to make you feel like you are under inspection, like the interviewer holds all the power. This is an illusion. Your labor is valuable — it is vital — and you should be scrutinizing them every bit as closely as they you. In fact, here is Tip #1:

  • If they allow you plenty of time to converse with your interviewers throughout the process, great. If they tack on a cursory “any questions for us?” while wrapping up, they don’t think it matters what you think of them. Pull the ripcord.

Collect and practice good interview questions for you to ask potential employers. Write them down — your mind is likely to go blank under stress, and you don’t want to let them off the hook. There is a LOT of signal to be gained by probing down below the surface answers.

Backchanneling

  1. Whisper networks and backchannels are incredibly important. It can be especially valuable to talk to someone who has recently left the company: why did they leave? Would they go there again?
  2. Alternately, do you know anyone who has worked for or with their leadership, even if not at that company?
  3. If you know any women or under-represented minorities (URM) who work there, buy them lunch and ask for the unvarnished truth. That’s where you usually turn up the real dirt. 🥂

Diversity, equity and inclusion

Just because a company has a diverse workforce doesn’t necessarily mean it is a healthy place to work. (But it’s fair to give some points up front, because that doesn’t usually happen by accident.)

  1. Do they have a diverse leadership team? A diverse board?
  2. Is their company diverse overall, or are minorities concentrated in a few (lower-paying, high-turnover) departments?
  3. You might not want to write off all the companies that don’t meet points one and two, if for no other reason than it dramatically shrinks your available option pool. If they don’t have a particularly diverse team, and this is something that matters to you, that’s your cue to dig deeper:
    • Are they bothered by their lack of diversity? What’s the plan? Do they just feel generically sad about it, or have they set specific goals to improve by specific dates? What investments are they making?
    • Who works on DEI stuff currently? (Answers like “HR and recruiting”, or “we have a woman who’s really good at it” are bad answers.)
    • Who is accountable for making sure those goals are hit? (The only right answer is “our execs”. Having a “chief diversity officer” is an anti-pattern in my book.)
    • If the team is all guys, for example, ask if they’ve ever had any women on the team in the past. Did she/they leave? Do they know why?
  4. This is a GREAT one: “As a white man, I’d ask what they’ve done to find qualified women and minorities for the role I’m interviewing for.” (via David Daly) 🔥🔥

Company stuff

  1. What are their values? Do they feel bloodless and ripped from the pages of HBR, or are they unique, lived-in, and give you a glimpse of what the people there care about? Are they mentioned over the course of your interview?
  2. Ask tough questions about the business and try to ascertain whether they are hitting their quarterly goals, how much funding they have in the bank, what the growth curve looks like, what users really think about their product, and what the biggest obstacles to success are.
    • Companies that are floundering are going to be really stressful places to work, and even if the leadership is decent, they may find themselves backed into making some really tough decisions.
    • You want to work at a company on a strong growth trajectory for lots of reasons, but a big one is your own growth potential. You will learn the most the fastest at places that are growing fast, and have way more openings for promotions and leadership roles than a slower-growing company.
  3. Are people willing to speak freely about things they’ve tried that have failed, and things that don’t work well currently? Being self-aware and comfortable with visible failure are two of the most important self-correcting mechanisms a company can cultivate.
  4. EVERYBODY thinks they value transparency, so I wouldn’t even bother asking. Instead, ask for specific examples of leadership being forthcoming with bad news to the team, and team members delivering hard feedback or bad news to upper leadership. Transparency shouldn’t be something they’re especially proud of, so much as it is taken for granted. It’s in the air that you breathe.

Planning and the unplanned

  1. Ask about how decisions get made. A chestnut is, “how does work end up on my plate?” — meaning is there a business strategy (owned by whom?), a technical strategy, a product strategy, quarterly KPIs, customer requests, manager delegation, JIRA tickets…? (The most important part may be how similar the answers you get are. 🙃)
  2. How often does work get pre-empted and why? It’s a good thing if product development has to get put on ice once in a while so the team can focus on reliability and maintenance work. It’s a bad thing if they’re expected to stuff reliability work in the cracks around their product development, or if they’re incapable of sticking to a plan.
  3. What does “crunch time” look like? Nearly every company has one from time to time (it might even be a bad sign if it never happens), but this is when you find out your leadership’s true colors.
    • Do they praise people or call them out to thank them for pulling all nighters and other extremist behaviors? 🚨BZZT🚨
    • Is it voluntary? Are you trusted to set your own pace, your own limits, or are you pressured to do more? Are people expected to participate to the extent that they are able, and not expected to justify how hard or how much (so long as they communicate their capacity, of course)?
    • How long did it last, and how often does it happen, and why? It should be rare (1-2x/year at most), involve the whole company, and move the business forward meaningfully
    • Did they follow through by making sure people took time off afterwards to recover? Not just give permission, but actually make sure the human beings had a chance to refresh themselves? Did leaders set a good example by taking a breather themselves?

Believe it or not, crunch time done correctly can be an enormously exciting, intense, bonding time for a group of people who love what they do, culminating in a surge of collective triumph and celebration, followed by recovery time. If it was done correctly, and you ask about it afterwards, people will 💡light up💡.

Team stuff

  1. Unfortunately, culture can vary widely from function to function, even from manager to manager. Make sure you get a real interview slot with your actual manager — not just a screener or wrap-up call — and as much of the team as possible, too.
  2. Ask your potential manager about the last person they had to let go. Why? What was the process? What was the impact on the team? How did the person feel afterwards?
  3. Who is on call? How often do people get paged outside of hours, and how frequently do they work an incident? (Do managers track this?) Are you expected to keep shipping product during on call weeks, or devote your time to making the system better?
  4. If you had to ship a single line of code to production using the deploy pipeline, how long would that take? Remember, the lower the deploy interval, the happier and more productive you are likely to be as an engineer there. Under 15 minutes is great. Under an hour is tolerable. More than that, proceed with great caution.

The interview itself

  1. Was your interview well-organized and conducted in a timely fashion? Were you given detailed information about what to expect, and were your interviewers well-prepared, and conversational? Were the questions fair, open-ended, and relevant to the job in question?
  2. If they asked you to perform any kind of take-home labor of more than an hour or so, did they compensate you for your time?
  3. Did they get back to you swiftly at each step of the way to let you know where you stand and what comes next?
  4. Did you find the questions interesting and challenging? Do they have a clear idea of what success looks like for you in this role? Did you leave excited and buzzing with ideas about the work you could do together?
    • This is 👆 definitely more of a “how good is this job” question than “is this a shithole” question, but one of our honeycombers brought it up as an example of how a great interview can make you decide to leave a job , even one you’re perfectly happy with.
  5. The questions they ask you while interviewing you are the questions they ask everyone else. So…did they ask you about your views on diversity and team dynamics while interviewing you? Or is that not part of their filters, only their advertised persona?

Three more

  1. Do their employees seem to speak freely on twitter? If you are an agitator of sorts, are there others who agitate about similar issues — either with company support, or at least lack of censorship?
  2. How does the company respond to criticism and feedback? For that matter, how do they treat their competitors? Being competitive is fine, being mean is not.
  3. Get clear on your own expectations. What’s on your wishlist, and what’s make-or-break for you? If something is very important to you, consider telling the hiring manager up front. For example, “These are my expectations for how women are treated. How do you think your company matches up?” Their answers will speak volumes, and so will their comfort level with the question.

In closing

If you a join a new company, and two or three weeks in you’re just not feeling it, you’re wondering if you made a mistake — leave. You do not owe them a year of your life. Trust your instincts. Just leave it off your resume entirely and roll the dice again.

Employers are all too accustomed to feeling (and acting) like they hold all the power. They do not. Every tech company is a talent business, which rises and falls on the caliber of the people they can convince to stay. They aren’t doing you a favor by employing you; you are doing THEM a favor by lending them your creativity, labor, and a third of the hours in your day.

Do they deserve it? Will their success make the world a better place? If not, stop supporting them with your work and lend your muscle to a company that deserves it.

In the hottest job market of my lifetime, with millions of opportunities newly open to people who live literally anywhere, you owe it to yourself, your future self, and your family to take a good hard look at where you sit. 🍄 Are you happy? 🍄 Are you compensated well, is your time valued? 🍄 Are you still learning new things and improving your skills every day? 🍄 Is your company still on a growth trajectory? 🍄 Do you trust your leadership and your team, 🍄 do still you believe in the mission, and 🍄 do you think your labor contributes meaningfully to making the world a better place?

If not, consider joining the Great Resignation. I hear they have cookies.

Huge thanks to Amy Davis, Phillip Carter, Ian Smith, Sarah Voegeli, Kent Quirk, Liz Fong-Jones, Amanda Shapiro, Nick Rycar, Fred Hebert and David Daly, all of whom contributed to this post!

How can you tell if the company you’re interviewing with is rotten on the inside?

How “Engineering-Driven” Leads to “Engineering-Supremacy”

Honeycomb has a reputation for being a very engineering-driven company. No surprise there, since it was founded by two engineers and our mission involves building an engineering product for other engineers.

We are never going to stop being engineering-driven in the sense that we are building for engineers and we always want engineers to have a seat at the table when it comes to what we build, and how, and why. But I am increasingly uncomfortable with the term “engineering-driven” and the asymmetry it implies.

We are less and less engineering-driven nowadays, for entirely good reasons — we want this to be just as much of a design-driven company and a product-driven company, and I would never want sales or marketing to feel like anything other than equal partners in our journey towards revolutionizing the way the world builds and runs code in production.

It is true that most honeycomb employees were engineers for the first few years, and our culture felt very engineer-centric. Other orgs were maybe comprised of a person or two, or had engineers trying to play them on TV, or just felt highly experimental.

But if there is one thing Christine and I were crystal clear on from the beginning, it’s this:

✨WE ARE HERE TO BUILD A BUSINESS✨

Not just the shiniest, most hardcore tech, not just the happiest, most diverse teams. These things matter to us — they matter a lot! But succeeding at business is what gives us the power to change the world in all of these other ways we care about changing it.

If business is booming and people are thirsty for more of whatever it is you’re serving, you pretty much get a blank check for radical experiments in sociotechnical transformation, be that libertarian or communitarian or anything in between.

If you don’t have the business to back it up, you get fuck-all.

Not only do you not get shit, you risk being pointed out as a Cautionary Tale of “what happens if $(thing you deeply care about) comes true.” Sit on THAT for a hot second. 😕

So yes, we cared. Which is not to say we knew how to build a great business. We most certainly did not. But both of us had been through too many startups (Linden Lab, Aardvark, Parse, etc) where the tech was amazing, the people were amazing, the product was amazing … and the business side just did not keep up. Which always leads to the same thing: heartbreak and devastation.

✨IF you can’t make it profitable✨
✨your destiny will inevitably be✨
✨taken out of your hands✨
✨and given to someone else.✨

So Christine and I have been repeating these twin facts back and forth to each other for over six years now:

  1. Honeycomb must succeed as a revenue-generating, eventually profitable business.
  2. We are not business experts. Therefore we have to make Honeycomb a place that explicitly values business expertise, that places it on the same level as engineering expertise.

We have worked hard to get better at understanding the business side (her more than me 🙃) but ultimately, we cannot be the domain experts in marketing or sales (or customer success, support, etc).

What we could do is demonstrate respect for those functions, bake that respect into our culture, and hire and support amazing business talent to run them with us.

On being “engineering-driven”

Self-described “engineering-driven” companies tend to fall into one of two traps. Either they alienate the business side by pinching their nose and holding business development at arm’s length (“aaahhh, i’m just an engineer! I have no interest in or capacity for participating in developing our marketing voice or sales pitch decks, get off me!”), or they act like engineering is a sort of super-skillset that makes you capable of doing everybody else’s job better than they possibly could. As though those other disciplines and skill sets aren’t every bit as deep and creative and challenging in their own right as developing software can be.

For the first few years we really did use engineers for all of those functions. We were trying to figure out how to build and explain and sell something new, which meant working out these things on the ground every day with our users. Engineer to engineer. What resonated? What clicked? What worked?

So we hired just a few engineers who were interested in how the business worked, and who were willing to work like Swiss Army knives across the org. We didn’t yet have a workable plan in place, which is what you need in order to bring domain experts on board, point them in a direction, and trust them to do what they do best in executing that plan.

Like I said, we didn’t know what to do or how to do it. But at least we knew that. Which kept us humble. And translated into a hard, fast rule which we set early on in our hiring process:

We DO NOT hire engineers who talk shit about sales and marketing.

If I was interviewing an engineer and they made any alienating sort of comment whatsoever about their counterparts on the business side, it was an automatic no. Easy out. We had a zero tolerance policy for talking down down about other functions, or joking, even for being unwilling to perform other business functions.

In retrospect, I think this is one of the best decisions we ever made.

Hiring engineers who respected other functions

We leaned hard into hiring engineers who asked curious questions about our business strategy and execution. We pursued engineers who talked about wanting to spend time directly with users, who were intrigued by the idea of writing marketing copy to help explain concepts to engineers, and who were ready, willing and able to go along on sales calls.

Once we finally found product-market fit, about 2-3 years ago, we stopped using engineers to play other roles and started hiring actual professionals in product, design, marketing, sales, customer success, etc to build and staff out their organizations. That was when we first started building the business for the longer term; until we found PMF, our event horizon was never more than 1-3 months ahead of “right now”.

(I’ll never forget going out to coffee with one of our earlier VPs of marketing, shortly after she was hired, and having her ask, in bemusement: “Why are all these engineers just sitting around in the #marketing channel? I’ve never had so many people giving opinions on my work!” 🙃)

Those early Swiss Army knife engineers have since stepped back, gratefully, into roles more centered around engineering. But that early knee-jerk reaction of ours established an important company norm that pays dividends to this day.

Every function of the business is equally challenging, creative, and worthy of respect. None of us are here to peacock; our skill sets serve the primary business goals. We all do our jobs better when we know more about how each other’s functions work.

These days, it’s not just about making sure we hire engineers who treat business counterparts like equals. It’s more about finding ways to stimulate the flow of information cross-functionally, creating a hunger for this information.

Caring about the big picture is a ✨learnable skill✨

You can try to hire people who care about the overall business outcomes, not just their own corner of reality, and we do select for this to  some degree — for all roles, not just engineers.

But you can also foster this curiosity and teach people to seek it out. Curiosity begets curiosity, and every single person at Honeycomb is doing something interesting. We all want to succeed and win together, and there’s something infectious and exciting about connecting all the dots that lead to success and reflecting that story back to the rest of the company.

For example,

    1. Every time we close a deal, a post gets written up and dropped into the announcements slack by the sales team. Not just who did we close and how much money did we make, but the full story of that customer’s interaction with honeycomb. How did they hear of us? Whose blog posts, training sessions, or office hours did they engage with? Did someone on the telemetry pull a record-fast turnaround on an integration they needed to get going? What pains did we solve effectively for them as a tool, and where were the rough edges that we can improve on in the future?

      The story is often half a page long or more, and tags a dozen or more people throughout all parts of the company, showing how everyone’s hard work added up and materially contributed to the final result.
    2. We have orgs take turn presenting in all hands — where they’re at, what they’ve built, and the impact of their contributions, week after week. Whether that’s the design team talking about how they’ve rolled out our new design system and how it is going to help everyone in the company experiment more and move more quickly, or it’s the people team showing how they’ve improved our recruiting, interviewing, and hiring processes to make people feel more seen, welcomed, and appreciated throughout the process.

      We expect people to be curious about the rest of the company. We expect honeybees to be interested in, excited about and celebratory of each other’s hard work. And it’s easy to be excited when you see people showing off work that they were excited to do.
    3. We have a weekly Friday “demo day” where people come and show off something they’ve built this week, rapid-fire. Whether it’s connecting to a mysql shell in the terminal to show off our newly consistent permissioning scheme, or product marketing showing off new work on the website. Everybody’s work counts. Everybody wants to see it.
    4. We have a #love channel in slack where you can drop in and tag someone when you’re feeling thankful for how much they just made your day better. We also have a “Gratefuls” section during all hands, where people speak up and give verbal props to coworkers who have really made a difference in their lives at work.

We have always attracted engineers who care about the business, not just the technology and the culture. As a result, we have consistently recruited and retained business leaders who are well above our weight class — our investors still sometimes marvel at the caliber of the business talent we have been able to attract. It is way above the norm for developer tools companies like ours.

“Engineering-driven” can be a mask for “engineering-supremacy”

Because the sad truth is that so many companies who pride themselves on being “engineering-driven” are actually what I would call more “engineering-supremacist”. Ask any top-tier sales or marketing leader out there about their experiences in the tech industry and you’ll hear a painful, rage-inducing list of times they were talked down to by technical founders, had their counsel blown off or overridden, had their plans scrapped and their budgets cut, and every other sort of disrespectful act you can imagine.

(I am aware that the opposite also exists; that there are companies and cultures out there that valorize and glorify sales or marketing while treating engineers like code monkeys and button pushers, but it’s less common around here. In neither direction is this okay.)

This isn’t good for business, and it isn’t good for people.

It is still true that engineering is the most mature and developed organization at the company, because it has been around the longest. But our other orgs are starting to catch up and figure out what it means to “be honeycomby” for them, on their own terms. How do our core values apply to the sales team, the developer evangelists, the marketing folks, the product people? We are starting to see this play out in real time, and it’s fascinating. It is better than forcing all teams to be “engineering-driven”.

Business success is what makes all things possible

We are known for the caliber of our engineering today. But none of that matters a whit if you never hear about us, or can’t buy us in a way that makes sense for you and your team, or if you can’t use the product, or if we don’t keep building the right things, the things you need to modernize your engineering teams and move into the future together.

When looking towards that future, I still want us to be known for our great engineering. But I also want us to be a magnet for great designers who trust that they can come here and be respected, for great product people who know they can come here and do the best work of their life. That won’t happen if we see ourselves as being “driven” by one third of the triad.

Supremacy destroys balance. Always.

And none of this, none of this works unless we have a surging, thriving business to keep the wind in our sails.

~charity

How “Engineering-Driven” Leads to “Engineering-Supremacy”

Why every software engineering interview should include ops questions

I’ve fallen way behind on my blog posts — my goal was to write one per month, and I haven’t published anything since MAY. Egads. So here I am dipping into the drafts archives! This one was written in April of 2016, when I was noodling over my CraftConf 2016 talk on “DevOps for Developers (see slides).”

So I got to the part in my talk where I’m talking about how to interview and hire software engineers who aren’t going to burn the fucking house down, and realized I could spend a solid hour on that question alone. That’s why I decided to turn it into a blog post instead.

Stop telling ops people to code better, start telling SWEs to ops better

Our industry has gotten very good at pressing operations engineers to get better at writing code, writing tests, and software engineering in general these past few years. Which is great! But we have not been nearly so good at pushing software engineers to level up their systems skills. Which is unfortunate, because it is just as important.

Most systems suffer from the syndrome of running too much software. Tossing more software into the heap is as likely to cause more problems as often as it solves them.

We see this play out at companies stacked with good software engineers who have built horrifying spaghetti messes of their infrastructure, and then commence paging themselves to death.

The only way to unwind this is to reset expectations, and make it clear that

  1. you are still responsible for your code after it’s been deployed to production, and 
  2. operational excellence is everyone’s job.

Operations is the constellation of tools, practices, policies, habits, and docs around shipping value to users, and every single one of us needs to participate in order to do this swiftly and safely.

Every software engineering interviewing loop should have an ops component.

Nobody interviews candidates for SRE or ops nowadays without asking some coding questions. You don’t have to be the greatest programmer in the world, but you can’t be functionally illiterate. The reverse is less common: asking software engineers basic, stupid questions about the lifecycle of their code, instrumentation best practices, etc. 

It’s common practice at lots of companies now to have a software engineer in the loop for hiring SREs to evaluate their coding abilities. It should be just as common to have an ops engineer in the loop for a SWE hire, especially for any SWE who is being considered for a key senior position. Those are the people you most rely on to be mentors and role models for junior hires. All engineers should embrace the ethos of owning their code in production, and nobody should be promoted or hired into a senior role if they don’t.

And yes, that means all engineers!  Even your iOS/Android engineers and website developers should be interested in what happens to their code after they hit deploy.  They should care about things like instrumentation, and what kind of data they may need later to debug their problems, and how their features may impact other infrastructure components.

You need to balance out your software engineers with engineers who don’t react to every problem by writing more code. You need engineers who write code begrudgingly, as a last resort. You’ll find these priceless gems in ops and SRE.

ops questions for software engineers

The best questions are broad and start off easy, with plenty of reasonable answers and pathways to explore. Even beginners can give a reasonable answer, while experts can go on talking for hours.

For example: give them the specs for a new feature, and ask them to talk through the infrastructure choices and dependencies to support that feature. Do they ask about things like which languages, databases, and frameworks are already supported by the team? Do they understand what kind of monitoring and observability tools to use, do they ask about local instrumentation best practices?

Or design a full deployment pipeline together. Probe what they know about generating artifacts, versioning, rollbacks, branching vs master, canarying, rolling restarts, green/blue deploys, etc. How might they design a deploy tool? Talk through the tradeoffs.

Some other good starting points:

  • “Tell me about the last time you caused a production outage. What happened, how did you find out, how was it resolved, and what did you learn?”
  • “What are some of your favorite tools for visibility, instrumentation, and debugging?
  • “Latency seems to have doubled over the last 6 hours. Where do you start looking, how do you start debugging?”
  • And this chestnut: “What happens when you type ‘google.com’ into a web browser?” You would be fucking *astonished* how many senior software engineers don’t know a thing about DNS, HTTP, SSL/TLS, cookies, TCP/IP, routing, load balancers, web servers, proxies, and on and on.

Another question I really like is: “what’s your favorite API (or database, or language) and why?” followed up by “… and what are the worst things about it?” (True love doesn’t mean blind worship.)

Remember, you’re exploring someone’s experience and depth here, not giving them a pass-fail quiz. It’s okay if they don’t know it all. You’re also evaluating them on communication skills, which is severely underrated by most people but is actually as a key technical skill.

Signals to look for

You’re not looking for perfection. You are teasing out signals for things like, how will this person perform on a team where software engineers are expected to own their code? How much do they know about the world outside the code they write themselves? Are they curious, eager, and willing to learn, or fearful, incurious and begrudging?

Do they expect networks to be reliable? Do they expect databases to respond, retries to succeed? Are they offended by the idea of being on call? Are they overly clever or do they look to simplify? (God, I hate clever software engineers 🙃.)

It’s valuable to get a feel for an engineer’s operational chops, but let’s be clear, you’re doing this for one big reason: to set expectations. By making ops questions part of the interview, you’re establishing from the start that you run an org where operations is valued, where ownership is non-optional. This is not an ivory tower where software engineers can merrily git push and go home for the day and let other people handle the fallout

It can be toxic when you have an engineer who thinks all ops work is toil and operations engineering is lesser-than. It tends to result in operations work being done very poorly. This is your best chance to let those people self-select out.

You know what, I’m actually feeling uncharacteristically optimistic right now. I’m remembering how controversial some of this stuff was when I first wrote it, five years ago in 2016. Nowadays it just sounds obvious. Like table stakes.

Hell yeah. 🤘

Why every software engineering interview should include ops questions

Questionable Advice: “What Should I Say In My Exit Interview?”

I recently received this gem of a note::

Hi Charity, I really enjoy your writing and a lot of it has directly contributed to me finally deciding to leave a company with a toxic management culture. I’ll also be leaving many great IC friends that will have lost a strong voice.

My exit interview will be next week. Any advice on how honest I should be?

I’ve googled quite a bit but there are only generic “don’t burn bridges” comments. Would love to see something a little more authoritative 🙂

–Anonymous Reader

Ew, fuck that. That’s exactly the kind of quivering, self-serving, ass covering advice you’d get from HR. It’s exactly the kind of advice that good people use to perpetuate harm.

I wouldn’t worry about “too much honesty” or personal repercussions or whatnot. I would worry about just one thing: being effective. This is your last chance to do the people you care about a solid, and you don’t want to waste it.

So … ranting about every awful person, boring project and offensive party theme of your tenure: not effective. Ranting about people who were personally irritating but had very limited power: not effective. Talking only in vague, high level abstractions (“toxic culture”), or about things only engineers understand and are bugged by: not effective.

What is effective? Hm, let’s think on this..

  • Start off with your high level assertion (toxic culture) and methodically assemble a list of stories, incidents, and consequences that support your thesis. Structure-wise, this is a lot like writing a good essay
  • Tie your critiques to the higher ups who enabled or encouraged the bad behavior, not just the flunkies who carried it out.
  • Wherever possible, draw a straight line to material consequences — people quitting, customers leaving, your company’s reputation suffering
  • Keep it crisp. No more than three pages total. Pick your top 1-3 points and drive them home. No detours.
  • This one sucks, but … if someone was perceived as an underperformer or a problem employee, avoid using them as evidence in support of your argument. It won’t help you or them, it will be used as an excuse to discredit you.
  • Keep it mostly professional. I am not saying don’t show any anger or strong emotion; it can be a powerful tool; just be careful with it. Get a proofread from someone with upper management experience, ideally with no connection to your work. (Me, if necessary.)
  • Put it in WRITING!✨ Deliver your feedback in person, but hand over a written copy as well. Written words are harder to ignore or distort.
  • For extra oomph, give a copy to any execs, managers, or high level ICs you trust. Don’t just email it to them, though. Have a face to face conversation where you state your case, and hand them a written copy at the end.

The sad fact is that most exit feedback is dutifully entered in by a low ranking employee who makes a third your salary and has no reason whatsoever to rock the boat, after which it gets tossed in a folder or the trash and is never seen again.

If you want to use your voice on your way out the door, the challenge you face isn’t one of retribution, it’s inertia and apathy. HR doesn’t care about your feedback … but they care if they think their boss saw it and cares about it

And I think you should use your voice! You clearly have some clout, and what’s the point of having power if you won’t extend yourself now and then on behalf of those who don’t?

Good luck!!

charity

Questionable Advice: “What Should I Say In My Exit Interview?”

Questionable Advice: “How can I sniff out bad managers while interviewing for a job?”

A few weeks ago I got a question from Stephane Bjorne on twitter, about how to screen for bad managers and/or management culture.

This is a great question. I’ve talked a lot about my philosophy for interviews, which boils down to equalizing the power dynamics as much as possible to reflect the reality that the candidate should be interviewing the company just as much as the reverse. You are all highly compensated subject matter experts who can find jobs relatively easily; there’s no reason all the judgment and critique should flow in a single direction.

But HOW? What questions can you ask, to get a feel for whether you will join this team and belatedly discover that you’re expected to be a jira ticket monkey, or that the manager is passive aggressive and won’t advocate for you or take a stand on anything?

Glad you asked.

First of all, backchannel like mad if you can. If you can’t, do ask the same question of multiple people and compare their answers. Pay particular attention to the different answers given by junior vs senior members of the team, and give extra weight to the experiences of any underrepresented minorities

Questions to consider asking the manager:

  • “How did you become a manager? Do you enjoy it?” Trust me, you never want to work for someone who was pushed into management against their will and still seems openly bitter about it.
  • “What do you like about your job?” Red flags include, “I was tired of being out of the loop and left out of decision-making processes.” That could be you next.
  • Is management a promotion here, or a lateral move? Do people ever go back from managing to engineering? Is that considered a viable career path?”
  • “What kind of training do you offer new managers, and what are they evaluated on? What are YOU evaluated on?”
  • “Do you have a job ladder for individual contributors (ICs)? Can I see it? Do the IC levels track management levels all the way to the top, or top out at some point?”
  • “What does your review and promotion process look like?”
  • “Are your levels public or private? What is the distribution of engineers across levels, roughly?” Here you are looking for how high the IC track goes, and how many engineers exist at the upper bound. Distribution should be scant at the upper levels, roughly an order of magnitude less for each level after “senior engineer”. Do the written level descriptions seem reasonable and appealing to you? Do you see yourself in them, and feel like there is a path for growth?
  • Do you have any junior or intermediate engineers, and how many? If not, why the fuck not?” Ask.
  • “How often do you have 1x1s with each of your direct reports? How often are the skip levels?”

And then there is the ur-question, which every one of you should ask in every single interview you ever have:

What is the process by which someone ends up working on a particular project?” In other words, how does work get decided and allocated? Bad signs: they get flustered, don’t have a clear answer, you have no say, there is no product/design process, it’s all done via jira assignments.

Pay just as much attention to their body language and signals here as the answer they give you. Are they telling the truth, or describing their ideal world? Ask what happens when there are problems with the  normal process, or how often it gets circumvented, or how you know if your work aligns with company strategy.

Questions to ask an engineer:

  • “What do you talk about in your 1x1s with your manager?”
  • “How often does your manager have career conversations with you, asking how you want to develop over the next few years?” Ideally at least a couple times a year, but really any answer is fine other than a blank stare.
  • “Does your manager care about you? Has working here moved your career forward? How so?”
  • “Have you ever been surprised by feedback you received in a review from your manager?” You should never be surprised.

Most of all, can you get the manager to tell you “no” on a thing you clearly want during the interview? (Maybe a level, or WFH schedule, or travel, etc.) Or are they squirmy, evasive, or hedging, or make you promises that they’ll look into it later, etc? Anyone who will look you straight in the eye and say “no, and here’s why”, is someone you can probably trust to be straight with you down the line, in good times and bad.

Depending on your level and career goals, it’s a good idea to ask questions to ascertain if the right gap exists for you to fill to reach your goals. Don’t join a team where five other people have the same exact skill sets as you do and are already eyeing the same role you want. (I wrote more on levels here.)

We also got some great contributions from @GergelyOrosz and @JillWohlner:

  • “Ask about specific situations, any half decent manager can BS on hypothetical stuff.”
  • “Who was the last person you promoted? Why/how?”
  • “How do you handle when X complains to you about Y? Can you give an example?”
  • “What was the best team you managed and why?”
  • “What is the biggest challenge the team currently has and why? What are you doing about it? Biggest strength?”
  • “How many people have left the team since you’ve been here?”
  • “Who was the last person to leave your team? Why did they leave?” (These leaving questions are tough to ask but will give you a lot of signal. Especially seeing how the manager frames it — are they playing victim, or owning up to it?)

Anyone who won’t be honest about their real personal weaknesses and struggles, probably isn’t someone you want to report to.

Jill says that she wants to hear from ICs:

  • when their manager has gone to bat for them
  • when their manager gave them hard to hear feedback (if it’s never, it’s a flag — all positive feedback = little growth)
  • when their manager coached them through a tricky situation

and from Managers:

  • when they went to bat for someone on their team
  • a team member’s growth they feel really proud of
  • when they got negative feedback about someone on their team and how it was handled

Obviously, you won’t be able to ask all of these questions in an hourlong slot. So ask a few, and lean in whenever they seem avoidant or uncomfortable — that’s where the juicy stuff comes from. And personally, I wouldn’t consider working somewhere that sees management as a promotion rather than a peer/support role, but that may be a high bar to clear in some parts of the world.

Good luck. Joining a team with a good manager can be one of the best ways to accelerate your career and open the door to opportunities, in part because it ensures healthy team dynamics. Workplace friction, bullying, harassment, passive aggressiveness, etc can be truly terrible, and even in the milder cases it’s still a huge distraction from  your work and a big quality of life issue.

A manager’s One Job is to make sure that shit doesn’t happen. It’s really is worth trying to find a good one you can trust.

Questionable Advice: “How can I sniff out bad managers while interviewing for a job?”