Mar 27, 2023
The first big project of my technology career was a company-wide rollout of Windows 95. Our tiny five person IT organization (which included desktop support, network admin, and software development) took on a herculean task of adding RAM and installing the new operating system on over 500 employee workstations. After each work day, we stayed around until midnight to process 50 or so computers and then we got to work early the next day to train and support the employees whose computers we upgraded. While we were all exhausted from the previous night, morning support was the most critical part of the project because there were problems on some machines and, even when there weren't, users needed help finding their way around the new OS. Those first few minutes determined whether they loved the new experience or felt screwed by IT.
That's when I learned one of the most important lessons of my career: follow your shot. I usually don't love using sports analogies for work but I think most people have seen (or maybe even been) a tenacious hockey/basketball/soccer/lacrosse... player who scores off their own rebound.
Following your shot means caring about the result more than just completing the task. It means, monitoring a new feature's utilization and fault rates, testing it the field, talking to customers... and making adjustments to redirect a miss. Teams that lack a shot following mentality have less impact because they don't notice missed steps or miscalculations. They just move onto the next task or project. Every team I have joined eventually notices a code branch that was never merged, a feature that was never fully dialed up, or a bug that made a feature inaccessible. Task-oriented teams also have a higher risk of burnout because checking off a task or completing a project is a relief, not a reward -- not like the satisfaction of having a measurable impact.
We often talk about the distinction between being "process oriented" vs. "results driven." Some criticize process orientation for not caring enough about outputs. Results driven cultures can be toxic for overly punishing failure and not appreciating effort. A "follow your shot" culture is a balance between these two extremes. It values the methodical execution of a defined process and also the agility to adapt to get the desired result.
Whatever the task, there is a "follow your shot" behavior that will increase its chance of success: checking the deployment pipeline after a code merge; monitoring dashboards during a dial-up; following up on a customer support case to ensure that the problem was resolved. You just need to ask yourself "how do I follow this shot?"
Feb 12, 2023
If you invested time and effort to build a new feature on your product, you had at least an informal hypothesis that the new capability would bring value to your customers. But did you actually test that hypothesis by validating the value? Chances are you obsessed over the design, implementation, and delivery of the feature but your attention shifted to the next project once the feature was launched. If you make a habit out of this, your product becomes a bloated jumble of features that your customers can't discover, don't understand, or don't like. You have fallen into the Feature Debt Trap. The cost of maintaining a product like this is high because of the large surface area for defects and you are vulnerable to disruption by a simpler product that delivers high value by scratching the right itches.
To avoid this fate, or at least slow the onset, you must concentrate your investment on the features that matter the most to your customers and ensure that they are delivering high value. If you look at one metric, start with retention - not for your entire product but for users of the feature. The strongest evidence of a feature delivering value is continued use. Retention means that customers have a recurring need and have found your solution the most convenient way to satisfy it.
The level of usage regularity one can expect from a feature depends on the feature. For example, your payroll product's W-2 generation function probably just gets used once per year. There are some important features that solve sporadic problems such as password reset. But, for most features, regular usage means high perceived value.
You measure retention by duration. 2-Week Retention is the percent of customers who continued to use the feature a week after they started to use it. 3-Week Retention is the percent of that group that continued to use it in the 3rd weeks and so on. Retention usually declines over weeks and then stabilizes. The shape of the decline is called a Survival Curve. The longest survivors are getting the most value from the feature. For some features, the time range when the survival curve starts to flatten can be considered the habit formation period - meaning that if someone sticks with something for this long, they continue to use it. For example, a meditation app. Or the survivor curve could be an asymptote approaching the ideal segment of users.
If the retention rate is low, your feature or product suffers from one or both of the following issues:
- Low usability. There are easier ways to solve the problem.
- Low utility. The problem that the feature solves isn't a regular priority for the customer.
You should fix those problems before you invest in promoting the feature or invest in support and maintenance.
The low utility problem is the harder one to address. It could mean you were undisciplined in managing the product ("wouldn't it be cool if") or didn't understand your customer very well. Or it could be that the customer doesn't understand the value. For example, they may not anticipate the value of having a backup or auto-save. Whatever the root cause, you need to determine whether you can establish value or figure out an exit. Otherwise, the cost of maintaining the feature will be all waste.
The low usability problem is easier to solve and there are great techniques and skilled people who can help. The only hitch is that you need to plan for the investment. If you are locked into a road map that promises a hit parade of new feature launches, you will need to reset expectations. Nobody wins if your feature launches don't deliver value to your customers.
If a feature is not fixable, you should eliminate it. But that is hard to do if it has a small group of loyal customers that have relied on this feature to address an important need. To stay out of this situation, give yourself a two way door by leveraging pilot or soft launch strategies. Often these approaches are just used to test scaling but they offer a great opportunity to monitor retention.
Once you have achieved a suitable retention rate, your attention can shift to building adoption (measured by penetration rate) by raising awareness for the feature. But you still need to monitor retention in the form of lapse and churn. But that is for another blog post.
Jan 04, 2023
I have recently run across two professional development practices that I am interested in adopting; and now I am starting to think that combining them together will make them even more useful. The first practice is Jacob Kaplan-Moss's recommendation to maintain a transition file. This document is designed to help you train your successor and should contain notes on team mechanisms, role responsibilities, projects, and (if you are a manager) the people you manage. Jacob's reasoning for maintaining and updating this file every 2-3 months is mostly around supporting your team and showing general professionalism when you decide to leave. But I think this introspection is valuable even if you never leave your role. By preparing to teach someone to do your job, you can leverage the "Protégé Effect" to understand your job even better. Preparing to explain why something is done affords a perspective to see gaps, inconsistencies, and opportunities.
That brings me to the "Be Even Better" practice, which I learned about in an internal Amazon email list for managers. The post described a quarterly personal development day that team members schedule for themselves with the subject "Be Even Better." I love the title because it encourages a positive form of self criticism that embraces a "Growth Mindset."
One of the suggested question prompts to initiate the "Be Even Better" exercise is "if someone came in tomorrow and took my place, what would they be shocked or surprised about?" By updating the transition file and then asking this question, you give yourself an opportunity to combine the energy of a newcomer who wants to have an impact, with the experience and context of an incumbent. The next steps in the process are to prioritize, plan, and set goals for improvement. If you were training your successor, it would feel much better to say "this is how we are solving this problem" than to appear content that the problem persists.
I think we humans have a natural tendency to settle into a ruts where we just go through the motions of our day - attending meetings, responding to emails, processing deliverables... We get tired of being annoyed by issues and develop an unhealthy tolerance of them. Then we complain about burnout and "needing a change." We under-appreciate the potential to change our perspectives and see new challenges and opportunities in our current work environment where we can harness our domain expertise and professional relationships to achieve more.
That is not to say that a change of work environment isn't also a good idea. Being on the messy side of a learning curve is great mental exercise and the ability to share practices and ideas from different places is great for organizations. But a job change is risky. The recruitment process isn't ideal for learning whether you will like the new role. At Amazon, we have many "boomerangs" who come back after jumping into sub-optimal jobs. The art of managing a career is being able to maximize the growth you can get out of every role that you have and seeing when growth opportunities start to actually diminish .. rather than just appear to diminish because you stopped paying attention.
Dec 17, 2022
In light of the recent drama over at Twitter, I have been re-evaluating my use of social media. I have been off Facebook for years and for months I have made an effort to immerse myself in longer form content (newspaper articles, books, etc.). But I hadn't been able to resist the temptation of the Twitter timeline - specifically the little dopamine hits I got from snarky jokes and also surges of outrage at the world's injustices.
When I joined Mastodon (my handle is @sgottlieb@better.boston), I didn't want to recreate the same experience that I had on Twitter; specifically, the habit of trolling for dopamine. But it is taking me some time to figure out what I want from a timeline. I don't want to rely on a timeline to "stay informed" or "keep up to date." With their journalism skills and editorial standards, newspapers are much better resources to satisfy those needs. And I don't need to sift through a disorganized timeline to find links to articles that I should read. Content is already nicely organized in news websites and various newsletters that I subscribe to.
I rarely used Twitter as a social space to have conversations with people I know. And I am not looking for Facebook-style highlight reels from my real world friends. To maintain connections, I am much better off having real conversations.
So what problem am I looking to Mastodon to solve? I am inspired by passionate people doing noble things like advocating for more livable cities, scientific research, or any kind of creative expression. I like the idea of an open space to talk about esoteric topics. I am trying to learn how to use Mastodon to provide that experience. I think the solution lies in hashtags and being very deliberate in who I follow. It's a work in progress and I am patient.
Reach out if you have cracked the Mastodon code to create a perfect timeline.
Feb 07, 2020
I am pretty pleased by the low tech search I just implemented for this statically generated site. It's just a simple HTML form that appends "site:https://www.contenthere.net" to the end of the search string and then submits to Google. It's the little things.
Feb 06, 2020
I have been reading some articles on digital transformation and change management and I am surprised about how little attention is given to the notion of comfort. The reason why change is hard is that it makes people feel uncomfortable. And when people are not comfortable, their confidence, morale, and productivity suffers.
At the very least, there is the loss of familiarity. Habituated routines that once hummed along on the edge of our consciousness all of the sudden require direct attention. Tasks take longer to do. Mistakes are made. The confidence of mastery can feel threatened.
There is always a group of people who benefited from the old way of doing things even if that way was inefficient or dysfunctional. Dysfunction can create jobs or mask poor performance so don't assume that everyone is onboard to eradicate it.
There is great advice for assessing change readiness and rolling out change. But be aware that the resistance that you feel is a primal tendency to seek and preserve comfort. In addition to your conventional change management best practices, focus your attention on people whose comfort will be the most impacted. These will be people who have developed mastery over specialized skills and knowledge or have accumulated power from the current dynamics. Think about what benefits would be the most motivating for those groups to buy in. If you don't have their support, you can count on their interference.
And remember to measure the impact of a change after people have had a chance to re-equilibrate and form new comfortable habits.
Jan 04, 2020
I have been steadily reducing my Google footprint over the past year. I switched from gmail. I no longer use Google drive or photos. And, most recently, I migrated this blog from Blogger to a statically generated site hosted on Amazon S3.
I use a framework called Pelican to generate the full site from content files written in Markdown. I am writing this post using a Markdown editor called Typora, although most text editors have very good Markdown support. Then I run a command that generates HTML files and pushes them to S3.
Migration was incredibly easy. Google Takeout allows you to export all your posts as an Atom file. Then I wrote a script that turned the Atom feed into Markdown files. It was easy to keep the same URLs thanks to the "blogger:filename" elements in the Atom file. For a template, I chose a theme called Blue Penguin. After changing a couple of colors, I was done!
I am still working out the finer points of the workflow. So far, the main limitation I see is that I can't post from any computer like I was able to do with Blogger (and before that Wordpress). Before being able to generate the site, you need to set up a local environment -- not hard thanks to GitHub and Pipenv, but not something that I would want to do on my work computer. Probably the next time I get inspired to blog while traveling, I will email myself a post to publish later.
Overall, I am pretty happy with this setup!
Dec 09, 2019
When I start working with an established software development team, my favorite tool for understanding their process is a "hot lot." Hot lot is a manufacturing term where an order is expedited by being allowed to jump the queue. Hot lots are closely watched for progress and potential delays. In the world of software development, a hot lot can be a feature request that goes through a process of design, implementation, testing, enablement (documentation), release, promotion, and evaluation. A hot lot should accelerate the process by adjusting priorities but it should not circumvent the process by breaking rules or cutting corners.
By prioritizing a feature and watching it go through the process at top speed, you can learn many things. For example, you can learn...
- Whether the process is even responsive enough to accept a hot lot. Sometimes engineering is tied to rigid roadmaps and nothing, no matter how important, can jump the line. This is concerning if those roadmaps stretch out beyond the opportunities you can reliably anticipate.
- Whether there even is a defined process. Is there a mechanism for ensuring all of the tasks (qa, documentation, deployment, etc.) are completed? Or maybe there is a process but nobody follows it. If there is no practiced process, you can't trust the integrity of the system or whether anything is really "done."
- How the process is structured into steps, roles, and hand-offs. How many people touch the feature? How much time does each step take? How much time is spent waiting between steps? Is there excessive back and forth? Lean Six Sigma has a great framework for studying process cycle time, wait times, and waste across the value stream.
- The theoretical speed limit of the current process. You will never know how responsive your process can be when you always slow it down with competing priorities. Often actual speed is much slower than potential speed because of various delays, distractions, and interruptions that are not the fault of the process.
- Whether there are structural blockers like "we only release every 3 months." Or maybe the team is distributed over many time zones with little overlap for hand-offs and feedback.
- Whether there are capacity blocks like "Joe is the only person who can do that step and he is not available."
- How easy it is to monitor the process. Can you go to one place and see the completed and remaining work?
- The amount of managerial overhead that the process requires. For example, is there a project manager that needs to track and delegate every task?
- The artifacts the process creates. Can you go back and see what was done and why?
- How the response to the feature was measured and incorporated into future improvement ideas.
After running through a couple of these experiments, I have a pretty good understanding of the process structure, its theoretical speed, its strengths, and its flaws. At that point, we can start to come up with ideas for improvement. The low hanging fruit is usually pretty obvious ... especially to people who have been participating in the process but not paying attention to the overall throughput. Optimizations can be designed collaboratively and tested by future hot lots. I find that teams are generally comfortable with this evaluation because it doesn't target people as much as the framework that people work in. Usually processes (at least as they are practiced) form organically so nobody feels threatened by process improvements -- especially if they are clearly supported by empirical evidence.
Even if you have been working with a team for a while, try pushing through a hot lot and pay close attention to it. There is really no better way to understand the execution of a process.
Nov 07, 2019
Before building any functionality, a product team should first start by fully understanding the problem they are being asked to solve. This may sound obvious but I can’t tell you how many times I see one-liner Jira tickets that ask for something without explaining why. But the “why” is the most important part for a number of reasons.
- The team has to agree that the problem exists and is worth solving. The impact and urgency is a primary factor in prioritization.
- Being grounded in the “why” informs creativity to answer the “what” and the “how.” Design begins with empathy and you can’t have empathy if you don’t know what your users are struggling with.
- Solutions should be evaluated on how well they address the problem. This evaluation should drive design, QA, and post-release review.
To help people focus on the problem, I use a simple tool that I call a “problem definition.” This is a document (preferably a wiki page) that describes the problem and why it is important: inefficiency, risk, etc. There is also a section for proposed solutions where the author can suggest their ideas. The problem definition then becomes a focal point for clarification and learning. Stakeholders can ask questions to explore the use case.
I think this type of document was the original intent behind the “User Story” used in various agile methodologies. But over time, the User Story has been corrupted into a formulaic and useless “As a _____, I want to ________ so I can ________”; I have yet to read a User Story that really got to the heart of the problem and why it was worth solving.
Problem definitions are precursors to project artifacts like specifications and work items. They should be easy for anyone to write in their own language. No commitment is made to implement a solution. Sometimes problems can be solved with training or better documentation. Even if no action is taken, expressing and hearing these issues is important in bridging the gap between the product team and its users.
Everyone on the team should be able to answer the question “why are we doing this?” If they can’t, they can’t be expected to be contribute to an effective solution.
Oct 23, 2019
One of my favorite tech presentations is Rich Hickey’s Simple Made Easy. The premise of the talk is that simple and easy are not the same thing. In fact, you often sacrifice simplicity in pursuit of easiness. As Rich says, we consider something easy if it is “at hand.” Like the Staples Easy Button. Simplicity is something totally different. It is the absence of complexity - lots of moving parts entwined together in intricate ways. If an “easy button” really existed, it would be supported by a complex network of solutions that could take care of any problem.
Rich was talking about programming and how to keep code maintainable. Simple code is easier to understand and extend. But I apply this perspective to lots of things. For example, a bicycle is a simple machine. A quick glance reveals how it works and what every part does. But pedaling up a hill is not easy. A modern car is complex. There is a lot of stuff going on under the hood and nearly all drivers accept that they have no hope of understanding it all.
In building software, I have come to realize that users only value ease. A user wants the features he/she likes “at hand.” In a mature, multi-featured application, UI design is mainly focused on hiding some features to make the frequently used ones stand out. Users don’t want simple. Take away any feature and there will be complaints.
Developers want simple. They want to work with code that is understandable and behaves predictably. They realize that every new feature is supported by hundreds of lines of code that need to be tested with every modification. Much of Rich's talk deals with programming styles that unnecessarily create complexity. But some requirements will force even well designed code to become complex. Ironically, these requirements are often driven by a desire for a "simple and easy" user experience (personalization, natural language inputs, voice control...).
Why does this matter?
If we don't acknowledge that "simple and easy" are in conflict, there will be unmet expectations that lead to friction between stakeholders. Users can become impatient discussing complex details about their "simple feature." Development teams can feel under-appreciated for the effort required to do a "simple thing." The time taken to wrestle with ignored complexity can look like incompetence.
Take, for example, the Google search box. It is easy to use... just type in some text and click the button. But it is anything but simple. There is an art to constructing effective queries and a whole industry (SEO) dedicated to manipulating what comes back. There is also a set of features that makes the search box like the command line for the web. I can't tell you how many times I have heard "I just want something simple like Google." Google isn't simple. But it is easy. What makes Google search feel easy is that the core functionality is obvious and it gives useful feedback to help you get what you want. You may not get what you want on the first try, but it is easy to refine your search to hone in on your target. Voice assistants aim for the same level of ease but I find the trial/error loop to be more frustrating. That is probably because the system can return only one response and the feedback loop takes longer.
I know how annoying it can be for someone to pick at language and I am not advocating to constantly correct people on their word choices. But I do think it is important for everyone to understand what they are asking for and what they are giving up when they get it. We can do that by probing into what the user means by "simple." That question is reasonable because both "simple" and "easy" are subjective terms that require elaboration. When we document requirements, we should avoid all subjective language. After all, most of the work to achieve the perception of ease and simplicity is through iteration and refinement. These qualities are not intrinsic to the feature but rather to the sentiment of the user.