Tools have been used since the dawn of civilization as a means of discovering the world more effectively. From old stone age tools to modern machinery and software tools, helps us to spend less energy to achieve a higher outcome.

Products are simply tools that are marketed and sold that make the buyer's lives easier.

A software product exists to solve a problem by providing the user with an experience that lets them do more with less thinking. It takes energy to think. It also takes energy to repair the damages done by spending time inside and on electronic devices.

When humans spend more time outside and in natural environments, their bodies and minds will recover that energy. This puts the body under less stress, which will help reverse or prevent disease. See my writings on LinkedIn to see how I reversed type 2 diabetes and what I'm learning about the biosciences.

In the modern world, we should aim to provide tools to humans that let them accomplish this fundamental goal, more time with friends and family outdoors. This will translate to higher-quality work and outcomes for the company.

As software builders, we should be giving time back to our users.

But anyone who has been building software products, especially a software as a service will see this is not the case.

Building Software Products

Building software takes research and development (R&D). It is even on the company's balance sheet as R&D expenses and sometimes makes up for over 50% of the company's operating expenses.

Many companies strive towards a trio of Product, Engineering, and Design working closely together as R&D. These are the three core tenets for building a software product.

Unfortunately, for many, this balance is hard to achieve. The reality is that Research is separated from Development with a hand-off to engineering.

With poor or a lack of research, the development spending can be wasted as they will build the wrong things.

The industry doubled down and segmented this into Product Discovery and Product Delivery tracks. The delivery is handled by engineering, while the product managers and designers mostly handle the discovery. Both processes run in parallel.

This separation essentially created two different teams. One team handled what's referred to as Continuous Discovery. The discovery team conducts the research and "hands off" the result to the delivery team through a backlog.

The delivery team takes items off the backlog and "iterates" on the implementation of the product until the discovery team is satisfied.

This separation, no matter how close the team works, causes the high-risk development of the wrong features.

Personal Experiences

I've seen firsthand a variety of these processes at work:

Around 2017, I saw a process referred to as Dual-Track Scrum, implemented by Jim Axelsson while working at The Network Inc. a company that got acquired by Navex Global. Discovery and Delivery are new names for old processes. It works well if done by having all stakeholders involved in both tracks.

Care must be taken that the research does not end in Delivery, and we measure what was shipped.

Around 2019, I saw a process referred to as the triad, that involved a weekly meeting with the tech lead, designer, and product manager. This was implemented by Meagan Gleason while working at Auth0, which got acquired by Okta. In their meetings, they had questions like "what have we learned?", "what do we plan to learn and how?".

These two teams and leaders had a few things in common, which was to bring the engineers over into the product research/discovery to increase learning throughput. This creates one cohesive team.

The speed at which the team learns determines its success. But the speed of learning is as fast as the slowest person on the team, as you'll find out below, it is highly dependent upon their tools.

Delivery - Engineering

In engineering as an industry, we can quickly test and learn something within minutes. We can write code that gets deployed around the world to learn how it does on real production traffic and users.

This problem has been well solved over the past ten years with the investment in Continuous Integration and Delivery (CI/CD). These are automations that help us build, test, deploy and monitor code. They refer to this as DevOps, development and server operations working together.

A team that is not able to do this in this day and age is due to cultural issues and not tooling. We are engineers, so we build the tooling to make lives easier.

Github is a very good example of a platform that makes this easy. They provide source control, CI/CD, and project/issue management.

The hosting providers are getting better, such as Vercel, with great integration between Github and their coding framework NextJS.

We can then measure with analytics tools like Heap or Datadog.

At Samelogic, we only need to learn two tools to move from ideas to real hard data within minutes; Github and Heap Analytics. The tooling has removed a significant amount of stress on the engineers, so we can focus on just getting things done.

Discovery - Product & Design

The product manager and designers function as researchers. Their main goal is to ensure that the engineers build the right things. This is why they are found mostly working under the Product Discovery umbrella. It is exploration and risk reduction.

Conducting research should be simple;

1. Observe properly

2. Ask the right questions

3. Test what you hear and observe

4. Reason out what you've learned

But it gets tricky because we do not know if what we are "handing off" to the engineers is the right solution until we have measurable data from real users.

No matter how much concept testing is done on a Figma mock using tools like Maze, if the real users are not using it to achieve their outcomes, then that's a wasted effort in totality. The planning, design, and development effort is the cost of learning. Some folks have a metric called Time To Learn.

This is why feature validation and prioritization is the most important aspect of research, which usually falls on the product manager.

They are working longer hours, sometimes 70+ hours per week. They have little time for their friends and families.

They cannot quickly observe user behavior, ask questions, and test ideas and reason without using 6+ tools that provide fragmented data. The cognitive load is high, which increases the barrier to entry in this field.

This chart is from a report about the state of product leadership in 2022 by Pendo. If you read this chart properly, you'll see that 95% is about tooling and processes.

Part of my studies in bioscience is mental stress, which is directly linked to biochemical markers of disease and immunity. See the work of Hans Selye or the section "The anticancer mind" in the book Anticancer by David Servan-Schreiber.

If the team delivers the wrong thing, the Product Manager faces the brunt of the backlash. The lack of tooling and processes causes them to play corporate politics with guesswork more than anything else. Their instinct is to tell everyone who asks for a feature "no", which is stress for both them and their peers.

Even on a very good product team, the product manager has a higher chance of dying early.

Experiences at Samelogic

I will share personal experiences and challenges of conducting product research and what we've learned in the past year.

At Samelogic, I share responsibility for Product and Engineering with my cofounder Dwayne. So I have been between research mode and build mode.

How do I conduct research?

When I want user behavioral insights, I use Heap Analytics or Amplitude. This is pretty straightforward.

When I talk to users, I want to organize this data. So I use tools like confluence, notion, or google docs.

When I want to survey users in the product and have this data connected. Tools like Chameleon and Sprig could help us here, but their developer integrations do not allow a tight integration. So we had to build our own Microsurvey tool; we even made it open source.

As we learned the hard way, we must take what users say with a grain of salt. So after hearing what they have to say, we need to test it for ourselves. The test is usually to build the MVP, which takes away resources from engineering.

We came across a method of testing called fake doors in a few products. Ever clicked on a feature, and it gave you a survey or asked you to vote on it? They are testing an idea and getting hard evidence to know if the feature will be used before building it.

We spent the past year investigating this and found many brave people are using this method, from large news sites like Financial Times to product-heavy companies like Trip Advisor.

Dwayne has been researching this from the Design side of things and came across Atomic Designs. So we now call this Atomic Concept Testing. Look out for blog posts from him about this.

We now have a product that allows you to set up and launch a test with surveys to your audience in under a minute.

But this is not enough. Fragmentation is hitting us hard. The cognitive load is high. Our product stack is a mess. Even though this tool is already very powerful, it is still another tool for product managers to learn.

[tooling process image here]

This is why we are taking a step back and tackling the original problem of why Samelogic was started. Product Managers have no central collaborative command and control center like Github.

The research and product must work hand in hand. Mocks and codes have to live side by side. The analytics must represent this in very simple terms. Researchers love to share learnings, so this platform has to support a truly "build-in-public" approach.

We'll be conducting deep research and testing over the coming months to see what we can come up with. But for the time being, feel free to take our Atomic Concept Testing tool for a spin.