2026 - A Testing Philosophy

I was recently asked by an interviewer to describe my testing philosophy. I stumbled through my answer but I could tell it was unsatisfactory. Around the same time I also came across the same challenge at work. In that context, I was the leader attempting guide my team to a more robust testing strategy. As a result of that I produced a document AND was able to codify the strategy into an LLM persona. Below you will find my current testing philosophy and the approach I take to testing software applications. I hope you find it useful.

As a software engineer it is crucial to my success that the applications I write work. Tests inherently improve the quality of code because it requires that you can break down the code into small composable chunks and verify the functionality of each chunk/unit.

This document outlines my current testing philosophy and how I approach testing for traditional software application. ==Note: AI Testing (aka "evaluations") is a different challenge.==

[!tldr] I believe that testing adds value through validation and regression checks to any piece of software and anything "production ready" should be well tested.

The tests themselves should follow these principles:

1. Tests only give you value when you put value in

• Garbage in/Garbage out
• Descriptions should highlight what is being tested and what value it adds
• Failures should provide meaningful feedback on why it failed

2. Tests (like code) should be cumulative

• Many focused tests are better than one cumulative test

3. Tests should focus on behavior and not implementation

• Answer the question: "How we expect the code to behave?"

4. Tests should be deterministic
   • Tests should produce the same result every time

When not to test (aka Testing Blacklist)

In general, adding tests to your code is a net benefit. It helps catch bugs related to information management and operational flow. Below we talk about the types of tests that I feel should be written for an application (see below), I want to first address that just because testing improves quality, does not mean everything needs to be tested.

The term "Testing Blacklist" refers to the idea that we would opt out of testing specific chunks of code. My preferred way to approach selecting which tests to blacklist is by answering the following question

[!question] Questions

1. Does the codebase I am testing have control over the outcomes of my tests?
2. Does the file/code do any processing?

These questions will primarily identify things like:

1. Libraries and static setup configurations
2. Configuration files (such as constants, interfaces, or definitions)
3. Registration or Barrel Files (think index.js or __init__.py files) which typically act as collectors and unified interfaces

Everything else, should be tested.

Testing Types

I found that having a clear definition of what a test was and what value it added, provided clarity for both myself and teams that I worked with around what is expected from a test AND what value a test provides.

Manual Testing (Discouraged)

There will be times where the value proposition of writing an automated test does not exist OR that you need a completely different process to test the code than you would need to execute the code.

This is where you may still need manual tests. This type of test needs to be manually executed by a human. In these scenarios the human doing the testing will need record what they did and how they did it. This allows us to record the value add and determinism that we get from automated tests.

---
date:   // Datetime test was run
result: // Pass | Fail
tester: // Name of the human doing the testing
versions:
	// Map of packages/libraries being tested
	// - [email protected]
	// - iEngineer
---

# Manual Testing Report

| | |
| --- | --- |
| Test Cases | 1 |
| Tests Passing | 1 |
| Tests Failing | 0 |

## Test Summary

This document outlines the manual testing needed to 

## Environment

**Base URL:** https://application.example.com
**Environment:** Staging

| Feature Flag | Enabled |
| --- | --- |
| ShowDebugging | No |

## Detailed Test Cases 

## 01 - User can login

**Tests Data:**

- Username: [email protected]
- Password: *******

**Tests Steps:**

1. Navigate to the application landing page
2. Click the "Sign In" button
3. Login using the users credentials
4. Confirm user home page is shown
  
**Expected Results:**

- [ ] Authentication Successful
- [ ] Redirected to correctly to the home page
- [ ] No Console Errors: [ ] Yes [ ] No

**Notes & Issues Encountered:**

- 

The above template can be generated for a manual tester to follow. It is both a guide and an interactive form. The manual tester will configure the environment, follow the steps and fill out the Expected Results and Notes & Issues Encountered sections for each test. Then populate the test case pass/fail count.

Developer Testing

Developer Testing is a form of testing done during development. These tests are meant to be run in isolation, focusing on logic written in the project. Developer tests should be executable anywhere without the need for externally running dependencies. This allows developers to run these tests locally OR in CI environments without heavy setup

The question we want to answer with these types of tests is, "do the individual elements of the application function as we expect them too." This question is broken out into three types of tests to highlight the 3 types of elements we have in our application: Unit Tests, Orchestration Tests, External Orchestration Tests

Frameworks:

• Javascript/Typescript - Mocha + Chai

General Best Practices

• Test are written alongside the code - Test files are written adjacent to the code that is being tested
• Tests should be used to accept new changes

Unit Tests

Unit tests are the core type of developer test. These will be the most frequently written test and should focus on isolated units of code that use only platform-native dependencies. Examples would include, using the JS Math API or the NodeJS fs api.

These tests should be exhaustive, testing as many scenarios as possible to ensure that there is no cap in how the application will behave to possible inputs.

Best Practices:

• Unit tests should be unique: Each test should testa specific scenario or edge case
• Use descriptive test names: Each test should have a descriptive name which clearly outline what is being tested. Example: should return error when input is null is better than should call validateInput function because it highlights what the input is and what the expected output is
• Test both positive and negative scenarios: Write test that cover both they "happy path" (positive tests) as well as unexpected inputs to ensure the unit of code behaves correctly in all situations
• Keep test simple: Avoid complex logic or conditionals in your tests. If a test is hard to understand, it is hard to maintain

Orchestration Tests

As we write more complex code (and follow the law of composability) we will start to drift away from individual/functional units of code by composing those units together into larger functions. We call these Orchestration functions. The real value in this type of test is that we want to isolate the logic the function to coordinate the nested unit functions and NOT the unit functions themselves

An example of an Orchestration function might be a function which calls our units based on a condition:

import { add, subtract } from './math.js';

enum Operator {
	ADD = 'add',
	SUBTRACT = 'substract',
}

function doMath(a: number, b: number, operator: Operator): number {
  switch(operator) {
    case Operator.ADD: {
      return add(a, b)
    }
    case Operator.SUBTRACT: {
      return subtract(a, b)
    }
    default:
	    throw new Error('Invalid Operator')
  }
}

In the above example we have a method called doMath which in turn calls an add and subtract functions. In our Orchestration tests we want to avoid calling add and subtract (since they already have unit tests) and instead focus on making sure that:

1. The correct function (add/subtract) was called based on the operator
2. The function that is called (add/subtract) was passed the correct arguments in the correct order
3. That our system correctly handles invalid operators

This ensures we do not duplicate testing the add and subtract methods.

Best Practices:

• Focus on the interactions between units - The primary goal of orchestration tests is to verify the correct units are called based on the inputs provided. Focus on testing the interactions and not the results of the underlying units.
• Use spying when possible over mocking

External Orchestration Tests

There are rarely cases where an application is 100% self-contained. We call out to Databases, External APIs, Caches, and the File System. All of which are other pieces of software. Importantly those external resources should have already tested themselves (or we should not use them if they are untested).

What our tests do need to cover is verification of data contracts. When we call a database, we want to verify that we are providing the correct input AND that if the database throws an error, the system can handle the response.

Best Practices:

• Always mock external services - We should never make real calls to external services. The input should be spied on and the response should be mocked to produce a deterministic result
• Test all documented error scenarios - External Orchestration tests should ensure the code can handle failures from those external systems
• Verify input construction with spies - Use spies to verify that the external orchestration correctly constructs the input arguments.
• Mock realistic response structures - Use actual response structures from documentation to build real examples. This allows the code to also act as a form of documetnation
• Document the API version - Keep track of which version of an API or SDK you are testing against so that it is clear if tests are outdated compared to dependencies.

End-to-end Testing

End-to-End (E2E) tests are named such because they test the entire application from start to finish. This can be simulating real scenarios or verifying a multi-step process works as expected. These tests interact with a live environment which uses external services like databases. This ensures the application works in real-world scenarios. The other big variant from Developer Tests is that these should focus on expected paths rather than being exhaustive.

E2E tests can further be broken up into 2 sub-categories: Functional Tests and User Workflow Tests.

Frameworks:

• Browser Based and REST APIs - Playwright

Functional Tests

Functional tests focus on non-user facing processes. These could be background tasks, automated jobs, or system integrations. An example might be a webhook which is triggered by an external service. This may generate some data and it will be important for the system to be able to confirm the data is created correctly.

Best Practices:

• Always clean up test data - Leave the environment better than you found it. Delete data after it was created in a test, or revert if it was modified.
• Always use the same data for each run - Ensure tests are deterministic by reusing the test pattern
• Verify actual state changes - Make sure to check beyond the UI to make sure changes were applied. Verify in the database, queues or other systems directly
• Test with realistic data - Use data that resembles production scenarios

User Workflow Tests

As the name implies, User Workflow Tests emulate the users workflow in an application. This is especially common with UI and focuses on a specific user task that needs to be completed, verifying that a user could complete those steps.

Each test should focus on a specific action and have an expected outcome. Ex: when a user uploads a file, the application should notify them that the file was accepted.

From a developers perspective you can think of these tests similar to the testing you do while developing a feature, simply turned into a codified (and repeatable process).

Best Practices:

• Use semantic selectors: Prefer selectors based on user-visible attributes like roles (getByRole('button')), labels, or discrete test IDs over CSS Selectors like class name or element hierarchy.
• Test from the user's perspective: Focus on what the users can see and do, not on what is implemented. Test workflows like "user can submit a form" not "form submit handler called"
• Write clear action descriptions: Describe actions in plain language as if instructing a human tester. Ex: "Click the Submit Button" rather than "Trigger click event on #submit-btn"
• Clean up test data: Remove any data that was created after the test has been completed (pass or fail)
• Use meaningful waits: Wait for specific conditions (element visible, text appears) rather than arbitrary timeouts. This ensures tests fail faster AND it is more clear why the test failed.

Testing Anti-Patterns

Understanding what not to do is just as important as knowing what to do. Here are common mistakes that make test brittle, unreliable, ineffective

Testing Log Messages

Logs are a common component of our applications. They let us record snapshots at points in time to see what is happening or when something happened.

❌ DONT

test('should create a record', () => {
	const consoleSpy = jest.spyOn(console, 'log');
	
	createRecord({})

	expect(consoleSpy).toHavebeenCalledWith('Record with ID: X - Created')
})

✅ DO

test('should create a record', () => {
  

})

Why is this a problem? - Log messages are nondeterministic, change frequently, and can be changed independent of the actual functionality.

Appendix A: Playwright Test Documentation

When writing playwright tests, I have found a couple of helpful patterns that allow me to organize tests as well as describe their value. One of the challenges with most testing frameworks is capturing intent behind a test.

Playwright gives us annotations as a tool to attach metadata about the test. I have found that I can use this to my advantage and have constructed this interface for a TestSuite:

interface TestSuite {
  id: number;
  name: string;
  notes: string;
  tags: [];
  description: string;      // Description of what the suite is testing
  purpose: string;          // Describes the value being added by the feature
  users?: [];               // List of user personas to use for testing
  steps: Array<{            // List of descrete steps to be executed
    tags: [];
    action: string;         // Describes what action is being taken
    expectedOutcome: string // Describes what we expect to happen
    test: Function          // The actutal test logic
  }>
}

This can be combined with Playwright to create a comprehensive report that not only shows the test results but also describes the why and how of the test.

Appendix B: Playwright Tags

Most test runners offer a way to "grep" for specific tests or to run a sub-set of tests in a single execution. One way playwright does this is through tags.

What I want to highlight in this section is not that tags are important but that tags should be a way to collect tests. I want to highlight here a couple of examples of tags which I have identified as useful for testing as well as where I use those tests.

[!note] Notation - Developer Tests For Developer Tests, I typically append the tags to the end of the test name:

// Mocha

enum TestTypes {
     UNIT = '[unit]',
     ORCHESTRATION = '[orchestration]',
     EXTERNAL = '[external-orchestration]'
}

it(`\should create a new record ${TestTypes.UNIT}\`, () => {})

it(`should throw an error when the input is invalid ${TestTypes.UNIT}\`, () => {})

[!note] Notation - E2E Testing For E2E Tests, I use the playwright Tag System:

// Playwright

enum TestTypes {
     FUNCTIONAL = '@functional',
     USER_WORKFLOE = '@user-workflow',
     SMOKE = '@smoke',
     COMPREHENSIVE = '@comprehensive'
}

it(`\should create a new record ${TestTypes.SMOKE}\`, () => {})

it(`should throw an error when the input is invalid ${TestTypes.COMPREHENSIVE}\`, () => {})

Some common types of tests that I like to have are: