Axioms

An axiom is a self-evident truth that serves as a foundational principle. We use axioms to guide us in our design and implementation of the DSFX codebase. They are not just rules; they are the philosophical underpinnings of our approach to software development. By following the axioms, we can create a codebase that is not only functional but also elegant and a joy to maintain.

Our Motivation

"We are what we repeatedly do. Excellence, then, is not an act, but a habit." – Aristotle

We believe that excellence in software development occurs when the human element is in harmony with the technical. There is a reason why the swing of analog recordings is often preferred over the perfection of digital recordings. It’s the human touch, the imperfections, that make it feel alive. This is the essence of our motivation: to create software that resonates with the human experience, that is both functional and beautiful. We prioritize correctness, simplicity, and elegance in our code, and we strive to make our codebase a place where developers can thrive.

Why Have Style?

Steve Jobs once said, "Design is not just what it looks like and feels like. Design is how it works." In the world of software, this means that style is not just about making our code look pretty; it’s about making it work well. It's not about producing a product that works, but about creating a product that is a joy to work with. The axioms that we follow ripple through the project, shaping everything from individual lines of code, to the overall philosophy of the project.

The priorities of this style are simple:

Safety first,
Performance next,
and then developer experience.

Safety First

Even in Go, safety is non‐negotiable. We frequently embed explicit checks and assertions to ensure our code behaves as expected. Use the internal/lib/assert package for assertions. While using assertions is not idiomatic in golang, we believe that the safety they provide is worth the trade-off.

It is important to clarify what should be asserted. Assertions should never be used as a crutch for proper error handling. Rather, they should assert obvious bugs and developer errors, such as:

Preconditions: Check that inputs are valid before processing.
Postconditions: Ensure that outputs are as expected after processing.
Invariants: Validate that the state of an object or system remains consistent throughout its lifecycle.
Critical states: Assert that certain conditions are met at key points in the code.

Assertions cover our bases as developers. If we forget to initialize something, should we really be reporting this error to the user? No! We should panic and fix the code. Assertions are our safety net. In this case, the code should not be passing tests and we should not be releasing this code. It's true that we could write an explicit unit test to check for this, but that would be redundant and unnecessary. Again, these are developer errors, not user errors. We shouldn't be writing explicit tests for developer errors. Instead, we should be asserting the positive and negative space of our system to immediately crash the program if something goes wrong. This will fail the user level tests by proxy and alert us to the issue before it reaches production, without needing to manage a complex test harness just for these types of errors.

// Assert panics with a message if the condition is not met.
// Not idiomatic Go, but used here to enforce critical invariants.
func Assert(condition bool, msg string) {
	if !condition {
		panic(fmt.Sprintf("Assertion failed: %s", msg))
	}
}

Key safety guidelines:

Use simple, explicit control flow. No recursion or function trampolines. Do not use goto.
Limit loops and queues: every loop should have an explicit upper bound to avoid infinite cycles.
Favor fixed-size types (e.g., use uint32 or int32 where appropriate) instead of architecture-dependent sizes like int or uint. This helps avoid overflow and underflow issues.
Use assertions liberally—both to check expected states before and after important operations. Aim for at least two assertions per function.
Keep function bodies short. In Go, we suggest striving for functions to be easily digestible (roughly keeping functions around 70 lines or less) by breaking out helper functions where it makes sense.
Line lengths should be kept to a maximum of 100 columns to ensure readability and maintainability.

Performance

Performance starts in the design phase—even before profiling code.

Back-of-the-envelope sketches: Before implementation, roughly gauge resource (network, disk, memory, CPU) usage.
Identify the slowest resource (network → disk → memory → CPU) and optimize that first.
Use batching to reduce overhead and context switching. For example, aggregate database writes rather than doing them one at a time.
Separate the control plane from the data plane. For “hot loops” or performance‐critical code, create small, stateless helper functions accepting primitive types. This helps both the compiler and human reviewers spot redundant computations.

In a typical Go server, many things happen at once. The server is constantly reading from the network, writing to the disk, and processing data in memory. This can lead to contention for resources, which can slow down the server. To mitigate this, we recommend:

Don't use goroutines directly: Instead, use a worker pool that you can dispatch work to. This allows for better control over cpu usage, giving the program the ability to scale up or down depending on the load. This can be done in a controlled manner, without flooding the scheduler with goroutines. Never use user input to determine the acceptable tolerance for load.
Use channels for communication: Channels are a powerful feature of Go that allow for safe communication between goroutines. They can be used to pass messages between the worker pool and the main program, allowing for better coordination and control over the flow of data.
Use context for cancellation: The context package in Go provides a way to manage cancellation and timeouts for goroutines. This is especially useful for long-running operations that may need to be cancelled if they take too long or if the program is shutting down.
Use the sync package for synchronization: The sync package provides a way to manage synchronization between goroutines. This is important for ensuring that shared resources are accessed safely and that data is not corrupted by concurrent access.
Use the sync/atomic package for atomic operations: The sync/atomic package provides a way to perform atomic operations on shared variables. This is important for ensuring that data is not corrupted by concurrent access and that performance is not degraded by unnecessary locking.
Use the sync.Pool for object reuse: The sync.Pool provides a way to reuse objects to reduce memory allocation and garbage collection overhead. This is especially useful for performance-critical code that creates and destroys many objects.
Use the testing package for benchmarking: The testing package provides a way to benchmark code to measure its performance. This is important for identifying performance bottlenecks and for ensuring that changes to the code do not degrade performance.

Handling Errors

Errors must always be handled explicitly - they are never ignored. In a function that can produce more than one error, wrap the errors using fmt.Errorf("%w") to provide additional debugging context in a way that preserves the original error for use with errors.Is and errors.As. This allows for better error handling and debugging, especially in complex systems where multiple errors can occur.

Transient errors from external resources (like network or disk) should be retried with exponential backoff. This is especially important for operations that can fail due to temporary issues, such as network timeouts or disk I/O errors. Assume that there will be transient errors, and design your code to handle them gracefully.

A note on data corruption

Data corruption is a serious issue that can occur in any system. It can happen due to hardware failures, software bugs, or even malicious attacks. To mitigate the risk of data corruption, we recommend the following:

Use checksums: Always use checksums to verify the integrity of data before and after processing. This helps ensure that the data has not been corrupted during transmission or storage.
Use versioning: Use versioning to track changes to data and to ensure that the correct version of the data is being used. This helps prevent data corruption due to software bugs or changes in the data format.
Use backups: Always keep backups of important data to prevent data loss due to corruption or other issues. This is especially important for critical data that cannot be easily recreated.

Always write software with the assumption that things will go wrong. Packets will arrive out of order, more than once, or not at all. Data that you wrote to the disk might have been overwritten or corrupted by another process. Maybe the disk is failing. We need to be prepared for these scenarios and design our code to handle them gracefully. This means using checksums, versioning, and backups to ensure data integrity, as well as implementing robust error handling and retry logic for transient errors. By doing so, we can minimize the risk of data corruption and ensure that our software is resilient to failures.

Dependencies

We strive to maintain a zero dependency codebase. This means that we avoid using third-party tools, even in development. This ensures that supply chain attacks are minimized exclusively to the go toolchain.

9.8 KiB Raw Blame History Unescape Escape