diff --git a/docs/README.md b/docs/README.md index f5ab3b3..5174ecb 100644 --- a/docs/README.md +++ b/docs/README.md @@ -4,7 +4,7 @@ This is the documentation for DSFX, the next generation decentralized file excha ## Overview -- [TigerStyle](./tigerstyle.md) is our Go style guide, which outlines our design philosophy and coding standards. +- [Axioms](./axioms.md) is our style guide, which outlines our design philosophy and coding standards. - [Hosting](./hosting.md) gets you started with your own DSFX server. - [Administration](./administration/) shows you how to manage your DSFX server and its users. - [Operating](./operating/) covers the usage of the dsfx client to securely upload and download files. diff --git a/docs/axioms.md b/docs/axioms.md new file mode 100644 index 0000000..3c93e18 --- /dev/null +++ b/docs/axioms.md @@ -0,0 +1,177 @@ +# Axioms + +An `axiom` is a self-evident truth that serves as a foundational principle. We use axioms to guide us +in our design and implementation of the DSFX codebase. They are not just rules; they are the +philosophical underpinnings of our approach to software development. By following the axioms, we can +create a codebase that is not only functional but also elegant and a joy to maintain. + +## Our Motivation + +> "We are what we repeatedly do. Excellence, then, is not an act, but a habit." – Aristotle + +We believe that excellence in software development occurs when the human element is in harmony with +the technical. There is a reason why the swing of analog recordings is often preferred over the +perfection of digital recordings. It’s the human touch, the imperfections, that make it feel alive. +This is the essence of our motivation: to create software that resonates with the human experience, +that is both functional and beautiful. We prioritize correctness, simplicity, and elegance in our +code, and we strive to make our codebase a place where developers can thrive. + +--- + +## Why Have Style? + +Steve Jobs once said, "Design is not just what it looks like and feels like. Design is how it works." +In the world of software, this means that style is not just about making our code look pretty; it’s +about making it work well. It's not about producing a product that works, but about creating a +product that is a joy to work with. The axioms that we follow ripple through the project, shaping +everything from individual lines of code, to the overall philosophy of the project. + +The priorities of this style are simple: + +- **Safety** first, +- **Performance** next, +- and then **developer experience**. + +--- + +## Safety First + +Even in Go, safety is non‐negotiable. We frequently embed explicit checks and assertions to ensure +our code behaves as expected. Use the `internal/lib/assert` package for assertions. While using +assertions is not idiomatic in golang, we believe that the safety they provide is worth the trade-off. + +It is important to clarify _what_ should be asserted. Assertions should never be used as a crutch for +proper error handling. Rather, they should assert obvious bugs and developer errors, such as: + +- **Preconditions**: Check that inputs are valid before processing. +- **Postconditions**: Ensure that outputs are as expected after processing. +- **Invariants**: Validate that the state of an object or system remains consistent throughout its lifecycle. +- **Critical states**: Assert that certain conditions are met at key points in the code. + +Assertions cover our bases as developers. If we forget to initialize something, should we really be +reporting this error to the user? No! We should panic and fix the code. Assertions are our safety net. +In this case, the code should not be passing tests and we should not be releasing this code. It's +true that we could write an explicit unit test to check for this, but that would be redundant and +unnecessary. Again, these are **developer errors**, not user errors. We shouldn't be writing explicit +tests for developer errors. Instead, we should be asserting the positive and negative space of our +system to immediately crash the program if something goes wrong. This will fail the user level tests +by proxy and alert us to the issue before it reaches production, without needing to manage a complex +test harness just for these types of errors. + +```go +// Assert panics with a message if the condition is not met. +// Not idiomatic Go, but used here to enforce critical invariants. +func Assert(condition bool, msg string) { + if !condition { + panic(fmt.Sprintf("Assertion failed: %s", msg)) + } +} +``` + +**Key safety guidelines**: + +- Use **simple, explicit control flow**. No recursion or function trampolines. Do not use `goto`. +- **Limit loops and queues**: every loop should have an explicit upper bound to avoid infinite cycles. +- Favor fixed-size types (e.g., use `uint32` or `int32` where appropriate) instead of + architecture-dependent sizes like `int` or `uint`. This helps avoid overflow and underflow issues. +- Use assertions liberally—both to check expected states _before_ and _after_ important operations. + Aim for at least two assertions per function. +- Keep function bodies short. In Go, we suggest striving for functions to be easily digestible + (roughly keeping functions around 70 lines or less) by breaking out helper functions where it makes + sense. +- Line lengths should be kept to a maximum of 100 columns to ensure readability and maintainability. + +--- + +## Performance + +Performance starts in the design phase—even before profiling code. + +- **Back-of-the-envelope sketches:** Before implementation, roughly gauge resource (network, disk, + memory, CPU) usage. +- Identify the slowest resource (network → disk → memory → CPU) and optimize that first. +- Use batching to reduce overhead and context switching. For example, aggregate database writes + rather than doing them one at a time. +- Separate the control plane from the data plane. For “hot loops” or performance‐critical code, + create small, stateless helper functions accepting primitive types. This helps both the compiler + and human reviewers spot redundant computations. + +In a typical Go server, many things happen at once. The server is constantly reading from the network, +writing to the disk, and processing data in memory. This can lead to contention for resources, which +can slow down the server. To mitigate this, we recommend: + +- **Don't use goroutines directly**: Instead, use a worker pool that you can dispatch work to. This + allows for better control over cpu usage, giving the program the ability to scale up or down + depending on the load. This can be done in a controlled manner, without flooding the scheduler with + goroutines. **Never use user input to determine the acceptable tolerance for load.** + +- **Use channels for communication**: Channels are a powerful feature of Go that allow for safe + communication between goroutines. They can be used to pass messages between the worker pool and + the main program, allowing for better coordination and control over the flow of data. + +- **Use context for cancellation**: The `context` package in Go provides a way to manage + cancellation and timeouts for goroutines. This is especially useful for long-running operations + that may need to be cancelled if they take too long or if the program is shutting down. + +- **Use the `sync` package for synchronization**: The `sync` package provides a way to manage + synchronization between goroutines. This is important for ensuring that shared resources are + accessed safely and that data is not corrupted by concurrent access. + +- **Use the `sync/atomic` package for atomic operations**: The `sync/atomic` package provides a way to + perform atomic operations on shared variables. This is important for ensuring that data is not + corrupted by concurrent access and that performance is not degraded by unnecessary locking. + +- **Use the `sync.Pool` for object reuse**: The `sync.Pool` provides a way to reuse objects to + reduce memory allocation and garbage collection overhead. This is especially useful for + performance-critical code that creates and destroys many objects. + +- **Use the `testing` package for benchmarking**: The `testing` package provides a way to + benchmark code to measure its performance. This is important for identifying performance + bottlenecks and for ensuring that changes to the code do not degrade performance. + +--- + +## Handling Errors + +Errors must always be handled explicitly - they are never ignored. In a function that can produce +more than one error, wrap the errors using `fmt.Errorf("%v")` to provide additional debugging context +in a way that preserves the original error for use with `errors.Is` and `errors.As`. This allows +for better error handling and debugging, especially in complex systems where multiple errors can +occur. + +Transient errors from external resources (like network or disk) should be retried with exponential +backoff. This is especially important for operations that can fail due to temporary issues, such as +network timeouts or disk I/O errors. Assume that there **will** be transient errors, and design your +code to handle them gracefully. + +**A note on data corruption** + +Data corruption is a serious issue that can occur in any system. It can happen due to hardware +failures, software bugs, or even malicious attacks. To mitigate the risk of data corruption, we +recommend the following: + +- **Use checksums**: Always use checksums to verify the integrity of data before and after + processing. This helps ensure that the data has not been corrupted during transmission or storage. + +- **Use versioning**: Use versioning to track changes to data and to ensure that the correct version + of the data is being used. This helps prevent data corruption due to software bugs or changes in + the data format. + +- **Use backups**: Always keep backups of important data to prevent data loss due to corruption or + other issues. This is especially important for critical data that cannot be easily recreated. + +Always write software with the assumption that things will go wrong. Packets will arrive out of order, +more than once, or not at all. Data that you wrote to the disk might have been overwritten or corrupted +by another process. Maybe the disk is failing. We need to be prepared for these scenarios and +design our code to handle them gracefully. This means using checksums, versioning, and backups to +ensure data integrity, as well as implementing robust error handling and retry logic for transient +errors. By doing so, we can minimize the risk of data corruption and ensure that our software is +resilient to failures. + +--- + +## Dependencies + +We strive to maintain a **zero dependency** codebase. This means that we avoid using third-party +tools, even in development. This ensures that supply chain attacks are minimized exclusively to the +go toolchain. diff --git a/docs/internals/README.md b/docs/internals/README.md index 82447a3..2fac153 100644 --- a/docs/internals/README.md +++ b/docs/internals/README.md @@ -7,6 +7,8 @@ DSFX, you don't need to read this and could head straight to our user-level docs If you want to learn how DSFX works inside, here's what we got: -- [TigerStyle](../TIGER_STYLE.md) is _the_ style guide, and more. This is the philosophy underlining +- [Axioms](../axioms.md) is _the_ style guide, and more. This is the philosophy underlining all the code here! +- [commit strategy](./commits.md) is the guide for writing commit messages that follow the Conventional Commits + specification. - [handshake](./handshake.md) is the protocol for establishing a secure connection between two parties. diff --git a/docs/internals/commits.md b/docs/internals/commits.md new file mode 100644 index 0000000..f3159e4 --- /dev/null +++ b/docs/internals/commits.md @@ -0,0 +1,154 @@ +# Commit Strategy + +The project utilizes convetional commits to standardize commit messages across the codebase. This +document outlines the guidelines for writing commit messages that adhere to the Conventional Commits +specification. + +--- + +## Commit Message Structure + +Every commit message **MUST** follow this structure: + +``` +type(scope): subject + +[body] + +[footer] +``` + +- The header (first line) **must not exceed 100 characters**. +- The body (if provided) should explain the **what** and the **why** behind the change. +- The footer is reserved for any breaking changes or referencing issues. + +--- + +## Allowed Commit Types + +Use one of the following types to describe your commits: + +- **feat**: A new feature for the user. +- **fix**: A bug fix. +- **docs**: Documentation changes (in-code or external docs). +- **style**: Changes that do not affect the code meaning (formatting, semicolons, white-space, etc.). +- **refactor**: A code change that neither fixes a bug nor adds a feature. +- **perf**: A code change that improves performance. +- **test**: Adding missing tests or correcting existing tests. +- **build**: Changes affecting the build system or external dependencies. +- **ci**: Changes to the Continuous Integration configuration files and scripts. +- **chore**: Maintenance tasks (updating auxiliary tools or routines). +- **revert**: Reverts a previous commit (use the prefix “Revert:” in the subject). + +--- + +## Allowed Scopes + +Our project’s structure suggests a set of scopes that reflect the affected areas. When you add a scope, choose one that best describes where your change applies. Below are our recommended scopes: + +### Command-line Tools + +- **cmd/dsfx** – Core DSFX application commands. +- **cmd/dsfxctl** – Client-side commands (for dsfxctl). +- **cmd/dsfxnode** – Node or peer functionality. + +### Client & Peer + +- **internal/client** – Client functionality (dsfxctl). +- **internal/peer** – Peer and node operations (dsfxnode). + +### Core Libraries (located in `internal/lib`) + +- **assert** – Assertion helpers. +- **buffer** – Buffer utilities (length-prefixed buffers). +- **crypto** + - **encryption** – AEAD encryption and decryption routines. + - **identity** – Ed25519 key generation, signing, and verification. + - **keyexchange** – ECDH key exchange routines. +- **disk** – File system abstractions. +- **frame** – Network frame handling. +- **handshake** – Handshake protocol implementation. +- **logging** – Internal logging and context-based utilities. +- **network** – Network addresses, connections, listener/dialer functionalities. +- **storage** – Scoped storage interfaces. +- **system** – System-level operations and environment access. + +### Simulation + +- **internal/sim** – In-memory file system and simulated system tools. + +### Tools + +- **internal/tool** – Auxiliary tools (e.g., benchmarks, key generators). + +### Documentation + +- **docs** – Documentation improvements or changes. + +_Note:_ When a commit affects multiple areas, pick the scope that best represents the overall change; you may also leave it empty. + +--- + +## Examples + +Here are a few examples of acceptable commit messages: + +- **Feature example:** + + ``` + feat(client): add new file upload progress indicator + + Introduces a visual progress indicator for file uploads in the dsfxctl client. + ``` + +- **Bug fix example:** + + ``` + fix(handshake): correct nonce handling in authentication message + + Adjusts nonce extraction to fix a potential off-by-one error during handshake. + ``` + +- **Documentation update example:** + + ``` + docs: update internal documentation for conventional commits + + Added a new internal doc explaining our Conventional Commits guidelines. + ``` + +- **Refactor example:** + + ``` + refactor(network): simplify dial and connection setup logic + + Consolidated redundant code in network dialers for improved maintainability. + ``` + +- **Performance improvement example:** + + ``` + perf(disk): optimize file write performance in SimDisk + + Reduced simulated disk latency by refining timeout computations. + ``` + +--- + +## Breaking Changes + +If a commit introduces a backward-incompatible change, include a `BREAKING CHANGE:` token in the footer. For example: + +``` +feat(core): update authentication protocol + +BREAKING CHANGE: The client handshake now requires additional parameters. +``` + +--- + +## Conclusion + +Adopting Conventional Commits is a step toward ensuring a clear revision history and facilitating automated changelog generation. Keep your commit messages short, descriptive, and consistent with these guidelines. + +Place this document in your internal docs folder (e.g., `/docs/internal/CONVENTIONAL_COMMITS.md`) and use it as a reference for all contributors. diff --git a/docs/tigerstyle.md b/docs/tigerstyle.md deleted file mode 100644 index 2285f6b..0000000 --- a/docs/tigerstyle.md +++ /dev/null @@ -1,127 +0,0 @@ -# TigerStyle - -Adopted from the original [Tiger Style](https://github.com/tigerbeetle/tigerbeetle/blob/main/docs/TIGER_STYLE.md) -for **zig**, created by tigerbeetle. - -## The Essence of Our Go Style - -> “There are three things extremely hard: steel, a diamond, and to know one's self.” — Benjamin Franklin - -Our Go style is not merely about formatting code—it’s our design philosophy. It blends engineering excellence with artful clarity to ensure that our systems are safe, high-performance, and enjoyable to develop and maintain. - ---- - -## Why Have Style? - -In our view, design is as much about how the system works as what it looks like. Our style makes each decision explicit, ensuring that safety, performance, and developer experience remain paramount. - -- **Safety** first, -- **Performance** next, -- and then **developer experience**. - -Good style does more than improve readability; it enforces the design decisions that prevent technical debt from creeping in. - ---- - -## On Simplicity, Elegance, and Technical Debt - -- **Simplicity** is our ultimate goal—but it is never a free pass. Achieving simplicity is the result of thoughtful design and disciplined iteration. -- A “zero technical debt” policy applies: we design and implement things right the first time. This saves each of us from the cost of firefighting production issues later. - ---- - -## Safety - -Even in Go, safety is non‐negotiable. We frequently embed explicit checks and assertions to ensure our code behaves as expected. Consider this sample assertion helper: - -```go -// Assert panics with a message if the condition is not met. -// Not idiomatic Go, but used here to enforce critical invariants. -func Assert(condition bool, msg string) { - if !condition { - panic(fmt.Sprintf("Assertion failed: %s", msg)) - } -} -``` - -Key safety guidelines: - -- Use **simple, explicit control flow** (avoid unnecessary recursion and hidden complex abstractions). -- **Limit loops and queues**: every loop should have an explicit upper bound to avoid infinite cycles. -- Favor fixed-size types (e.g., use `uint32` or `int32` where appropriate) instead of architecture-dependent sizes. -- Use assertions liberally—both to check expected states _before_ and _after_ important operations. Aim for at least two assertions per function. -- All memory usage should be planned up-front. Although Go provides garbage collection, design your modules with a fixed resource strategy in mind for predictability. -- Keep function bodies short. In Go, we suggest striving for functions to be easily digestible (roughly keeping functions around 70 lines or less) by breaking out helper functions where it makes sense. - ---- - -## Performance - -Performance starts in the design phase—even before profiling code. - -- **Back-of-the-envelope sketches:** Before implementation, roughly gauge resource (network, disk, memory, CPU) usage. -- Identify the slowest resource (network → disk → memory → CPU) and optimize that first. -- Use batching to reduce overhead and context switching. For example, aggregate database writes rather than doing them one at a time. -- Separate the control plane from the data plane. For “hot loops” or performance‐critical code, create small, stateless helper functions accepting primitive types. This helps both the compiler and human reviewers spot redundant computations. - ---- - -## Developer Experience - -Our goal is to make our code easy to read, maintain, and extend. - -### Naming Conventions - -- **Go idioms:** For exported functions, use `CamelCase` (e.g., `ReadSector`), and for local functions and variables, use `camelCase`. -- Avoid abbreviations unless they are well-known acronyms (for example, use `HTTPStatus` rather than `httpStatus`). -- Include units and qualifiers in variable names at the end, so names like `latencyMsMax` (where “Ms” stands for milliseconds) are both clear and neatly aligned with related variables. -- When a function delegates work to helpers, consider prefixing the helper name with the parent function (e.g., `readSectorCallback`) to provide context. -- Place important constructs (like `main`) or critical types near the top of files for clarity. - -### Comments and Commit Messages - -- Write clear and complete comments. Comments should be full sentences with proper punctuation. -- Document why a decision was made, not just what the code does. Commit messages should convey context and rationale. - ---- - -## Handling Errors - -Errors must always be handled explicitly: - -- In Go, this means checking returned errors immediately. Do not ignore them. -- Test error paths thoroughly, including non-fatal errors, to mitigate the risk of catastrophic failures found in distributed systems. -- Use assertions where appropriate to ensure that numbers or states remain within expected ranges. - ---- - -## Avoiding Off-By-One Errors - -Misunderstandings between zero-based indexes and one-based counts can be problematic: - -- Clearly differentiate between an index, a count, and a size. For example, increment an index when calculating a count. -- Use helper functions for mathematical operations—consider naming them in a way that demonstrates intent (like `divExact`, `divFloor`, or `divCeil`). - ---- - -## Formatting and Dependencies - -- Run `go fmt` on all code. We also recommend using linters (such as `golangci-lint`) for consistency. -- Set a limit on line lengths (suggested maximum of 100 columns) to ensure clear visibility of code segments. -- Aim for a “zero dependencies” mindset where possible. Rely on Go’s standard library to reduce external risk and simplify builds. - ---- - -## Tooling - -- Choose tools that streamline development. When writing scripts or utilities, consider writing them in Go. -- A standardized toolchain reduces personal variations and ensures consistency across the team. - ---- - -## Final Thoughts - -In our adapted TigerStyle for Go, every detail—from strict assertions to deliberate naming and error handling—is designed to create robust, clear, and high-performance software. Remember: even though some of these practices (like assertions) aren’t common in Go, we include them as an essential part of our commitment to safety and clarity. - -> "You don’t really suppose, do you, that all your adventures and escapes were managed by mere luck?" -> Keep iterating, keep improving, and let your code be both a precise tool and a piece of art.