Merge branch 'docs'

docs/adr/0001-architecture-overview.md

@@ -0,0 +1,56 @@
+# ADR-0001: Architecture Overview
+
+## Status
+
+Proposed
+
+## Context
+
+Our system is a self-hosted, privacy-focused file storage solution that enforces client-side encryption, robust file versioning, and decentralized peer-to-peer (P2P) interactions. Users must have full control over their data, with all operations being transparent, auditable, and reversible. In addition, our requirements demand that tests simulate real-world conditions such as network and disk latency and failures (see Functional Requirements FR-1, FR-4, FR-6 and Non-Functional Requirements NFR-2, NFR-6). This necessitates an architecture that not only provides a seamless user experience in normal operating conditions but also supports deterministic simulation of edge-case scenarios.
+
+## Decision
+
+We will adopt a _hybrid architecture_ that splits responsibilities between a client application and self-hosted server/P2P nodes. The design includes the following elements:
+
+- **Client-Side Encryption and Processing:**
+  All file encryption, decryption, partitioning, and version tracking is handled at the client. This ensures that files are secured (FR-1) before leaving the client and that the user remains in full control of their data. This design choice reinforces our core principle of user-controlled digital self-ownership.
+
+- **Peer-to-Peer (P2P) Connectivity:**
+  In addition to the client-server interaction, nodes will interact directly with one another via secure P2P connections (FR-4). These connections will leverage public-key based authentication, and support NAT traversal techniques to simulate and handle various networking conditions, including latency and intermittent disconnections (NFR-2).
+
+- **Robust Replication and Redundancy:**
+  A replication layer ensures that file chunks and versions are consistently available across multiple nodes (FR-5). The architecture supports automatic replication and rebalancing, even under simulated failure conditions such as disk or network outages (NFR-3, NFR-6).
+
+- **Deterministic Simulation Capability:**
+  The system will be designed as a non-I/O state machine at its core, allowing deterministic simulations of network delays, disk failures, and other real-world problems. This design supports our testing strategy, ensuring that performance, reliability, and auditability requirements are testable under controlled conditions (see overall testing strategy integrated in FR and NFR sections).
+
+## Consequences
+
+- **Advantages:**
+
+  - **Enhanced Security:** By performing all sensitive operations on the client, our system better protects user data from unauthorized access.
+  - **Improved Testing:** The non-I/O state machine design allows for deterministic simulations, making it easier to verify system behavior in the face of network and disk failures.
+  - **Scalability and Resilience:** The decentralized approach via P2P interactions and replication supports scalability and reduces the risk of single points of failure.
+  - **User Empowerment:** The architecture reinforces our commitment to privacy and user control by ensuring that all operations are transparent and fully auditable.
+
+- **Trade-offs:**
+  - **Complexity:** Introducing a hybrid model with P2P nodes increases the complexity of both development and maintenance compared to a purely centralized solution.
+  - **Testing Overhead:** While deterministic simulations improve reliability, they require a robust testing framework that accurately models various failure conditions, potentially increasing development time.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - FR-1: Secure file upload and download (ensuring client-side encryption).
+  - FR-4: Peer-to-peer node interaction (supporting direct P2P connections).
+  - FR-5: Replication and redundancy management (ensuring data integrity across nodes).
+  - FR-6: User control and access management (ensuring auditability in all operations).
+
+- **Non-Functional Requirements:**
+  - NFR-2: Performance & responsiveness (accounting for simulated network and disk latency).
+  - NFR-3: Scalability & capacity (ensuring robust operations under increasing loads).
+  - NFR-6: Deployability & maintainability (facilitating deterministic simulation of deployment scenarios).
+
+## Conclusion
+
+The proposed hybrid architecture meets our overarching mission by ensuring privacy, user control, and robustness. It supports both regular operations and deterministic simulation of adverse conditions, satisfying our functional and non-functional requirements. This architecture lays a solid foundation to build a trustworthy, resilient system where every user interaction remains secure, fully auditable, and under the complete control of the user.

docs/adr/0002-encryption-algorithms-and-modes.md

@@ -0,0 +1,58 @@
+# ADR-0002: Encryption Algorithms and Modes
+
+## Status
+
+Proposed
+
+## Context
+
+Our system mandates that all file data is protected by strong, client-side encryption to ensure that no plaintext data leaves the client environment (FR-1). This is essential to safeguarding user privacy and maintaining their complete control over their digital data. The encryption module must be designed to operate under a variety of conditions—including simulated network and disk failures—to ensure performance and reliability consistent with our Non-Functional Requirements (NFR-1 and NFR-2).
+
+The encryption component is a cornerstone of our security strategy and directly affects how the system performs under normal and adverse conditions. It must deliver both robust security guarantees and high performance, even when subjected to simulated delays and failure modes in our deterministic testing framework.
+
+## Decision
+
+We will implement authenticated encryption for all file operations. In particular:
+
+- **Algorithm Selection:**
+  We will evaluate and use either AES-GCM or ChaCha20-Poly1305 as our authenticated encryption algorithm. Both options are recognized for their strong cryptographic assurances and efficiency.
+
+  - **AES-GCM** is preferred when hardware acceleration (e.g., AES-NI) is available.
+  - **ChaCha20-Poly1305** offers strong performance on platforms without dedicated AES hardware support.
+
+- **Mode of Operation:**
+  The chosen encryption algorithm will be implemented in an authenticated encryption mode to ensure not just secrecy but also data integrity. This operation mode is critical for detecting any tampering or corruption, aligning with our auditability and transparency requirement (FR-6, NFR-5).
+
+- **Integration with Testing Framework:**
+  The encryption module will be designed to support deterministic simulation testing. This will allow us to simulate network and disk latency as well as failure conditions (refer to NFR-2 and NFR-6) without compromising the encryption process.
+
+- **Key Management:**
+  Encryption keys are to be derived and managed strictly on the client side using secure key derivation functions such as Argon2, scrypt, or PBKDF2. This fits into our overall mandate to ensure that sensitive operations remain isolated from external systems, directly supporting FR-1 and NFR-1.
+
+## Consequences
+
+- **Advantages:**
+
+  - **Enhanced Security:** Using authenticated encryption guarantees that both the confidentiality and integrity of the user’s data are preserved, reinforcing our commitment to strong privacy (FR-1, NFR-1).
+  - **Flexibility:** Supporting multiple algorithms (AES-GCM and ChaCha20-Poly1305) allows optimization based on the target hardware environment, ensuring consistent performance (NFR-2).
+  - **Robust Testing:** Designing the encryption component as part of a deterministic simulation environment ensures that its performance and reliability can be rigorously tested under simulated adverse conditions.
+
+- **Trade-offs:**
+  - **Complexity in Implementation:** Integrating multiple algorithm options and ensuring seamless fallback based on hardware capabilities adds complexity to the development process.
+  - **Performance Overhead:** While authenticated encryption is critical for security, it may introduce slight performance overhead. However, this is offset by the benefits of enhanced data integrity and confidentiality.
+  - **Testing Complexity:** Incorporating deterministic simulation of real-world conditions such as network and disk latency requires additional development and integration effort within the testing framework.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - FR-1: Secure file upload & download requires encryption to be performed on the client before any file transmission.
+  - FR-6: User control and access management relies on robust encryption to secure data access and audit logs.
+
+- **Non-Functional Requirements:**
+  - NFR-1: Security & privacy mandates the use of strong cryptographic standards.
+  - NFR-2: Performance & responsiveness demands that encryption operations meet established benchmarks even under simulated failure conditions.
+
+## Conclusion
+
+The decision to use authenticated encryption with either AES-GCM or ChaCha20-Poly1305 meets our core requirement for strong client-side encryption while supporting our broader goals of privacy, auditability, and robustness. This approach not only secures file data at the point of entry and exit but also integrates seamlessly with our deterministic simulation framework, ensuring that performance and reliability are upheld even under adverse conditions. This decision directly supports our mission to empower users with secure, verified, and fully controlled digital storage.

docs/adr/0003-key-management-strategy.md

@@ -0,0 +1,56 @@
+# ADR-0003: Key Management Strategy
+
+## Status
+
+Proposed
+
+## Context
+
+A fundamental aspect of our secure file storage system is ensuring that encryption keys remain under the exclusive control of the user. As all file encryption and decryption is performed on the client side (FR-1), it is critical that the key management strategy prevents external exposure and compromise of encryption keys. In addition, our system must provide a secure and efficient method for deriving keys from user inputs such as passphrases, while remaining compatible with our deterministic simulation testing framework to mimic real-world scenarios, including network and disk failure conditions (NFR-1, NFR-2).
+
+The key management approach must also support integration with OS-level keychains or hardware-based key management modules when available, ensuring that keys are stored and accessed in a secure manner, thereby enhancing overall user privacy and data security.
+
+## Decision
+
+We will implement a client-side key management strategy based on secure key derivation and strict local storage practices. The strategy will include the following elements:
+
+- **Key Derivation Function (KDF):**
+  We will use one of the recognized strong key derivation functions such as Argon2, scrypt, or PBKDF2 to derive encryption keys from user-supplied passphrases. This approach aligns with our requirement to derive keys securely on the client side (FR-1, NFR-1) while ensuring that the derivation process is robust against brute-force attacks.
+
+- **Local Key Storage:**
+  Keys will be managed and stored exclusively on the client. Where possible, integration with OS-level secure keychains or hardware security modules (HSMs) will be provided to further protect key material. This approach bolsters the privacy guarantee by ensuring that keys never leave the user’s controlled environment.
+
+- **Deterministic Simulation Support:**
+  The key management module will be designed to allow deterministic simulation of key derivation and storage processes. Testing will simulate conditions such as network and disk latencies and possible failures to ensure that the key management functions remain reliable under all conditions (NFR-2, NFR-6).
+
+- **Auditability and Transparency:**
+  Any operation involving key management (such as key derivation, updates, and revocations) will generate audit logs that are tamper-evident (FR-6, NFR-5). This ensures that every change in key status is tracked, and the process remains transparent and verifiable to the user.
+
+## Consequences
+
+- **Advantages:**
+
+  - **Enhanced Security:** By deriving and managing keys entirely on the client side and leveraging strong KDFs, the strategy isolates sensitive key material from external exposure, upholding our commitment to rigorous data security (FR-1, NFR-1).
+  - **User Empowerment:** Integration with OS-level keychains or HSMs gives users additional confidence in the security of their keys, strengthening the principle of digital self-ownership.
+  - **Robust Testing Capability:** The designed compatibility with deterministic simulation testing allows for thorough evaluation of key management processes under adverse conditions, ensuring continuous performance and reliability (NFR-2, NFR-6).
+
+- **Trade-offs:**
+  - **Performance Impact:** Advanced key derivation functions such as Argon2 may introduce additional computational overhead. However, this is a necessary trade-off to ensure strong protection against brute-force attacks.
+  - **Implementation Complexity:** Integrating with multiple potential key storage mechanisms (OS-level or hardware-based modules) may increase the overall complexity of the implementation.
+  - **Testing Complexity:** Simulating the key management process under various network and disk failure scenarios requires careful design of the deterministic testing framework, potentially increasing the testing and validation effort.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - FR-1: Secure file upload & download, ensuring that all encryption and decryption happens on the client, backed by strong key management.
+  - FR-6: User control and access management is supported by precise and auditable key management operations.
+
+- **Non-Functional Requirements:**
+  - NFR-1: Security & privacy, which mandates the use of robust cryptographic techniques and secure key management practices.
+  - NFR-2: Performance & responsiveness, as the key derivation process should operate efficiently even under simulated network and disk latency conditions.
+  - NFR-6: Deployability & maintainability, which requires that key operations be thoroughly testable under deterministic simulation environments to ensure reliability.
+
+## Conclusion
+
+The proposed key management strategy, centered on strong, client-side key derivation and secure key storage, meets our core system requirements. It preserves user privacy and data control by ensuring that encryption keys are derived, stored, and managed exclusively on the client. The design’s compatibility with deterministic simulation testing further ensures that key management remains robust under a variety of adverse conditions, fulfilling our mission to create a secure, transparent, and user-controlled file storage system.

docs/adr/0004-file-partitioning-and-chunking-strategy.md

@@ -0,0 +1,54 @@
+# ADR-0004: File Partitioning and Chunking Strategy
+
+## Status
+
+Proposed
+
+## Context
+
+Efficient file storage and transfer are critical to our system, particularly because files must be securely encrypted on the client before transmission. To meet our requirements, files are partitioned into discrete chunks to enable efficient deduplication, versioning, and reassembly (FR-2). This strategy also supports deterministic simulation testing of disk latency, failures, and other adverse conditions, thereby ensuring robust performance and stability under both normal and simulated failure conditions (NFR-2, NFR-3, NFR-6).
+
+## Decision
+
+We will adopt a fixed-size chunking strategy for processing files:
+
+- **Fixed-Size Chunking:**
+  Files will be divided into fixed-size chunks (e.g., 1 MB per chunk). This method simplifies chunk management by providing predictable boundaries for deduplication, encryption, and reassembly.
+
+  - **Advantages:**
+    - Simplicity and predictability in processing.
+    - Straightforward reassembly of files.
+    - Efficient deduplication based on consistent chunk sizes.
+
+- **Deterministic Simulation Support:**
+  The fixed-size chunking mechanism will be designed to integrate seamlessly with our deterministic simulation framework. This ensures that we can simulate disk latency, disk failures, and other adverse storage conditions during both the partitioning and reassembly processes (NFR-2, NFR-3, NFR-6).
+
+- **Data Integrity:**
+  Each chunk will be individually encrypted and indexed with a cryptographic hash (e.g., SHA-256) to guarantee data integrity during storage and reassembly, supporting our goal of secure, reliable storage.
+
+## Consequences
+
+- **Advantages:**
+
+  - **Efficiency and Simplicity:** Fixed-size chunks simplify the overall design, making it easier to manage and duplicate data, thereby enhancing system performance.
+  - **Robust Testing:** The defined chunking approach easily integrates with our deterministic simulation framework, enabling rigorous testing under simulated adverse conditions.
+  - **Predictable Behavior:** Known chunk sizes facilitate efficient planning for both storage and network resource allocation.
+
+- **Trade-offs:**
+  - **Chunk Boundary Issues:** Fixed-size chunking may lead to some inefficiency when file updates occur at arbitrary boundaries, resulting in reprocessing of entire chunks even if only a small portion is modified.
+  - **Lack of Adaptive Chunking:** Without content-defined chunking, there is less optimization for files that have variable update patterns. However, for our use case, the simplicity and performance benefits of fixed-size chunking outweigh this limitation.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - FR-2: File Partitioning & Chunk Management – mandates that files be split into chunks to support deduplication and efficient updating.
+
+- **Non-Functional Requirements:**
+  - NFR-2: Performance & Responsiveness – the chunking process must maintain performance even under simulated disk and network latency conditions.
+  - NFR-3: Scalability & Capacity – efficient chunking supports overall system scalability by minimizing redundant data storage.
+  - NFR-6: Deployability & Maintainability – a simpler chunking strategy eases deployment and testing, including deterministic simulations of adverse conditions.
+
+## Conclusion
+
+The decision to use a fixed-size chunking strategy meets our system's need for efficient file partitioning that supports secure deduplication, versioning, and reliable reassembly. By limiting our approach to fixed-size chunking, we simplify the design and implementation, ensuring that our system remains robust and easy to test under a variety of conditions. This decision directly supports our functional and non-functional requirements and reinforces our commitment to building a secure, efficient, and maintainable file storage solution.

docs/adr/0005-automatic-replication-and-redundancy-strategy.md

@@ -0,0 +1,53 @@
+# ADR-0005: Automatic Replication and Redundancy Strategy
+
+## Status
+
+Proposed
+
+## Context
+
+Our file storage system must ensure high data availability and fault tolerance regardless of node, network, or disk failures. According to our functional requirements (FR-5), file chunks need to be replicated across nodes to maintain redundancy, and our non-functional requirements (NFR-2, NFR-3, NFR-6) demand robust behavior even under simulated adverse conditions such as network and disk failures. In order to keep the user experience simple and focused on privacy and control, the replication process should be fully automatic and transparent with no need for direct user configuration.
+
+## Decision
+
+We will implement an automatic replication strategy that is integrated within the system's internal management layers, making replication entirely transparent to the user and client application. The key decisions are as follows:
+
+- **Automatic Replication Management:**
+  The system will autonomously monitor node availability and data integrity. When a node failure or data inconsistency is detected, the system automatically replicates the affected file chunks to other nodes to maintain the desired redundancy level. This process is initiated by an internal monitoring mechanism and does not require any user intervention.
+
+- **Replication Under Deterministic Simulation:**
+  The replication operations will integrate with our deterministic simulation framework. This allows controlled testing of scenarios such as network latency and disk failures, ensuring that the automatic replication mechanism reliably maintains data availability under a variety of conditions (NFR-2, NFR-3, NFR-6).
+
+- **Data Integrity Verification:**
+  Each file chunk is associated with a cryptographic hash (e.g., SHA-256) to verify its integrity before and after replication. The verification process is automatically performed to ensure that replicated data remains consistent and unaltered in all nodes, contributing to the overall auditability and transparency of the system (FR-5, NFR-5).
+
+- **Transparent Operation:**
+  All replication processes are handled internally by the system. No replication policy or replication-related configuration is exposed to the user. This design choice reinforces our mission to provide a secure and user-centric solution where complex technical details—such as data redundancy strategies—are completely abstracted from the end user.
+
+## Consequences
+
+- **Advantages:**
+
+  - **Simplicity for Users:** The replication mechanism is entirely automatic, ensuring a frictionless user experience where data availability and redundancy are maintained without any manual configuration.
+  - **Enhanced Robustness:** The automatic system continuously monitors and resolves data integrity issues, ensuring minimal downtime and high availability of file data even under network and disk failure conditions.
+  - **Testability:** Integration with the deterministic simulation framework enables rigorous testing of replication processes under varied adverse conditions, ensuring reliable performance (NFR-2, NFR-3, NFR-6).
+
+- **Trade-offs:**
+  - **System Complexity:** Automating replication and integrating continuous monitoring require additional internal system complexity. However, this complexity is encapsulated so that users remain unaffected.
+  - **Resource Overhead:** While automatic replication may use additional resources for data duplication and monitoring, this overhead is justified by the benefits of improved resilience and data availability.
+  - **Development Effort:** Developing a robust, automatic replication mechanism demands a comprehensive design and extensive testing, but it aligns with our overall goals of fault tolerance and user-centric design.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - FR-5: Replication and Redundancy Management – Ensures file chunks are replicated across nodes for continuous data availability.
+
+- **Non-Functional Requirements:**
+  - NFR-2: Performance & Responsiveness – Replication must operate efficiently even under simulated adverse network and disk conditions.
+  - NFR-3: Scalability & Capacity – Automatic replication supports a growing number of files and nodes while sustaining performance.
+  - NFR-6: Deployability & Maintainability – The replication process, managed automatically by the system, should simplify deployment and support rigorous deterministic simulation testing.
+
+## Conclusion
+
+The chosen automatic replication and redundancy strategy meets our system’s needs by ensuring continuous data availability and integrity through an entirely transparent process. By automating replication management, integrating deterministic simulation testing, and verifying data integrity through cryptographic measures, the strategy fulfills our functional and non-functional requirements. This decision supports our mission by maintaining a resilient, secure storage environment while keeping the complexity hidden from the end user.

docs/adr/0006-manifest-design.md

@@ -0,0 +1,56 @@
+# ADR-0006: Manifest Design (Append-Only Log)
+
+## Status
+
+Proposed
+
+## Context
+
+Our system requires robust support for file versioning, change tracking, and auditability while maintaining data integrity. To achieve this, the manifest will serve as an append-only log that records every file operation. This design is essential for ensuring that changes to file data and metadata can be traced over time, which is core to our mission of transparency and user control.
+
+The manifest must be tamper-evident through cryptographic chaining of log entries and include periodic snapshots that summarize the state up to a given point. These snapshots help balance performance with auditability while keeping the historical log verifiable. Additionally, this design must support deterministic simulation testing to validate its behavior under conditions such as simulated network and disk delays or failures (NFR-2, NFR-6).
+
+## Decision
+
+We will implement the manifest as an append-only log structured with the following characteristics:
+
+- **Append-Only Structure and Cryptographic Chaining:**
+  Each log entry will record a discrete change (e.g., file addition, modification, or deletion) along with associated metadata (such as timestamps and file or chunk identifiers). Every entry will include a cryptographic hash that chains it to the previous entry, ensuring that any tampering is immediately detectable. This design supports auditability and transparency (FR-6, NFR-5).
+
+- **Snapshot Mechanism:**
+  To avoid performance degradation as the log grows, the manifest will incorporate periodic snapshots. A snapshot captures the current state of the manifest along with an aggregate hash referencing all previous entries. Future log entries will reference the most recent snapshot, enabling efficient verification of the complete history while pruning old log details if needed.
+
+- **Versioning and Conflict Management:**
+  The manifest will serve as the authoritative source for file versioning across multiple devices and nodes. Strategies to handle version merging and conflict resolution will be embedded within the manifest design, ensuring that concurrent updates can be reconciled consistently (FR-10).
+
+- **Deterministic Simulation Integration:**
+  The manifest design will allow for deterministic simulation of its append-only behavior. This means that in testing, we can simulate network and disk latency or failures and ensure that log entries are recorded, chained, and snapshotted correctly. This approach is critical to validating our performance and robustness requirements under adverse conditions (NFR-2, NFR-6).
+
+## Consequences
+
+- **Advantages:**
+
+  - **Auditability and Transparency:** The chained, append-only log design provides clear, verifiable evidence of every operation, which is central to our commitment to user control and data integrity (FR-6, NFR-5).
+  - **Resilience and Consistency:** With proper snapshots and cryptographic chaining, the manifest provides a resilient mechanism for tracking file changes, aiding in conflict detection and resolution.
+  - **Efficient Verification:** Periodic snapshots reduce the burden of verifying a long history of entries, helping maintain performance while still offering complete audit trails.
+
+- **Trade-offs:**
+  - **Increased Storage and Processing Overhead:** Maintaining and validating a constantly growing log and producing periodic snapshots will add processing and storage overhead. This trade-off is justified by the need for thorough auditability and data integrity.
+  - **Complexity in Conflict Management:** As multiple devices interact and update the manifest concurrently, embedded versioning and conflict resolution mechanisms add to system complexity. This complexity is managed internally and remains transparent to the user.
+  - **Testing Considerations:** To ensure the manifest performs correctly under simulated failure conditions, the deterministic testing framework must robustly emulate various latencies and failures, which increases testing effort.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - **Versioning and Change Tracking (FR-3 and FR-10):** The manifest is essential for recording file versions and managing conflict resolution across devices.
+  - **Auditability (FR-6):** The tamper-evident, chained log design provides verifiable audit trails for all file operations.
+
+- **Non-Functional Requirements:**
+  - **Performance & Responsiveness (NFR-2):** The use of snapshots and efficient verification techniques is critical to maintaining performance as the manifest grows.
+  - **Deployability & Maintainability (NFR-6):** The design supports a deterministic simulation framework that verifies behavior during network and disk latency or failure conditions.
+  - **Auditability & Transparency (NFR-5):** Ensuring that every log entry’s integrity can be cryptographically verified directly supports these principles.
+
+## Conclusion
+
+Implementing the manifest as an append-only, cryptographically chained log with periodic snapshots ensures a robust, transparent audit trail critical for file versioning and user-controlled data management. This design not only satisfies our functional and non-functional requirements but also aligns with our mission of providing a secure, resilient, and fully verifiable storage system.

docs/adr/0007-communication-protocols-and-api-design.md

@@ -0,0 +1,61 @@
+# ADR-0007: Communication Protocols and API Design
+
+## Status
+
+Proposed
+
+## Context
+
+Our system requires a robust communication layer to enable secure and efficient interactions between the client and the self-hosted server/P2P nodes. This includes both data transfers for file operations and control messages for tasks such as replication, manifest management, and access control. The communication protocols must support secure endpoints, transparent operations, and reliable performance even under simulated conditions (see FR-4, NFR-2, and NFR-6).
+
+In order to minimize external vulnerabilities, we strive to avoid third-party dependencies. As such, the client-to-server interactions will be implemented using a standard RESTful API, while inter-node communication will rely on a custom-built binary protocol. This custom solution mitigates supply chain risks by removing reliance on external libraries such as gRPC and maintains full control over our protocol’s security features.
+
+## Decision
+
+We will design the communication layer with two distinct components:
+
+- **RESTful API for Client-Server Interactions:**
+  The client application will interact with the server using a RESTful API. This API will handle file uploads/downloads, user commands, and management operations. REST was chosen for its simplicity, ease of use, and wide support while ensuring that sophisticated security measures (e.g., TLS and strong authentication) are applied. This design meets our requirements for transparency and secure user interactions (FR-4, FR-6, NFR-1).
+
+- **Custom Binary Protocol for Inter-Node Communication:**
+  For communication between nodes, we will develop a custom binary protocol. This protocol will be designed to:
+
+  - Ensure efficient, low-latency communication suitable for P2P replication and synchronization.
+  - Provide built-in security features, including mutual authentication and encrypted payloads, without relying on external dependencies.
+  - Integrate with our deterministic simulation framework to simulate network latency, failures, and other adverse conditions, ensuring the protocol meets performance and reliability benchmarks (NFR-2, NFR-6).
+
+- **Secure Communication Practices:**
+  Both the RESTful API and the custom binary protocol will enforce TLS for securing communications over the network. All endpoints will require strong authentication to prevent unauthorized access, aligning with our security expectations (NFR-1).
+
+- **Stateless and Versioned API Design:**
+  The RESTful API will be designed in a stateless manner to facilitate scalability and ease of load balancing. Additionally, the API will be versioned to support backward compatibility as the system evolves, ensuring stability and maintainability.
+
+## Consequences
+
+- **Advantages:**
+
+  - **Eliminated Third-Party Dependency:** The use of a custom binary protocol removes reliance on external libraries such as gRPC, reducing the risk of a supply chain attack.
+  - **Enhanced Security Control:** Developing our own inter-node protocol allows full control over security features tailored to our specific needs, ensuring robust encryption, mutual authentication, and auditability (NFR-1, FR-6).
+  - **Performance and Reliability:** With a custom solution targeted for inter-node communication, we can optimize performance for low-latency and high-throughput scenarios. Integration with deterministic simulation ensures adherence to performance and reliability targets under various network conditions (NFR-2, NFR-6).
+  - **Simple Client Interaction:** Using a RESTful API for client-server interactions maintains simplicity for the end user, making the system easier to use and maintain.
+
+- **Trade-offs:**
+  - **Development Complexity:** Crafting a custom binary protocol introduces additional development effort compared to adopting an existing solution. However, this is outweighed by the increased control over security and reduced external risk.
+  - **Maintenance Overhead:** Custom protocols require ongoing maintenance and robust testing to remain secure and performant across various network conditions.
+  - **Learning Curve:** Developers will need to familiarize themselves with the custom protocol specifications, even though the RESTful API helps to mitigate user-facing complexity.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - FR-4: Peer-to-Peer Node Interaction – The system’s inter-node communication is essential for secure file sharing and replication.
+  - FR-6: User Control and Access Management – Secure communication is critical to ensuring transparent and audit-friendly operations.
+
+- **Non-Functional Requirements:**
+  - NFR-1: Security & Privacy – Enforcing TLS and mutual authentication as part of both the RESTful API and custom binary protocol meets our strong security requirements.
+  - NFR-2: Performance & Responsiveness – The custom binary protocol is designed to operate efficiently even under simulated network latency.
+  - NFR-6: Deployability & Maintainability – A stateless, versioned API paired with a tailored inter-node protocol facilitates straightforward deployment and rigorous deterministic testing.
+
+## Conclusion
+
+By implementing a RESTful API for client-server interactions and a custom binary protocol for inter-node communications, our design not only meets our functional and non-functional requirements but also mitigates external dependencies, reducing supply chain risk. This approach preserves security, performance, and transparency in all communication layers, ensuring a resilient system that remains fully within our control.

docs/adr/0008-p2p-node-connectivity-and-security.md

@@ -0,0 +1,56 @@
+# ADR-0008: P2P Node Connectivity and Security
+
+## Status
+
+Proposed
+
+## Context
+
+Our system facilitates secure file sharing and replication across self-hosted nodes. However, rather than relying on automatic node discovery, each user will explicitly configure which nodes to connect to. In our design, a user's network is built organically—a personal web of trusted nodes—ensuring that connectivity remains fully under the control of the user. This design supports our goal of maintaining strict privacy and security (FR-4, FR-5, NFR-1) while also accommodating deterministic testing of network conditions (NFR-2, NFR-6).
+
+User-configured connections mean that nodes will only establish connectivity with explicitly defined peers rather than automatically searching for or connecting to unknown nodes. This approach is in line with our mission to provide full user control over their digital networks and minimizes external dependencies and unexpected interactions.
+
+## Decision
+
+We will implement a P2P node connectivity framework that emphasizes explicit user control and secure, manually configured connections with the following key features:
+
+- **Explicit Node Configuration:**
+  The user will be responsible for configuring the list of nodes with which to connect. No automatic node discovery or unsolicited connection attempts will occur. Users can build and maintain personalized networks, similar to managing a social graph, where connections reflect direct trust and controlled relationships.
+
+- **Mutual Authentication Using IP Address and Public Keys:**
+  Connections between nodes will be established only after mutual authentication using predefined IP addresses and associated public keys. This ensures that nodes connect only to explicitly authorized peers, safeguarding against unauthorized access and bolstering our overall security posture (FR-4, NFR-1).
+
+- **Encrypted Communication Channels:**
+  All inter-node communications will be secured via encryption. Our custom binary protocol, used exclusively for inter-node messaging, will incorporate strong encryption practices similar to TLS, ensuring data confidentiality and integrity across all explicitly configured connections.
+
+- **Deterministic Simulation Support:**
+  The connectivity framework will be designed to support deterministic simulation testing, allowing us to validate performance and reliability under various simulated network conditions—including latency, intermittent connectivity, and failure scenarios—while preserving the explicit configuration model (NFR-2, NFR-6).
+
+## Consequences
+
+- **Advantages:**
+
+  - **Enhanced User Control:** Users maintain complete authority over which nodes are part of their network, reinforcing the principle of self-ownership and ensuring that network growth is deliberate and trusted.
+  - **Increased Security:** By connecting only to user-approved nodes, the potential attack surface is reduced. Mutual authentication using IP addresses and public keys further ensures that only verified entities participate in the user's network (NFR-1).
+  - **Predictable Behavior:** A manually configured network is easier to simulate and test deterministically, as the connection topology is fixed and known, enhancing reliability and performance under controlled test conditions (NFR-2, NFR-6).
+
+- **Trade-offs:**
+  - **Limited Network Growth:** Without automatic discovery, users must manually add each node, potentially limiting the organic expansion of their network. However, this aligns with our emphasis on explicit trust relationships and user control.
+  - **Increased Configuration Effort:** Users or administrators are responsible for setting up and maintaining node configurations, which may increase the initial setup effort. This complexity is acceptable given the benefits of enhanced privacy and security.
+  - **Maintenance Overhead:** Changes to network configurations must be managed manually, requiring additional processes for updating and revoking node connections as trust relationships evolve.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - FR-4: Peer-to-Peer Node Interaction – Connectivity is critically important for file sharing and replication, and connecting only to explicitly configured nodes enhances trusted operations.
+  - FR-5: Replication and Redundancy Management – Reliance on user-configured nodes ensures replication occurs only among trusted peers, maintaining data integrity and availability.
+
+- **Non-Functional Requirements:**
+  - NFR-1: Security & Privacy – Explicit node configuration, mutual authentication, and encrypted communications all contribute to a secure, user-controlled environment.
+  - NFR-2: Performance & Responsiveness – Fixed, user-defined connectivity simplifies deterministic simulation testing, ensuring that the system performs reliably under varying conditions.
+  - NFR-6: Deployability & Maintainability – The lack of automatic node discovery reduces dependency on external systems and minimizes unexpected dynamic behavior, aiding in straightforward deployment and predictable maintenance.
+
+## Conclusion
+
+The decision to use explicit, user-configured node connectivity supports our secure, privacy-focused design by ensuring that only trusted nodes participate in a user's network. By employing mutual authentication, encrypted communication channels, and integrating deterministic simulation testing into our custom protocol, this approach meets our functional and non-functional requirements while keeping network growth and connection management fully under the user's control.

docs/adr/0009-trust-access-control-and-replication-policies.md

@@ -0,0 +1,57 @@
+# ADR-0009: Trust, Access Control, and Replication Policies
+
+## Status
+
+Proposed
+
+## Context
+
+Our system’s secure and resilient operation depends on how nodes establish trust, manage access, and enforce replication policies. Since users explicitly configure their node networks, trust is built on manually approved connections using public key credentials, ensuring that data sharing and replication occur strictly among verified peers (FR-4, FR-5). Further, robust access control mechanisms are essential to prevent unauthorized access while ensuring that all replication actions remain transparent and auditable (FR-6, NFR-1).
+
+Given our focus on privacy and user control, the system must enforce policies that guarantee replication happens only within a set of trusted nodes and that access privileges can be granted or revoked as needed. These policies need to integrate seamlessly with our deterministic simulation testing framework to validate behavior under various adverse conditions (NFR-2, NFR-6).
+
+## Decision
+
+We will implement trust, access control, and replication policies with the following design principles:
+
+- **Explicit Trust Management:**
+  Trust is established based on user-approved connections. Each node connection is verified through public key authentication and explicit configuration by the user. No automatic trust propagation is allowed, ensuring that all participating nodes have been deliberately authorized.
+
+- **Access Control Mechanisms:**
+  The system will enforce access controls using cryptographic credentials. Each access request to file data or node resources must be signed and verified. Audit logs will capture every access grant or revocation, enhancing transparency and security in line with our auditability requirements (FR-6, NFR-1).
+
+- **Automated Replication Within Trusted Networks:**
+  Replication policies are enforced automatically within the user-configured network. The system continuously monitors the availability and integrity of file chunks, replicating data only among nodes that have been explicitly authorized by the user. This automated process is transparent to the user, ensuring that data remains available and consistent without manual intervention.
+
+- **Policy Verification and Simulation:**
+  The entire trust and access control mechanism is designed to be fully testable within our deterministic simulation framework. This will allow us to validate the performance and reliability of trust validation, access control, and replication policies under simulated network and disk failure conditions (NFR-2, NFR-6).
+
+## Consequences
+
+- **Advantages:**
+
+  - **Enhanced Security:** Explicit trust management and strong cryptographic access controls ensure that only authorized nodes participate in data replication and file sharing, greatly reducing the risk of unauthorized access (FR-4, NFR-1).
+  - **Transparent Operations:** All access and replication events are logged and auditable, reinforcing our commitment to transparency and enabling clear tracking of every file operation (FR-6, NFR-5).
+  - **User-Centric Control:** By centralizing trust management in the hands of the user, the system upholds the principle of digital self-ownership and ensures that replication policies operate strictly within user-defined networks.
+
+- **Trade-offs:**
+  - **Increased Configuration and Management Overhead:** Relying on explicit, user-managed trust relationships demands careful configuration and may result in higher initial setup efforts. However, this overhead is justified by the resulting security and privacy benefits.
+  - **Complexity in Policy Enforcement:** Integrating automated replication with strict access controls adds layers to the system’s complexity. These must be rigorously tested to maintain performance under various conditions.
+  - **Testing Requirements:** Ensuring that trust and access control mechanisms behave as intended under simulated adverse conditions requires a robust deterministic simulation framework, increasing the scope of testing efforts.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - FR-4: Peer-to-Peer Node Interaction – Trust management ensures that nodes only interact when explicitly authorized.
+  - FR-5: Replication and Redundancy Management – Automated replication is restricted to user-approved nodes for data consistency.
+  - FR-6: User Control and Access Management – Strong access control is vital for ensuring that all operations are transparent and auditable.
+
+- **Non-Functional Requirements:**
+  - NFR-1: Security & Privacy – Cryptographic access controls and explicit trust verification directly support our stringent security and privacy mandates.
+  - NFR-2: Performance & Responsiveness – The design of these policies ensures that, even under simulated adverse conditions, the system maintains timely responses through efficient policy enforcement.
+  - NFR-6: Deployability & Maintainability – The mechanisms are designed to be verifiable through deterministic simulations, aiding in predictable deployment and straightforward maintenance.
+
+## Conclusion
+
+Implementing stringent trust, access control, and replication policies ensures that our file storage system operates securely and transparently. By requiring explicit, user-approved node connections and enforcing robust cryptographic access controls, the system guarantees that replication occurs only among trusted nodes. This design not only meets our functional and non-functional requirements but also embodies our overarching mission of providing secure, user-controlled digital storage.

docs/adr/0010-versioning-and-conflict-resolution-strategy.md

@@ -0,0 +1,60 @@
+# ADR-0010: Versioning and Conflict Resolution Strategy
+
+## Status
+
+Proposed
+
+## Context
+
+Our system must maintain an accurate, verifiable history of all file changes and support seamless multi-device collaboration. This necessitates a robust versioning mechanism that can track changes over time and resolve conflicts that occur when multiple devices update the same file concurrently. Our functional requirements (FR-3, FR-10) call for maintaining multiple versions of files with the ability to merge manifests from various devices, while our non-functional requirements (NFR-2, NFR-6) demand that such operations perform reliably even under adverse conditions.
+
+Versioning is not only essential for maintaining historical records and supporting file restoration but also plays a key role in conflict resolution. When simultaneous updates occur, the system must efficiently determine the correct ordering of operations to reconcile differences, ensuring data integrity. This design must support deterministic simulation testing to verify conflict resolution under simulated network delays and failures.
+
+## Decision
+
+We will implement a versioning and conflict resolution strategy that consists of the following elements:
+
+- **Append-Only Manifest for Versioning:**
+  Each file operation (create, update, or delete) is recorded as an entry in an immutable, append-only manifest. This manifest forms the backbone of our versioning system, ensuring that every change is verifiable and auditable (see ADR-0006). The manifest entries are cryptographically chained to prevent tampering.
+
+- **Timestamps and Order Identification:**
+  Each log entry will be tagged with a logical timestamp. We propose using Lamport timestamps as a simple and effective mechanism to order events across multiple devices. This method enables us to determine a consistent ordering of updates even in a distributed system where events are generated concurrently (FR-10).
+
+- **Conflict Detection and Automated Resolution:**
+  When simultaneous updates are detected—identifiable through overlapping timestamps or non-linear history in the manifest—the system will apply a deterministic conflict resolution algorithm. This algorithm may involve choosing the update with the lower Lamport timestamp or, in cases where timestamps are equal, a predefined rule such as favoring the update from a primary device as configured by the user. This process ensures that conflicts are resolved consistently while still allowing for manual intervention if needed.
+
+- **Merge and Reconciliation Process:**
+  Before merging manifest logs from multiple devices, the system will reconcile differences by comparing the chronological order of operations based on logical timestamps. Any conflicts will be flagged, and the resolution algorithm will be applied to merge changes without data loss. The system logs every resolution decision, supporting auditability and transparency (FR-3, FR-6).
+
+- **Integration with Deterministic Simulation:**
+  The versioning and conflict resolution framework will be fully integrated into our deterministic simulation environment. This integration ensures that the process can be rigorously tested and verified under simulated conditions such as network latency and disk failures, meeting our performance (NFR-2) and maintainability (NFR-6) requirements.
+
+## Consequences
+
+- **Advantages:**
+
+  - **Strong Auditability and Transparency:** Using an append-only manifest with cryptographic chaining guarantees a tamper-evident history, fulfilling our auditability requirements.
+  - **Deterministic Conflict Resolution:** Leveraging Lamport timestamps and a deterministic resolution algorithm ensures that conflicts are resolved consistently across devices, contributing to data integrity.
+  - **Efficient Multi-Device Synchronization:** The clear ordering of operations facilitates smooth merging of updates from multiple sources, reducing the possibility of data loss or inconsistency.
+  - **Robust Testing:** Integration with our deterministic simulation framework ensures that the versioning and conflict resolution system is validated under varied and adverse real-world conditions.
+
+- **Trade-offs:**
+  - **Computational Overhead:** The use of logical timestamps and the maintenance of an append-only log may introduce additional computational load, but this is justified by the benefits of improved consistency and auditability.
+  - **Complexity in Conflict Resolution:** Implementing a deterministic algorithm that handles all possible conflict scenarios across a distributed system increases the complexity of the system. However, this is mitigated by a clear set of rules and transparency in the resolution process.
+  - **Edge Case Handling:** Certain conflict situations may require manual intervention when automatic resolution is ambiguous, necessitating additional user interface considerations.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - FR-3: Versioning and Change Tracking – The manifest-based versioning directly records file evolution.
+  - FR-10: Versioning and Conflict Resolution – The strategy ensures that concurrent updates are reconciled in a consistent manner.
+  - FR-6: Auditability – All operations, including conflict resolutions, are logged and verifiable.
+
+- **Non-Functional Requirements:**
+  - NFR-2: Performance & Responsiveness – The system must efficiently process and merge updates even under simulated adverse conditions.
+  - NFR-6: Deployability & Maintainability – The deterministic nature of our versioning and conflict resolution supports thorough testing and ease of maintenance.
+
+## Conclusion
+
+The proposed versioning and conflict resolution strategy meets our requirements by combining an immutable, cryptographically chained manifest with logical timestamps (via Lamport timestamps) to maintain a consistent order of operations. This approach facilitates automated resolution of conflicts in a distributed multi-device environment while supporting full auditability and transparency. By integrating with our deterministic simulation framework, our design ensures reliable performance and data integrity even under adverse conditions, aligning with both our functional and non-functional requirements.

docs/adr/0011-deployment-and-containerization-strategy.md

@@ -0,0 +1,58 @@
+# ADR-0011: Deployment and Containerization Strategy
+
+## Status
+
+Proposed
+
+## Context
+
+Our system is designed for self-hosting and must be deployable in diverse environments with minimal configuration effort. Users expect an easy installation process, whether on a local server or in a containerized environment. In addition, our non-functional requirements (NFR-6) emphasize deployability and maintainability, making it critical that we support robust containerization and configuration management strategies.
+
+This strategy is fundamental to ensuring that our privacy-focused application remains accessible and straightforward to install while maintaining consistent behavior across various operating environments. It also aids in deterministic testing by providing uniform deployment scenarios that simulate real-world conditions such as network and disk performance.
+
+## Decision
+
+We will adopt a containerized deployment strategy with the following key elements:
+
+- **Containerization with Docker:**
+  The application will be packaged as Docker containers to provide a consistent runtime environment across different host systems. This approach simplifies deployment, reduces configuration inconsistencies, and enhances portability. Users will be able to deploy our system easily on any platform that supports Docker.
+
+- **Support for Orchestration Platforms:**
+  Although the primary deployment target is a self-hosted environment, the containerized design will also enable deployments using orchestration platforms such as Kubernetes. This support allows for scalable, resilient deployments with automated recovery and load balancing, ensuring that the system meets performance targets even under simulated adverse conditions (NFR-2, NFR-6).
+
+- **Configuration and Secrets Management:**
+  Deployment will incorporate predefined configuration templates and support integration with secure secrets management systems. This not only simplifies the setup process for users but also ensures that sensitive deployment parameters are handled securely in line with our security requirements (NFR-1).
+
+- **Self-Hosting Focus:**
+  The deployment strategy is tailored for self-hosting scenarios, ensuring that users can manage and maintain their own servers without needing to depend on centralized services. This supports our commitment to user control and full transparency while maintaining robust security and privacy standards.
+
+- **Deterministic Testing Environment:**
+  Our containerized deployment design enables the creation of deterministic simulation environments. Through container orchestration, we can simulate network and disk latency or failure scenarios consistently, aiding in rigorous testing and validation of system performance (NFR-2, NFR-6).
+
+## Consequences
+
+- **Advantages:**
+
+  - **Consistency and Portability:** Containerization ensures that the application runs uniformly across various platforms, reducing deployment errors and configuration drift.
+  - **Ease of Deployment:** Users can deploy our system with minimal manual configuration, making it accessible to a wider audience of self-hosting enthusiasts.
+  - **Scalability:** Integration with orchestration platforms like Kubernetes enables smooth scaling and high availability, supporting robust performance under challenging conditions.
+  - **Improved Security and Configuration Management:** Secure handling of configuration and secrets reduces risks and aligns with our strict security requirements (NFR-1).
+
+- **Trade-offs:**
+  - **Increased Setup Complexity for Orchestration:** While Docker deployment is straightforward, leveraging orchestration platforms may introduce additional complexity that advanced users must manage. However, this is an optional pathway for those requiring enterprise-scale deployments.
+  - **Maintenance Overhead:** Keeping container images updated and managing deployment scripts requires continuous effort, though this trade-off is balanced by the benefits of streamlined, reproducible deployments.
+  - **Resource Overhead:** Containerized environments can add some overhead in terms of resource utilization; however, the benefits of consistency and ease-of-deployment justify this cost.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - Not directly functional, but essential for ensuring that users can operate the system reliably in varied environments.
+
+- **Non-Functional Requirements:**
+  - NFR-6: Deployability & Maintainability – The containerization strategy meets the need for easy, consistent deployment and supports deterministic simulation testing.
+  - NFR-2: Performance & Responsiveness – Standardized environments help ensure that performance benchmarks are achieved even under simulated adverse conditions.
+
+## Conclusion
+
+The proposed containerization and deployment strategy enables consistent, straightforward self-hosted installations while supporting scalable and secure deployment options through orchestration platforms like Kubernetes. By adopting Docker-based containerization and focusing on robust configuration and secrets management, this approach fully meets our non-functional requirements for deployability, maintainability, and performance. This decision reinforces our commitment to providing an accessible and resilient system that retains user control and transparency across diverse deployment environments.

docs/adr/0012-storage-backends-and-data-persistence-strategy.md

@@ -0,0 +1,71 @@
+# ADR-0012: Storage Backends and Data Persistence Strategy
+
+## Status
+
+Proposed
+
+## Context
+
+Our self-hosted file storage system needs to ensure that encrypted file chunks are reliably stored and remain available even in the event of hardware, disk, or network failures. To meet this challenge, our system must use a storage backend and data persistence strategy that supports both local storage and distributed redundancy. This is essential for:
+
+- **File Management and Deduplication (FR-2):** Ensuring that file chunks are correctly managed and can be efficiently retrieved using content addressable methods (SHA-256 indexing).
+- **Replication and Redundancy (FR-5):** Guaranteeing that data remains available across nodes even when individual nodes fail.
+
+Furthermore, our strategy must align with non-functional requirements such as:
+
+- **Security & Privacy (NFR-1):** Data must be encrypted and integrity-verified to prevent unauthorized modifications.
+- **Performance & Responsiveness (NFR-2):** The system ought to remain fast and efficient, even when simulating adverse disk or network conditions.
+- **Scalability (NFR-3):** The storage strategy must support system growth in a modular fashion.
+- **Deployability & Maintainability (NFR-6):** The design should facilitate deterministic testing and easy maintenance.
+
+## Decision
+
+We will implement a hybrid storage backend strategy that includes the following elements:
+
+- **Local Filesystem Storage:**
+  Each node will manage its file storage on a local filesystem. Key features include:
+
+  - **Write-Ahead Logging:** Operations will be logged prior to execution to support crash recovery.
+  - **Content Addressable Storage:** File chunks are indexed using cryptographic hashes (e.g., SHA-256) to ensure integrity, prevent duplication, and streamline retrieval.
+
+- **Distributed Redundancy Through Replication:**
+
+  - **Automatic Replication:** As outlined in ADR-0005, file chunks will be replicated to multiple nodes automatically. This replication ensures that data is available even if one or more nodes are offline.
+  - **Seamless Recovery:** In the event of a node failure, redundant copies on other nodes will be used to restore data, thus maintaining high availability.
+
+- **Data Integrity Verification:**
+  Each file chunk is associated with a cryptographic hash to verify its integrity regularly. This continuous verification helps detect data corruption or discrepancies, ensuring that the system fulfills its security (NFR-1) and reliability (FR-5) requirements.
+
+- **Deterministic Simulation Support:**
+  The storage backend will integrate with our deterministic simulation testing framework. This integration allows us to simulate disk latency, disk failures, and other adverse conditions, ensuring that the system meets the performance (NFR-2) and maintainability (NFR-6) targets under varied scenarios.
+
+## Consequences
+
+- **Advantages:**
+
+  - **High Availability and Resilience:** Distributed replication ensures that data remains accessible despite individual node failures.
+  - **Strong Data Integrity:** The use of content addressable storage with SHA-256 indexing and periodic integrity verification maintains data correctness.
+  - **Scalability:** Combining local storage with distributed redundancy allows the system to efficiently handle increased loads as the network of nodes grows.
+  - **Robust Testing:** Integration with deterministic simulation provides a structured environment to test and validate data persistence and recovery under adverse conditions.
+
+- **Trade-offs:**
+  - **System Complexity:** Managing both local storage and distributed replication adds architectural complexity, though this is mitigated by clear logging and automated recovery processes.
+  - **Resource Overhead:** Redundant storage increases disk space and network utilization, a necessary trade-off for improved resilience and availability.
+  - **Maintenance Effort:** Automated integrity checks and replication processes require ongoing maintenance and monitoring but are essential for long-term reliability.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - **FR-2:** File Partitioning & Chunk Management – Local storage and content addressable indexing support efficient file operations.
+  - **FR-5:** Replication and Redundancy Management – Distributed replication ensures that data remains available across nodes.
+
+- **Non-Functional Requirements:**
+  - **NFR-1:** Security & Privacy – Encryption and hash-based integrity checks safeguard user data.
+  - **NFR-2:** Performance & Responsiveness – The design supports efficient data access even under simulated adverse conditions.
+  - **NFR-3:** Scalability & Capacity – The strategy is built to accommodate growth in data volume and the number of nodes.
+  - **NFR-6:** Deployability & Maintainability – Integration with deterministic simulation ensures predictable behavior and easier maintenance.
+
+## Conclusion
+
+The proposed storage backend and data persistence strategy, which combines local filesystem storage with distributed replication and robust integrity verification, meets the system’s needs for high availability, security, and scalability. By indexing file chunks using cryptographic hashes and ensuring fault tolerance through automatic replication, this approach aligns with our functional and non-functional requirements. The integration of deterministic simulation testing further validates performance and recovery under varied conditions, ensuring a resilient and maintainable system.

docs/adr/0013-performance-optimizations-and-resource-management.md

@@ -0,0 +1,78 @@
+# ADR-0013: Performance Optimizations and Resource Management
+
+## Status
+
+Proposed
+
+## Context
+
+Our system is designed to handle extensive file operations, including encryption, decryption, file partitioning, replication, and versioning across a distributed network of nodes. As such, ensuring that the system performs efficiently and manages resources optimally is critical. This ADR addresses methods to enhance throughput, minimize latency, and manage computational and storage resources effectively.
+
+Performance and resource management are especially important given:
+
+- **Functional requirements (FR-2, FR-5):** which demand efficient partitioning, deduplication, and replication of file chunks across nodes.
+- **Non-functional requirements (NFR-2, NFR-3, NFR-6):** which stipulate that operations should meet performance benchmarks even when testing under simulated adverse network and disk conditions.
+
+The system should be capable of processing file operations with minimal delay while scaling to manage growing data volumes and an increasing number of nodes. This ADR also needs to consider the impact of encryption and audit logging (FR-1, FR-6) on overall resource utilization.
+
+## Decision
+
+We will incorporate several key strategies to optimize performance and manage resources effectively:
+
+- **Efficient Algorithms and Optimized Code Paths:**
+  Critical operations such as file chunking, encryption/decryption, and hashing (using SHA-256) will be implemented with optimized algorithms to minimize computational overhead. The design will include profiling and benchmarking during development to identify and address performance bottlenecks.
+
+  - Use hardware acceleration (where available) for cryptographic functions.
+  - Optimize file I/O through buffer management and asynchronous processing where appropriate.
+
+- **Caching and In-Memory Data Structures:**
+  Frequently accessed data (e.g., recently used file chunks, manifest metadata) will be cached in memory to reduce repeated disk I/O overhead.
+
+  - Implement lightweight in-memory caches to store hash values and intermediate processing results.
+  - Ensure the cache size is configurable to balance memory usage and performance.
+
+- **Asynchronous and Parallel Processing:**
+  Operations that are independent, such as processing individual file chunks or replicating file data across nodes, will be executed asynchronously or in parallel.
+
+  - Leverage concurrent processing for encryption, hashing, and network data transfers.
+  - Use non-blocking I/O patterns to improve responsiveness, especially under simulated high-load conditions.
+
+- **Resource Utilization Monitoring:**
+  The system will include internal monitoring to track CPU, memory, disk I/O, and network usage.
+
+  - Build diagnostic tools that integrate with our deterministic simulation framework to allow testing under varied load conditions, as stipulated in NFR-2 and NFR-6.
+  - Use collected data to dynamically adjust cache sizes or processing concurrency levels.
+
+- **Optimized Replication and Audit Logging:**
+  To minimize the performance impact of data replication and audit logging:
+  - Use batched replication operations where possible to reduce network overhead.
+  - Implement efficient append-only logging with periodic snapshotting to limit the overhead of continuously processing long audit trails (refer to ADR-0006).
+
+## Consequences
+
+- **Advantages:**
+
+  - **Improved Throughput & Responsiveness:** Optimized algorithms, caching, and asynchronous processing will reduce latency, ensuring that file operations meet the speed and performance benchmarks specified in NFR-2.
+  - **Scalability:** Effective resource management will support the system's ability to scale horizontally as the number of nodes and data volume increases, aligning with our NFR-3 requirements.
+  - **Predictable Performance:** Integrating performance monitoring with deterministic testing provides a means to consistently validate that resource utilization remains within acceptable bounds under varied simulated conditions (NFR-6).
+
+- **Trade-offs:**
+  - **Implementation Complexity:** Introducing caching, asynchronous processing, and parallelism can significantly increase system complexity. However, modular design techniques and careful documentation will mitigate long-term maintenance challenges.
+  - **Resource Overhead:** Proactive resource monitoring may add some overhead; nonetheless, this cost is justified by the benefits of proactive performance optimization.
+  - **Testing and Tuning:** Optimizing performance requires ongoing profiling and tuning, which may extend initial development time but ensures that the system maintains high performance as it scales.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - FR-2: Efficient file partitioning and chunk management require that file operations are processed quickly and reliably.
+  - FR-5: Replication and redundancy management must occur efficiently to ensure that data remains available even under adverse conditions.
+
+- **Non-Functional Requirements:**
+  - NFR-2: Performance & Responsiveness – The system must complete operations within acceptable time frames under simulated network and disk latency.
+  - NFR-3: Scalability & Capacity – The system should handle a growing number of files, versions, and nodes while maintaining high performance.
+  - NFR-6: Deployability & Maintainability – Deterministic simulation testing of resource utilization ensures consistent performance and ease of maintenance.
+
+## Conclusion
+
+The proposed performance optimizations and resource management strategy is designed to ensure that the system remains responsive and scalable even under heavy load or simulated network and disk adversities. By employing efficient algorithms, caching, asynchronous processing, and proactive resource monitoring, this strategy fulfills our functional and non-functional requirements while supporting a robust, high-performance file storage system.

docs/adr/0014-auditability-logging-and-security-monitoring.md

@@ -0,0 +1,63 @@
+# ADR-0014: Auditability, Logging, and Security Monitoring
+
+## Status
+
+Proposed
+
+## Context
+
+Our system is built on the core principles of user control, transparency, and data integrity. For users to trust that their data is secure and managed correctly, all file operations and node interactions must be auditable and subject to thorough logging. In addition, security monitoring is paramount to quickly detect anomalies, tampering, or breaches. This ADR addresses the need to design an audit trail and security monitoring framework that satisfies the following:
+
+- **Functional requirements (FR-6):** Every file operation, version change, and node interaction must be logged in a tamper-evident manner to support auditability and transparency.
+- **Non-functional requirements (NFR-1, NFR-2, NFR-5, NFR-6):** Security and privacy mandates that logs must be secured and verified, while performance and maintainability indicators require that logging mechanisms impose minimal overhead. The framework must also integrate with deterministic simulation testing so that logging and security events can be consistently reproduced under simulated adverse conditions.
+
+The logging framework must be designed not only to record normal operations but to flag and alert on aberrant activities, ensuring robust security monitoring across all system components.
+
+## Decision
+
+We will implement a comprehensive logging and security monitoring solution with the following components:
+
+- **Append-Only Logging:**
+  All system operations—including file uploads/downloads, manifest updates, conflict resolutions, and node interactions—will be recorded in an immutable, append-only log. This log will be cryptographically chained (see ADR-0006) to help ensure tamper-evidence and to maintain a verifiable history of system events.
+
+- **Granular Audit Trails:**
+  Logs will capture key metadata for each operation, including timestamps, device identifiers, user and node credentials, and the outcome of operations. These detailed audit trails provide full transparency and support retrospective security investigations (FR-6).
+
+- **Real-Time Security Monitoring:**
+  A monitoring component will actively analyze log entries and system events in real time to detect anomalous behavior, potential tampering, or security breaches. Automated alerting mechanisms will notify administrators when suspicious activity is detected, enabling rapid response in line with our security and privacy requirements (NFR-1).
+
+- **Integration with Deterministic Simulation:**
+  The logging and monitoring framework will support deterministic simulation testing. This capability allows us to replay logged events under simulated network or disk failure conditions, ensuring that the logging system consistently captures and reports events, even under adverse operating conditions (NFR-2, NFR-6).
+
+- **Secure Log Storage:**
+  Logs will be stored in a secure manner using encryption at rest and in transit. Access controls will ensure that log data is only available to authorized entities, further protecting sensitive audit information in compliance with our privacy requirements.
+
+## Consequences
+
+- **Advantages:**
+
+  - **Enhanced Transparency and Trust:** A robust, tamper-evident logging solution supports full auditability, empowering users to verify that all operations are performed in accord with security policies.
+  - **Improved Security Posture:** Real-time monitoring and alerting enable quick detection and remediation of anomalies or intrusions, reinforcing our overall security and privacy framework (NFR-1).
+  - **Consistent Auditing Across Conditions:** Integration with deterministic simulation testing ensures that logging remains reliable and verifiable even under simulated adverse network or disk conditions (NFR-2, NFR-6).
+  - **Regulatory Compliance:** Detailed audit trails facilitate compliance with security standards and legal requirements for data handling and breach notifications.
+
+- **Trade-offs:**
+  - **Performance Impact:** Comprehensive logging and real-time monitoring can add overhead in terms of processing and storage. However, careful optimization and the use of asynchronous logging techniques will help mitigate this impact.
+  - **Complexity in Log Management:** Ensuring that log data remains secure, accessible, and tamper-evident over time requires robust management strategies and additional development effort.
+  - **Data Volume:** Detailed logs can grow rapidly in size, which necessitates strategies for log rotation, archival, and secure deletion to avoid storage bloat.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - FR-6: Auditability and Access Management require that every operation is logged in a verifiable manner.
+
+- **Non-Functional Requirements:**
+  - NFR-1: Security & Privacy – Secure, encrypted log storage and access controls are essential components.
+  - NFR-2: Performance & Responsiveness – The logging system must operate efficiently even under adverse conditions.
+  - NFR-5: Auditability & Transparency – The design directly supports complete audit trails and tamper-evident logging.
+  - NFR-6: Deployability & Maintainability – Integration with deterministic simulation testing ensures that the logging and monitoring mechanisms are both testable and maintainable.
+
+## Conclusion
+
+Implementing a comprehensive auditability, logging, and security monitoring framework is essential for building trust in our self-hosted file storage system. By adopting an immutable, chained logging strategy combined with real-time security monitoring and integration with deterministic simulation testing, we ensure that all actions on the system are transparent and verifiable. This approach aligns with our core functional and non-functional requirements, ensuring that our system remains secure, reliable, and fully under user control.

docs/adr/0015-user-interface-and-multi-device-synchronization.md

@@ -0,0 +1,86 @@
+# ADR-0015: User Interface and Multi-Device Synchronization
+
+## Status
+
+Proposed
+
+## Context
+
+Our system is designed to empower users with complete control over their data while ensuring that file operations are consistent and synchronized across multiple devices and nodes. Users must be able to interact with the system through an intuitive interface that provides clear feedback on file operations, synchronization status, and network connectivity. Further, when users operate on multiple devices, the system must reliably merge changes and resolve conflicts to maintain data integrity (see FR-15 and FR-10).
+
+Key aspects of this ADR include:
+
+- **User Interface:**
+  The initial interface will be a command-line interface (CLI) that offers direct control and feedback over file operations, audit logs, and synchronization status. Future plans include a graphical web-based interface to enhance usability.
+
+- **Multi-Device Synchronization:**
+  The system must support seamless synchronization across devices. This involves merging manifest entries, tracking versions, and resolving conflicts in a deterministic manner. Changes initiated from one device should be transparently propagated to others while preserving auditability (FR-3, FR-10).
+
+- **Deterministic Testing:**
+  Multi-device synchronization scenarios, including scenarios with simulated network latency or failures, must be deterministically testable. This ensures that synchronization remains robust under varied conditions, meeting our performance and maintainability goals (NFR-2, NFR-6).
+
+## Decision
+
+We will implement a user interface and multi-device synchronization strategy that incorporates the following components:
+
+- **Intuitive User Interface (UI):**
+
+  - **Command-Line Interface (CLI):**
+    The initial UI will be CLI-based, offering clear, concise commands for file operations, status queries, and viewing audit logs. The CLI will provide immediate feedback to users regarding synchronization events and potential conflicts.
+
+  - **Future Web Interface:**
+    While the CLI is the primary interface in the initial release, the architecture will support a future web-based interface for users who prefer a graphical approach. This web interface will offer similar functionalities with additional visual aids for system status and history.
+
+- **Robust Multi-Device Synchronization:**
+
+  - **Manifest-Based Synchronization:**
+    Synchronization across devices is primarily achieved through a shared, immutable manifest. Each device maintains its own copy of the manifest, which is periodically merged with those from other devices. Conflict resolution and version tracking (as detailed in ADR-0010) ensure that changes are integrated consistently.
+
+  - **Deterministic Conflict Resolution:**
+    The synchronization mechanism relies on logical timestamps (e.g., Lamport timestamps) to order events, ensuring that conflicts are resolved in a consistent and predictable manner across devices.
+
+  - **Status and Feedback:**
+    The UI will display real-time synchronization status, including:
+    - A summary of pending updates and recent activity.
+    - Notifications of resolved conflicts or required user actions.
+    - A detailed view of synchronization history for audit and troubleshooting purposes.
+
+- **Simulation and Testing Integration:**
+  - The synchronization process is designed to work seamlessly in the deterministic simulation environment. This allows us to reproduce and verify multi-device coordination, performance under network latency, and disk failure scenarios during testing (NFR-2, NFR-6).
+
+## Consequences
+
+- **Advantages:**
+
+  - **Enhanced User Control and Transparency:**
+    Users benefit from a clear view of the system state, including file changes and synchronization events, ensuring trust in the system’s operations.
+
+  - **Seamless Multi-Device Experience:**
+    The manifest-based synchronization mechanism along with deterministic conflict resolution ensures that users experience consistent data across all their devices.
+
+  - **Future-Proofing:**
+    The planned transition to a web interface builds upon the initial CLI, ensuring that the system can evolve to meet diverse user preferences without rearchitecting the core synchronization logic.
+
+- **Trade-offs:**
+
+  - **Complexity in Synchronization Logic:**
+    Merging multiple manifest logs and resolving conflicts in a distributed environment adds architectural and implementation complexity. However, this is justified by the need for robust multi-device synchronization.
+
+  - **User Onboarding:**
+    For users unfamiliar with a CLI, there may be an initial learning curve. This is mitigated by comprehensive documentation and the planned development of a more user-friendly graphical interface.
+
+## References to Requirements
+
+- **Functional Requirements:**
+
+  - FR-15: User Interface and Multi-Device Synchronization require that users can manage file operations and view the state of synchronization across devices.
+  - FR-3 & FR-10: Versioning, change tracking, and conflict resolution in the manifest underpin the synchronization process.
+
+- **Non-Functional Requirements:**
+  - NFR-4: Usability & User Experience – The UI design must be clear and intuitive to provide a good user experience.
+  - NFR-2: Performance & Responsiveness – The synchronization mechanisms must perform efficiently, even under simulated network or disk latency.
+  - NFR-6: Deployability & Maintainability – The synchronization process and UI must integrate with deterministic simulation testing to ensure reliability and ease of maintenance.
+
+## Conclusion
+
+The proposed strategy for user interface design and multi-device synchronization ensures that users receive a consistent, transparent, and efficient experience when managing their data. By leveraging an intuitive CLI (with plans for a future web interface) alongside a robust, manifest-based synchronization mechanism, our system meets both the functional and non-functional requirements of reliability, auditability, and performance. This decision supports our mission to empower users with full control over their digital data across all devices.

docs/discovery.md

@@ -0,0 +1,173 @@
+# Encrypted File Storage System – Initial Discovery Document
+
+## 1. Overview
+
+This document outlines the discovery and design considerations for a secure, encrypted file storage system intended for self-hosting enthusiasts. The system is designed not only to securely store and version files with client-side encryption and an immutable manifest but also to extend into a peer-to-peer (P2P) network for redundancy, backup, and collaborative sharing between trusted nodes.
+
+## 2. Goals & High-Level Vision
+
+### 2.1. Security & Privacy
+
+- **End-to-End Encryption:**
+
+  - Files are encrypted on the client before being uploaded, ensuring that the server or any peer only ever handles ciphertext.
+  - Decryption occurs solely on the client, preserving data privacy even in a distributed network.
+
+- **Robust Key Management:**
+  - Encryption keys are derived client-side using strong key derivation functions (e.g., Argon2, scrypt, or PBKDF2).
+  - Integration with OS-level or hardware-based key management solutions is considered for enhanced security.
+
+### 2.2. Efficient Versioning & Storage
+
+- **File Partitioning & Chunking:**
+
+  - Files are divided into fixed-size or content-defined chunks (e.g., 1 MB per chunk).
+  - Only updated chunks are re-uploaded which reduces upload times and storage consumption.
+
+- **Content Addressable Storage & Deduplication:**
+
+  - Each chunk is encrypted separately and indexed by its SHA-256 hash.
+  - This method allows for deduplication, reducing redundant uploads across versions and even across files.
+
+- **Verifiable Append-Only Manifest:**
+  - An append-only log (referred to as the manifest) maintains a complete history of chunk operations and file versions.
+  - Periodic snapshots (that include a hash of the log state) allow for pruning of older logs while ensuring verifiability of data integrity.
+
+### 2.3. P2P Node Network & Redundancy
+
+- **Peer-to-Peer (P2P) Connectivity:**
+
+  - Users can connect their nodes with those of friends using an IP address plus public key combination for secure identification and mutual authentication.
+  - Nodes establish encrypted channels (e.g., via TLS or secure Diffie-Hellman exchanges) to share encrypted file blobs.
+
+- **Data Redundancy and Backup:**
+
+  - Shared file blobs are replicated across nodes to provide data redundancy and backup.
+  - Users can define replication policies, choosing which peers should store copies.
+
+- **Dynamic Trust and Access Control:**
+  - Nodes employ certificate or public-key-based validation mechanisms to grant or restrict access.
+  - A decentralized trust framework ensures that only authorized nodes participate in file sharing.
+
+## 3. Architecture & Component Overview
+
+### 3.1. Client Application
+
+- **User Interaction:**
+
+  - Initially a command-line interface (CLI) is used for file management; a web interface is planned for future development.
+
+- **Processing & Encryption:**
+
+  - Files are partitioned into chunks which are each individually encrypted using authenticated encryption algorithms (e.g., AES-GCM or ChaCha20-Poly1305).
+  - The client maintains an immutable, append-only manifest that records all operations such as additions, modifications, and deletions.
+
+- **Synchronization:**
+  - The manifest is used for multi-device synchronization, allowing devices to merge their change logs seamlessly.
+  - The manifest incorporates periodic snapshots to prune old entries while keeping a cryptographic chain for integrity verification.
+
+### 3.2. Server/P2P Node
+
+- **Self-Hosting Friendly:**
+
+  - Designed to run on user-managed servers with simple configuration.
+  - Containerized deployment (e.g., Docker) ensures consistent and easy installation.
+
+- **API & Data Handling:**
+
+  - The node provides a secure, stateless API (RESTful or gRPC) for interactions with clients.
+  - Encrypted blobs (chunks) are stored and indexed using content addressing to support deduplication and versioning.
+
+- **P2P Functionality & Node Connectivity:**
+
+  - Nodes establish direct secure connections using an IP address and public key identifier.
+  - Support for NAT traversal techniques (e.g., UPnP, STUN, TURN) is built in to facilitate peer connections.
+  - A decentralized discovery mechanism (potentially via a DHT or bootstrap nodes) helps nodes locate one another.
+
+- **Redundancy & Data Synchronization:**
+  - Each node maintains information in the manifest regarding which peers store which file chunks.
+  - Health checks and heartbeat signals maintain up-to-date replication and rebalancing across the network.
+
+## 4. Detailed Design Considerations
+
+### 4.1. File Partitioning and Deduplication
+
+- **Chunking Strategy:**
+
+  - Fixed-size chunks provide predictability, while content-defined chunking (e.g., using Rabin fingerprinting) may help reduce unnecessary re-uploads when file content shifts.
+
+- **Content Addressing:**
+  - SHA-256 is used to hash each chunk before storage, allowing duplicate chunks to be recognized and only stored once across nodes.
+
+### 4.2. Manifest (Append-Only Log)
+
+- **Structure & Integrity:**
+
+  - The manifest is an append-only log where each entry records a discrete change (addition, deletion, modification of chunks), along with metadata such as timestamps, device identifiers, and operation details.
+  - Cryptographic chaining (each log entry containing a hash of the previous one) ensures tamper-evident history.
+
+- **Snapshot Mechanism:**
+  - Periodic snapshots capture the full state of the manifest up to that point by incorporating an aggregate hash.
+  - Future log entries reference the last snapshot, allowing previous data to be pruned while maintaining verifiability.
+
+### 4.3. Multi-Device & P2P Synchronization
+
+- **Manifest Merging:**
+
+  - Devices (including those on separate nodes) merge their manifest logs using ordering methods such as vector clocks or Lamport timestamps to resolve concurrent updates.
+
+- **Node-to-Node Sharing:**
+  - Secure node connections based on IP address and public key identifiers facilitate the sharing of encrypted blobs between trusted peers.
+  - A decentralized model ensures that file updates, redundancy, and replication policies are distributed and maintained across the network.
+
+### 4.4. Security and Access Control
+
+- **Connection Security:**
+
+  - All node-to-node communications use strong encryption (TLS, Diffie-Hellman key exchange, or equivalent) to protect data in transit.
+
+- **Trust & Authentication:**
+  - Nodes exchange public keys during initial handshake for mutual authentication.
+  - Certificate-based or signed permission systems safeguard against unauthorized access, ensuring only trusted peers participate.
+
+### 4.5. Scalability, Resilience, and Management
+
+- **NAT Traversal & Discovery:**
+
+  - Implementation of NAT traversal techniques (UPnP, STUN, TURN) and decentralized peer discovery ensures reliable connectivity, even behind firewalls.
+
+- **Monitoring & Conflict Resolution:**
+  - Systems to monitor node availability (heartbeats, health checks) are essential for maintaining replication and redundancy.
+  - Conflict resolution protocols are implemented in the manifest for consistent state management across devices and nodes.
+
+## 5. Threat Model & Security Audit
+
+- **Threat Model Perspective:**
+
+  - Protect against potential vulnerabilities including unauthorized access, tampering with the manifest, compromised nodes, and interception of communications.
+  - Emphasize that all sensitive operations (encryption, key management, and manifest maintenance) are performed client-side or within a trusted node environment.
+
+- **Audit & Transparency:**
+  - An immutable, cryptographically chained manifest offers auditability.
+  - The design encourages regular security audits and community reviews to validate the security framework.
+
+## 6. Roadmap & Future Enhancements
+
+- **Initial Development:**
+
+  - Build a robust CLI client for file operations, including adding files, updating versions, and restoring files, backed by an append-only manifest.
+  - Develop the server/P2P node with secure API endpoints and replication functionality.
+
+- **P2P Network Expansion:**
+
+  - Implement automated peer discovery, NAT traversal support, and dynamic trust mechanisms.
+  - Create user-friendly tools for configuring peer connections and managing replication policies.
+
+- **Advanced Features:**
+  - Extend the system with a web interface.
+  - Explore integration with existing distributed systems or version control systems.
+  - Consider plugins or additional tools to automate backup management and network health monitoring.
+
+## 7. Conclusion
+
+This discovery document establishes the fundamental framework for a secure, self-hosted encrypted file storage system that integrates efficient versioning, a verifiable append-only manifest, and a P2P network for file sharing and redundancy. By combining client-side encryption, content-addressable storage, robust manifest management, and secure node connectivity, the product aims to deliver high security, privacy, and resilience in a distributed self-hosting environment.

docs/mission-statement.md

@@ -0,0 +1,3 @@
+# Mission Statement
+
+> Our mission is to champion the fundamental right to digital self-ownership. We believe that every person deserves complete control over their data, free from centralized oversight. Our goal is to provide a user-driven, self-hosted storage solution that puts privacy, transparency, and accountability at the forefront. We are committed to ensuring that every interaction with your data is clear, verifiable, and fully revocable—because in our view, privacy is not just a feature, it's a human right.

docs/requirements/fr-1-secure-file-upload-and-download.md

@@ -0,0 +1,120 @@
+# FR-1: Secure File Upload & Download
+
+## 1. Overview
+
+This document defines the detailed requirements for secure file upload and download, ensuring that every file operation adheres to stringent security and privacy standards. The primary goal is to protect user data by encrypting all files on the client before they are uploaded and decrypting them only after they are safely downloaded back to the client. The design ensures that plaintext data never leaves the client environment, thereby eliminating potential exposure to untrusted servers or intermediaries.
+
+## 2. Objectives & Scope
+
+### 2.1 Objectives
+
+- **Client-Side Encryption:** Guarantee that all file data is encrypted on the client before transmission.
+- **End-to-End Data Privacy:** Ensure that files remain concealed (only as ciphertext) during network transit and on any intermediary storage.
+- **Data Integrity Verification:** Verify that files are identical before upload and after download via integrity checking mechanisms.
+- **Robust Error Handling:** Provide predictable, secure responses to network interruptions, transfer errors, and simulated failure scenarios.
+
+### 2.2 Scope
+
+- Applies to all file upload and download operations within the system.
+- Covers interactions between the client and server/P2P nodes.
+- Encompasses encryption/decryption process, key management integration, and data integrity checks.
+
+## 3. Detailed Requirements
+
+### 3.1 Client-Side Encryption & Decryption
+
+- **Encryption Requirement:**
+
+  - All file content must be encrypted entirely on the client side prior to any data leaving the client machine.
+  - Encryption must use an authenticated encryption algorithm (supported options may include AES-GCM or ChaCha20-Poly1305, as per ADR-0002).
+  - The encryption process should remain seamless and performant even when simulating high-latency or failure modes.
+
+- **Decryption Requirement:**
+  - Files must be decrypted solely on the client after download. Under no circumstances should ciphertext be decrypted on the server or during transit.
+  - Successful decryption must only occur if the file’s integrity is verified (e.g., via a cryptographic hash check).
+
+### 3.2 End-to-End Data Privacy
+
+- **Data Secrecy:**
+
+  - The system shall guarantee that the encryption keys used for client-side encryption are never transmitted or stored outside the client’s secure environment.
+  - Encrypted data (ciphertext) is the only form available in transit or when stored at intermediary nodes.
+
+- **Secure Transport:**
+  - All file uploads/downloads shall use secure transport protocols (e.g., TLS) to further protect transmitted data from interception.
+  - Mutual authentication between client and server/P2P node should be enforced to prevent man-in-the-middle attacks.
+
+### 3.3 Data Integrity Verification
+
+- **Integrity Check:**
+
+  - Each file shall be associated with a cryptographic hash (e.g., SHA-256) computed on the plaintext. This hash is used to verify that the file hasn’t been altered during transmission.
+  - Upload processes must include a step for embedding or associating the hash with the encrypted file so that after decryption the client can verify data integrity.
+
+- **Error Detection & Recovery:**
+  - If integration or confirmation of data integrity fails on the client during download, the system shall trigger a secure, automatic re-request or error notification.
+
+### 3.4 Robust Error Handling & Deterministic Simulation Integration
+
+- **Simulated Failure Scenarios:**
+
+  - The encryption and transmission components must be designed with deterministic simulation capabilities. This includes simulating network latency, packet loss, or disrupted connections to ensure the encryption and transfer pipelines respond predictably.
+
+- **Timeouts and Retries:**
+
+  - The system must implement robust timeout management and automatic retry logic for failed upload or download operations. During these events, the client must re-initiate encryption and hash verification to avoid propagation of partial or corrupted uploads.
+
+- **Feedback Mechanisms:**
+  - The user interface (CLI or future web interface) should provide clear feedback on the status of file transfers, including progress, success, or errors.
+  - In the case of an error, informative messages assisting the user in troubleshooting (while maintaining security principles) must be shown.
+
+## 4. Measurable Criteria & Test Cases
+
+### 4.1 Automated End-to-End Tests
+
+- **Encryption Verification Test:**
+
+  - Create a file and simulate an upload operation while capturing the data leaving the client.
+  - Validate that no substring of plaintext is present in the captured data.
+
+- **Integrity Verification Test:**
+
+  - Upload a file and then simulate a download, decrypting the content on the client.
+  - Compare the decrypted file’s hash against the expected SHA-256 hash to confirm integrity.
+
+- **Simulated Fault Conditions Test:**
+  - Execute parallel tests with introduced network latency and intermittent failures.
+  - Ensure that file transfers maintain encryption integrity and error-handling mechanisms appropriately reinitiate failed transfers.
+
+### 4.2 Performance Measurements
+
+- **Encryption/Decryption Overhead:**
+
+  - Record the time required to encrypt and decrypt files under different file sizes.
+  - Ensure that the additional delay stays within acceptable performance limits as defined in NFR-2.
+
+- **Timeout and Retry Efficacy:**
+  - Verify through simulation that the configured timeout/retry strategy minimizes transfer failures without exposing decryption keys or compromising integrity.
+
+## 5. Dependencies & Integration
+
+- **Key Management:**
+
+  - This requirement integrates closely with the Key Management Strategy (refer to ADR-0003), ensuring that keys are securely derived and stored locally.
+
+- **Transport Security:**
+
+  - Utilization of secure transfer protocols (TLS) extends from the Communication Protocols design (ADR-0007).
+
+- **Deterministic Simulation Framework:**
+  - Integration with the deterministic simulation testing environment allows for reproducible tests for file transfer under fault injection scenarios.
+
+## 6. Security Considerations
+
+- All encryption routines should be reviewed for side-channel vulnerabilities.
+- The client-side application must be hardened against local attacks (e.g., memory dumps) that might compromise the encryption keys.
+- Logging for file transfer operations must exclude any sensitive data and focus solely on audit and performance metrics.
+
+## 7. Conclusion
+
+FR-1 sets the foundation for a secure file transfer mechanism that prioritizes client-side encryption, continuous data integrity, and robust error-handling under real-world and simulated adverse conditions. This document serves as the blueprint for developers and testers to ensure that the file upload and download operations meet the highest standards for security and privacy in the system.

docs/requirements/fr-2-file-partitioning-and-chunk-management.md

@@ -0,0 +1,145 @@
+# FR-2: File Partitioning & Chunk Management
+
+## 1. Overview
+
+This document outlines the detailed requirements for partitioning files into manageable chunks and the associated management of these chunks during storage and retrieval. The aim is to optimize file transfer, deduplication, and versioning while ensuring that the file reassembly process maintains data integrity even under adverse conditions such as simulated network or disk latency and failures.
+
+## 2. Objectives & Scope
+
+### 2.1 Objectives
+
+- **Efficient Upload & Download:** Enable efficient file transfers by breaking large files into smaller chunks.
+- **Deduplication:** Support content-addressable storage by indexing each chunk via a cryptographic hash (e.g., SHA-256) to allow redundant data detection and avoidance.
+- **Resilient Reassembly:** Ensure that files can be accurately reassembled from individual chunks in the correct order, preserving the original file's integrity.
+- **Performance Under Failure Conditions:** Validate that the chunking mechanism performs robustly even during simulated network and disk failure scenarios.
+
+### 2.2 Scope
+
+- Applies to all file operations where files are partitioned before upload.
+- Covers algorithms to partition files, assign identifiers, and maintain metadata for reassembly.
+- Includes supporting tools for integrity checking and deduplication.
+- Integrates with deterministic simulation testing to simulate adverse conditions during chunk processing.
+
+## 3. Detailed Requirements
+
+### 3.1 File Partitioning Strategy
+
+- **Chunking Approach:**
+
+  - Files must be divided into discrete segments (chunks) either using a fixed-size approach (for example, 1 MB per chunk) or a content-defined boundary method.
+  - The chosen approach should favor predictability in reassembly while balancing performance and effective deduplication.
+  - Implementation should remain flexible, with the possibility to choose fixed-size chunking during initial deployment and exploring content-defined methods later if necessary.
+
+- **Metadata Association:**
+  - Each chunk shall be assigned metadata, including:
+    • A unique identifier derived from a cryptographic hash (e.g., SHA-256) computed on the plaintext of the chunk.
+    • Sequence information to allow correct reassembly.
+    • Timestamps and any additional markers for version history.
+  - The metadata must be stored alongside the encrypted chunk ensuring the data’s lineage and integrity during recovery.
+
+### 3.2 Management of Chunked Data
+
+- **Encryption and Indexing of Chunks:**
+
+  - Each file chunk is to be encrypted individually on the client side before transmission.
+  - Post encryption, a cryptographic hash is computed (or preserved from pre-encryption, if applicable) to serve as a unique identifier for deduplication and integrity verification.
+  - The encryption must ensure that even if two identical chunks occur (or near-identical), the system can recognize and avoid storing duplicates.
+
+- **Chunk Deduplication:**
+  - The system shall implement a deduplication mechanism whereby if a chunk with an identical hash already exists within the system, the chunk is not stored a second time.
+  - When a duplicate is detected, the system should update its reference count or metadata to indicate reuse, reducing storage overhead.
+
+### 3.3 Reassembly of Files
+
+- **Reassembly Process:**
+
+  - A complete file must be reconstructible by sequentially fetching each required chunk based on its stored metadata and then decrypting and concatenating them in their correct order.
+  - Reassembly operations must include integrity verification, ensuring that each chunk matches its expected hash before being integrated into the final output.
+
+- **Error Handling during Reassembly:**
+  - In the event of a missing or corrupted chunk, the system must trigger a re-request from alternative nodes (if replication is available) and retry the operation.
+  - The user must be notified if reassembly fails after all recovery attempts, and logs should capture the details of the error for auditability.
+
+### 3.4 Robustness Under Simulated Adverse Conditions
+
+- **Deterministic Simulation Integration:**
+
+  - The chunking and reassembly processes must integrate with the deterministic simulation testing framework, allowing testers to simulate:
+    • Network latency and intermittent connectivity during chunk upload/download.
+    • Disk I/O delays or failures during chunk read/write operations.
+  - These simulations must be repeatable, ensuring that robustness and error handling can be thoroughly validated in controlled test scenarios.
+
+- **Timeouts and Retries:**
+  - In case of errors during chunk transfer, the system must implement appropriate retry logic and timeouts.
+  - Retried operations should ensure consistency and integrity checks are reapplied to avoid propagating errors.
+
+### 3.5 Performance Considerations
+
+- **Optimized Partitioning:**
+
+  - The chunking algorithm should be highly efficient to minimize processing overhead, even for large files.
+  - Use of multithreading or asynchronous processing shall be considered to achieve higher throughput during chunking, encryption, and uploading.
+
+- **Resource Management:**
+  - The system must monitor and regulate the resource usage (CPU, memory, disk I/O) during the process of partitioning and reassembly.
+  - Built-in diagnostic tools should measure performance metrics (e.g., chunk processing time, deduplication success rates) and enable adjustments to the configuration parameters dynamically.
+
+## 4. Measurable Criteria & Test Cases
+
+### 4.1 Automated Tests
+
+- **File Chunking and Reassembly Test:**
+
+  - Create test files of varying sizes.
+  - Verify that the file is partitioned into chunks of the expected size.
+  - Ensure that upon download and correct decryption, the reassembled file’s content exactly matches the original using byte-for-byte comparisons (via cryptographic hash comparison).
+
+- **Deduplication Validation Test:**
+
+  - Upload files with repeated content.
+  - Verify that identical chunks are recognized and stored only once while references to the duplicate are maintained.
+
+- **Simulated Failure Conditions Test:**
+  - Run tests with aggressive simulated network and disk latencies or intentional failures during chunk processing.
+  - Confirm that robust retry logic successfully fetches all chunks and that error paths are adequately logged.
+
+### 4.2 Performance Benchmarks
+
+- **Chunk Processing Speed:**
+
+  - Measure the average time required for file partitioning and reassembly across different file sizes.
+  - Validate that these times meet the performance targets as defined in the broader NFR-2 (Performance & Responsiveness) requirements.
+
+- **Resource Utilization:**
+  - Monitor CPU, memory, and disk I/O usage during extensive chunk processing tasks.
+  - Compare against baseline performance profiles to ensure that the process scales efficiently, even under high load scenarios.
+
+## 5. Dependencies & Integration Points
+
+- **Encryption Module:**
+
+  - Fully integrated with the encryption procedures defined in FR-1 and ADR-0002, ensuring that each chunk is securely encrypted.
+
+- **Manifest Management:**
+
+  - The chunk metadata and ordering information is recorded in the immutable, append-only manifest as described in ADR-0006. This ensures a verifiable history for file reassembly and version control.
+
+- **Deterministic Simulation Framework:**
+  - Integration with the simulation framework is critical to ensure that the chunking strategy is robust under adverse conditions (refer to the deterministic simulation aspects in FR-1).
+
+## 6. Security Considerations
+
+- **Data Confidentiality:**
+
+  - Ensure that no sensitive plaintext is written to disk during or after the chunking process.
+
+- **Integrity and Auditability:**
+
+  - All operations on chunks (creation, encryption, deduplication, assembly) must be logged in a tamper-evident manner, supporting the overall auditability requirements of the system.
+
+- **Fallback Mechanisms:**
+  - Consider secure fallback strategies if one type of partitioning algorithm fails or does not perform adequately under certain environmental conditions.
+
+## 7. Conclusion
+
+FR-2 establishes the core mechanism for partitioning files into chunks, ensuring robust management for efficient uploads and downloads in a secured environment. By combining efficient, flexible chunking with strong deduplication, metadata tracking, and robust reassembly mechanisms, this requirement supports the broader system goals of performance, scalability, and data integrity even under simulated failure conditions.

docs/requirements/fr-3-versioning-and-change-tracking.md

@@ -0,0 +1,148 @@
+# FR-3: Versioning and Change Tracking
+
+## 1. Overview
+
+This document defines the detailed requirements for maintaining a comprehensive versioning system and change tracking mechanism throughout the file storage solution. The objective is to record every file operation—such as creation, modifications, and deletions—in an immutable audit trail. This not only enables users to revert changes and restore previous versions but also provides full transparency and auditability across multi-device interactions. The design leverages an immutable, cryptographically chained manifest (as described in ADR-0006) to ensure that every update is verifiable and resistant to tampering.
+
+## 2. Objectives & Scope
+
+### 2.1 Objectives
+
+- **Comprehensive History:** Record every change made to a file, including creation, updates, and deletions.
+- **Auditability & Transparency:** Guarantee that the change history is tamper-evident and fully auditable.
+- **Conflict Resolution & Merging:** Enable automatic and deterministic conflict resolution in multi-device scenarios.
+- **Reversion:** Allow users to revert to any previous version of a file.
+- **Synchronization Support:** Facilitate efficient merging of version histories across devices while preserving the ordering of events.
+
+### 2.2 Scope
+
+- Applies to all file operations across the system, including file modifications on the client and updates propagated through nodes.
+- Involves tracking metadata for every operation (e.g., timestamps, author information, and device identifiers).
+- Integrates with the immutable manifest (ADR-0006) to leverage cryptographic chaining and snapshots.
+- Supports deterministic simulation of versioning and conflict resolution processes under various network and disk conditions.
+
+## 3. Detailed Requirements
+
+### 3.1 Immutable, Append-Only Manifest
+
+- **Manifest Structure:**
+
+  - Every file operation (create, update, delete) must be recorded as an entry in an append-only manifest.
+  - Each entry must include essential metadata, such as a unique change identifier, timestamp, device/client identifier, and a description of the operation.
+  - Entries are cryptographically chained (e.g., each entry includes a hash of the previous one) to ensure tamper detection.
+
+- **Snapshots:**
+  - To prevent performance degradation, periodic snapshots capturing the cumulative state of the manifest must be generated.
+  - Future log entries should reference the most recent snapshot, allowing earlier portions of the manifest to be pruned (while remaining verifiable).
+
+### 3.2 Version Tracking and History
+
+- **Version Creation:**
+
+  - Every modification to a file results in a new version being recorded. Metadata such as version number, change description, and timestamp shall be associated with each version.
+  - The full version history must be retrievable, allowing users to inspect the progression of changes over time.
+
+- **Multi-Device Versioning:**
+  - As files may be modified on multiple devices, each device’s changes must be merged deterministically.
+  - Logical timestamps (for example, Lamport timestamps) will be used to order events across devices to ensure that the sequence of modifications is consistent.
+
+### 3.3 Conflict Detection and Resolution
+
+- **Automated Conflict Detection:**
+
+  - When concurrent modifications are detected (e.g., overlapping timestamps or divergent change paths in the manifest), the system must flag the conflict.
+  - The system shall initiate a deterministic conflict resolution process by applying predefined rules (e.g., favoring the update with the lower Lamport timestamp or deferring to a primary device designation).
+
+- **User Intervention:**
+  - If the automatic resolution algorithm fails to clearly resolve a conflict, the system must flag the conflicted version for manual review.
+  - Clear documentation within the user interface should guide users in resolving conflicts when necessary.
+
+### 3.4 Reversion and Rollback Mechanisms
+
+- **Rollback Capability:**
+
+  - Users must be able to revert a file to any previous version recorded in the manifest.
+  - Reversion operations should restore the file content and update the manifest to record the rollback as a new change, preserving the audit trail.
+
+- **Verification:**
+  - Every reversion must undergo an integrity check (using cryptographic hash comparisons) to ensure that the restored version accurately reflects its original state.
+
+### 3.5 Integration with Deterministic Simulation Testing
+
+- **Simulation of Versioning Under Adverse Conditions:**
+
+  - The complete versioning and conflict resolution logic should be integrated with the deterministic simulation framework.
+  - Simulated tests will introduce network and disk delays or failures during file modifications to validate that the versioning mechanism responds predictably.
+
+- **Logging and Monitoring:**
+  - All versioning activities (including conflict detection and resolution events) must be logged in detail for reproducibility in simulated failure conditions.
+
+## 4. Measurable Criteria & Test Cases
+
+### 4.1 Automated End-to-End Versioning Tests
+
+- **Manifest Consistency Test:**
+
+  - Create a file and perform a series of modifications.
+  - Verify that an immutable manifest accurately records each change and that entries are properly linked via cryptographic chaining.
+
+- **Version History Accuracy Test:**
+  - Retrieve the full version history of a file and compare the recorded data with the expected series of changes.
+  - Verify that metadata (timestamps, device IDs, operation descriptions) matches the executed operations.
+
+### 4.2 Conflict Resolution Testing
+
+- **Simultaneous Update Simulation:**
+
+  - From multiple simulated devices, perform overlapping modifications on the same file.
+  - Validate that conflicts are detected and resolved deterministically according to the predefined rules.
+  - Confirm that any unresolved conflicts are flagged for user intervention.
+
+- **Rollback Verification:**
+  - Initiate a rollback operation to restore an earlier file version.
+  - Verify that the rollback creates a new version entry in the manifest and that the restored file content passes integrity checks.
+
+### 4.3 Performance and Stress Testing
+
+- **Scalability Test:**
+
+  - Simulate rapid, consecutive file modifications to stress-test the versioning system.
+  - Monitor performance metrics, including manifest update latency and system responsiveness under heavy load.
+
+- **Deterministic Simulation:**
+  - Use the simulation framework to introduce network delays and disk I/O latencies while performing versioning operations.
+  - Confirm that the system maintains consistency and the manifest remains verifiable under stressed conditions.
+
+## 5. Dependencies & Integration Points
+
+- **Immutable Manifest System:**
+
+  - This requirement relies heavily on the manifest design (ADR-0006) to store and maintain a tamper-evident history of changes.
+
+- **Conflict Resolution Algorithms:**
+
+  - Integration with the logical timestamp approach (e.g., Lamport timestamps) used in conflict detection and resolution is essential.
+
+- **User Interface:**
+
+  - The UI must present the version history clearly, allow easy navigation of changes, and support rollback/reversion operations.
+
+- **Deterministic Simulation Framework:**
+  - Integration ensures that versioning and conflict resolution behave consistently under simulated adverse conditions in accordance with NFR guidelines.
+
+## 6. Security Considerations
+
+- **Integrity of Version Log:**
+
+  - The cryptographic chaining of manifest entries must be secure against tampering to prevent unauthorized alteration of the file history.
+
+- **Auditability:**
+
+  - All versioning operations must be logged in a tamper-evident manner, ensuring that an audit trail is maintained for regulatory compliance and user trust.
+
+- **Access Controls:**
+  - Only authorized users should be permitted to view or alter the version history. Appropriate authentication mechanisms must be enforced.
+
+## 7. Conclusion
+
+FR-3 establishes a robust versioning and change tracking system that meets the critical needs for auditability, transparency, and user control in a distributed, multi-device environment. By leveraging an immutable, cryptographically chained manifest, robust conflict resolution algorithms, and reversion capabilities, the system provides a trustworthy mechanism for managing file versions while ensuring resilience even under simulated adverse conditions.

docs/requirements/fr-4-peer-to-peer-node-interaction.md

@@ -0,0 +1,144 @@
+# FR-4: Peer-to-Peer Node Interaction
+
+## 1. Overview
+
+This document details the requirements for establishing and maintaining secure peer-to-peer (P2P) node interactions throughout the system. The goal is to enable direct, authenticated communication between trusted nodes, thereby supporting file sharing, replication, and redundancy without relying solely on centralized servers. This requirement is critical to ensuring high availability, resilience, and user-controlled data exchange, while integrating with the overall deterministic simulation framework to test network conditions under varied scenarios.
+
+## 2. Objectives & Scope
+
+### 2.1 Objectives
+
+- **Secure Connectivity:** Ensure that all node-to-node communications use robust encryption and mutual authentication.
+- **Explicit Trust Establishment:** Require that connections between nodes are established based on user-configured trust relationships—using pre-approved IP addresses and associated public key credentials.
+- **Transparent and Resilient Interaction:** Maintain reliable connectivity across nodes even in scenarios with adverse network conditions (e.g., latency, intermittent connectivity, or failures).
+- **Support for Redundancy and Backup:** Enable efficient file replication and synchronization by leveraging direct P2P communication channels.
+- **Deterministic Simulation:** Facilitate testing of P2P interactions under simulated network delays, packet loss, and other disruptions to validate performance and recovery.
+
+### 2.2 Scope
+
+- Applies to all interactions between self-hosted nodes participating in file sharing and replication.
+- Covers the establishment, maintenance, and teardown of direct P2P connections.
+- Encompasses both the initial secure connection handshake and ongoing secure data exchange.
+- Integrates with the deterministic simulation framework to reproduce network scenarios.
+
+## 3. Detailed Requirements
+
+### 3.1 Secure Connection Setup
+
+- **Mutual Authentication:**
+
+  - Nodes must authenticate each other using explicit exchange of pre-configured public key credentials.
+  - Authentication relies on a user-defined list of trusted nodes, where each node is identified by its IP address and public key fingerprint.
+  - No automatic node discovery is permitted; connections must be initiated only with user-approved peers.
+
+- **Encryption of Data:**
+  - All P2P communications must be secured by employing strong encryption methods (e.g., TLS or a custom binary protocol with built-in encryption features).
+  - Encryption should ensure confidentiality and integrity of data in transit so that even if intercepted, the payload remains unusable to unauthorized entities.
+
+### 3.2 Communication Protocols
+
+- **Custom Binary Protocol for Inter-Node Messaging:**
+
+  - A custom-designed binary protocol will be used for node-to-node communication. This minimizes external dependencies and improves control over security features.
+  - The protocol should support low-latency, high-throughput data exchange, tailored specifically for replication and synchronization tasks.
+  - Built-in security features, such as message integrity checks and automatic retransmission for lost packets, are required.
+
+- **Fallback & Secure Transport:**
+  - In environments where the custom protocol might be incompatible, or fallback measures are needed, nodes should be able to leverage standardized secure protocols (e.g., TLS).
+  - The design must facilitate seamless switching between protocols without compromising security or performance.
+
+### 3.3 Explicit Trust and Configuration
+
+- **User-Configured Node List:**
+
+  - The system must provide mechanisms for users to explicitly list nodes (using IP addresses and corresponding public keys) with which to establish connections.
+  - Nodes not on the approved list must be denied connection requests.
+
+- **Trust Management:**
+  - A trust management module is required to verify the identity of each node during the connection handshake process.
+  - The module must store and update trust relationships locally, ensuring that any changes (e.g., revoked access) are promptly enforced.
+
+### 3.4 Resilience and Robustness
+
+- **Network Condition Simulation:**
+
+  - The connectivity framework must integrate with the deterministic simulation system to allow controlled testing scenarios. These include:
+    • Simulating network latency, jitter, and packet loss.
+    • Simulating intermittent connectivity or complete node unavailability.
+  - These tests validate that P2P communications degrade gracefully and that link recovery mechanisms work effectively.
+
+- **Error Recovery and Reconnections:**
+  - In the event of a connection drop or node failure, the system must automatically attempt to re-establish connections with predefined backoff strategies.
+  - When reconnections occur, the entire authentication process must be repeated to ensure continued security.
+
+### 3.5 Monitoring and Feedback
+
+- **Status Indicators:**
+
+  - Nodes shall provide real-time feedback on the status of all P2P connections, including:
+    • Connection establishment successes and failures.
+    • Latency and throughput metrics.
+    • Alerts when a node is unreachable or when security credentials fail verification.
+
+- **Audit Logging:**
+  - All connection events, including successful handshakes and failures, must be recorded in a tamper-evident log.
+  - Logs should record details such as timestamps, node identifiers, error codes (if any), and the outcome of the authentication process.
+
+## 4. Measurable Criteria & Test Cases
+
+### 4.1 Automated Connection Tests
+
+- **Connection Establishment Test:**
+
+  - Pre-configure a set of trusted nodes with valid IP and public key data.
+  - Test that connections are only established between these nodes and that any connection attempts from unapproved nodes are rejected.
+
+- **Encryption Validation Test:**
+  - Verify, by capturing network traffic in a controlled environment, that all inter-node communications are encrypted.
+  - Ensure that no data is transmitted in plaintext.
+
+### 4.2 Resilience Tests
+
+- **Simulated Network Adversity:**
+
+  - Use the deterministic simulation framework to introduce network delays, jitter, and packet loss.
+  - Confirm that nodes automatically reconnect upon transient failures and that connection handshakes are successfully re-initiated.
+
+- **Failure Handling Test:**
+  - Simulate node unavailability (e.g., power cycle a node) and verify that the remaining nodes detect the failure and attempt reconnection according to the defined retry strategies.
+
+### 4.3 Performance and Throughput
+
+- **Latency and Throughput Measurements:**
+  - Record key metrics (latency, throughput, handshake time) during normal operations and under simulated network conditions.
+  - Compare performance against pre-defined benchmarks to ensure that the system meets the targeted responsiveness and efficiency.
+
+## 5. Dependencies & Integration Points
+
+- **Trust and Access Control Module:**
+
+  - FR-4 relies on integration with modules managing explicit trust and public key verification (as described in ADR-0008 and ADR-0009).
+
+- **Encryption and Communication Protocols:**
+
+  - The secure protocol for node communication is further defined in ADR-0007, ensuring alignment in cryptographic practices.
+
+- **Deterministic Simulation Framework:**
+  - Integration with the simulation framework is essential to validate robustness under controlled fault injection scenarios.
+
+## 6. Security Considerations
+
+- **Key Security:**
+
+  - Public keys and associated credentials must be stored securely on each node and never transmitted insecurely.
+
+- **Man-in-the-Middle (MitM) Protection:**
+
+  - The mutual authentication process and encrypted communication channels must be designed to effectively mitigate MitM attacks.
+
+- **Audit and Logging:**
+  - All steps in the connection handshake and subsequent communication sessions need to be logged without leaking sensitive information.
+
+## 7. Conclusion
+
+FR-4 establishes the framework for secure, resilient peer-to-peer node interactions that underpin the system’s replication and synchronization capabilities. By mandating explicit trust configurations, robust mutual authentication, and integration with deterministic simulation testing, this requirement ensures that file sharing between nodes is both secure and reliable even under adverse network conditions. This design upholds the system’s overall objectives of privacy, data availability, and user empowerment.

docs/requirements/fr-5-replication-and-redundancy-management.md

@@ -0,0 +1,140 @@
+# FR-5: Replication and Redundancy Management
+
+## 1. Overview
+
+This document specifies the requirements for ensuring high availability and fault tolerance through replication and redundancy of file chunks across multiple nodes. The aim is to guarantee that file data remains accessible even when individual nodes encounter failures or network disruptions. By automatically replicating data and maintaining redundancy, the system can support continuous file access and preserve data integrity under both expected and adverse conditions.
+
+## 2. Objectives & Scope
+
+### 2.1 Objectives
+
+- **High Data Availability:** Ensure that file chunks are stored redundantly across multiple nodes so that failures of individual nodes do not result in loss of data.
+- **Fault Tolerance:** Enable automatic recovery from node, network, or disk failures through robust replication mechanisms.
+- **Seamless Replication:** Perform replication without user intervention, ensuring that data redundancy remains up-to-date.
+- **Efficient Resource Usage:** Optimize replication to reduce unnecessary data transfer and storage overhead, using techniques such as deduplication and reference counting.
+- **Integration with Deterministic Simulation:** Validate replication and redundancy mechanisms under simulated conditions such as network latency and disk failures.
+
+### 2.2 Scope
+
+- Applies to all file chunks managed by the system.
+- Covers the automatic replication process that spreads file chunks to multiple nodes.
+- Includes verification of replicated data using integrity checks (e.g., SHA-256 hashes).
+- Integrates with the system’s deterministic simulation framework for testing under controlled adverse conditions.
+
+## 3. Detailed Requirements
+
+### 3.1 Automatic Replication Mechanism
+
+- **Replication Trigger:**
+
+  - The system shall automatically monitor the availability and integrity of file chunks.
+  - On detecting a node failure or data inconsistency, the system must trigger a replication process to copy affected chunks to other healthy nodes.
+
+- **Predefined Replication Factor:**
+  - Configuration must allow specifying a minimum replication factor (e.g., a chunk should exist on at least three different nodes) for fault tolerance.
+  - In cases where a chunk is stored on fewer nodes than specified, a replication process is initiated to restore the desired redundancy level.
+
+### 3.2 Data Integrity and Verification
+
+- **Integrity Checks:**
+
+  - Each file chunk must be associated with a cryptographic hash (e.g., SHA-256) that is used to verify the chunk’s integrity before and after replication.
+  - During the replication process, a verification step must confirm that the replicated data matches the expected hash.
+
+- **Auditability:**
+  - All replication operations, including successful replications and error events, must be recorded in an immutable, tamper-evident log.
+  - Logs should include details such as timestamps, node identifiers, replication actions, and outcomes to support audit and troubleshooting activities.
+
+### 3.3 Transparent Redundancy Management
+
+- **Automatic Monitoring:**
+
+  - A built-in monitoring service (or daemon) should continuously assess the health and availability of file chunks across nodes.
+  - The monitoring service should also check for outdated replicas and trigger rebalancing if necessary.
+
+- **User Transparency:**
+  - The replication mechanism must operate in the background, without requiring manual configuration or intervention by the user during normal operations.
+  - Users should, however, be provided with a summary view of replication status and data redundancy levels via the user interface.
+
+### 3.4 Handling Node Failures and Network Disruptions
+
+- **Resilience under Failure:**
+
+  - In the event of a node or disk failure, the system must automatically redistribute file chunks based on the pre-configured replication policy.
+  - The replication strategy must include fallback mechanisms that ensure replication continues despite intermittent network connectivity or node unavailability.
+
+- **Retry and Recovery:**
+  - Replication operations should include retry logic and exponential backoff strategies in case of transient errors.
+  - After restoration of connectivity or node recovery, the system must reassess and adjust the replication factor if necessary.
+
+### 3.5 Integration with Deterministic Simulation Testing
+
+- **Simulated Failure Conditions:**
+
+  - The replication process must be fully testable within a deterministic simulation framework that emulates network latency, packet loss, and disk failures.
+  - Testing scenarios should include verifying that replication operations complete successfully despite simulated adverse conditions, and that logs accurately reflect the replication events.
+
+- **Performance Benchmarks:**
+  - The system should meet performance targets under simulated failure conditions, ensuring that replication delays do not significantly impact overall system responsiveness (see NFR-2 for performance benchmarks).
+
+## 4. Measurable Criteria & Test Cases
+
+### 4.1 Automated End-to-End Replication Tests
+
+- **Replication Factor Test:**
+
+  - Simulate a scenario where one or more nodes become unavailable. Verify that file chunks are replicated up to the pre-defined replication factor.
+  - Confirm that the system automatically detects insufficient replication and initiates additional replication cycles.
+
+- **Data Integrity Test:**
+
+  - For each replicated chunk, compare the cryptographic hash of the replica against the original value.
+  - Ensure that any mismatch triggers an error recovery process and logs the event for audit purposes.
+
+- **Recovery and Retry Test:**
+  - Simulate transient network outages and node failures. Verify that the replication process successfully recovers once the system conditions stabilize and that all affected chunks reach the required replication level.
+
+### 4.2 Performance and Stress Testing
+
+- **Replication Throughput Test:**
+
+  - Measure the time taken to replicate a set of file chunks under normal conditions versus under simulated network latency/disruptions.
+  - Ensure that performance remains within acceptable bounds as outlined in the NFR-2 specifications.
+
+- **Logging and Audit Test:**
+  - Validate that all replication actions are recorded in the system log with accurate timestamps, node IDs, and operation details.
+  - Confirm that auditor reviews can successfully trace and verify all replication events.
+
+## 5. Dependencies & Integration Points
+
+- **Storage Backend:**
+
+  - Reliance on the underlying storage management system (see ADR-0012) which manages local file systems and content-addressable storage.
+
+- **Manifest and Audit Log:**
+
+  - Integration with the immutable manifest (ADR-0006) ensures that replication events are incorporated into the overall audit trail.
+
+- **Deterministic Simulation Framework:**
+
+  - Testing and validation of the replication mechanisms under simulated adverse conditions are critical, as defined by the deterministic simulation framework requirements.
+
+- **Monitoring and Diagnostic Tools:**
+  - The replication system should integrate with internal monitoring tools to continuously track node health, replication status, and resource utilization.
+
+## 6. Security Considerations
+
+- **Data Confidentiality:**
+
+  - Replicated data must remain encrypted at all times, with no decryption occurring during normal replication operations.
+
+- **Tamper-Evident Logging:**
+
+  - All replication-related events must be logged securely, ensuring that logs cannot be altered or tampered with to hide replication failures or anomalies.
+
+- **Access Controls:**
+  - Only authorized system components and administrators should be able to trigger or modify replication operations. Proper authentication measures must be enforced.
+
+## 7. Conclusion
+
+FR-5 establishes a robust replication and redundancy management mechanism that is essential for ensuring high data availability and fault tolerance in the system. By automatically replicating file chunks across multiple nodes while maintaining strict integrity checks and tamper-evident logging, the system efficiently safeguards user data against node, network, or disk failures. Integration with deterministic simulation further ensures that these mechanisms are reliable under both normal and adverse operating conditions.

docs/requirements/fr-6-user-control-and-access-management.md

@@ -0,0 +1,156 @@
+# FR-6: User Control and Access Management
+
+## 1. Overview
+
+This document outlines the detailed requirements for ensuring that users retain complete control over their data through robust access management mechanisms. In FR-6, every user interaction—whether sharing files, granting or revoking access, or reviewing audit logs—must be transparent, auditable, and secure. The design requires that all access requests are cryptographically verified and logged in a tamper-evident manner. This functionality is critical for sustaining user trust, as it empowers individuals to manage who can access, modify, or share their information.
+
+## 2. Objectives & Scope
+
+### 2.1 Objectives
+
+- **User Empowerment:**
+  - Provide intuitive controls that allow users to manage permissions for file access and sharing.
+  - Ensure that users maintain direct control over data access without relying on intermediaries.
+- **Transparency and Auditability:**
+  - Guarantee that all access-related operations are recorded in an immutable, auditable log.
+  - Enable users to review historical access events and changes in permissions.
+- **Secure Access Management:**
+  - Employ strong cryptographic measures for authenticating access requests and validating user identity.
+  - Ensure that operations such as grant, revoke, and modification of access rights are executed securely.
+- **Multi-Device Coordination:**
+  - Support consistency of access controls across devices in a multi-device environment by synchronizing audit logs and control data.
+
+### 2.2 Scope
+
+- Covers all interactions related to granting, modifying, or revoking access to file data and shared nodes.
+- Applies to both local user interactions (via CLI or future web interfaces) and automated peer-to-peer replication processes.
+- Includes integration with the audit logging system, ensuring that every access control change is verifiable.
+- Interfaces with authentication, key management (ADR-0003), and secure communication modules (ADR-0007) to safely carry out access operations.
+
+## 3. Detailed Requirements
+
+### 3.1 Access Control Mechanisms
+
+- **Cryptographic Credentials:**
+
+  - All access requests (both granting and revoking rights) must be digitally signed using each user’s private key.
+  - The system must verify these signatures on the server or node before processing permission changes.
+
+- **Access Rights Definitions:**
+  - Users shall be able to define granular access rights for different operations (read, write, share, delete) on individual files or groups of files.
+  - Access rights definitions must be stored securely and associated with file metadata in the manifest.
+
+### 3.2 User-Driven Permission Management
+
+- **Granting Access:**
+
+  - Users must be able to grant access to file data or specific file operations by explicitly specifying user identities or node credentials (public key-based).
+  - The granting process should provide options for temporary or indefinite access, as needed.
+
+- **Revoking Access:**
+
+  - Users must have the ability to revoke access permissions at any time.
+  - Revocation actions must immediately take effect and propagate to all nodes holding cached or replicated data.
+
+- **Access Review and Audit:**
+  - The interface should allow users to view a detailed list of all current access permissions for their files.
+  - Audit logs must capture every grant, modification, or revocation event, including timestamps, user/node identifiers, and the nature of the access change.
+
+### 3.3 Audit Logging and Transparency
+
+- **Tamper-Evident Logs:**
+
+  - All access control events must be recorded in an immutable, append-only log that is chained cryptographically (following the design from ADR-0006).
+  - These logs should be integrated into the overall manifest, ensuring that any unauthorized changes will be immediately detectable.
+
+- **Real-Time Notifications:**
+  - The system should provide real-time feedback on critical access control events such as unexpected revocations or unauthorized access attempts.
+  - Alerts should also be available in the UI for administrative review.
+
+### 3.4 Interfacing with Multi-Device Synchronization
+
+- **Consistent Access State:**
+
+  - In a multi-device environment, access control changes should be synchronized across all active devices in near real-time.
+  - Mechanisms such as manifest merging (ADR-0010) must be employed to ensure that permission changes are consistently reflected across each device.
+
+- **Conflict Management:**
+  - When multiple devices simultaneously update access permissions, the system must detect and resolve conflicts deterministically using logical timestamps.
+  - Any ambiguities must be flagged for user review.
+
+### 3.5 Error Handling and Recovery
+
+- **Robustness in Adverse Conditions:**
+
+  - The system should provide clear error messages and fallback procedures if an access control change fails (e.g., due to network issues or simulation-induced faults).
+  - Automatic retry mechanisms must be integrated so that transient failures do not compromise overall security.
+
+- **Verification on Recovery:**
+  - After a reconnection or system recovery event, all access permissions must be revalidated to ensure consistency with the audit log.
+
+## 4. Measurable Criteria & Test Cases
+
+### 4.1 Automated End-to-End Tests
+
+- **Access Permission Grant Test:**
+
+  - Simulate a user granting read and write permissions to another user.
+  - Verify via audit logs and manifest entries that the access grant is recorded accurately and that the delegated user can access the designated file operations accordingly.
+
+- **Revocation Integrity Test:**
+
+  - Execute user-driven revocation of access, then attempt access with the revoked credentials to ensure the operation is immediately blocked.
+  - Check corresponding audit logs for the revocation event and ensure notifications were issued.
+
+- **Synchronization Consistency Test:**
+  - On multiple devices, change access permissions and confirm that all devices reflect the updated permissions within a predefined time window.
+  - Validate that conflicting updates are resolved deterministically and logged appropriately.
+
+### 4.2 Performance and Stress Testing
+
+- **Response Time Measurement:**
+
+  - Benchmark the time taken from initiating an access control change to the update propagating across the entire system.
+  - Validate that response times meet target criteria under both normal and simulated adverse network conditions.
+
+- **Audit Log Integrity Under Load:**
+  - Stress-test the audit logging subsystem by generating a high volume of access control changes.
+  - Ensure that logs remain tamper-evident and that retrieval and verification operations continue to perform within acceptable limits.
+
+## 5. Dependencies & Integration Points
+
+- **Key Management Module:**
+
+  - This requirement collaborates closely with the key management strategy (ADR-0003) to ensure that cryptographic keys and digital signatures are managed securely.
+
+- **Immutable Manifest and Audit Log:**
+
+  - Integrates with the manifest system (ADR-0006) to provide a tamper-evident history of access control operations.
+
+- **Communication Protocols:**
+
+  - Secure messaging defined in ADR-0007 ensures that all access control messages are transmitted with confidentiality and integrity.
+
+- **Multi-Device Synchronization Mechanisms:**
+  - Leverages the versioning and manifest merging processes described in ADR-0010 to maintain consistent access rules across devices.
+
+## 6. Security Considerations
+
+- **Protection of Credentials:**
+
+  - Ensure that private keys used for signing access control requests remain strictly on the client and are never transmitted over the network.
+
+- **Prevention of Unauthorized Access:**
+
+  - Implement robust authentication to prevent impersonation and unauthorized modifications of access rights.
+
+- **Audit and Compliance:**
+
+  - Strictly enforce audit logging to support compliance with policies or regulatory requirements regarding data access and user privacy.
+
+- **Mitigation of Replay Attacks:**
+  - Utilize nonce values or timestamps in access control messages to protect against replay attacks.
+
+## 7. Conclusion
+
+FR-6 establishes a comprehensive, secure, and user-centric model for managing access permissions and ensuring that all operations are fully transparent and auditable. By integrating strong cryptographic verification, tamper-evident logging, and consistent updates across multiple devices, this requirement not only empowers users with full control over their data but also reinforces the trustworthiness and integrity of the entire system—even under simulated adverse conditions.

docs/requirements/nfr-1-security-and-privacy.md

@@ -0,0 +1,141 @@
+# NFR-1: Security & Privacy
+
+## 1. Overview
+
+This document defines the non-functional requirements related to the security and privacy aspects of the self-hosted encrypted file storage system. The goal is to ensure that every component of the system—from client-side encryption to inter-node communication—is designed, implemented, and maintained with robust security measures. These measures must protect user data against unauthorized access, tampering, and breaches while maintaining user privacy, even in the face of simulated adverse conditions.
+
+## 2. Objectives & Scope
+
+### 2.1 Objectives
+
+- **Data Confidentiality:**
+  Ensure that no sensitive plaintext data is exposed during file upload, download, storage, or transmission.
+
+- **Data Integrity:**
+  Guarantee that data remains unaltered during transit and storage by using cryptographic checks (e.g., SHA-256) and authenticated encryption.
+
+- **Strong Cryptographic Standards:**
+  Enforce the use of industry-standard, authenticated encryption algorithms (e.g., AES-GCM or ChaCha20-Poly1305) and robust key derivation functions (e.g., Argon2, scrypt, or PBKDF2).
+
+- **Secure Communication:**
+  Ensure all network communications (client-to-server, inter-node) are secured with protocols like TLS or equivalent secure channels that are resistant to man-in-the-middle (MitM) attacks.
+
+- **Privacy by Design:**
+  Architect the system so that sensitive operations, including encryption, decryption, and key management, occur entirely on the client side, minimizing exposure of sensitive data.
+
+### 2.2 Scope
+
+- Applies to all components and interactions within the system including file operations, node communication, audit logging, and user access controls.
+- Covers cryptographic operations, secure transport layers, and the handling and storage of keys.
+- Includes integration with deterministic simulation testing to ensure security measures remain effective even under simulated network and system failure conditions.
+
+## 3. Detailed Requirements
+
+### 3.1 Strong Cryptographic Practices
+
+- **Encryption Algorithms:**
+
+  - The system must utilize authenticated encryption algorithms (e.g., AES-GCM or ChaCha20-Poly1305) to ensure both confidentiality and integrity.
+  - Algorithms must be chosen based on hardware capabilities (e.g., AES-NI for AES-GCM) and fallback to alternatives when necessary.
+
+- **Key Derivation and Management:**
+  - Keys must be derived using strong key derivation functions (e.g., Argon2, scrypt, or PBKDF2) on the client side.
+  - Private encryption keys shall be generated and stored securely on the client, never transmitted or replicated externally.
+  - Integration with OS-level keychains or HSMs should be supported for environments that offer hardware-backed key management.
+
+### 3.2 Secure Communication Protocols
+
+- **Transport Layer Security:**
+
+  - All communications (client-server and inter-node) must use secure transport protocols such as TLS.
+  - Certificates and mutual authentication should be enforced to prevent unauthorized access and MitM attacks.
+
+- **Custom Protocol Security:**
+  - For inter-node communication through custom binary protocols, built-in encryption and integrity checks (e.g., message authentication codes) must be provided, ensuring those protocols meet equivalent security standards to TLS.
+
+### 3.3 Data Integrity and Verification
+
+- **Cryptographic Hashes:**
+
+  - File data, including subdivided file chunks, must be associated with cryptographic hashes (e.g., SHA-256) that are used to verify data integrity during storage and transmission.
+  - Integrity checks must be performed before and after encryption/decryption to ensure that no data corruption or tampering has occurred.
+
+- **Auditability:**
+  - Security events, including encryption operations and communication handshakes, must be logged in a tamper-evident format.
+  - Logs should be maintained in an append-only manner with cryptographic chaining to support auditability and forensic analysis.
+
+### 3.4 Privacy Preservation
+
+- **Client-Side Security:**
+
+  - Critical operations such as encryption, decryption, and key derivation must occur exclusively on the client side.
+  - Sensitive data (e.g., plaintext files, encryption keys) must not leave the client environment.
+
+- **Minimal Exposure:**
+  - Only ciphertext should be transmitted or stored; any metadata transmitted must be carefully chosen to avoid leaking sensitive information.
+
+### 3.5 Integration with Deterministic Simulation
+
+- **Simulation of Adverse Conditions:**
+  - The security measures must be verified under deterministic simulation to test for vulnerabilities and operational integrity under scenarios like network intercepts, simulated tampering, or system faults.
+  - The simulation framework should be used to validate that secure communications remain intact and that cryptographic operations are consistently performed without exposing sensitive data.
+
+## 4. Measurable Criteria & Test Cases
+
+### 4.1 Security Testing
+
+- **Penetration Testing:**
+
+  - Conduct penetration tests to confirm that no critical vulnerabilities (e.g., exposure of plaintext data, weak ciphers) are present.
+  - Tests should simulate various attack vectors including MitM attacks, replay attacks, and side-channel attacks.
+
+- **Cryptographic Validation:**
+  - Automated tests must verify that encryption and decryption functions produce correct and consistent results (i.e., round-trip tests) and that decryption fails when data integrity is compromised.
+  - Performance metrics for key derivation should be measured to ensure adequate protection against brute-force attacks.
+
+### 4.2 Communication & Network Security Tests
+
+- **TLS and Protocol Testing:**
+  - Validate that all endpoints enforce TLS, and that communications cannot be downgraded to insecure channels.
+  - Use controlled environments to capture network traffic and confirm that no plaintext data is transmitted.
+
+### 4.3 Integration with Simulation
+
+- **Simulated Attack Scenarios:**
+  - Utilize the deterministic simulation framework to simulate network attacks (e.g., packet tampering, replay) and confirm that security mechanisms correctly detect and mitigate such events.
+  - Test the system’s ability to maintain integrity and confidentiality under induced adversarial conditions.
+
+## 5. Dependencies & Integration Points
+
+- **Key Management Module:**
+
+  - Relies on secure key management strategies (described in ADR-0003) to ensure that encryption keys are generated, stored, and utilized exclusively on the client side.
+
+- **Communication Protocols:**
+
+  - Built upon secure protocols as defined in ADR-0007 and the custom security measures listed therein.
+
+- **Audit and Manifest Systems:**
+
+  - Works in conjunction with the immutable manifest (ADR-0006) to guarantee that all security events are logged and traceable.
+
+- **Deterministic Simulation Framework:**
+  - Integration with the simulation framework is critical to validating security functionalities under controlled test conditions.
+
+## 6. Security Considerations
+
+- **Resilience Against Breaches:**
+
+  - The system should be designed to minimize damage in case of a breach, ensuring that attackers cannot easily derive keys or intercept sensitive operations.
+
+- **Monitoring and Response:**
+
+  - Real-time monitoring should detect abnormal activities (e.g., unusual authentication failures) and alert administrators or trigger automated countermeasures.
+
+- **Compliance and Best Practices:**
+  - The implementation must comply with industry-standard security practices and regulatory requirements where applicable.
+  - Regular security audits and updates must be scheduled to address emerging threats and vulnerabilities.
+
+## 7. Conclusion
+
+NFR-1 establishes the comprehensive security and privacy framework essential for a trustworthy encrypted file storage system. By enforcing strong cryptographic practices, ensuring secure communications, maintaining data integrity, and integrating rigorous testing and simulation, the system is designed to protect user data throughout its lifecycle. These measures form the foundation for user trust and the overall resilience of the system.

docs/requirements/nfr-2-performance-and-responsiveness.md

@@ -0,0 +1,143 @@
+# NFR-2: Performance & Responsiveness
+
+## 1. Overview
+
+This document defines the performance and responsiveness criteria that the self-hosted encrypted file storage system must meet. The goal is to ensure that all file operations—including encryption/decryption, file partitioning, upload/download, replication, and synchronization—underpin user interactions with minimal latency and overhead. The system must remain responsive under normal operational loads as well as under simulated adverse conditions using deterministic simulation testing.
+
+## 2. Objectives & Scope
+
+### 2.1 Objectives
+
+- **Low Latency Operations:**
+  Ensure that file uploads, downloads, and internal file processing complete within acceptable response times so that users experience no significant delays.
+
+- **High Throughput:**
+  Optimize system workflows (e.g., encryption, chunking, replication) to handle multiple simultaneous operations efficiently, thereby supporting scalability.
+
+- **Consistent Responsiveness Under Stress:**
+  Maintain speedy processing even under conditions of high load, intermittent network conditions, and disk latency or failure scenarios simulated via deterministic testing.
+
+- **Predictable and Deterministic Performance:**
+  Provide measurable performance benchmarks across various components of the system to allow reliable prediction and repeatability of operations.
+
+### 2.2 Scope
+
+- Applies to all system modules including client-side file processing (encryption/decryption and file partitioning), server/peer communications, replication processes, and manifest updates.
+- Covers performance both in normal operating conditions and simulated adverse scenarios (using the deterministic simulation framework for network/disk conditions).
+- Includes interface responsiveness in both the command-line and any future web-based implementations.
+
+## 3. Detailed Requirements
+
+### 3.1 File Operation Timeliness
+
+- **Encryption/Decryption Overhead:**
+
+  - File encryption and decryption operations must complete within a predefined time range relative to file size (e.g., X seconds per MB) under typical hardware conditions.
+  - The chosen cryptographic operations (e.g., AES-GCM or ChaCha20-Poly1305) must be benchmarked, and implementations should leverage hardware acceleration where available.
+
+- **Chunking and Reassembly Speed:**
+  - Partitioning a file into chunks (or reassembling chunks back to a file) must be optimized to avoid bottlenecks. Target processing times should be established and validated through benchmarking.
+
+### 3.2 Network and Communication Performance
+
+- **Upload/Download Latency:**
+
+  - The latency between initiating a file upload or download and obtaining a response must satisfy predetermined performance targets (e.g., response times within a few hundred milliseconds plus the transmission time).
+  - Endpoints must regulate protocol overhead, ensuring that secure channels (e.g., TLS) and custom binary protocols do not adversely affect throughput.
+
+- **Inter-Node Communication:**
+  - Peer-to-peer node communications for replication and synchronization must be optimized for low-latency data transfer.
+  - Performance metrics such as handshake times, data transfer latency, and retransmission delays (under simulated adverse network conditions) should be regularly measured and meet target benchmarks.
+
+### 3.3 System Throughput and Load Handling
+
+- **High-Concurrency Support:**
+
+  - The system must support multiple simultaneous file operations across different nodes. Concurrency handling mechanisms (such as asynchronous I/O and multithreading) should be in place to avoid performance degradation.
+  - Performance tests must be conducted to simulate multiple concurrent users updating files, replicating chunks, and synchronizing manifests.
+
+- **Scaling Efficiency:**
+  - As the number of users, files, or nodes increases, overall system throughput should scale linearly (or near-linearly) with no single performance bottleneck.
+  - The system shall define performance baselines against which scaled deployments (e.g., hundreds of nodes or thousands of file operations per minute) can be measured.
+
+### 3.4 Deterministic Testing and Simulation of Adverse Conditions
+
+- **Integration with Simulation Framework:**
+
+  - The performance requirement must integrate with the deterministic simulation framework to simulate adverse conditions such as network latency, jitter, and disk I/O delays.
+  - Performance tests under these simulation conditions must confirm that the system maintains acceptable response times with gracefully degrading performance rather than catastrophic failure.
+
+- **Predictable Recovery:**
+  - The system must implement robust timeout and retry strategies for transient failures, ensuring that performance remains predictable even under intermittent network or disk disruptions.
+  - Comprehensive logging and diagnostic outputs should be available to monitor and tune timeout/retry settings as needed.
+
+### 3.5 User Interface Responsiveness
+
+- **CLI and Web UI Feedback:**
+  - The user interface should provide immediate feedback on file operation progress (e.g., encryption progress bars, replication status, synchronization updates).
+  - UI response times—such as command acknowledgement and display of operation status—must be optimized to deliver a responsive user experience, even in environments under load.
+
+## 4. Measurable Criteria & Test Cases
+
+### 4.1 Benchmarking and Profiling
+
+- **Per-Operation Benchmarking:**
+  - Establish benchmarks for typical file sizes (e.g., 1 MB, 10 MB, 100 MB) to measure encryption/decryption times, chunking/reassembly times, and upload/download latency.
+  - Use profiling tools to measure CPU, memory, and I/O usage during operation and ensure that system resource consumption remains within acceptable limits.
+
+### 4.2 Load Testing
+
+- **Concurrency Testing:**
+  - Simulate multiple concurrent file operations to validate the system's scalability.
+  - Stress tests should trigger peak loads and record response times, ensuring that performance remains within defined thresholds.
+
+### 4.3 Simulation Testing
+
+- **Injected Network/Disk Latency:**
+
+  - Apply the deterministic simulation framework to induce network latency, jitter, and disk I/O delays, then measure the system’s responsiveness under these conditions.
+  - Document recovery time for operations after simulated adverse conditions subside.
+
+- **Timeout and Retry Efficiency:**
+  - Test scenarios where connections drop, ensuring that retry logic recovers the operation promptly without significant additional delay.
+
+### 4.4 End-to-End Operation Tests
+
+- **Complete File Transfer:**
+  - Execute a full cycle of file upload (including client-side encryption, partitioning, transfer) and subsequent download (including decryption and reassembly), measuring total time and per-stage performance.
+  - Verify that file integrity is maintained and that individual stage durations meet target performance criteria.
+
+## 5. Dependencies & Integration Points
+
+- **Cryptographic Modules:**
+
+  - The performance of encryption/decryption functions (as defined in NFR-1) is a critical dependency.
+  - Integration with hardware acceleration components (e.g., AES-NI) must be verified and optimized.
+
+- **Network and Communication Modules:**
+
+  - Performance of secure communication protocols (as defined in ADR-0007 for client-server and inter-node communications) directly impacts overall responsiveness.
+
+- **Simulation Framework:**
+
+  - Integration with the deterministic simulation framework is essential to reliably test system behavior under enforced adverse conditions.
+
+- **Monitoring Tools:**
+  - Diagnostic tools and system monitors must be integrated to continuously profile system resource usage and ensure that performance thresholds are met.
+
+## 6. Security and Responsiveness Considerations
+
+- **Balancing Security Overhead:**
+
+  - While strong cryptographic measures and secure communication protocols add inherent overhead, these must be balanced with performance optimizations to prevent latency degradation.
+
+- **Resource Management:**
+
+  - Efficient system resource management (CPU, memory, disk I/O) is crucial to maintaining high throughput. Strategies such as caching, asynchronous processing, and load balancing should be employed.
+
+- **Scalability Impact:**
+  - The system design must account for the impact of scaling on performance and implement mechanisms to mitigate potential bottlenecks that could affect responsiveness.
+
+## 7. Conclusion
+
+NFR-2 defines the performance and responsiveness criteria that are critical to delivering a seamless, user-friendly experience in the self-hosted encrypted file storage system. By setting measurable benchmarks, integrating robust testing under simulated adverse conditions, and ensuring efficient resource management, the system is designed to maintain low latency and high throughput in both typical and stressed operational environments. These requirements support the overall goals of scalability, efficiency, and user satisfaction while upholding stringent security and privacy standards.

docs/requirements/nfr-3-scalability-and-capacity.md

@@ -0,0 +1,86 @@
+# NFR-3: Scalability & Capacity
+
+## 1. Overview
+
+This document defines the requirements for ensuring that the system can scale efficiently as usage, file sizes, user numbers, and node counts grow. The goal is to ensure that the system’s performance and resource utilization remain within acceptable limits even when handling increased loads and when deployed across many nodes. Scalability and capacity planning are essential to guarantee consistent performance under high volume and stress conditions.
+
+## 2. Objectives & Scope
+
+### 2.1 Objectives
+
+- **Linear or Near-Linear Scaling:**
+  The system should scale its processing, storage, and communication capabilities in proportion to growth in users, file operations, and nodes.
+
+- **Efficient Resource Utilization:**
+  Optimize CPU, memory, disk I/O, and network bandwidth to accommodate expanding workloads without degradation.
+
+- **High Throughput Maintenance:**
+  Ensure that increases in data volume or node count do not cause bottlenecks in file processing, replication, or synchronization.
+
+### 2.2 Scope
+
+- Applies to backend processing (encryption, partitioning, replication), data storage management, and inter-node communications.
+- Covers performance under normal growth conditions as well as stress scenarios simulated via deterministic testing.
+- Encompasses both vertical scaling (improving performance on a single node) and horizontal scaling (adding more nodes).
+
+## 3. Detailed Requirements
+
+### 3.1 System Performance Under Load
+
+- **Throughput Benchmarks:**
+
+  - Define target throughput with respect to number of file operations per minute/hour.
+  - Establish benchmarks for performance as nodes or user counts increase.
+
+- **Resource Efficiency:**
+  - Implement mechanisms for dynamic load balancing and resource allocation to prevent any single node from being overwhelmed.
+  - Support distributed storage paradigms (such as content-addressable storage) to minimize duplication and resource waste.
+
+### 3.2 Elastic Scaling Strategies
+
+- **Horizontal Scalability:**
+
+  - Enable the addition of nodes to increase processing power and storage capacity with minimal reconfiguration.
+  - Integrate with orchestration tools (e.g., Kubernetes) to facilitate automatic scaling based on load.
+
+- **Vertical Scalability:**
+  - Allow nodes to use hardware acceleration and optimized resources to improve performance on larger workloads.
+
+### 3.3 Bottleneck Identification and Mitigation
+
+- **Profiling and Stress Testing:**
+  - Establish periodic testing procedures under simulated high-load conditions to identify performance bottlenecks (e.g., CPU, memory, network).
+  - Implement caching strategies, asynchronous processes, and efficient algorithms to mitigate identified bottlenecks.
+
+## 4. Measurable Criteria & Test Cases
+
+### 4.1 Load Testing
+
+- Simulate increased numbers of concurrent file uploads, downloads, and replication operations across many nodes.
+- Measure throughput, latency, and resource usage as the load increases.
+- Validate that performance metrics remain within pre-defined acceptable thresholds.
+
+### 4.2 Scalability Benchmarks
+
+- Establish baseline performance with a known number of nodes and users.
+- Incrementally increase load and record the scaling behavior; ideally, performance degradation should be minimal or linear.
+- Use horizontal scaling tests to add nodes dynamically and verify that overall throughput increases proportionately.
+
+### 4.3 Capacity Planning Tests
+
+- Simulate scenarios with large file sizes and high data volumes.
+- Monitor system behavior (e.g., disk I/O, network bandwidth usage) to ensure that capacity limits are not exceeded.
+- Validate that distributed storage mechanisms efficiently leverage available resources.
+
+## 5. Dependencies & Integration Points
+
+- **Monitoring and Logging Tools:**
+  - Integration with system monitors to continuously track performance metrics and resource utilization.
+- **Orchestration Platforms:**
+  - Compatibility with container orchestration tools to facilitate horizontal scaling.
+- **Deterministic Simulation Framework:**
+  - Use simulation to reproduce high-load scenarios reliably.
+
+## 6. Conclusion
+
+NFR-3 ensures that the system can scale as demand grows, maintaining high performance and efficient resource utilization. By establishing clear throughput benchmarks, elastic scaling strategies, and regular stress testing, the system will remain robust and responsive even under significant load and capacity stress.

docs/requirements/nfr-4-usability-and-user-experience.md

@@ -0,0 +1,90 @@
+# NFR-4: Usability & User Experience
+
+## 1. Overview
+
+This document defines the non-functional requirements related to usability and user experience. The focus is on delivering an intuitive, accessible, and efficient interface—initially through a command-line interface (CLI) with future plans for a web-based solution—that empowers users to easily manage file operations, view system status, and interact with access control and synchronization features.
+
+## 2. Objectives & Scope
+
+### 2.1 Objectives
+
+- **Intuitive Interface:**
+  Design user interfaces that are simple to use, with clear instructions, feedback, and error messages.
+
+- **Efficient User Interactions:**
+  Minimize the number of commands or steps required for common tasks such as uploading, syncing, and viewing logs.
+
+- **Consistency Across Devices:**
+  Ensure that user experience is consistent across multiple devices and interface modalities.
+
+- **Accessibility:**
+  Provide support for users across different skill levels, from advanced CLI users to those preferring graphical interactions.
+
+### 2.2 Scope
+
+- Applies to both the CLI and any future web or graphical user interfaces.
+- Covers feedback on system operations such as file encryption, chunking, replication, and synchronization status.
+- Encompasses features for troubleshooting, help documentation, and real-time notifications.
+
+## 3. Detailed Requirements
+
+### 3.1 User Interface Design
+
+- **Clarity and Simplicity:**
+
+  - Commands and outputs must be clear and concise, with appropriate use of color, formatting, and context-sensitive help.
+  - Error messages should be descriptive and recommend corrective action.
+
+- **Feedback Mechanisms:**
+  - Real-time indicators for operation progress (e.g., progress bars, status notifications).
+  - Notification system for significant system events, including access changes, replication status, and errors.
+
+### 3.2 Accessibility and Adaptability
+
+- **Multi-Modal Interfaces:**
+
+  - Initially provide a robust CLI with comprehensive help menus and usage documentation.
+  - Design the system architecture to support future development of a web-based UI with consistent functionality.
+
+- **User Customization:**
+  - Allow users to configure settings (e.g., verbosity of logs, notification preferences) to suit their expertise and preference.
+
+### 3.3 Usability Testing and Iteration
+
+- **Feedback Loop:**
+
+  - Conduct user testing sessions to gather feedback on ease-of-use, clarity, and navigability.
+  - Iterate on the design based on user feedback and usability metrics.
+
+- **Documentation and Training:**
+  - Provide thorough documentation, tutorials, and usage examples to help users quickly learn the system’s features.
+
+## 4. Measurable Criteria & Test Cases
+
+### 4.1 Usability Studies
+
+- Conduct surveys and usability tests with target users and measure satisfaction scores.
+- Quantify the average time required for users to complete common tasks (e.g., uploading a file, changing access rights).
+
+### 4.2 Benchmark Tests on Interface Responsiveness
+
+- Measure the time from issuing a command to receiving a response, ensuring it meets target performance thresholds.
+- Validate that real-time operation feedback (e.g., progress bars) updates without perceptible lag.
+
+### 4.3 Accessibility and Consistency Checks
+
+- Test across different devices and terminal types to ensure consistency.
+- Validate that error messages and help documentation are readily accessible and clear.
+
+## 5. Dependencies & Integration Points
+
+- **CLI Framework:**
+  - Relies on a well-designed command-line framework that supports text formatting and real-time feedback.
+- **Backend Performance:**
+  - Must integrate with NFR-2 (Performance & Responsiveness) to ensure that UI response times are not adversely affected during heavy operations.
+- **Documentation Systems:**
+  - Integration with help and documentation systems to provide contextual guidance.
+
+## 6. Conclusion
+
+NFR-4 ensures that the system offers an excellent user experience, characterized by an intuitive interface, rapid feedback, and consistent operation across platforms. By focusing on clarity, efficiency, and accessibility, the system will empower users to manage their data securely and confidently with minimal friction.

docs/requirements/nfr-5-audibility-and-transparency.md

@@ -0,0 +1,95 @@
+# NFR-5: Auditability & Transparency
+
+## 1. Overview
+
+This document defines the requirements necessary to guarantee that all system operations are fully auditable and transparent. The objective is to build trust in the system by ensuring that every user action, file operation, and inter-node communication is logged in a tamper-evident manner. These logs provide a verifiable history for forensic analysis, regulatory compliance, and user accountability.
+
+## 2. Objectives & Scope
+
+### 2.1 Objectives
+
+- **Immutable Logging:**
+  Record all critical operations in an append-only, tamper-evident log.
+
+- **Action Traceability:**
+  Ensure that every action, permission change, or file operation is traceable to its origin (user, node, timestamp).
+
+- **User Transparency:**
+  Allow users to access audit logs to review system activity and verify that operations have been executed correctly.
+
+- **Forensic Readiness:**
+  Provide detailed logs that support forensic investigation and compliance with security standards.
+
+### 2.2 Scope
+
+- Applies to file operations (upload, download, encryption, decryption), replication events, access control changes, and inter-node communications.
+- Encompasses logging mechanisms across client-side and server/node operations.
+- Integrates with the immutable manifest design (ADR-0006) to ensure cryptographic chaining of logs.
+- Covers both real-time monitoring and historical log retrieval capabilities.
+
+## 3. Detailed Requirements
+
+### 3.1 Logging Mechanisms
+
+- **Append-Only Log Structure:**
+
+  - Every critical operation must be logged with relevant details (timestamp, actor, operation details, outcomes).
+  - Logs must be written in an immutable, append-only format and protected by cryptographic chaining.
+
+- **Granularity of Logging:**
+  - Define the level of detail required for each type of operation (e.g., file transfers, access changes, replication events).
+  - Logs should include sufficient metadata to allow comprehensive reviews without revealing sensitive data.
+
+### 3.2 Transparency and User Access
+
+- **Audit Log Retrieval:**
+
+  - Provide interfaces (CLI and future web UI) for users to query and review audit logs.
+  - Allow users to filter logs by date, operation type, or actor.
+
+- **Real-Time Alerts:**
+
+  - Implement real-time monitoring that triggers alerts for suspicious or critical operations (e.g., unexpected access revocations, repeated failures).
+
+- **Compliance Reporting:**
+  - Support export formats for audit logs to facilitate external compliance reviews and security audits.
+
+### 3.3 Integration with Other Modules
+
+- **Immutable Manifest Integration:**
+
+  - Integrate logging with the manifest to ensure that every change is recorded and verifiable.
+  - Maintain consistency between audit logs and manifest snapshots for integrity.
+
+- **Security and Key Management:**
+  - Ensure that logs related to sensitive operations (e.g., key changes) are encrypted and access-controlled.
+
+## 4. Measurable Criteria & Test Cases
+
+### 4.1 Log Integrity Verification
+
+- Regularly perform cryptographic verification of log chains to confirm that no tampering has occurred.
+- Simulate unauthorized modifications to verify that the system detects breaches in log integrity.
+
+### 4.2 User Audit Access Testing
+
+- Conduct tests where users retrieve logs and verify that filtering and search functions return accurate results.
+- Measure the performance of log retrieval operations under various loads.
+
+### 4.3 Forensic Readiness Tests
+
+- Test the export functionality to ensure that logs can be exported in standard, compliant formats.
+- Validate that all audit events include complete metadata for forensic analysis.
+
+## 5. Dependencies & Integration Points
+
+- **Immutable Manifest System:**
+  - Critical reliance on ADR-0006 for cryptographic chaining and tamper-evidence.
+- **Access Control Systems:**
+  - Integration with FR-6 and other access management functionalities to log all permission changes.
+- **Monitoring Tools:**
+  - Use of real-time monitoring systems to trigger alerts and collect log data continuously.
+
+## 6. Conclusion
+
+NFR-5 ensures that all system activities are transparent, traceable, and verifiable, providing a robust audit trail critical to user trust and regulatory compliance. Through immutable logging, detailed action traceability, and user-accessible audit reports, the system will be capable of robust forensic analysis and continuous security monitoring.

docs/requirements/nfr-6-deployability-and-maintainability.md

@@ -0,0 +1,102 @@
+# NFR-6: Deployability & Maintainability
+
+## 1. Overview
+
+This document outlines the requirements for ensuring that the encrypted file storage system is easy to deploy, configure, and maintain in diverse environments. It focuses on strategies such as containerization, configuration automation, and deterministic testing environments to simplify operations and updates, while ensuring consistent behavior across different deployment scenarios.
+
+## 2. Objectives & Scope
+
+### 2.1 Objectives
+
+- **Simplified Deployment:**
+  Ensure that the system can be deployed with minimal manual configuration using standardized deployment packages.
+
+- **Ease of Maintenance:**
+  Provide robust mechanisms for system updates, monitoring, and troubleshooting that reduce downtime and intervention effort.
+
+- **Deterministic Testing:**
+  Enable deterministic simulation of deployment scenarios (including network and disk failures) to ensure behavior consistency and ease of reproducibility.
+
+- **Modular and Extensible Architecture:**
+  Ensure that system components are modular to facilitate easy updates and integration of new features.
+
+### 2.2 Scope
+
+- Covers the full lifecycle of deployment: configuration, installation, operation, and maintenance.
+- Encompasses deployment on self-hosted servers, containerized environments (e.g., Docker), and orchestration platforms.
+- Includes integration with automated configuration management and deterministic simulation frameworks.
+
+## 3. Detailed Requirements
+
+### 3.1 Containerization and Standardized Deployment
+
+- **Containerization:**
+
+  - Package all components as Docker containers to ensure a consistent runtime environment.
+  - Provide container orchestration templates (e.g., Kubernetes manifests) for scalable deployments.
+
+- **Configuration Management:**
+  - Develop configuration templates with support for environment variables and secure secrets management.
+  - Automate deployment procedures via scripts or configuration management tools to minimize manual intervention.
+
+### 3.2 Maintenance and Updatability
+
+- **Graceful Updates:**
+
+  - Implement rolling update mechanisms to allow seamless system patching without significant downtime.
+  - Ensure that updates are backward compatible or provide clear migration paths.
+
+- **Monitoring and Diagnostics:**
+  - Integrate diagnostic tools and monitoring dashboards to track system health, performance metrics, and error logs.
+  - Provide detailed logs and alerts as part of the maintenance interface.
+
+### 3.3 Deterministic Simulation for Deployment
+
+- **Testing Deployment Scenarios:**
+
+  - Integrate the deployment process with a deterministic simulation environment to test behavior under simulated adverse conditions (e.g., partial network failures, disk errors).
+  - Validate that the system recovers gracefully from simulated deployment faults.
+
+- **Automated Rollback and Recovery:**
+  - Define procedures for automated rollback if an update or deployment fails in the simulated test.
+  - Maintain logs of deployment events for auditing and troubleshooting.
+
+### 3.4 Documentation and Support
+
+- **Comprehensive Documentation:**
+
+  - Provide detailed deployment instructions and troubleshooting guides.
+  - Ensure that documentation is updated alongside system updates to reflect new procedures or changes.
+
+- **Community and Vendor Support:**
+  - Establish support channels (e.g., forums, issue trackers) for deployment issues to facilitate rapid resolution.
+
+## 4. Measurable Criteria & Test Cases
+
+### 4.1 Deployment Automation Tests
+
+- Verify that containers can be deployed using provided scripts/templates across multiple platforms.
+- Test automated configuration and secrets management to ensure that deployments require minimal manual intervention.
+
+### 4.2 Update and Rollback Testing
+
+- Simulate rolling update procedures and measure downtime or service interruptions.
+- Conduct rollback tests after simulated deployment failures to confirm that the system returns to a stable state.
+
+### 4.3 Simulation of Fault Conditions
+
+- Utilize the deterministic simulation framework to introduce network or disk failures during deployment.
+- Validate that the system’s recovery mechanisms trigger as expected and that comprehensive logs are maintained for analysis.
+
+## 5. Dependencies & Integration Points
+
+- **Containerization Infrastructure:**
+  - Relies on Docker and orchestration platforms (e.g., Kubernetes) for deployment consistency.
+- **Configuration Management Tools:**
+  - Integration with tools such as Ansible, Helm, or similar to automate deployments.
+- **Deterministic Simulation Framework:**
+  - Essential to validate behavior under simulated fault conditions and to document recovery procedures.
+
+## 6. Conclusion
+
+NFR-6 ensures that the system is easily deployable, maintainable, and updatable in diverse operational environments. By leveraging containerization, automated configuration management, and deterministic simulation testing, the deployment process is streamlined, predictable, and resilient to common failure scenarios. This allows system administrators and users to focus on leveraging the storage solution without being burdened by complex deployment procedures.