Detail Whitepaper GitHub Comment Analysis

WebSite	https://storj.io/
GitHub	storj/storj
Update Date	2018 Aug 31 10:08:50

Circulating Supply	135,787,440
Total Supply	425,000,000
Max Supply	0
Price	$0.26265272
Volume 24h	$1,256,989
Market Cap	$35,664,940
Change 24h	-8.22%
Update Date	September 26th 2018, 1:58:10 am

v1

Storj is a protocol that creates a distributed network for the formation and execution of storage contracts between peers. The Storj protocol enables peers on the network to negotiate contracts, transfer data, verify the integrity and availability of remote data, retrieve data, and pay other nodes. Each peer is an autonomous agent, capable of performing these actions without signiﬁcant human interaction. Many of the basic tools for these interactions are described in this document. Full protocol documentation can be found elsewhere [1].
2.1 Files as Encrypted Shards
A shard is a portion of an encrypted ﬁle to be stored on this network. Sharding has a number of advantages to security, privacy, performance, and availability.
Files should be encrypted client-side before being sharded. The reference implementation uses AES256-CTR, but convergent encryption or any other desirable system could be implemented. This protects the content of the data from the storage provider, or farmer, housing the data. The data owner retains complete control over the encryption key, and thus over access to the data.
The data owner may separately secure knowledge of how a ﬁle is sharded and where in the network the shards are located. As the set of shards in the network grows, it becomes exponentially more diﬃcult to locate any given shard set without prior knowledge of their locations (see Section 6.3). This implies that security of the ﬁle is proportional to the square of the size of the network.
Shard size is a negotiable contract parameter. To preserve privacy, it is recommended that shard sizes be standardized as a byte multiple, such as 8 or 32 MB. Smaller ﬁles may be ﬁlled with zeroes or random data. Standardized sizes dissuade side-channel attempts to determine the content of a given shard, and can mask the ﬂow of shards through the network.
Sharding large ﬁles like video content and distributing the shards across nodes reduces the impact of content delivery on any given node. Bandwidth demands are distributed more evenly across the network. In addition, the enduser can take advantage of parallel transfer, similar to BitTorrent [2] or other peer-to-peer networks.
Because peers generally rely on separate hardware and infrastructure, data failure is not correlated. This implies that creating redundant mirrors of shards, or applying a parity scheme across the set of shards is an extremely eﬀective method of securing availability. Availability is proportional to the number of nodes storing the data.

2.2 Kademlia and Modiﬁcations

2.2.1 Signature Veriﬁcation

2.3 Proofs of Retrievability

Proofs of retrievability guarantee the existence of a certain piece of data on a remote host. The ideal proof minimizes message size, can be calculated quickly, requires minimal pre-processing, and provides a high degree of conﬁdence that the ﬁle is available and intact. To provide knowledge of data integrity and availability to the data owner, Storj provides a standard format for issuing and verifying proofs of retrievability via a challenge-response interaction called an audit or heartbeat.
Our reference implementation uses Merkle trees [5] and Merkle proofs. After the sharding process the data owner generates a set of n random challenge salts s0,s1,...sn−1 and stores the set of salts s. The challenge salts are each prepended to the data d, and the resulting string is hashed to form a pre-leaf p as such: pi = H(si + d). Pre-leaves are hashed again, and the resulting digests become the set of leaves l of a standard Merkle tree such that li = H(H(si + d)). The leaf set is ﬁlled with hashes of a blank string until its cardinality is a power of two, to simplify the proof process.

The data owner stores the set of challenges, the Merkle root and the depth of the Merkle tree, then transmits the Merkle trees leaves to the farmer. The farmer stores the leaves along with the shard. Periodically, the data owner selects a challenge from the stored set, and transmits it to the farmer. Challenges may be selected according to any reasonable pattern, but should not be reused. The farmer uses the challenge and the data to generate the pre-leaf. The preleaf, along with the set of leaves, is used to generate a Merkle proof, which is sent back to the data owner.
The Storj Merkle proof always consists of exactly log2(|l|) + 1 hashes, and thus is a compact transmission, even for large trees. The data owner uses the stored Merkle root and tree depth to verify the proof by verifying that its length is equal to the tree depth and the hashes provided recreate the stored root. This scheme does not allow false negatives or false positives, as the hash function requires each bit to remain intact to produce the same output.

Storj is exploring a reputation system that leverages public Blockchains to solve a narrow set of identity problems [28]. This system requires nodes to spend money directly to maintain reputation. Nodes invest in their identity over time by making small standardized payments to their own Storj network node ID. Because the ID is a Bitcoin address to which the node holds the private key, these funds are fully recoupable, except for the miners fees. In this system, nodes prefer to interact with nodes that have a long history of regular transactions. Over time these indicate monetary investment in an identity equal to the sum of the miners fees paid.

The payment required to participate in this system should be signiﬁcantly less than the expected return of operating a network node. If set correctly, this recurring monetary payment for an identity bounds the size and duration of Sybil attacks without aﬀecting cooperative nodes. Legitimate nodes would easily recoup their identity expense, while Sybil operators would ﬁnd their expenses outstripping their returns. Unfortunately, this approach solves a relatively small subset of identity issues on the network, and it is diﬃcult to see how it could be extended to other problem sets.

Permissioning in this system may be managed by a service like a Bridge, tied to a web of trust identity via services like Keybase, or handled by a distributed self-sovereign identity system. Smart contract systems, e.g. Ethereum [22] contracts, seem like a sensible long-term choice, as they can provide ﬁle permissions based on arbitrary code execution. Some problems may exist with respect to management of the private information required for identity and permissioning systems, but suﬃcient solutions likely exist.

The Bridge API uses public-key cryptography to verify clients. Rather than the Bridge server issuing an API key to each user, users register public keys with the Bridge. API requests are signed, and the Bridge veriﬁes that the signature matches a registered public key. Bridge organizes ﬁle metadata into buckets to facilitate management. Buckets can be permissioned individually by registering a set of public keys to the Bucket.
Application developers can use this to easily delegate permissions to applications, servers, or other developers. For instance, the developer of a ﬁle syncing service could create a keypair for each user of that service, and divide each user into a separate Bucket accessible only by that users keypair. Usage of each Bucket is tracked separately, so users who have exceeded their allotment could have write permissions revoked programmatically. This provides a logical separation of user permissions, as well as a variety of organizational tools.

5 Attacks
As with any distributed system, a variety of attack vectors exist. Many of these are common to all distributed systems. Some are storage-speciﬁc, and will apply to any distributed storage system.

5.4 Hostage Bytes
The hostage byte attack is a storage-speciﬁc attack where malicious farmers refuse to transfer shards, or portions of shards, in order to extort additional payments from data owners. Data owners should protect themselves against hostage byte attacks by storing shards redundantly across several nodes (see Section 2.7). As long as the client keeps the bounds of its erasure encoding a secret, the malicious farmer cannot know what the last byte is. Redundant storage is not a complete solution for this attack, but addresses the vast majority of practical applications of this attack. Defeating redundancy requires collusion across multiple malicious nodes, which is diﬃcult to execute in practice.
5.5 Cheating Owner
A data owner may attempt to avoid paying a farmer for data storage by refusing to verify a correct audit. In response the farmer may drop the data-owners shard. This attack primarily poses a problem for any future distributed reputation system, as it is diﬃcult for outside observers to verify the claims of either party. There is no known practical publicly veriﬁable proof of storage, and no known scheme for independently verifying that a privately veriﬁable audit was issued or answered as claimed. This indicates that a cheating client attack is a large unsolved problem for any reputation system.

Storjcoin allows much more granular payments than other candidate currencies, thereby minimizing trust between parties. In addition, the mechanics of micropayment channels require the total value of the channel to be escrowed for the life of the channel. This decreases currency velocity, and implies that value ﬂuctuations severely impact the economic incentives of micropayment channels. The use of a separate token creates a certain amount of insulation from outside volatility, and Storjcoin’s large supply minimizes the impact of token escrow on the market.

In cases where the cost of delegating trust is not excessively high, clients may use third-party Bridges. Because Bridges do not store data and have no access to keys, this is still a large improvement on the traditional data-center model. Many of the features Bridge servers provide, like permissioning and intelligent contracting, leverage considerable network eﬀects. Data sets grow exponentially more useful as they increse in size, indicating that there are strong economic incentives to share infrastructure and information in a Bridge.

While this system represents a signiﬁcant step up in both usability and value, there are unmitigable security issues. Unfortunately, as in any cryptographic system, it is impossible to revoke access to data. Applications may cache data or forward it to third parties. Users, by deﬁnition, trust application developers to handle their data responsibly. To mitigate these risks, Storj Labs intends to provide incentives to developers to build free and open-source software. No application can be completely secure, but auditable code is the best defense of users privacy and security.

By encouraging use of data format and access standards, Storj aims to allow portability of data between applications. Unlike a traditional model, where control of data is tied to the service used to access the data, data access may be tied to individual users because Storj forms a common underlying layer. User data can be tied to persistent cryptographic identities, and authenticated without exposing data to third parties. Siloing data in applications is a harmful relic of traditional models. Building cross-compatibility into the future of data storage greatly improves user privacy and user experience.
Applications implementing these standards would be broadly compatible. When access is tied to users rather than services, privacy and control are preserved. A user may grant access to a service that backs up their hard drive, which places those ﬁles in Storj. The user could separately grant access to a photo-sharing service, which could then access any photos in the backup. The user gains seamless portability of data across many applications, and application developers gain access to a large pool of existing users.

Permissioning in this system may be managed by a service like a Bridge, tied to a web of trust identity via services like Keybase, or handled by a distributed self-sovereign identity system. Smart contract systems, e.g. Ethereum [22] contracts, seem like a sensible long-term choice, as they can provide ﬁle permissions based on arbitrary code execution. Some problems may exist with respect to management of the private information required for identity and permissioning systems, but suﬃcient solutions likely exist.
While this system represents a signiﬁcant step up in both usability and value, there are unmitigable security issues. Unfortunately, as in any cryptographic system, it is impossible to revoke access to data. Applications may cache data or forward it to third parties. Users, by deﬁnition, trust application developers to handle their data responsibly. To mitigate these risks, Storj Labs intends to provide incentives to developers to build free and open-source software. No application can be completely secure, but auditable code is the best defense of users privacy and security.
The potential advantages in terms of user experience and privacy are great, but more research is needed. Many open questions exist with respect to permissioning mechanisms. At worst a uniﬁed backend powering interoperable applications provides equivalent security to current data-center based models. Storj hopes to collaborate with other forward-thinking data-driven projects to create and advocate for these open standards.