Decentralized Storage: Where Web 3.0 Meets Metaverse
Produced by: LD Capital Research
Author:Betty, YY
Research Associates: James Kuo, Noise Zhou, Lightmanben
Translation: Lightmanben, Betty
TL;DR
- The action of shifting physical life online and user-generated content (UGC) in full swing creates massive demand for data storage with significant commercial and application prospects.
- Due to the high cost and factors such as pre-required storage settings, it is difficult for centralized storage to keep up with the changes in capacity, read/ write speed, security, and data relationship brought by the exponential growth of online data. Therefore, decentralized storage came into being.
- Decentralized storage is based on blockchain technology and has the characteristics of solider scalability, higher security, higher efficiency, automatic fault tolerance, higher reliability, and lower cost.
- For Web 3.0, users’ massive online footprints and online contents generate incredibly numerous data that depends on a reliable, stable, and secure data storage network.
- In Metaverse, the monetized virtual asset will become a worthless cheque with no redemption if it loses protection from a trusted blockchain storage and its corresponding metadata becomes invalid or tampered with.
- The current decentralized storage solutions are IPFS/ Filecoin and Arweave. The IPFS innovatively introduces the incentive layer Filecoin to ensure reliable data storing within the agreed period, and the Arweave achieves permanent storage for on-chain data through its technological innovation.
1.Introduction
In the summer of 2021, a set of strange-looking figures composed of 10,000 irregular pixels suddenly became popular. What surprised the market was that any single of these figures could be sold in ETH that was worth tens of millions of dollars. Since then, similar items as Bored Ape Yacht Club and Loot have become the new darling of the crypto market. From crypto art and games to avatars and texts, the rise of NFTs has ignited people’s endless imagination of the metaverse and Web 3.0.
However, when Cryptopunks inspire us as the status symbol, enthusiastically play-to-earn games as Axie Infinity, and the promising future of metaverse, have the thought of where the data would be stored ever crossed your mind? Do you worry about any tampering of the stored data? No one wants to spend tens of millions of dollars on avatars and end up with worthless assets due to the tampering of the underlying data. No one hopes the precious memories of interacting with friends in the metaverse disappeared forever due to the system outage. At the starting point of the metaverse and Web 3.0 and the explosive growth in data, we urgently need a safe, open, free, and reliable storage network.
This article will walk you through the profound meaning of storage in Web 3.0 and metaverse and understand the exciting applications of decentralized storage in the ecosystem. Also, we will demonstrate the study and comparison between different decentralized storage solutions based on the current situation.
2. Industrial Ecology of Storage
The widespread Covid 19 pandemic has accelerated the boundary-blurring between the physical world and the virtual one. People have shifted their activities online from the physical world regarding recruitment, social networks, entertainment, dining, shopping, traveling, etc. At the same time, from Web 1.0 to Web 3.0, data production has exploded and will continue with exponential growth due to the shifting from passive data browsing to enthusiastic content generation. According to industry research conducted by 360. cn, the global digital content creation market is expected to reach a value of US$1.69 billion in the year 2025. Together with the full swing of UGC model and the exponential growth in data lead to massive data storage needs. The development of chip technology, algorithm, and hardware has promoted the advancement of storage technology; The unlocked value of data and rigid demand of data security have forced the innovation and infrastructure construction for data storage. In the iteration and integration of software and algorithms, the storage industry has gradually evolved from traditional disk storage to cloud data storage beyond the hardware’s limitation. As one of the pillars of blockchain technology (that refers to computation, storage, and network), storage is significant for the development of the Metaverse. From the application perspective, the stable operation of the virtual world relies on the ultimate affiliation relationship between data security and data sovereignty; From the economic point of view, there are substantial commercial prospects in the massive data storing demand generated by the Metaverse and Web 3.0.
Due to various factors, we currently cannot assess the overall storage market size. However, taking cloud storage as an example, it is estimated that by 2027 the global cloud storage market will reach US$222 billion with a compound annual growth rate of 21.9% during the forecasted period ( Sneha Korad Rachita Rake & Vineet Kumar, 2021 ).
The business model of data storage is mainly divided into centralized and decentralized storage. Centralized storage stores the data entirely on the centralized server, and the decentralized one uses distributed storage technology to store sliced data in multiple independent storage providers. Centralized data storage meets the enterprise-level storage need with its high stability and low cost; Decentralized storage is preferred by the long-tail market for its customized storage solutions, economic incentive models, and strong privacy to fulfill the particular requirements of these firms. The importance of decentralized storage has become more significant in both Web 3.0 and the Metaverse, raising awareness of data security and user data ownership.
2.1 Centralized Storage
Decentralized storage is from the era of the mainframe that the entire storage is concentrated on multiple sets of devices in a single system with a fixed number of nodes. Because data only passes through a controller, decentralized storage has the advantage of low latency but relatively high costs, inadequate security, and low scalability of the service providers. One critical factor to keep the smooth operation of the centralized storage system is the stability of the storage server. Therefore, it has become the bottleneck of the storage system performance and the focus point of reliability. The stability issue puts high requirements on the storage environment and hardware. Thus the market is monopolized by tech giants like AWS, Azure, Google Cloud, and Alibaba Cloud, which account for 67% of the total market share in cloud storage.
With the popularity of UGC and the development of social networking platforms like Weibo and Facebook, we have entered an era of exponential data growth. The explosive growth of data raises the higher requirements on storage capacity, expansion speed, and data backup ability; The real-time interaction between users requires faster read/ write speed and higher security. At the same time, because of the variety and complicity of UGC data and its interrelationship, the storage system needs to transfer the model from a single file type to various semi-structured and unstructured data relationship models. Decentralized storage has emerged due to the high infrastructure cost and the entry barrier of centralized storage. These features make the centralized approach challenging to keep up with data production in capacity and data types.
2.2 Decentralized storage
Just like Feitong Hu, the early evangelist of IPFS (InterPlanetary File System), said, “In the world of centralized data storage, users not only pay for data storage but also do not get high-quality data protection. By the decentralized method, users can manage their data, encrypt stored data, authorize access to data, and backup the storage. The features are expected to greatly ensure user privacy, enhance long-term data storage, and reduce costs through the sharing model. It is the inevitable phase of data storage development.”
Decentralized storage is based on blockchain technology using open-source applications and algorithms to store sliced data to multiple independent network nodes. It advocates privacy protection, data standby redundancy, and value-oriented data by providing network nodes and content uploaders with incentives. Among the mentioned features, the incentive model is the finishing touch of decentralized storage because it allows data to be kept long-term and secured.
The integration of decentralized storage with blockchain technology provides strong scalability, high security, high efficiency, automatic fault tolerance, high reliability, and low costs.（ Shike Jiao, 2020）
Scalability: The number of storage nodes can be expanded indefinitely, and the storage capacity of each node can also be flexibly changed.
Security: The stored data is divided into fragments and allocated in multiple nodes without knowing the service providers or seeking the trust of the third parties. The data encryption process is not limited to the end-users and software but also occurs in all links of the storage networks by methods such as zero-knowledge proof and private network keys access.
High efficiency: Nodes on the same network can directly share files.
Automatic fault tolerance: because stored data has been distributed to numbers of nodes in the network, the multiple copies of data come in handy when data transmission or storage errors occur.
High reliability: The storage system contains a verification mechanism that ensures the stored files are complete and truthful, with users’ 24–7 access.
Low cost: It is advised that iQiyi will have to pay $1 million to the traditional storage provider for storing a single episode of The Rap of China at the cost of $0.001/ GB. By contrast, the IPFS distributed transmission can save 60% of bandwidth cost.(Shike Jiao, 2020 ）
3.Decentralized storage: Where Web 3.0 Meets Metaverse
3.1 Decentralized storage and Web 3.0
The origin of the Internet can be traced back to the innovation of the World Wide Web in 1989. If the website portal opened the era of Web 1.0, then the rise of blogs in 2003 had marked the beginning of Web 2.0. The transformation from Web 1.0 to Web 2.0 indicates the passive information receiving is gradually replaced by self-created content and online interactions. These changes shift Internet users’ attention to the application experience such as UGC, usability, and interoperability and eventually bring Web 3.0 to life. As Timothy Berners-Lee said, “People keep asking what Web 3.0 is. I think maybe when you’ve got an overlay of scalable vector graphics — everything rippling and folding and looking misty — on Web 2.0 and access to a semantic Web integrated across a huge space of data, you’ll have access to an unbelievable data resource”. （Victoria Shannon, 2006）Although we have not yet reached a consensus on the Web 3.0 definition, it should have the following characteristics（ Fabric Venture, 2020）:
Open: Open-source software is built by the open and accessible developer community, and operations performed on the chain can be viewed by others.
Trustless: the network allows interacting publicly or privately without the trusted third party. Web 3.0 ensures the participants follow the rules by adopting a decentralized incentive mechanism. Web 3.0 also verifies the authenticity of the account and the effectiveness of the trade transfer by the consensus mechanism of the blockchain protocol to achieve the disintermediation of value transfer.
Permissionless: Either users or storage providers can participate without the authorization of the administration.
Web 3.0 is blurring the boundary of the digital and physical world. The massive online users’ digital footprints and the online content creations require a stable and reliable data storage network behind the scenes. Otherwise, what has happened in Web 3.0 will end up being useless data fragments.
3.2 Decentralized Storage and Metaverse
In 2021, with the listing of Roblox, the concept of metaverse suddenly stormed the major industries such as the Internet, VR/AR, and financial investment. People have come to the edge of the metaverse by developing microcomputers, VR equipment, and high-speed network technology represented by 5G. As our last research report, The Metaverse Overview: From the Past to the Future, stated, the finishing touch of the metaverse is its economic system. First, the open payment and settlement system based on the blockchain can meet real-time, fair, transparent, and fast P2P payment requirements. Second, monetizing the virtual items or creations of users in metaverse provides users the protection of rights and interests similar to those in the physical world and promotes the circulation and transaction of assets in metaverse further to encourage the accumulation of wealth by user innovations. Behind it all, the monetized virtual asset will become a worthless check with no acceptance if it loses the protection from the trusted blockchain storage and its corresponding metadata becomes invalid or tampered with.
3.3 Major Decentralized Storage Solutions
3.3.1 Bittorrent
BitTorrent, the so-called BT download, was the earliest decentralized transmission solution proposed by software engineer Bram Cohen in 2003. BitTorrent overcomes the limitations of the traditional download approach by P2P download. It did not require the content resource publisher to own the high-performance server to transfer the data, and the more users downloading the same file, the faster the download speed can be. Additionally, the free-of-charge model attracted a vast number of netizens to use. However, BitTorrent must use the torrent file containing all targeted content addresses to perform the download. The downloading content is strictly restricted within the scope of the torrent file. But the torrent file will reduce or disappear over time due to losing relevant data content in spreading.
Furthermore, BitTorrent lacked incentives to motivate users to share unpaid files. In the absence of the appropriate incentive mechanism, only those popular contents that are widely spread and constantly downloaded could exist. Those equally important contents that failed to attract the general public might disappear in the long history.
3.3.2 IPFS/ Filecoin
IFPS (InterPlanetary File System) is another decentralized storage solution proposed in 2015. It overcomes the shortcoming of BitTorrent by creating multiple versions of the file, and pre-downloading content to achieve decentralized, fast, high efficiency, reliable and secure content storage method. Notably, IPFS introduces its incentive layer, Filecoin, a groundbreaking approach to ensure reliable data storing within the agreed period.
Filecoin allows any individual to provide data storage and retrieval service, monetizing their idle hard drive and bandwidth capacity. And it adopts a mechanism of Proof-of-Replication (PoRep) and Proof-of-Spacetime (PoSt) to prove the condition of the stored data and prevent attacks such as Sybil attacks, outsourcing, and generation attacks. In short, PoRep verifies the integrity of the stored data, and PoSt audits the storage offered by miners at the time of response. The operating principle of Filecoin is shown below: a client finds a suitable storage miner by issuing order information, then the cryptographically sealed and sliced data is sent to the miner. Finally, the miner adds data to sectors and continuously proves the targeted content is stored to gain rewards.
The business model of Filecoin, of course, is not perfect. It requires professional equipment, GPU mining machines to participate. The higher mining cost and entry barrier make ordinary people unable to provide storage directly, and the entire storage network will be occupied by a small number of giant storage providers and large-scale mining pools. IPFS’s ideally decentralized storage solution has once again centralized to some extent.
Filecoin launched its mainnet in October 2020, currently with 3,175 active nodes and mainly distributed in North America, Europe, China, and Korea. As of 14th Oct 2021, the sum of the effective computing power is 12.393 EiB, its base fee is 0.3591 nanoFIL and the cost of sealing 32G sector is 4.958 Fil/TiB (data from filscout.com).
With the improvement of Filecoin’s underlying network and infrastructure, the early dividends of its network effect have attracted numbers of applications to participate. As of now, Filecoin has established an ecosystem covering fields of “Consumer Storage Applications, Data Marketplace, Data Retrieval, NFT, Defi, Verifiable Storage and Privacy” that includes projects like Fileswan, NFT Storage, Opera, and Livepeer.
3.3.3 Arweave
Unlike Filecoin’s off-chain data storage, the novel solution approach of Arweave stores data directly in the blockchain, focusing on permanent storage and on-chain decentralized storage. The framework of Arweave is shown below: the underlying blockchain network Blockweave and the upper layer Permaweb based on HTTP. The highlights of Arweave is its underlying Blockweave network:
Blockweave:The project introduces a brand new blockchain network, Blockweave, based on the consensus mechanism and particular block generation mode. It generates tables for all previous blocks’ hash values and active wallets for miners later to verify works instead of verifying the entire node as BTC and ETH. This approach reduces energy consumption and improves node efficiency.
Incentive Algorithm, Wildfire:Unlike Filecoin needs to charge storage fees after the agreed storing period continuously, Arweave only charges a one-time permanent fee when data is stored. After that, the users are free to read data later. To encourage nodes to respond to data read requests, Arweave introduced the incentive algorithm wildfire. It is a node ranking system based on the response speed and amount of accepting data. The more reliable data obtained from the adjacent node, the higher score is given and the greater probability of block generation.
SmartWeave:Instead of requiring from network nodes, SmartWeave now requires users to execute the smart contract. In this model, the Arweave network is performed as a generic data consensus and sharing layer, and it helps to enable computation-heavy Dapps on top of the Arweave network.
Although the Arweave storage solution stores data on-chain to guarantee the non-tampering, permanent storage, and public access of the data, the model has its drawback. Firstly, the data cannot be tempered, which means the only way to modify data is by updating a new version. While this action will add up the iteration cost. Secondly, to ensure the permanent storage of data, Arweave needs to continuously attract participants of new nodes while keeping marginal benefit exceeding the marginal cost. However, the total number of nodes has been decreasing. There are currently 146 nodes, reduced by 70% compared with 490 nodes in May 2020.
Arweave launched its mainnet on January 29th, 2019, and published Arweave 2.0 with the bundled transactions function in May 2020. The on-chain transaction volume boosted from the start of 2021 and reached its peak in September (over 80,000 for the average daily transaction). The launch of Metaplex Candy Machine on Solana, the release of cross-chain bridge ipfs2arweave, etc. had made the brilliant performance of Arweave.
Across projects in the Arweave ecosystem, the most active ones are in NFT, Defi, and infrastructure fields like Koii, Limestone. The Arweave ecosystem is still in its early stage of construction.
4 Outlook
Decentralized storage has become more significant for Web 3.0 and Metaverse as there is raising awareness of data security and user data ownership. Decentralized storage differs from centralized storage in multiple perspectives: storage source, bandwidth source, security, utility, etc. These factors cause the scale of decentralized storage currently far smaller than the centralized one. Projects like IPFS and Arweave have overcome the shortage of decentralized storage by introducing different incentive mechanisms to expand the total network computation power. However, facts such as the cost structure and the inconsistent interests between the project team and users prevent its rapid growth. The decentralized storage ecosystem is still in the early stage, and it is necessary to balance the cost, the number of nodes, and the number of users to consider its performance and scale.
We look forward to the future of decentralized storage in Metaverse and Web 3.0. The flywheel effect of the networks will boost the surge of applications in decentralized storage. With improvements in storage technology and optimized incentive models, the decentralized storage industry will finally make significant and stable progress.
5 Reference
- Cloud Storage Market (2021), Sneha Korad Rachita Rake , Vineet Kumar. www.alliedmarketresearch.com/cloud-storage-market
- Distributed Storage, the battle of Blue Ocean in hundreds of billions Worth Storage Market. www.8btc.com/article/613165
- Distributed Storage Industry Chain Research Report in 2020, Shike Jiao. www.jinse.com/news/blockchain/835473.html
- A ‘more revolutionary’ Web(2006), Victoria Shannon,www.nytimes.com/2006/05/23/technology/23iht-web.html
- What Is Web 3.0 & Why It Matters （2020），Fabric Venture. medium.com/fabric-ventures/what-is-web-3–0-why-it-matters-934eb07f3d2b
- IPFS and Friends: A Qualitative Comparison of Next Generation Peer-to-Peer Data Networks (2021), Erik Daniel and Florian Tshorsch
Disclaimer
The information in this research report comes from publicly disclosed information, and the opinions in this article are for research purposes only and do not represent any investment opinions. The opinions and predictions issued in the report are only analysis and judgment on the date of issuance and do not have permanent validity. In addition, under no circumstances will the organization and the author be liable to anyone for any losses caused by the use of any content in this report.
LD Capital is a leading crypto fund in investment and trading in primary and secondary markets, whose sub-funds include dedicated eco fund, FoF and hedge fund.
Owing to industrial resources and professional investment and research team, LD Capital has successively discovered and invested more than 300 companies in DeFi/infra/protocol/NFT/Dapp/Metaverse/DAO/Layer2/GameFi fields since 2016.
We have 30+ industrial experts around the world based in China, Singapore, Australia, EU and US, who worked for financial and high-tech companies like Visa, Cisco and now actively invest in blockchain companies.
website: ldcap.com
twitter: twitter.com/ld_capital
mail: BP@ldcap.com
medium：ld-capital.medium.com
