Arweave: State of the Ecosystem

8 min readSep 20, 2022

Thoughts on the key components of the Arweave ecosystem as we build out a permanent, digital, open commons for humanity’s knowledge

Note: Originally posted on 14 August, 2022 here.

About 8 months ago, I came across Sam Williams’ video on Mechanism Design (linked below). This got me interested in the field, and specifically in Sam’s own data storage blockchain — Arweave. I reached out via a cold DM on twitter to see if I could work with him. Turns out I could. In the last few months I have done some of the most interesting work of my life thus far, here. 😃

With this post, I hope to provide developers and operators who are looking to make a mark in web3 a broad overview of the Arweave ecosystem. A basic understanding of blockchains (Bitcoin/Proof-of-work) is assumed. This post also sets context for the new innovations that launched last week, which I will continue to outline in future threads. At the end of this post I’ve also collated a list of helpful resources for diving into Arweave.

Understanding Arweave

For the first time in history, Arweave is enabling the creation of a permanent digital store of knowledge. To understand Arweave, you need to understand two basic things:

Decentralized storage: If you upload a file to Arweave, it is replicated to and stored in the hard drives of a number of miners in the world. These miners are rewarded with AR tokens when (during mining) they prove they hold your data, similar to rewards in Bitcoin. Arweave employs a variant of Proof-of-Work called Proof-of-Access that requires proving storage of random chunks of past data.
Endowment model: Arweave storage financing works on an a perpetual endowment model. When you upload a piece of data, you pay up front for 200 years of storage, in AR (the native token of Arweave). A large portion of this goes into an endowment and is paid out to miners over time, proportional to their cost of storage of your data every block. The key here is that the cost of storage of data is perpetually declining due to technological advancement. If this rate is just 0.5%, then your up-front payment is perpetual, since 0.005*200 = 1 year.

With this basic mental model, you can now dive in to projects in the ecosystem. We start with core infrastructure projects, then explain some of the other interesting developments on Arweave like permaweb apps and atomic assets.

Infrastructure

Bundlers

The following image shows what an Arweave transaction contains. The key rows to observe here are quantity and the data_ fields. These fields both represent the only two types of Arweave transactions, transfers (represented by quantity) and data uploads (represented by data).

{
  "format": 2,
  "id": "3pXpj43Tk8QzDAoERjHE3ED7oEKLKephjnVakvkiHF8",
  "last_tx": "NpeIbi93igKhE5lKUMhH5dFmyEsNGC0fb2Qysggd-kM",
  "owner": "posmE...psEok",
  "tags": [],
  "target": "pEbU_SLfRzEseum0_hMB1Ie-hqvpeHWypRhZiPoioDI",
  "quantity": "10000000000",
  "data_root": "PGh0b...RtbD4",
  "data_size": "234234",
  "data": "VGVzdA",
  "reward": "321579659",
  "signature": "fjL0N...f2UMk"
}

Arweave imposes a limit of 1000 such transactions per block, which is about ~8 tps at an average block time of 2 minutes. This is not enough to support requirements of everyone’s data storage.

In order to solve this - about a year ago, in probably the biggest step-function improvement in the Arweave ecosystem since its launch - the ANS-104 bundling standard was defined. This format for sub-transactions (DataItems) allows a transaction to be constructed of many data uploads bundled into one.

In contrast to an Arweave base layer txn, this is what a DataItem looks like.

I won’t go into detail here as bundling enables a whole slew of innovations, but note the appearance of signature_type field. By allowing any type of signature for data items, bundles allow for signing using arbitrary keys.

This feature has allowed bundlers like the bundlr.network (the largest bundling service on top of Arweave) to integrate with other L1s. This means that you can pay for Arweave storage and sign using your Ethereum, or Solana or other private key. Other bundlers have now emerged, as Ever Finance recently shipped Arseeding 1.0 that supports ANS-104 bundles as a service.

Summarizing, bundles -

Allow for arbitrarily scaling Arweave data uploads.
Allow for signing and paying for Arweave data with other blockchain wallets.
Cannot perform AR transfers.

Gateways

While the Arweave protocol bakes-in incentives for sharing data among nodes, this data is often stored in hard drives that are not conducive for instantaneous retrieval and querying.

If bundles are the default way to get data into the weave, gateways are a way to get data out of it. They are a way to access data stored on Arweave quickly and granularly, acting as a relational read-database.

DMac (if you’ve ever visited Arweave discord, you probably know who he is) compares gateways to data warehouses on top of the Arweave’s data lake.

While this is a helpful simplification, gateways vary in functionality, and often end up providing a lot more services on top of Arweave in addition to indexing, including -

Unbundling
Transaction/block header querying (via GraphQL)
Caching
Secure Sockets
App rendering

While the AR.IO team has recently taken over the default arweave.net gateway (that has served the community for many years), there are many other gateways being built. The textury team recently released their own gateway services, and another team is building enterprise-grade gateways on Arweave.

Smartweave

Arweave smart contracts have an interesting design. While the protocol itself does not specify execution and is very different from a typical smart contract chain, smart contracting is done using lazy evaluation.

Simply put, Arweave smart contracts are simply an ordered tuple (C, I, Ts) of transactions, where C represents the txn containing the contract code, I represents the txn containing the initial state and Ts represents an ordered list of transactions that interact with the contract. The state is lazily-evaluated on the client by using the code in C, the initial state in I, and applying every transaction after it (given that it’s a valid transaction) based on the contract code. Here’s an overview of the design,

Smartweave Architecture

Smartweave is an architecture, and there are many ecosystem teams that are in the business of making a reliable, fast and production-ready smart contracting engine on Arweave. Perhaps the most advanced of these is Redstone’s Warp Smartweave engine. For new programmers coming to Arweave, Redstone also have an excellent onboarding tutorial that you should absolutely check out!

Another up-and-coming smartweave engine on Arweave is 3em built by Community Labs, the team and company behind ArConnect — the most prominent Arweave wallet.

If you understand these 3 basic infrastructure pieces — Bundlers, Gateways and Smartweave — you have the necessary building blocks to understand all of the things that Arweave enables and the interesting developments currently playing out in the ecosystem.

Ecosystem

Before discussing some of the more recent developments, here is a quick overview of the ecosystem and important use cases for Arweave.

NFTs

Arweave’s quality of permanence ensures that it continues to be the platform of choice for storing the metadata & media in all NFT projects in crypto. Any serious NFT project must ensure that its data is permanently stored and available forever, and Arweave is the only blockchain that enables this today. As a result, an overwhelming majority of Solana NFTs and a significant portion of Ethereum NFTs are stored permanently on Arweave.

Atomic NFTs

Using a combination of SmartWeave and Arweave’s data upload feature, you can create NFTs where the data (the music/video file or the monkey jpeg) is atomically linked to its metadata and its contract state. This is in stark contrast to other smart contracting chains, where the “NFT” is just a piece of contract code that links to an external data source where media is actually stored. This is illustrated below,

Atomic NFTs (Assets) on Arweave. [Credit: David]

Aside on Tags

This is a good time to mention Arweave tags. Arweave tags are a unique feature of Arweave that allows data and arbitrary metadata to be linked together in the same transaction ID. This means that any data stored on Arweave, and therefore the permaweb, is atomically linked to its metadata.

This enables features like associating data with contract state (as in Atomic NFTs), licenses, etc.

Permaweb

If you’ve been in the Arweave ecosystem long enough, you have probably heard of the “Permaweb”. The Permaweb is a collection of interlinked web applications, files and media — all stored permanently on Arweave — that will exist forever (with game-theoretic guarantees), without mutation. It is a decentralized, permanent web built on Arweave. Applications on Arweave are thus deployed once and available for use forever in their current form. Check out Nader’s explanation of hyperstructures/apps for perspective and textury’s very helpful permaweb dropper for an example of a simple permaweb app. Also check out the Permaweb wiki.

How does the permaweb work? Arweave gateways render web-apps using data stored in transactions. A full web-app can be uploaded along with a manifest that specifies how to render the web application.

TL;DR. If Gmail in 2007 was deployed as a permaweb application (without terms that allowed them to misuse your activity and info), then today you wouldn’t have shared all your private data with them for them to monetize by selling it.

Apps

Permaweb applications and other applications that use Arweave for storage are too many to cover in detail here. Examples include permapages, akord, pianity, ardrive, permacast.

Ecosystem Apps [Credit: Pascal]

That’s it for this time! If you found this overview of Arweave helpful, share this post with fellow web3/blockchain devs and let me know below what you’d like to learn more about.