Skip to main content

The data availability layer

Experimental

The data availability layer is an experimental feature that is not yet available on Tezos Mainnet. The way the DAL works may change significantly before it is generally available.

The data availability layer (DAL) is a companion peer-to-peer network for the Tezos blockchain, designed to provide additional data bandwidth to Smart Rollups. It allows users to share large amounts of data in a way that is decentralized and permissionless, because anyone can join the network and post and read data on it.

For a tutorial on how to use the DAL, see Implement a file archive with the DAL and a Smart Rollup.

How the DAL works​

The DAL relies on a network of DAL nodes that distribute data via a peer-to-peer network. Layer 1 bakers verify that the data is available. After the bakers attest that the data is available, the DAL nodes provide the data to Smart Rollups. Smart Rollups that need the data must store it, because it is available only temporarily on the DAL.

The DAL works like this:

  1. Users post data to a DAL node.

  2. The DAL node returns a certificate, which includes two parts:

    • The commitment is like a hash of the data but has the additional ability to identify individual shards of the data and reconstruct the original data from a certain percentage of the shards. The number of shards needed depends on how the data is spread across shards, which is controlled by a parameter called the redundancy factor.
    • The proof certifies the length of the data to prevent malicious users from overloading the layer with data.

  3. Users post the certificate to layer 1 via the Octez client.

  4. When the certificate is confirmed in a block, layer 1 splits the data into shards and assigns those shards to bakers, who verify that the data is available.

  5. Bakers verify that the data is available and attest that the data is available in their usual block attestations to layer 1. They have a certain number of blocks to do so, known as the attestation lag, and if enough shards are not attested by the end of this period, the certificate is considered bogus and the related data is dropped.

  6. Other DAL nodes get the data from the initial DAL node through the peer-to-peer network.

  7. The Smart Rollup node monitors the blocks and when it sees attested DAL data, it connects to a DAL node to request the data.

  8. The Smart Rollup node uses and stores the data.

The overall workflow is summarized in the following figure:

Data structure​

The data availability layer stores information about the available data in layer 1 blocks. Each block has several byte-vectors called slots, each with a maximum size. DAL users can add information about the available data as pages in these slots, as shown in this figure:

The data in a slot is broken into pages to ensure that each piece of data can fit in a single Tezos operation. This data must fit in a single operation to allow the Smart Rollup refutation game to work, in which every execution step of the Smart Rollup must be provable to layer 1.

When clients add data, they must specify which slot to add it to. Note that because the DAL is permissionless, clients may try to add data to the same slot in the same block. In this case, the first operation in the block takes precedence, which leaves the baker that creates the block in control of which data makes it into the block. Other operations that try to add data to the same slot fail.

The number and size of these slots can change. Different networks can have different DAL parameters. Future changes to the protocol may allow the DAL to resize dynamically based on usage.

Getting the DAL parameters​

Clients can get information about the current DAL parameters from the RPC endpoint GET /chains/main/blocks/head/context/constants or the Smart Rollup kernel SDK function reveal_dal_parameters. These parameters include:

  • number_of_slots: The number of slots in each block
  • slot_size: The size of each slot in bytes
  • page_size: The size of each page in bytes
  • attestation_lag: The number of subsequent blocks in which bakers can attest that the data is available; if enough attestations are available by the time this number of blocks have been created, the data becomes available to Smart Rollups
  • redundancy_factor: How much redundancy is used to split the data into shards; for example, a redundancy factor of 2 means that half of all shards are enough to reconstruct the original data and a redundancy factor of 4 means that 25% of all shards are required

Sending data to the DAL​

Sending data to the DAL is a two-step process:

  1. Send the data to a DAL node by passing it to its POST /slot endpoint, as in this example:

    curl -X POST http://dal-node.example.com:10732/slot --data '"Hello, world!"' -H 'Content-Type: application/json'

    The DAL node returns the commitment and proof of the data, as in this abbreviated example:

    {
      "commitment": "sh1u3tr3YKPDY",
      "commitment_proof": "8229c63b8e858d9a9"
    }

  2. Send an operation to include the commitment and proof in a block by running this Octez client command, using an RPC endpoint for $ENDPOINT and an account alias or address for MY_ACCOUNT:

    commitment="sh1u3tr3YKPDY"
    proof="8229c63b8e858d9a9"
    octez-client --endpoint ${ENDPOINT} \
      publish dal commitment "${commitment}" from ${MY_ACCOUNT} for slot 10 \
      with proof "${proof}"

For an example of sending larger amounts of data, see Implement a file archive with the DAL and a Smart Rollup.

Getting data from the DAL​

Smart Rollups can use data from the DAL only after it has been attested by the bakers. Therefore, they cannot access DAL data in the current level, because not enough blocks have elapsed to allow bakers to attest the data.

The latest level that Smart Rollups can access is the current level minus the attestation lag. They can access the data in that level with the Smart Rollup kernel SDK function reveal_dal_page, which accepts the level, slot, and page to receive, as in this example:

let param = host.reveal_dal_parameters();

let sol = host.read_input()?.unwrap();

let target_level = sol.level as usize - param.attestation_lag as usize;

let mut buffer = vec![0u8; param.page_size as usize];

let bytes_read = host.reveal_dal_page(target_level as i32, slot_index, 0, &mut buffer)?;

if 0 < bytes_read {
    debug_msg!(
        host,
        "Attested slot at index {} for level {}: {:?}\n",
        slot_index,
        target_level,
        &buffer.as_slice()[0..10]
    );
} else {
    debug_msg!(
        host,
        "No attested slot at index {} for level {}\n",
        slot_index,
        target_level
    );
}

Reference​

For more information about the DAL, see DAL overview in the Octez documentation.

Last updated on