Skip to content

Commit

Permalink
[Future Impl] Wiki Reorg: Sharding and Asynchronous Backing (#4871)
Browse files Browse the repository at this point in the history 
* added async backing

* remove async backing, add mermaid js diagram

* depds

* remove diagram

* removed mermaid locally

* lock delete

* actual diagram

* grammar

* Update docs/learn/learn-async-backing.md

Co-authored-by: Filippo <110459737+filippoweb3@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Filippo <110459737+filippoweb3@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Filippo <110459737+filippoweb3@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Filippo <110459737+filippoweb3@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Filippo <110459737+filippoweb3@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Filippo <110459737+filippoweb3@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Filippo <110459737+filippoweb3@users.noreply.github.com>

* start refactor

* fix diagram + clarify

* add more

* slight change

* Filippo edits

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* clarify diagram + remove dank compare

* block-labels

* link fix

* refactor / clarify

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* initial round applied

* minor grammar / tweaks

* slight polish

* sync with master

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Filippo <110459737+filippoweb3@users.noreply.github.com>

* Update docs/learn/learn-async-backing.md

Co-authored-by: Filippo <110459737+filippoweb3@users.noreply.github.com>

* clarified small nit

---------

Co-authored-by: Filippo <110459737+filippoweb3@users.noreply.github.com>
Co-authored-by: filippoweb3 <filippo@web3.foundation>
Co-authored-by: Radha <86818441+DrW3RK@users.noreply.github.com>
Co-authored-by: Bradley Olson <34992650+BradleyOlson64@users.noreply.github.com>
  • Loading branch information
@CrackTheCode016 @filippoweb3
@DrW3RK @BradleyOlson64
5 people authored Sep 8, 2023
1 parent b1e0b27 commit bb9f317
Show file tree
Hide file tree
Showing 4 changed files with 1,619 additions and 1,514 deletions.
2 changes: 1 addition & 1 deletion docs/general/faq.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ use TPS as a measure for the network performance if all the transactions carry t
[Performance benchmark tests](https://twitter.com/gavofyork/status/1270025498580656134) show that
Substrate-based blockchains can achieve over 1000 TPS for balance transfer transactions. Assuming
Polkadot is running over 100 parachains; the projected TPS is well over 100,000. With
[asynchronous backing upgrade](../learn/learn-async-backing), the TPS is expected to increase
[asynchronous backing upgrade](../learn/learn-async-backing.md), the TPS is expected to increase
tenfold to 1,000,000.

It is essential to realize that TPS is inherently a subjective measurement with numerous factors
Expand Down
116 changes: 116 additions & 0 deletions docs/learn/learn-async-backing.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
---
id: learn-async-backing
title: Asynchronous Backing
sidebar_label: Asynchronous Backing
description: A brief overview of asynchronous backing, and how it affects Polkadot's scalability.
keywords: [parachains, slots, backing, parablock]
slug: ../learn-async-backing
---

Asynchronous backing is a feature that introduces
[pipelining](https://www.techtarget.com/whatis/definition/pipelining) to the parachain block
generation and validation process. It is analogous to the logical pipelining of processor
instruction in "traditional" architectures, where some instructions may be executed before others
are complete. Instructions may also be executed in parallel, enabling multiple parts of the
processor to be working on potentially different instructions at the same time.

Bundles of state transitions represented as blocks may be processed similarly. In the context of
Polkadot, it aims to increase the throughput of the entire network.

:::info

In order to realize which exact processes this upgrade improves, it is recommended to be familiar
with the stages of parablock validation, which you may [read here](./learn-parachains-protocol.md).

:::

Asynchronous backing enables logical pipelining over the parablock
[**generation**](https://wiki.polkadot.network/docs/learn-parachains-protocol#collators),
[**backing**](https://wiki.polkadot.network/docs/learn-parachains-protocol#parachain-phase), and
[**inclusion**](https://wiki.polkadot.network/docs/learn-parachains-protocol#inclusion-pipeline)
processes. A parablock may be at different stages, but multiple parablocks can be processed
simultaneously (in parallel) if needed. Most notably, parablock N + 1 can be generated and backed
while its predecessor, parablock N, is undergoing inclusion on the relay chain. Processes can occur
while their ancestors are included on the relay chain.

This pipeline will allow [collators](./learn-parachains-protocol.md#collators) to include an
estimated ~3-5x more transactions/data while speeding up parachain block times from 12 to 6 seconds.
In short, Polkadot with asynchronous backing will deliver an estimated ~6-10x more blockspace to its
parachains.

Asynchronous Backing has three overarching goals:

1. Decrease parachain block (parablock) validation time to **6 seconds** from **12 seconds**
2. Increase the amount of usable blockspace, allowing more state changes per relay chain block.
3. Allow for parablocks to be re-proposed to the network if they are not included successfully on
the first attempt.

Asynchronous backing provides a form of **contextual execution**, which allows for more time for
parachain collators to fit more transactions and prepare block candidates for backing and inclusion.
**Contextual execution** refers to how a parablock can be built earlier using the context provided
by an _unincluded segment_ of recent block ancestors.

**Unincluded segments** are chains of candidate blocks that are not yet included in the relay chain.
Parablocks can be added to this unincluded segment without waiting for the latest included parent
block of the relay chain. The core functionality that asynchronous backing brings is the ability to
build on these unincluded segments of block ancestors rather than ancestors included in the relay
chain state.

Currently, parablocks rely on the most recent relay chain block (often referred to as the **parent**
block, as the parablock anchors itself to it). Each parablock must be generated and go through the
entire backing process in a single relay block. It then proceeds to the availability + inclusion
process during the next block.

:::info

It is important to note the following clarifications:

1. The parablock is not placed on the relay chain in its entirety, but rather the **paraheader**
2. The relay chain does not access the entire state of a parachain but only the values that changed
during that block and the merkelized hashes of the unchanged values.

:::

For more information on the validity and availability process, be sure to visit the
[parachain protocol](../learn/learn-parachains-protocol.md) page.

## Synchronous Backing on Polkadot

With synchronous backing, there was only about a single relay chain block (6-second window), to
complete the parablock candidate generation and backing process. This was tightly coupled to the
relay chain's progress, where blocks had to be created within this window.

The main limitation of synchronous backing is that parablock validation is tightly coupled to the
relay chain's progression on a 1-1 basis, meaning every parablock must be generated and backed
within six seconds. This time limit reduces the amount of data a collator has time to add to each
block.

A particular parablock, `P1`, would **only** be valid for backing at relay chain parent `R1 + 1` and
subsequently, be included at `R1 + 2` should it be backed successfully. Essentially, a parablock is
rushing to be backed and included within this two-block window due to the inherent requirement for
synchrony between the parachain and relay chain.

## Asynchronous Backing on Polkadot

With asynchronous backing, the window of time is customizable and will most likely sit around the
6-18 second range. It also introduces a parameter to aid in defining the maximum amount of ancestor
blocks, which allows for a parablock to be backed later in the future, enabling more computational
and storage time per block. The context for the latest parablock is derived from the unincluded
segment of block ancestors upon which the newest parablock is built.

These blocks can be prepared in anticipation of being included later rather than keeping in sync
with the relay chain's progress 1-1. The block can be in a different stage from another block being
built, as long as it abides by the parameters set forth by the asynchronous backing configuration.

This combination of lower latency, higher storage per block, and a logical pipeline spanning
Polkadot's networking, runtime, and collation aspects will allow for higher, more robust throughput.

## Learn More

This topic is evolving and being implemented at the moment, keep up to date using the following
resources:

- [Polkadot Roadmap Roundup](https://polkadot.network/blog/polkadot-roadmap-roundup) - Article by
Rob Habermeier, Polkadot founder, details the plans for Polkadot for 2023.
- [Asynchronous Backing Spec & Tracking Issue](https://github.com/paritytech/polkadot/issues/3779) -
The implementation tracking issue for asynchronous backing
1 change: 1 addition & 0 deletions polkadot-wiki/sidebars.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -543,6 +543,7 @@ module.exports = {
items: [
'learn/learn-spree',
"learn/learn-parathreads",
"learn/learn-async",
],
},
{
Expand Down
Loading

0 comments on commit bb9f317

Please sign in to comment.