Skip to content

Commit

Permalink
Bump version 8.0.0a1 (#1008)
Browse files Browse the repository at this point in the history
  • Loading branch information
droserasprout authored May 6, 2024
1 parent fa11f11 commit 7110c94
Show file tree
Hide file tree
Showing 82 changed files with 1,210 additions and 876 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ concurrency:
on:
push:
tags:
- '*.*.*'
- 'docs'
- '8.*.*'
- 'docs-next'
branches:
- '**'
paths:
Expand All @@ -17,7 +17,7 @@ on:
- '.github/workflows/docs.yml'

env:
FRONTEND_BRANCH: master
FRONTEND_BRANCH: dev
GITHUB_TOKEN: ${{ secrets.DOCS_GITHUB_TOKEN }}

jobs:
Expand Down
6 changes: 3 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog], and this project adheres to [Semantic Versioning].

## [Unreleased]
## [8.0.0a1] - 2024-05-06

### Added

Expand Down Expand Up @@ -36,9 +36,9 @@ The format is based on [Keep a Changelog], and this project adheres to [Semantic
- deps: `datamodel-code-generator` updated to 0.25.
- deps: `pyarrow` updated to 16.0.
- deps: `pydantic`updated to 2.2.
- deps: `sentry-sdk`updated to 2.0.
- deps: `sentry-sdk`updated to 2.1.
- deps: `tortoise-orm` updated to 0.20.1.
- deps: `web3` updated to 6.17.
- deps: `web3` updated to 6.18.

## [7.5.5] - 2024-04-17

Expand Down
2 changes: 1 addition & 1 deletion docs/0.quickstart-evm.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ A modern Linux/macOS distribution with Python 3.12 installed is required to run
The easiest way to install DipDup as a CLI application [pipx](https://pipx.pypa.io/stable/). We have a convenient wrapper script that installs DipDup for the current user. Run the following command in your terminal:

```shell [Terminal]
curl -Lsf https://dipdup.io/install.py | python3
curl -Lsf https://dipdup.io/install.py | python3.12
```

See the [Installation](https://dipdup.io/docs/installation) page for all options.
Expand Down
2 changes: 1 addition & 1 deletion docs/0.quickstart-tezos.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ A modern Linux/macOS distribution with Python 3.12 installed is required to run
The easiest way to install DipDup as a CLI application [pipx](https://pipx.pypa.io/stable/). We have a convenient wrapper script that installs DipDup for the current user. Run the following command in your terminal:

```shell [Terminal]
curl -Lsf https://dipdup.io/install.py | python3
curl -Lsf https://dipdup.io/install.py | python3.12
```

See the [Installation](https://dipdup.io/docs/installation) page for all options.
Expand Down
2 changes: 1 addition & 1 deletion docs/1.getting-started/1.installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Minimum hardware requirements are 256 MB RAM, 1 CPU core, and some disk space fo
The easiest way to install DipDup is to use our interactive installer script. It will ask you a few questions and install DipDup with all dependencies. Run the following command in your terminal:

```shell [Terminal]
curl -Lsf https://dipdup.io/install.py | python3
curl -Lsf https://dipdup.io/install.py | python3.12
```

That's it! DipDup is installed as a CLI application and is available everywhere in a system. Now you can run `dipdup new` to spawn a new project from lots of ready-to-use templates and proceed to the next section: [Core concepts](2.core-concepts.md)
Expand Down
21 changes: 11 additions & 10 deletions docs/1.getting-started/6.datasources.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,16 @@ Datasources are DipDup connectors to various APIs. They are defined in config an

Index datasources, ones that can be attached to a specific index, are prefixed with blockchain name, e.g. `tezos.tzkt` or `evm.subsquid`.

| kind | blockchain | description |
| ---------------------------------------------------------- | ---------------- | ------------------------------- |
| [abi.etherscan](../3.datasources/1.abi_etherscan.md) | ⟠ EVM-compatible | Provides ABIs for EVM contracts |
| [coinbase](../3.datasources/2.coinbase.md) | any | Coinbase price feed |
| [evm.node](../3.datasources/3.evm_node.md) | ⟠ EVM-compatible | Ethereum node |
| [evm.subsquid](../3.datasources/4.evm_subsquid.md) | ⟠ EVM-compatible | Subsquid Network node |
| [http](../3.datasources/5.http.md) | any | Generic HTTP API |
| [ipfs](../3.datasources/6.ipfs.md) | any | IPFS gateway |
| [tezos.tzkt](../3.datasources/7.tezos_tzkt.md) | ꜩ Tezos | TzKT API |
| [tezos.tzip_metadata](../3.datasources/8.tzip_metadata.md) | ꜩ Tezos | TZIP-16 metadata |
| kind | blockchain | description |
| ---------------------------------------------------- | ---------------- | ------------------------------- |
| [abi.etherscan](../3.datasources/1.abi_etherscan.md) | ⟠ EVM-compatible | Provides ABIs for EVM contracts |
| [coinbase](../3.datasources/2.coinbase.md) | any | Coinbase price feed |
| [evm.node](../3.datasources/3.evm_node.md) | ⟠ EVM-compatible | Ethereum node |
| [evm.subsquid](../3.datasources/4.evm_subsquid.md) | ⟠ EVM-compatible | Subsquid Network node |
| [http](../3.datasources/5.http.md) | any | Generic HTTP API |
| [ipfs](../3.datasources/6.ipfs.md) | any | IPFS gateway |
| [tezos.tzkt](../3.datasources/7.tezos_tzkt.md) | ꜩ Tezos | TzKT API |
| [tzip_metadata](../3.datasources/8.tzip_metadata.md) | ꜩ Tezos | TZIP-16 metadata |

## Connection settings

Expand All @@ -40,6 +40,7 @@ datasources:
connection_timeout: 60
request_timeout: 60
batch_size: 10000
polling_interval: 1.0
replay_path: None
alias: None
```
Expand Down
22 changes: 11 additions & 11 deletions docs/1.getting-started/7.indexes.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,17 @@ Index is a primary DipDup entity connecting the inventory and data handling rule

Multiple indexes are available for different workloads. Every index is linked to a specific datasource and provides a set of handlers for different kinds of data. Use this table to choose the right index for the task:

| kind | blockchain | datasource | indexed data |
| -------------------------------------------------------------------------------------- | ---------------- | ---------- | --------------------------- |
| [evm.events](../2.indexes/1.evm_events.md) | ⟠ EVM-compatible | Subsquid | event logs |
| [evm.transactions](../2.indexes/2.evm_transactions.md) | ⟠ EVM-compatible | Subsquid | transactions |
| [tezos.big_maps](../2.indexes/3.tezos_big_maps.md) | ꜩ Tezos | TzKT | big map diffs |
| [tezos.events](../2.indexes/4.tezos_events.md) | ꜩ Tezos | TzKT | events |
| [tezos.head](../2.indexes/5.tezos_head.md) | ꜩ Tezos | TzKT | head blocks (realtime only) |
| [tezos.operations](../2.indexes/6.tezos_operations.md) | ꜩ Tezos | TzKT | typed operations |
| [tezos.operations_unfiltered](../2.indexes/7.tezos_operations_unfiltered.md) | ꜩ Tezos | TzKT | untyped operations |
| [tezos.token_balances](../2.indexes/8.tezos_token_balances.md) | ꜩ Tezos | TzKT | TZIP-12/16 token balances |
| [tezos.token_transfers](../2.indexes/9.tezos_token_transfers.md) | ꜩ Tezos | TzKT | TZIP-12/16 token transfers |
| kind | blockchain | datasources | indexed data |
| ---------------------------------------------------------------------------- | ---------------- | ----------- | --------------------------- |
| [evm.events](../2.indexes/1.evm_events.md) | ⟠ EVM-compatible | `evm` | event logs |
| [evm.transactions](../2.indexes/2.evm_transactions.md) | ⟠ EVM-compatible | `evm` | transactions |
| [tezos.big_maps](../2.indexes/3.tezos_big_maps.md) | ꜩ Tezos | `tezos` | big map diffs |
| [tezos.events](../2.indexes/4.tezos_events.md) | ꜩ Tezos | `tezos` | events |
| [tezos.head](../2.indexes/5.tezos_head.md) | ꜩ Tezos | `tezos` | head blocks (realtime only) |
| [tezos.operations](../2.indexes/6.tezos_operations.md) | ꜩ Tezos | `tezos` | typed operations |
| [tezos.operations_unfiltered](../2.indexes/7.tezos_operations_unfiltered.md) | ꜩ Tezos | `tezos` | untyped operations |
| [tezos.token_balances](../2.indexes/8.tezos_token_balances.md) | ꜩ Tezos | `tezos` | TZIP-12/16 token balances |
| [tezos.token_transfers](../2.indexes/9.tezos_token_transfers.md) | ꜩ Tezos | `tezos` | TZIP-12/16 token transfers |

Indexes can join multiple contracts considered as a single application. Also, contracts can be used by multiple indexes of any kind, but make sure that they are independent of each other and that indexed data don't overlap.

Expand Down
44 changes: 44 additions & 0 deletions docs/10.supported-networks/0.overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
title: "→ Overview"
description: "DipDup can index any EVM-compatible network as long as there's enough historical data. This page contains a list of supported networks and instructions on how to configure your indexer for them."
network: "ethereum"
---

<!-- markdownlint-disable no-bare-urls no-inline-html no-emphasis-as-heading -->

_Latest update: 2024-05-04._

# Supported networks

DipDup can index any EVM-compatible network as long as there's enough historical data. This page contains a list of supported networks and instructions on how to configure your indexer for them.

We aim to support all networks available in [Subsquid Network](https://docs.subsquid.io/subsquid-network/reference/evm-networks/#raw-urls), and several others in node-only mode.

If you want to share your test results or have found an error, please open an issue on [GitHub](https://github.com/dipdup-io/dipdup).

## Configuring datasources

If you've created a new project using `dipdup new` command and used one of the EVM templates, `datasources` section in your config file should look like this:

```yaml [dipdup.yaml]
datasources:
subsquid:
kind: evm.subsquid
url: ${SUBSQUID_URL:-https://v2.archive.subsquid.io/network/ethereum-mainnet}
etherscan:
kind: abi.etherscan
url: ${ETHERSCAN_URL:-https://api.etherscan.io/api}
api_key: ${ETHERSCAN_API_KEY:-''}
evm_node:
kind: evm.node
url: ${NODE_URL:-https://eth-mainnet.g.alchemy.com/v2}/${NODE_API_KEY:-''}
ws_url: ${NODE_WS_URL:-wss://eth-mainnet.g.alchemy.com/v2}/${NODE_API_KEY:-''}
```
To configure datasources for other networks, you need to change URLs and API keys. You can do it in the config file directly, but it's better to use environment variables. Check the `deploy/.env.default` file in your project directory; it contains all the variables used in config.

[evm.subsquid](../3.datasources/4.evm_subsquid.md) - Subsquid Network is the main source of historical data for EVM-compatible networks. It's free and available for many networks.

[abi.etherscan](../3.datasources/1.abi_etherscan.md) - Etherscan is a source of contract ABIs, which are used to generate types for the indexer. Many explorers have Etherscan-like API which could be used to retrieve ABIs. Some of them require an API key, which you can get on their website. If there's no Etherscan-like API available, you need to obtain contract ABI JSON somewhere and put it to the `abi/<typename>/abi.json` path. Don't forget to run `dipdup init` after that to generate all necessary types.

[evm.node](../3.datasources/3.evm_node.md) - EVM node datasource can be used to fetch recent data not yet in Subsquid Network. API methods could vary a lot across different networks, but DipDup only uses a few of them, so most of the nodes will work. WebSocket URL can be specified to get real-time updates. This option can save you some requests to the node, but otherwise, it's not required. If Subsquid for your network is not available yet, you can use this datasource to fetch historical data, but it's significantly slower.
50 changes: 50 additions & 0 deletions docs/10.supported-networks/1.arbitrum.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
title: "Arbitrum"
description: "Arbitrum network support"
---

<!-- markdownlint-disable single-h1 heading-increment no-inline-html -->

# Arbitrum

{{ #include 10.supported-networks/_intro.md }}

See also: [RPC endpoints and providers | Arbitrum Docs](https://docs.arbitrum.io/node-running/node-providers)

### Arbitrum One

Explorer: [Arbiscan](https://arbiscan.io/)

| datasource | status | URLs |
| -----------------:|:-------- | -------------------------------------------------------------------------------- |
| **evm.subsquid** | 🟢 works | `https://v2.archive.subsquid.io/network/arbitrum-one` |
| **abi.etherscan** | 🟢 works | `https://api.arbiscan.io/api` |
| **evm.node** | 🟢 works | `https://arb-mainnet.g.alchemy.com/v2` <br> `wss://arb-mainnet.g.alchemy.com/v2` |

### Arbitrum Goerli

| datasource | status | URLs |
| -----------------:|:------------- | -------------------------------------------------------- |
| **evm.subsquid** | ⚰️ deprecated | `https://v2.archive.subsquid.io/network/arbitrum-goerli` |
| **abi.etherscan** | ⚰️ deprecated | |
| **evm.node** | ⚰️ deprecated | |

### Arbitrum Nova

Explorer: [Arbiscan](https://nova.arbiscan.io/)

| datasource | status | URLs |
| -----------------:|:---------------- | ------------------------------------------------------ |
| **evm.subsquid** | 🟢 works | `https://v2.archive.subsquid.io/network/arbitrum-nova` |
| **abi.etherscan** | 🟢 works | `https://api-nova.arbiscan.io/api` |
| **evm.node** | 🤔 WS not tested | `https://nova.arbitrum.io/rpc` |

### Arbitrum Sepolia

Explorer: [Arbiscan](https://sepolia.arbiscan.io/)

| datasource | status | URLs |
| -----------------:|:-------- | -------------------------------------------------------------------------------- |
| **evm.subsquid** | 🟢 works | `https://v2.archive.subsquid.io/network/arbitrum-sepolia` |
| **abi.etherscan** | 🟢 works | `https://api-sepolia.arbiscan.io/api` |
| **evm.node** | 🟢 works | `https://arb-sepolia.g.alchemy.com/v2` <br> `wss://arb-sepolia.g.alchemy.com/v2` |
18 changes: 18 additions & 0 deletions docs/10.supported-networks/10.dogechain.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
title: "Dogechain"
description: "Dogechain network support"
---

<!-- markdownlint-disable single-h1 heading-increment no-inline-html -->

# Dogechain

{{ #include 10.supported-networks/_intro.md }}

Explorers: [Dogechain](https://dogechain.info/), [Blockscout](https://explorer.dogechain.dog/)

| datasource | status | URLs |
| -----------------:|:------------- | -------------------------------------------------- |
| **evm.subsquid** | 🤔 not tested | `https://v2.archive.subsquid.io/network/dogechain` |
| **abi.etherscan** | 🤔 not tested | `https://explorer.dogechain.dog/api` |
| **evm.node** | 🤔 not tested | |
50 changes: 50 additions & 0 deletions docs/10.supported-networks/11.ethereum.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
title: "Ethereum"
description: "Ethereum network support"
---

<!-- markdownlint-disable single-h1 heading-increment no-inline-html -->

# Ethereum

{{ #include 10.supported-networks/_intro.md }}

### Ethereum Mainnet

Explorer: [Etherscan](https://etherscan.io/)

| datasource | status | URLs |
| -----------------:|:-------- | -------------------------------------------------------------------------------- |
| **evm.subsquid** | 🟢 works | `https://v2.archive.subsquid.io/network/ethereum-mainnet` |
| **abi.etherscan** | 🟢 works | `https://api.etherscan.io/api` |
| **evm.node** | 🟢 works | `https://eth-mainnet.g.alchemy.com/v2` <br> `wss://eth-mainnet.g.alchemy.com/v2` |

### Ethereum Goerli

Explorer: [Etherscan](https://goerli.etherscan.io/)

| datasource | status | URLs |
| -----------------:|:------------- | -------------------------------------------------------- |
| **evm.subsquid** | 🤔 not tested | `https://v2.archive.subsquid.io/network/ethereum-goerli` |
| **abi.etherscan** | 🤔 not tested | `https://api-goerli.etherscan.io/api` |
| **evm.node** | 🤔 not tested | |

### Ethereum Holesky

Explorer: [Etherscan](https://holesky.etherscan.io/)

| datasource | status | URLs |
| -----------------:|:------------- | --------------------------------------------------------- |
| **evm.subsquid** | 🤔 not tested | `https://v2.archive.subsquid.io/network/ethereum-holesky` |
| **abi.etherscan** | 🤔 not tested | `https://api-holesky.etherscan.io/api` |
| **evm.node** | 🤔 not tested | |

### Ethereum Sepolia

Explorer: [Etherscan](https://sepolia.etherscan.io/)

| datasource | status | URLs |
| -----------------:|:------------- | --------------------------------------------------------- |
| **evm.subsquid** | 🤔 not tested | `https://v2.archive.subsquid.io/network/ethereum-sepolia` |
| **abi.etherscan** | 🤔 not tested | `https://api-sepolia.etherscan.io/api` |
| **evm.node** | 🤔 not tested | |
20 changes: 20 additions & 0 deletions docs/10.supported-networks/12.etherlink.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
title: "Etherlink"
description: "Etherlink network support"
---

<!-- markdownlint-disable single-h1 heading-increment no-inline-html -->

# Etherlink

{{ #include 10.supported-networks/_intro.md }}

### Etherlink Testnet

Explorer: [Blockscout](https://testnet-explorer.etherlink.com/)

| datasource | status | URLs |
| -----------------:|:------------- | ---------------------------------------------------------- |
| **evm.subsquid** | 🤔 not tested | `https://v2.archive.subsquid.io/network/etherlink-testnet` |
| **abi.etherscan** | 🤔 not tested | `https://testnet-explorer.etherlink.com/api` |
| **evm.node** | 🤔 not tested | |
18 changes: 18 additions & 0 deletions docs/10.supported-networks/13.exosama.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
title: "Exosama"
description: "Exosama network support"
---

<!-- markdownlint-disable single-h1 heading-increment no-inline-html -->

# Exosama

{{ #include 10.supported-networks/_intro.md }}

Explorer: [Blockscout](https://explorer.exosama.com/)

| datasource | status | URLs |
| -----------------:|:------------- | ------------------------------------------------ |
| **evm.subsquid** | 🟢 works | `https://v2.archive.subsquid.io/network/exosama` |
| **abi.etherscan** | 🤔 not tested | `https://explorer.exosama.com/api` |
| **evm.node** | 🤔 not tested | |
Loading

0 comments on commit 7110c94

Please sign in to comment.