Arbitrum Manifest File
Arbitrum Manifest File
We use Ethereum packages, runtimes, and handlers (e.g.
ethereum/*Handler) for Arbitrum. Since Arbitrum is an EVM-compatible layer-2 scaling solution, we can use the core Ethereum framework to index it.
project.yaml file can be seen as an entry point of your project and it defines most of the details on how SubQuery will index and transform the chain data. It clearly indicates where we are indexing data from, and to what on chain events we are subscribing to.
The Manifest can be in either YAML or JSON format. In this document, we will use YAML in all the examples.
Below is a standard example of a basic Arbitrum One
project.yaml. Arbitrum Nova is essentially the same but with different settings under
specVersion: "1.0.0" name: "arbitrum-one-subql-starter" version: "0.0.1" runner: node: name: "@subql/node-ethereum" version: "*" query: name: "@subql/query" version: "*" description: "This project can be use as a starting point for developing your new Arbitrum One SubQuery project" repository: "https://github.com/subquery/ethereum-subql-starter" schema: file: "./schema.graphql" network: # chainId is the EVM Chain ID, for Arbitrum One this is 42161 and Abitrum Nova is 42170 # https://chainlist.org/chain/42161 chainId: "42161" # This endpoint must be a public non-pruned archive node # We recommend providing more than one endpoint for improved reliability, performance, and uptime # Public nodes may be rate limited, which can affect indexing speed # When developing your project we suggest getting a private API key # You can get them from OnFinality for free https://app.onfinality.io # https://documentation.onfinality.io/support/the-enhanced-api-service endpoint: ["https://arbitrum.api.onfinality.io/public"] # Recommended to provide the HTTP endpoint of a full chain dictionary to speed up processing dictionary: "https://api.subquery.network/sq/subquery/arbitrum-one-dictionary" dataSources: - kind: ethereum/Runtime # We use ethereum runtime since Arbitrum One and Nova are layer-2s that are compatible startBlock: 2591 # This is the block that the contract was deployed on https://arbiscan.io/tx/0x8ebe1945f039f865af8b3079df3819534340ee41a5e6b8bfefb9c36a857778c9 options: # Must be a key of assets abi: erc20 address: "0x2f2a2543b76a4166549f7aab2e75bef0aefc5b0f" # This is the contract address for wrapped BTC https://arbiscan.io/token/0x2f2a2543b76a4166549f7aab2e75bef0aefc5b0f assets: erc20: file: "./abis/erc20.abi.json" mapping: file: "./dist/index.js" handlers: - handler: handleTransaction kind: ethereum/TransactionHandler # We use ethereum handlers since Arbitrum One and Nova are layer-2s that are compatible filter: ## The function can either be the function fragment or signature # function: '0x095ea7b3' # function: '0x7ff36ab500000000000000000000000000000000000000000000000000000000' function: approve(address spender, uint256 amount) - handler: handleLog kind: ethereum/LogHandler # We use ethereum handlers since Arbitrum One and Nova are layer-2s that are compatible filter: topics: ## Follows standard log filters https://docs.ethers.io/v5/concepts/events/ - Transfer(address indexed from, address indexed to, uint256 amount) # address: "0x60781C2586D68229fde47564546784ab3fACA982"
Top Level Spec
|specVersion||String||The spec version of the manifest file|
|name||String||Name of your project|
|version||String||Version of your project|
|description||String||Description of your project|
|runner||Runner Spec||Runner specs info|
|repository||String||Git repository address of your project|
|schema||Schema Spec||The location of your GraphQL schema file|
|network||Network Spec||Detail of the network to be indexed|
|dataSources||DataSource Spec||The datasource to your project|
|templates||Templates Spec||Allows creating new datasources from this templates|
|file||String||The location of your GraphQL schema file|
If you start your project by using the
subql init command, you'll generally receive a starter project with the correct network settings. If you are changing the target chain of an existing project, you'll need to edit the Network Spec section of this manifest.
chainId is the network identifier of the blockchain. Examples in Arbitrum is
42161 for Arbitrum One and
42170 for Arbitrum Nova. See https://chainlist.org/chain/42161
Additionally you will need to update the
endpoint. This defines the (HTTP or WSS) endpoint of the blockchain to be indexed - this must be a full archive node. This property can be a string or an array of strings (e.g.
endpoint: ['rpc1.endpoint.com', 'rpc2.endpoint.com']). We suggest providing an array of endpoints as it has the following benefits:
- Increased speed - When enabled with worker threads, RPC calls are distributed and parallelised among RPC providers. Historically, RPC latency is often the limiting factor with SubQuery.
- Increased reliability - If an endpoint goes offline, SubQuery will automatically switch to other RPC providers to continue indexing without interruption.
- Reduced load on RPC providers - Indexing is a computationally expensive process on RPC providers, by distributing requests among RPC providers you are lowering the chance that your project will be rate limited.
Public nodes may be rate limited which can affect indexing speed, when developing your project we suggest getting a private API key from a professional RPC provider like OnFinality.
There is only a dictionary for Arbitrum One
https://gx.api.subquery.network/sq/subquery/arbitrum-one-dictionary. For Arbitrum Nova, please omit the dictionary setting.
|chainId||String||A network identifier for the blockchain|
|endpoint||String||Defines the endpoint of the blockchain to be indexed - This must be a full archive node.|
|port||Number||Optional port number on the |
|dictionary||String||It is suggested to provide the HTTP endpoint of a full chain dictionary to speed up processing - read how a SubQuery Dictionary works.|
|bypassBlocks||Array||Bypasses stated block numbers, the values can be a |
|node||Runner node spec||Describe the node service use for indexing|
|query||Runner query spec||Describe the query service|
Runner Node Spec
|version||String||Version of the indexer Node service, it must follow the SEMVER rules or |
Runner Query Spec
|version||String||Version of the Query service, available versions can be found here, it also must follow the SEMVER rules or |
Defines the data that will be filtered and extracted and the location of the mapping function handler for the data transformation to be applied.
|kind||string||ethereum/Runtime We use the Ethereum runtime for Arbitrum since it is compatible with the Ethereum framework|
|startBlock||Integer||This changes your indexing start block, set this higher to skip initial blocks with less data|
|handlers & filters||Default handlers and filters||List all the mapping functions and their corresponding handler types, with additional mapping filters.|
Data Sources and Mapping
In this section, we will talk about the default Arbitrum runtime and its mapping. Here is an example:
dataSources: - kind: ethereum/Runtime # We use ethereum runtime since Arbitrum is a layer-2 that is compatible startBlock: 2591 # This is the block that the contract was deployed on https://arbiscan.io/tx/0x8ebe1945f039f865af8b3079df3819534340ee41a5e6b8bfefb9c36a857778c9 options: # Must be a key of assets abi: erc20 address: "0x2f2a2543b76a4166549f7aab2e75bef0aefc5b0f" # This is the contract address for wrapped BTC https://arbiscan.io/token/0x2f2a2543b76a4166549f7aab2e75bef0aefc5b0f assets: erc20: file: "./abis/erc20.abi.json" mapping: file: "./dist/index.js" handlers: ...
Mapping Handlers and Filters
The following table explains filters supported by different handlers.
Your SubQuery project will be much more efficient when you only use
LogHandler handlers with appropriate mapping filters (e.g. NOT a
Default runtime mapping filters are an extremely useful feature to decide what block, event, or extrinsic will trigger a mapping handler.
Only incoming data that satisfies the filter conditions will be processed by the mapping functions. Mapping filters are optional but are highly recommended as they significantly reduce the amount of data processed by your SubQuery project and will improve indexing performance.
modulo filter allows handling every N blocks, which is useful if you want to group or calculate data at a set interval. The following example shows how to use this filter.
filter: modulo: 50 # Index every 50 blocks: 0, 50, 100, 150....
Real-time indexing (Block Confirmations)
As indexers are an additional layer in your data processing pipeline, they can introduce a massive delay between when an on-chain event occurs and when the data is processed and able to be queried from the indexer.
SubQuery provides real time indexing of unconfirmed data directly from the RPC endpoint that solves this problem. SubQuery takes the most probabilistic data before it is confirmed to provide to the app. In the unlikely event that the data isn’t confirmed and a reorg occurs, SubQuery will automatically roll back and correct its mistakes quickly and efficiently - resulting in an insanely quick user experience for your customers.
Bypass Blocks allows you to skip the stated blocks, this is useful when there are erroneous blocks in the chain or when a chain skips a block after an outage or a hard fork. It accepts both a
range or single
integer entry in the array.
When declaring a
range use an string in the format of
"start - end". Both start and end are inclusive, e.g. a range of
"100-102" will skip blocks
network: chainId: "1" endpoint: "https://eth.api.onfinality.io/public" bypassBlocks: [1, 2, 3, "105-200", 290]
You can validate your project manifest by running
subql validate. This will check that it has the correct structure, valid values where possible and provide useful feedback as to where any fixes should be made.