Running SubQuery Locally

... 2022-11-16 About 4 min

# Running SubQuery Locally

This guide works through how to run a local SubQuery node on your infrastructure, which includes both the indexer and query service. Don't want to worry about running your own SubQuery infrastructure? SubQuery provides a Managed Service (opens new window) to the community for free. Follow our publishing guide to see how you can upload your project to SubQuery Projects (opens new window).

# Using Docker

An alternative solution is to run a Docker Container, defined by the docker-compose.yml file. For a new project that has been just initialised you won't need to change anything here.

Under the project directory run the following command:

docker-compose pull && docker-compose up
1

Note It may take some time to download the required packages ([`@subql/node`](https://www.npmjs.com/package/@subql/node), [`@subql/query`](https://www.npmjs.com/package/@subql/query), and Postgres) for the first time but soon you'll see a running SubQuery node. :::

# Running an Indexer (subql/node)

Requirements:

A SubQuery node is an implementation that extracts Substrate/Polkadot-based blockchain data per the SubQuery project and saves it into a Postgres database.

If you are running your project locally using subql-node or subql-node-<network>, make sure you enable the pg_extension btree_gist

You can run the following SQL query:

CREATE EXTENSION IF NOT EXISTS btree_gist;
1

# Installation

Please note that we **DO NOT** encourage the use of `yarn global` due to its poor dependency management which may lead to an errors down the line. :::

Once installed, you can start a node with the following command:

# Key Commands

The following commands will assist you to complete the configuration of a SubQuery node and begin indexing. To find out more, you can always run --help.

# Point to local project path

# Connect to database

export DB_USER=postgres
export DB_PASS=postgres
export DB_DATABASE=postgres
export DB_HOST=localhost
export DB_PORT=5432
subql-node -f your-project-path
1
2
3
4
5
6

Depending on the configuration of your Postgres database (e.g. a different database password), please ensure also that both the indexer (subql/node) and the query service (subql/query) can establish a connection to it.

# Specify a configuration file

This will point the query node to a manifest file which can be in YAML or JSON format.

# Change the block fetching batch size

subql-node -f your-project-path --batch-size 200

Result:
[IndexerManager] fetch block [203, 402]
[IndexerManager] fetch block [403, 602]
1
2
3
4
5

When the indexer first indexes the chain, fetching single blocks will significantly decrease the performance. Increasing the batch size to adjust the number of blocks fetched will decrease the overall processing time. The current default batch size is 100.

# Run in local mode

For debugging purposes, users can run the node in local mode. Switching to local model will create Postgres tables in the default schema public.

If local mode is not used, a new Postgres schema with the initial subquery_ and corresponding project tables will be created.

# Check your node health

There are 2 endpoints that you can use to check and monitor the health of a running SubQuery node.

  • Health check endpoint that returns a simple 200 response.
  • Metadata endpoint that includes additional analytics of your running SubQuery node.

Append this to the base URL of your SubQuery node. Eg http://localhost:3000/meta will return:

{
    "currentProcessingHeight": 1000699,
    "currentProcessingTimestamp": 1631517883547,
    "targetHeight": 6807295,
    "bestHeight": 6807298,
    "indexerNodeVersion": "0.19.1",
    "lastProcessedHeight": 1000699,
    "lastProcessedTimestamp": 1631517883555,
    "uptime": 41.151789063,
    "polkadotSdkVersion": "5.4.1",
    "apiConnected": true,
    "injectedApiConnected": true,
    "usingDictionary": false,
    "chain": "Polkadot",
    "specName": "polkadot",
    "genesisHash": "0x91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3",
    "blockTime": 6000
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

http://localhost:3000/health will return HTTP 200 if successful.

A 500 error will be returned if the indexer is not healthy. This can often be seen when the node is booting up.

{
    "status": 500,
    "error": "Indexer is not healthy"
}
1
2
3
4

If an incorrect URL is used, a 404 not found error will be returned.

{
"statusCode": 404,
"message": "Cannot GET /healthy",
"error": "Not Found"
}
1
2
3
4
5

# Debug your project

Use the node inspector (opens new window) to run the following command.

node --inspect-brk <path to subql-node> -f <path to subQuery project>
1

For example:

node --inspect-brk /usr/local/bin/subql-node -f ~/Code/subQuery/projects/subql-helloworld/
Debugger listening on ws://127.0.0.1:9229/56156753-c07d-4bbe-af2d-2c7ff4bcc5ad
For help, see: https://nodejs.org/en/docs/inspector
Debugger attached.
1
2
3
4

Then open up the Chrome dev tools, go to Source > Filesystem and add your project to the workspace and start debugging. For more information, check out How to debug a SubQuery project.

# Running a Query Service (subql/query)

# Installation

# NPM
npm install -g @subql/query
1
2

Please note that we **DO NOT** encourage the use of `yarn global` due to its poor dependency management which may lead to an errors down the line. :::

# Running the Query service

export DB_HOST=localhost
subql-query --name <project_name> --playground
1
2

Make sure the project name is the same as the project name when you initialize the project. Also, check the environment variables are correct.

After running the subql-query service successfully, open your browser and head to http://localhost:3000. You should see a GraphQL playground showing in the Explorer and the schema that is ready to query.

Last update: November 16, 2022 22:54