Command Line Flags

SubQuery TeamAbout 11 min

Command Line Flags

subql (cli)

--help

> subql --help

COMMANDS
  build     Build this SubQuery project code
  codegen   Generate schemas for graph node
  help      display help for subql
  init      Initialize a scaffold subquery project
  migrate   Migrate Subquery project manifest v0.0.1 to v0.2.0
  publish   Upload this SubQuery project to IPFS
  validate  Check a folder or github repo is a validate subquery project

build

This command is uses webpack to generate a bundle of a subquery project.

Options	Description
-l, --location	local folder of subquery project (if not in folder already)
-o, --output	specify output folder of build e.g. build-folder
--mode	production or development (default: production)

With subql build you can specify additional entry points in exports field although it will always build index.ts automatically.
You need to have @subql/cli v0.19.0 or above to use exports field.
Any exports field must map to string type (e.g. "entry": "./src/file.ts"), else it will be ignored from build.

For more info, visit basic workflows.

subql-node

--help

This shows the help options.

> subql-node --help
Commands:
  run force-clean  Clean the database dropping project schemas and tables. Once
                   the command is executed, the application would exit upon
                   completion.
  run reindex      Reindex to specified block height. Historical must be enabled
                   for the targeted project (--disable-historical=false). Once
                   the command is executed, the application would exit upon
                   completion.
Options:
      --help                Show help                                  [boolean]
      --version             Show version number                        [boolean]
  -f, --subquery            Local path of the subquery project          [string]
      --subquery-name       Name of the subquery project   [deprecated] [string]
  -c, --config              Specify configuration file                  [string]
      --local               Use local mode                [deprecated] [boolean]
      --db-schema           Db schema name of the project               [string]
      --unsafe              Allows usage of various other features that compromise a projects determinism                    [boolean][default: false]
      --batch-size          Batch size of blocks to fetch in one round  [number]
      --scale-batch-size    scale batch size based on memory usage
                                                      [boolean] [default: false]
      --timeout             Timeout for indexer sandbox to execute the mapping
                            functions                                   [number]
      --debug               Show debug information to console output. will
                            forcefully set log level to debug
                                                      [boolean] [default: false]
      --profiler            Show profiler information to console output
                                                      [boolean] [default: false]
      --subscription        Enable subscription       [boolean] [default: false]
      --network-endpoint    Blockchain network endpoint to connect      [string]
      --output-fmt          Print log as json or plain text
                                           [string] [choices: "json", "colored"]
      --log-level           Specify log level to print. Ignored when --debug is
                            used
          [string] [choices: "fatal", "error", "warn", "info", "debug", "trace",
                                                                       "silent"]
      --migrate             Migrate db schema (for management tables only)
                                                      [boolean] [default: false]
      --timestamp-field     Enable/disable created_at and updated_at in schema
                                                      [boolean] [default: false]
      --unfinalized-blocks  Enable/disable unfinalized blocks indexing
                                                       [boolean] [default: false]
  -d, --network-dictionary  Specify the dictionary api for this network [string]
  -m, --mmr-path            Local path of the merkle mountain range (.mmr) file
                                                                        [string]
      --proof-of-index      Enable/disable proof of index
                                                      [boolean] [default: false]
  -p, --port                The port the service will bind to           [number]
      --disable-historical  Disable storing historical state entities
                                                       [boolean] [default: true]
  -w, --workers             Number of worker threads to use for fetching and
                            processing blocks. Disabled by default.     [number]

--batch-size

This flag allows you to set the batch size in the command line. If batch size is also set in the config file, this takes precedent.

> subql-node -f . --batch-size=20
2021-08-09T23:24:43.775Z <fetch> INFO fetch block [6601,6620], total 20 blocks
2021-08-09T23:24:45.606Z <fetch> INFO fetch block [6621,6640], total 20 blocks
2021-08-09T23:24:47.415Z <fetch> INFO fetch block [6641,6660], total 20 blocks
2021-08-09T23:24:49.235Z <fetch> INFO fetch block [6661,6680], total 20 blocks

-c, --config

All these various configurations can be placed into a .yml or .json file and then referenced with the config flag.

Sample subquery_config.yml file:

subquery: . // Mandatory. This is the local path of the project. The period here means the current local directory.
subqueryName: hello // Optional name
batchSize: 55 // Optional config

Place this file in the same directory as the project. Then in the current project directory, run:

> subql-node -c ./subquery_config.yml

-d, --network-dictionary

This allows you to specify a dictionary endpoint which is a free service that is provided and hosted at SubQuery's Project Exploreropen in new window (search for dictionary) and presents an API endpoint of: https://api.subquery.network/sq/subquery/dictionary-polkadot.

Typically this would be set in your manifest file but below shows an example of using it as an argument in the command line.

subql-node -f . -d "https://api.subquery.network/sq/subquery/dictionary-polkadot"

For more info, visit How does a SubQuery Dictionary works?

--dictionary-timeout

Changes the timeout for dictionary queries, this number is expressed in seconds. By default we use 30 seconds.

--db-schema

This flag allows you to provide a name for the project database schema. Upon providing a new name, a new database schema is created with the configured name and block indexing starts.

subql-node -f . --db-schema=test2

--debug

This outputs debug information to the console output and forcefully sets the log level to debug.

> subql-node -f . --debug
2021-08-10T11:45:39.471Z <db> DEBUG Executing (1b0d0c23-d7c7-4adb-a703-e4e5c414e035): INSERT INTO "subquery_1"."starter_entities" ("id","block_height","created_at","updated_at") VALUES ($1,$2,$3,$4) ON CONFLICT ("id") DO UPDATE SET "id"=EXCLUDED."id","block_height"=EXCLUDED."block_height","updated_at"=EXCLUDED."updated_at" RETURNING "id","block_height","created_at","updated_at";
2021-08-10T11:45:39.472Z <db> DEBUG Executing (default): UPDATE "subqueries" SET "next_block_height"=$1,"updated_at"=$2 WHERE "id" = $3
2021-08-10T11:45:39.472Z <db> DEBUG Executing (1b0d0c23-d7c7-4adb-a703-e4e5c414e035): COMMIT;

--disable-historical

Disables automated historical state tracking, see Historic State Tracking. By default this is set to false.

-f, --subquery

Use this flag to start the SubQuery project.

subql-node -f . // OR
subql-node --subquery .

force-clean

In order to use this command you need to have @subql/node v1.10.0 or above.

This command forces the project schemas and tables to be regenerated. It is helpful to use when iteratively developing graphql schemas in order to ensure a clean state when starting a project. Note that this flag will also wipe all indexed data. This will also drop all related schema and tables of the project.

-f, --subquery flag must be passed in, to set path of the targeted project.

Note

Similar to reindex command, the application would exit upon completion.

subql-node -f /example/subql-project force-clean

--local (deprecated)

This flag is primarily used for debugging purposes where it creates the default starter_entity table in the default "postgres" schema.

subql-node -f . --local

Note that once you use this flag, removing it won't mean that it will point to another database. To repoint to another database you will have to create a NEW database and change the env settings to this new database. In other words, "export DB_DATABASE=<new_db_here>".

--log-level

There are 7 options to choose from. “fatal”, “error”, “warn”, “info”, “debug”, “trace”, “silent”. The example below shows silent. Nothing will be printed in the terminal so the only way to tell if the node is working or not is to query the database for row count (select count(*) from subquery_1.starter_entities) or query the block height.

> subql-node -f . --log-level=silent
(node:24686) [PINODEP007] Warning: bindings.level is deprecated, use options.level option instead
(Use `node --trace-warnings ...` to show where the warning was created)
(node:24686) [PINODEP007] Warning: bindings.level is deprecated, use options.level option instead
(node:24686) [PINODEP007] Warning: bindings.level is deprecated, use options.level option instead
(node:24686) [PINODEP007] Warning: bindings.level is deprecated, use options.level option instead
(node:24686) [PINODEP007] Warning: bindings.level is deprecated, use options.level option instead
(node:24686) [PINODEP007] Warning: bindings.level is deprecated, use options.level option instead
(node:24686) [PINODEP007] Warning: bindings.level is deprecated, use options.level option instead
(node:24686) [PINODEP007] Warning: bindings.level is deprecated, use options.level option instead
(node:24686) [PINODEP007] Warning: bindings.level is deprecated, use options.level option instead
(node:24686) [DEP0152] DeprecationWarning: Custom PerformanceEntry accessors are deprecated. Please use the detail property.
(node:24686) [PINODEP007] Warning: bindings.level is deprecated, use options.level option instead

--multi-chain

Enables indexing multiple subquery projects into the same database schema.

> subql-node -f . --multi-chain --db-schema=SCHEMA_NAME

For more info, visit Multi-Chain Support.

--network-endpoint

This flag allows users to override the network endpoint configuration from the manifest file.

subql-node -f . --network-endpoint="wss://polkadot.api.onfinality.io/public-ws"

Note that this must also be set in the manifest file, otherwise you'll get:

ERROR Create Subquery project from given path failed! Error: failed to parse project.yaml.
An instance of ProjectManifestImpl has failed the validation:
 - property network has failed the following constraints: isObject
 - property network.network has failed the following constraints: nestedValidation

--output-fmt

There are two different terminal output formats. JSON or colored. Colored is the default and contains colored text.

> subql-node -f . --output-fmt=json
{"level":"info","timestamp":"2021-08-10T11:58:18.087Z","pid":24714,"hostname":"P.local","category":"fetch","message":"fetch block [10501,10600], total 100 blocks"}

> subql-node -f . --output-fmt=colored
2021-08-10T11:57:41.480Z <subql-node> INFO node started
(node:24707) [PINODEP007] Warning: bindings.level is deprecated, use options.level option instead
2021-08-10T11:57:48.981Z <fetch> INFO fetch block [10201,10300], total 100 blocks
2021-08-10T11:57:51.862Z <fetch> INFO fetch block [10301,10400], total 100 blocks

-p, --port

The port the subquery indexing service binds to. By default this is set to 3000.

--pg-ca

When connecting to a postgres database via SSL, the path to the server certificate (in .pem format)

--pg-cert

When connecting to a postgres database via SSL, the path to the client certificate (in .pem format)

--pg-key

When connecting to a postgres database via SSL, the path to the client key file (in .key format)

--profiler

This shows profiler information.

subql-node -f . --local --profiler
2021-08-10T10:57:07.234Z <profiler> INFO FetchService, fetchMeta, 3876 ms
2021-08-10T10:57:08.095Z <profiler> INFO FetchService, fetchMeta, 774 ms
2021-08-10T10:57:10.361Z <profiler> INFO SubstrateUtil, fetchBlocksBatches, 2265 ms
2021-08-10T10:57:10.361Z <fetch> INFO fetch block [3801,3900], total 100 blocks

reindex

Note

In order to use this command, you require @subql/node:v1.10.0/@subql/node-YOURNETWORK:v1.10.0 or above.

When using reindex command, historical must be enabled for the targeted project (--disable-historical=false). After starting the project, it would print out a log stating if historical is enabled or not.

For more info, visit Automated Historical State Tracking

Use --targetHeight=<blockNumber> with reindex to remove indexed data and reindex from specified block height.

-f or --subquery flag must be passed in, to set path of the targeted project.

If the targetHeight is less than the declared starting height, it will execute the --force-clean command.

subql-node -f /example/subql-project reindex --targetHeight=30

Note

Once the command is executed and the state has been rolled back the the specified height, the application will exit. You can then start up the indexer to proceed again from this height.

--scale-batch-size

Scale the block fetch batch size with memory usage.

--subscription

This will create a notification trigger on entity, this also is the prerequisite to enable subscription feature in query service.

--subquery-name (deprecated)

This flag allows you to provide a name for your project which acts as if it creates an instance of your project. Upon providing a new name, a new database schema is created and block synchronisation starts from zero. Deprecated in favour of --db-schema

subql-node -f . --subquery-name=test2

--timeout

Set custom timeout for the javascript sandbox to execute mapping functions over a block before the block mapping function throws a timeout exception.

--timestamp-field

By default this is true. When set to false, it removes the created_at and updated_at columns in the starter_entities table.

> subql-node -f . –timestamp-field=false

--unfinalized-blocks

This allows you to index blocks before they become finalized. It can be very useful if you want the most up-to-date data possible. It will detect any forks and remove any blocks that don't become finalized. By default it is set to false. To change it to true run following command:

> subql-node -f . --unfinalized-blocks

Tip

Note that this feature requires historical indexing to be enabled. Learn more here.

Note

This feature is only available for Substrate-based blockchains; more networks will be supported in the future.

--unsafe (Node Service)

Unsafe mode controls various features that compromise the determinism of a SubQuery project. It makes it impossible to guarantee that the data within two identical projects running independently, will be 100% consistent.

One way we control this is by running all projects in a JS sandbox for security to limit the scope of access the project has to your system. The sandbox limits the available javascript imports to the following modules:

["assert", "buffer", "crypto", "util", "path"];

Although this enhances security, we understand that this limits the available functionality of your SubQuery project. The --unsafe command allows any import which greatly increases functionality with the tradeoff of decreased security.

By extension, the --unsafe command on the SubQuery Node also allows:

making external requests (e.g. via Fetch to an external HTTP address or fs)
querying block data at any height via the unsafeApi

Note that users must be on a paid plan to run projects with the --unsafe command (on the node service) within SubQuery's Managed Serviceopen in new window. Additionally, it will prevent your project from being run in the SubQuery Network in the future.

Also review the --unsafe command on the query service.

--version

This displays the current version.

> subql-node --version
0.19.1

-w, --workers

This will move block fetching and processing into a worker. By default, this feature is disabled. You can enable it with the --workers=<number> flag.

Note that the number of available CPU cores strictly limits the usage of worker threads. So, when using the --workers=<number> flag, always specify the number of workers. With no flag provided, everything will run in the same thread.

Tip

It can increase performance by up to 4 times. Give it a try and let us know your feedback!

It is at an early experimental stage at the moment, but we plan to enable it by default.

On initialisation, once the main thread is established, then the fetching and processing workload is distributed across all worker threads. Each worker has their own buffer (a set of blocks that they are responsible to fetch/process). For example:

Worker A: Will execute the fetch and indexing of blocks [n,..n+10]
Worker B: Will execute the fetch and indexing of blocks [n+11,..n+20]
Worker C: Will execute the fetch and indexing of blocks [n+21,..n+30]
Then repeat with n = n + 30

In the case where Worker C completes its fetch prior to Worker A and B, it will remain in an idle state until A and B have completed, as the processing phase executes sequentially.

subql-query

--help

This shows the help options.

Options:
      --help          Show help                                          [boolean]
      --version       Show version number                                [boolean]
  -n, --name          Project name                             [string] [required]
      --playground    Enable graphql playground                          [boolean]
      --subscription  Enable subscription               [boolean] [default: false]
      --output-fmt    Print log as json or plain text
                        [string] [choices: "json", "colored"] [default: "colored"]
      --log-level     Specify log level to print.
            [string] [choices: "fatal", "error", "warn", "info", "debug", "trace",
                                                       "silent"] [default: "info"]
      --log-path      Path to create log file e.g ./src/name.log          [string]
      --log-rotate    Rotate log files in directory specified by log-path
                                                      [boolean] [default: false]
      --indexer       Url that allows query to access indexer metadata    [string]
      --unsafe        Disable limits on query depth and allowable number returned
                      query records and enables aggregation functions                                          [boolean]
  -p, --port          The port the service will bind to                   [number]

--aggregate

Enables or disables the GraphQL aggregation feature, read more about this here. By default this is set to true.

disable-hot-schema

Disables the hot reload schema on project schema changes, by default this is set to false.

--indexer

Set a custom url for the location of the endpoints of the indexer, the query service uses these endpoints for indexer health, metadata and readiness status.

--log-level

See --log-level.

--log-path

Enable file logging by providing a path to a file to log to.

--log-rotate

Enable file log rotations with the options of a 1d rotation interval, a maximum of 7 files and with a max file size of 1GB.

--max-connection

The maximum simultaneous connections allowed to this GraphQL query service expressed as a positive integer. The default value is 10.

-n, --name

This flag is used to start the query service. If the --subquery-name flag is not provided when running an indexer, the name here will refer to the default project name. If --subquery-name is set, then the name here should match what was set.

> subql-node -f . // --subquery-name not set

> subql-query -n subql-helloworld  --playground // the name defaults to the project directory name

> subql-node -f . --subquery-name=hiworld // --subquery-name set

> subql-query -n hiworld --playground  // the name points to the subql-helloworld project but with the name of hiworld

--output-fmt

See --output-fmt.

--playground

This flag enables the graphql playground so should always be included by default to be of any use.

--playground-settings

You can use this flag to pass additional settings to the GraphQL playground (in JSON format). Additional settings can be found here https://github.com/graphql/graphql-playground#settings

--port

The port the subquery query service binds to. By default this is set to 3000

--query-complexity

The level of query complexity that this service will accept expressed as a positive integer. By default this is set to 10. If a client makes a query with a query complexity higher than this level, the GraphQL query service will reject the request.

We use the graphqql-query-complexityopen in new window plugin to calculate this value.

--query-limit

The query service by default has a limit of 100 entities for to prevent unbounded GraphQL queries and encourage the use of pagination. This flag accepts a positive integer value that will change this limit (by default this is set to 100). Setting a high value may cause performance issues on the query service, it is recommended instead that queries are paginatedopen in new window.

--query-timeout

The timeout for long running graphql queries expressed in milliseconds, by default this value is 10000 milliseconds

--subscription

This flag enables GraphQL Subscriptions, to enable this feature requires subql-node also enable --subscription.

--unsafe (Query Service)

This flag enables certain aggregation functions including sum, max, avg and others. Read more about this feature here.

These are disabled by default for database performance reasons.

Note that must be on a Partner plan if you would like to run projects with the --unsafe command (on the query service) within SubQuery's Managed Serviceopen in new window. Additionally, it will prevent your project from being run in the SubQuery Network in the future.

--version

This displays the current version.

> subql-query --version
0.7.0

Command Line Flags

# Command Line Flags

# subql (cli)

# --help

# build

# subql-node

# --help

# --batch-size

# -c, --config

# -d, --network-dictionary

# --dictionary-timeout

# --db-schema

# --debug

# --disable-historical

# -f, --subquery

# force-clean

# --local (deprecated)

# --log-level

# --multi-chain

# --network-endpoint

# --output-fmt

# -p, --port

# --pg-ca

# --pg-cert

# --pg-key

# --profiler

# reindex

# --scale-batch-size

# --subscription

# --subquery-name (deprecated)

# --timeout

# --timestamp-field

# --unfinalized-blocks

# --unsafe (Node Service)

# --version

# -w, --workers

# subql-query

# --help

# --aggregate

# disable-hot-schema

# --indexer

# --log-level

# --log-path

# --log-rotate

# --max-connection

# -n, --name

# --output-fmt

# --playground

# --playground-settings

# --port

# --query-complexity

# --query-limit

# --query-timeout

# --subscription

# --unsafe (Query Service)

# --version

Command Line Flags

subql (cli)

--help

build

subql-node

--help

--batch-size

-c, --config

-d, --network-dictionary

--dictionary-timeout

--db-schema

--debug

--disable-historical

-f, --subquery

force-clean

--local (deprecated)

--log-level

--multi-chain

--network-endpoint

--output-fmt

-p, --port

--pg-ca

--pg-cert

--pg-key

--profiler

reindex

--scale-batch-size

--subscription

--subquery-name (deprecated)

--timeout

--timestamp-field

--unfinalized-blocks

--unsafe (Node Service)

--version

-w, --workers

subql-query

--help

--aggregate

disable-hot-schema

--indexer

--log-level

--log-path

--log-rotate

--max-connection

-n, --name

--output-fmt

--playground

--playground-settings

--port

--query-complexity

--query-limit

--query-timeout

--subscription

--unsafe (Query Service)

--version