Skip to main content

NEAR Manifest File

SubQuery TeamAbout 6 min

NEAR Manifest File

The Manifest project.ts file can be seen as an entry point of your project and it defines most of the details on how SubQuery will index and transform the chain data. It clearly indicates where we are indexing data from, and to what on chain events we are subscribing to.

The Manifest can be in either Typescript, Yaml, or JSON format.

With the number of new features we are adding to SubQuery, and the slight differences between each chain that mostly occur in the manifest, the project manifest is now written by default in Typescript. This means that you get a fully typed project manifest with documentation and examples provided your code editor.

Below is a standard example of a basic project.ts.

import {
  NearDatasourceKind,
  NearHandlerKind,
  NearProject,
} from "@subql/types-near";

const project: NearProject = {
  // This project can be use as a starting point for developing your new NEAR SubQuery project
  specVersion: "1.0.0",
  name: "near-subql-starter",
  version: "0.0.1",
  runner: {
    node: {
      name: "@subql/node-near",
      version: "*",
    },
    query: {
      name: "@subql/query",
      version: "*",
    },
  },
  description:
    "This project can be use as a starting point for developing your new NEAR SubQuery project",
  repository: "https://github.com/subquery/near-subql-starter",
  schema: {
    // This endpoint must be a public non-pruned archive node
    // We recommend providing more than one endpoint for improved reliability, performance, and uptime
    // Public nodes may be rate limited, which can affect indexing speed
    // When developing your project we suggest getting a private API key from a commercial provider
    file: "./schema.graphql",
  },
  network: {
    chainId: "mainnet",
    endpoint: ["https://archival-rpc.mainnet.near.org"],
    // Optionally provide the HTTP endpoint of a full chain dictionary to speed up processing
    dictionary: "https://api.subquery.network/sq/subquery/near-dictionary",
    // This is a missing block from the NEAR mainnet chain that we are skipping
    bypassBlocks: [81003306],
  },
  dataSources: [
    {
      kind: NearDatasourceKind.Runtime,
      // You can set any start block you want here. This block was when the sweat_welcome.near address was created
      startBlock: 80980000,
      mapping: {
        file: "./dist/index.js",
        handlers: [
          // Using block handlers slows your project down as they can be executed with each and every block. Only use if you need to
          // {
          //   handler: "handleBlock",
          //   kind: "near/BlockHandler",
          //   filter: {
          //     modulo: 10,
          //   },
          // },
          {
            handler: "handleTransaction",
            kind: NearHandlerKind.Transaction,
            filter: {
              sender: "sweat_welcome.near",
              receiver: "token.sweat",
            },
          },
          {
            handler: "handleAction",
            kind: NearHandlerKind.Action,
            filter: {
              type: "FunctionCall",
              methodName: "storage_deposit",
              receiver: "token.sweat",
            },
          },
          // Some other filter examples
          // {
          //   handler: "handleAction",
          //   kind: NearHandlerKind.Action,
          //   filter: {
          //     type: "DeleteAccount",
          //     beneficiaryId: "",
          //   },
          // },
          // {
          //   handler: "handleAction",
          //   kind: NearHandlerKind.Action,
          //   filter: {
          //     type: "AddKey",
          //     publicKey: "",
          //     accessKey: "",
          //   },
          // },
          // {
          //   handler: "handleAction",
          //   kind: NearHandlerKind.Action,
          //   filter: {
          //     type: "DeleteKey",
          //     publicKey: "",
          //   },
          // },
          // {
          //   handler: "handleAction",
          //   kind: NearHandlerKind.Action,
          //   filter: {
          //     type: "Stake",
          //     publicKey: "",
          //   },
          // },
        ],
      },
    },
  ],
};

// Must set default to the project instance
export default project;

Below is a standard example of the legacy YAML version (project.yaml).

Legacy YAML Manifest
specVersion: 1.0.0

name: near-subql-starter
version: 0.0.1
runner:
  node:
    name: "@subql/node-near"
    version: "*"
  query:
    name: "@subql/query"
    version: "*"
description: "This project can be use as a starting point for developing your new NEAR SubQuery project"
repository: "https://github.com/subquery/near-subql-starter"

schema:
  file: ./schema.graphql

network:
  chainId: mainnet
  # This endpoint must be a public non-pruned archive node
  # We recommend providing more than one endpoint for improved reliability, performance, and uptime
  # Public nodes may be rate limited, which can affect indexing speed
  # When developing your project we suggest getting a private API key from a commercial provider
  endpoint: ["https://archival-rpc.mainnet.near.org"]
  # Optionally provide the HTTP endpoint of a full chain dictionary to speed up processing
  dictionary: https://api.subquery.network/sq/subquery/near-dictionary
  bypassBlocks: [81003306] # This is a missing block from the NEAR mainnet chain that we are skipping
dataSources:
  - kind: near/Runtime
    startBlock: 80980000 # You can set any start block you want here. This block was when the sweat_welcome.near address was created
    mapping:
      file: "./dist/index.js"
      handlers:
        # Using block handlers slows your project down as they can be executed with each and every block. Only use if you need to
        # - handler: handleBlock
        #   kind: near/BlockHandler
        #   filter:
        #     modulo: 10
        - handler: handleTransaction
          kind: near/TransactionHandler
          filter:
            sender: sweat_welcome.near
            receiver: token.sweat
        - handler: handleAction
          kind: near/ActionHandler
          filter:
            type: FunctionCall
            methodName: storage_deposit
            receiver: token.sweat

Overview

Top Level Spec

FieldTypeDescription
specVersionStringThe spec version of the manifest file
nameStringName of your project
versionStringVersion of your project
descriptionStringDiscription of your project
repositoryStringGit repository address of your project
schemaSchema SpecThe location of your GraphQL schema file
networkNetwork SpecDetail of the network to be indexed
dataSourcesDataSource SpecThe datasource to your project
templatesTemplates SpecAllows creating new datasources from this templates
runnerRunner SpecRunner specs info

Schema Spec

FieldTypeDescription
fileStringThe location of your GraphQL schema file

Network Spec

If you start your project by using the subql init command, you'll generally receive a starter project with the correct network settings. If you are changing the target chain of an existing project, you'll need to edit the Network Spec section of this manifest.

The chainId is the network identifier of the blockchain. In NEAR, it's either mainnet or testnet.

Additionally you will need to update the endpoint. This defines the (HTTP or WSS) endpoint of the blockchain to be indexed - this must be a full archive node. This property can be a string or an array of strings (e.g. endpoint: ['rpc1.endpoint.com', 'rpc2.endpoint.com']). We suggest providing an array of endpoints as it has the following benefits:

  • Increased speed - When enabled with worker threads, RPC calls are distributed and parallelised among RPC providers. Historically, RPC latency is often the limiting factor with SubQuery.
  • Increased reliability - If an endpoint goes offline, SubQuery will automatically switch to other RPC providers to continue indexing without interruption.
  • Reduced load on RPC providers - Indexing is a computationally expensive process on RPC providers, by distributing requests among RPC providers you are lowering the chance that your project will be rate limited.

Public nodes may be rate limited which can affect indexing speed, when developing your project we suggest getting a private API key from a professional RPC provider.

FieldTypeDescription
chainIdStringA network identifier for the blockchain
endpointStringDefines the RPC endpoint of the blockchain to be indexed - This must be a full archive node. You can retrieve endpoints for all parachains for free from OnFinalityopen in new window
portNumberOptional port number on the endpoint to connect to
dictionaryStringIt is suggested to provide the HTTP endpoint of a full chain dictionary to speed up processing - read how a SubQuery Dictionary works. For NEAR mainnet this is provided (we don't run a dictionary for the testnet)
bypassBlocksArrayBypasses stated block numbers, the values can be a range(e.g. "10- 50") or integer, see Bypass Blocks

Runner Spec

FieldTypeDescription
nodeRunner node specDescribe the node service use for indexing
queryRunner query specDescribe the query service

Runner Node Spec

FieldTypeDescription
nameString@subql/node-near
versionStringVersion of the indexer Node service, it must follow the SEMVERopen in new window rules or latest, you can also find available versions in subquery SDK releasesopen in new window
optionsRunner Node OptionsRunner specific options for how to run your project. These will have an impact on the data your project produces. CLI flags can be used to override these.

Runner Query Spec

FieldTypeDescription
nameStringWe currently support @subql/query
versionStringVersion of the Query service, available versions can be found hereopen in new window, it also must follow the SEMVER rules or latest.

Runner Node Options

Fieldv1.0.0 (default)Description
historicalBoolean (true)Historical indexing allows you to query the state at a specific block height. e.g A users balance in the past.
unfinalizedBlocksBoolean (false)If enabled unfinalized blocks will be indexed, when a fork is detected the project will be reindexed from the fork. Requires historical.
unsafeBoolean (false)Removes all sandbox restrictions and allows access to all inbuilt node packages as well as being able to make network requests. WARNING: this can make your project non-deterministic.

Datasource Spec

Defines the data that will be filtered and extracted and the location of the mapping function handler for the data transformation to be applied.

FieldTypeDescription
kindStringnear/Runtime
startBlockIntegerThis changes your indexing start block for this datasource, set this as high as possible to skip initial blocks with no relevant data
endBlockIntegerThis sets a end block for processing on the datasource. After this block is processed, this datasource will no longer index your data.

Useful when your contracts change at a certain block height, or when you want to insert data at genesis. For example, setting both the startBlock and endBlock to 320, will mean this datasource only operates on block 320
mappingMapping Spec

Mapping Spec

FieldTypeDescription
handlers & filtersDefault handlers and filtersList all the mapping functions and their corresponding handler types, with additional mapping filters.

Data Sources and Mapping

In this section, we will talk about the default NEAR runtime and its mapping. Here is an example:

{
  ...
  dataSources: [
    {
      kind: NearDataSourceKind.Runtime, // Indicates that this is default runtime
      startBlock: 1, // This changes your indexing start block, set this higher to skip initial blocks with less data
      mapping: {
        file: "./dist/index.js", // Entry path for this mapping
        handlers: [
          /* Enter handers here */
        ],
      }
    }
  ]
}

Mapping Handlers and Filters

The following table explains filters supported by different handlers.

Your SubQuery project will be much more efficient when you only use TransactionHandler or ActionHandler handlers with appropriate mapping filters (e.g. NOT a BlockHandler).

HandlerSupported filter
near/BlockHandlermodulo, timestamp
near/TransactionHandlersender, reciever
near/ActionHandlertype, sender, receiver, methodName, 'args', 'publicKey', 'accessKey', 'beneficiaryId'

Default runtime mapping filters are an extremely useful feature to decide what block, transaction, or action will trigger a mapping handler.

Only incoming data that satisfy the filter conditions will be processed by the mapping functions. Mapping filters are optional but are highly recommended as they significantly reduce the amount of data processed by your SubQuery project and will improve indexing performance.

# Example filter from TransactionHandler
filter:
  sender: sweat_welcome.near
  receiver: token.sweat

# Example filter from ActionHandler:
filter:
  type: FunctionCall
  methodName: 'storage_deposit'

The modulo filter allows handling every N blocks, which is useful if you want to group or calculate data at a set interval. The following example shows how to use this filter.

filter:
  modulo: 50 # Index every 50 blocks: 0, 50, 100, 150....

Action Types

There are several types of actions as defined hereopen in new window

Bypass Blocks

Bypass Blocks allow you to skip the stated blocks, this is useful when there are erroneous blocks in the chain or when a chain skips a block after an outage or a hard fork. It accepts both a range or single integer entry in the array.

When declaring a range use an string in the format of "start - end". Both start and end are inclusive, e.g. a range of "100-102" will skip blocks 100, 101, and 102.

{
  network: {
    bypassBlocks: [1, 2, 3, "105-200", 290];
  }
}