A step-by-step guide on how to configure a mirror node and query specific data on the Hedera Network
Last updated
Hedera Mirror Node is a useful tool that lets developers and users access past transaction data on the Hedera network. You can view and analyze network data such as transactions, transfers, balances, and events from the past in a reliable, scalable, and efficient way. The best feature is the capability to configure your mirror node to query only data that meet your tailored specifications.
This guide provides step-by-step instructions on how to configure and use a Hedera Mirror Node to access past transaction data on the Hedera network. You will learn how to configure your mirror node to store only the latest 90 days of data or data for a specific entity (account, smart contract, etc.) and how to use basic SQL queries to analyze the data.
Basic understanding of Hedera Mirror Nodes.
Basic understanding of terminal commands and SQL.
(openjdk@17: Java version 17), (the latest version), and (the latest version) are installed on your machine.
(>= v20.10.x) installed and open on your machine. Run docker -v in your terminal to check the version you have installed.
Clone the Hedera mirror node repository and navigate to the project directory:
In this example, we will configure the mirror node to store the last 90 days of data or data for a specific account ID (entity). To achieve this, we will need to create and modify an application.yml file. This file contains configuration settings for the mirror node, such as which data to store and how long to store it.
First, create a new configuration folder and file inside the hedera-mirror-importer directory. The Mirror Node importer directory contains the source code for the importer tool, which allows users to import data from the Hedera Mainnet, Testnet, or Previewnet. This creates and imports a read-only instance of the Hedera network data stored in its own database.
Run the following command to create the right folder and file:
The mirror node may be configured only to store a subset of data for entities and/or transaction types of interest — essentially, which rows of data to retain.
Here's an overview the application.yml file. Copy the following lines into application.yml and save it.
Here's a breakdown of what each section of the configuration file does:
behaviorhedera: This is the root section of the configuration file, indicating that the settings apply to the Hedera network.
mirror: This is a sub-section that pertains specifically to the Mirror node.
importer: This sub-section defines settings for the Mirror node's importer, which is responsible for retrieving transaction data from the network and storing it in a local database for querying.
importer.retention: This sub-section specifies the retention period and frequency for importing data. In this case, the importer will clean data that is older than 90 days every 60 seconds. If you omit the frequency key, the default behavior for cleaning data is once a day.
parser: This sub-section defines settings for the data parser, which determines the data that gets stored in the database or the data that should be filtered.
parser.exclude: This specifies the entities or transaction types that should be excluded from the imported data. In this case, the parser.exclude.entity with ID 0.0.111478 is excluded.
parser.include: This specifies the entities or transaction types that should be included from the imported data. In this case, the parser.include.entity with ID 0.0.111478 is included, and two specific transaction types (CRYPTOTRANSFER and CRYPTOCREATEACCOUNT) are included via parser.include.transaction.
You can also combine entity and transaction fields. In our example, we only want to store CONTRACTCREATEINSTANCE transactions for the entity with ID 0.0.111710.
parser.record: This sub-section specifies how the imported data should be recorded. In this case, the entity object is specified, which means that data should be recorded for each unique entity (account) involved in the transactions. The persist setting is set to false, which means that topic data for entities should not be persisted.
period: 90D: This indicates that the importer should retain the imported data for a period of 90 days. After this period, the data will be deleted.
The PostgreSQL container is responsible for creating the database for the mirror node instance, and the REST API container allows you to use the REST APIs to query the mirror node instance. The database stores the transaction data retrieved by the importer component of the mirror node, and the REST API provides an interface for accessing that data using HTTP requests. The importer component is responsible for retrieving the transaction data from the Hedera network and storing it in the database. Let's start up the database!
Open Docker and start the PostgreSQL and REST API containers in the root directory:
Wait until you see the "database system is ready to accept connections" message in the console log, then control + c to terminate the current process.
Now the database is ready, let's import demo data. Run the importer in the same root directory:
This process may take some time, but once you see this in your console and the process is at 92%, you kill the process with control + c. If you let the process run, it will import more data that you don't need for this tutorial.
Note: Should you encounter an error during this step or if the command doesn't execute successfully, it's recommended to run the command again.
To connect the PostgreSQL database, we need to retrieve the database credentials. Open a new terminal window and run the following command in the root directory. Copy the database password that your console returns. You need this for the next step.
In the same directory, run the following command that will connect the database:
Enter the database password when prompted. If you successfully connect to the PostgreSQL database, your console should be ready to execute queries and look something like this:
In this section, you can try out multiple queries that show you how to retrieve data from the PostgreSQL database. You need a basic understanding of SQL queries to craft your own queries.
Execute the following queries to analyze the data stored in your local database.
To exit the psql console, run the quit command:
Lastly, run the following command to stop and remove the created containers:
In this example, we'll use the application.yml format for demonstration purposes. This configuration retains transaction and crypto transfer data for 90 days, excludes data for entity 0.0.111478, and includes specific transactions for entities 0.0.111710 and 0.0.111734. Furthermore, it prevents the storage of topic data. You can check out the other two alternative formats if you don't like working with the YAML format.
importer.network: DEMO: This specifies that the importer should connect to a bucket with demo data. It's the easiest way to experiment with the mirror node and importer. If you want to connect to the TESTNET, MAINNET, or PREVIEWNET, you need to follow this .
importer.retention.include: This specifies the database tables that should be included in the imported data. The tables specified are transaction and crypto_transfer. You can find all tables in the for the mirror node.
See the hedera.mirror.importer.parser.include.* and hedera.mirror.importer.parser.exclude.* properties listed in .
More details about retention and transaction and entity filtering .
Check out the complete list of transaction types in the .
The PostgreSQL database is not an end product; the Hedera Mirror Node REST API is. Hence, SQL queries might fail when the engineering team updates the underlying database model in new releases. This tutorial has been tested with .
Writer: Krystal, Technical Writer
Editor: Michiel, Developer Advocate
1.
2.
3.
4.
5.
|
|
