Get Started with the Hedera Consensus Service Fabric Plugin

You must have a basic understanding of the Hyperledger Fabric network, its key concepts, first-network sample, and transaction flow. Please visit the Hyperledger Fabric docs to familiarize yourself with these concepts if you have not done so already.

Background

Hyperledger Fabric is one of the most popular private/permissioned enterprise blockchain frameworks available today. Fabric's modular architecture approach enables users to specify network components like network members and choice of consensus protocol (ordering service).

In this tutorial, you will create a Hyperledger Fabric network that leverages the Hedera Consensus Service Fabric plug-in to use Hedera as the ordering service via the first-network sample. The HCS Hyperledger Fabric network will be composed of two organizations. Each organization will host two peer nodes. The two organizations will privately communicate and transact within a Hyperledger Fabric channel.

If you already have experience with the Hyperledger Fabric network and the first-network sample, you can skip down to the requirements section to get started.

Transaction Flow

1. A client application creates a transaction proposal and sends it to Hyperledger Fabric network peers. The transaction proposal is endorsed (signed) by the Hyperledger Fabric network peers. The endorsed transaction proposal is sent back to the client application.

2. Client application submits the endorsed transaction to the Hyperledger Fabric network HCS ordering node. That Hyperledger Fabric ordering node interacts with the HCS fabric plugin as the ordering service

3. Fabric transactions are fragmented into many messages and submitted to the Hedera network against a particular topic ID

4. Hedera mainnet nodes timestamp and order the fragmented messages

5. Hedera mirror nodes read the fragmented messages in consensus order from the Hedera mainnet

6. Hedera mirror node relays the ordered messages back to the Hyperledger Fabric ordering node

7. Hyperledger Fabric ordering nodes reassembles the fragmented messages into fabric transactions, form fabric blocks, and communicate the blocks to the Hyperledger Fabric peer nodes

8. Hyperledger Fabric peer nodes update to have the same world state

Concepts

A basic description of the concepts for the purposes of this tutorial.

Client Application

A client application communicates to a Hyperledger Fabric peer node to submit a transaction proposal and collects the endorsed (signed) transaction proposal. The transaction proposal may initiate a modification of the current state of the distributed ledger.

Hedera Consensus Service Fabric Plugin

A consensus service plugin for the Hyperledger Fabric network that leverages the Hedera Consensus Service to order transactions using the unique properties of the hashgraph consensus algorithm that powers the Hedera network.

Hedera Mirror Node

A Hedera mirror node reads the ordered messages from a Hedera mainnet node and communicates that information back to the Hyperledger Fabric network. Each message has the following information:

• ConsensusTimestamp

• Message Body

• Topic Running Hash

• Topic Sequence Number

Hedera Mainnet Node

A Hedera mainnet node receives the Fabric transaction in fragmented message transactions from the HCS Hyperledger ordering node. That mainnet node gossips the transactions to the other mainnet nodes which collectively assigns the transactions a consensus timestamp and order within the corresponding topic.

fabric-hcs Repository

The project that contains all the necessary files to run the HCS Hyperledger Fabric network sample.

Hyperledger Fabric Network

The Hyperledger Fabric network is composed of peer nodes, ordering nodes, chaincode (smart contracts), organizations, channels, a distributed ledger and an ordering service.

Hyperledger Fabric Network Peers

Peers are nodes in the Hyperledger Fabric network that host instances of the ledger and instances of the chaincode. Client applications interact with the peer nodes to endorse a transaction proposal.

Hyperledger Fabric Channels

Channels are a mechanism by which organizations can communicate and transact privately with one another.

Hyperledger Fabric Organizations

Organizations are participants that would like to communicate and transact privately with one another.

Hyperledger Fabric Orderer

A node that orders the transactions received from a client application (also known as an "ordering node"). In this case, the ordering node interacts with the HCS fabric plugin as the ordering service.

Requirements:

If you would like to run the sample using a virtual environment, please follow the instructions here.

Hedera Consensus Service

• Testnet account ID and private key

  • Please follow the instructions here

pluggable-hcs Repository

• You will be directed to clone this repository in the outlined steps below

The HCS Fabric plugin supports Fabric ordering service 2.0 and is compatible with older versions peers.

Hyperledger Fabric Network

Download and install the following if you do not already have them on your computer.

• Git

  • Download Git
  • Check to see if you have it installed from your terminal: git --version

• cURL

  • Download cURL
  • Check to see if you have it installed from your terminal: curl --version

• wget

  • Install from your terminal (MacOS):brew install wget

  • Check to see if you have it installed from your terminal: wget --version

• Docker and Docker Compose

  • Download Docker
  • Docker version 17.06.2-ce or greater is required

  • Check to see if you have it installed from your terminal: docker --version && docker-compose --version

• Go Programming Language

  • Download Go
  • Go version 1.13.x is required

  • Check to see if you have it installed from your terminal: go version

• make

  • Install from your terminal (MacOS): brew install make

  • Check to see if you have installed from your terminal: make --version

• gcc

  • Check to see if you have it installed from your terminal: gcc --version

Additionally, you may reference Hyperledger Fabric's documentation for the prerequisites.

Terminal/IDE

You should be able to use the commands provided in this tutorial in terminal prompt or IDE of choice.

1. Clone the pluggable-hcs Project

• Open your terminal or favorite IDE

• Enter the following commands to set-up your environment variables (required)

  • Optionally you can make these settings permanent by placing them in the appropriate startup file, such as your personal ~/.bashrc file if you are using the bash shell under Linux.

$ export GOPATH=$HOME/go
$ export PATH=$PATH:$GOPATH/bin

• Create a hyperledger directory and navigate into that directory

$ mkdir -p $GOPATH/src/github.com/hyperledger && cd $GOPATH/src/github.com/hyperledger

• Clone the pluggable-hcs repository and rename it to fabric

• You must rename the folder to fabric otherwise you will run into issues in the following steps

$ git clone https://github.com/hyperledger-labs/pluggable-hcs fabric
$ cd fabric

• Confirm you are on the main branch

$ git branch

⭐ You have now successfully set your Go path variables and installed the pluggable-hcs/fabric repository.

2. Build Fabric Binaries and Docker Images

• cd to the fabric directory if you are not there already from your terminal or favorite IDE

$ cd fabric

• Follow the commands below to build the required fabric binaries and docker images

  • Note: This process may take a few minutes to complete

  • Note: Newer versions of Docker enable gRPC FUSE file sharing implementation by default which result in a failure. Switch to osxfs (Legacy) in Docker Settings

$ make clean
$ make configtxgen configtxlator cryptogen orderer peer docker

3. Hedera Network & HCS Hyperledger Fabric Orderer Configuration

You will now enter your Hedera testnet account ID and private key information to the relevant configuration files. If you have not previously generated your testnet account, please follow the instructions here.

• Navigate to the first-network directory from your terminal or favorite IDE

$ cd first-network

• Edit the hedera_env.json file via the terminal or IDE

  • This sets up the Hedera environment and allows you to create topics and submit messages to the Hedera network. The Hedera account entered here pays for the transaction fees associated with creating and submitting messages.

  • If you are using an IDE, you may skip this command and edit the file

$ nano hedera_env.json

• Enter your Hedera account ID (e.g. 0.0.1234) in the operatorId field

• Enter your account private key in the operatorKey field

{
    "operatorId": "put your testnet account id here",
    "operatorKey": "put your account private key here",
    "nodeId": "",
    "nodeAddress": "",
    "network": {
        "0.testnet.hedera.com:50211": "0.0.3",
        "1.testnet.hedera.com:50211": "0.0.4",
        "2.testnet.hedera.com:50211": "0.0.5",
        "3.testnet.hedera.com:50211": "0.0.6"
    },
    "mirrorNodeAddress": "hcs.testnet.mirrornode.hedera.com:5600"
}

• Save the hedera_env.json file

• Open the orderer.yaml file from your terminal or favorite IDE

  • Here you will set the configurations for the HCS Hyperledger Fabric orderer node

    • This file configures hcs as the ordering service

    • This also contains the node address and node ID to interact with the Hedera mainnet nodes

    • The Hedera mirror node address to receive ordered transaction from

  • If you are using an IDE, you may skip this command and edit the file

$ nano orderer.yaml

• Scroll down to the "SECTION: Hcs" heading

• Enter your Hedera account ID in the Operator.Id field

• Enter your Hedera account private key in the Operator.PrivateKey.Key field

# operator set for the orderer
Operator:
    Id: Your Hedera testnet account id here
    PrivateKey:
        # type of the private key
        Type: ed25519
        # key string (hex or PEM encoded) or path to the key file
        Key: Your Hedera testnet account private key string here

• Save the orderer.yaml file

Please make sure you have entered your Hedera information correctly with no syntax errors as it will cause issues when trying to run the network later.

⭐ You have now successfully set up your configuration variables for the Hedera operator and for the HCS Hyperledger ordering node.

4. Run Your Network

In this step you will create your HCS Hyperledger Fabric Network.

Make sure you are within the first-network directory before running these commands.

• Enter the following command to start the HCS Hyperledger Fabric network

$ ./byfn.sh up -t 20

• Enter "Y" to accept the 20 second timeout and CLI delay of 3 seconds

Please note it's necessary to set the timeout to 20 seconds since otherwise some checks in the script may fail due to HCS having a larger consensus delay than raft/kafka based ordering service.

• The script generates two HCS topics

  • One topic will be for the Hyperledger Fabric system channel

  • One topic will be for the Hyperledger Fabric application channel

  • HCS topics n this example are configured so that anyone can submit messages to them

  • HCS messages can be configured to be private topics where the sender would require the submitKey to successfully publish messages to them

• Transactions submitted to either of these channels will be visible from a mirror node explorer

  • Transactions can be fragmented into smaller chunks resulting in multiple HCS messages for a single transaction

The sample is almost identical with the upstream first-network sample as of 2.0.0 release. In summary, the modifications are:

1. Updated docker compose files to use hcs-based orderers.

2. Added a function in byfn.sh to generate a 256-bit AES key for data encryption/decryption between fabric orderers and hedera testnet.

3. Added a function in byfn.sh to use the hcscli tool for HCS topic creation and mapping topics to fabric channels.

A successful run will end with the following message:

`========= All GOOD, BYFN execution completed ===========

| _| | | | |** \ | _| | | | | | | |_ _| |**_ | | | | || | |__| || \| |__/`

5. Tear down the network

• It is required to run the following command to tear down the network

• If you do not tear down the network and try to restart the network you may run into issues

$ ./byfn.sh down

6. Verify Topics & Messages

• Topics and messages created in this tutorial can be verified on any available mirror node explorer

• At the start of the script, you can see the two HCS topic IDs that were created

installing hcscli ... generated HCS topics: 0.0.23419 0.0.23420 0.0.23419 will be used for the system channel, and 0.0.23420 will be used for the application channel

• Visit a Hedera mirror node explorer to verify the topics and messages that were created on testnet by searching the two topic IDs
• You will be able to view the application channel and system channel topics and all associated messages from this example

  • A single Fabric transaction sent to an ordering node could result in multiple HCS consensus messages as HCS messages have a 6k message size limit

  • i.e. there may not be a 1:1 correlation between a Fabric transaction and HCS message

• A fabric transaction payload is encrypted by the ordering node therefore the subsequent HCS transaction payload is also encrypted

• All messages on the mirror node explorer will be displayed in encrypted format.

Make sure you have selected the testnet network toggle in the explorer as the topics and messages created through this tutorial will not appear on the main network.

You have successfully done the following:

• Created a Hyperledger Fabric network using the HCS Fabric plugin as the ordering service

• Verified the topics and messages created in this example network

Running into issues or have suggestions? Visit the developer advocates in Discord and post your comments to the hedera-consensus-service channel 🤓 .