importblocks CLI Tool

Overview

importblocks is a command-line tool designed to test JAM implementations by generating and sending a stream of blocks (both valid and invalid) to a team-specified HTTP or QUIC endpoint. The importblocks is part of a larger test suite to help teams verify their block validation logic by sending a variety of block scenarios in different modes and checking if an implementation is correctly able to validate blocks. Implementations should determine if the blocks are valid and subsequently advance their chain based on this validation, updating the state trie if valid and outputting key value pairs with a state root. A genesis state/configuration may be supplied.

The specific goal for importblocks to support JAM implementation teams to reach M1 Import Blocks + M2 Author milestones at present. We envision it can support mature JAM implementations beyond this use case as part of a larger advanced JAM Supercomputing Test Suite.

Important: As of December 2024, this is in concept stage only and is a community-generated effort, not from W3F or Parity. Any implementation of importblocks should be considered unofficial and any success metrics, while aiming to be 100% diagnostic, have nothing to do technically for JAM Prize milestone achievements. A more official version of this can be expected to be supplied by W3F (see this issue).

importblocks supports the following modes:

fallback: Generates extrinsic-less blocks.
safrole: Generates blocks with tickets extrinsics only.
assurances: Generates blocks with extrinsics of guarantees, assurances, and preimages in addition to tickets

Additional modes are planned, as documented below.

Features

HTTP and QUIC Communication: Choose between HTTP or QUIC to deliver blocks to a team’s validation endpoint.
Block Generation: Programmatically generates blocks based on different modes, producing valid and invalid cases for testing.
Randomized Validity: Each block generated by importblocks can be valid or invalid, challenging teams to determine the block’s correctness.
Detailed Grading: When enabled, the --verbose flag dumps the correct answer (expected key-value pairs and state root) for comparison with the team’s response.

Installation

(PLANNED/UNDER DEVELOPMENT by JAM Duna team (colorful notion), but any team can build this with their own JAM implementation.)

Prerequisites

Docker: Ensure Docker is installed and running on your system.

Running with Docker

Pull the importblocks Docker image from the GCP container registry:

docker pull gcr.io/jamduna-org/importblocks:latest

Run the importblocks container with the required flags:

docker run --rm \
      gcr.io/jamduna-org/importblocks:latest \
      --mode=fallback --http=http://yourcorejam.network/validate --verbose

Usage

Command Syntax

docker run --rm \
  gcr.io/jamduna-org/importblocks:latest \
  --mode=<mode> [--http=<url> | --quic=<ip:port>] [options]

Modes

fallback: Generates extrinsic-less blocks ( $|E_T|=|E_A|=|E_G|=|E_P|=|E_D|=0$ ).
safrole: Generates blocks with ticket extrinsics where $|E_T| \ge 0$ but no other extrinsics ( $|E_A|=|E_G|=|E_P|=|E_D|=0$ )
assurances: Generates blocks with guarantees ( $|E_G|\ge 0$ ), assurances ( $|E_A|\ge 0$ ), and preimages ( $|E_P| \ge 0$ ) in addition to ticket extrinsics but no disputes ( $|E_D| = 0$ )
orderedaccumulation: same as assurances but also adjusting C(14), including prereqs
authorization: same as assurances, but also adjusting C(1)+C(2)
recenthistory: same as assurances, but also adjusting C(3)
blessed: same as assurances, but also adjusting C(5), C(12)
basichostfunctions: same as assurances, but using most common host functions
finalization: combines all of the above, adjusting everything in C(1)-C(15), except disputes
disputes: combines all of the above , including dispute extrinsics ( $|E_D|\ge 0$ ).
conformance: every single host function

Required Flags

You must specify either an HTTP URL or a QUIC address for the endpoint where importblocks will send blocks.

--http=<url>: HTTP URL where importblocks will POST generated blocks.
--quic=<ip:port>: IP and port for sending generated blocks over QUIC.

Options

--mode <mode>: Sets the mode for block generation. Options are fallback, safrole, assurances, and disputes.
--verbose: Enables detailed logging output and dumps the correct answer (including JSON key vals and state root) for comparison.
--numblocks <numblocks>: runs a test for up to numblocks valid blocks (default: 100).
--invalidrate <invalidrate>: percentage of blocks that are invalid (default: 25).
--statistics <N>: Dumps statistics every N blocks. (default is same as numblocks). Useful for long running tests.
--genesis <file>: Initial genesis state (e.g. genesis.json).
--network <network>: JAM Network size. Options are tiny and full.

Example Usage

Sending Blocks via HTTP

docker run --rm gcr.io/jamduna-org/importblocks:latest \
  --mode=fallback --http=http://yourcorejam.network/validate --verbose`

Sending Blocks via QUIC

docker run --rm gcr.io/jamduna-org/importblocks:latest \
  --mode=safrole --quic=127.0.0.1:9000

How `importblocks` Works

Block Generation: Based on the selected mode, importblocks programmatically generates blocks, which may be valid or invalid. The rate of invalid block generation is controlled by invalidrate
Endpoint Submission: importblocks sends the generated blocks to the specified HTTP or QUIC endpoint. As soon as a response is received, another block is sent.
Team Validation: It is up to the endpoint to determine whether each block is valid or invalid. Upon receiving a valid block, the endpoint should advance its state accordingly and return a full state trie dump and a state root.
Response Handling: importblocks expects for HTTP endpoint:
- 200 OK: For a valid block, the endpoint should return a JSON response with:
  - keyvals: A set of key-value pairs representing the state trie.
  - stateroot: The state root hash of the trie.
- 406 Bad Request: For an invalid block, the endpoint should return a 406 Bad Request response.
A similar expectation for QUIC endpoints will be devised.

Example of a Team’s Expected Response

If a block is valid, the team’s endpoint should return a 200 OK status with a JSON response containing a keyvals section (key-value pairs representing the state trie) and a stateroot (state root hash). For an invalid block, the endpoint should return a 400 Bad Request response.

Example response for a valid block:

{
  "keyvals": [
        [
            "0x00fe00ff00ff00ff9fe68beeb224fffce8d74982e6e37f554e670548e26ac754",
            "0x000000000000001000000084000000000072051100000005100000000518000000055f04071300040a0400fffe040b24040713000211f8031004031504050000fffe01582004070000fffe04090020040a0010040b0030040c00404e090d0503570404090400fffe04070000fffe040804040a044e03011004011502110813000407130021842a4825050922222a4190945201"
        ],
        [
            "0x0100000000000000000000000000000000000000000000000000000000000000",
            "0x0000"
        ],
        ...
  ],
  "stateroot": "0xd3adb33f..."
}

In this example, keyvals contains the state trie’s key-value pairs and stateroot represents the posterior state root hash of the trie using those keyvals. Both keys and values must be 0x prefixed. The genesis.json file that may be supplied as a genesis parameter must obey the same format.

Example response for an invalid block:

The endpoint should simply return an HTTP 406 Not Acceptable with a JSON with an error text attribute.

{
  "error": "Epoch marker expected"
}

Any free form text is acceptable.

Verbose Mode

When --verbose is enabled, importblocks will log the expected answer, which includes the correct keyvals and stateroot. This helps teams compare their endpoint’s output with the expected values, facilitating debugging and validation of the implementation.

Grading Methods

Multiple grading methods are supported:

binary: Grades correctness based on 200 vs 406 (http) alone (and analogous equivalent for QUIC), independent of the content of the key-value pairs or stateroot.
keyvals: Grades correctness based on key-value pairs for valid blocks.
stateroot: Grades correctness based on state root exact match alone for valid blocks.

JAM Network Settings

Blocks are sent by importblocks as fast as the implementation can provide responses, without regard to JCE Time.

Different network sizes are supported: full and tiny, following the chain spec.

Private / Public Keys for $V$ participants can be programmatically generated for different genesis configurations:

https://github.com/jam-duna/jamtestnet/tree/main/key

For tiny: https://docs.jamcha.in/basics/dev-accounts

Development Roadmap

As of December 2024, this is a Proof of Concept but can be reduced to practice. JAM Implementation Teams are encouraged to provide a list of fuzzing ideas and general feedback.

Full QUIC Implementation: Development of QUIC functionality for a robust testing setup.
Expanded Modes and Fuzzing: More modes and fuzzing for block generation.
Enhanced Grading and Statistics: Additional grading parameters as determined to be useful.

RFP for Multiple `importblocks` Implementations

For the same reasons JAM is being implemented by multiple teams in different languages to support resilient and non-buggy implementations (no 2 implementations having the same bug, in principle), it may be desirable for multiple teams to build different implementations of importblocks.

importblocks Implementers would likely use their own JAM implementation as a source of blocks, and likely will be closed source.

In addition to being JAM Implementers, ideal teams will be able to build other components of the JAM Trustless Supercomputing Test Suite.

Overview​

Features​

Installation​

Prerequisites​

Running with Docker​

Usage​

Command Syntax​

Modes​

Required Flags​

Options​

Example Usage​

Sending Blocks via HTTP​

Sending Blocks via QUIC​

How importblocks Works​

Example of a Team’s Expected Response​

Example response for a valid block:​

Example response for an invalid block:​

Verbose Mode​

Grading Methods​