My Blockchain Journey

Basics & Development

 

 

 

 

Company / Organization: becke.ch 

Scope: 0.0 {language={en};technology--document={ott};organization={becke.ch};interest={business};}

Version: 1.1.0

File-name: becke-ch--blockchain--s0-v1.odt

Author: blockchain--s0-v1@becke.ch

Copyright © 2018 becke.ch – All rights reserved

 

Document Version History

Version

Date

Author

Description

1.0.0

21.05.2018

Raoul Becke

Modified this document according to requirement below 1.0.0.

1.1.0

14.09.2018

Raoul Becke

Modified this document according to requirement below 1.1.0.

 

 

Module / Artifact / Component / Work-Product Version History

Version

Date

Author

Requirements

Components Changed

1.0.0

21.05.2018

Raoul Becke

Create documentation on: Blockchain, Ethereum Overview (Networks, Nodes, Cryptocurrency Wallet, Smart Contracts, DApps, Ether, Ethereum Virtual Machine), Nodes (Parity, Infura), Cryptocurrency Wallet (MetaMask), Testnet (Ropsten), Solidity Programming Language (Development, Compilation), web3.js (Installation, Integration, Instantiation, Sign Transaction, Callbacks, API), Deployment (web3), Truffle,  Tools

This document

1.1.0

14.09.2018

Raoul Becke

Restructured the entire document, new “Introduction and Overview” chapter, new “Architecture Overview” chapter, new chapter on “Fork”, reworked chapter “Parity” regarding Docker and Proof of Authority, extended chapter "Solidity – Smart Contract Programming Language" regarding samples and language specification, reworked chapter "web3.js - Ethereum JavaScript API" regarding packages and security, moved chapter “Truffle” in the appendix, moved chapter “Azure Blockchain Workbench” in the appendix, moved all “Error & Solution” chapters in the appendix

This document

 

 

 

 

 

Table of Contents

1. Introduction and Overview

1.1. Architecture Overview

1.1.1. Node address format

1.1.2. Communication Protocol

1.1.2.1. Light Ethereum Subprotocol (LES)

1.1.2.2. JSON RPC

1.1.3. Information Flow

1.1.4. Interfaces

2. Definitions

2.1. Fork

2.2. Wallet

2.2.1. HD (Hierarchical Deterministic) wallet & mnemonics

3. Ethereum

3.1. Networks

3.2. The Ethereum Virtual Machine

3.2.1. Overview

3.2.2. Accounts

3.2.3. Transactions

3.2.4. Gas

3.2.5. Storage, Memory and the Stack

3.2.6. Instruction Set

3.2.7. Message Calls

3.2.8. Delegatecall / Callcode and Libraries

3.2.9. Logs

3.2.10. Create

3.2.11. Self-destruct

3.3. Parity (Node)

3.3.1. Installation

3.3.1.1. Docker

3.3.2. Setup & Run

3.3.2.1. Docker

3.3.3. JSON RPC

3.3.4. Node Configuration

3.3.4.1. config file

3.3.4.2. cli parameters

3.3.5. Networks & Chains (Configuration)

3.3.5.1. Private chains

3.3.6. Proof of Authority (Consensus Algorithm Configuration)

3.3.6.1. Setting up a Proof of Authority network

3.3.6.2. validators

3.3.6.2.1. Non-reporting contract: safeContract

3.3.6.2.2. Reporting Contract: contract

3.3.7. Parity UI

3.3.8. Infura

3.4. MetaMask (Wallet)

3.4.1. Installation and configuration

3.4.1.1. Import existing DEN (from Ganache)

3.4.2. Import accounts

3.5. Ropsten (Test Network)

3.5.1. Create Account

3.5.2. Faucet

3.6. Solidity – Smart Contract Programming Language

3.6.1. Development

3.6.1.1. Structure of a Contract

3.6.1.1.1. State Variables

3.6.1.2. Types

3.6.1.2.1. Value Types

address

enum

3.6.1.2.2. Reference Types

Arrays

Structs

3.6.1.2.3. mapping

3.6.1.2.4. delete

3.6.1.3. Units and Globally Available Variables

3.6.1.3.1. Ether Units

3.6.1.3.2. Time Units

3.6.1.3.3. Special Variables and Functions

Block and Transaction Properties

3.6.1.4. Expressions and Control Structures

3.6.1.4.1. Function Calls

3.6.1.5. Contracts

3.6.1.5.1. Visibility and Getters

3.6.1.5.2. Functions

Fallback Function

3.6.2. Compilation

3.7. web3.js - Ethereum JavaScript API

3.7.1. Installation & Integration

3.7.1.1. NPM

3.7.1.2. HTML & JavaScript

3.7.1.3. fs

3.7.2. Instantiation – setting a provider

3.7.2.1. Proprietary (HD Wallet) Provider

3.7.3. Account Creation

3.7.3.1. web3.eth.accounts.create

3.7.3.2. Java

3.7.4. Sign Transaction

3.7.4.1. web3.eth.accounts.wallet

3.7.4.1.1. JavaScript

3.7.4.2. web3.eth.accounts.signTransaction

3.7.4.2.1. JavaScript

3.7.4.3. web3.eth.personal.unlockAccount

3.7.4.4. Proprietary & Tool Specific

3.7.5. Callbacks

3.7.6. big numbers

3.7.7. API

3.8. Deployment

3.8.1. web3

3.8.1.1. wallet

3.8.1.2. signTransaction

4. Tools

4.1. VSCode:

4.1.1. Extension

4.1.2. Solidity Development

4.2. IntelliJ

4.2.1. New Project

4.2.2. Plugin

4.2.3. Solidity Development

4.3. NPM

5. Landscape

6. References and glossary

6.1. References

6.2. Glossary (terms, abbreviations, acronyms)

A. Appendix - Truffle (Ethereum Development Framework)

A.1. Tools

A.2. Development (Ethereum Pet Shop)

A.2.1. Test the version

A.2.2. Create directory structure:

A.2.3. Download and install:

A.2.4. Writing the smart contract

A.2.5. Compilation

A.2.6. Migration (Deployment & Versioning) & Installation of Ganache (Ethereum Node)

A.2.7. Testing the smart contract

A.2.8. Running the tests

A.2.9. Writing Tests in JavaScript

A.2.10. Interacting with your contracts

A.2.11. Package management via EthPM

A.2.12. Package management via NPM

A.2.13. Debugging your contracts

A.2.14. Using Truffle Develop and the console

A.2.15. Writing external scripts

A.2.16. Using the build pipeline

A.2.17. Configuration

A.2.18. Networks and app deployment

A.2.19. Creating a user interface to interact with the smart contract

A.2.20. Interacting with the dapp in a browser

A.2.21. Installing and configuring lite-server

B. Appendix – Azure Blockchain Workbench

B.1. Create Blockchain app

B.1.1. Configuration File

B.1.2. Smart contract code file

B.1.3. Security

B.1.4. Add blockchain application to Blockchain Workbench

C. Appendix – Parity on Docker Installation

D. Errors & Solutions

D.1. Web3.js

D.2. Parity on Docker

D.3. Azure Blockchain Workbench Setup

 

 

Illustration Index

Illustration Index

 Illustration 1: Elimination of Intermediary: Move from Centralized Ledger to Distributed Ledger        8

 Illustration 2: Chain of Blocks        8

 Illustration 3: Blockchain: Ethereum: Information Flow        12

 Illustration 4: Blockchain: Ethereum: Interfaces        13

 Illustration 5: Soft- versus Hard-Fork        14

Illustration 6: Infura: Sign-Up        34

Illustration 7: MetaMask: Add Chrome Extension        36

Illustration 8: MetaMask: Browser Icon        36

 Illustration 9: MetaMask: Privacy Notice        37

 Illustration 10: MetaMask: Create DEN (HD Wallet)        38

 Illustration 11: MetaMask: Mnemonic        38

 Illustration 12: Testnetwork: Ropsten: Metamask: Create Account        40

Illustration 13: Testnetwork: Ropsten: Metamask: Account        40

Illustration 14: Testnetwork: Ropsten: Faucet        41

Illustration 15: Solidity Extension        62

Illustration 16: IntelliJ: New project from existing sources        63

 Illustration 17: IntelliJ: Create project from existing resource        64

Illustration 18: IntelliJ: Project setting        64

Illustration 19: IntelliJ: Import Source Files        65

Illustration 20: IntelliJ: No Frameworks Detected        65

Illustration 21: Intellij-Solidity Plug-In        66

 

 

Index of Tables

Index of Tables

Table 1: References        71

Table 2: Glossary        71

 

1. Introduction and Overview

«Elimination» Intermediary: Centralized Ledger -> Distributed Ledger: One of the main drivers for blockchain is the “elimination” of the intermediary e.g. bank and moving from a centralized- to a distributed-ledger approach where the ledger is not hosted on a central node but instead on all nodes participating in the network. There exist different reasons to do so: I don’t have access to the intermediary, I don’t have trust in the intermediary, the fees of the intermediary are too high, the transaction handling of the intermediary is too slow, etc. On the cons side with the elimination of the intermediary as well some regulatory aspects and expectations enforced by the intermediary are eliminated like for example the know your customer principle in the context of the prevention of money laundering act.

 
 
Ledger -> Blockchain: The ledger is a (time sorted) list of transactions and at a time when a certain number of transactions respective the current block size limit has been reached these transactions are put into a block1. This block is then chained with the previous block by writing the address respective hash-value of the previous block into the current block and last but not least the transactions in the block are secured by calculating the hash-value of the contained transactions and persisting this value as well in the current block. The hash value guarantees that the transactions and previous block cannot be tampered with otherwise the hash value is not valid anymore. The resulting chain is secure and immutable.
Blocks are created aka mined and the hash values calculated by special nodes in the network called mining nodes. Because there exist a lot of nodes that can hand in new transactions into the network and because there exist a lot of mining nodes it can happen that several blocks, containing different set of transactions, are created at the same time and the chain is getting forked. A fork can as well happen when some part of the network and the nodes contained in this network segment are getting separated from the others. But in the end, after some time the chain focuses again on a single branch, namely the longest where most of the nodes are working on. The longest in this context is the branch respective chain where most of the cumulative effort in terms of block hash calculation was invested in. Due to this behavior one should wait until the block containing the transaction has reached a certain depth before shipping the goods related to this transaction to make sure the transaction is not on a temporary branch that is getting reverted.
 
 
Transaction: A transaction contains: the from- respective sender-address2, the to- respective receiver-address, the amount that should be transferred, the transaction-fee the miner gets as reward when mining this transaction and optional some data respective code in case this transaction runs in the context of smart contracts. And last but not least the transaction is signed with the private-key of the sender to secure the transaction and making sure that nobody can modify the content (e.g. amount or receiver) of the transaction.

Network: A network consists of regular-nodes and mining-nodes. There exist separate networks for the different blockchain technologies e.g. bitcoin and ethereum and within the different technologies there exist further separate networks for different  purposes e.g.: production network (the official public network where money/coins are traded), networks for testing and even different networks for private or consortium purpose.

Cryptocurrency: Cryptocurrency is the term for the currency that is used in the context of blockchain when transferring assets (money/coins) within a transaction. A cryptocurrency is bound to a blockchain i.e. cannot be used across different blockchains3 and the value of the currency grows with the number of participants, transactions and goods that are exchanged and of course with the supply and demand. The two most famous cryptocurrencies are: bitcoin and ether.

Coins versus Tokens: Both are cryptocurrencies, but while a coin – Bitcoin, Litecoin, Dogecoin – operates on its own blockchain, a token lives on top of an existing blockchain infrastructure like Ethereum. Coins can be created by starting a new blockchain network (and e.g. giving it the name of your coin) or hard forking an existing blockchain by modifying the underlying software and/or configuration (see chapter 2.1 and e.g. giving the fork the name of your coin). Tokens are created respective traded using smart contracts (see below) which manage the exchange of assets (e.g. a concert ticket / token) for coins.

Consensus Algorithm: To calculate the hash value in a block there exist different approaches aka consensus algorithms used in different networks. The most famous algorithm that is used in the public bitcoin and ethereum network is the proof-of-work algorithm. In the proof-of-work algorithm the difficulty of the algorithm is permanently adjusted to make sure that an average of 6 blocks per hour are getting mined and to make sure that the chain is not spammed with transactions (besides that every transaction costs coins). In private and consortium networks where the participant nodes are known and trusted, the proof-of-authority algorithm is normally getting used. In these networks mining- aka authority-nodes are only added to the network if they are trusted by the other miners. Further consensus algorithms we will not go into are: proof-of-stake, proof-of-burn, proof-of-activity.

Wallet: A wallet stores the public- and private-keys of a participant, is connected to a node and hence meets the conditions to create and participate in transactions. Loosing a wallet means loosing all coins because the private-key to create and sign transactions and thus spending coins belonging to this account (public-key) is lost. And therefore a wallet respective its public and private keys should always be backed up!

Smart-Contract: A smart-contract is a piece of code that runs in the network, has a public-key and can therefore participate in transactions and accordingly receive and spend coins. Furthermore a smart contract exposes an interface (ABI: Application Binary Interface) whose methods can be invoked in the context of a transaction.
A smart contract has/needs no private-key because the smart-contract code and virtual machine (bitcoin, ethereum, etc.) are carefully defined and fully transparent (source code available) and therefore everyone will always agree on what the outcome of the operation was. This includes whether or not the contract sent a transaction to any other contract. Because everyone can agree on the origin of the message, no further degree of verification (such as that provided by a signature) is necessary.

Dapps Decentralized Application: Dapps are programs that run on top of smart contracts and communicate with them via their exposed ABI in the context of a transaction.
When we compare blockchain to a database and smart-contracts to stored procedures then Dapps are the applications that run on top of the stored procedures.

Pros and Cons: Some pros and cons of using blockchain have already been listed in the beginning of this chapter “«Elimination» Intermediary”. For a “full” list of pros and cons see [4]. But two arguments that were not listed I want to discuss here:

Funny: Blockchain & Cryptocurrency explained see [13].

The remainder of this document (starting with chapter 3) gives an introduction on blockchain technology with focus on the ethereum implementation.

In red font are the different issues the blockchain exposes and risks you need to consider.

In yellow highlighted are keywords, definitions and aspects that require your attention.

1.1. Architecture Overview

The architecture overview is based on ethereum but conceptual identical to other blockchain implementations.

1.1.1. Node address format

An Ethereum node can be addressed using the URL scheme "enode" notation.

The hexadecimal node ID is encoded in the username portion of the URL, separated from the host by an @ sign. The hostname can only be given as an IP address, DNS domain names are not allowed. The port in the host name section is the TCP listening port. If the TCP and UDP (discovery) ports differ, the UDP port is specified as query parameter "discport".

In the following example, the node URL describes a node with IP address 10.3.58.6, TCP listening port 30303 and UDP discovery port 30301.

enode://6f8a80d14311c39f35f516fa664deaaaa13e85b2f7493f37f6144d86991ec012937307647bd3b9a82abe2974e1407241d54947bbb39763a4cac9f77166ad92a0@10.3.58.6:30303?discport=30301

The enode url scheme is used by the Node discovery protocol and can be used in the bootnodes command line option of the client.

More information regarding the node address format can be found in [26].

1.1.2. Communication Protocol

Ethereum nodes use a peer-to-peer communication - listening (TCP) port and a discovery (UDP) port, both on 30303 by default.

Additional information regarding the different protocols can be found in [27].

1.1.2.1. Light Ethereum Subprotocol (LES)

Light Clients & Light Ethereum Subprotocol (LES): Work in progress: LES is a sub-protocol of the ÐΞVp2p Wire Protocol. In Ethereum, a light client is a client that downloads only block headers by default, and verifies only a small portion of what needs to be verified. Light clients do not interact directly with the blockchain; they instead use full nodes as intermediaries. Light clients rely on full nodes for many operations, from requesting the latest headers to asking for the balance of an account. As light clients need to send several requests to do simple operations, the overall network bandwidth needed is short term higher than that of a full node BUT only when the client is in use, compared to a full-node that is always online and participates entirely in all blockchain communications. Furthermore the amount of resources and storage needed is several orders of magnitude lower than that of a full node while achieving a very high level of security. Requiring only about 100 MB of storage and low computational power, a light node can run on a mobile device! But consider:

Light clients do not expose peer-to-peer communication and discovery (UDP) ports i.e. there is no incoming communication on these ports and protocols (because a light client is only up when used and relies on full-nodes i.e. cannot act and reply on its own)! But light clients rely on full nodes and LES as sub-protocol of the ÐΞVp2p Wire Protocol; therefore communication outgoing is on the same ports:  peer-to-peer communication (TCP) port and a discovery (UDP) port, both on 30303.

1.1.2.2. JSON RPC

JSON RPC: Dapps connect to a node and communicate with smart contracts via the JSON RPC protocol. JSON RPC in the context of ethereum can run on top of different transport layer protocols namely: HTTPS( port 8545), WebSocket (port 8546) and IPC (Inter Process Communication) running locally on the file system. But consider:

1.1.3. Information Flow

 
 

Full-nodes and mining-nodes are very similar and run on the same software e.g. parity or geth, with the main difference that mining-nodes produce blocks and require a lot computing power. For this reason on the full-node outgoing flow we put “[block]” in square brackets meaning that full-nodes, because they have all information, can provide block information to other nodes but full-nodes do not produce blocks and spread them into the network.

Light nodes only retrieve and store all block-headers. “[block, receipt, proof]” information is only retrieved on demand when needed and therefore put into square brackets.

ping”, “pong” and “neighbor” information is exchanged as part of the node discovery. But light clients do not expose a discovery port i.e. cannot receive a “ping” and therefore do not send a “pong”.

hello” and “status” is exchanged when two nodes connect to each other, exchange their status and start synchronizing the chain.

 

1.1.5. Interfaces

 
 

3. Definitions

Based on the introduction and overview given in chapter 1, this chapter further details and defines terms used in the blockchain context.

3.1. Fork

Besides temporary forks that happen when different miners create blocks at the same time or due to temporary network segregation see chapter 1, a hard- respective soft-fork is caused by a non-forward- respective forward-compatible change of the blockchain protocol rule-set in the underlying node software and/or configuration.

 
 

Soft fork: A soft fork is a forward compatible change and tightens the rule-set of the blockchain protocol. All transactions (and blocks) created using the new software will be accepted by all nodes regardless whether they are running the old or new software. Transactions (and blocks) that are created using the old software will be accepted by nodes running the old software but will only be accepted by nodes running the new software if they don’t violate the new tightened protocol rule-set.

Hard Fork: Bilateral versus Strictly Expanding: A hard fork is a non-forward compatible change. Hard forks require “immediately” that all nodes upgrade their software because “old” nodes will not accept (certain) “new” transactions and blocks and therefore no soft migration is possible compared to a soft-fork but instead a “big-bang” upgrade is required.

Whether a fork takes place or not is finally decided by the miners whether the majority of them update to the new software version or not. Because in the end only miners can create blocks and add them to the chain.

The DAO: A decentralized autonomous organization and a form of investor directed venture capital fund, written in Solidity and deployed as smart-contract on 30.04.2016 attracted in 20 days crowdsale more than 150 Mio US$ Ether. BUT the software had a bug and a hacker transferred a third of the entire fund to his account!
A soft-fork was planned to restrict the protocol rule-set and reject all blocks containing a transaction to or from the hacker account. BUT this software change could have resulted in DoS attacks where hackers could flood the network with computation intensive (and therefore ether-expensive) transactions and as last transaction in a block they could have created a transaction to or from the DAO hacker account which would then reject the entire block and giving the hackers back their money they “invested” before in the computation intensive transactions. Therefore in the end a hard-fork was performed which splitted the chain into two separate networks.

There exist dozens of articles about soft- and hard-forks see [10].

3.2. Wallet

See chapter 1 A wallet stores the public- and private-keys of a participant, is connected to a full-node and hence meets the conditions to create and participate in transactions. Loosing a wallet means loosing all coins because the private-key to create and sign transactions and thus spending coins belonging to this account (public-key) is lost. And therefore a wallet respective its public and private keys should always be backed up!

When choosing a wallet, the owner must keep in mind who is supposed to have access to (a copy of) the private keys and thus has potentially access to the coins/asset. Just like with a bank, the user needs to trust the provider to keep the coins safe. Downloading a wallet from a wallet provider to a computer or phone does not automatically mean that the owner is the only one who has a copy of the private keys. For example with Coinbase, it is possible to install a wallet on a phone and to also have access to the same wallet through their website.

A backup of a wallet can come in different forms like:

When the private keys and the backup are lost then that coins are lost forever. When using a webwallet, the private keys are managed by the provider. When owning coins, those trusted with managing the private keys should be carefully selected. An (encrypted) copy of the wallet should be kept in a trusted place. Preferably off-line. Some people 'write' their mnemonic sentence or private key on metal, because it is robust.

Alternatively a “paranoid” user can write the code to create a public-/private-key pair and the code to sign the transaction with the private-key himself see chapter 3.7.3 and 3.7.4.

Further interesting articles see [14].

3.2.1. HD (Hierarchical Deterministic) wallet & mnemonics

Most content in this chapter is extracted from the article “Deterministic Wallet” see [15].

A deterministic wallet is a system respective algorithm of deriving keys from a single starting point known as a seed. The seed allows a user to easily back up and restore a wallet without needing any other information and can allow the creation of public addresses without the knowledge of the private key. Seeds are typically serialized into human-readable words in a Mnemonic phrase. In other words instead of memorizing all the public- / private-key pairs contained in the wallet the user only needs to memorize the mnemonic sentence because from this mnemonic all public- / private-key pairs can be derived (calculated).

Deterministic wallets can generate an unlimited number of addresses on the fly and as the addresses are generated in a known fashion rather than randomly some clients can be used on multiple devices without the risk of losing funds. Users can conveniently create a single backup of the seed in a human readable format that will last the life of the wallet, without the worry of this backup becoming stale.

Certain types of deterministic wallet (BIP0032, Armory, Coinkite and Coinb.in ) additionally allow for the complete separation of private and public key creation for greater security and convenience. In this model a server can be set up to only know the Master Public Key of a particular deterministic wallet. This allows the server to create as many public keys as is necessary for receiving funds, but a compromise of the MPK will not allow an attacker to spend from the wallet.

Mnemonic (dt “Eselsbrücke”): See [16]: Pronounced “ne-manik,” in its purest form a mnemonic is a pattern of letters, words, or associations which allows the user to easily remember information, and has been used by humans for thousands of years. In other words, it can be a very useful tool to help us memorize important information we need to remember.

Further interesting articles see [14].

5. Ethereum

Ethereum is an open-source, blockchain-based distributed computing platform and operating system featuring smart contract (scripting) functionality.

Ether is the cryptocurrency of the ethereum blockchain. Ether can be transferred between accounts and used to compensate participant mining nodes for computations performed. Ethereum provides a decentralized Turing-complete virtual machine, the Ethereum Virtual Machine (EVM), which can execute smart contracts (scripts) on a network of nodes. "Gas", an internal transaction fee/pricing mechanism, is used to mitigate spam and allocate resources on the network.
Ethereum has a metric system of denominations used as units of ether. The smallest denomination aka base unit of ether is called Wei. Below is a list of the named denominations and their value in Wei.

Further detailed information about ethereum can be found in the ethereum white paper see [5] and yellow paper see [6] which are the basis for ethereum.

5.1. Networks

Each network version gets a separate name (and id). Here is an overview – see as well [12].

 

The current protocol version is Homestead and Ropsten is is the public Homestead equivalent testnet.

Despite the differences in name, Olympic, Morden and Ropsten have the network ids 0, 2 and 3. Frontier, Homestead are the main network with id 1. You can run your own chain by specifying a network id other than 0, 1, 2, or 3.

5.2. The Ethereum Virtual Machine

Most content in this chapter is extracted from the article “Introduction to Smart Contracts” see [17].

5.2.1. Overview

The Ethereum Virtual Machine or EVM is the runtime environment for smart contracts in Ethereum. It is not only sandboxed but actually completely isolated, which means that code running inside the EVM has no access to network, filesystem or other processes. Smart contracts even have limited access to other smart contracts.

5.2.2. Accounts

There are two kinds of accounts in Ethereum which share the same address space: External accounts that are controlled by public-private key pairs (i.e. humans) and contract accounts which are controlled by the code stored together with the account.

The address of an external account is determined from the public key while the address of a contract is determined at the time the contract is created respective deployed (it is derived from the creator address and the number of transactions sent from that address, the so-called “nonce”).

Regardless of whether or not the account stores code, the two types are treated equally by the EVM.

Every account has a persistent key-value store mapping 256-bit words to 256-bit words called storage.

Furthermore, every account has a balance in Ether (in “Wei” to be exact) which can be modified by sending transactions that include Ether.

5.2.3. Transactions

A transaction is a message that is sent from one account to another account (which might be the same or the special zero-account, see below). It can include binary data (its payload) and Ether.

If the target account contains code, that code is executed and the payload is provided as input data.

If the target account is the zero-account (the account with the address 0), the transaction creates a new contract. As already mentioned, the address of that contract is not the zero address but an address derived from the sender and its number of transactions sent (the “nonce”). The payload of such a contract creation transaction is taken to be EVM bytecode and executed. The output of this execution is permanently stored as the code of the contract. This means that in order to create a contract, you do not send the actual code of the contract, but in fact code that returns that code.

5.2.4. Gas

Upon creation, each transaction is charged with a certain amount of gas, whose purpose is to limit the amount of work that is needed to execute the transaction and to pay for this execution. While the EVM executes the transaction, the gas is gradually depleted according to specific rules.

The gas price is a value set by the creator of the transaction, who has to pay gas_price * gas up front from the sending account. If some gas is left after the execution, it is refunded in the same way.

If the gas is used up at any point (i.e. it is negative), an out-of-gas exception is triggered, which reverts all modifications made to the state in the current call frame.

With Bitcoin miners prioritise transaction with the highest mining fees. The same is true of Ethereum where miners are free to ignore transactions whose gas price limit is too low.

The gas price per transaction or contract is set up to deal with the Turing Complete nature of Ethereum and its EVM (Ethereum Virtual Machine Code) – the idea being to limit infinite loops. So for example 10 Szabo, or 0.00001 Ether or 1 Gas can execute a line of code or some command. If there is not enough Ether in the account to perform the transaction or message then it is considered invalid. The idea is to stop denial of service attacks from infinite loops, encourage efficiency in the code – and to make an attacker pay for the resources they use, from bandwidth through to CPU calculations through to storage.

5.2.5. Storage, Memory and the Stack

Each account has a persistent memory area which is called storage. Storage is a key-value store that maps 256-bit words to 256-bit words. It is not possible to enumerate storage from within a contract and it is comparatively costly to read and even more so, to modify storage. A contract can neither read nor write to any storage apart from its own.

The second memory area is called memory, of which a contract obtains a freshly cleared instance for each message call. Memory is linear and can be addressed at byte level, but reads are limited to a width of 256 bits, while writes can be either 8 bits or 256 bits wide. Memory is expanded by a word (256-bit), when accessing (either reading or writing) a previously untouched memory word (ie. any offset within a word). At the time of expansion, the cost in gas must be paid. Memory is more costly the larger it grows (it scales quadratically).

The EVM is not a register machine but a stack machine, so all computations are performed on an area called the stack. It has a maximum size of 1024 elements and contains words of 256 bits. Access to the stack is limited to the top end in the following way: It is possible to copy one of the topmost 16 elements to the top of the stack or swap the topmost element with one of the 16 elements below it. All other operations take the topmost two (or one, or more, depending on the operation) elements from the stack and push the result onto the stack. Of course it is possible to move stack elements to storage or memory, but it is not possible to just access arbitrary elements deeper in the stack without first removing the top of the stack.

5.2.6. Instruction Set

The instruction set of the EVM is kept minimal in order to avoid incorrect implementations which could cause consensus problems. All instructions operate on the basic data type, 256-bit words. The usual arithmetic, bit, logical and comparison operations are present. Conditional and unconditional jumps are possible. Furthermore, contracts can access relevant properties of the current block like its number and timestamp.

5.2.7. Message Calls

Contracts can call other contracts or send Ether to non-contract accounts by the means of message calls. Message calls are similar to transactions, in that they have a source, a target, data payload, Ether, gas and return data. In fact, every transaction consists of a top-level message call which in turn can create further message calls.

A contract can decide how much of its remaining gas should be sent with the inner message call and how much it wants to retain. If an out-of-gas exception happens in the inner call (or any other exception), this will be signalled by an error value put onto the stack. In this case, only the gas sent together with the call is used up. In Solidity, the calling contract causes a manual exception by default in such situations, so that exceptions “bubble up” the call stack.

As already said, the called contract (which can be the same as the caller) will receive a freshly cleared instance of memory and has access to the call payload - which will be provided in a separate area called the calldata. After it has finished execution, it can return data which will be stored at a location in the caller’s memory preallocated by the caller.

Calls are limited to a depth of 1024, which means that for more complex operations, loops should be preferred over recursive calls.

5.2.8. Delegatecall / Callcode and Libraries

There exists a special variant of a message call, named delegatecall which is identical to a message call apart from the fact that the code at the target address is executed in the context of the calling contract and msg.sender and msg.value do not change their values.

This means that a contract can dynamically load code from a different address at runtime. Storage, current address and balance still refer to the calling contract, only the code is taken from the called address.

This makes it possible to implement the “library” feature in Solidity: Reusable library code that can be applied to a contract’s storage, e.g. in order to implement a complex data structure.

5.2.9. Logs

It is possible to store data in a specially indexed data structure that maps all the way up to the block level. This feature called logs is used by Solidity in order to implement events. Contracts cannot access log data after it has been created, but they can be efficiently accessed from outside the blockchain. Since some part of the log data is stored in bloom filters, it is possible to search for this data in an efficient and cryptographically secure way, so network peers that do not download the whole blockchain (“light clients”) can still find these logs.

5.2.10. Create

Contracts can even create other contracts using a special opcode (i.e. they do not simply call the zero address). The only difference between these create calls and normal message calls is that the payload data is executed and the result stored as code and the caller / creator receives the address of the new contract on the stack.

5.2.11. Self-destruct

The only possibility that code is removed from the blockchain is when a contract at that address performs the selfdestruct operation. The remaining Ether stored at that address is sent to a designated target and then the storage and code is removed from the state.

5.4. Parity (Node)

A node is a computer, that is part of the ethereum network. This node either stores a part (light client) or a full (full node) copy of the blockchain and updates the blockchain continuously. Furthermore there exist mining-nodes that confirm transactions aka perform mining.

See [7]: What does the client software do?

It downloads the whole blockchain onto your system on a regular basis, keeping the tab on the whole network. It verifies all transactions and contracts on the blockchain. If you are building your own contracts, it broadcasts them to the network so that they are included in the next block and confirmed by the miners. Client software can also do the mining but these days you may need a super-computer do make any ether this way.

Both geth and parity require 2-4GB of RAM and 50-100GB of hard drive space for storing the blockchain. And both require extensive memory so ~4GB swap needs to be set up.

There exist different ethereum clients (listed here descending, according to popularity): go-ethereum (geth), Parity, cpp-ethereum, pyethapp, ethereumjs-lib, Ethereum(J), ruby-ethereum, ethereumH. But there exists no resilient comparison between them. Some people suggest parity because it is popular, flexible and seems to be a little bit faster than the others. What I’ve seen so far is that the test network support is different (especially regarding PoA) – see chapter 3.1.

In this document we will focus on the parity client/node. Most content in this chapter is extracted from the parity wiki see [18].

Summary: Parity is an Ethereum client, written from the ground-up for correctness-verifiability, modularisation, low-footprint and high-performance. To this end it utilizes the Rust language, a hybrid imperative/OO/functional language with an emphasis on efficiency.

5.4.1. Installation

You can download and run the binaries, you can build the binaries from source, you can run parity in a docker container or you can run parity in the cloud e.g. Microsoft Azure.

5.4.1.1. Docker

Precondition is that you’ve docker installed – see chapter “Installation” and “Network” in [3].

Lately the parity docker image support has been improved greatly and different base images are available out of the box see [20].

But in case you want to build it on docker yourself follow the instruction in this chapter. All details respective output and issues encountered can be found in appendix  C. Listed here is the final resulting dockerfile:

# Get an image as basis for building the app

FROM ubuntu AS builder

 

# Set the working directory to /app

WORKDIR /app

 

# 1. Download and install all packages required to build parity

RUN apt-get update -y && apt-get install -y build-essential && apt-get install -y libudev-dev && apt-get install -y git && apt-get install -y curl && curl https://sh.rustup.rs -sSf > rustup.sh && chmod +x rustup.sh && ./rustup.sh -y && git clone https://github.com/paritytech/parity

 

# (2. Optionally download and install the test packages required to run the parity tests)

#RUN cd parity && git submodule init && git submodule update

 

# 3. Finally run the compilation and build the parity node

# There is an issue running “source $HOME/.cargo/env”, replacing with “. $HOME/.cargo/env”

RUN cd parity && . $HOME/.cargo/env && cargo build

 

# Run app.py when the container launches

#CMD ["python", "app.py"]

 

docker build -t becke-ch--parity--s0-v1:s0-0-v1-0 .

docker run --net docker--s0-0-v1-0 --ip 10.0.0.10 -it becke-ch--parity--s0-v1

5.4.2. Setup & Run

Full Node: Running a full node with the stadard configuration for the Ethereum Mainnet requires a lot of computer ressources. The blockchain download and validation process is particularly heavy on CPU and disk IO. It is therefore recommended to run a full node on a computer with multi-core CPU, 4GB RAM and a SSD drive with at least 60GB free space. Internet connection can also be a limiting factor. A decent DSL connection is required.

Light Node: Running a light node using the flag --light does not require to download and perform validation of the whole blockchain. A light node relies on full node peers to receive block headers and verify transactions. It is therefore far less resource demanding than a full node. For additional information see as well chapter 1.1.2.1.

--warp: State snapshotting, or warp-sync, allows for an extremely fast “synchronization” that skips almost all of the block processing, simply injecting the appropriate data directly into the database. Parity 1.6 or 1.7, --warp does nothing as it is enabled by default.

parity --chain ropsten : Connect to the Ropsten testnet.

parity --chain kovan : Connect to the Kovan testnet.

parity --bootnodes enode://YOUR_BOOT_NODE_ID_HERE@127.0.0.1:30303 : You can override the normal boot nodes with --bootnodes, i.e., you might run a local bootnode and sync from that. Further information on bootnodes see chapter 1.1.2 and regarding configuration see chapter 3.3.4, 3.3.4.1 and 3.3.4.2.

parity --reserved-peers /path/to/reserved.txt : Instead of using a bootnode to discover and connect to peers maintain a permanent connection to own set of nodes, you can wire them with the --reserved-peers feature. Simply place all node addresses you want to connect to (enode://..., one per line) into a text file, e.g., reserved.txt.
Using a list of reserved peers hinders the dynamics of of a network to grow (and shrink) because maintaining and distributing such a file between the nodes is painful and against a decentralized approach. And there exist better ways and technologies to restrict nodes respective IP ranges.

parity --chain /path/to/chain-config.json --config /path/to/node-config.toml --base-path /path/to/base-data-storage/ --jsonrpc-interface [IP] --jsonrpc-apis [APIS]:  Start parity on a private/consortium chain defined in the “--chain” configuration, with a customized node configuration according to “--config”, with a dedicated base data storage location “--base-path” and accessible via JSON-RPC over HTTP on a (fix) IP address “--jsonrpc-interface [IP]”. For example see PoA chain in chapter 3.3.6.1.

$ mkdir -p /data/becke-ch--parity-chain--s0-v1

$ touch /data/becke-ch--parity-chain--s0-v1/chain-config.json

$ mkdir -p /data/becke-ch--parity-node--s0-v1/data

$ mkdir -p /data/becke-ch--parity-node--s0-v1/configuration

$ touch /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml

“Minimal”: Minimal according to my point of view in matters of clean separation and reasonable functionality with minimal security exposure:

$ parity --chain /data/becke-ch--parity-chain--s0-v1/chain-config.json --config /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml --base-path /data/becke-ch--parity-node--s0-v1/data/ --jsonrpc-interface 10.0.0.10

Extended”:

$ parity --chain /data/becke-ch--parity-chain--s0-v1/chain-config.json --config /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml --base-path /data/becke-ch--parity-node--s0-v1/data/ --jsonrpc-interface 10.0.0.10 --jsonrpc-apis web3,eth,pubsub,net,shh,shh_pubsub

 

5.4.2.1. Docker

Follow the installation in chapter 3.3.1.1 and setup see chapter above.

  1. 1.Because docker in some environments (e.g. Azure Container Instance (ACI)) has issues mounting more than one directory, link the chain configuration file into the node configuration directory (Attention symbolic links are not working in docker – use hard link instead!): 

$ ln /data/becke-ch--parity-chain--s0-v1/chain-config.json /data/becke-ch--parity-node--s0-v1/configuration/

  1. 2.Run docker, mount the node (and chain) directory and pass the chain-configuration-, node-configuration- and base-data-storage-path as startup parameters: “--chain”, “--config” and “--base-path
    Do NOT map the parity ports using “-p” otherwise you will get conflicts with several docker nodes running on your host! Instead see chapter “Network” in [3] run the container on an existing bridge (and if visibility is required outside of the host, expose this bridge via a physical NIC externally). 

  2. 3.Run the docker containers on fix IP addresses to make sure that they will find each other even after a restart. It is mandatory at least for the boot-nodes to run on fix IP addresses otherwise the other nodes in the network cannot startup (in case the boot-node is hard-coded / configured)! 

“Minimal”:

$ docker run -ti --net docker--s0-0-v1-0 --ip 10.0.0.10 -v /data/becke-ch--parity-node--s0-v1/:/data/becke-ch--parity-node--s0-v1/ parity/parity:v1.11.8 --chain /data/becke-ch--parity-node--s0-v1/configuration/chain-config.json --config /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml --base-path /data/becke-ch--parity-node--s0-v1/data/ --jsonrpc-interface 10.0.0.10

Extended”:

$ docker run -ti --net docker--s0-0-v1-0 --ip 10.0.0.10 -v /data/becke-ch--parity-node--s0-v1/:/data/becke-ch--parity-node--s0-v1/ parity/parity:v1.11.8 --chain /data/becke-ch--parity-node--s0-v1/configuration/chain-config.json --config /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml --base-path /data/becke-ch--parity-node--s0-v1/data/ --jsonrpc-interface 10.0.0.10 --jsonrpc-apis web3,eth,pubsub,net,shh,shh_pubsub

 

5.4.3. JSON RPC

Based on chapter 1.1.2.2 parity exposes the following JSON RPC APIs by default – see below and next to them I’ve listed the web3js (see chapter 3.7.7) packages they map to:

And the following JSON RPC APIs are NOT exposed by default:

Finally I suggest to enable the following minimal set of APIs (in the configuration file – see chapter below):

apis = ["web3", "eth", "pubsub", "net", "shh", "shh_pubsub"]

The following (proprietary and/or not documented) APIs I’ve removed from the minimal default: parity, parity_pubsub, traces and rpc!

 

5.4.4. Node Configuration

As shown in chapter 3.3.2 parity can be started without any specific node configuration. In this case the default configuration is used which matches the configuration of the other nodes in the public blockchain.

Configuration can either be provided as command line arguments or in a configuration file. Should the command line arguments and the config file disagree about a setting, the CLI takes precedence. Therefore the config file should contain the default, minimal and most restrictive configuration settings relevant for all participating nodes and (temporary) exception and dynamic behaviors should be provided via command line!

5.4.4.1. config file

The config file: “--config /data/becke-ch--parity-node--sX-vY/configuration/node-config.toml” should contain the following information:

apis = [...]: By default only the minimal set of APIs as listed in the previous chapter should be exposed.
Attention due to a security issue in the parity WebSocket implementation, the disabling of WebSocket APIs is not working as expected – see chapter
1.1.2.2. In other words you can configure it but it will get ignored under certain circumstances and therefore the WebSocket interface should not be exposed at allsee next chapter below!:

[rpc]

apis = ["web3", "eth", "pubsub", "net", "shh", "shh_pubsub"]

[websockets]

apis = ["web3", "eth", "pubsub", "net", "shh", "shh_pubsub"]

bootnodes = [“enode://YOUR_BOOT_NODE_ID_HERE@IP_ADDRESS:30303”]: The configuration file should contain the initial boot node(s) of the chain founder(s). For (temporary) exception see next chapter. For additional remarks regarding boot nodes see chapter 1.1.2.
Using a list of reserved peers instead hinders the dynamics of of a network to grow (and shrink) because maintaining and distributing such a file between the nodes is painful and against a decentralized approach. And there exist better ways and technologies to restrict nodes respective IP ranges.

[network]

bootnodes = ["enode://2806efe7adb55d001d9ffabefab6a1711540672ae84308133940c366dc08aaf3080839e14da17e3d694156f54c4cf33397fe00ac6aa377454433dc81aeb906f3@10.0.0.10:30303"]

 

The resulting configuration looks as follows:

[network]

bootnodes = ["enode://2806efe7adb55d001d9ffabefab6a1711540672ae84308133940c366dc08aaf3080839e14da17e3d694156f54c4cf33397fe00ac6aa377454433dc81aeb906f3@10.0.0.10:30303"]

[rpc]

apis = ["web3", "eth", "pubsub", "net", "shh", "shh_pubsub"]

[websockets]

apis = ["web3", "eth", "pubsub", "net", "shh", "shh_pubsub"]

 

5.4.4.2. cli parameters

--bootnodes enode://YOUR_BOOT_NODE_ID_HERE@IP_ADDRESS:30303: In case a bootnode changes (dynamic IP address or downtime) this information should be provided as startup parameter overriding the default configuration in the config file see previous chapter. For additional remarks regarding boot nodes see chapter 1.1.2.

--bootnodes enode://2806efe7adb55d001d9ffabefab6a1711540672ae84308133940c366dc08aaf3080839e14da17e3d694156f54c4cf33397fe00ac6aa377454433dc81aeb906f3@10.0.0.10:30303

 

--jsonrpc-interface=[IP]:  To access the node via HTTP from externally (respective from the host where the docker container node is running) the IP address where the interface is listening respective assigned to should be allowed. Because the IP address of the node respective interface might change in case of dynamic IPs this information should be provided as CLI parameter (in the sample below replace “10.0.0.10” with the IP address where your node is running respective the interface is listening!).
--ws-interface=[IP]: Because there exists a security issue with the parity WebSocket implementation and the disabling of exposed APIs, see previous chapter above and chapter 1.1.2.2, the IP address where the interface is listening respective assigned to should NOT be allowed! Instead only the “--jsonrpc-interface=[IP]” should be allowed!
For HTTP and WebSocket the default interface is “
local”.

--jsonrpc-interface 10.0.0.10

 

--jsonrpc-apis [APIS]:  In exceptional cases when additional administration APIs (for a list see chapter 3.3.3) are required they can temporarily (for a short time) be activated via command line and they will then oversteer the configuration in the config file see previous chapter.
--ws-apis [APIS]: As already mentioned above the disabling of WebSocket APIs is not working as expected and therefore should not be used (currently)!

--jsonrpc-apis web3,eth,pubsub,net,shh,shh_pubsub

 

5.4.5. Networks & Chains (Configuration)

By default, when simply running parity, Parity Ethereum will connect to the official public Ethereum network.

In order to run a chain different to the official public Ethereum one, Parity has to run with the --chain option e.g. “parity --chain ropsten:

For more information on networks see chapter 3.1.

5.4.5.1. Private chains

Parity supports private chains and private network configurations via Chain specification file provided with --chain. In addition to the usual Proof of Work Chains, Parity also includes Proof of Authority Chains which do not require mining:

JSON chain spec format:

{

 "name": "becke-ch--parity-chain--sX-vY",

 "engine": {

  "ENGINE_NAME": {

   "params": {

    ENGINE_PARAMETERS

   }

  }

 },

 "genesis": {

  "seal": {

   ENGINE_SPECIFIC_GENESIS_SEAL

  },

  "difficulty": "0x20000",

  "gasLimit": "0x2fefd8"

 },

 "params": {

   "networkID" : "0xYOUR_NETWORK_ID",

   "maximumExtraDataSize": "0x20",

   "minGasLimit": "0x1388"

 },

 "accounts": {

  GENESIS_ACCOUNTS

 }

}

"accounts": {

    "0x0000000000000000000000000000000000000001": { "balance": "1", "builtin": { "name":   "ecrecover", "pricing": { "linear": { "base": 3000, "word": 0 } } } },

    "0x0000000000000000000000000000000000000002": { "balance": "1", "builtin": { "name": "sha256", "pricing": { "linear": { "base": 60, "word": 12 } } } },

    "0x0000000000000000000000000000000000000003": { "balance": "1", "builtin": { "name": "ripemd160", "pricing": { "linear": { "base": 600, "word": 120 } } } },

    "0x0000000000000000000000000000000000000004": { "balance": "1", "builtin": { "name": "identity", "pricing": { "linear": { "base": 15, "word": 3 } } } }

}

Other types of accounts that can be specified:

5.4.6. Proof of Authority (Consensus Algorithm Configuration)

Proof of Authority does not depend on nodes solving arbitrarily difficult mathematical problems, but instead uses a set of “authorities” - nodes that are explicitly allowed to create new blocks and secure the blockchain. The block has to be signed off by the majority of authorities, in which case it becomes a part of the permanent record. This makes it easier to maintain a private chain and keep the block issuers accountable.

For consortium setting there are no disadvantages of PoA network as compared to PoW. It is more secure (since an attacker with unwanted connection or hacked authority can not overwhelm a network potentially reverting all transactions), less computationally intensive (mining with difficulty which provides security requires lots of computation), more performant (Aura consensus provides lower transaction acceptance latency) and more predictable (blocks are issued at steady time intervals). PoA deployments are used by the enterprise and by the public (e.g. popular Kovan test network).

The two available algorithms are the Aura and Tendermint. Because Tendermint is still experimental we will focus on Aura (authorityRound) in the following sub chapters.

5.4.6.1. Setting up a Proof of Authority network

  1. 1.Pick real world entities that should control the network, so called authorities. E.g different organizations that want to collaborate without an intermediary. Optionally establish the authority as a new organization and give it a DNS name e.g. “becke.ch” (respective “becke-ch” depending on the context). 

  2. 2.Nominate a promoter respective administrator of the authority round / chain, responsible for specifying and distributing the configurations and running an (initial) boot node the others can connect to. 

  3. 3.Each authority should create two accounts – an authority account and a user account. The authority account is used by the node itself to issue blocks. To issue blocks the node requires the password (file) for this authority account AND because this is a security issue (from my point of view) and as well for separation of concerns, we use a separate user account that has the ethers required to send transactions. The accounts can be created by using a wallet or simply running the code shown in chapter 3.7.3 or reusing an existing account (public- / private-key pair). 

  4. 4.Each authority sends its authority account (public key) and user account (public key) to the authority round administrator.  

  5. 5.The administrator creates the configuration files: 

    1. a.Prepares the network configuration as shown below and stores it in a file: 

$ vi /data/becke-ch--parity-chain--s0-v1/chain-config.json

 

{

 "name": "becke-ch--parity-chain--s0-v1",

 "engine": {

    "authorityRound": {

        "params": {

            "stepDuration": "5",

            "validators" : {

                "list": ["0xDfBad97d7AD160804762c36C255082a705D985E8","0x1C8592BFB4f2A842fC2eFBEB54463638739c2B93","0x169b1feCEb85044b36d6B7A3f2076Af6b372843A"]

            }

        }

    }

},

 "genesis": {

  "seal": {

   "authorityRound": {

    "step": "0x0",

    "signature": "0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"

            }

        },

  "difficulty": "0x20000",

  "gasLimit": "0x2fefd8"

 },

 "params": {

   "networkID" : "0x777",

   "maximumExtraDataSize": "0x20",

   "minGasLimit": "0x1388",

   "gasLimitBoundDivisor": "0x400",

   "eip155Transition": 0,

   "validateChainIdTransition": 0,

   "eip140Transition": 0,

   "eip211Transition": 0,

   "eip214Transition": 0,

   "eip658Transition": 0

 },

 "accounts": {

        "0x0000000000000000000000000000000000000001": { "balance": "1", "builtin": { "name": "ecrecover", "pricing": { "linear": { "base": 3000, "word": 0 } } } },

        "0x0000000000000000000000000000000000000002": { "balance": "1", "builtin": { "name": "sha256", "pricing": { "linear": { "base": 60, "word": 12 } } } },

        "0x0000000000000000000000000000000000000003": { "balance": "1", "builtin": { "name": "ripemd160", "pricing": { "linear": { "base": 600, "word": 120 } } } },

        "0x0000000000000000000000000000000000000004": { "balance": "1", "builtin": { "name": "identity", "pricing": { "linear": { "base": 15, "word": 3 } } } },

        "0x26D9E695Bd95fD6cd9e2C08Fc3450FCD6958fC1E": { "balance": "10000000000000000000000" },

        "0xDdf06643ec9970568424c690c27462e090B9bd66": { "balance": "10000000000000000000000" },

        "0x9929ff47aDFb2D3D2Ab5D65E156d9a9EA46B5435": { "balance": "10000000000000000000000" }

 }

}

            "validators" : {

                "multi": {

                        "0": { "list": ["0xc6d9d2cd449a754c494264e1809c50e34d64562b"] },

                        "10": { "safeContract": ["0xd6d9d2cd449a754c494264e1809c50e34d64562b"] },

                        "20": { "contract": "0xc6d9d2cd449a754c494264e1809c50e34d64562b" }

                }

            }

 

    1. b.The administrator prepares the node configuration as shown in chapter 3.3.4, 3.3.4.1 and store it in a file. The bootnode line is commented because (of course) we don’t know its value until the first time we’ve started the node: 

$ vi /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml

 

[network]

#bootnodes = ["enode://2806efe7adb55d001d9ffabefab6a1711540672ae84308133940c366dc08aaf3080839e14da17e3d694156f54c4cf33397fe00ac6aa377454433dc81aeb906f3@172.17.0.2:30303"]

[rpc]

apis = ["web3", "eth", "pubsub", "net", "shh", "shh_pubsub"]

[websockets]

apis = ["web3", "eth", "pubsub", "net", "shh", "shh_pubsub"]

 

    1. c.Run the node as described in chapter 3.3.2 respective 3.3.2.1 

“Minimal”:

raoul-becke--s0-v1@hp-elitebook-850-g5--s0-v1:~$ docker run -ti --net docker--s0-0-v1-0 --ip 10.0.0.10 -v /data/becke-ch--parity-node--s0-v1/:/data/becke-ch--parity-node--s0-v1/ parity/parity:v1.11.8 --chain /data/becke-ch--parity-node--s0-v1/configuration/chain-config.json --config /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml --base-path /data/becke-ch—parity-node--s0-v1/data/ --jsonrpc-interface 10.0.0.10

Loading config file from /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml

2018-09-13 06:09:00 UTC Starting Parity/v1.11.8-stable-c754a02-20180725/x86_64-linux-gnu/rustc1.27.2

2018-09-13 06:09:00 UTC Keys path /data/becke-ch--parity-node--s0-v1/data//keys/becke-ch--parity-chain--s0-v1

2018-09-13 06:09:00 UTC DB path /data/becke-ch--parity-node--s0-v1/data//chains/becke-ch--parity-chain--s0-v1/db/675318460daeba05

2018-09-13 06:09:00 UTC Path to dapps /data/becke-ch--parity-node--s0-v1/data//dapps

2018-09-13 06:09:00 UTC State DB configuration: fast

2018-09-13 06:09:00 UTC Operating mode: active

2018-09-13 06:09:00 UTC Configured for becke-ch--parity-chain--s0-v1 using AuthorityRound engine

2018-09-13 06:09:05 UTC Public node URL: enode://3b9d3ff377a6f49e812c1ecfb89f52ddd300b1ff4610a74a7289ab4da110751b7bf41f1a2e85b4e8fb19d852ce4ea6b96e221910f58830e2ce0b3b61d65f3423@10.0.0.10:30303

2018-09-13 06:09:30 UTC    0/25 peers   8 KiB chain 9 KiB db 0 bytes queue 448 bytes sync  RPC:  0 conn,  0 req/s,   0 µs

...

    1. d.Stop the node by pressing “Ctrl+C” and adapt the node configuration with the bootnode information we got from the startup: 

$ vi /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml

 

[network]

bootnodes = ["enode://3b9d3ff377a6f49e812c1ecfb89f52ddd300b1ff4610a74a7289ab4da110751b7bf41f1a2e85b4e8fb19d852ce4ea6b96e221910f58830e2ce0b3b61d65f3423@10.0.0.10:30303"]

[rpc]

apis = ["web3", "eth", "pubsub", "net", "shh", "shh_pubsub"]

[websockets]

apis = ["web3", "eth", "pubsub", "net", "shh", "shh_pubsub"]

 

    1. e.The authority account that signs consensus messages and issues blocks needs to be added to the node that is performing these actions:  

      1. i.The authority account must be listed in the validators list  (see above) (and belongs to the authority maintaining this node).  

      2. ii.parity_newAccountFromSecret: Assuming that the authority used the method listed in chapter 3.7.3 to create the account, they already have the private key for this account and therefore use the method “parity_newAccountFromSecret” to add this account to the node store.
        To perform this action we need to allow parity_accountsAPI BUT be careful when exposing this API because it is potentially unsafe, therefore stop parity right after the account has been added: 

docker run -ti --net docker--s0-0-v1-0 --ip 10.0.0.10 -v /data/becke-ch--parity-node--s0-v1/:/data/becke-ch--parity-node--s0-v1/ parity/parity:v1.11.8 --chain /data/becke-ch--parity-node--s0-v1/configuration/chain-config.json --config /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml --base-path /data/becke-ch--parity-node--s0-v1/data/ --jsonrpc-interface 10.0.0.10 --jsonrpc-apis web3,eth,pubsub,net,shh,shh_pubsub,parity_accounts

The private key and password need to be replaced with the one of the authority (the private key listed here belongs to the authority round administrator 0xDfBad97d7AD160804762c36C255082a705D985E8):

curl --data '{"method":"parity_newAccountFromSecret","params":["0x3930fe64c8381001a351068688542ae71cb5c9badeccd74744cab889da9b2771","hunter2"],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST 10.0.0.10:8545

 

{"jsonrpc":"2.0","result":"0xdfbad97d7ad160804762c36c255082a705d985e8","id":1}

Optional: To check whether the account was added successfully run “parity_allAccountsInfo”:

curl --data '{"method":"parity_allAccountsInfo","params":[],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST 10.0.0.10:8545

 

{"jsonrpc":"2.0","result":{"0xdfbad97d7ad160804762c36c255082a705d985e8":{"meta":"{}","name":"","uuid":"6d8449ba-90c3-8870-107f-d0c2c3cff33d"}},"id":1}

 

      1. iii.--engine-signer [ADDRESS]: Specify the authority address which should be used to sign consensus messages and issue blocks (relevant only to non-PoW chains). This parameter is provided as command line argument because it is (of course) different for each authority node.5
        --password [FILE]: Provide a file containing a password for unlocking an account (leading and trailing whitespace is trimmed). This password is required to unlock the account that was provided in the parameter “--engine-signer [ADDRESS]” see above.6 

$ vi /data/becke-ch--parity-node--s0-v1/configuration/engine-signer.pwd

hunter2

 

docker run -ti --net docker--s0-0-v1-0 --ip 10.0.0.10 -v /data/becke-ch--parity-node--s0-v1/:/data/becke-ch--parity-node--s0-v1/ parity/parity:v1.11.8 --chain /data/becke-ch--parity-node--s0-v1/configuration/chain-config.json --config /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml --base-path /data/becke-ch--parity-node--s0-v1/data/ --jsonrpc-interface 10.0.0.10 --engine-signer 0xDfBad97d7AD160804762c36C255082a705D985E8 --password /data/becke-ch--parity-node--s0-v1/configuration/engine-signer.pwd

    1. f.Optional: For example in case we need to interact with the account information: “personal” and/or “parity_accounts”; docker needs to be started as follows BUT attention be careful when exposing these APIs because they are potentially unsafe – alternative safer approaches to create accounts and sign transactions see 3.7.3 and 3.7.4: 

docker run -ti --net docker--s0-0-v1-0 --ip 10.0.0.10 -v /data/becke-ch--parity-node--s0-v1/:/data/becke-ch--parity-node--s0-v1/ parity/parity:v1.11.8 --chain /data/becke-ch--parity-node--s0-v1/configuration/chain-config.json --config /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml --base-path /data/becke-ch--parity-node--s0-v1/data/ --jsonrpc-interface 10.0.0.10 --jsonrpc-apis web3,eth,pubsub,net,shh,shh_pubsub,personal,parity_accounts

BUT instead of using the UI see chapter 3.3.7, to interact with the accounts, which is based on the WebSocket transport layer protocol, which has security issues regarding disabling of APIs; I rather suggest to directly use HTTPS and communicate with the node using “curl --data ‘{KEY-VALUE-PAIRS}’ -H “Content-Type: application/json" -X POST 10.0.0.10:8545”:

curl --data '{"method":"personal_listAccounts","params":[],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST 10.0.0.10:8545

curl --data '{"method":"parity_allAccountsInfo","params":[],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST 10.0.0.10:8545

curl --data '{"jsonrpc":"2.0","method":"parity_newAccountFromPhrase","params":["node0", "node0"],"id":0}' -H "Content-Type: application/json" -X POST 10.0.0.10:8545

...

 

  1. 6.The administrator distributes the config files: “/data/becke-ch--parity-chain--s0-v1/chain-config.jsonand “/data/becke-ch—parity-node--s0-v1/configuration/node-config.toml” to the different authorities. 

  2. 7.Each authority stores the config files and starts a parity node (applying some of the steps already listed above) as follows: 

docker run -ti --net docker--s0-0-v1-0 --ip 10.0.0.10 -v /data/becke-ch--parity-node--s1-v1/:/data/becke-ch--parity-node--s1-v1/ parity/parity:v1.11.8 --chain /data/becke-ch--parity-node--s1-v1/configuration/chain-config.json --config /data/becke-ch--parity-node--s1-v1/configuration/node-config.toml --base-path /data/becke-ch--parity-node--s1-v1/data/ --jsonrpc-interface 10.0.0.11 --jsonrpc-apis web3,eth,pubsub,net,shh,shh_pubsub,parity_accounts

The private key and password need to be replaced with the one of the authority (the private key listed here belongs to the authority account “0x1C8592BFB4f2A842fC2eFBEB54463638739c2B93):

curl --data '{"method":"parity_newAccountFromSecret","params":["0x19fd5781e9ba1c21b1a56b5a2f867778a15978fcda2cfc31fc744e71e273348d","hunter3"],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST 10.0.0.11:8545

 

{"jsonrpc":"2.0","result":"0xdfbad97d7ad160804762c36c255082a705d985e8","id":1}

 

$ vi /data/becke-ch--parity-node--s1-v1/configuration/engine-signer.pwd

hunter3

 

docker run -ti --net docker--s0-0-v1-0 --ip 10.0.0.10 -v /data/becke-ch--parity-node--s1-v1/:/data/becke-ch--parity-node--s1-v1/ parity/parity:v1.11.8 --chain /data/becke-ch--parity-node--s1-v1/configuration/chain-config.json --config /data/becke-ch--parity-node--s1-v1/configuration/node-config.toml --base-path /data/becke-ch--parity-node--s1-v1/data/ --jsonrpc-interface 10.0.0.11 --engine-signer 0x1C8592BFB4f2A842fC2eFBEB54463638739c2B93 --password /data/becke-ch--parity-node--s1-v1/configuration/engine-signer.pwd

  1. 8.Additional user nodes can connect to the network as well. 

  2. 9.Use the network just like a public Ethereum network (transactions, contracts etc.). 

 

Hacker note: When a hacker tries to modify: “chain-config.json”, starts his own node and joins the network then the following will happen:

 

5.4.6.2.  validators

A number of Engines available in Parity achieve consensus by referring to a list of “validators” (referred to as authorities if they are linked to physical entities). Validators are a group of accounts which are allowed to participate in the consensus, they validate the transactions and blocks to later sign messages about them.

Since parity 1.7 the list can also be a part of the blockchain state by being stored in an Ethereum contract.

It is best to include the contract in the genesis, placing the bytecode as a “constructor” in the “accounts” field like so:

{

  "name": "becke-ch--parity-chain--s0-v1",

  "engine": {

    "authorityRound": {

      "params": {

        "stepDuration": "5",

        "validators": {

          "safeContract": "0x0000000000000000000000000000000000000010"

        }

      }

    }

  },

 "accounts": {

        "0x0000000000000000000000000000000000000001": { "balance": "1", "builtin": { "name": "ecrecover", "pricing": { "linear": { "base": 3000, "word": 0 } } } },

        "0x0000000000000000000000000000000000000010": { "balance": "1", "constructor": "0x6080...50029"},

        "0x26D9E695Bd95fD6cd9e2C08Fc3450FCD6958fC1E": { "balance": "10000000000000000000000" },

 }

}

And (of course) in addition this contract needs to be referenced from the validators section as safeContract – see chapter 3.3.6.2.1 or as contract – see chapter 3.3.6.2.2.

If the constructor takes arguments they must be encoded and appended to the contract bytecode (using e.g. ethabi). Also if the contract initializes any address with msg.sender (for example as a contract owner) you must take into account that when defining the contract in genesis, msg.sender will be set to the system address (SYSTEM_ADDRESS: 2^160 – 2).

Additional information see [35].

5.4.6.2.1. Non-reporting contract: safeContract

Compared to the reporting contract – see chapter 3.3.6.2.2 - this contract does not offer functionality to report bad behavior of validators and to act on such behavior. Basically it just returns an array of validators: getValidators

The function getValidators should always return the active set or the initial set if the contract hasn’t been activated yet. Switching the set should be done by issuing a InitiateChange event with the parent block hash and new set, storing the pending set, and then waiting for call to finalizeChange (by the SYSTEM_ADDRESS: 2^160 - 2) before setting the active set to the pending set.

A very simple contract that just replaces the static validators list shown in chapter 3.3.6.1 could look as follows:

pragma solidity ^0.4.22;

 

contract ValidatorSet {

 

    // Current list of addresses entitled to participate in the consensus.

    address[] validators;

 

    /// Issue this log event to signal a desired change in validator set.

    /// This will not lead to a change in active validator set until

    /// finalizeChange is called.

    ///

    /// Only the last log event of any block can take effect.

    /// If a signal is issued while another is being finalized it may never

    /// take effect.

    ///

    /// _parentHash here should be the parent block hash, or the

    /// signal will not be recognized.

    event InitiateChange(bytes32 indexed _parentHash, address[] _newSet);

 

    constructor() public {

        validators.push(0xDfBad97d7AD160804762c36C255082a705D985E8);

        validators.push(0x1C8592BFB4f2A842fC2eFBEB54463638739c2B93);

        validators.push(0x169b1feCEb85044b36d6B7A3f2076Af6b372843A);

    }

 

    /// Called when an initiated change reaches finality and is activated.

    /// Only valid when msg.sender == SYSTEM (EIP96, 2**160 - 2).

    ///

    /// Also called when the contract is first enabled for consensus. In this case,

    /// the "change" finalized is the activation of the initial set.

    function finalizeChange() external{

    }

 

    /// Get current validator set (last enacted or initial if no changes ever made).

    function getValidators() external view returns (address[]){

        return validators;

    }

}

This contract can be compiled as follows – additional information see chapter 3.6.2

solc ValidatorSetImpl.sol --combined-json abi,asm,ast,bin,bin-runtime,clone-bin,devdoc,interface,opcodes,srcmap,srcmap-runtime,userdoc > ValidatorSetImpl.json

In the resulting compiled JSON file: ValidatorSetImpl.json copy the bin value:

{

  "contracts": {

    "ValidatorSetImpl.sol:ValidatorSet": {

      "bin": "608060405...d8a16cf5f350029",

    }

  },

}

Into the chain configuration file, into the accounts section AND reference this contract address from the validators section as follows:

{

  "name": "becke-ch--parity-chain--s0-v1",

  "engine": {

    "authorityRound": {

      "params": {

        "stepDuration": "5",

        "validators": {

          "safeContract": "0x0000000000000000000000000000000000000010"

        }

      }

    }

  },

 "accounts": {

        "0x0000000000000000000000000000000000000001": { "balance": "1", "builtin": { "name": "ecrecover", "pricing": { "linear": { "base": 3000, "word": 0 } } } },

        "0x0000000000000000000000000000000000000010": { "balance": "1", "constructor": "0x6080...50029"},

        "0x26D9E695Bd95fD6cd9e2C08Fc3450FCD6958fC1E": { "balance": "10000000000000000000000" },

 }

}

And last but not least when starting parity these “Initial contract validators” should be listed in the output:

raoul-becke--s0-v1@hp-elitebook-850-g5--s0-v1:~$ docker run -ti --net docker--s0-0-v1-0 --ip 10.0.0.10 -v /data/becke-ch--parity-node--s0-v1/:/data/becke-ch--parity-node--s0-v1/ parity/parity:v1.11.8 --chain /data/becke-ch--parity-node--s0-v1/configuration/chain-config.json --config /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml --base-path /data/becke-ch--parity-node--s0-v1/data/

Loading config file from /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml

2018-09-11 05:02:45 UTC Starting Parity/v1.11.8-stable-c754a02-20180725/x86_64-linux-gnu/rustc1.27.2

2018-09-11 05:02:45 UTC Keys path /data/becke-ch--parity-node--s0-v1/data//keys/becke-ch--parity-chain--s0-v1

2018-09-11 05:02:45 UTC DB path /data/becke-ch--parity-node--s0-v1/data//chains/becke-ch--parity-chain--s0-v1/db/675318460daeba05

2018-09-11 05:02:45 UTC Path to dapps /data/becke-ch--parity-node--s0-v1/data//dapps

2018-09-11 05:02:45 UTC State DB configuration: fast

2018-09-11 05:02:45 UTC Operating mode: active

2018-09-11 05:02:45 UTC Configured for becke-ch--parity-chain--s0-v1 using AuthorityRound engine

2018-09-11 05:02:45 UTC Signal for switch to contract-based validator set.

2018-09-11 05:02:45 UTC Initial contract validators: [0xdfbad97d7ad160804762c36c255082a705d985e8, 0x1c8592bfb4f2a842fc2efbeb54463638739c2b93, 0x169b1feceb85044b36d6b7a3f2076af6b372843a]

2018-09-11 05:02:50 UTC Public node URL: enode://4326994a843cd64591998ac2c80b437215a18d793f925fe95972470728943610599ca3082c7103295923a2bc482d3003bdd627fcb9edfb761028a92b27f5fd7d@10.0.0.10:30303

2018-09-11 05:03:15 UTC    0/25 peers   8 KiB chain 9 KiB db 0 bytes queue 448 bytes sync  RPC:  0 conn,  0 req/s,   0 µs

...

 

5.4.6.2.2. Reporting Contract: contract

Sometimes one might want to automatically take action when one of the validators behaves badly. The definition of bad behaviour depends on a consensus engine and there are two types of bad behaviour:

Benign misbehaviour in Aura may be simply not receiving a block from a designated primary, while malicious misbehaviour would be releasing two different blocks for the same step.

This type of contract can listen to misbehaviour reports from the consensus engine and decide what are the consequences for the validators.

The contract structure that needs to be implemented is superset to the safeContract with the following additional methods see differences highlighted below:

pragma solidity ^0.4.22;

 

contract ValidatorSet {

    event InitiateChange(bytes32 indexed _parentHash, address[] _newSet);

 

    function finalizeChange() external{

 ...

    }

 

    /// Reports benign misbehavior of validator of the current validator set

    /// (e.g. validator offline).

    function reportBenign(address validator, uint256 blockNumber) external{

 ...

    }

 

    /// Reports malicious misbehavior of validator of the current validator set

    /// and provides proof of that misbehavor, which varies by engine

    /// (e.g. double vote).

    function reportMalicious(address validator, uint256 blockNumber, bytes proof) external{

 ...

    }

 

    function getValidators() external view returns (address[]){

        ...

    }

 

}

 

 

5.4.7. Parity UI

Starting from Parity v1.10, Parity Ethereum client has been separated from the Parity User Interface (UI). The user interface previously accessible from the browser in versions <=1.9 is now released as a standalone app.

As already mentioned in chapter 1.1.2.2 and 3.3.4 the WebSocket communication is not secure respective the exposed APIs cannot be disabled as expected and therefore WebSocket should not be used! (Actually the ParityUI can access all APIs via the WebSocket using a signer token but the security issue here is that the first ParityUI does not need to request a signer token but instead gets it “for free”!) Most operations can be performed using the HTTP(S) protocol see [36]. In case you really want to quickly temporarily startup the WebSocket interface perform the following command:

docker run -ti --net docker--s0-0-v1-0 --ip 10.0.0.10 -v /data/becke-ch--parity-node--s0-v1/:/data/becke-ch--parity-node--s0-v1/ parity/parity:v1.11.8 --chain /data/becke-ch--parity-node--s0-v1/configuration/chain-config.json --config /data/becke-ch--parity-node--s0-v1/configuration/node-config.toml --base-path /data/becke-ch--parity-node--s0-v1/data/ --jsonrpc-interface 10.0.0.10 --ws-interface 10.0.0.10

  1. 1.Download latest version of “parity-ui-X.Y.Z.tar.xz” from https://github.com/Parity-JS/shell/releases into directory “/tool/” and extract file. 

  2. 2.Launch “/tool/parity-ui-0.3.4/parity-ui --ws-interface=NODE_IP --ws-port=8546”  replace “NODE_IP” with the IP address of your node e.g. 10.0.0.10 

Optional:

  1. 3.To delete the configurations performed by the client remove the following directories: 

$ rm -rf .config/parity-ui

$ rm -rf .local/share/io.parity.ethereum

 

5.4.8. Infura

Instead of running a parity node locally and connecting your dapp to this node via JSON RPC and this node to the rest of the network; you can alternatively connect to a remote infura full-node, which is already connected to the network, and connect your dapp to this remote node instead. But in order to do this some steps are necessary:

  1. 1.Sign-up: https://infura.io/signup: You first need to sign-up to Infura with: first-, last-name and email address and accept the terms of service: 

 

Illustration 6: Infura: Sign-Up

 
  1. 2.You will then get a list of nodes respective url-end-points in the different networks/chains you can connect to: 

Main Ethereum Network

https://mainnet.infura.io/...

Test Ethereum Network (Ropsten)

https://ropsten.infura.io/...

 

Test Ethereum Network (Rinkeby)

https://rinkeby.infura.io/...

 

Test Ethereum Network (Kovan)

https://kovan.infura.io/...

 

IPFS Gateway

https://ipfs.infura.io

 

IPFS RPC

https://ipfs.infura.io:5001

 

5.6. MetaMask (Wallet)

Most content in this chapter is extracted from different MetaMask articles see [23].

MetaMask allows you to run Ethereum dApps right in your browser without running a full Ethereum node. MetaMask includes a secure identity vault, providing a user interface to manage your identities on different sites and sign blockchain transactions.

MetaMask is more than just an Ether wallet. It's an Ethereum Browser, like Mist! It allows you all the same functions, features and ease of access from regular Ethereum Wallets, and it allows you to interact with Dapps and Smart Contracts, and all without the need to download the blockchain or install any software, you can just install it as a Google Chrome Extension!

Under the hood, MetaMask stores your public & private keys securely (per network), connects to the infura full-nodes see chapter 3.3.8: “https://mainnet.infura.io/metamask”, “https://ropsten.infura.io/metamask”, “https://kovan.infura.io/metamask”, “https://rinkeby.infura.io/metamask” or to local nodes using the JSON RPC protocol and injects a web3 provider which intercepts the web3 via JSON RPC communication and facilitates the interaction with the MetaMask public/private key-store.

Basically what MetaMask plugin does is to inject a web3 provider; see chapter 3.7.2; and intercept web3 api calls to simplify and facilitate communication with nodes (see chapter 3.3) and contracts in the Ethereum network (see chapter 3.1)

5.6.1. Installation and configuration

  1. 1.Go to: https://metamask.io/ 

  2. 2.Click on “Get Chrome extension” 

  3. 3.Click on “Add to Chrome” and click “Add Extension”