r1

test/functional/README.md

@@ -0,0 +1,177 @@
+# Functional tests
+
+### Writing Functional Tests
+
+#### Example test
+
+The [example_test.py](example_test.py) is a heavily commented example of a test case that uses both
+the RPC and P2P interfaces. If you are writing your first test, copy that file
+and modify to fit your needs.
+
+#### Coverage
+
+Running `test_runner.py` with the `--coverage` argument tracks which RPCs are
+called by the tests and prints a report of uncovered RPCs in the summary. This
+can be used (along with the `--extended` argument) to find out which RPCs we
+don't have test cases for.
+
+#### Style guidelines
+
+- Where possible, try to adhere to [PEP-8 guidelines](https://www.python.org/dev/peps/pep-0008/)
+- Use a python linter like flake8 before submitting PRs to catch common style
+  nits (eg trailing whitespace, unused imports, etc)
+- The oldest supported Python version is specified in [doc/dependencies.md](/doc/dependencies.md).
+  Consider using [pyenv](https://github.com/pyenv/pyenv), which checks [.python-version](/.python-version),
+  to prevent accidentally introducing modern syntax from an unsupported Python version.
+  The Travis linter also checks this, but [possibly not in all cases](https://github.com/bitcoin/bitcoin/pull/14884#discussion_r239585126).
+- See [the python lint script](/test/lint/lint-python.sh) that checks for violations that
+  could lead to bugs and issues in the test code.
+- Avoid wildcard imports
+- Use a module-level docstring to describe what the test is testing, and how it
+  is testing it.
+- When subclassing the BitcoinTestFramwork, place overrides for the
+  `set_test_params()`, `add_options()` and `setup_xxxx()` methods at the top of
+  the subclass, then locally-defined helper methods, then the `run_test()` method.
+- Use `'{}'.format(x)` for string formatting, not `'%s' % x`.
+
+#### Naming guidelines
+
+- Name the test `<area>_test.py`, where area can be one of the following:
+    - `feature` for tests for full features that aren't wallet/mining/mempool, eg `feature_rbf.py`
+    - `interface` for tests for other interfaces (REST, ZMQ, etc), eg `interface_rest.py`
+    - `mempool` for tests for mempool behaviour, eg `mempool_reorg.py`
+    - `mining` for tests for mining features, eg `mining_prioritisetransaction.py`
+    - `p2p` for tests that explicitly test the p2p interface, eg `p2p_disconnect_ban.py`
+    - `rpc` for tests for individual RPC methods or features, eg `rpc_listtransactions.py`
+    - `tool` for tests for tools, eg `tool_wallet.py`
+    - `wallet` for tests for wallet features, eg `wallet_keypool.py`
+- use an underscore to separate words
+    - exception: for tests for specific RPCs or command line options which don't include underscores, name the test after the exact RPC or argument name, eg `rpc_decodescript.py`, not `rpc_decode_script.py`
+- Don't use the redundant word `test` in the name, eg `interface_zmq.py`, not `interface_zmq_test.py`
+
+#### General test-writing advice
+
+- Set `self.num_nodes` to the minimum number of nodes necessary for the test.
+  Having additional unrequired nodes adds to the execution time of the test as
+  well as memory/CPU/disk requirements (which is important when running tests in
+  parallel or on Travis).
+- Avoid stop-starting the nodes multiple times during the test if possible. A
+  stop-start takes several seconds, so doing it several times blows up the
+  runtime of the test.
+- Set the `self.setup_clean_chain` variable in `set_test_params()` to control whether
+  or not to use the cached data directories. The cached data directories
+  contain a 200-block pre-mined blockchain and wallets for four nodes. Each node
+  has 25 mature blocks (25x50=1250 BTC) in its wallet.
+- When calling RPCs with lots of arguments, consider using named keyword
+  arguments instead of positional arguments to make the intent of the call
+  clear to readers.
+- Many of the core test framework classes such as `CBlock` and `CTransaction`
+  don't allow new attributes to be added to their objects at runtime like
+  typical Python objects allow. This helps prevent unpredictable side effects
+  from typographical errors or usage of the objects outside of their intended
+  purpose.
+
+#### RPC and P2P definitions
+
+Test writers may find it helpful to refer to the definitions for the RPC and
+P2P messages. These can be found in the following source files:
+
+- `/src/rpc/*` for RPCs
+- `/src/wallet/rpc*` for wallet RPCs
+- `ProcessMessage()` in `/src/net_processing.cpp` for parsing P2P messages
+
+#### Using the P2P interface
+
+- `messages.py` contains all the definitions for objects that pass
+over the network (`CBlock`, `CTransaction`, etc, along with the network-level
+wrappers for them, `msg_block`, `msg_tx`, etc).
+
+- P2P tests have two threads. One thread handles all network communication
+with the agrariand(s) being tested in a callback-based event loop; the other
+implements the test logic.
+
+- `P2PConnection` is the class used to connect to a agrariand.  `P2PInterface`
+contains the higher level logic for processing P2P payloads and connecting to
+the Bitcoin Core node application logic. For custom behaviour, subclass the
+P2PInterface object and override the callback methods.
+
+- Can be used to write tests where specific P2P protocol behavior is tested.
+Examples tests are `p2p_unrequested_blocks.py`, `p2p_compactblocks.py`.
+
+### test-framework modules
+
+#### [test_framework/authproxy.py](test_framework/authproxy.py)
+Taken from the [python-bitcoinrpc repository](https://github.com/jgarzik/python-bitcoinrpc).
+
+#### [test_framework/test_framework.py](test_framework/test_framework.py)
+Base class for functional tests.
+
+#### [test_framework/util.py](test_framework/util.py)
+Generally useful functions.
+
+#### [test_framework/mininode.py](test_framework/mininode.py)
+Basic code to support P2P connectivity to a agrariand.
+
+#### [test_framework/comptool.py](test_framework/comptool.py)
+Framework for comparison-tool style, p2p tests.
+
+#### [test_framework/script.py](test_framework/script.py)
+Utilities for manipulating transaction scripts (originally from python-bitcoinlib)
+
+#### [test_framework/blockstore.py](test_framework/blockstore.py)
+Implements disk-backed block and tx storage.
+
+#### [test_framework/key.py](test_framework/key.py)
+Wrapper around OpenSSL EC_Key (originally from python-bitcoinlib)
+
+#### [test_framework/bignum.py](test_framework/bignum.py)
+Helpers for script.py
+
+#### [test_framework/blocktools.py](test_framework/blocktools.py)
+Helper functions for creating blocks and transactions.
+
+### Comptool
+
+* Testing framework for writing tests that compare the block/tx acceptance
+behavior of a agrariand against 1 or more other agrariand instances, or against
+known outcomes, or both.
+
+* Set the ```num_nodes``` variable (defined in ```ComparisonTestFramework```) to start up
+1 or more nodes.  If using 1 node, then ```--testbinary``` can be used as a command line
+option to change the agrariand binary used by the test.  If using 2 or more nodes,
+then ```--refbinary``` can be optionally used to change the agrariand that will be used
+on nodes 2 and up.
+
+* Implement a (generator) function called ```get_tests()``` which yields ```TestInstance```s.
+Each ```TestInstance``` consists of:
+  - a list of ```[object, outcome, hash]``` entries
+    * ```object``` is a ```CBlock```, ```CTransaction```, or
+    ```CBlockHeader```.  ```CBlock```'s and ```CTransaction```'s are tested for
+    acceptance.  ```CBlockHeader```s can be used so that the test runner can deliver
+    complete headers-chains when requested from the agrariand, to allow writing
+    tests where blocks can be delivered out of order but still processed by
+    headers-first agrariand's.
+    * ```outcome``` is ```True```, ```False```, or ```None```.  If ```True```
+    or ```False```, the tip is compared with the expected tip -- either the
+    block passed in, or the hash specified as the optional 3rd entry.  If
+    ```None``` is specified, then the test will compare all the agrariand's
+    being tested to see if they all agree on what the best tip is.
+    * ```hash``` is the block hash of the tip to compare against. Optional to
+    specify; if left out then the hash of the block passed in will be used as
+    the expected tip.  This allows for specifying an expected tip while testing
+    the handling of either invalid blocks or blocks delivered out of order,
+    which complete a longer chain.
+  - ```sync_every_block```: ```True/False```.  If ```False```, then all blocks
+    are inv'ed together, and the test runner waits until the node receives the
+    last one, and tests only the last block for tip acceptance using the
+    outcome and specified tip.  If ```True```, then each block is tested in
+    sequence and synced (this is slower when processing many blocks).
+  - ```sync_every_transaction```: ```True/False```.  Analogous to
+    ```sync_every_block```, except if the outcome on the last tx is "None",
+    then the contents of the entire mempool are compared across all agrariand
+    connections.  If ```True``` or ```False```, then only the last tx's
+    acceptance is tested against the given outcome.
+
+* For examples of tests written in this framework, see
+  ```invalidblockrequest.py``` and ```p2p-fullblocktest.py```.
+

test/functional/combine_logs.py

@@ -0,0 +1,114 @@
+#!/usr/bin/env python3
+"""Combine logs from multiple bitcoin nodes as well as the test_framework log.
+
+This streams the combined log output to stdout. Use combine_logs.py > outputfile
+to write to an outputfile."""
+
+import argparse
+from collections import defaultdict, namedtuple
+import heapq
+import itertools
+import os
+import re
+import sys
+
+# Matches on the date format at the start of the log event
+TIMESTAMP_PATTERN = re.compile(r"^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d{6}")
+
+LogEvent = namedtuple('LogEvent', ['timestamp', 'source', 'event'])
+
+def main():
+    """Main function. Parses args, reads the log files and renders them as text or html."""
+
+    parser = argparse.ArgumentParser(usage='%(prog)s [options] <test temporary directory>', description=__doc__)
+    parser.add_argument('-c', '--color', dest='color', action='store_true', help='outputs the combined log with events colored by source (requires posix terminal colors. Use less -r for viewing)')
+    parser.add_argument('--html', dest='html', action='store_true', help='outputs the combined log as html. Requires jinja2. pip install jinja2')
+    args, unknown_args = parser.parse_known_args()
+
+    if args.color and os.name != 'posix':
+        print("Color output requires posix terminal colors.")
+        sys.exit(1)
+
+    if args.html and args.color:
+        print("Only one out of --color or --html should be specified")
+        sys.exit(1)
+
+    # There should only be one unknown argument - the path of the temporary test directory
+    if len(unknown_args) != 1:
+        print("Unexpected arguments" + str(unknown_args))
+        sys.exit(1)
+
+    log_events = read_logs(unknown_args[0])
+
+    print_logs(log_events, color=args.color, html=args.html)
+
+def read_logs(tmp_dir):
+    """Reads log files.
+
+    Delegates to generator function get_log_events() to provide individual log events
+    for each of the input log files."""
+
+    files = [("test", "%s/test_framework.log" % tmp_dir)]
+    for i in itertools.count():
+        logfile = "{}/node{}/regtest/debug.log".format(tmp_dir, i)
+        if not os.path.isfile(logfile):
+            break
+        files.append(("node%d" % i, logfile))
+
+    return heapq.merge(*[get_log_events(source, f) for source, f in files])
+
+def get_log_events(source, logfile):
+    """Generator function that returns individual log events.
+
+    Log events may be split over multiple lines. We use the timestamp
+    regex match as the marker for a new log event."""
+    try:
+        with open(logfile, 'r') as infile:
+            event = ''
+            timestamp = ''
+            for line in infile:
+                # skip blank lines
+                if line == '\n':
+                    continue
+                # if this line has a timestamp, it's the start of a new log event.
+                time_match = TIMESTAMP_PATTERN.match(line)
+                if time_match:
+                    if event:
+                        yield LogEvent(timestamp=timestamp, source=source, event=event.rstrip())
+                    event = line
+                    timestamp = time_match.group()
+                # if it doesn't have a timestamp, it's a continuation line of the previous log.
+                else:
+                    event += "\n" + line
+            # Flush the final event
+            yield LogEvent(timestamp=timestamp, source=source, event=event.rstrip())
+    except FileNotFoundError:
+        print("File %s could not be opened. Continuing without it." % logfile, file=sys.stderr)
+
+def print_logs(log_events, color=False, html=False):
+    """Renders the iterator of log events into text or html."""
+    if not html:
+        colors = defaultdict(lambda: '')
+        if color:
+            colors["test"] = "\033[0;36m"   # CYAN
+            colors["node0"] = "\033[0;34m"  # BLUE
+            colors["node1"] = "\033[0;32m"  # GREEN
+            colors["node2"] = "\033[0;31m"  # RED
+            colors["node3"] = "\033[0;33m"  # YELLOW
+            colors["reset"] = "\033[0m"     # Reset font color
+
+        for event in log_events:
+            print("{0} {1: <5} {2} {3}".format(colors[event.source.rstrip()], event.source, event.event, colors["reset"]))
+
+    else:
+        try:
+            import jinja2
+        except ImportError:
+            print("jinja2 not found. Try `pip install jinja2`")
+            sys.exit(1)
+        print(jinja2.Environment(loader=jinja2.FileSystemLoader('./'))
+                    .get_template('combined_log_template.html')
+                    .render(title="Combined Logs from testcase", log_events=[event._asdict() for event in log_events]))
+
+if __name__ == '__main__':
+    main()

test/functional/combined_log_template.html

@@ -0,0 +1,40 @@
+<html lang="en">
+<head>
+    <title> {{ title }} </title>
+    <style>
+        ul {
+            list-style-type: none;
+            font-family: monospace;
+        }
+        li {
+            border: 1px solid slategray;
+            margin-bottom: 1px;
+        }
+        li:hover {
+            filter: brightness(85%);
+        }
+        li.log-test {
+            background-color: cyan;
+        }
+        li.log-node0 {
+            background-color: lightblue;
+        }
+        li.log-node1 {
+            background-color: lightgreen;
+        }
+        li.log-node2 {
+            background-color: lightsalmon;
+        }
+        li.log-node3 {
+            background-color: lightyellow;
+        }
+    </style>
+</head>
+<body>
+<ul>
+{% for event in log_events %}
+<li class="log-{{ event.source }}"> {{ event.source }} {{ event.timestamp }} {{event.event}}</li>
+{% endfor %}
+</ul>
+</body>
+</html>

test/functional/create_cache.py

@@ -0,0 +1,28 @@
+#!/usr/bin/env python3
+# Copyright (c) 2016-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Create a blockchain cache.
+
+Creating a cache of the blockchain speeds up test execution when running
+multiple functional tests. This helper script is executed by test_runner when multiple
+tests are being run in parallel.
+"""
+
+from test_framework.test_framework import BitcoinTestFramework
+
+class CreateCache(BitcoinTestFramework):
+    # Test network and test nodes are not required:
+
+    def set_test_params(self):
+        self.num_nodes = 0
+        self.supports_cli = True
+
+    def setup_network(self):
+        pass
+
+    def run_test(self):
+        pass
+
+if __name__ == '__main__':
+    CreateCache().main()

test/functional/example_test.py

@@ -0,0 +1,222 @@
+#!/usr/bin/env python3
+# Copyright (c) 2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""An example functional test
+
+The module-level docstring should include a high-level description of
+what the test is doing. It's the first thing people see when they open
+the file and should give the reader information about *what* the test
+is testing and *how* it's being tested
+"""
+# Imports should be in PEP8 ordering (std library first, then third party
+# libraries then local imports).
+from collections import defaultdict
+
+# Avoid wildcard * imports if possible
+from test_framework.blocktools import (create_block, create_coinbase)
+from test_framework.mininode import (
+    CInv,
+    P2PInterface,
+    mininode_lock,
+    msg_block,
+    msg_getdata,
+    network_thread_join,
+    network_thread_start,
+)
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (
+    assert_equal,
+    connect_nodes,
+    wait_until,
+)
+
+# P2PInterface is a class containing callbacks to be executed when a P2P
+# message is received from the node-under-test. Subclass P2PInterface and
+# override the on_*() methods if you need custom behaviour.
+class BaseNode(P2PInterface):
+    def __init__(self):
+        """Initialize the P2PInterface
+
+        Used to inialize custom properties for the Node that aren't
+        included by default in the base class. Be aware that the P2PInterface
+        base class already stores a counter for each P2P message type and the
+        last received message of each type, which should be sufficient for the
+        needs of most tests.
+
+        Call super().__init__() first for standard initialization and then
+        initialize custom properties."""
+        super().__init__()
+        # Stores a dictionary of all blocks received
+        self.block_receive_map = defaultdict(int)
+
+    def on_block(self, message):
+        """Override the standard on_block callback
+
+        Store the hash of a received block in the dictionary."""
+        message.block.calc_sha256()
+        self.block_receive_map[message.block.sha256] += 1
+
+    def on_inv(self, message):
+        """Override the standard on_inv callback"""
+        pass
+
+def custom_function():
+    """Do some custom behaviour
+
+    If this function is more generally useful for other tests, consider
+    moving it to a module in test_framework."""
+    # self.log.info("running custom_function")  # Oops! Can't run self.log outside the BitcoinTestFramework
+    pass
+
+class ExampleTest(BitcoinTestFramework):
+    # Each functional test is a subclass of the BitcoinTestFramework class.
+
+    # Override the set_test_params(), add_options(), setup_chain(), setup_network()
+    # and setup_nodes() methods to customize the test setup as required.
+
+    def set_test_params(self):
+        """Override test parameters for your individual test.
+
+        This method must be overridden and num_nodes must be exlicitly set."""
+        self.setup_clean_chain = True
+        self.num_nodes = 3
+        # Use self.extra_args to change command-line arguments for the nodes
+        self.extra_args = [[], ["-logips"], []]
+
+        # self.log.info("I've finished set_test_params")  # Oops! Can't run self.log before run_test()
+
+    # Use add_options() to add specific command-line options for your test.
+    # In practice this is not used very much, since the tests are mostly written
+    # to be run in automated environments without command-line options.
+    # def add_options()
+    #     pass
+
+    # Use setup_chain() to customize the node data directories. In practice
+    # this is not used very much since the default behaviour is almost always
+    # fine
+    # def setup_chain():
+    #     pass
+
+    def setup_network(self):
+        """Setup the test network topology
+
+        Often you won't need to override this, since the standard network topology
+        (linear: node0 <-> node1 <-> node2 <-> ...) is fine for most tests.
+
+        If you do override this method, remember to start the nodes, assign
+        them to self.nodes, connect them and then sync."""
+
+        self.setup_nodes()
+
+        # In this test, we're not connecting node2 to node0 or node1. Calls to
+        # sync_all() should not include node2, since we're not expecting it to
+        # sync.
+        connect_nodes(self.nodes[0], 1)
+        self.sync_all([self.nodes[0:1]])
+
+    # Use setup_nodes() to customize the node start behaviour (for example if
+    # you don't want to start all nodes at the start of the test).
+    # def setup_nodes():
+    #     pass
+
+    def custom_method(self):
+        """Do some custom behaviour for this test
+
+        Define it in a method here because you're going to use it repeatedly.
+        If you think it's useful in general, consider moving it to the base
+        BitcoinTestFramework class so other tests can use it."""
+
+        self.log.info("Running custom_method")
+
+    def run_test(self):
+        """Main test logic"""
+
+        # Create P2P connections to two of the nodes
+        self.nodes[0].add_p2p_connection(BaseNode())
+
+        # Start up network handling in another thread. This needs to be called
+        # after the P2P connections have been created.
+        network_thread_start()
+        # wait_for_verack ensures that the P2P connection is fully up.
+        self.nodes[0].p2p.wait_for_verack()
+
+        # Generating a block on one of the nodes will get us out of IBD
+        blocks = [int(self.nodes[0].generate(1)[0], 16)]
+        self.sync_all([self.nodes[0:1]])
+
+        # Notice above how we called an RPC by calling a method with the same
+        # name on the node object. Notice also how we used a keyword argument
+        # to specify a named RPC argument. Neither of those are defined on the
+        # node object. Instead there's some __getattr__() magic going on under
+        # the covers to dispatch unrecognised attribute calls to the RPC
+        # interface.
+
+        # Logs are nice. Do plenty of them. They can be used in place of comments for
+        # breaking the test into sub-sections.
+        self.log.info("Starting test!")
+
+        self.log.info("Calling a custom function")
+        custom_function()
+
+        self.log.info("Calling a custom method")
+        self.custom_method()
+
+        self.log.info("Create some blocks")
+        self.tip = int(self.nodes[0].getbestblockhash(), 16)
+        self.block_time = self.nodes[0].getblock(self.nodes[0].getbestblockhash())['time'] + 1
+
+        height = 1
+
+        for i in range(10):
+            # Use the mininode and blocktools functionality to manually build a block
+            # Calling the generate() rpc is easier, but this allows us to exactly
+            # control the blocks and transactions.
+            block = create_block(self.tip, create_coinbase(height), self.block_time)
+            block.solve()
+            block_message = msg_block(block)
+            # Send message is used to send a P2P message to the node over our P2PInterface
+            self.nodes[0].p2p.send_message(block_message)
+            self.tip = block.sha256
+            blocks.append(self.tip)
+            self.block_time += 1
+            height += 1
+
+        self.log.info("Wait for node1 to reach current tip (height 11) using RPC")
+        self.nodes[1].waitforblockheight(11)
+
+        self.log.info("Connect node2 and node1")
+        connect_nodes(self.nodes[1], 2)
+
+        self.log.info("Add P2P connection to node2")
+        # We can't add additional P2P connections once the network thread has started. Disconnect the connection
+        # to node0, wait for the network thread to terminate, then connect to node2. This is specific to
+        # the current implementation of the network thread and may be improved in future.
+        self.nodes[0].disconnect_p2ps()
+        network_thread_join()
+
+        self.nodes[2].add_p2p_connection(BaseNode())
+        network_thread_start()
+        self.nodes[2].p2p.wait_for_verack()
+
+        self.log.info("Wait for node2 reach current tip. Test that it has propagated all the blocks to us")
+
+        getdata_request = msg_getdata()
+        for block in blocks:
+            getdata_request.inv.append(CInv(2, block))
+        self.nodes[2].p2p.send_message(getdata_request)
+
+        # wait_until() will loop until a predicate condition is met. Use it to test properties of the
+        # P2PInterface objects.
+        wait_until(lambda: sorted(blocks) == sorted(list(self.nodes[2].p2p.block_receive_map.keys())), timeout=5, lock=mininode_lock)
+
+        self.log.info("Check that each block was received only once")
+        # The network thread uses a global lock on data access to the P2PConnection objects when sending and receiving
+        # messages. The test thread should acquire the global lock before accessing any P2PConnection data to avoid locking
+        # and synchronization issues. Note wait_until() acquires this global lock when testing the predicate.
+        with mininode_lock:
+            for block in self.nodes[2].p2p.block_receive_map.values():
+                assert_equal(block, 1)
+
+if __name__ == '__main__':
+    ExampleTest().main()

test/functional/fake_stake/base_test.py

@@ -0,0 +1,435 @@
+#!/usr/bin/env python3
+# Copyright (c) 2019 The PIVX developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+# -*- coding: utf-8 -*-
+
+from io import BytesIO
+from struct import pack
+from random import randint, choice
+import time
+
+from test_framework.authproxy import JSONRPCException
+from test_framework.blocktools import create_coinbase, create_block
+from test_framework.key import CECKey
+from test_framework.messages import CTransaction, CTxOut, CTxIn, COIN, msg_block
+from test_framework.mininode import network_thread_start
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.script import CScript, OP_CHECKSIG
+from test_framework.util import hash256, bytes_to_hex_str, hex_str_to_bytes, connect_nodes_bi, p2p_port
+
+from .util import TestNode, create_transaction, utxo_to_stakingPrevOuts, dir_size
+''' -------------------------------------------------------------------------
+Agrarian_FakeStakeTest CLASS ----------------------------------------------------
+
+General Test Class to be extended by individual tests for each attack test
+'''
+class Agrarian_FakeStakeTest(BitcoinTestFramework):
+
+    def set_test_params(self):
+        ''' Setup test environment
+        :param:
+        :return:
+        '''
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+        self.extra_args = [['-staking=1', '-debug=net']]*self.num_nodes
+
+
+    def setup_network(self):
+        ''' Can't rely on syncing all the nodes when staking=1
+        :param:
+        :return:
+        '''
+        self.setup_nodes()
+        for i in range(self.num_nodes - 1):
+            for j in range(i+1, self.num_nodes):
+                connect_nodes_bi(self.nodes, i, j)
+
+    def init_test(self):
+        ''' Initializes test parameters
+        :param:
+        :return:
+        '''
+        title = "*** Starting %s ***" % self.__class__.__name__
+        underline = "-" * len(title)
+        self.log.info("\n\n%s\n%s\n%s\n", title, underline, self.description)
+        # Global Test parameters (override in run_test)
+        self.DEFAULT_FEE = 0.1
+        # Spam blocks to send in current test
+        self.NUM_BLOCKS = 30
+
+        # Setup the p2p connections and start up the network thread.
+        self.test_nodes = []
+        for i in range(self.num_nodes):
+            self.test_nodes.append(TestNode())
+            self.test_nodes[i].peer_connect('127.0.0.1', p2p_port(i))
+
+        network_thread_start()  # Start up network handling in another thread
+        self.node = self.nodes[0]
+
+        # Let the test nodes get in sync
+        for i in range(self.num_nodes):
+            self.test_nodes[i].wait_for_verack()
+
+
+    def run_test(self):
+        ''' Performs the attack of this test - run init_test first.
+        :param:
+        :return:
+        '''
+        self.description = ""
+        self.init_test()
+        return
+
+
+
+    def create_spam_block(self, hashPrevBlock, stakingPrevOuts, height, fStakeDoubleSpent=False, fZPoS=False, spendingPrevOuts={}):
+        ''' creates a block to spam the network with
+        :param   hashPrevBlock:      (hex string) hash of previous block
+                 stakingPrevOuts:    ({COutPoint --> (int, int, int, str)} dictionary)
+                                      map outpoints (to be used as staking inputs) to amount, block_time, nStakeModifier, hashStake
+                 height:             (int) block height
+                 fStakeDoubleSpent:  (bool) spend the coinstake input inside the block
+                 fZPoS:              (bool) stake the block with zerocoin
+                 spendingPrevOuts:   ({COutPoint --> (int, int, int, str)} dictionary)
+                                      map outpoints (to be used as tx inputs) to amount, block_time, nStakeModifier, hashStake
+        :return  block:              (CBlock) generated block
+        '''
+
+        # If not given inputs to create spam txes, use a copy of the staking inputs
+        if len(spendingPrevOuts) == 0:
+            spendingPrevOuts = dict(stakingPrevOuts)
+
+        # Get current time
+        current_time = int(time.time())
+        nTime = current_time & 0xfffffff0
+
+        # Create coinbase TX
+        # Even if PoS blocks have empty coinbase vout, the height is required for the vin script
+        coinbase = create_coinbase(height)
+        coinbase.vout[0].nValue = 0
+        coinbase.vout[0].scriptPubKey = b""
+        coinbase.nTime = nTime
+        coinbase.rehash()
+
+        # Create Block with coinbase
+        block = create_block(int(hashPrevBlock, 16), coinbase, nTime)
+
+        # Find valid kernel hash - Create a new private key used for block signing.
+        if not block.solve_stake(stakingPrevOuts):
+            raise Exception("Not able to solve for any prev_outpoint")
+
+        # Sign coinstake TX and add it to the block
+        signed_stake_tx = self.sign_stake_tx(block, stakingPrevOuts[block.prevoutStake][0], fZPoS)
+        block.vtx.append(signed_stake_tx)
+
+        # Remove coinstake input prevout unless we want to try double spending in the same block.
+        # Skip for zPoS as the spendingPrevouts are just regular UTXOs
+        if not fZPoS and not fStakeDoubleSpent:
+            del spendingPrevOuts[block.prevoutStake]
+
+        # remove a random prevout from the list
+        # (to randomize block creation if the same height is picked two times)
+        if len(spendingPrevOuts) > 0:
+            del spendingPrevOuts[choice(list(spendingPrevOuts))]
+
+        # Create spam for the block. Sign the spendingPrevouts
+        for outPoint in spendingPrevOuts:
+            value_out = int(spendingPrevOuts[outPoint][0] - self.DEFAULT_FEE * COIN)
+            tx = create_transaction(outPoint, b"", value_out, nTime, scriptPubKey=CScript([self.block_sig_key.get_pubkey(), OP_CHECKSIG]))
+            # sign txes
+            signed_tx_hex = self.node.signrawtransaction(bytes_to_hex_str(tx.serialize()))['hex']
+            signed_tx = CTransaction()
+            signed_tx.deserialize(BytesIO(hex_str_to_bytes(signed_tx_hex)))
+            block.vtx.append(signed_tx)
+
+        # Get correct MerkleRoot and rehash block
+        block.hashMerkleRoot = block.calc_merkle_root()
+        block.rehash()
+
+        # Sign block with coinstake key and return it
+        block.sign_block(self.block_sig_key)
+        return block
+
+
+    def spend_utxo(self, utxo, address_list):
+        ''' spend amount from previously unspent output to a provided address
+        :param      utxo:           (JSON) returned from listunspent used as input
+                    addresslist:    (string) destination address
+        :return:    txhash:         (string) tx hash if successful, empty string otherwise
+        '''
+        try:
+            inputs = [{"txid":utxo["txid"], "vout":utxo["vout"]}]
+            out_amount = (float(utxo["amount"]) - self.DEFAULT_FEE)/len(address_list)
+            outputs = {}
+            for address in address_list:
+                outputs[address] = out_amount
+            spendingTx = self.node.createrawtransaction(inputs, outputs)
+            spendingTx_signed = self.node.signrawtransaction(spendingTx)
+            if spendingTx_signed["complete"]:
+                txhash = self.node.sendrawtransaction(spendingTx_signed["hex"])
+                return txhash
+            else:
+                self.log.warning("Error: %s" % str(spendingTx_signed["errors"]))
+                return ""
+        except JSONRPCException as e:
+            self.log.error("JSONRPCException: %s" % str(e))
+            return ""
+
+
+    def spend_utxos(self, utxo_list, address_list = []):
+        ''' spend utxos to provided list of addresses or 10 new generate ones.
+        :param      utxo_list:      (JSON list) returned from listunspent used as input
+                    address_list:   (string list) [optional] recipient Agrarian addresses. if not set,
+                                    10 new addresses will be generated from the wallet for each tx.
+        :return:    txHashes        (string list) tx hashes
+        '''
+        txHashes = []
+
+        # If not given, get 10 new addresses from self.node wallet
+        if address_list == []:
+            for i in range(10):
+                address_list.append(self.node.getnewaddress())
+
+        for utxo in utxo_list:
+            try:
+                # spend current utxo to provided addresses
+                txHash = self.spend_utxo(utxo, address_list)
+                if txHash != "":
+                    txHashes.append(txHash)
+            except JSONRPCException as e:
+                self.log.error("JSONRPCException: %s" % str(e))
+                continue
+        return txHashes
+
+
+    def stake_amplification_step(self, utxo_list, address_list = []):
+        ''' spends a list of utxos providing the list of new outputs
+        :param      utxo_list:     (JSON list) returned from listunspent used as input
+                    address_list:  (string list) [optional] recipient Agrarian addresses.
+        :return:    new_utxos:     (JSON list) list of new (valid) inputs after the spends
+        '''
+        self.log.info("--> Stake Amplification step started with %d UTXOs", len(utxo_list))
+        txHashes = self.spend_utxos(utxo_list, address_list)
+        num_of_txes = len(txHashes)
+        new_utxos = []
+        if num_of_txes> 0:
+            self.log.info("Created %d transactions...Mining 2 blocks to include them..." % num_of_txes)
+            self.node.generate(2)
+            time.sleep(2)
+            new_utxos = self.node.listunspent()
+
+        self.log.info("Amplification step produced %d new \"Fake Stake\" inputs:" % len(new_utxos))
+        return new_utxos
+
+
+
+    def stake_amplification(self, utxo_list, iterations, address_list = []):
+        ''' performs the "stake amplification" which gives higher chances at finding fake stakes
+        :param      utxo_list:    (JSON list) returned from listunspent used as input
+                    iterations:   (int) amount of stake amplification steps to perform
+                    address_list: (string list) [optional] recipient Agrarian addresses.
+        :return:    all_inputs:   (JSON list) list of all spent inputs
+        '''
+        self.log.info("** Stake Amplification started with %d UTXOs", len(utxo_list))
+        valid_inputs = utxo_list
+        all_inputs = []
+        for i in range(iterations):
+            all_inputs = all_inputs + valid_inputs
+            old_inputs = valid_inputs
+            valid_inputs = self.stake_amplification_step(old_inputs, address_list)
+        self.log.info("** Stake Amplification ended with %d \"fake\" UTXOs", len(all_inputs))
+        return all_inputs
+
+
+
+    def sign_stake_tx(self, block, stake_in_value, fZPoS=False):
+        ''' signs a coinstake transaction
+        :param      block:           (CBlock) block with stake to sign
+                    stake_in_value:  (int) staked amount
+                    fZPoS:           (bool) zerocoin stake
+        :return:    stake_tx_signed: (CTransaction) signed tx
+        '''
+        self.block_sig_key = CECKey()
+
+        if fZPoS:
+            self.log.info("Signing zPoS stake...")
+            # Create raw zerocoin stake TX (signed)
+            raw_stake = self.node.createrawzerocoinstake(block.prevoutStake)
+            stake_tx_signed_raw_hex = raw_stake["hex"]
+            # Get stake TX private key to sign the block with
+            stake_pkey = raw_stake["private-key"]
+            self.block_sig_key.set_compressed(True)
+            self.block_sig_key.set_secretbytes(bytes.fromhex(stake_pkey))
+
+        else:
+            # Create a new private key and get the corresponding public key
+            self.block_sig_key.set_secretbytes(hash256(pack('<I', 0xffff)))
+            pubkey = self.block_sig_key.get_pubkey()
+            # Create the raw stake TX (unsigned)
+            scriptPubKey = CScript([pubkey, OP_CHECKSIG])
+            outNValue = int(stake_in_value + 2*COIN)
+            stake_tx_unsigned = CTransaction()
+            stake_tx_unsigned.nTime = block.nTime
+            stake_tx_unsigned.vin.append(CTxIn(block.prevoutStake))
+            stake_tx_unsigned.vin[0].nSequence = 0xffffffff
+            stake_tx_unsigned.vout.append(CTxOut())
+            stake_tx_unsigned.vout.append(CTxOut(outNValue, scriptPubKey))
+            # Sign the stake TX
+            stake_tx_signed_raw_hex = self.node.signrawtransaction(bytes_to_hex_str(stake_tx_unsigned.serialize()))['hex']
+
+        # Deserialize the signed raw tx into a CTransaction object and return it
+        stake_tx_signed = CTransaction()
+        stake_tx_signed.deserialize(BytesIO(hex_str_to_bytes(stake_tx_signed_raw_hex)))
+        return stake_tx_signed
+
+
+    def get_prevouts(self, utxo_list, blockHeight, zpos=False):
+        ''' get prevouts (map) for each utxo in a list
+        :param   utxo_list: <if zpos=False> (JSON list) utxos returned from listunspent used as input
+                            <if zpos=True>  (JSON list) mints returned from listmintedzerocoins used as input
+                 blockHeight:               (int) height of the previous block
+                 zpos:                      (bool) type of utxo_list
+        :return: stakingPrevOuts:           ({COutPoint --> (int, int, int, str)} dictionary)
+                                            map outpoints to amount, block_time, nStakeModifier, hashStake
+        '''
+        zerocoinDenomList = [1, 5, 10, 50, 100, 500, 1000, 5000]
+        stakingPrevOuts = {}
+
+        for utxo in utxo_list:
+            if zpos:
+                # get mint checkpoint
+                checkpointHeight = blockHeight - 200
+                checkpointBlock = self.node.getblock(self.node.getblockhash(checkpointHeight), True)
+                checkpoint = int(checkpointBlock['acc_checkpoint'], 16)
+                # parse checksum and get checksumblock
+                pos = zerocoinDenomList.index(utxo['denomination'])
+                checksum = (checkpoint >> (32 * (len(zerocoinDenomList) - 1 - pos))) & 0xFFFFFFFF
+                checksumBlock = self.node.getchecksumblock(hex(checksum), utxo['denomination'], True)
+                # get block hash and block time
+                txBlockhash = checksumBlock['hash']
+                txBlocktime = checksumBlock['time']
+            else:
+                # get raw transaction for current input
+                utxo_tx = self.node.getrawtransaction(utxo['txid'], 1)
+                # get block hash and block time
+                txBlocktime = utxo_tx['blocktime']
+                txBlockhash = utxo_tx['blockhash']
+
+            # get Stake Modifier
+            stakeModifier = int(self.node.getblock(txBlockhash)['modifier'], 16)
+            # assemble prevout object
+            utxo_to_stakingPrevOuts(utxo, stakingPrevOuts, txBlocktime, stakeModifier, zpos)
+
+        return stakingPrevOuts
+
+
+
+    def log_data_dir_size(self):
+        ''' Prints the size of the '/regtest/blocks' directory.
+        :param:
+        :return:
+        '''
+        init_size = dir_size(self.node.datadir + "/regtest/blocks")
+        self.log.info("Size of data dir: %s kilobytes" % str(init_size))
+
+
+
+    def test_spam(self, name, staking_utxo_list,
+                  fRandomHeight=False, randomRange=0, randomRange2=0,
+                  fDoubleSpend=False, fMustPass=False, fZPoS=False,
+                  spending_utxo_list=[]):
+        ''' General method to create, send and test the spam blocks
+        :param    name:               (string) chain branch (usually either "Main" or "Forked")
+                  staking_utxo_list:  (string list) utxos to use for staking
+                  fRandomHeight:      (bool) send blocks at random height
+                  randomRange:        (int) if fRandomHeight=True, height is >= current-randomRange
+                  randomRange2:       (int) if fRandomHeight=True, height is < current-randomRange2
+                  fDoubleSpend:       (bool) if true, stake input is double spent in block.vtx
+                  fMustPass:          (bool) if true, the blocks must be stored on disk
+                  fZPoS:              (bool) stake the block with zerocoin
+                  spending_utxo_list: (string list) utxos to use for spending
+        :return:  err_msgs:           (string list) reports error messages from the test
+                                      or an empty list if test is successful
+        '''
+        # Create empty error messages list
+        err_msgs = []
+        # Log initial datadir size
+        self.log_data_dir_size()
+        # Get latest block number and hash
+        block_count = self.node.getblockcount()
+        pastBlockHash = self.node.getblockhash(block_count)
+        randomCount = block_count
+        self.log.info("Current height: %d" % block_count)
+        for i in range(0, self.NUM_BLOCKS):
+            if i !=0:
+                self.log.info("Sent %d blocks out of %d" % (i, self.NUM_BLOCKS))
+
+            # if fRandomHeight=True get a random block number (in range) and corresponding hash
+            if fRandomHeight:
+                randomCount = randint(block_count - randomRange, block_count - randomRange2)
+                pastBlockHash = self.node.getblockhash(randomCount)
+
+            # Get spending prevouts and staking prevouts for the height of current block
+            current_block_n = randomCount + 1
+            stakingPrevOuts = self.get_prevouts(staking_utxo_list, randomCount, zpos=fZPoS)
+            spendingPrevOuts = self.get_prevouts(spending_utxo_list, randomCount)
+
+            # Create the spam block
+            block = self.create_spam_block(pastBlockHash, stakingPrevOuts, current_block_n,
+                                           fStakeDoubleSpent=fDoubleSpend, fZPoS=fZPoS, spendingPrevOuts=spendingPrevOuts)
+
+            # Log time and size of the block
+            block_time = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(block.nTime))
+            block_size = len(block.serialize())/1000
+            self.log.info("Sending block %d [%s...] - nTime: %s - Size (kb): %.2f",
+                          current_block_n, block.hash[:7], block_time, block_size)
+
+            # Try submitblock
+            var = self.node.submitblock(bytes_to_hex_str(block.serialize()))
+            time.sleep(1)
+            if (not fMustPass and var not in [None, "bad-txns-invalid-zagr"]) or (fMustPass and var != "inconclusive"):
+                self.log.error("submitblock [fMustPass=%s] result: %s" % (str(fMustPass), str(var)))
+                err_msgs.append("submitblock %d: %s" % (current_block_n, str(var)))
+
+            # Try sending the message block
+            msg = msg_block(block)
+            try:
+                self.test_nodes[0].handle_connect()
+                self.test_nodes[0].send_message(msg)
+                time.sleep(2)
+                block_ret = self.node.getblock(block.hash)
+                if not fMustPass and block_ret is not None:
+                    self.log.error("Error, block stored in %s chain" % name)
+                    err_msgs.append("getblock %d: result not None" % current_block_n)
+                if fMustPass:
+                    if block_ret is None:
+                        self.log.error("Error, block NOT stored in %s chain" % name)
+                        err_msgs.append("getblock %d: result is None" % current_block_n)
+                    else:
+                        self.log.info("Good. Block IS stored on disk.")
+
+            except JSONRPCException as e:
+                exc_msg = str(e)
+                if exc_msg == "Can't read block from disk (-32603)":
+                    if fMustPass:
+                        self.log.warning("Bad! Block was NOT stored to disk.")
+                        err_msgs.append(exc_msg)
+                    else:
+                        self.log.info("Good. Block was not stored on disk.")
+                else:
+                    self.log.warning(exc_msg)
+                    err_msgs.append(exc_msg)
+
+            except Exception as e:
+                exc_msg = str(e)
+                self.log.error(exc_msg)
+                err_msgs.append(exc_msg)
+
+
+        self.log.info("Sent all %s blocks." % str(self.NUM_BLOCKS))
+        # Log final datadir size
+        self.log_data_dir_size()
+        # Return errors list
+        return err_msgs

test/functional/fake_stake/util.py

@@ -0,0 +1,143 @@
+#!/usr/bin/env python3
+# Copyright (c) 2019 The PIVX developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+# -*- coding: utf-8 -*-
+
+import subprocess
+
+from test_framework.messages import CTransaction, CTxIn, CTxOut, COutPoint, COIN
+from test_framework.messages import msg_getheaders, msg_headers, CBlockHeader
+from test_framework.mininode import P2PInterface, mininode_lock
+from test_framework.script import CScript
+from test_framework.util import wait_until
+
+''' -------------------------------------------------------------------------
+TestNode CLASS --------------------------------------------------------------
+
+A peer we use to send messsages to agrariand and store responses
+Extends P2PInterface.
+'''
+
+# TestNode: A peer we use to send messages to bitcoind, and store responses.
+class TestNode(P2PInterface):
+    def __init__(self):
+        super().__init__()
+        self.last_sendcmpct = []
+        self.block_announced = False
+        # Store the hashes of blocks we've seen announced.
+        # This is for synchronizing the p2p message traffic,
+        # so we can eg wait until a particular block is announced.
+        self.announced_blockhashes = set()
+
+    def on_sendcmpct(self, message):
+        self.last_sendcmpct.append(message)
+
+    def on_cmpctblock(self, message):
+        self.block_announced = True
+        self.last_message["cmpctblock"].header_and_shortids.header.calc_sha256()
+        self.announced_blockhashes.add(self.last_message["cmpctblock"].header_and_shortids.header.sha256)
+
+    def on_headers(self, message):
+        self.block_announced = True
+        for x in self.last_message["headers"].headers:
+            x.calc_sha256()
+            self.announced_blockhashes.add(x.sha256)
+
+    def on_inv(self, message):
+        for x in self.last_message["inv"].inv:
+            if x.type == 2:
+                self.block_announced = True
+                self.announced_blockhashes.add(x.hash)
+
+    # Requires caller to hold mininode_lock
+    def received_block_announcement(self):
+        return self.block_announced
+
+    def clear_block_announcement(self):
+        with mininode_lock:
+            self.block_announced = False
+            self.last_message.pop("inv", None)
+            self.last_message.pop("headers", None)
+            self.last_message.pop("cmpctblock", None)
+
+    def get_headers(self, locator, hashstop):
+        msg = msg_getheaders()
+        msg.locator.vHave = locator
+        msg.hashstop = hashstop
+        self.connection.send_message(msg)
+
+    def send_header_for_blocks(self, new_blocks):
+        headers_message = msg_headers()
+        headers_message.headers = [CBlockHeader(b) for b in new_blocks]
+        self.send_message(headers_message)
+
+    def request_headers_and_sync(self, locator, hashstop=0):
+        self.clear_block_announcement()
+        self.get_headers(locator, hashstop)
+        wait_until(self.received_block_announcement, timeout=30, lock=mininode_lock)
+        self.clear_block_announcement()
+
+    # Block until a block announcement for a particular block hash is received.
+    def wait_for_block_announcement(self, block_hash, timeout=30):
+        def received_hash():
+            return (block_hash in self.announced_blockhashes)
+        wait_until(received_hash, timeout=timeout, lock=mininode_lock)
+
+    def send_await_disconnect(self, message, timeout=30):
+        """Sends a message to the node and wait for disconnect.
+
+        This is used when we want to send a message into the node that we expect
+        will get us disconnected, eg an invalid block."""
+        self.send_message(message)
+        wait_until(lambda: not self.connected, timeout=timeout, lock=mininode_lock)
+
+
+
+''' -------------------------------------------------------------------------
+MISC METHODS ----------------------------------------------------------------
+'''
+
+def dir_size(path):
+    ''' returns the size in bytes of the directory at given path
+    '''
+    size = subprocess.check_output(['du','-shk', path]).split()[0].decode('utf-8')
+    return int(size)
+
+
+def create_transaction(outPoint, sig, value, nTime, scriptPubKey=CScript()):
+    ''' creates a CTransaction object provided input-output data
+    '''
+    tx = CTransaction()
+    tx.vin.append(CTxIn(outPoint, sig, 0xffffffff))
+    tx.vout.append(CTxOut(value, scriptPubKey))
+    tx.nTime = nTime
+    tx.calc_sha256()
+    return tx
+
+
+def utxo_to_stakingPrevOuts(utxo, stakingPrevOuts, txBlocktime, stakeModifier, zpos=False):
+    '''
+    Updates a map of unspent outputs to (amount, blocktime) to be used as stake inputs
+    :param   utxo:     <if zpos=False>  (map) utxo JSON object returned from listunspent
+                       <if zpos=True>   (map) mint JSON object returned from listmintedzerocoins
+             stakingPrevOuts:   ({COutPoint --> (int, int, int, str)} dictionary)
+                                map outpoints to amount, block_time, nStakeModifier, hashStake hex
+             txBlocktime:       (int) block time of the stake Modifier
+             stakeModifier:     (int) stake modifier for the current utxo
+             zpos:              (bool) if true, utxo holds a zerocoin serial hash
+    :return
+    '''
+
+    COINBASE_MATURITY = 200 if zpos else 100
+    if utxo['confirmations'] > COINBASE_MATURITY:
+        if zpos:
+            outPoint = utxo["serial hash"]
+            stakingPrevOuts[outPoint] = (int(utxo["denomination"]) * COIN, txBlocktime, stakeModifier, utxo['hash stake'])
+        else:
+            outPoint = COutPoint(int(utxo['txid'], 16), utxo['vout'])
+            stakingPrevOuts[outPoint] = (int(utxo['amount'])*COIN, txBlocktime, stakeModifier, "")
+
+    return
+
+

test/functional/feature_cltv.py

@@ -0,0 +1,154 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test BIP65 (CHECKLOCKTIMEVERIFY).
+
+Test that the CHECKLOCKTIMEVERIFY soft-fork activates at (regtest) block height
+1351.
+"""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+from test_framework.mininode import *
+from test_framework.blocktools import create_coinbase, create_block
+from test_framework.script import CScript, OP_1NEGATE, OP_CHECKLOCKTIMEVERIFY, OP_DROP, CScriptNum
+from io import BytesIO
+
+CLTV_HEIGHT = 750
+
+# Reject codes that we might receive in this test
+REJECT_INVALID = 16
+REJECT_OBSOLETE = 17
+REJECT_NONSTANDARD = 64
+
+def cltv_invalidate(tx):
+    '''Modify the signature in vin 0 of the tx to fail CLTV
+
+    Prepends -1 CLTV DROP in the scriptSig itself.
+
+    TODO: test more ways that transactions using CLTV could be invalid (eg
+    locktime requirements fail, sequence time requirements fail, etc).
+    '''
+    tx.vin[0].scriptSig = CScript([OP_1NEGATE, OP_CHECKLOCKTIMEVERIFY, OP_DROP] +
+                                  list(CScript(tx.vin[0].scriptSig)))
+
+def cltv_validate(node, tx, height):
+    '''Modify the signature in vin 0 of the tx to pass CLTV
+    Prepends <height> CLTV DROP in the scriptSig, and sets
+    the locktime to height'''
+    tx.vin[0].nSequence = 0
+    tx.nLockTime = height
+
+    # Need to re-sign, since nSequence and nLockTime changed
+    signed_result = node.signrawtransaction(ToHex(tx))
+    new_tx = CTransaction()
+    new_tx.deserialize(BytesIO(hex_str_to_bytes(signed_result['hex'])))
+
+    new_tx.vin[0].scriptSig = CScript([CScriptNum(height), OP_CHECKLOCKTIMEVERIFY, OP_DROP] +
+                                  list(CScript(new_tx.vin[0].scriptSig)))
+    return new_tx
+
+def create_transaction(node, coinbase, to_address, amount):
+    from_txid = node.getblock(coinbase)['tx'][0]
+    inputs = [{ "txid" : from_txid, "vout" : 0}]
+    outputs = { to_address : amount }
+    rawtx = node.createrawtransaction(inputs, outputs)
+    signresult = node.signrawtransaction(rawtx)
+    tx = CTransaction()
+    tx.deserialize(BytesIO(hex_str_to_bytes(signresult['hex'])))
+    return tx
+
+class BIP65Test(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 1
+        self.extra_args = [['-promiscuousmempoolflags=1', '-whitelist=127.0.0.1']]
+        self.setup_clean_chain = True
+
+    def run_test(self):
+        self.nodes[0].add_p2p_connection(P2PInterface())
+
+        network_thread_start()
+
+        # wait_for_verack ensures that the P2P connection is fully up.
+        self.nodes[0].p2p.wait_for_verack()
+
+        self.log.info("Mining %d blocks", CLTV_HEIGHT - 2)
+        self.coinbase_blocks = self.nodes[0].generate(CLTV_HEIGHT - 2)
+        self.nodeaddress = self.nodes[0].getnewaddress()
+
+        self.log.info("Test that an invalid-according-to-CLTV transaction can still appear in a block")
+
+        spendtx = create_transaction(self.nodes[0], self.coinbase_blocks[0],
+                self.nodeaddress, 1.0)
+        cltv_invalidate(spendtx)
+        spendtx.rehash()
+
+        tip = self.nodes[0].getbestblockhash()
+        block_time = self.nodes[0].getblockheader(tip)['mediantime'] + 1
+        block = create_block(int(tip, 16), create_coinbase(CLTV_HEIGHT - 1), block_time)
+        block.nVersion = 4
+        block.vtx.append(spendtx)
+        block.hashMerkleRoot = block.calc_merkle_root()
+        block.solve()
+
+        self.nodes[0].p2p.send_and_ping(msg_block(block))
+        assert_equal(self.nodes[0].getbestblockhash(), block.hash)
+
+        self.nodes[0].generate(205)
+
+        self.log.info("Test that blocks must now be at least version 5")
+        tip = self.nodes[0].getbestblockhash()
+        block_time = self.nodes[0].getblockheader(tip)['mediantime'] + 1
+        block = create_block(int(tip, 16), create_coinbase(CLTV_HEIGHT + 205), block_time)
+        block.nVersion = 4
+        block.solve()
+        self.nodes[0].p2p.send_and_ping(msg_block(block))
+
+        wait_until(lambda: "reject" in self.nodes[0].p2p.last_message.keys(), lock=mininode_lock)
+        with mininode_lock:
+            assert_equal(self.nodes[0].p2p.last_message["reject"].code, REJECT_OBSOLETE)
+            assert_equal(self.nodes[0].p2p.last_message["reject"].reason, b'bad-version')
+            assert_equal(self.nodes[0].p2p.last_message["reject"].data, block.sha256)
+            del self.nodes[0].p2p.last_message["reject"]
+
+        self.log.info("Test that invalid-according-to-cltv transactions cannot appear in a block")
+        block.nVersion = 5
+
+        spendtx = create_transaction(self.nodes[0], self.coinbase_blocks[1],
+                self.nodeaddress, 1.0)
+        cltv_invalidate(spendtx)
+        spendtx.rehash()
+
+        # Verify that a block with this transaction is invalid.
+        block.vtx.append(spendtx)
+        block.hashMerkleRoot = block.calc_merkle_root()
+        block.solve()
+
+        self.nodes[0].p2p.send_and_ping(msg_block(block))
+
+        wait_until(lambda: "reject" in self.nodes[0].p2p.last_message.keys(), lock=mininode_lock)
+        with mininode_lock:
+            assert self.nodes[0].p2p.last_message["reject"].code in [REJECT_INVALID, REJECT_NONSTANDARD]
+            assert_equal(self.nodes[0].p2p.last_message["reject"].data, block.sha256)
+            if self.nodes[0].p2p.last_message["reject"].code == REJECT_INVALID:
+                # Generic rejection when a block is invalid
+                assert_equal(self.nodes[0].p2p.last_message["reject"].reason, b'block-validation-failed')
+            else:
+                assert b'Negative locktime' in self.nodes[0].p2p.last_message["reject"].reason
+
+        self.log.info("Test that a version 5 block with a valid-according-to-CLTV transaction is accepted")
+        spendtx = cltv_validate(self.nodes[0], spendtx, CLTV_HEIGHT - 1)
+        spendtx.rehash()
+
+        block.vtx.pop(1)
+        block.vtx.append(spendtx)
+        block.hashMerkleRoot = block.calc_merkle_root()
+        block.solve()
+
+        self.nodes[0].p2p.send_and_ping(msg_block(block))
+        assert_equal(int(self.nodes[0].getbestblockhash(), 16), block.sha256)
+
+
+if __name__ == '__main__':
+    BIP65Test().main()

test/functional/feature_config_args.py

@@ -0,0 +1,49 @@
+#!/usr/bin/env python3
+# Copyright (c) 2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test various command line arguments and configuration file parameters."""
+
+import os
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import get_datadir_path
+
+class ConfArgsTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+
+    def run_test(self):
+        self.stop_node(0)
+        # Remove the -datadir argument so it doesn't override the config file
+        self.nodes[0].args = [arg for arg in self.nodes[0].args if not arg.startswith("-datadir")]
+
+        default_data_dir = get_datadir_path(self.options.tmpdir, 0)
+        new_data_dir = os.path.join(default_data_dir, 'newdatadir')
+        new_data_dir_2 = os.path.join(default_data_dir, 'newdatadir2')
+
+        # Check that using -datadir argument on non-existent directory fails
+        self.nodes[0].datadir = new_data_dir
+        self.assert_start_raises_init_error(0, ['-datadir='+new_data_dir], 'Error: Specified data directory "' + new_data_dir + '" does not exist.')
+
+        # Check that using non-existent datadir in conf file fails
+        conf_file = os.path.join(default_data_dir, "bitcoin.conf")
+        with open(conf_file, 'a', encoding='utf8') as f:
+            f.write("datadir=" + new_data_dir + "\n")
+        self.assert_start_raises_init_error(0, ['-conf='+conf_file], 'Error reading configuration file: specified data directory "' + new_data_dir + '" does not exist.')
+
+        # Create the directory and ensure the config file now works
+        os.mkdir(new_data_dir)
+        self.start_node(0, ['-conf='+conf_file, '-wallet=w1'])
+        self.stop_node(0)
+        assert os.path.isfile(os.path.join(new_data_dir, 'regtest', 'wallets', 'w1'))
+
+        # Ensure command line argument overrides datadir in conf
+        os.mkdir(new_data_dir_2)
+        self.nodes[0].datadir = new_data_dir_2
+        self.start_node(0, ['-datadir='+new_data_dir_2, '-conf='+conf_file, '-wallet=w2'])
+        assert os.path.isfile(os.path.join(new_data_dir_2, 'regtest', 'wallets', 'w2'))
+
+if __name__ == '__main__':
+    ConfArgsTest().main()

test/functional/feature_fee_estimation.py

@@ -0,0 +1,262 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test fee estimation code."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+from test_framework.script import CScript, OP_1, OP_DROP, OP_2, OP_HASH160, OP_EQUAL, hash160, OP_TRUE
+from test_framework.mininode import CTransaction, CTxIn, CTxOut, COutPoint, ToHex, COIN
+
+# Construct 2 trivial P2SH's and the ScriptSigs that spend them
+# So we can create many transactions without needing to spend
+# time signing.
+redeem_script_1 = CScript([OP_1, OP_DROP])
+redeem_script_2 = CScript([OP_2, OP_DROP])
+P2SH_1 = CScript([OP_HASH160, hash160(redeem_script_1), OP_EQUAL])
+P2SH_2 = CScript([OP_HASH160, hash160(redeem_script_2), OP_EQUAL])
+
+# Associated ScriptSig's to spend satisfy P2SH_1 and P2SH_2
+SCRIPT_SIG = [CScript([OP_TRUE, redeem_script_1]), CScript([OP_TRUE, redeem_script_2])]
+
+global log
+
+def small_txpuzzle_randfee(from_node, conflist, unconflist, amount, min_fee, fee_increment):
+    """
+    Create and send a transaction with a random fee.
+    The transaction pays to a trivial P2SH script, and assumes that its inputs
+    are of the same form.
+    The function takes a list of confirmed outputs and unconfirmed outputs
+    and attempts to use the confirmed list first for its inputs.
+    It adds the newly created outputs to the unconfirmed list.
+    Returns (raw transaction, fee)
+    """
+    # It's best to exponentially distribute our random fees
+    # because the buckets are exponentially spaced.
+    # Exponentially distributed from 1-128 * fee_increment
+    rand_fee = float(fee_increment)*(1.1892**random.randint(0,28))
+    # Total fee ranges from min_fee to min_fee + 127*fee_increment
+    fee = min_fee - fee_increment + satoshi_round(rand_fee)
+    tx = CTransaction()
+    total_in = Decimal("0.00000000")
+    while total_in <= (amount + fee) and len(conflist) > 0:
+        t = conflist.pop(0)
+        total_in += t["amount"]
+        tx.vin.append(CTxIn(COutPoint(int(t["txid"], 16), t["vout"]), b""))
+    if total_in <= amount + fee:
+        while total_in <= (amount + fee) and len(unconflist) > 0:
+            t = unconflist.pop(0)
+            total_in += t["amount"]
+            tx.vin.append(CTxIn(COutPoint(int(t["txid"], 16), t["vout"]), b""))
+        if total_in <= amount + fee:
+            raise RuntimeError("Insufficient funds: need %d, have %d"%(amount+fee, total_in))
+    tx.vout.append(CTxOut(int((total_in - amount - fee)*COIN), P2SH_1))
+    tx.vout.append(CTxOut(int(amount*COIN), P2SH_2))
+    # These transactions don't need to be signed, but we still have to insert
+    # the ScriptSig that will satisfy the ScriptPubKey.
+    for inp in tx.vin:
+        inp.scriptSig = SCRIPT_SIG[inp.prevout.n]
+    txid = from_node.sendrawtransaction(ToHex(tx), True)
+    unconflist.append({ "txid" : txid, "vout" : 0 , "amount" : total_in - amount - fee})
+    unconflist.append({ "txid" : txid, "vout" : 1 , "amount" : amount})
+
+    return (ToHex(tx), fee)
+
+def split_inputs(from_node, txins, txouts, initial_split = False):
+    """
+    We need to generate a lot of inputs so we can generate a ton of transactions.
+    This function takes an input from txins, and creates and sends a transaction
+    which splits the value into 2 outputs which are appended to txouts.
+    Previously this was designed to be small inputs so they wouldn't have
+    a high coin age when the notion of priority still existed.
+    """
+    prevtxout = txins.pop()
+    tx = CTransaction()
+    tx.vin.append(CTxIn(COutPoint(int(prevtxout["txid"], 16), prevtxout["vout"]), b""))
+
+    half_change = satoshi_round(prevtxout["amount"]/2)
+    rem_change = prevtxout["amount"] - half_change  - Decimal("0.00001000")
+    tx.vout.append(CTxOut(int(half_change*COIN), P2SH_1))
+    tx.vout.append(CTxOut(int(rem_change*COIN), P2SH_2))
+
+    # If this is the initial split we actually need to sign the transaction
+    # Otherwise we just need to insert the proper ScriptSig
+    if (initial_split) :
+        completetx = from_node.signrawtransaction(ToHex(tx))["hex"]
+    else :
+        tx.vin[0].scriptSig = SCRIPT_SIG[prevtxout["vout"]]
+        completetx = ToHex(tx)
+    txid = from_node.sendrawtransaction(completetx, True)
+    txouts.append({ "txid" : txid, "vout" : 0 , "amount" : half_change})
+    txouts.append({ "txid" : txid, "vout" : 1 , "amount" : rem_change})
+
+def check_estimates(node, fees_seen, max_invalid, print_estimates = True):
+    """
+    This function calls estimatefee and verifies that the estimates
+    meet certain invariants.
+    """
+    all_estimates = [ node.estimatefee(i) for i in range(1,26) ]
+    if print_estimates:
+        log.info([str(all_estimates[e-1]) for e in [1,2,3,6,15,25]])
+    delta = 1.0e-6 # account for rounding error
+    last_e = max(fees_seen)
+    for e in [x for x in all_estimates if x >= 0]:
+        # Estimates should be within the bounds of what transactions fees actually were:
+        if float(e)+delta < min(fees_seen) or float(e)-delta > max(fees_seen):
+            raise AssertionError("Estimated fee (%f) out of range (%f,%f)"
+                                 %(float(e), min(fees_seen), max(fees_seen)))
+        # Estimates should be monotonically decreasing
+        if float(e)-delta > last_e:
+            raise AssertionError("Estimated fee (%f) larger than last fee (%f) for lower number of confirms"
+                                 %(float(e),float(last_e)))
+        last_e = e
+    valid_estimate = False
+    invalid_estimates = 0
+    for i,e in enumerate(all_estimates): # estimate is for i+1
+        if e >= 0:
+            valid_estimate = True
+            if i >= 13:  # for n>=14 estimatesmartfee(n/2) should be at least as high as estimatefee(n)
+                assert(node.estimatesmartfee((i+1)//2)["feerate"] > float(e) - delta)
+
+        else:
+            invalid_estimates += 1
+
+            # estimatesmartfee should still be valid
+            approx_estimate = node.estimatesmartfee(i+1)["feerate"]
+            answer_found = node.estimatesmartfee(i+1)["blocks"]
+            assert(approx_estimate > 0)
+            assert(answer_found > i+1)
+
+            # Once we're at a high enough confirmation count that we can give an estimate
+            # We should have estimates for all higher confirmation counts
+            if valid_estimate:
+                raise AssertionError("Invalid estimate appears at higher confirm count than valid estimate")
+
+    # Check on the expected number of different confirmation counts
+    # that we might not have valid estimates for
+    if invalid_estimates > max_invalid:
+        raise AssertionError("More than (%d) invalid estimates"%(max_invalid))
+    return all_estimates
+
+
+class EstimateFeeTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 3
+
+    def setup_network(self):
+        """
+        We'll setup the network to have 3 nodes that all mine with different parameters.
+        But first we need to use one node to create a lot of outputs
+        which we will use to generate our transactions.
+        """
+        self.add_nodes(3, extra_args=[["-maxorphantx=1000", "-whitelist=127.0.0.1"],
+                                      ["-maxorphantx=1000", "-deprecatedrpc=estimatefee"],
+                                      ["-maxorphantx=1000"]])
+        # Use node0 to mine blocks for input splitting
+        # Node1 mines small blocks but that are bigger than the expected transaction rate.
+        # NOTE: the CreateNewBlock code starts counting block size at 1,000 bytes,
+        # (17k is room enough for 110 or so transactions)
+        # Node2 is a stingy miner, that
+        # produces too small blocks (room for only 55 or so transactions)
+
+
+    def transact_and_mine(self, numblocks, mining_node):
+        min_fee = Decimal("0.00001")
+        # We will now mine numblocks blocks generating on average 100 transactions between each block
+        # We shuffle our confirmed txout set before each set of transactions
+        # small_txpuzzle_randfee will use the transactions that have inputs already in the chain when possible
+        # resorting to tx's that depend on the mempool when those run out
+        for i in range(numblocks):
+            random.shuffle(self.confutxo)
+            for j in range(random.randrange(100-50,100+50)):
+                from_index = random.randint(1,2)
+                (txhex, fee) = small_txpuzzle_randfee(self.nodes[from_index], self.confutxo,
+                                                      self.memutxo, Decimal("0.005"), min_fee, min_fee)
+                tx_kbytes = (len(txhex) // 2) / 1000.0
+                self.fees_per_kb.append(float(fee)/tx_kbytes)
+            sync_mempools(self.nodes[0:3], wait=.1)
+            mined = mining_node.getblock(mining_node.generate(1)[0],True)["tx"]
+            sync_blocks(self.nodes[0:3], wait=.1)
+            # update which txouts are confirmed
+            newmem = []
+            for utx in self.memutxo:
+                if utx["txid"] in mined:
+                    self.confutxo.append(utx)
+                else:
+                    newmem.append(utx)
+            self.memutxo = newmem
+
+    def run_test(self):
+        self.log.info("This test is time consuming, please be patient")
+        self.log.info("Splitting inputs so we can generate tx's")
+
+        # Make log handler available to helper functions
+        global log
+        log = self.log
+
+        # Start node0
+        self.start_node(0)
+        self.txouts = []
+        self.txouts2 = []
+        # Split a coinbase into two transaction puzzle outputs
+        split_inputs(self.nodes[0], self.nodes[0].listunspent(0), self.txouts, True)
+
+        # Mine
+        while (len(self.nodes[0].getrawmempool()) > 0):
+            self.nodes[0].generate(1)
+
+        # Repeatedly split those 2 outputs, doubling twice for each rep
+        # Use txouts to monitor the available utxo, since these won't be tracked in wallet
+        reps = 0
+        while (reps < 5):
+            #Double txouts to txouts2
+            while (len(self.txouts)>0):
+                split_inputs(self.nodes[0], self.txouts, self.txouts2)
+            while (len(self.nodes[0].getrawmempool()) > 0):
+                self.nodes[0].generate(1)
+            #Double txouts2 to txouts
+            while (len(self.txouts2)>0):
+                split_inputs(self.nodes[0], self.txouts2, self.txouts)
+            while (len(self.nodes[0].getrawmempool()) > 0):
+                self.nodes[0].generate(1)
+            reps += 1
+        self.log.info("Finished splitting")
+
+        # Now we can connect the other nodes, didn't want to connect them earlier
+        # so the estimates would not be affected by the splitting transactions
+        self.start_node(1)
+        self.start_node(2)
+        connect_nodes(self.nodes[1], 0)
+        connect_nodes(self.nodes[0], 2)
+        connect_nodes(self.nodes[2], 1)
+
+        self.sync_all()
+
+        self.fees_per_kb = []
+        self.memutxo = []
+        self.confutxo = self.txouts # Start with the set of confirmed txouts after splitting
+        self.log.info("Will output estimates for 1/2/3/6/15/25 blocks")
+
+        for i in range(2):
+            self.log.info("Creating transactions and mining them with a block size that can't keep up")
+            # Create transactions and mine 10 small blocks with node 2, but create txs faster than we can mine
+            self.transact_and_mine(10, self.nodes[2])
+            check_estimates(self.nodes[1], self.fees_per_kb, 14)
+
+            self.log.info("Creating transactions and mining them at a block size that is just big enough")
+            # Generate transactions while mining 10 more blocks, this time with node1
+            # which mines blocks with capacity just above the rate that transactions are being created
+            self.transact_and_mine(10, self.nodes[1])
+            check_estimates(self.nodes[1], self.fees_per_kb, 2)
+
+        # Finish by mining a normal-sized block:
+        while len(self.nodes[1].getrawmempool()) > 0:
+            self.nodes[1].generate(1)
+
+        sync_blocks(self.nodes[0:3], wait=.1)
+        self.log.info("Final estimates after emptying mempools")
+        check_estimates(self.nodes[1], self.fees_per_kb, 2)
+
+if __name__ == '__main__':
+    EstimateFeeTest().main()

test/functional/feature_help.py

@@ -0,0 +1,46 @@
+#!/usr/bin/env python3
+# Copyright (c) 2018 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Verify that starting bitcoin with -h works as expected."""
+import subprocess
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import assert_equal
+
+class HelpTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+
+    def setup_network(self):
+        self.add_nodes(self.num_nodes)
+        # Don't start the node
+
+    def run_test(self):
+        self.log.info("Start bitcoin with -h for help text")
+        self.nodes[0].start(extra_args=['-?'], stderr=subprocess.PIPE, stdout=subprocess.PIPE)
+        # Node should exit immediately and output help to stdout.
+        ret_code = self.nodes[0].process.wait(timeout=1)
+        assert_equal(ret_code, 1)
+        output = self.nodes[0].process.stdout.read()
+        assert b'Options' in output
+        self.log.info("Help text received: {} (...)".format(output[0:60]))
+        self.nodes[0].running = False
+
+        self.log.info("Start bitcoin with -version for version information")
+        self.nodes[0].start(extra_args=['-version'], stderr=subprocess.PIPE, stdout=subprocess.PIPE)
+        # Node should exit immediately and output version to stdout.
+        ret_code = self.nodes[0].process.wait(timeout=1)
+        assert_equal(ret_code, 1)
+        output = self.nodes[0].process.stdout.read()
+        assert b'version' in output
+        self.log.info("Version text received: {} (...)".format(output[0:60]))
+        # Clean up TestNode state
+        self.nodes[0].running = False
+        self.nodes[0].process = None
+        self.nodes[0].rpc_connected = False
+        self.nodes[0].rpc = None
+
+if __name__ == '__main__':
+    HelpTest().main()

test/functional/feature_minchainwork.py

@@ -0,0 +1,89 @@
+#!/usr/bin/env python3
+# Copyright (c) 2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test logic for setting nMinimumChainWork on command line.
+
+Nodes don't consider themselves out of "initial block download" until
+their active chain has more work than nMinimumChainWork.
+
+Nodes don't download blocks from a peer unless the peer's best known block
+has more work than nMinimumChainWork.
+
+While in initial block download, nodes won't relay blocks to their peers, so
+test that this parameter functions as intended by verifying that block relay
+only succeeds past a given node once its nMinimumChainWork has been exceeded.
+"""
+
+import time
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import connect_nodes, assert_equal
+
+# 2 hashes required per regtest block (with no difficulty adjustment)
+REGTEST_WORK_PER_BLOCK = 2
+
+class MinimumChainWorkTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 3
+
+        self.extra_args = [[], ["-minimumchainwork=0x65"], ["-minimumchainwork=0x65"]]
+        self.node_min_work = [0, 101, 101]
+
+    def setup_network(self):
+        # This test relies on the chain setup being:
+        # node0 <- node1 <- node2
+        # Before leaving IBD, nodes prefer to download blocks from outbound
+        # peers, so ensure that we're mining on an outbound peer and testing
+        # block relay to inbound peers.
+        self.setup_nodes()
+        for i in range(self.num_nodes-1):
+            connect_nodes(self.nodes[i+1], i)
+
+    def run_test(self):
+        # Start building a chain on node0.  node2 shouldn't be able to sync until node1's
+        # minchainwork is exceeded
+        starting_chain_work = REGTEST_WORK_PER_BLOCK # Genesis block's work
+        self.log.info("Testing relay across node %d (minChainWork = %d)", 1, self.node_min_work[1])
+
+        starting_blockcount = self.nodes[2].getblockcount()
+
+        num_blocks_to_generate = int((self.node_min_work[1] - starting_chain_work) / REGTEST_WORK_PER_BLOCK)
+        self.log.info("Generating %d blocks on node0", num_blocks_to_generate)
+        hashes = self.nodes[0].generate(num_blocks_to_generate)
+
+        self.log.info("Node0 current chain work: %s", self.nodes[0].getblockheader(hashes[-1])['chainwork'])
+
+        # Sleep a few seconds and verify that node2 didn't get any new blocks
+        # or headers.  We sleep, rather than sync_blocks(node0, node1) because
+        # it's reasonable either way for node1 to get the blocks, or not get
+        # them (since they're below node1's minchainwork).
+        time.sleep(3)
+
+        self.log.info("Verifying node 2 has no more blocks than before")
+        self.log.info("Blockcounts: %s", [n.getblockcount() for n in self.nodes])
+        # Node2 shouldn't have any new headers yet, because node1 should not
+        # have relayed anything.
+        assert_equal(len(self.nodes[2].getchaintips()), 1)
+        assert_equal(self.nodes[2].getchaintips()[0]['height'], 0)
+
+        assert self.nodes[1].getbestblockhash() != self.nodes[0].getbestblockhash()
+        assert_equal(self.nodes[2].getblockcount(), starting_blockcount)
+
+        self.log.info("Generating one more block")
+        self.nodes[0].generate(1)
+
+        self.log.info("Verifying nodes are all synced")
+
+        # Because nodes in regtest are all manual connections (eg using
+        # addnode), node1 should not have disconnected node0. If not for that,
+        # we'd expect node1 to have disconnected node0 for serving an
+        # insufficient work chain, in which case we'd need to reconnect them to
+        # continue the test.
+
+        self.sync_all()
+        self.log.info("Blockcounts: %s", [n.getblockcount() for n in self.nodes])
+
+if __name__ == '__main__':
+    MinimumChainWorkTest().main()

test/functional/feature_notifications.py

@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the -alertnotify, -blocknotify and -walletnotify options."""
+import os
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import assert_equal, wait_until, connect_nodes_bi
+
+class NotificationsTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+        self.setup_clean_chain = True
+
+    def setup_network(self):
+        self.alert_filename = os.path.join(self.options.tmpdir, "alert.txt")
+        self.block_filename = os.path.join(self.options.tmpdir, "blocks.txt")
+        self.tx_filename = os.path.join(self.options.tmpdir, "transactions.txt")
+
+        # -alertnotify and -blocknotify on node0, walletnotify on node1
+        self.extra_args = [["-blockversion=4",
+                            "-alertnotify=echo %%s >> %s" % self.alert_filename,
+                            "-blocknotify=echo %%s >> %s" % self.block_filename],
+                           ["-blockversion=211",
+                            "-rescan",
+                            "-walletnotify=echo %%s >> %s" % self.tx_filename]]
+        super().setup_network()
+
+    def run_test(self):
+        self.log.info("test -blocknotify")
+        block_count = 10
+        blocks = self.nodes[1].generate(block_count)
+
+        # wait at most 10 seconds for expected file size before reading the content
+        wait_until(lambda: os.path.isfile(self.block_filename) and os.stat(self.block_filename).st_size >= (block_count * 65), timeout=10)
+
+        # file content should equal the generated blocks hashes
+        with open(self.block_filename, 'r') as f:
+            assert_equal(sorted(blocks), sorted(f.read().splitlines()))
+
+        self.log.info("test -walletnotify")
+        # wait at most 10 seconds for expected file size before reading the content
+        wait_until(lambda: os.path.isfile(self.tx_filename) and os.stat(self.tx_filename).st_size >= (block_count * 65), timeout=10)
+
+        # file content should equal the generated transaction hashes
+        txids_rpc = list(map(lambda t: t['txid'], self.nodes[1].listtransactions("*", block_count)))
+        with open(self.tx_filename, 'r') as f:
+            assert_equal(sorted(txids_rpc), sorted(f.read().splitlines()))
+        os.remove(self.tx_filename)
+
+        self.log.info("test -walletnotify after rescan")
+        # restart node to rescan to force wallet notifications
+        self.restart_node(1)
+        connect_nodes_bi(self.nodes, 0, 1)
+
+        wait_until(lambda: os.path.isfile(self.tx_filename) and os.stat(self.tx_filename).st_size >= (block_count * 65), timeout=10)
+
+        # file content should equal the generated transaction hashes
+        txids_rpc = list(map(lambda t: t['txid'], self.nodes[1].listtransactions("*", block_count)))
+        with open(self.tx_filename, 'r') as f:
+            assert_equal(sorted(txids_rpc), sorted(f.read().splitlines()))
+
+        # Mine another 41 up-version blocks. -alertnotify should trigger on the 51st.
+        self.log.info("test -alertnotify")
+        self.nodes[1].generate(51)
+        self.sync_all()
+
+        # Give bitcoind 10 seconds to write the alert notification
+        wait_until(lambda: os.path.isfile(self.alert_filename) and os.path.getsize(self.alert_filename), timeout=10)
+
+        with open(self.alert_filename, 'r', encoding='utf8') as f:
+            alert_text = f.read()
+
+        # Mine more up-version blocks, should not get more alerts:
+        self.nodes[1].generate(2)
+        self.sync_all()
+
+        with open(self.alert_filename, 'r', encoding='utf8') as f:
+            alert_text2 = f.read()
+
+        self.log.info("-alertnotify should not continue notifying for more unknown version blocks")
+        assert_equal(alert_text, alert_text2)
+
+if __name__ == '__main__':
+    NotificationsTest().main()

test/functional/feature_nulldummy.py

@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+# Copyright (c) 2016-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test NULLDUMMY softfork.
+
+Connect to a single node.
+Generate 2 blocks (save the coinbases for later).
+Generate 427 more blocks.
+[Policy/Consensus] Check that NULLDUMMY compliant transactions are accepted in the 430th block.
+[Policy] Check that non-NULLDUMMY transactions are rejected before activation.
+[Consensus] Check that the new NULLDUMMY rules are not enforced on the 431st block.
+[Policy/Consensus] Check that the new NULLDUMMY rules are enforced on the 432nd block.
+"""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+from test_framework.mininode import CTransaction, network_thread_start
+from test_framework.blocktools import create_coinbase, create_block, add_witness_commitment
+from test_framework.script import CScript
+from io import BytesIO
+import time
+
+NULLDUMMY_ERROR = "64: non-mandatory-script-verify-flag (Dummy CHECKMULTISIG argument must be zero)"
+
+def trueDummy(tx):
+    scriptSig = CScript(tx.vin[0].scriptSig)
+    newscript = []
+    for i in scriptSig:
+        if (len(newscript) == 0):
+            assert(len(i) == 0)
+            newscript.append(b'\x51')
+        else:
+            newscript.append(i)
+    tx.vin[0].scriptSig = CScript(newscript)
+    tx.rehash()
+
+class NULLDUMMYTest(BitcoinTestFramework):
+
+    def set_test_params(self):
+        self.num_nodes = 1
+        self.setup_clean_chain = True
+        # This script tests NULLDUMMY activation, which is part of the 'segwit' deployment, so we go through
+        # Must set the blockversion for this test
+        self.extra_args = [['-whitelist=127.0.0.1']]
+
+    def run_test(self):
+        self.address = self.nodes[0].getnewaddress()
+        self.ms_address = self.nodes[0].addmultisigaddress(1,[self.address])
+
+        network_thread_start()
+        self.coinbase_blocks = self.nodes[0].generate(2) # Block 2
+        coinbase_txid = []
+        for i in self.coinbase_blocks:
+            coinbase_txid.append(self.nodes[0].getblock(i)['tx'][0])
+        self.nodes[0].generate(427) # Block 429
+        self.lastblockhash = self.nodes[0].getbestblockhash()
+        self.tip = int("0x" + self.lastblockhash, 0)
+        self.lastblockheight = 429
+        self.lastblocktime = int(time.time()) + 429
+
+        self.log.info("Test 1: NULLDUMMY compliant base transactions should be accepted to mempool and mined before activation [430]")
+        test1txs = [self.create_transaction(self.nodes[0], coinbase_txid[0], self.ms_address, 49)]
+        txid1 = self.nodes[0].sendrawtransaction(bytes_to_hex_str(test1txs[0].serialize_without_witness()), True)
+        test1txs.append(self.create_transaction(self.nodes[0], txid1, self.ms_address, 48))
+        txid2 = self.nodes[0].sendrawtransaction(bytes_to_hex_str(test1txs[1].serialize_without_witness()), True)
+        self.block_submit(self.nodes[0], test1txs, True)
+
+        self.log.info("Test 2: Non-NULLDUMMY base multisig transaction should not be accepted to mempool before activation")
+        test2tx = self.create_transaction(self.nodes[0], txid2, self.ms_address, 48)
+        trueDummy(test2tx)
+        txid4 = self.tx_submit(self.nodes[0], test2tx, NULLDUMMY_ERROR)
+
+        self.log.info("Test 3: Non-NULLDUMMY base transactions should be accepted in a block before activation [431]")
+        self.block_submit(self.nodes[0], [test2tx], True)
+
+        self.log.info("Test 4: Non-NULLDUMMY base multisig transaction is invalid after activation")
+        test4tx = self.create_transaction(self.nodes[0], txid4, self.address, 47)
+        test6txs=[CTransaction(test4tx)]
+        trueDummy(test4tx)
+        self.tx_submit(self.nodes[0], test4tx, NULLDUMMY_ERROR)
+        self.block_submit(self.nodes[0], [test4tx])
+
+        self.log.info("Test 6: NULLDUMMY compliant transactions should be accepted to mempool and in block after activation [432]")
+        for i in test6txs:
+            self.nodes[0].sendrawtransaction(bytes_to_hex_str(i.serialize_without_witness()), True)
+        self.block_submit(self.nodes[0], test6txs, True)
+
+
+    def create_transaction(self, node, txid, to_address, amount):
+        inputs = [{ "txid" : txid, "vout" : 0}]
+        outputs = { to_address : amount }
+        rawtx = node.createrawtransaction(inputs, outputs)
+        signresult = node.signrawtransaction(rawtx)
+        tx = CTransaction()
+        f = BytesIO(hex_str_to_bytes(signresult['hex']))
+        tx.deserialize(f)
+        return tx
+
+
+    def tx_submit(self, node, tx, msg = ""):
+        tx.rehash()
+        try:
+            node.sendrawtransaction(bytes_to_hex_str(tx.serialize()), True)
+        except JSONRPCException as exp:
+            assert_equal(exp.error["message"], msg)
+        else:
+            assert_equal('', msg)
+        return tx.hash
+
+
+    def block_submit(self, node, txs, accept = False):
+        block = create_block(self.tip, create_coinbase(self.lastblockheight + 1), self.lastblocktime + 1)
+        block.nVersion = 4
+        for tx in txs:
+            tx.rehash()
+            block.vtx.append(tx)
+        block.hashMerkleRoot = block.calc_merkle_root()
+        block.rehash()
+        block.solve()
+        node.submitblock(bytes_to_hex_str(block.serialize()))
+        if (accept):
+            assert_equal(node.getbestblockhash(), block.hash)
+            self.tip = block.sha256
+            self.lastblockhash = block.hash
+            self.lastblocktime += 1
+            self.lastblockheight += 1
+        else:
+            assert_equal(node.getbestblockhash(), self.lastblockhash)
+
+if __name__ == '__main__':
+    NULLDUMMYTest().main()

test/functional/feature_proxy.py

@@ -0,0 +1,201 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test bitcoind with different proxy configuration.
+
+Test plan:
+- Start agrariand's with different proxy configurations
+- Use addnode to initiate connections
+- Verify that proxies are connected to, and the right connection command is given
+- Proxy configurations to test on agrariand side:
+    - `-proxy` (proxy everything)
+    - `-onion` (proxy just onions)
+    - `-proxyrandomize` Circuit randomization
+- Proxy configurations to test on proxy side,
+    - support no authentication (other proxy)
+    - support no authentication + user/pass authentication (Tor)
+    - proxy on IPv6
+
+- Create various proxies (as threads)
+- Create agrariands that connect to them
+- Manipulate the agrariands using addnode (onetry) an observe effects
+
+addnode connect to IPv4
+addnode connect to IPv6
+addnode connect to onion
+addnode connect to generic DNS name
+"""
+
+import socket
+import os
+
+from test_framework.socks5 import Socks5Configuration, Socks5Command, Socks5Server, AddressType
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (
+    PORT_MIN,
+    PORT_RANGE,
+    assert_equal,
+)
+from test_framework.netutil import test_ipv6_local
+
+RANGE_BEGIN = PORT_MIN + 2 * PORT_RANGE  # Start after p2p and rpc ports
+
+class ProxyTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 4
+
+    def setup_nodes(self):
+        self.have_ipv6 = test_ipv6_local()
+        # Create two proxies on different ports
+        # ... one unauthenticated
+        self.conf1 = Socks5Configuration()
+        self.conf1.addr = ('127.0.0.1', RANGE_BEGIN + (os.getpid() % 1000))
+        self.conf1.unauth = True
+        self.conf1.auth = False
+        # ... one supporting authenticated and unauthenticated (Tor)
+        self.conf2 = Socks5Configuration()
+        self.conf2.addr = ('127.0.0.1', RANGE_BEGIN + 1000 + (os.getpid() % 1000))
+        self.conf2.unauth = True
+        self.conf2.auth = True
+        if self.have_ipv6:
+            # ... one on IPv6 with similar configuration
+            self.conf3 = Socks5Configuration()
+            self.conf3.af = socket.AF_INET6
+            self.conf3.addr = ('::1', RANGE_BEGIN + 2000 + (os.getpid() % 1000))
+            self.conf3.unauth = True
+            self.conf3.auth = True
+        else:
+            self.log.warning("Testing without local IPv6 support")
+
+        self.serv1 = Socks5Server(self.conf1)
+        self.serv1.start()
+        self.serv2 = Socks5Server(self.conf2)
+        self.serv2.start()
+        if self.have_ipv6:
+            self.serv3 = Socks5Server(self.conf3)
+            self.serv3.start()
+
+        # Note: proxies are not used to connect to local nodes
+        # this is because the proxy to use is based on CService.GetNetwork(), which return NET_UNROUTABLE for localhost
+        args = [
+            ['-listen', '-proxy=%s:%i' % (self.conf1.addr),'-proxyrandomize=1'],
+            ['-listen', '-proxy=%s:%i' % (self.conf1.addr),'-onion=%s:%i' % (self.conf2.addr),'-proxyrandomize=0'],
+            ['-listen', '-proxy=%s:%i' % (self.conf2.addr),'-proxyrandomize=1'],
+            []
+            ]
+        if self.have_ipv6:
+            args[3] = ['-listen', '-proxy=[%s]:%i' % (self.conf3.addr),'-proxyrandomize=0', '-noonion']
+        self.add_nodes(self.num_nodes, extra_args=args)
+        self.start_nodes()
+
+    def node_test(self, node, proxies, auth, test_onion=True):
+        rv = []
+        # Test: outgoing IPv4 connection through node
+        node.addnode("15.61.23.23:1234", "onetry")
+        cmd = proxies[0].queue.get()
+        assert(isinstance(cmd, Socks5Command))
+        # Note: bitcoind's SOCKS5 implementation only sends atyp DOMAINNAME, even if connecting directly to IPv4/IPv6
+        assert_equal(cmd.atyp, AddressType.DOMAINNAME)
+        assert_equal(cmd.addr, b"15.61.23.23")
+        assert_equal(cmd.port, 1234)
+        if not auth:
+            assert_equal(cmd.username, None)
+            assert_equal(cmd.password, None)
+        rv.append(cmd)
+
+        if self.have_ipv6:
+            # Test: outgoing IPv6 connection through node
+            node.addnode("[1233:3432:2434:2343:3234:2345:6546:4534]:5443", "onetry")
+            cmd = proxies[1].queue.get()
+            assert(isinstance(cmd, Socks5Command))
+            # Note: bitcoind's SOCKS5 implementation only sends atyp DOMAINNAME, even if connecting directly to IPv4/IPv6
+            assert_equal(cmd.atyp, AddressType.DOMAINNAME)
+            assert_equal(cmd.addr, b"1233:3432:2434:2343:3234:2345:6546:4534")
+            assert_equal(cmd.port, 5443)
+            if not auth:
+                assert_equal(cmd.username, None)
+                assert_equal(cmd.password, None)
+            rv.append(cmd)
+
+        if test_onion:
+            # Test: outgoing onion connection through node
+            node.addnode("bitcoinostk4e4re.onion:8333", "onetry")
+            cmd = proxies[2].queue.get()
+            assert(isinstance(cmd, Socks5Command))
+            assert_equal(cmd.atyp, AddressType.DOMAINNAME)
+            assert_equal(cmd.addr, b"bitcoinostk4e4re.onion")
+            assert_equal(cmd.port, 8333)
+            if not auth:
+                assert_equal(cmd.username, None)
+                assert_equal(cmd.password, None)
+            rv.append(cmd)
+
+        # Test: outgoing DNS name connection through node
+        node.addnode("node.noumenon:8333", "onetry")
+        cmd = proxies[3].queue.get()
+        assert(isinstance(cmd, Socks5Command))
+        assert_equal(cmd.atyp, AddressType.DOMAINNAME)
+        assert_equal(cmd.addr, b"node.noumenon")
+        assert_equal(cmd.port, 8333)
+        if not auth:
+            assert_equal(cmd.username, None)
+            assert_equal(cmd.password, None)
+        rv.append(cmd)
+
+        return rv
+
+    def run_test(self):
+        # basic -proxy
+        self.node_test(self.nodes[0], [self.serv1, self.serv1, self.serv1, self.serv1], False)
+
+        # -proxy plus -onion
+        self.node_test(self.nodes[1], [self.serv1, self.serv1, self.serv2, self.serv1], False)
+
+        # -proxy plus -onion, -proxyrandomize
+        rv = self.node_test(self.nodes[2], [self.serv2, self.serv2, self.serv2, self.serv2], True)
+        # Check that credentials as used for -proxyrandomize connections are unique
+        credentials = set((x.username,x.password) for x in rv)
+        assert_equal(len(credentials), len(rv))
+
+        if self.have_ipv6:
+            # proxy on IPv6 localhost
+            self.node_test(self.nodes[3], [self.serv3, self.serv3, self.serv3, self.serv3], False, False)
+
+        def networks_dict(d):
+            r = {}
+            for x in d['networks']:
+                r[x['name']] = x
+            return r
+
+        # test RPC getnetworkinfo
+        n0 = networks_dict(self.nodes[0].getnetworkinfo())
+        for net in ['ipv4','ipv6','onion']:
+            assert_equal(n0[net]['proxy'], '%s:%i' % (self.conf1.addr))
+            assert_equal(n0[net]['proxy_randomize_credentials'], True)
+        assert_equal(n0['onion']['reachable'], True)
+
+        n1 = networks_dict(self.nodes[1].getnetworkinfo())
+        for net in ['ipv4','ipv6']:
+            assert_equal(n1[net]['proxy'], '%s:%i' % (self.conf1.addr))
+            assert_equal(n1[net]['proxy_randomize_credentials'], False)
+        assert_equal(n1['onion']['proxy'], '%s:%i' % (self.conf2.addr))
+        assert_equal(n1['onion']['proxy_randomize_credentials'], False)
+        assert_equal(n1['onion']['reachable'], True)
+
+        n2 = networks_dict(self.nodes[2].getnetworkinfo())
+        for net in ['ipv4','ipv6','onion']:
+            assert_equal(n2[net]['proxy'], '%s:%i' % (self.conf2.addr))
+            assert_equal(n2[net]['proxy_randomize_credentials'], True)
+        assert_equal(n2['onion']['reachable'], True)
+
+        if self.have_ipv6:
+            n3 = networks_dict(self.nodes[3].getnetworkinfo())
+            for net in ['ipv4','ipv6']:
+                assert_equal(n3[net]['proxy'], '[%s]:%i' % (self.conf3.addr))
+                assert_equal(n3[net]['proxy_randomize_credentials'], False)
+            assert_equal(n3['onion']['reachable'], False)
+
+if __name__ == '__main__':
+    ProxyTest().main()
+

test/functional/feature_reindex.py

@@ -0,0 +1,37 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test running bitcoind with -reindex and -reindex-chainstate options.
+
+- Start a single node and generate 3 blocks.
+- Stop the node and restart it with -reindex. Verify that the node has reindexed up to block 3.
+- Stop the node and restart it with -reindex-chainstate. Verify that the node has reindexed up to block 3.
+"""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import wait_until
+import time
+
+class ReindexTest(BitcoinTestFramework):
+
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+
+    def reindex(self):
+        self.nodes[0].generate(3)
+        blockcount = self.nodes[0].getblockcount()
+        self.stop_nodes()
+        time.sleep(5)
+        extra_args = [["-reindex", "-checkblockindex=1"]]
+        self.start_nodes(extra_args)
+        time.sleep(15)
+        wait_until(lambda: self.nodes[0].getblockcount() == blockcount)
+        self.log.info("Success")
+
+    def run_test(self):
+        self.reindex()
+
+if __name__ == '__main__':
+    ReindexTest().main()

test/functional/feature_uacomment.py

@@ -0,0 +1,35 @@
+#!/usr/bin/env python3
+# Copyright (c) 2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the -uacomment option."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import assert_equal
+
+class UacommentTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 1
+        self.setup_clean_chain = True
+
+    def run_test(self):
+        self.log.info("test multiple -uacomment")
+        test_uacomment = self.nodes[0].getnetworkinfo()["subversion"][-12:-1]
+        assert_equal(test_uacomment, "(testnode0)")
+
+        self.restart_node(0, ["-uacomment=foo"])
+        foo_uacomment = self.nodes[0].getnetworkinfo()["subversion"][-17:-1]
+        assert_equal(foo_uacomment, "(testnode0; foo)")
+
+        self.log.info("test -uacomment max length")
+        self.stop_node(0)
+        expected = "exceeds maximum length (256). Reduce the number or size of uacomments."
+        self.assert_start_raises_init_error(0, ["-uacomment=" + 'a' * 256], expected)
+
+        self.log.info("test -uacomment unsafe characters")
+        for unsafe_char in ['/', ':', '(', ')']:
+            expected = "User Agent comment (" + unsafe_char + ") contains unsafe characters"
+            self.assert_start_raises_init_error(0, ["-uacomment=" + unsafe_char], expected)
+
+if __name__ == '__main__':
+    UacommentTest().main()

test/functional/interface_bitcoin_cli.py

@@ -0,0 +1,58 @@
+#!/usr/bin/env python3
+# Copyright (c) 2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test agrarian-cli"""
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import assert_equal, assert_raises_process_error, get_auth_cookie
+import time
+
+class TestBitcoinCli(BitcoinTestFramework):
+
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+
+    def run_test(self):
+        """Main test logic"""
+
+        self.log.info("Sleeping 30 seconds...")
+        time.sleep(30)
+
+        self.log.info("Compare responses from gewalletinfo RPC and `agrarian-cli getwalletinfo`")
+        cli_response = self.nodes[0].cli.getwalletinfo()
+        rpc_response = self.nodes[0].getwalletinfo()
+        assert_equal(cli_response, rpc_response)
+
+        self.log.info("Compare responses from getblockchaininfo RPC and `agrarian-cli getblockchaininfo`")
+        cli_response = self.nodes[0].cli.getblockchaininfo()
+        rpc_response = self.nodes[0].getblockchaininfo()
+        assert_equal(cli_response, rpc_response)
+
+        user, password = get_auth_cookie(self.nodes[0].datadir)
+
+        self.log.info("Compare responses from `agrarian-cli -getinfo` and the RPCs data is retrieved from.")
+        cli_get_info = self.nodes[0].cli('getinfo').send_cli()
+        wallet_info = self.nodes[0].getwalletinfo()
+        network_info = self.nodes[0].getnetworkinfo()
+        blockchain_info = self.nodes[0].getblockchaininfo()
+
+        assert_equal(cli_get_info['version'], network_info['version'])
+        assert_equal(cli_get_info['protocolversion'], network_info['protocolversion'])
+        assert_equal(cli_get_info['walletversion'], wallet_info['walletversion'])
+        assert_equal(cli_get_info['balance'], wallet_info['balance'])
+        assert_equal(cli_get_info['blocks'], blockchain_info['blocks'])
+        assert_equal(cli_get_info['timeoffset'], network_info['timeoffset'])
+        assert_equal(cli_get_info['connections'], network_info['connections'])
+        assert_equal(cli_get_info['proxy'], network_info['networks'][0]['proxy'])
+        assert_equal(cli_get_info['difficulty'], blockchain_info['difficulty'])
+        assert_equal(cli_get_info['testnet'], blockchain_info['chain'] == "test")
+        assert_equal(cli_get_info['balance'], wallet_info['balance'])
+        assert_equal(cli_get_info['keypoololdest'], wallet_info['keypoololdest'])
+        assert_equal(cli_get_info['keypoolsize'], wallet_info['keypoolsize'])
+        assert_equal(cli_get_info['paytxfee'], wallet_info['paytxfee'])
+        assert_equal(cli_get_info['relayfee'], network_info['relayfee'])
+        # unlocked_until is not tested because the wallet is not encrypted
+
+if __name__ == '__main__':
+    TestBitcoinCli().main()

test/functional/interface_http.py

@@ -0,0 +1,108 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the RPC HTTP basics."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+import http.client
+import urllib.parse
+
+class HTTPBasicsTest (BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 3
+
+    def setup_network(self):
+        self.setup_nodes()
+
+    def run_test(self):
+
+        #################################################
+        # lowlevel check for http persistent connection #
+        #################################################
+        url = urllib.parse.urlparse(self.nodes[0].url)
+        authpair = url.username + ':' + url.password
+        headers = {"Authorization": "Basic " + str_to_b64str(authpair)}
+
+        conn = http.client.HTTPConnection(url.hostname, url.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        out1 = conn.getresponse().read()
+        assert(b'"error":null' in out1)
+        assert(conn.sock!=None) #according to http/1.1 connection must still be open!
+
+        #send 2nd request without closing connection
+        conn.request('POST', '/', '{"method": "getchaintips"}', headers)
+        out1 = conn.getresponse().read()
+        assert(b'"error":null' in out1) #must also response with a correct json-rpc message
+        assert(conn.sock!=None) #according to http/1.1 connection must still be open!
+        conn.close()
+
+        #same should be if we add keep-alive because this should be the std. behaviour
+        headers = {"Authorization": "Basic " + str_to_b64str(authpair), "Connection": "keep-alive"}
+
+        conn = http.client.HTTPConnection(url.hostname, url.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        out1 = conn.getresponse().read()
+        assert(b'"error":null' in out1)
+        assert(conn.sock!=None) #according to http/1.1 connection must still be open!
+
+        #send 2nd request without closing connection
+        conn.request('POST', '/', '{"method": "getchaintips"}', headers)
+        out1 = conn.getresponse().read()
+        assert(b'"error":null' in out1) #must also response with a correct json-rpc message
+        assert(conn.sock!=None) #according to http/1.1 connection must still be open!
+        conn.close()
+
+        #now do the same with "Connection: close"
+        headers = {"Authorization": "Basic " + str_to_b64str(authpair), "Connection":"close"}
+
+        conn = http.client.HTTPConnection(url.hostname, url.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        out1 = conn.getresponse().read()
+        assert(b'"error":null' in out1)
+        assert(conn.sock==None) #now the connection must be closed after the response
+
+        #node1 (2nd node) is running with disabled keep-alive option
+        urlNode1 = urllib.parse.urlparse(self.nodes[1].url)
+        authpair = urlNode1.username + ':' + urlNode1.password
+        headers = {"Authorization": "Basic " + str_to_b64str(authpair)}
+
+        conn = http.client.HTTPConnection(urlNode1.hostname, urlNode1.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        out1 = conn.getresponse().read()
+        assert(b'"error":null' in out1)
+
+        #node2 (third node) is running with standard keep-alive parameters which means keep-alive is on
+        urlNode2 = urllib.parse.urlparse(self.nodes[2].url)
+        authpair = urlNode2.username + ':' + urlNode2.password
+        headers = {"Authorization": "Basic " + str_to_b64str(authpair)}
+
+        conn = http.client.HTTPConnection(urlNode2.hostname, urlNode2.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        out1 = conn.getresponse().read()
+        assert(b'"error":null' in out1)
+        assert(conn.sock!=None) #connection must be closed because agrariand should use keep-alive by default
+
+        # Check excessive request size
+        conn = http.client.HTTPConnection(urlNode2.hostname, urlNode2.port)
+        conn.connect()
+        conn.request('GET', '/' + ('x'*1000), '', headers)
+        out1 = conn.getresponse()
+        assert_equal(out1.status, http.client.NOT_FOUND)
+
+        conn = http.client.HTTPConnection(urlNode2.hostname, urlNode2.port)
+        conn.connect()
+        conn.request('GET', '/' + ('x'*10000), '', headers)
+        out1 = conn.getresponse()
+        assert_equal(out1.status, http.client.BAD_REQUEST)
+
+
+if __name__ == '__main__':
+    HTTPBasicsTest ().main ()

test/functional/interface_rest.py

@@ -0,0 +1,326 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the REST API."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+from struct import *
+from io import BytesIO
+from codecs import encode
+
+import http.client
+import urllib.parse
+
+def deser_uint256(f):
+    r = 0
+    for i in range(8):
+        t = unpack(b"<I", f.read(4))[0]
+        r += t << (i * 32)
+    return r
+
+#allows simple http get calls
+def http_get_call(host, port, path, response_object = 0):
+    conn = http.client.HTTPConnection(host, port)
+    conn.request('GET', path)
+
+    if response_object:
+        return conn.getresponse()
+
+    return conn.getresponse().read().decode('utf-8')
+
+#allows simple http post calls with a request body
+def http_post_call(host, port, path, requestdata = '', response_object = 0):
+    conn = http.client.HTTPConnection(host, port)
+    conn.request('POST', path, requestdata)
+
+    if response_object:
+        return conn.getresponse()
+
+    return conn.getresponse().read()
+
+class RESTTest (BitcoinTestFramework):
+    FORMAT_SEPARATOR = "."
+
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 3
+
+    def setup_network(self, split=False):
+        super().setup_network()
+        connect_nodes_bi(self.nodes, 0, 2)
+
+    def run_test(self):
+        url = urllib.parse.urlparse(self.nodes[0].url)
+        self.log.info("Mining blocks...")
+
+        self.nodes[0].generate(1)
+        self.sync_all()
+        self.nodes[2].generate(100)
+        self.sync_all()
+
+        assert_equal(self.nodes[0].getbalance(), 250)
+
+        txid = self.nodes[0].sendtoaddress(self.nodes[1].getnewaddress(), 0.1)
+        self.sync_all()
+        self.nodes[2].generate(1)
+        self.sync_all()
+        bb_hash = self.nodes[0].getbestblockhash()
+
+        assert_equal(self.nodes[1].getbalance(), Decimal("0.1")) #balance now should be 0.1 on node 1
+
+        # load the latest 0.1 tx over the REST API
+        json_string = http_get_call(url.hostname, url.port, '/rest/tx/'+txid+self.FORMAT_SEPARATOR+"json")
+        json_obj = json.loads(json_string)
+        vintx = json_obj['vin'][0]['txid'] # get the vin to later check for utxo (should be spent by then)
+        # get n of 0.1 outpoint
+        n = 0
+        for vout in json_obj['vout']:
+            if vout['value'] == 0.1:
+                n = vout['n']
+
+
+        #######################################
+        # GETUTXOS: query an unspent outpoint #
+        #######################################
+        json_request = '/checkmempool/'+txid+'-'+str(n)
+        json_string = http_get_call(url.hostname, url.port, '/rest/getutxos'+json_request+self.FORMAT_SEPARATOR+'json')
+        json_obj = json.loads(json_string)
+
+        #check chainTip response
+        assert_equal(json_obj['chaintipHash'], bb_hash)
+
+        #make sure there is one utxo
+        assert_equal(len(json_obj['utxos']), 1)
+        assert_equal(json_obj['utxos'][0]['value'], 0.1)
+
+
+        #################################################
+        # GETUTXOS: now query an already spent outpoint #
+        #################################################
+        json_request = '/checkmempool/'+vintx+'-0'
+        json_string = http_get_call(url.hostname, url.port, '/rest/getutxos'+json_request+self.FORMAT_SEPARATOR+'json')
+        json_obj = json.loads(json_string)
+
+        #check chainTip response
+        assert_equal(json_obj['chaintipHash'], bb_hash)
+
+        #make sure there is no utox in the response because this oupoint has been spent
+        assert_equal(len(json_obj['utxos']), 0)
+
+        #check bitmap
+        assert_equal(json_obj['bitmap'], "0")
+
+
+        ##################################################
+        # GETUTXOS: now check both with the same request #
+        ##################################################
+        json_request = '/checkmempool/'+txid+'-'+str(n)+'/'+vintx+'-0'
+        json_string = http_get_call(url.hostname, url.port, '/rest/getutxos'+json_request+self.FORMAT_SEPARATOR+'json')
+        json_obj = json.loads(json_string)
+        assert_equal(len(json_obj['utxos']), 1)
+        assert_equal(json_obj['bitmap'], "10")
+
+        #test binary response
+        bb_hash = self.nodes[0].getbestblockhash()
+
+        binaryRequest = b'\x01\x02'
+        binaryRequest += hex_str_to_bytes(txid)
+        binaryRequest += pack("i", n)
+        binaryRequest += hex_str_to_bytes(vintx)
+        binaryRequest += pack("i", 0)
+
+        bin_response = http_post_call(url.hostname, url.port, '/rest/getutxos'+self.FORMAT_SEPARATOR+'bin', binaryRequest)
+        output = BytesIO()
+        output.write(bin_response)
+        output.seek(0)
+        chainHeight = unpack("i", output.read(4))[0]
+        hashFromBinResponse = hex(deser_uint256(output))[2:].zfill(64)
+
+        assert_equal(bb_hash, hashFromBinResponse) #check if getutxo's chaintip during calculation was fine
+        assert_equal(chainHeight, 102) #chain height must be 102
+
+
+        ############################
+        # GETUTXOS: mempool checks #
+        ############################
+
+        # do a tx and don't sync
+        txid = self.nodes[0].sendtoaddress(self.nodes[1].getnewaddress(), 0.1)
+        json_string = http_get_call(url.hostname, url.port, '/rest/tx/'+txid+self.FORMAT_SEPARATOR+"json")
+        json_obj = json.loads(json_string)
+        vintx = json_obj['vin'][0]['txid'] # get the vin to later check for utxo (should be spent by then)
+        # get n of 0.1 outpoint
+        n = 0
+        for vout in json_obj['vout']:
+            if vout['value'] == 0.1:
+                n = vout['n']
+
+        json_request = '/'+txid+'-'+str(n)
+        json_string = http_get_call(url.hostname, url.port, '/rest/getutxos'+json_request+self.FORMAT_SEPARATOR+'json')
+        json_obj = json.loads(json_string)
+        assert_equal(len(json_obj['utxos']), 0) #there should be an outpoint because it has just added to the mempool
+
+        json_request = '/checkmempool/'+txid+'-'+str(n)
+        json_string = http_get_call(url.hostname, url.port, '/rest/getutxos'+json_request+self.FORMAT_SEPARATOR+'json')
+        json_obj = json.loads(json_string)
+        assert_equal(len(json_obj['utxos']), 1) #there should be an outpoint because it has just added to the mempool
+
+        #do some invalid requests
+        json_request = '{"checkmempool'
+        response = http_post_call(url.hostname, url.port, '/rest/getutxos'+self.FORMAT_SEPARATOR+'json', json_request, True)
+        assert_equal(response.status, 500) #must be a 500 because we send a invalid json request
+
+        json_request = '{"checkmempool'
+        response = http_post_call(url.hostname, url.port, '/rest/getutxos'+self.FORMAT_SEPARATOR+'bin', json_request, True)
+        assert_equal(response.status, 500) #must be a 500 because we send a invalid bin request
+
+        response = http_post_call(url.hostname, url.port, '/rest/getutxos/checkmempool'+self.FORMAT_SEPARATOR+'bin', '', True)
+        assert_equal(response.status, 500) #must be a 500 because we send a invalid bin request
+
+        #test limits
+        json_request = '/checkmempool/'
+        for x in range(0, 20):
+            json_request += txid+'-'+str(n)+'/'
+        json_request = json_request.rstrip("/")
+        response = http_post_call(url.hostname, url.port, '/rest/getutxos'+json_request+self.FORMAT_SEPARATOR+'json', '', True)
+        assert_equal(response.status, 500) #must be a 500 because we exceeding the limits
+
+        json_request = '/checkmempool/'
+        for x in range(0, 15):
+            json_request += txid+'-'+str(n)+'/'
+        json_request = json_request.rstrip("/")
+        response = http_post_call(url.hostname, url.port, '/rest/getutxos'+json_request+self.FORMAT_SEPARATOR+'json', '', True)
+        assert_equal(response.status, 200) #must be a 500 because we exceeding the limits
+
+        self.nodes[0].generate(1) #generate block to not affect upcoming tests
+        self.sync_all()
+
+        ################
+        # /rest/block/ #
+        ################
+
+        # check binary format
+        response = http_get_call(url.hostname, url.port, '/rest/block/'+bb_hash+self.FORMAT_SEPARATOR+"bin", True)
+        assert_equal(response.status, 200)
+        assert_greater_than(int(response.getheader('content-length')), 80)
+        response_str = response.read()
+
+        # compare with block header
+        response_header = http_get_call(url.hostname, url.port, '/rest/headers/1/'+bb_hash+self.FORMAT_SEPARATOR+"bin", True)
+        assert_equal(response_header.status, 200)
+        assert_equal(int(response_header.getheader('content-length')), 80)
+        response_header_str = response_header.read()
+        assert_equal(response_str[0:80], response_header_str)
+
+        # check block hex format
+        response_hex = http_get_call(url.hostname, url.port, '/rest/block/'+bb_hash+self.FORMAT_SEPARATOR+"hex", True)
+        assert_equal(response_hex.status, 200)
+        assert_greater_than(int(response_hex.getheader('content-length')), 160)
+        response_hex_str = response_hex.read()
+        assert_equal(encode(response_str, "hex_codec")[0:160], response_hex_str[0:160])
+
+        # compare with hex block header
+        response_header_hex = http_get_call(url.hostname, url.port, '/rest/headers/1/'+bb_hash+self.FORMAT_SEPARATOR+"hex", True)
+        assert_equal(response_header_hex.status, 200)
+        assert_greater_than(int(response_header_hex.getheader('content-length')), 160)
+        response_header_hex_str = response_header_hex.read()
+        assert_equal(response_hex_str[0:160], response_header_hex_str[0:160])
+        assert_equal(encode(response_header_str, "hex_codec")[0:160], response_header_hex_str[0:160])
+
+        # check json format
+        block_json_string = http_get_call(url.hostname, url.port, '/rest/block/'+bb_hash+self.FORMAT_SEPARATOR+'json')
+        block_json_obj = json.loads(block_json_string)
+        assert_equal(block_json_obj['hash'], bb_hash)
+
+        # compare with json block header
+        response_header_json = http_get_call(url.hostname, url.port, '/rest/headers/1/'+bb_hash+self.FORMAT_SEPARATOR+"json", True)
+        assert_equal(response_header_json.status, 200)
+        response_header_json_str = response_header_json.read().decode('utf-8')
+        json_obj = json.loads(response_header_json_str, parse_float=Decimal)
+        assert_equal(len(json_obj), 1) #ensure that there is one header in the json response
+        assert_equal(json_obj[0]['hash'], bb_hash) #request/response hash should be the same
+
+        #compare with normal RPC block response
+        rpc_block_json = self.nodes[0].getblock(bb_hash)
+        assert_equal(json_obj[0]['hash'],               rpc_block_json['hash'])
+        assert_equal(json_obj[0]['confirmations'],      rpc_block_json['confirmations'])
+        assert_equal(json_obj[0]['height'],             rpc_block_json['height'])
+        assert_equal(json_obj[0]['version'],            rpc_block_json['version'])
+        assert_equal(json_obj[0]['merkleroot'],         rpc_block_json['merkleroot'])
+        assert_equal(json_obj[0]['time'],               rpc_block_json['time'])
+        assert_equal(json_obj[0]['nonce'],              rpc_block_json['nonce'])
+        assert_equal(json_obj[0]['bits'],               rpc_block_json['bits'])
+        assert_equal(json_obj[0]['difficulty'],         rpc_block_json['difficulty'])
+        assert_equal(json_obj[0]['chainwork'],          rpc_block_json['chainwork'])
+        assert_equal(json_obj[0]['previousblockhash'],  rpc_block_json['previousblockhash'])
+
+        #see if we can get 5 headers in one response
+        self.nodes[1].generate(5)
+        self.sync_all()
+        response_header_json = http_get_call(url.hostname, url.port, '/rest/headers/5/'+bb_hash+self.FORMAT_SEPARATOR+"json", True)
+        assert_equal(response_header_json.status, 200)
+        response_header_json_str = response_header_json.read().decode('utf-8')
+        json_obj = json.loads(response_header_json_str)
+        assert_equal(len(json_obj), 5) #now we should have 5 header objects
+
+        # do tx test
+        tx_hash = block_json_obj['tx'][0]['txid']
+        json_string = http_get_call(url.hostname, url.port, '/rest/tx/'+tx_hash+self.FORMAT_SEPARATOR+"json")
+        json_obj = json.loads(json_string)
+        assert_equal(json_obj['txid'], tx_hash)
+
+        # check hex format response
+        hex_string = http_get_call(url.hostname, url.port, '/rest/tx/'+tx_hash+self.FORMAT_SEPARATOR+"hex", True)
+        assert_equal(hex_string.status, 200)
+        assert_greater_than(int(response.getheader('content-length')), 10)
+
+
+        # check block tx details
+        # let's make 3 tx and mine them on node 1
+        txs = []
+        txs.append(self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), 11))
+        txs.append(self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), 11))
+        txs.append(self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), 11))
+        self.sync_all()
+
+        # check that there are exactly 3 transactions in the TX memory pool before generating the block
+        json_string = http_get_call(url.hostname, url.port, '/rest/mempool/info'+self.FORMAT_SEPARATOR+'json')
+        json_obj = json.loads(json_string)
+        assert_equal(json_obj['size'], 3)
+        # the size of the memory pool should be greater than 3x ~100 bytes
+        assert_greater_than(json_obj['bytes'], 300)
+
+        # check that there are our submitted transactions in the TX memory pool
+        json_string = http_get_call(url.hostname, url.port, '/rest/mempool/contents'+self.FORMAT_SEPARATOR+'json')
+        json_obj = json.loads(json_string)
+        for tx in txs:
+            assert_equal(tx in json_obj, True)
+
+        # now mine the transactions
+        newblockhash = self.nodes[1].generate(1)
+        self.sync_all()
+
+        #check if the 3 tx show up in the new block
+        json_string = http_get_call(url.hostname, url.port, '/rest/block/'+newblockhash[0]+self.FORMAT_SEPARATOR+'json')
+        json_obj = json.loads(json_string)
+        for tx in json_obj['tx']:
+            if not 'coinbase' in tx['vin'][0]: #exclude coinbase
+                assert_equal(tx['txid'] in txs, True)
+
+        #check the same but without tx details
+        json_string = http_get_call(url.hostname, url.port, '/rest/block/notxdetails/'+newblockhash[0]+self.FORMAT_SEPARATOR+'json')
+        json_obj = json.loads(json_string)
+        for tx in txs:
+            assert_equal(tx in json_obj['tx'], True)
+
+        #test rest bestblock
+        bb_hash = self.nodes[0].getbestblockhash()
+
+        json_string = http_get_call(url.hostname, url.port, '/rest/chaininfo.json')
+        json_obj = json.loads(json_string)
+        assert_equal(json_obj['bestblockhash'], bb_hash)
+
+if __name__ == '__main__':
+    RESTTest ().main ()

test/functional/interface_zmq.py

@@ -0,0 +1,128 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the ZMQ notification interface."""
+import configparser
+import os
+import struct
+import time
+
+from test_framework.test_framework import BitcoinTestFramework, SkipTest
+from test_framework.mininode import CTransaction
+from test_framework.util import (assert_equal,
+                                 bytes_to_hex_str,
+                                 hash256,
+                                )
+from io import BytesIO
+
+class ZMQSubscriber:
+    def __init__(self, socket, topic):
+        self.sequence = 0
+        self.socket = socket
+        self.topic = topic
+
+        import zmq
+        self.socket.setsockopt(zmq.SUBSCRIBE, self.topic)
+
+    def receive(self):
+        topic, body, seq = self.socket.recv_multipart()
+        # Topic should match the subscriber topic.
+        assert_equal(topic, self.topic)
+        # Sequence should be incremental.
+        assert_equal(struct.unpack('<I', seq)[-1], self.sequence)
+        self.sequence += 1
+        return body
+
+
+class ZMQTest (BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+
+    def setup_nodes(self):
+        # Try to import python3-zmq. Skip this test if the import fails.
+        try:
+            import zmq
+        except ImportError:
+            raise SkipTest("python3-zmq module not available.")
+
+        # Check that bitcoin has been built with ZMQ enabled.
+        config = configparser.ConfigParser()
+        if not self.options.configfile:
+            self.options.configfile = os.path.abspath(os.path.join(os.path.dirname(__file__), "../config.ini"))
+        config.read_file(open(self.options.configfile))
+
+        if not config["components"].getboolean("ENABLE_ZMQ"):
+            raise SkipTest("bitcoind has not been built with zmq enabled.")
+
+        # Initialize ZMQ context and socket.
+        # All messages are received in the same socket which means
+        # that this test fails if the publishing order changes.
+        # Note that the publishing order is not defined in the documentation and
+        # is subject to change.
+        address = "tcp://127.0.0.1:28332"
+        self.zmq_context = zmq.Context()
+        socket = self.zmq_context.socket(zmq.SUB)
+        socket.set(zmq.RCVTIMEO, 60000)
+        socket.connect(address)
+
+        # Subscribe to all available topics.
+        self.hashblock = ZMQSubscriber(socket, b"hashblock")
+        self.hashtx = ZMQSubscriber(socket, b"hashtx")
+        self.rawblock = ZMQSubscriber(socket, b"rawblock")
+        self.rawtx = ZMQSubscriber(socket, b"rawtx")
+
+        self.extra_args = [["-zmqpub%s=%s" % (sub.topic.decode(), address) for sub in [self.hashblock, self.hashtx, self.rawblock, self.rawtx]], []]
+        self.add_nodes(self.num_nodes, self.extra_args)
+        self.start_nodes()
+        time.sleep(10)
+
+    def run_test(self):
+        try:
+            self._zmq_test()
+        finally:
+            # Destroy the ZMQ context.
+            self.log.debug("Destroying ZMQ context")
+            self.zmq_context.destroy(linger=None)
+
+    def _zmq_test(self):
+        num_blocks = 5
+        self.log.info("Generate %(n)d blocks (and %(n)d coinbase txes)" % {"n": num_blocks})
+        genhashes = self.nodes[0].generate(num_blocks)
+        self.sync_all()
+
+        for x in range(num_blocks):
+            # Should receive the coinbase txid.
+            txid = self.hashtx.receive()
+
+            # Should receive the coinbase raw transaction.
+            hex = self.rawtx.receive()
+            tx = CTransaction()
+            tx.deserialize(BytesIO(hex))
+            tx.calc_sha256()
+            assert_equal(tx.hash, bytes_to_hex_str(txid))
+
+            # Should receive the generated block hash.
+            hash = bytes_to_hex_str(self.hashblock.receive())
+            assert_equal(genhashes[x], hash)
+            # The block should only have the coinbase txid.
+            assert_equal([bytes_to_hex_str(txid)], self.nodes[1].getblock(hash)["tx"])
+
+            # Should receive the generated raw block.
+            block = self.rawblock.receive()
+            assert_equal(genhashes[x], bytes_to_hex_str(hash256(block[:112])))
+
+        self.log.info("Wait for tx from second node")
+        payment_txid = self.nodes[1].sendtoaddress(self.nodes[0].getnewaddress(), 1.0)
+        self.sync_all()
+
+        # Should receive the broadcasted txid.
+        txid = self.hashtx.receive()
+        assert_equal(payment_txid, bytes_to_hex_str(txid))
+
+        # Should receive the broadcasted raw transaction.
+        hex = self.rawtx.receive()
+        assert_equal(payment_txid, bytes_to_hex_str(hash256(hex)))
+
+if __name__ == '__main__':
+    ZMQTest().main()

test/functional/mempool_limit.py

@@ -0,0 +1,54 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test mempool limiting together/eviction with the wallet."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+class MempoolLimitTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+        self.extra_args = [["-maxmempool=5", "-spendzeroconfchange=0"]]
+
+    def run_test(self):
+        txouts = gen_return_txouts()
+        relayfee = self.nodes[0].getnetworkinfo()['relayfee']
+
+        self.log.info('Check that mempoolminfee is minrelytxfee')
+        assert_equal(self.nodes[0].getmempoolinfo()['minrelaytxfee'], Decimal('0.00001000'))
+        assert_equal(self.nodes[0].getmempoolinfo()['mempoolminfee'], Decimal('0.00001000'))
+
+        txids = []
+        utxos = create_confirmed_utxos(relayfee, self.nodes[0], 91)
+
+        self.log.info('Create a mempool tx that will be evicted')
+        us0 = utxos.pop()
+        inputs = [{ "txid" : us0["txid"], "vout" : us0["vout"]}]
+        outputs = {self.nodes[0].getnewaddress() : 0.0001}
+        tx = self.nodes[0].createrawtransaction(inputs, outputs)
+        self.nodes[0].settxfee(relayfee) # specifically fund this tx with low fee
+        txF = self.nodes[0].fundrawtransaction(tx)
+        self.nodes[0].settxfee(0) # return to automatic fee selection
+        txFS = self.nodes[0].signrawtransaction(txF['hex'])
+        txid = self.nodes[0].sendrawtransaction(txFS['hex'])
+
+        relayfee = self.nodes[0].getnetworkinfo()['relayfee']
+        base_fee = relayfee*100
+        for i in range (3):
+            txids.append([])
+            txids[i] = create_lots_of_big_transactions(self.nodes[0], txouts, utxos[30*i:30*i+30], 30, (i+1)*base_fee)
+
+        self.log.info('The tx should be evicted by now')
+        assert(txid not in self.nodes[0].getrawmempool())
+        txdata = self.nodes[0].gettransaction(txid)
+        assert(txdata['confirmations'] ==  0) #confirmation should still be 0
+
+        self.log.info('Check that mempoolminfee is larger than minrelytxfee')
+        assert_equal(self.nodes[0].getmempoolinfo()['minrelaytxfee'], Decimal('0.00001000'))
+        assert_greater_than(self.nodes[0].getmempoolinfo()['mempoolminfee'], Decimal('0.00001000'))
+
+if __name__ == '__main__':
+    MempoolLimitTest().main()

test/functional/mempool_persist.py

@@ -0,0 +1,117 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test mempool persistence.
+
+By default, bitcoind will dump mempool on shutdown and
+then reload it on startup. This can be overridden with
+the -persistmempool=0 command line option.
+
+Test is as follows:
+
+  - start node0, node1 and node2. node1 has -persistmempool=0
+  - create 5 transactions on node2 to its own address. Note that these
+    are not sent to node0 or node1 addresses because we don't want
+    them to be saved in the wallet.
+  - check that node0 and node1 have 5 transactions in their mempools
+  - shutdown all nodes.
+  - startup node0. Verify that it still has 5 transactions
+    in its mempool. Shutdown node0. This tests that by default the
+    mempool is persistent.
+  - startup node1. Verify that its mempool is empty. Shutdown node1.
+    This tests that with -persistmempool=0, the mempool is not
+    dumped to disk when the node is shut down.
+  - Restart node0 with -persistmempool=0. Verify that its mempool is
+    empty. Shutdown node0. This tests that with -persistmempool=0,
+    the mempool is not loaded from disk on start up.
+  - Restart node0 with -persistmempool. Verify that it has 5
+    transactions in its mempool. This tests that -persistmempool=0
+    does not overwrite a previously valid mempool stored on disk.
+  - Remove node0 mempool.dat and verify savemempool RPC recreates it
+    and verify that node1 can load it and has 5 transaction in its
+    mempool.
+  - Verify that savemempool throws when the RPC is called if
+    node1 can't write to disk.
+
+"""
+import os
+import time
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+class MempoolPersistTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 3
+        self.extra_args = [[], ["-persistmempool=0"], []]
+
+    def run_test(self):
+        chain_height = self.nodes[0].getblockcount()
+        assert_equal(chain_height, 200)
+
+        self.log.debug("Mine a single block to get out of IBD")
+        self.nodes[0].generate(1)
+        self.sync_all()
+
+        self.log.debug("Send 5 transactions from node2 (to its own address)")
+        for i in range(5):
+            self.nodes[2].sendtoaddress(self.nodes[2].getnewaddress(), Decimal("10"))
+        node2_balance = self.nodes[2].getbalance()
+        self.sync_all()
+
+        self.log.debug("Verify that node0 and node1 have 5 transactions in their mempools")
+        assert_equal(len(self.nodes[0].getrawmempool()), 5)
+        assert_equal(len(self.nodes[1].getrawmempool()), 5)
+
+        self.log.debug("Stop-start the nodes. Verify that node0 has the transactions in its mempool and node1 does not. Verify that node2 calculates its balance correctly after loading wallet transactions.")
+        self.stop_nodes()
+        self.start_node(1)  # Give this one a head-start, so we can be "extra-sure" that it didn't load anything later
+        self.start_node(0)
+        self.start_node(2)
+        # Give bitcoind a second to reload the mempool
+        wait_until(lambda: len(self.nodes[0].getrawmempool()) == 5, timeout=1)
+        wait_until(lambda: len(self.nodes[2].getrawmempool()) == 5, timeout=1)
+        # The others have loaded their mempool. If node_1 loaded anything, we'd probably notice by now:
+        assert_equal(len(self.nodes[1].getrawmempool()), 0)
+
+        # Verify accounting of mempool transactions after restart is correct
+        self.nodes[2].syncwithvalidationinterfacequeue()  # Flush mempool to wallet
+        assert_equal(node2_balance, self.nodes[2].getbalance())
+
+        self.log.debug("Stop-start node0 with -persistmempool=0. Verify that it doesn't load its mempool.dat file.")
+        self.stop_nodes()
+        self.start_node(0, extra_args=["-persistmempool=0"])
+        # Give bitcoind a second to reload the mempool
+        time.sleep(1)
+        assert_equal(len(self.nodes[0].getrawmempool()), 0)
+
+        self.log.debug("Stop-start node0. Verify that it has the transactions in its mempool.")
+        self.stop_nodes()
+        self.start_node(0)
+        wait_until(lambda: len(self.nodes[0].getrawmempool()) == 5)
+
+        mempooldat0 = os.path.join(self.options.tmpdir, 'node0', 'regtest', 'mempool.dat')
+        mempooldat1 = os.path.join(self.options.tmpdir, 'node1', 'regtest', 'mempool.dat')
+        self.log.debug("Remove the mempool.dat file. Verify that savemempool to disk via RPC re-creates it")
+        os.remove(mempooldat0)
+        self.nodes[0].savemempool()
+        assert os.path.isfile(mempooldat0)
+
+        self.log.debug("Stop nodes, make node1 use mempool.dat from node0. Verify it has 5 transactions")
+        os.rename(mempooldat0, mempooldat1)
+        self.stop_nodes()
+        self.start_node(1, extra_args=[])
+        wait_until(lambda: len(self.nodes[1].getrawmempool()) == 5)
+
+        self.log.debug("Prevent bitcoind from writing mempool.dat to disk. Verify that `savemempool` fails")
+        # to test the exception we are setting bad permissions on a tmp file called mempool.dat.new
+        # which is an implementation detail that could change and break this test
+        mempooldotnew1 = mempooldat1 + '.new'
+        with os.fdopen(os.open(mempooldotnew1, os.O_CREAT, 0o000), 'w'):
+            pass
+        assert_raises_rpc_error(-1, "Unable to dump mempool to disk", self.nodes[1].savemempool)
+        os.remove(mempooldotnew1)
+
+if __name__ == '__main__':
+    MempoolPersistTest().main()

test/functional/mempool_reorg.py

@@ -0,0 +1,96 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test mempool re-org scenarios.
+
+Test re-org scenarios with a mempool that contains transactions
+that spend (directly or indirectly) coinbase transactions.
+"""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+import time
+
+# Create one-input, one-output, no-fee transaction:
+class MempoolCoinbaseTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+        self.extra_args = [["-checkmempool"]] * 2
+
+    alert_filename = None  # Set by setup_network
+
+    def run_test(self):
+        # Start with a 200 block chain
+        assert_equal(self.nodes[0].getblockcount(), 200)
+
+        # Mine four blocks. After this, nodes[0] blocks
+        # 101, 102, and 103 are spend-able.
+        new_blocks = self.nodes[1].generate(4)
+        self.sync_all()
+
+        node0_address = self.nodes[0].getnewaddress()
+        node1_address = self.nodes[1].getnewaddress()
+
+        # Three scenarios for re-orging coinbase spends in the memory pool:
+        # 1. Direct coinbase spend  :  spend_101
+        # 2. Indirect (coinbase spend in chain, child in mempool) : spend_102 and spend_102_1
+        # 3. Indirect (coinbase and child both in chain) : spend_103 and spend_103_1
+        # Use invalidatblock to make all of the above coinbase spends invalid (immature coinbase),
+        # and make sure the mempool code behaves correctly.
+        b = [ self.nodes[0].getblockhash(n) for n in range(101, 105) ]
+        coinbase_txids = [ self.nodes[0].getblock(h)['tx'][0] for h in b ]
+        spend_101_raw = create_tx(self.nodes[0], coinbase_txids[1], node1_address, 249.99)
+        spend_102_raw = create_tx(self.nodes[0], coinbase_txids[2], node0_address, 249.99)
+        spend_103_raw = create_tx(self.nodes[0], coinbase_txids[3], node0_address, 249.99)
+
+        # Create a transaction which is time-locked to two blocks in the future
+        timelock_tx = self.nodes[0].createrawtransaction([{"txid": coinbase_txids[0], "vout": 0}], {node0_address: 249.99})
+        # Set the time lock
+        timelock_tx = timelock_tx.replace("ffffffff", "11111191", 1)
+        timelock_tx = timelock_tx[:-8] + hex(self.nodes[0].getblockcount() + 2)[2:] + "000000"
+        timelock_tx = self.nodes[0].signrawtransaction(timelock_tx)["hex"]
+        # This will raise an exception because the timelock transaction is too immature to spend
+        assert_raises_rpc_error(-26, "non-final", self.nodes[0].sendrawtransaction, timelock_tx)
+
+        # Broadcast and mine spend_102 and 103:
+        spend_102_id = self.nodes[0].sendrawtransaction(spend_102_raw)
+        spend_103_id = self.nodes[0].sendrawtransaction(spend_103_raw)
+        self.nodes[0].generate(1)
+        # Time-locked transaction is still too immature to spend
+        assert_raises_rpc_error(-26,'non-final', self.nodes[0].sendrawtransaction, timelock_tx)
+
+        # Create 102_1 and 103_1:
+        spend_102_1_raw = create_tx(self.nodes[0], spend_102_id, node1_address, 249.98)
+        spend_103_1_raw = create_tx(self.nodes[0], spend_103_id, node1_address, 249.98)
+
+        # Broadcast and mine 103_1:
+        spend_103_1_id = self.nodes[0].sendrawtransaction(spend_103_1_raw)
+        last_block = self.nodes[0].generate(1)
+        # Time-locked transaction can now be spent
+        #timelock_tx_id = self.nodes[0].sendrawtransaction(timelock_tx)
+
+        # ... now put spend_101 and spend_102_1 in memory pools:
+        spend_101_id = self.nodes[0].sendrawtransaction(spend_101_raw)
+        spend_102_1_id = self.nodes[0].sendrawtransaction(spend_102_1_raw)
+
+        self.sync_all()
+
+        assert_equal(set(self.nodes[0].getrawmempool()), {spend_101_id, spend_102_1_id})
+
+        for node in self.nodes:
+            node.invalidateblock(last_block[0])
+        # Time-locked transaction is now too immature and has been removed from the mempool
+        # spend_103_1 has been re-orged out of the chain and is back in the mempool
+        assert_equal(set(self.nodes[0].getrawmempool()), {spend_101_id, spend_102_1_id, spend_103_1_id})
+
+        # Use invalidateblock to re-org back and make all those coinbase spends
+        # immature/invalid:
+        for node in self.nodes:
+            node.invalidateblock(new_blocks[0])
+
+        # mempool should be empty.
+        assert_equal(set(self.nodes[0].getrawmempool()), set())
+
+if __name__ == '__main__':
+    MempoolCoinbaseTest().main()

test/functional/mempool_resurrect.py

@@ -0,0 +1,68 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test resurrection of mined transactions when the blockchain is re-organized."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+# Create one-input, one-output, no-fee transaction:
+class MempoolCoinbaseTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 1
+        self.extra_args = [["-checkmempool"]]
+
+    def run_test(self):
+        node0_address = self.nodes[0].getnewaddress()
+        # Spend block 1/2/3's coinbase transactions
+        # Mine a block.
+        # Create three more transactions, spending the spends
+        # Mine another block.
+        # ... make sure all the transactions are confirmed
+        # Invalidate both blocks
+        # ... make sure all the transactions are put back in the mempool
+        # Mine a new block
+        # ... make sure all the transactions are confirmed again.
+
+        b = [ self.nodes[0].getblockhash(n) for n in range(1, 4) ]
+        coinbase_txids = [ self.nodes[0].getblock(h)['tx'][0] for h in b ]
+        spends1_raw = [ create_tx(self.nodes[0], txid, node0_address, 249.99) for txid in coinbase_txids ]
+        spends1_id = [ self.nodes[0].sendrawtransaction(tx) for tx in spends1_raw ]
+
+        blocks = []
+        blocks.extend(self.nodes[0].generate(1))
+
+        spends2_raw = [ create_tx(self.nodes[0], txid, node0_address, 249.98) for txid in spends1_id ]
+        spends2_id = [ self.nodes[0].sendrawtransaction(tx) for tx in spends2_raw ]
+
+        blocks.extend(self.nodes[0].generate(1))
+
+        # mempool should be empty, all txns confirmed
+        assert_equal(set(self.nodes[0].getrawmempool()), set())
+        for txid in spends1_id+spends2_id:
+            tx = self.nodes[0].gettransaction(txid)
+            assert(tx["confirmations"] > 0)
+
+        # Use invalidateblock to re-org back; all transactions should
+        # end up unconfirmed and back in the mempool
+        for node in self.nodes:
+            node.invalidateblock(blocks[0])
+
+        # mempool should be empty, all txns confirmed
+        assert_equal(set(self.nodes[0].getrawmempool()), set(spends1_id+spends2_id))
+        for txid in spends1_id+spends2_id:
+            tx = self.nodes[0].gettransaction(txid)
+            assert(tx["confirmations"] == 0)
+
+        # Generate another block, they should all get mined
+        self.nodes[0].generate(1)
+        # mempool should be empty, all txns confirmed
+        assert_equal(set(self.nodes[0].getrawmempool()), set())
+        for txid in spends1_id+spends2_id:
+            tx = self.nodes[0].gettransaction(txid)
+            assert(tx["confirmations"] > 0)
+
+
+if __name__ == '__main__':
+    MempoolCoinbaseTest().main()

test/functional/mempool_spend_coinbase.py

@@ -0,0 +1,53 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test spending coinbase transactions.
+
+The coinbase transaction in block N can appear in block
+N+100... so is valid in the mempool when the best block
+height is N+99.
+This test makes sure coinbase spends that will be mature
+in the next block are accepted into the memory pool,
+but less mature coinbase spends are NOT.
+"""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+# Create one-input, one-output, no-fee transaction:
+class MempoolSpendCoinbaseTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 1
+        self.extra_args = [["-checkmempool"]]
+
+    def run_test(self):
+        chain_height = self.nodes[0].getblockcount()
+        assert_equal(chain_height, 200)
+        node0_address = self.nodes[0].getnewaddress()
+
+        # Coinbase at height chain_height-100+1 ok in mempool, should
+        # get mined. Coinbase at height chain_height-100+2 is
+        # is too immature to spend.
+        b = [ self.nodes[0].getblockhash(n) for n in range(101, 103) ]
+        coinbase_txids = [ self.nodes[0].getblock(h)['tx'][0] for h in b ]
+        spends_raw = [ create_tx(self.nodes[0], txid, node0_address, 249.99) for txid in coinbase_txids ]
+
+        spend_101_id = self.nodes[0].sendrawtransaction(spends_raw[0])
+
+        # coinbase at height 102 should be too immature to spend
+        assert_raises_rpc_error(-26,"bad-txns-premature-spend-of-coinbase", self.nodes[0].sendrawtransaction, spends_raw[1])
+
+        # mempool should have just spend_101:
+        assert_equal(self.nodes[0].getrawmempool(), [ spend_101_id ])
+
+        # mine a block, spend_101 should get confirmed
+        self.nodes[0].generate(1)
+        assert_equal(set(self.nodes[0].getrawmempool()), set())
+
+        # ... and now height 102 can be spent:
+        spend_102_id = self.nodes[0].sendrawtransaction(spends_raw[1])
+        assert_equal(self.nodes[0].getrawmempool(), [ spend_102_id ])
+
+if __name__ == '__main__':
+    MempoolSpendCoinbaseTest().main()

test/functional/mining_basic.py

@@ -0,0 +1,134 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test mining RPCs
+
+- getmininginfo
+- getblocktemplate proposal mode
+- submitblock"""
+
+import copy
+from binascii import b2a_hex
+from decimal import Decimal
+
+from test_framework.blocktools import create_coinbase
+from test_framework.mininode import CBlock
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import assert_equal, assert_raises_rpc_error
+
+def b2x(b):
+    return b2a_hex(b).decode('ascii')
+
+def assert_template(node, block, expect, rehash=True):
+    if rehash:
+        block.hashMerkleRoot = block.calc_merkle_root()
+    rsp = node.getblocktemplate({'data': b2x(block.serialize()), 'mode': 'proposal'})
+    assert_equal(rsp, expect)
+
+class MiningTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+        self.setup_clean_chain = False
+
+    def run_test(self):
+        node = self.nodes[0]
+
+        self.log.info('getmininginfo')
+        mining_info = node.getmininginfo()
+        assert_equal(mining_info['blocks'], 200)
+        assert_equal(mining_info['chain'], 'regtest')
+        assert_equal(mining_info['currentblocktx'], 0)
+        assert_equal(mining_info['difficulty'], Decimal('0.000244140625'))
+        assert_equal(mining_info['networkhashps'], Decimal('17476'))
+        assert_equal(mining_info['pooledtx'], 0)
+
+        # Mine a block to leave initial block download
+        node.generate(1)
+        tmpl = node.getblocktemplate()
+        self.log.info("getblocktemplate: Test capability advertised")
+        assert 'proposal' in tmpl['capabilities']
+        assert 'coinbasetxn' not in tmpl
+
+        coinbase_tx = create_coinbase(height=int(tmpl["height"]) + 1)
+        # sequence numbers must not be max for nLockTime to have effect
+        coinbase_tx.vin[0].nSequence = 2 ** 32 - 2
+        coinbase_tx.rehash()
+
+        block = CBlock()
+        block.nVersion = tmpl["version"]
+        block.hashPrevBlock = int(tmpl["previousblockhash"], 16)
+        block.nTime = tmpl["curtime"]
+        block.nBits = int(tmpl["bits"], 16)
+        block.nNonce = 0
+        block.vtx = [coinbase_tx]
+
+        self.log.info("getblocktemplate: Test valid block")
+        assert_template(node, block, None)
+
+        self.log.info("submitblock: Test block decode failure")
+        assert_raises_rpc_error(-22, "Block decode failed", node.submitblock, b2x(block.serialize()[:-15]))
+
+        self.log.info("getblocktemplate: Test bad input hash for coinbase transaction")
+        bad_block = copy.deepcopy(block)
+        bad_block.vtx[0].vin[0].prevout.hash += 1
+        bad_block.vtx[0].rehash()
+        assert_template(node, bad_block, 'bad-cb-missing')
+
+        self.log.info("submitblock: Test invalid coinbase transaction")
+        assert_raises_rpc_error(-22, "Block does not start with a coinbase", node.submitblock, b2x(bad_block.serialize()))
+
+        self.log.info("getblocktemplate: Test truncated final transaction")
+        assert_raises_rpc_error(-22, "Block decode failed", node.getblocktemplate, {'data': b2x(block.serialize()[:-1]), 'mode': 'proposal'})
+
+        self.log.info("getblocktemplate: Test duplicate transaction")
+        bad_block = copy.deepcopy(block)
+        bad_block.vtx.append(bad_block.vtx[0])
+        assert_template(node, bad_block, 'bad-txns-duplicate')
+
+        self.log.info("getblocktemplate: Test invalid transaction")
+        bad_block = copy.deepcopy(block)
+        bad_tx = copy.deepcopy(bad_block.vtx[0])
+        bad_tx.vin[0].prevout.hash = 255
+        bad_tx.rehash()
+        bad_block.vtx.append(bad_tx)
+        assert_template(node, bad_block, 'bad-txns-inputs-missingorspent')
+
+        self.log.info("getblocktemplate: Test nonfinal transaction")
+        bad_block = copy.deepcopy(block)
+        bad_block.vtx[0].nLockTime = 2 ** 32 - 1
+        bad_block.vtx[0].rehash()
+        assert_template(node, bad_block, 'bad-txns-nonfinal')
+
+        self.log.info("getblocktemplate: Test bad tx count")
+        # The tx count is immediately after the block header
+        TX_COUNT_OFFSET = 112
+        bad_block_sn = bytearray(block.serialize())
+        assert_equal(bad_block_sn[TX_COUNT_OFFSET], 1)
+        bad_block_sn[TX_COUNT_OFFSET] += 1
+        assert_raises_rpc_error(-22, "Block decode failed", node.getblocktemplate, {'data': b2x(bad_block_sn), 'mode': 'proposal'})
+
+        self.log.info("getblocktemplate: Test bad bits")
+        bad_block = copy.deepcopy(block)
+        bad_block.nBits = 469762303  # impossible in the real world
+        assert_template(node, bad_block, 'bad-diffbits')
+
+        self.log.info("getblocktemplate: Test bad merkle root")
+        bad_block = copy.deepcopy(block)
+        bad_block.hashMerkleRoot += 1
+        assert_template(node, bad_block, 'bad-txnmrklroot', False)
+
+        self.log.info("getblocktemplate: Test bad timestamps")
+        bad_block = copy.deepcopy(block)
+        bad_block.nTime = 2 ** 31 - 1
+        assert_template(node, bad_block, 'time-too-new')
+        bad_block.nTime = 0
+        assert_template(node, bad_block, 'time-too-old')
+
+        self.log.info("getblocktemplate: Test not best block")
+        bad_block = copy.deepcopy(block)
+        bad_block.hashPrevBlock = 123
+        assert_template(node, bad_block, 'inconclusive-not-best-prevblk')
+
+if __name__ == '__main__':
+    MiningTest().main()

test/functional/mining_getblocktemplate_longpoll.py

@@ -0,0 +1,71 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test longpolling with getblocktemplate."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+import threading
+
+class LongpollThread(threading.Thread):
+    def __init__(self, node):
+        threading.Thread.__init__(self)
+        # query current longpollid
+        templat = node.getblocktemplate()
+        self.longpollid = templat['longpollid']
+        # create a new connection to the node, we can't use the same
+        # connection from two threads
+        self.node = get_rpc_proxy(node.url, 1, timeout=600, coveragedir=node.coverage_dir)
+
+    def run(self):
+        self.node.getblocktemplate({'longpollid':int(self.longpollid)})
+
+class GetBlockTemplateLPTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+
+    def run_test(self):
+        self.log.info("Warning: this test will take about 70 seconds in the best case. Be patient.")
+        self.nodes[0].generate(10)
+        templat = self.nodes[0].getblocktemplate()
+        longpollid = templat['longpollid']
+        # longpollid should not change between successive invocations if nothing else happens
+        templat2 = self.nodes[0].getblocktemplate()
+        assert(templat2['longpollid'] == longpollid)
+
+        # Test 1: test that the longpolling wait if we do nothing
+        thr = LongpollThread(self.nodes[0])
+        thr.start()
+        # check that thread still lives
+        thr.join(5)  # wait 5 seconds or until thread exits
+        assert(thr.is_alive())
+
+        # Test 2: test that longpoll will terminate if another node generates a block
+        self.nodes[1].generate(1)  # generate a block on another node
+        # check that thread will exit now that new transaction entered mempool
+        thr.join(5)  # wait 5 seconds or until thread exits
+        assert(not thr.is_alive())
+
+        # Test 3: test that longpoll will terminate if we generate a block ourselves
+        thr = LongpollThread(self.nodes[0])
+        thr.start()
+        self.nodes[0].generate(1)  # generate a block on another node
+        thr.join(5)  # wait 5 seconds or until thread exits
+        assert(not thr.is_alive())
+
+        # Test 4: test that introducing a new transaction into the mempool will terminate the longpoll
+        thr = LongpollThread(self.nodes[0])
+        thr.start()
+        # generate a random transaction and submit it
+        min_relay_fee = self.nodes[0].getnetworkinfo()["relayfee"]
+        # min_relay_fee is fee per 1000 bytes, which should be more than enough.
+        (txid, txhex, fee) = random_transaction(self.nodes, Decimal("1.1"), min_relay_fee, Decimal("0.001"), 20)
+        # after one minute, every 10 seconds the mempool is probed, so in 80 seconds it should have returned
+        thr.join(60 + 20)
+        assert(not thr.is_alive())
+
+if __name__ == '__main__':
+    GetBlockTemplateLPTest().main()
+

test/functional/mining_prioritisetransaction.py

@@ -0,0 +1,147 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the prioritisetransaction mining RPC."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+from test_framework.mininode import COIN, MAX_BLOCK_BASE_SIZE
+
+class PrioritiseTransactionTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 2
+        self.extra_args = [["-printpriority=1"], ["-printpriority=1"]]
+
+    def run_test(self):
+        # Test `prioritisetransaction` required parameters
+        assert_raises_rpc_error(-1, "prioritisetransaction", self.nodes[0].prioritisetransaction)
+        assert_raises_rpc_error(-1, "prioritisetransaction", self.nodes[0].prioritisetransaction, '')
+        assert_raises_rpc_error(-1, "prioritisetransaction", self.nodes[0].prioritisetransaction, '', 0)
+
+        # Test `prioritisetransaction` invalid extra parameters
+        assert_raises_rpc_error(-1, "prioritisetransaction", self.nodes[0].prioritisetransaction, '', 0, 0, 0)
+
+        # Test `prioritisetransaction` invalid `txid`
+        assert_raises_rpc_error(-1, "txid must be hexadecimal string", self.nodes[0].prioritisetransaction, 'foo', 0, 0)
+
+        # Test `prioritisetransaction` invalid `dummy`
+        txid = '1d1d4e24ed99057e84c3f80fd8fbec79ed9e1acee37da269356ecea000000000'
+        assert_raises_rpc_error(-1, "JSON value is not a number as expected", self.nodes[0].prioritisetransaction, txid, 'foo', 0)
+        #assert_raises_rpc_error(-8, "Priority is no longer supported, dummy argument to prioritisetransaction must be 0.", self.nodes[0].prioritisetransaction, txid, 1, 0)
+
+        # Test `prioritisetransaction` invalid `fee_delta`
+        assert_raises_rpc_error(-1, "JSON value is not an integer as expected", self.nodes[0].prioritisetransaction, txid, 0, 'foo')
+
+        self.txouts = gen_return_txouts()
+        self.relayfee = self.nodes[0].getnetworkinfo()['relayfee']
+
+        utxo_count = 90
+        utxos = create_confirmed_utxos(self.relayfee, self.nodes[0], utxo_count)
+        base_fee = self.relayfee*100 # our transactions are smaller than 100kb
+        txids = []
+
+        # Create 3 batches of transactions at 3 different fee rate levels
+        range_size = utxo_count // 3
+        for i in range(3):
+            txids.append([])
+            start_range = i * range_size
+            end_range = start_range + range_size
+            txids[i] = create_lots_of_big_transactions(self.nodes[0], self.txouts, utxos[start_range:end_range], end_range - start_range, (i+1)*base_fee)
+
+        # Make sure that the size of each group of transactions exceeds
+        # MAX_BLOCK_BASE_SIZE -- otherwise the test needs to be revised to create
+        # more transactions.
+        mempool = self.nodes[0].getrawmempool(True)
+        sizes = [0, 0, 0]
+        for i in range(3):
+            for j in txids[i]:
+                assert(j in mempool)
+                sizes[i] += mempool[j]['size']
+            assert(sizes[i] > MAX_BLOCK_BASE_SIZE) # Fail => raise utxo_count
+
+        # add a fee delta to something in the cheapest bucket and make sure it gets mined
+        # also check that a different entry in the cheapest bucket is NOT mined
+        self.nodes[0].prioritisetransaction(txids[0][0], 0, int(3*base_fee*COIN))
+
+        self.nodes[0].generate(1)
+
+        mempool = self.nodes[0].getrawmempool()
+        self.log.info("Assert that prioritised transaction was mined")
+        assert(txids[0][0] not in mempool)
+        assert(txids[0][1] in mempool)
+
+        high_fee_tx = None
+        for x in txids[2]:
+            if x not in mempool:
+                high_fee_tx = x
+
+        # Something high-fee should have been mined!
+        assert(high_fee_tx != None)
+
+        # Add a prioritisation before a tx is in the mempool (de-prioritising a
+        # high-fee transaction so that it's now low fee).
+        self.nodes[0].prioritisetransaction(high_fee_tx, 0, -int(2*base_fee*COIN))
+
+        # Add everything back to mempool
+        self.nodes[0].invalidateblock(self.nodes[0].getbestblockhash())
+
+        # Check to make sure our high fee rate tx is back in the mempool
+        mempool = self.nodes[0].getrawmempool()
+        assert(high_fee_tx in mempool)
+
+        # Now verify the modified-high feerate transaction isn't mined before
+        # the other high fee transactions. Keep mining until our mempool has
+        # decreased by all the high fee size that we calculated above.
+        while (self.nodes[0].getmempoolinfo()['bytes'] > sizes[0] + sizes[1]):
+            self.nodes[0].generate(1)
+
+        # High fee transaction should not have been mined, but other high fee rate
+        # transactions should have been.
+        mempool = self.nodes[0].getrawmempool()
+        self.log.info("Assert that de-prioritised transaction is still in mempool")
+        assert(high_fee_tx in mempool)
+        for x in txids[2]:
+            if (x != high_fee_tx):
+                assert(x not in mempool)
+
+        # Create a free transaction.  Should be rejected.
+        utxo_list = self.nodes[0].listunspent()
+        assert(len(utxo_list) > 0)
+        utxo = utxo_list[0]
+
+        inputs = []
+        outputs = {}
+        inputs.append({"txid" : utxo["txid"], "vout" : utxo["vout"]})
+        outputs[self.nodes[0].getnewaddress()] = float(utxo["amount"])
+        raw_tx = self.nodes[0].createrawtransaction(inputs, outputs)
+        tx_hex = self.nodes[0].signrawtransaction(raw_tx)["hex"]
+        tx_id = self.nodes[0].decoderawtransaction(tx_hex)["txid"]
+
+        # This will raise an exception due to min relay fee not being met
+        #assert_raises_rpc_error(-26, "66: min relay fee not met", self.nodes[0].sendrawtransaction, tx_hex)
+        #assert(tx_id not in self.nodes[0].getrawmempool())
+
+        # This is a less than 1000-byte transaction, so just set the fee
+        # to be the minimum for a 1000 byte transaction and check that it is
+        # accepted.
+        self.nodes[0].prioritisetransaction(tx_id, 0, int(self.relayfee*COIN*2))
+
+        self.log.info("Assert that prioritised free transaction is accepted to mempool")
+        assert_equal(self.nodes[0].sendrawtransaction(tx_hex), tx_id)
+        assert(tx_id in self.nodes[0].getrawmempool())
+
+        # Test that calling prioritisetransaction is sufficient to trigger
+        # getblocktemplate to (eventually) return a new block.
+        mock_time = int(time.time())
+        self.nodes[0].setmocktime(mock_time)
+        template = self.nodes[0].getblocktemplate()
+        self.nodes[0].prioritisetransaction(tx_id, 0, -int(self.relayfee*COIN))
+        self.nodes[0].setmocktime(mock_time+10)
+        new_template = self.nodes[0].getblocktemplate()
+
+        assert(template != new_template)
+
+if __name__ == '__main__':
+    PrioritiseTransactionTest().main()

test/functional/p2p_disconnect_ban.py

@@ -0,0 +1,107 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test node disconnect and ban behavior"""
+import time
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (
+    assert_equal,
+    assert_raises_rpc_error,
+    connect_nodes_bi,
+    wait_until,
+)
+
+class DisconnectBanTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+
+    def run_test(self):
+        self.log.info("Test setban and listbanned RPCs")
+
+        self.log.info("setban: successfully ban single IP address")
+        assert_equal(len(self.nodes[1].getpeerinfo()), 2)  # node1 should have 2 connections to node0 at this point
+        self.nodes[1].setban("127.0.0.1", "add")
+        wait_until(lambda: len(self.nodes[1].getpeerinfo()) == 0, timeout=10)
+        assert_equal(len(self.nodes[1].getpeerinfo()), 0)  # all nodes must be disconnected at this point
+        assert_equal(len(self.nodes[1].listbanned()), 1)
+
+        self.log.info("clearbanned: successfully clear ban list")
+        self.nodes[1].clearbanned()
+        assert_equal(len(self.nodes[1].listbanned()), 0)
+        self.nodes[1].setban("127.0.0.0/24", "add")
+
+        self.log.info("setban: fail to ban an already banned subnet")
+        assert_equal(len(self.nodes[1].listbanned()), 1)
+        assert_raises_rpc_error(-23, "IP/Subnet already banned", self.nodes[1].setban, "127.0.0.1", "add")
+
+        self.log.info("setban: fail to ban an invalid subnet")
+        assert_raises_rpc_error(-23, "Error: Invalid IP/Subnet", self.nodes[1].setban, "127.0.0.1/42", "add")
+        assert_equal(len(self.nodes[1].listbanned()), 1)  # still only one banned ip because 127.0.0.1 is within the range of 127.0.0.0/24
+
+        self.log.info("setban remove: fail to unban a non-banned subnet")
+        assert_raises_rpc_error(-1, "Error: Unban failed", self.nodes[1].setban, "127.0.0.1", "remove")
+        assert_equal(len(self.nodes[1].listbanned()), 1)
+
+        self.log.info("setban remove: successfully unban subnet")
+        self.nodes[1].setban("127.0.0.0/24", "remove")
+        assert_equal(len(self.nodes[1].listbanned()), 0)
+        self.nodes[1].clearbanned()
+        assert_equal(len(self.nodes[1].listbanned()), 0)
+
+        self.log.info("setban: test persistence across node restart")
+        self.nodes[1].setban("127.0.0.0/32", "add")
+        self.nodes[1].setban("127.0.0.0/24", "add")
+        # Set the mocktime so we can control when bans expire
+        old_time = int(time.time())
+        self.nodes[1].setmocktime(old_time)
+        self.nodes[1].setban("192.168.0.1", "add", 1)  # ban for 1 seconds
+        self.nodes[1].setban("2001:4d48:ac57:400:cacf:e9ff:fe1d:9c63/19", "add", 1000)  # ban for 1000 seconds
+        listBeforeShutdown = self.nodes[1].listbanned()
+        assert_equal("192.168.0.1/32", listBeforeShutdown[2]['address'])
+        # Move time forward by 3 seconds so the third ban has expired
+        self.nodes[1].setmocktime(old_time + 3)
+        assert_equal(len(self.nodes[1].listbanned()), 4)
+
+        self.stop_node(1)
+        self.start_node(1)
+
+        listAfterShutdown = self.nodes[1].listbanned()
+        assert_equal("127.0.0.0/24", listAfterShutdown[0]['address'])
+        assert_equal("127.0.0.0/32", listAfterShutdown[1]['address'])
+        assert_equal("/19" in listAfterShutdown[2]['address'], True)
+
+        # Clear ban lists
+        self.nodes[1].clearbanned()
+        connect_nodes_bi(self.nodes, 0, 1)
+
+        self.log.info("Test disconnectnode RPCs")
+
+        #self.log.info("disconnectnode: fail to disconnect when calling with address and nodeid")
+        #address1 = self.nodes[0].getpeerinfo()[0]['addr']
+        #node1 = self.nodes[0].getpeerinfo()[0]['addr']
+        #assert_raises_rpc_error(-32602, "Only one of address and nodeid should be provided.", self.nodes[0].disconnectnode, address=address1, nodeid=node1)
+
+        self.log.info("disconnectnode: fail to disconnect when calling with junk address")
+        assert_raises_rpc_error(-29, "Node not found in connected nodes", self.nodes[0].disconnectnode, "221B Baker Street")
+
+        self.log.info("disconnectnode: successfully disconnect node by address")
+        address1 = self.nodes[0].getpeerinfo()[0]['addr']
+        self.nodes[0].disconnectnode(address1)
+        wait_until(lambda: len(self.nodes[0].getpeerinfo()) == 1, timeout=10)
+        assert not [node for node in self.nodes[0].getpeerinfo() if node['addr'] == address1]
+
+        self.log.info("disconnectnode: successfully reconnect node")
+        connect_nodes_bi(self.nodes, 0, 1)  # reconnect the node
+        assert_equal(len(self.nodes[0].getpeerinfo()), 2)
+        assert [node for node in self.nodes[0].getpeerinfo() if node['addr'] == address1]
+
+        #self.log.info("disconnectnode: successfully disconnect node by node id")
+        #id1 = self.nodes[0].getpeerinfo()[0]['id']
+        #self.nodes[0].disconnectnode(nodeid=id1)
+        #wait_until(lambda: len(self.nodes[0].getpeerinfo()) == 1, timeout=10)
+        #assert not [node for node in self.nodes[0].getpeerinfo() if node['id'] == id1]
+
+if __name__ == '__main__':
+    DisconnectBanTest().main()

test/functional/p2p_feefilter.py

@@ -0,0 +1,93 @@
+#!/usr/bin/env python3
+# Copyright (c) 2016-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test processing of feefilter messages."""
+
+from test_framework.mininode import *
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+import time
+
+
+def hashToHex(hash):
+    return format(hash, '064x')
+
+# Wait up to 60 secs to see if the testnode has received all the expected invs
+def allInvsMatch(invsExpected, testnode):
+    for x in range(60):
+        with mininode_lock:
+            if (sorted(invsExpected) == sorted(testnode.txinvs)):
+                return True
+        time.sleep(1)
+    return False
+
+class TestNode(P2PInterface):
+    def __init__(self):
+        super().__init__()
+        self.txinvs = []
+
+    def on_inv(self, message):
+        for i in message.inv:
+            if (i.type == 1):
+                self.txinvs.append(hashToHex(i.hash))
+
+    def clear_invs(self):
+        with mininode_lock:
+            self.txinvs = []
+
+class FeeFilterTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+
+    def run_test(self):
+        node1 = self.nodes[1]
+        node0 = self.nodes[0]
+        # Get out of IBD
+        node1.generate(1)
+        sync_blocks(self.nodes)
+
+        # Setup the p2p connections and start up the network thread.
+        self.nodes[0].add_p2p_connection(TestNode())
+        network_thread_start()
+        self.nodes[0].p2p.wait_for_verack()
+
+        # Test that invs are received for all txs at feerate of 20 sat/byte
+        node1.settxfee(float(0.00020000))
+        txids = [node1.sendtoaddress(node1.getnewaddress(), 1) for x in range(3)]
+        assert(allInvsMatch(txids, self.nodes[0].p2p))
+        self.nodes[0].p2p.clear_invs()
+
+        # Set a filter of 15 sat/byte
+        self.nodes[0].p2p.send_and_ping(msg_feefilter(15000))
+
+        # Test that txs are still being received (paying 20 sat/byte)
+        txids = [node1.sendtoaddress(node1.getnewaddress(), 1) for x in range(3)]
+        assert(allInvsMatch(txids, self.nodes[0].p2p))
+        self.nodes[0].p2p.clear_invs()
+
+        # Change tx fee rate to 10 sat/byte and test they are no longer received
+        node1.settxfee(float(0.00010000))
+        [node1.sendtoaddress(node1.getnewaddress(), 1) for x in range(3)]
+        sync_mempools(self.nodes) # must be sure node 0 has received all txs
+
+        # Send one transaction from node0 that should be received, so that we
+        # we can sync the test on receipt (if node1's txs were relayed, they'd
+        # be received by the time this node0 tx is received). This is
+        # unfortunately reliant on the current relay behavior where we batch up
+        # to 35 entries in an inv, which means that when this next transaction
+        # is eligible for relay, the prior transactions from node1 are eligible
+        # as well.
+        node0.settxfee(float(0.00020000))
+        txids = [node0.sendtoaddress(node0.getnewaddress(), 1)]
+        assert(allInvsMatch(txids, self.nodes[0].p2p))
+        self.nodes[0].p2p.clear_invs()
+
+        # Remove fee filter and check that txs are received again
+        self.nodes[0].p2p.send_and_ping(msg_feefilter(0))
+        txids = [node1.sendtoaddress(node1.getnewaddress(), 1) for x in range(3)]
+        assert(allInvsMatch(txids, self.nodes[0].p2p))
+        self.nodes[0].p2p.clear_invs()
+
+if __name__ == '__main__':
+    FeeFilterTest().main()

test/functional/p2p_fingerprint.py

@@ -0,0 +1,152 @@
+#!/usr/bin/env python3
+# Copyright (c) 2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test various fingerprinting protections.
+
+If an stale block more than a month old or its header are requested by a peer,
+the node should pretend that it does not have it to avoid fingerprinting.
+"""
+
+import time
+
+from test_framework.blocktools import (create_block, create_coinbase)
+from test_framework.mininode import (
+    CInv,
+    P2PInterface,
+    msg_headers,
+    msg_block,
+    msg_getdata,
+    msg_getheaders,
+    network_thread_start,
+    wait_until,
+)
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (
+    assert_equal,
+)
+
+class P2PFingerprintTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+
+    # Build a chain of blocks on top of given one
+    def build_chain(self, nblocks, prev_hash, prev_height, prev_median_time):
+        blocks = []
+        for _ in range(nblocks):
+            coinbase = create_coinbase(prev_height + 1)
+            block_time = prev_median_time + 1
+            block = create_block(int(prev_hash, 16), coinbase, block_time)
+            block.solve()
+
+            blocks.append(block)
+            prev_hash = block.hash
+            prev_height += 1
+            prev_median_time = block_time
+        return blocks
+
+    # Send a getdata request for a given block hash
+    def send_block_request(self, block_hash, node):
+        msg = msg_getdata()
+        msg.inv.append(CInv(2, block_hash))  # 2 == "Block"
+        node.send_message(msg)
+
+    # Send a getheaders request for a given single block hash
+    def send_header_request(self, block_hash, node):
+        msg = msg_getheaders()
+        msg.hashstop = block_hash
+        node.send_message(msg)
+
+    # Check whether last block received from node has a given hash
+    def last_block_equals(self, expected_hash, node):
+        block_msg = node.last_message.get("block")
+        return block_msg and block_msg.block.rehash() == expected_hash
+
+    # Check whether last block header received from node has a given hash
+    def last_header_equals(self, expected_hash, node):
+        headers_msg = node.last_message.get("headers")
+        return (headers_msg and
+                headers_msg.headers and
+                headers_msg.headers[0].rehash() == expected_hash)
+
+    # Checks that stale blocks timestamped more than a month ago are not served
+    # by the node while recent stale blocks and old active chain blocks are.
+    # This does not currently test that stale blocks timestamped within the
+    # last month but that have over a month's worth of work are also withheld.
+    def run_test(self):
+        node0 = self.nodes[0].add_p2p_connection(P2PInterface())
+
+        network_thread_start()
+        node0.wait_for_verack()
+
+        # Set node time to 60 days ago
+        self.nodes[0].setmocktime(int(time.time()) - 60 * 24 * 60 * 60)
+
+        # Generating a chain of 10 blocks
+        block_hashes = self.nodes[0].generate(10)
+
+        # Create longer chain starting 2 blocks before current tip
+        height = len(block_hashes) - 2
+        block_hash = block_hashes[height - 1]
+        block_time = self.nodes[0].getblockheader(block_hash)["mediantime"] + 1
+        new_blocks = self.build_chain(5, block_hash, height, block_time)
+
+        # Force reorg to a longer chain
+        node0.send_message(msg_headers(new_blocks))
+        node0.wait_for_getdata()
+        for block in new_blocks:
+            node0.send_and_ping(msg_block(block))
+
+        # Check that reorg succeeded
+        assert_equal(self.nodes[0].getblockcount(), 13)
+
+        stale_hash = int(block_hashes[-1], 16)
+
+        # Check that getdata request for stale block succeeds
+        self.send_block_request(stale_hash, node0)
+        test_function = lambda: self.last_block_equals(stale_hash, node0)
+        wait_until(test_function, timeout=3)
+
+        # Check that getheader request for stale block header succeeds
+        self.send_header_request(stale_hash, node0)
+        test_function = lambda: self.last_header_equals(stale_hash, node0)
+        wait_until(test_function, timeout=3)
+
+        # Longest chain is extended so stale is much older than chain tip
+        self.nodes[0].setmocktime(0)
+        tip = self.nodes[0].generate(nblocks=1)[0]
+        assert_equal(self.nodes[0].getblockcount(), 14)
+
+        # Send getdata & getheaders to refresh last received getheader message
+        block_hash = int(tip, 16)
+        self.send_block_request(block_hash, node0)
+        self.send_header_request(block_hash, node0)
+        node0.sync_with_ping()
+
+        # Request for very old stale block should now fail
+        self.send_block_request(stale_hash, node0)
+        time.sleep(3)
+        assert not self.last_block_equals(stale_hash, node0)
+
+        # Request for very old stale block header should now fail
+        self.send_header_request(stale_hash, node0)
+        time.sleep(3)
+        assert not self.last_header_equals(stale_hash, node0)
+
+        # Verify we can fetch very old blocks and headers on the active chain
+        block_hash = int(block_hashes[2], 16)
+        self.send_block_request(block_hash, node0)
+        self.send_header_request(block_hash, node0)
+        node0.sync_with_ping()
+
+        self.send_block_request(block_hash, node0)
+        test_function = lambda: self.last_block_equals(block_hash, node0)
+        wait_until(test_function, timeout=3)
+
+        self.send_header_request(block_hash, node0)
+        test_function = lambda: self.last_header_equals(block_hash, node0)
+        wait_until(test_function, timeout=3)
+
+if __name__ == '__main__':
+    P2PFingerprintTest().main()

test/functional/p2p_invalid_block.py

@@ -0,0 +1,119 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2016 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+from test_framework.test_framework import ComparisonTestFramework
+from test_framework.util import *
+from test_framework.comptool import TestManager, TestInstance, RejectResult
+from test_framework.blocktools import *
+import copy
+import time
+
+
+'''
+In this test we connect to one node over p2p, and test block requests:
+1) Valid blocks should be requested and become chain tip.
+2) Invalid block with duplicated transaction should be re-requested.
+3) Invalid block with bad coinbase value should be rejected and not
+re-requested.
+'''
+
+# Use the ComparisonTestFramework with 1 node: only use --testbinary.
+class InvalidBlockRequestTest(ComparisonTestFramework):
+
+    '''
+    Can either run this test as 1 node with expected answers, or two and compare them.
+    Change the "outcome" variable from each TestInstance object to only do the comparison.
+    '''
+    def __init__(self):
+        super().__init__()
+        self.num_nodes = 1
+
+    def run_test(self):
+        test = TestManager(self, self.options.tmpdir)
+        test.add_all_connections(self.nodes)
+        self.tip = None
+        self.block_time = None
+        NetworkThread().start() # Start up network handling in another thread
+        test.run()
+
+    def get_tests(self):
+        if self.tip is None:
+            self.tip = int("0x" + self.nodes[0].getbestblockhash(), 0)
+        self.block_time = int(time.time())+1
+
+        '''
+        Create a new block with an anyone-can-spend coinbase
+        '''
+        height = 1
+        block = create_block(self.tip, create_coinbase(height), self.block_time)
+        self.block_time += 1
+        block.solve()
+        # Save the coinbase for later
+        self.block1 = block
+        self.tip = block.sha256
+        height += 1
+        yield TestInstance([[block, True]])
+
+        '''
+        Now we need that block to mature so we can spend the coinbase.
+        '''
+        test = TestInstance(sync_every_block=False)
+        for i in range(100):
+            block = create_block(self.tip, create_coinbase(height), self.block_time)
+            block.solve()
+            self.tip = block.sha256
+            self.block_time += 1
+            test.blocks_and_transactions.append([block, True])
+            height += 1
+        yield test
+
+        '''
+        Now we use merkle-root malleability to generate an invalid block with
+        same blockheader.
+        Manufacture a block with 3 transactions (coinbase, spend of prior
+        coinbase, spend of that spend).  Duplicate the 3rd transaction to
+        leave merkle root and blockheader unchanged but invalidate the block.
+        '''
+        block2 = create_block(self.tip, create_coinbase(height), self.block_time)
+        self.block_time += 1
+
+        # b'0x51' is OP_TRUE
+        tx1 = create_transaction(self.block1.vtx[0], 0, b'\x51', 50 * COIN)
+        tx2 = create_transaction(tx1, 0, b'\x51', 50 * COIN)
+
+        block2.vtx.extend([tx1, tx2])
+        block2.hashMerkleRoot = block2.calc_merkle_root()
+        block2.rehash()
+        block2.solve()
+        orig_hash = block2.sha256
+        block2_orig = copy.deepcopy(block2)
+
+        # Mutate block 2
+        block2.vtx.append(tx2)
+        assert_equal(block2.hashMerkleRoot, block2.calc_merkle_root())
+        assert_equal(orig_hash, block2.rehash())
+        assert(block2_orig.vtx != block2.vtx)
+
+        self.tip = block2.sha256
+        yield TestInstance([[block2, RejectResult(16, b'bad-txns-duplicate')], [block2_orig, True]])
+        height += 1
+
+        '''
+        Make sure that a totally screwed up block is not valid.
+        '''
+        block3 = create_block(self.tip, create_coinbase(height), self.block_time)
+        self.block_time += 1
+        block3.vtx[0].vout[0].nValue = 100 * COIN # Too high!
+        block3.vtx[0].sha256=None
+        block3.vtx[0].calc_sha256()
+        block3.hashMerkleRoot = block3.calc_merkle_root()
+        block3.rehash()
+        block3.solve()
+
+        yield TestInstance([[block3, RejectResult(16, b'bad-cb-amount')]])
+
+
+if __name__ == '__main__':
+    InvalidBlockRequestTest().main()

test/functional/p2p_invalid_tx.py

@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2016 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+from test_framework.test_framework import ComparisonTestFramework
+from test_framework.comptool import TestManager, TestInstance, RejectResult
+from test_framework.blocktools import *
+import time
+
+
+'''
+In this test we connect to one node over p2p, and test tx requests.
+'''
+
+# Use the ComparisonTestFramework with 1 node: only use --testbinary.
+class InvalidTxRequestTest(ComparisonTestFramework):
+
+    '''
+    Can either run this test as 1 node with expected answers, or two and compare them.
+    Change the "outcome" variable from each TestInstance object to only do the comparison.
+    '''
+    def __init__(self):
+        super().__init__()
+        self.num_nodes = 1
+
+    def run_test(self):
+        test = TestManager(self, self.options.tmpdir)
+        test.add_all_connections(self.nodes)
+        self.tip = None
+        self.block_time = None
+        NetworkThread().start() # Start up network handling in another thread
+        test.run()
+
+    def get_tests(self):
+        if self.tip is None:
+            self.tip = int("0x" + self.nodes[0].getbestblockhash(), 0)
+        self.block_time = int(time.time())+1
+
+        '''
+        Create a new block with an anyone-can-spend coinbase
+        '''
+        height = 1
+        block = create_block(self.tip, create_coinbase(height), self.block_time)
+        self.block_time += 1
+        block.solve()
+        # Save the coinbase for later
+        self.block1 = block
+        self.tip = block.sha256
+        height += 1
+        yield TestInstance([[block, True]])
+
+        '''
+        Now we need that block to mature so we can spend the coinbase.
+        '''
+        test = TestInstance(sync_every_block=False)
+        for i in range(100):
+            block = create_block(self.tip, create_coinbase(height), self.block_time)
+            block.solve()
+            self.tip = block.sha256
+            self.block_time += 1
+            test.blocks_and_transactions.append([block, True])
+            height += 1
+        yield test
+
+        # b'\x64' is OP_NOTIF
+        # Transaction will be rejected with code 16 (REJECT_INVALID)
+        tx1 = create_transaction(self.block1.vtx[0], 0, b'\x64', 50 * COIN - 12000)
+        yield TestInstance([[tx1, RejectResult(16, b'mandatory-script-verify-flag-failed')]])
+
+        # TODO: test further transactions...
+
+if __name__ == '__main__':
+    InvalidTxRequestTest().main()

test/functional/p2p_leak.py

@@ -0,0 +1,153 @@
+#!/usr/bin/env python3
+# Copyright (c) 2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test message sending before handshake completion.
+
+A node should never send anything other than VERSION/VERACK/REJECT until it's
+received a VERACK.
+
+This test connects to a node and sends it a few messages, trying to intice it
+into sending us something it shouldn't.
+
+Also test that nodes that send unsupported service bits to bitcoind are disconnected
+and don't receive a VERACK. Unsupported service bits are currently 1 << 5 and
+1 << 7 (until August 1st 2018)."""
+
+from test_framework.mininode import *
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+banscore = 10
+
+class CLazyNode(P2PInterface):
+    def __init__(self):
+        super().__init__()
+        self.unexpected_msg = False
+        self.ever_connected = False
+
+    def bad_message(self, message):
+        self.unexpected_msg = True
+        self.log.info("should not have received message: %s" % message.command)
+
+    def on_open(self):
+        self.ever_connected = True
+
+    def on_version(self, message): self.bad_message(message)
+    def on_verack(self, message): self.bad_message(message)
+    def on_reject(self, message): self.bad_message(message)
+    def on_inv(self, message): self.bad_message(message)
+    def on_addr(self, message): self.bad_message(message)
+    def on_getdata(self, message): self.bad_message(message)
+    def on_getblocks(self, message): self.bad_message(message)
+    def on_tx(self, message): self.bad_message(message)
+    def on_block(self, message): self.bad_message(message)
+    def on_getaddr(self, message): self.bad_message(message)
+    def on_headers(self, message): self.bad_message(message)
+    def on_getheaders(self, message): self.bad_message(message)
+    def on_ping(self, message): self.bad_message(message)
+    def on_mempool(self, message): self.bad_message(message)
+    def on_pong(self, message): self.bad_message(message)
+    def on_feefilter(self, message): self.bad_message(message)
+    def on_sendheaders(self, message): self.bad_message(message)
+    def on_sendcmpct(self, message): self.bad_message(message)
+    def on_cmpctblock(self, message): self.bad_message(message)
+    def on_getblocktxn(self, message): self.bad_message(message)
+    def on_blocktxn(self, message): self.bad_message(message)
+
+# Node that never sends a version. We'll use this to send a bunch of messages
+# anyway, and eventually get disconnected.
+class CNodeNoVersionBan(CLazyNode):
+    # send a bunch of veracks without sending a message. This should get us disconnected.
+    # NOTE: implementation-specific check here. Remove if bitcoind ban behavior changes
+    def on_open(self):
+        super().on_open()
+        for i in range(banscore):
+            self.send_message(msg_verack())
+
+    def on_reject(self, message): pass
+
+# Node that never sends a version. This one just sits idle and hopes to receive
+# any message (it shouldn't!)
+class CNodeNoVersionIdle(CLazyNode):
+    def __init__(self):
+        super().__init__()
+
+# Node that sends a version but not a verack.
+class CNodeNoVerackIdle(CLazyNode):
+    def __init__(self):
+        self.version_received = False
+        super().__init__()
+
+    def on_reject(self, message): pass
+    def on_verack(self, message): pass
+    # When version is received, don't reply with a verack. Instead, see if the
+    # node will give us a message that it shouldn't. This is not an exhaustive
+    # list!
+    def on_version(self, message):
+        self.version_received = True
+        self.send_message(msg_ping())
+        self.send_message(msg_getaddr())
+
+class P2PLeakTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 1
+        self.extra_args = [['-banscore='+str(banscore)]]
+
+    def run_test(self):
+        self.nodes[0].setmocktime(1501545600)  # August 1st 2017
+
+        no_version_bannode = self.nodes[0].add_p2p_connection(CNodeNoVersionBan(), send_version=False)
+        no_version_idlenode = self.nodes[0].add_p2p_connection(CNodeNoVersionIdle(), send_version=False)
+        no_verack_idlenode = self.nodes[0].add_p2p_connection(CNodeNoVerackIdle())
+        unsupported_service_bit5_node = self.nodes[0].add_p2p_connection(CLazyNode(), services=NODE_NETWORK)
+        unsupported_service_bit7_node = self.nodes[0].add_p2p_connection(CLazyNode(), services=NODE_NETWORK)
+
+        network_thread_start()
+
+        wait_until(lambda: no_version_bannode.ever_connected, timeout=10, lock=mininode_lock)
+        wait_until(lambda: no_version_idlenode.ever_connected, timeout=10, lock=mininode_lock)
+        wait_until(lambda: no_verack_idlenode.version_received, timeout=10, lock=mininode_lock)
+        wait_until(lambda: unsupported_service_bit5_node.ever_connected, timeout=10, lock=mininode_lock)
+        wait_until(lambda: unsupported_service_bit7_node.ever_connected, timeout=10, lock=mininode_lock)
+
+        # Mine a block and make sure that it's not sent to the connected nodes
+        self.nodes[0].generate(1)
+
+        #Give the node enough time to possibly leak out a message
+        time.sleep(5)
+
+        #This node should have been banned
+        assert no_version_bannode.state != "connected"
+
+        # These nodes should have been disconnected
+        assert unsupported_service_bit5_node.state != "connected"
+        assert unsupported_service_bit7_node.state != "connected"
+
+        self.nodes[0].disconnect_p2ps()
+
+        # Wait until all connections are closed and the network thread has terminated
+        wait_until(lambda: len(self.nodes[0].getpeerinfo()) == 0)
+        network_thread_join()
+
+        # Make sure no unexpected messages came in
+        assert(no_version_bannode.unexpected_msg == False)
+        assert(no_version_idlenode.unexpected_msg == False)
+        assert(no_verack_idlenode.unexpected_msg == False)
+        assert not unsupported_service_bit5_node.unexpected_msg
+        assert not unsupported_service_bit7_node.unexpected_msg
+
+        self.log.info("Service bits 5 and 7 are allowed after August 1st 2018")
+        self.nodes[0].setmocktime(1533168000)  # August 2nd 2018
+
+        allowed_service_bit5_node = self.nodes[0].add_p2p_connection(P2PInterface(), services=NODE_NETWORK)
+        allowed_service_bit7_node = self.nodes[0].add_p2p_connection(P2PInterface(), services=NODE_NETWORK)
+
+        # Network thread stopped when all previous P2PInterfaces disconnected. Restart it
+        network_thread_start()
+
+        wait_until(lambda: allowed_service_bit5_node.message_count["verack"], lock=mininode_lock)
+        wait_until(lambda: allowed_service_bit7_node.message_count["verack"], lock=mininode_lock)
+
+if __name__ == '__main__':
+    P2PLeakTest().main()

test/functional/p2p_mempool.py

@@ -0,0 +1,35 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test p2p mempool message.
+
+Test that nodes are disconnected if they send mempool messages when bloom
+filters are not enabled.
+"""
+
+from test_framework.mininode import *
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+class P2PMempoolTests(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+        self.extra_args = [["-peerbloomfilters=0"]]
+
+    def run_test(self):
+        # Add a p2p connection
+        self.nodes[0].add_p2p_connection(P2PInterface())
+        network_thread_start()
+        self.nodes[0].p2p.wait_for_verack()
+
+        #request mempool
+        self.nodes[0].p2p.send_message(msg_mempool())
+        self.nodes[0].p2p.wait_for_disconnect()
+
+        #mininode must be disconnected at this point
+        assert_equal(len(self.nodes[0].getpeerinfo()), 0)
+
+if __name__ == '__main__':
+    P2PMempoolTests().main()

test/functional/p2p_pos_doublespend.py

@@ -0,0 +1,54 @@
+#!/usr/bin/env python3
+# Copyright (c) 2019 The PIVX developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+'''
+Covers the scenario of a valid PoS block with a valid coinstake transaction where the
+coinstake input prevout is double spent in one of the other transactions in the same block.
+'''
+
+from time import sleep
+
+from fake_stake.base_test import Agrarian_FakeStakeTest
+
+
+class PoSDoubleSpend(Agrarian_FakeStakeTest):
+
+    def run_test(self):
+        self.description = "Covers the scenario of a valid PoS block with a valid coinstake transaction where the coinstake input prevout is double spent in one of the other transactions in the same block."
+        self.init_test()
+        INITAL_MINED_BLOCKS = 300
+        FORK_DEPTH = 30
+        self.NUM_BLOCKS = 3
+
+        # 1) Starting mining blocks
+        self.log.info("Mining %d blocks.." % INITAL_MINED_BLOCKS)
+        self.node.generate(INITAL_MINED_BLOCKS)
+
+        # 2) Collect the possible prevouts
+        self.log.info("Collecting all unspent coins which we generated from mining...")
+        staking_utxo_list = self.node.listunspent()
+
+        # 3) Spam Blocks on the main chain
+        self.log.info("-- Main chain blocks first")
+        self.test_spam("Main", staking_utxo_list, fDoubleSpend=True)
+        sleep(2)
+
+        # 4) Mine some block as buffer
+        self.log.info("Mining %d more blocks..." % FORK_DEPTH)
+        self.node.generate(FORK_DEPTH)
+        sleep(2)
+
+        # 5) Spam Blocks on a forked chain
+        self.log.info("-- Forked chain blocks now")
+        err_msgs = self.test_spam("Forked", staking_utxo_list, fRandomHeight=True, randomRange=FORK_DEPTH, fDoubleSpend=True)
+
+        if not len(err_msgs) == 0:
+            self.log.error("result: " + " | ".join(err_msgs))
+            raise AssertionError("TEST FAILED")
+
+        self.log.info("%s PASSED" % self.__class__.__name__)
+
+if __name__ == '__main__':
+    PoSDoubleSpend().main()

test/functional/p2p_pos_fakestake.py

@@ -0,0 +1,58 @@
+#!/usr/bin/env python3
+# Copyright (c) 2019 The PIVX developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+'''
+Covers the scenario of a PoS block where the coinstake input prevout is already spent.
+'''
+
+from time import sleep
+
+from fake_stake.base_test import Agrarian_FakeStakeTest
+
+class PoSFakeStake(Agrarian_FakeStakeTest):
+
+    def run_test(self):
+        self.description = "Covers the scenario of a PoS block where the coinstake input prevout is already spent."
+        self.init_test()
+
+        INITAL_MINED_BLOCKS = 150   # First mined blocks (rewards collected to spend)
+        MORE_MINED_BLOCKS = 100     # Blocks mined after spending
+        STAKE_AMPL_ROUNDS = 2       # Rounds of stake amplification
+        self.NUM_BLOCKS = 3         # Number of spammed blocks
+
+        # 1) Starting mining blocks
+        self.log.info("Mining %d blocks.." % INITAL_MINED_BLOCKS)
+        self.node.generate(INITAL_MINED_BLOCKS)
+
+        # 2) Collect the possible prevouts
+        self.log.info("Collecting all unspent coins which we generated from mining...")
+
+        # 3) Create 10 addresses - Do the stake amplification
+        self.log.info("Performing the stake amplification (%d rounds)..." % STAKE_AMPL_ROUNDS)
+        utxo_list = self.node.listunspent()
+        address_list = []
+        for i in range(10):
+            address_list.append(self.node.getnewaddress())
+        utxo_list = self.stake_amplification(utxo_list, STAKE_AMPL_ROUNDS, address_list)
+
+        self.log.info("Done. Utxo list has %d elements." % len(utxo_list))
+        sleep(2)
+
+        # 4) Start mining again so that spent prevouts get confirmted in a block.
+        self.log.info("Mining %d more blocks..." % MORE_MINED_BLOCKS)
+        self.node.generate(MORE_MINED_BLOCKS)
+        sleep(2)
+
+        # 5) Create "Fake Stake" blocks and send them
+        self.log.info("Creating Fake stake blocks")
+        err_msgs = self.test_spam("Main", utxo_list)
+        if not len(err_msgs) == 0:
+            self.log.error("result: " + " | ".join(err_msgs))
+            raise AssertionError("TEST FAILED")
+
+        self.log.info("%s PASSED" % self.__class__.__name__)
+
+if __name__ == '__main__':
+    PoSFakeStake().main()

test/functional/p2p_pos_fakestake_accepted.py

@@ -0,0 +1,60 @@
+#!/usr/bin/env python3
+# Copyright (c) 2019 The PIVX developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+'''
+Covers the scenario of a valid PoS block where the coinstake input prevout is spent on main chain,
+but not on the fork branch. These blocks must be accepted.
+'''
+
+from time import sleep
+
+from fake_stake.base_test import Agrarian_FakeStakeTest
+
+class PoSFakeStakeAccepted(Agrarian_FakeStakeTest):
+
+    def run_test(self):
+        self.description = "Covers the scenario of a valid PoS block where the coinstake input prevout is spent on main chain, but not on the fork branch. These blocks must be accepted."
+        self.init_test()
+        INITAL_MINED_BLOCKS = 200   # First mined blocks (rewards collected to spend)
+        FORK_DEPTH = 50             # number of blocks after INITIAL_MINED_BLOCKS before the coins are spent
+        MORE_MINED_BLOCKS = 10      # number of blocks after spending of the collected coins
+        self.NUM_BLOCKS = 3         # Number of spammed blocks
+
+        # 1) Starting mining blocks
+        self.log.info("Mining %d blocks.." % INITAL_MINED_BLOCKS)
+        self.node.generate(INITAL_MINED_BLOCKS)
+
+        # 2) Collect the possible prevouts
+        self.log.info("Collecting all unspent coins which we generated from mining...")
+        staking_utxo_list = self.node.listunspent()
+        sleep(2)
+
+        # 3) Mine more blocks
+        self.log.info("Mining %d more blocks.." % (FORK_DEPTH+1))
+        self.node.generate(FORK_DEPTH+1)
+        sleep(2)
+
+        # 4) Spend the coins collected in 2 (mined in the first 100 blocks)
+        self.log.info("Spending the coins mined in the first %d blocks..." % INITAL_MINED_BLOCKS)
+        tx_hashes = self.spend_utxos(staking_utxo_list)
+        self.log.info("Spent %d transactions" % len(tx_hashes))
+        sleep(2)
+
+        # 5) Mine 10 more blocks
+        self.log.info("Mining %d more blocks to include the TXs in chain..." % MORE_MINED_BLOCKS)
+        self.node.generate(MORE_MINED_BLOCKS)
+        sleep(2)
+
+        # 6) Create "Fake Stake" blocks and send them
+        self.log.info("Creating Fake stake blocks")
+        err_msgs = self.test_spam("Fork", staking_utxo_list, fRandomHeight=True, randomRange=FORK_DEPTH, randomRange2=MORE_MINED_BLOCKS-2, fMustPass=True)
+        if not len(err_msgs) == 0:
+            self.log.error("result: " + " | ".join(err_msgs))
+            raise AssertionError("TEST FAILED")
+
+        self.log.info("%s PASSED" % self.__class__.__name__)
+
+if __name__ == '__main__':
+    PoSFakeStakeAccepted().main()

test/functional/p2p_sendheaders.py

@@ -0,0 +1,607 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2016 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+from test_framework.mininode import *
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+from test_framework.blocktools import create_block, create_coinbase
+
+'''
+SendHeadersTest -- test behavior of headers messages to announce blocks.
+
+Setup:
+
+- Two nodes, two p2p connections to node0. One p2p connection should only ever
+  receive inv's (omitted from testing description below, this is our control).
+  Second node is used for creating reorgs.
+
+Part 1: No headers announcements before "sendheaders"
+a. node mines a block [expect: inv]
+   send getdata for the block [expect: block]
+b. node mines another block [expect: inv]
+   send getheaders and getdata [expect: headers, then block]
+c. node mines another block [expect: inv]
+   peer mines a block, announces with header [expect: getdata]
+d. node mines another block [expect: inv]
+
+Part 2: After "sendheaders", headers announcements should generally work.
+a. peer sends sendheaders [expect: no response]
+   peer sends getheaders with current tip [expect: no response]
+b. node mines a block [expect: tip header]
+c. for N in 1, ..., 10:
+   * for announce-type in {inv, header}
+     - peer mines N blocks, announces with announce-type
+       [ expect: getheaders/getdata or getdata, deliver block(s) ]
+     - node mines a block [ expect: 1 header ]
+
+Part 3: Headers announcements stop after large reorg and resume after getheaders or inv from peer.
+- For response-type in {inv, getheaders}
+  * node mines a 7 block reorg [ expect: headers announcement of 8 blocks ]
+  * node mines an 8-block reorg [ expect: inv at tip ]
+  * peer responds with getblocks/getdata [expect: inv, blocks ]
+  * node mines another block [ expect: inv at tip, peer sends getdata, expect: block ]
+  * node mines another block at tip [ expect: inv ]
+  * peer responds with getheaders with an old hashstop more than 8 blocks back [expect: headers]
+  * peer requests block [ expect: block ]
+  * node mines another block at tip [ expect: inv, peer sends getdata, expect: block ]
+  * peer sends response-type [expect headers if getheaders, getheaders/getdata if mining new block]
+  * node mines 1 block [expect: 1 header, peer responds with getdata]
+
+Part 4: Test direct fetch behavior
+a. Announce 2 old block headers.
+   Expect: no getdata requests.
+b. Announce 3 new blocks via 1 headers message.
+   Expect: one getdata request for all 3 blocks.
+   (Send blocks.)
+c. Announce 1 header that forks off the last two blocks.
+   Expect: no response.
+d. Announce 1 more header that builds on that fork.
+   Expect: one getdata request for two blocks.
+e. Announce 16 more headers that build on that fork.
+   Expect: getdata request for 14 more blocks.
+f. Announce 1 more header that builds on that fork.
+   Expect: no response.
+
+Part 5: Test handling of headers that don't connect.
+a. Repeat 10 times:
+   1. Announce a header that doesn't connect.
+      Expect: getheaders message
+   2. Send headers chain.
+      Expect: getdata for the missing blocks, tip update.
+b. Then send 9 more headers that don't connect.
+   Expect: getheaders message each time.
+c. Announce a header that does connect.
+   Expect: no response.
+d. Announce 49 headers that don't connect.
+   Expect: getheaders message each time.
+e. Announce one more that doesn't connect.
+   Expect: disconnect.
+'''
+
+direct_fetch_response_time = 0.05
+
+class BaseNode(SingleNodeConnCB):
+    def __init__(self):
+        SingleNodeConnCB.__init__(self)
+        self.last_inv = None
+        self.last_headers = None
+        self.last_block = None
+        self.last_getdata = None
+        self.block_announced = False
+        self.last_getheaders = None
+        self.disconnected = False
+        self.last_blockhash_announced = None
+
+    def clear_last_announcement(self):
+        with mininode_lock:
+            self.block_announced = False
+            self.last_inv = None
+            self.last_headers = None
+
+    # Request data for a list of block hashes
+    def get_data(self, block_hashes):
+        msg = msg_getdata()
+        for x in block_hashes:
+            msg.inv.append(CInv(2, x))
+        self.connection.send_message(msg)
+
+    def get_headers(self, locator, hashstop):
+        msg = msg_getheaders()
+        msg.locator.vHave = locator
+        msg.hashstop = hashstop
+        self.connection.send_message(msg)
+
+    def send_block_inv(self, blockhash):
+        msg = msg_inv()
+        msg.inv = [CInv(2, blockhash)]
+        self.connection.send_message(msg)
+
+    def on_inv(self, conn, message):
+        self.last_inv = message
+        self.block_announced = True
+        self.last_blockhash_announced = message.inv[-1].hash
+
+    def on_headers(self, conn, message):
+        self.last_headers = message
+        if len(message.headers):
+            self.block_announced = True
+            message.headers[-1].calc_sha256()
+            self.last_blockhash_announced = message.headers[-1].sha256
+
+    def on_block(self, conn, message):
+        self.last_block = message.block
+        self.last_block.calc_sha256()
+
+    def on_getdata(self, conn, message):
+        self.last_getdata = message
+
+    def on_getheaders(self, conn, message):
+        self.last_getheaders = message
+
+    def on_close(self, conn):
+        self.disconnected = True
+
+    # Test whether the last announcement we received had the
+    # right header or the right inv
+    # inv and headers should be lists of block hashes
+    def check_last_announcement(self, headers=None, inv=None):
+        expect_headers = headers if headers != None else []
+        expect_inv = inv if inv != None else []
+        test_function = lambda: self.block_announced
+        assert(wait_until(test_function, timeout=60))
+        with mininode_lock:
+            self.block_announced = False
+
+            success = True
+            compare_inv = []
+            if self.last_inv != None:
+                compare_inv = [x.hash for x in self.last_inv.inv]
+            if compare_inv != expect_inv:
+                success = False
+
+            hash_headers = []
+            if self.last_headers != None:
+                # treat headers as a list of block hashes
+                hash_headers = [ x.sha256 for x in self.last_headers.headers ]
+            if hash_headers != expect_headers:
+                success = False
+
+            self.last_inv = None
+            self.last_headers = None
+        return success
+
+    # Syncing helpers
+    def wait_for_block(self, blockhash, timeout=60):
+        test_function = lambda: self.last_block != None and self.last_block.sha256 == blockhash
+        assert(wait_until(test_function, timeout=timeout))
+        return
+
+    def wait_for_getheaders(self, timeout=60):
+        test_function = lambda: self.last_getheaders != None
+        assert(wait_until(test_function, timeout=timeout))
+        return
+
+    def wait_for_getdata(self, hash_list, timeout=60):
+        if hash_list == []:
+            return
+
+        test_function = lambda: self.last_getdata != None and [x.hash for x in self.last_getdata.inv] == hash_list
+        assert(wait_until(test_function, timeout=timeout))
+        return
+
+    def wait_for_disconnect(self, timeout=60):
+        test_function = lambda: self.disconnected
+        assert(wait_until(test_function, timeout=timeout))
+        return
+
+    def wait_for_block_announcement(self, block_hash, timeout=60):
+        test_function = lambda: self.last_blockhash_announced == block_hash
+        assert(wait_until(test_function, timeout=timeout))
+        return
+
+    def send_header_for_blocks(self, new_blocks):
+        headers_message = msg_headers()
+        headers_message.headers = [ CBlockHeader(b) for b in new_blocks ]
+        self.send_message(headers_message)
+
+    def send_getblocks(self, locator):
+        getblocks_message = msg_getblocks()
+        getblocks_message.locator.vHave = locator
+        self.send_message(getblocks_message)
+
+# InvNode: This peer should only ever receive inv's, because it doesn't ever send a
+# "sendheaders" message.
+class InvNode(BaseNode):
+    def __init__(self):
+        BaseNode.__init__(self)
+
+# TestNode: This peer is the one we use for most of the testing.
+class TestNode(BaseNode):
+    def __init__(self):
+        BaseNode.__init__(self)
+
+class SendHeadersTest(BitcoinTestFramework):
+    def __init__(self):
+        super().__init__()
+        self.setup_clean_chain = True
+        self.num_nodes = 2
+
+    def setup_network(self):
+        self.nodes = []
+        self.nodes = start_nodes(self.num_nodes, self.options.tmpdir, [["-debug", "-logtimemicros=1"]]*2)
+        connect_nodes(self.nodes[0], 1)
+
+    # mine count blocks and return the new tip
+    def mine_blocks(self, count):
+        # Clear out last block announcement from each p2p listener
+        [ x.clear_last_announcement() for x in self.p2p_connections ]
+        self.nodes[0].generate(count)
+        return int(self.nodes[0].getbestblockhash(), 16)
+
+    # mine a reorg that invalidates length blocks (replacing them with
+    # length+1 blocks).
+    # Note: we clear the state of our p2p connections after the
+    # to-be-reorged-out blocks are mined, so that we don't break later tests.
+    # return the list of block hashes newly mined
+    def mine_reorg(self, length):
+        self.nodes[0].generate(length) # make sure all invalidated blocks are node0's
+        sync_blocks(self.nodes, wait=0.1)
+        for x in self.p2p_connections:
+            x.wait_for_block_announcement(int(self.nodes[0].getbestblockhash(), 16))
+            x.clear_last_announcement()
+
+        tip_height = self.nodes[1].getblockcount()
+        hash_to_invalidate = self.nodes[1].getblockhash(tip_height-(length-1))
+        self.nodes[1].invalidateblock(hash_to_invalidate)
+        all_hashes = self.nodes[1].generate(length+1) # Must be longer than the orig chain
+        sync_blocks(self.nodes, wait=0.1)
+        return [int(x, 16) for x in all_hashes]
+
+    def run_test(self):
+        # Setup the p2p connections and start up the network thread.
+        inv_node = InvNode()
+        test_node = TestNode()
+
+        self.p2p_connections = [inv_node, test_node]
+
+        connections = []
+        connections.append(NodeConn('127.0.0.1', p2p_port(0), self.nodes[0], inv_node))
+        # Set nServices to 0 for test_node, so no block download will occur outside of
+        # direct fetching
+        connections.append(NodeConn('127.0.0.1', p2p_port(0), self.nodes[0], test_node, services=0))
+        inv_node.add_connection(connections[0])
+        test_node.add_connection(connections[1])
+
+        NetworkThread().start() # Start up network handling in another thread
+
+        # Test logic begins here
+        inv_node.wait_for_verack()
+        test_node.wait_for_verack()
+
+        tip = int(self.nodes[0].getbestblockhash(), 16)
+
+        # PART 1
+        # 1. Mine a block; expect inv announcements each time
+        print("Part 1: headers don't start before sendheaders message...")
+        for i in range(4):
+            old_tip = tip
+            tip = self.mine_blocks(1)
+            assert_equal(inv_node.check_last_announcement(inv=[tip]), True)
+            assert_equal(test_node.check_last_announcement(inv=[tip]), True)
+            # Try a few different responses; none should affect next announcement
+            if i == 0:
+                # first request the block
+                test_node.get_data([tip])
+                test_node.wait_for_block(tip, timeout=5)
+            elif i == 1:
+                # next try requesting header and block
+                test_node.get_headers(locator=[old_tip], hashstop=tip)
+                test_node.get_data([tip])
+                test_node.wait_for_block(tip)
+                test_node.clear_last_announcement() # since we requested headers...
+            elif i == 2:
+                # this time announce own block via headers
+                height = self.nodes[0].getblockcount()
+                last_time = self.nodes[0].getblock(self.nodes[0].getbestblockhash())['time']
+                block_time = last_time + 1
+                new_block = create_block(tip, create_coinbase(height+1), block_time)
+                new_block.solve()
+                test_node.send_header_for_blocks([new_block])
+                test_node.wait_for_getdata([new_block.sha256], timeout=5)
+                test_node.send_message(msg_block(new_block))
+                test_node.sync_with_ping() # make sure this block is processed
+                inv_node.clear_last_announcement()
+                test_node.clear_last_announcement()
+
+        print("Part 1: success!")
+        print("Part 2: announce blocks with headers after sendheaders message...")
+        # PART 2
+        # 2. Send a sendheaders message and test that headers announcements
+        # commence and keep working.
+        test_node.send_message(msg_sendheaders())
+        prev_tip = int(self.nodes[0].getbestblockhash(), 16)
+        test_node.get_headers(locator=[prev_tip], hashstop=0)
+        test_node.sync_with_ping()
+
+        # Now that we've synced headers, headers announcements should work
+        tip = self.mine_blocks(1)
+        assert_equal(inv_node.check_last_announcement(inv=[tip]), True)
+        assert_equal(test_node.check_last_announcement(headers=[tip]), True)
+
+        height = self.nodes[0].getblockcount()+1
+        block_time += 10  # Advance far enough ahead
+        for i in range(10):
+            # Mine i blocks, and alternate announcing either via
+            # inv (of tip) or via headers. After each, new blocks
+            # mined by the node should successfully be announced
+            # with block header, even though the blocks are never requested
+            for j in range(2):
+                blocks = []
+                for b in range(i+1):
+                    blocks.append(create_block(tip, create_coinbase(height), block_time))
+                    blocks[-1].solve()
+                    tip = blocks[-1].sha256
+                    block_time += 1
+                    height += 1
+                if j == 0:
+                    # Announce via inv
+                    test_node.send_block_inv(tip)
+                    test_node.wait_for_getdata([tip], timeout=5)
+                    # Test that duplicate inv's won't result in duplicate
+                    # getdata requests, or duplicate headers announcements
+                    inv_node.send_block_inv(tip)
+                    # Should have received a getheaders as well!
+                    test_node.send_header_for_blocks(blocks)
+                    test_node.wait_for_getdata([x.sha256 for x in blocks[0:-1]], timeout=5)
+                    [ inv_node.send_block_inv(x.sha256) for x in blocks[0:-1] ]
+                    inv_node.sync_with_ping()
+                else:
+                    # Announce via headers
+                    test_node.send_header_for_blocks(blocks)
+                    test_node.wait_for_getdata([x.sha256 for x in blocks], timeout=5)
+                    # Test that duplicate headers won't result in duplicate
+                    # getdata requests (the check is further down)
+                    inv_node.send_header_for_blocks(blocks)
+                    inv_node.sync_with_ping()
+                [ test_node.send_message(msg_block(x)) for x in blocks ]
+                test_node.sync_with_ping()
+                inv_node.sync_with_ping()
+                # This block should not be announced to the inv node (since it also
+                # broadcast it)
+                assert_equal(inv_node.last_inv, None)
+                assert_equal(inv_node.last_headers, None)
+                tip = self.mine_blocks(1)
+                assert_equal(inv_node.check_last_announcement(inv=[tip]), True)
+                assert_equal(test_node.check_last_announcement(headers=[tip]), True)
+                height += 1
+                block_time += 1
+
+        print("Part 2: success!")
+
+        print("Part 3: headers announcements can stop after large reorg, and resume after headers/inv from peer...")
+
+        # PART 3.  Headers announcements can stop after large reorg, and resume after
+        # getheaders or inv from peer.
+        for j in range(2):
+            # First try mining a reorg that can propagate with header announcement
+            new_block_hashes = self.mine_reorg(length=7)
+            tip = new_block_hashes[-1]
+            assert_equal(inv_node.check_last_announcement(inv=[tip]), True)
+            assert_equal(test_node.check_last_announcement(headers=new_block_hashes), True)
+
+            block_time += 8
+
+            # Mine a too-large reorg, which should be announced with a single inv
+            new_block_hashes = self.mine_reorg(length=8)
+            tip = new_block_hashes[-1]
+            assert_equal(inv_node.check_last_announcement(inv=[tip]), True)
+            assert_equal(test_node.check_last_announcement(inv=[tip]), True)
+
+            block_time += 9
+
+            fork_point = self.nodes[0].getblock("%02x" % new_block_hashes[0])["previousblockhash"]
+            fork_point = int(fork_point, 16)
+
+            # Use getblocks/getdata
+            test_node.send_getblocks(locator = [fork_point])
+            assert_equal(test_node.check_last_announcement(inv=new_block_hashes), True)
+            test_node.get_data(new_block_hashes)
+            test_node.wait_for_block(new_block_hashes[-1])
+
+            for i in range(3):
+                # Mine another block, still should get only an inv
+                tip = self.mine_blocks(1)
+                assert_equal(inv_node.check_last_announcement(inv=[tip]), True)
+                assert_equal(test_node.check_last_announcement(inv=[tip]), True)
+                if i == 0:
+                    # Just get the data -- shouldn't cause headers announcements to resume
+                    test_node.get_data([tip])
+                    test_node.wait_for_block(tip)
+                elif i == 1:
+                    # Send a getheaders message that shouldn't trigger headers announcements
+                    # to resume (best header sent will be too old)
+                    test_node.get_headers(locator=[fork_point], hashstop=new_block_hashes[1])
+                    test_node.get_data([tip])
+                    test_node.wait_for_block(tip)
+                elif i == 2:
+                    test_node.get_data([tip])
+                    test_node.wait_for_block(tip)
+                    # This time, try sending either a getheaders to trigger resumption
+                    # of headers announcements, or mine a new block and inv it, also
+                    # triggering resumption of headers announcements.
+                    if j == 0:
+                        test_node.get_headers(locator=[tip], hashstop=0)
+                        test_node.sync_with_ping()
+                    else:
+                        test_node.send_block_inv(tip)
+                        test_node.sync_with_ping()
+            # New blocks should now be announced with header
+            tip = self.mine_blocks(1)
+            assert_equal(inv_node.check_last_announcement(inv=[tip]), True)
+            assert_equal(test_node.check_last_announcement(headers=[tip]), True)
+
+        print("Part 3: success!")
+
+        print("Part 4: Testing direct fetch behavior...")
+        tip = self.mine_blocks(1)
+        height = self.nodes[0].getblockcount() + 1
+        last_time = self.nodes[0].getblock(self.nodes[0].getbestblockhash())['time']
+        block_time = last_time + 1
+
+        # Create 2 blocks.  Send the blocks, then send the headers.
+        blocks = []
+        for b in range(2):
+            blocks.append(create_block(tip, create_coinbase(height), block_time))
+            blocks[-1].solve()
+            tip = blocks[-1].sha256
+            block_time += 1
+            height += 1
+            inv_node.send_message(msg_block(blocks[-1]))
+
+        inv_node.sync_with_ping() # Make sure blocks are processed
+        test_node.last_getdata = None
+        test_node.send_header_for_blocks(blocks)
+        test_node.sync_with_ping()
+        # should not have received any getdata messages
+        with mininode_lock:
+            assert_equal(test_node.last_getdata, None)
+
+        # This time, direct fetch should work
+        blocks = []
+        for b in range(3):
+            blocks.append(create_block(tip, create_coinbase(height), block_time))
+            blocks[-1].solve()
+            tip = blocks[-1].sha256
+            block_time += 1
+            height += 1
+
+        test_node.send_header_for_blocks(blocks)
+        test_node.sync_with_ping()
+        test_node.wait_for_getdata([x.sha256 for x in blocks], timeout=direct_fetch_response_time)
+
+        [ test_node.send_message(msg_block(x)) for x in blocks ]
+
+        test_node.sync_with_ping()
+
+        # Now announce a header that forks the last two blocks
+        tip = blocks[0].sha256
+        height -= 1
+        blocks = []
+
+        # Create extra blocks for later
+        for b in range(20):
+            blocks.append(create_block(tip, create_coinbase(height), block_time))
+            blocks[-1].solve()
+            tip = blocks[-1].sha256
+            block_time += 1
+            height += 1
+
+        # Announcing one block on fork should not trigger direct fetch
+        # (less work than tip)
+        test_node.last_getdata = None
+        test_node.send_header_for_blocks(blocks[0:1])
+        test_node.sync_with_ping()
+        with mininode_lock:
+            assert_equal(test_node.last_getdata, None)
+
+        # Announcing one more block on fork should trigger direct fetch for
+        # both blocks (same work as tip)
+        test_node.send_header_for_blocks(blocks[1:2])
+        test_node.sync_with_ping()
+        test_node.wait_for_getdata([x.sha256 for x in blocks[0:2]], timeout=direct_fetch_response_time)
+
+        # Announcing 16 more headers should trigger direct fetch for 14 more
+        # blocks
+        test_node.send_header_for_blocks(blocks[2:18])
+        test_node.sync_with_ping()
+        test_node.wait_for_getdata([x.sha256 for x in blocks[2:16]], timeout=direct_fetch_response_time)
+
+        # Announcing 1 more header should not trigger any response
+        test_node.last_getdata = None
+        test_node.send_header_for_blocks(blocks[18:19])
+        test_node.sync_with_ping()
+        with mininode_lock:
+            assert_equal(test_node.last_getdata, None)
+
+        print("Part 4: success!")
+
+        # Now deliver all those blocks we announced.
+        [ test_node.send_message(msg_block(x)) for x in blocks ]
+
+        print("Part 5: Testing handling of unconnecting headers")
+        # First we test that receipt of an unconnecting header doesn't prevent
+        # chain sync.
+        for i in range(10):
+            test_node.last_getdata = None
+            blocks = []
+            # Create two more blocks.
+            for j in range(2):
+                blocks.append(create_block(tip, create_coinbase(height), block_time))
+                blocks[-1].solve()
+                tip = blocks[-1].sha256
+                block_time += 1
+                height += 1
+            # Send the header of the second block -> this won't connect.
+            with mininode_lock:
+                test_node.last_getheaders = None
+            test_node.send_header_for_blocks([blocks[1]])
+            test_node.wait_for_getheaders(timeout=1)
+            test_node.send_header_for_blocks(blocks)
+            test_node.wait_for_getdata([x.sha256 for x in blocks])
+            [ test_node.send_message(msg_block(x)) for x in blocks ]
+            test_node.sync_with_ping()
+            assert_equal(int(self.nodes[0].getbestblockhash(), 16), blocks[1].sha256)
+
+        blocks = []
+        # Now we test that if we repeatedly don't send connecting headers, we
+        # don't go into an infinite loop trying to get them to connect.
+        MAX_UNCONNECTING_HEADERS = 10
+        for j in range(MAX_UNCONNECTING_HEADERS+1):
+            blocks.append(create_block(tip, create_coinbase(height), block_time))
+            blocks[-1].solve()
+            tip = blocks[-1].sha256
+            block_time += 1
+            height += 1
+
+        for i in range(1, MAX_UNCONNECTING_HEADERS):
+            # Send a header that doesn't connect, check that we get a getheaders.
+            with mininode_lock:
+                test_node.last_getheaders = None
+            test_node.send_header_for_blocks([blocks[i]])
+            test_node.wait_for_getheaders(timeout=1)
+
+        # Next header will connect, should re-set our count:
+        test_node.send_header_for_blocks([blocks[0]])
+
+        # Remove the first two entries (blocks[1] would connect):
+        blocks = blocks[2:]
+
+        # Now try to see how many unconnecting headers we can send
+        # before we get disconnected.  Should be 5*MAX_UNCONNECTING_HEADERS
+        for i in range(5*MAX_UNCONNECTING_HEADERS - 1):
+            # Send a header that doesn't connect, check that we get a getheaders.
+            with mininode_lock:
+                test_node.last_getheaders = None
+            test_node.send_header_for_blocks([blocks[i%len(blocks)]])
+            test_node.wait_for_getheaders(timeout=1)
+
+        # Eventually this stops working.
+        with mininode_lock:
+            self.last_getheaders = None
+        test_node.send_header_for_blocks([blocks[-1]])
+
+        # Should get disconnected
+        test_node.wait_for_disconnect()
+        with mininode_lock:
+            self.last_getheaders = True
+
+        print("Part 5: success!")
+
+        # Finally, check that the inv node never received a getdata request,
+        # throughout the test
+        assert_equal(inv_node.last_getdata, None)
+
+if __name__ == '__main__':
+    SendHeadersTest().main()

test/functional/p2p_timeouts.py

@@ -0,0 +1,75 @@
+#!/usr/bin/env python3
+# Copyright (c) 2016-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test various net timeouts.
+
+- Create three bitcoind nodes:
+
+    no_verack_node - we never send a verack in response to their version
+    no_version_node - we never send a version (only a ping)
+    no_send_node - we never send any P2P message.
+
+- Start all three nodes
+- Wait 1 second
+- Assert that we're connected
+- Send a ping to no_verack_node and no_version_node
+- Wait 30 seconds
+- Assert that we're still connected
+- Send a ping to no_verack_node and no_version_node
+- Wait 31 seconds
+- Assert that we're no longer connected (timeout to receive version/verack is 60 seconds)
+"""
+
+from time import sleep
+
+from test_framework.mininode import *
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+class TestNode(P2PInterface):
+    def on_version(self, message):
+        # Don't send a verack in response
+        pass
+
+class TimeoutsTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+
+    def run_test(self):
+        # Setup the p2p connections and start up the network thread.
+        no_verack_node = self.nodes[0].add_p2p_connection(TestNode())
+        no_version_node = self.nodes[0].add_p2p_connection(TestNode(), send_version=False)
+        no_send_node = self.nodes[0].add_p2p_connection(TestNode(), send_version=False)
+
+        network_thread_start()
+
+        sleep(1)
+
+        assert no_verack_node.connected
+        assert no_version_node.connected
+        assert no_send_node.connected
+
+        no_verack_node.send_message(msg_ping())
+        no_version_node.send_message(msg_ping())
+
+        sleep(30)
+
+        assert "version" in no_verack_node.last_message
+
+        assert no_verack_node.connected
+        assert no_version_node.connected
+        assert no_send_node.connected
+
+        no_verack_node.send_message(msg_ping())
+        no_version_node.send_message(msg_ping())
+
+        sleep(31)
+
+        assert not no_verack_node.connected
+        assert not no_version_node.connected
+        assert not no_send_node.connected
+
+if __name__ == '__main__':
+    TimeoutsTest().main()

test/functional/p2p_unrequested_blocks.py

@@ -0,0 +1,323 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test processing of unrequested blocks.
+
+Setup: two nodes, node0+node1, not connected to each other. Node1 will have
+nMinimumChainWork set to 0x10, so it won't process low-work unrequested blocks.
+
+We have one P2PInterface connection to node0 called test_node, and one to node1
+called min_work_node.
+
+The test:
+1. Generate one block on each node, to leave IBD.
+
+2. Mine a new block on each tip, and deliver to each node from node's peer.
+   The tip should advance for node0, but node1 should skip processing due to
+   nMinimumChainWork.
+
+Node1 is unused in tests 3-7:
+
+3. Mine a block that forks from the genesis block, and deliver to test_node.
+   Node0 should not process this block (just accept the header), because it
+   is unrequested and doesn't have more or equal work to the tip.
+
+4a,b. Send another two blocks that build on the forking block.
+   Node0 should process the second block but be stuck on the shorter chain,
+   because it's missing an intermediate block.
+
+4c.Send 288 more blocks on the longer chain (the number of blocks ahead
+   we currently store).
+   Node0 should process all but the last block (too far ahead in height).
+
+5. Send a duplicate of the block in #3 to Node0.
+   Node0 should not process the block because it is unrequested, and stay on
+   the shorter chain.
+
+6. Send Node0 an inv for the height 3 block produced in #4 above.
+   Node0 should figure out that Node0 has the missing height 2 block and send a
+   getdata.
+
+7. Send Node0 the missing block again.
+   Node0 should process and the tip should advance.
+
+8. Create a fork which is invalid at a height longer than the current chain
+   (ie to which the node will try to reorg) but which has headers built on top
+   of the invalid block. Check that we get disconnected if we send more headers
+   on the chain the node now knows to be invalid.
+
+9. Test Node1 is able to sync when connected to node0 (which should have sufficient
+   work on its chain).
+"""
+
+from test_framework.mininode import *
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+import time
+from test_framework.blocktools import create_block, create_coinbase, create_transaction
+
+class AcceptBlockTest(BitcoinTestFramework):
+    def add_options(self, parser):
+        parser.add_option("--testbinary", dest="testbinary",
+                          default=os.getenv("BITCOIND", "bitcoind"),
+                          help="bitcoind binary to test")
+
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 2
+        self.extra_args = [[], ["-minimumchainwork=0x10"]]
+
+    def setup_network(self):
+        # Node0 will be used to test behavior of processing unrequested blocks
+        # from peers which are not whitelisted, while Node1 will be used for
+        # the whitelisted case.
+        # Node2 will be used for non-whitelisted peers to test the interaction
+        # with nMinimumChainWork.
+        self.setup_nodes()
+
+    def run_test(self):
+        # Setup the p2p connections and start up the network thread.
+        # test_node connects to node0 (not whitelisted)
+        test_node = self.nodes[0].add_p2p_connection(P2PInterface())
+        # min_work_node connects to node1 (whitelisted)
+        min_work_node = self.nodes[1].add_p2p_connection(P2PInterface())
+
+        network_thread_start()
+
+        # Test logic begins here
+        test_node.wait_for_verack()
+        min_work_node.wait_for_verack()
+
+        # 1. Have nodes mine a block (leave IBD)
+        [ n.generate(1) for n in self.nodes ]
+        tips = [ int("0x" + n.getbestblockhash(), 0) for n in self.nodes ]
+
+        # 2. Send one block that builds on each tip.
+        # This should be accepted by node0
+        blocks_h2 = []  # the height 2 blocks on each node's chain
+        block_time = int(time.time()) + 1
+        for i in range(2):
+            blocks_h2.append(create_block(tips[i], create_coinbase(2), block_time))
+            blocks_h2[i].solve()
+            block_time += 1
+        test_node.send_message(msg_block(blocks_h2[0]))
+        min_work_node.send_message(msg_block(blocks_h2[1]))
+
+        for x in [test_node, min_work_node]:
+            x.sync_with_ping()
+        assert_equal(self.nodes[0].getblockcount(), 2)
+        assert_equal(self.nodes[1].getblockcount(), 1)
+        self.log.info("First height 2 block accepted by node0; correctly rejected by node1")
+
+        # 3. Send another block that builds on genesis.
+        block_h1f = create_block(int("0x" + self.nodes[0].getblockhash(0), 0), create_coinbase(1), block_time)
+        block_time += 1
+        block_h1f.solve()
+        test_node.send_message(msg_block(block_h1f))
+
+        test_node.sync_with_ping()
+        tip_entry_found = False
+        for x in self.nodes[0].getchaintips():
+            if x['hash'] == block_h1f.hash:
+                assert_equal(x['status'], "headers-only")
+                tip_entry_found = True
+        assert(tip_entry_found)
+        assert_raises_rpc_error(-1, "Block not found on disk", self.nodes[0].getblock, block_h1f.hash)
+
+        # 4. Send another two block that build on the fork.
+        block_h2f = create_block(block_h1f.sha256, create_coinbase(2), block_time)
+        block_time += 1
+        block_h2f.solve()
+        test_node.send_message(msg_block(block_h2f))
+
+        test_node.sync_with_ping()
+        # Since the earlier block was not processed by node, the new block
+        # can't be fully validated.
+        tip_entry_found = False
+        for x in self.nodes[0].getchaintips():
+            if x['hash'] == block_h2f.hash:
+                assert_equal(x['status'], "headers-only")
+                tip_entry_found = True
+        assert(tip_entry_found)
+
+        # But this block should be accepted by node since it has equal work.
+        self.nodes[0].getblock(block_h2f.hash)
+        self.log.info("Second height 2 block accepted, but not reorg'ed to")
+
+        # 4b. Now send another block that builds on the forking chain.
+        block_h3 = create_block(block_h2f.sha256, create_coinbase(3), block_h2f.nTime+1)
+        block_h3.solve()
+        test_node.send_message(msg_block(block_h3))
+
+        test_node.sync_with_ping()
+        # Since the earlier block was not processed by node, the new block
+        # can't be fully validated.
+        tip_entry_found = False
+        for x in self.nodes[0].getchaintips():
+            if x['hash'] == block_h3.hash:
+                assert_equal(x['status'], "headers-only")
+                tip_entry_found = True
+        assert(tip_entry_found)
+        self.nodes[0].getblock(block_h3.hash)
+
+        # But this block should be accepted by node since it has more work.
+        self.nodes[0].getblock(block_h3.hash)
+        self.log.info("Unrequested more-work block accepted")
+
+        # 4c. Now mine 288 more blocks and deliver; all should be processed but
+        # the last (height-too-high) on node (as long as its not missing any headers)
+        tip = block_h3
+        all_blocks = []
+        for i in range(288):
+            next_block = create_block(tip.sha256, create_coinbase(i + 4), tip.nTime+1)
+            next_block.solve()
+            all_blocks.append(next_block)
+            tip = next_block
+
+        # Now send the block at height 5 and check that it wasn't accepted (missing header)
+        test_node.send_message(msg_block(all_blocks[1]))
+        test_node.sync_with_ping()
+        assert_raises_rpc_error(-5, "Block not found", self.nodes[0].getblock, all_blocks[1].hash)
+        assert_raises_rpc_error(-5, "Block not found", self.nodes[0].getblockheader, all_blocks[1].hash)
+
+        # The block at height 5 should be accepted if we provide the missing header, though
+        headers_message = msg_headers()
+        headers_message.headers.append(CBlockHeader(all_blocks[0]))
+        test_node.send_message(headers_message)
+        test_node.send_message(msg_block(all_blocks[1]))
+        test_node.sync_with_ping()
+        self.nodes[0].getblock(all_blocks[1].hash)
+
+        # Now send the blocks in all_blocks
+        for i in range(288):
+            test_node.send_message(msg_block(all_blocks[i]))
+        test_node.sync_with_ping()
+
+        # Blocks 1-287 should be accepted, block 288 should be ignored because it's too far ahead
+        for x in all_blocks[:-1]:
+            self.nodes[0].getblock(x.hash)
+        assert_raises_rpc_error(-1, "Block not found on disk", self.nodes[0].getblock, all_blocks[-1].hash)
+
+        # 5. Test handling of unrequested block on the node that didn't process
+        # Should still not be processed (even though it has a child that has more
+        # work).
+
+        # The node should have requested the blocks at some point, so
+        # disconnect/reconnect first
+
+        self.nodes[0].disconnect_p2ps()
+        self.nodes[1].disconnect_p2ps()
+        network_thread_join()
+
+        test_node = self.nodes[0].add_p2p_connection(P2PInterface())
+        network_thread_start()
+        test_node.wait_for_verack()
+
+        test_node.send_message(msg_block(block_h1f))
+
+        test_node.sync_with_ping()
+        assert_equal(self.nodes[0].getblockcount(), 2)
+        self.log.info("Unrequested block that would complete more-work chain was ignored")
+
+        # 6. Try to get node to request the missing block.
+        # Poke the node with an inv for block at height 3 and see if that
+        # triggers a getdata on block 2 (it should if block 2 is missing).
+        with mininode_lock:
+            # Clear state so we can check the getdata request
+            test_node.last_message.pop("getdata", None)
+            test_node.send_message(msg_inv([CInv(2, block_h3.sha256)]))
+
+        test_node.sync_with_ping()
+        with mininode_lock:
+            getdata = test_node.last_message["getdata"]
+
+        # Check that the getdata includes the right block
+        assert_equal(getdata.inv[0].hash, block_h1f.sha256)
+        self.log.info("Inv at tip triggered getdata for unprocessed block")
+
+        # 7. Send the missing block for the third time (now it is requested)
+        test_node.send_message(msg_block(block_h1f))
+
+        test_node.sync_with_ping()
+        assert_equal(self.nodes[0].getblockcount(), 290)
+        self.nodes[0].getblock(all_blocks[286].hash)
+        assert_equal(self.nodes[0].getbestblockhash(), all_blocks[286].hash)
+        assert_raises_rpc_error(-1, "Block not found on disk", self.nodes[0].getblock, all_blocks[287].hash)
+        self.log.info("Successfully reorged to longer chain from non-whitelisted peer")
+
+        # 8. Create a chain which is invalid at a height longer than the
+        # current chain, but which has more blocks on top of that
+        block_289f = create_block(all_blocks[284].sha256, create_coinbase(289), all_blocks[284].nTime+1)
+        block_289f.solve()
+        block_290f = create_block(block_289f.sha256, create_coinbase(290), block_289f.nTime+1)
+        block_290f.solve()
+        block_291 = create_block(block_290f.sha256, create_coinbase(291), block_290f.nTime+1)
+        # block_291 spends a coinbase below maturity!
+        block_291.vtx.append(create_transaction(block_290f.vtx[0], 0, b"42", 1))
+        block_291.hashMerkleRoot = block_291.calc_merkle_root()
+        block_291.solve()
+        block_292 = create_block(block_291.sha256, create_coinbase(292), block_291.nTime+1)
+        block_292.solve()
+
+        # Now send all the headers on the chain and enough blocks to trigger reorg
+        headers_message = msg_headers()
+        headers_message.headers.append(CBlockHeader(block_289f))
+        headers_message.headers.append(CBlockHeader(block_290f))
+        headers_message.headers.append(CBlockHeader(block_291))
+        headers_message.headers.append(CBlockHeader(block_292))
+        test_node.send_message(headers_message)
+
+        test_node.sync_with_ping()
+        tip_entry_found = False
+        for x in self.nodes[0].getchaintips():
+            if x['hash'] == block_292.hash:
+                assert_equal(x['status'], "headers-only")
+                tip_entry_found = True
+        assert(tip_entry_found)
+        assert_raises_rpc_error(-1, "Block not found on disk", self.nodes[0].getblock, block_292.hash)
+
+        test_node.send_message(msg_block(block_289f))
+        test_node.send_message(msg_block(block_290f))
+
+        test_node.sync_with_ping()
+        self.nodes[0].getblock(block_289f.hash)
+        self.nodes[0].getblock(block_290f.hash)
+
+        test_node.send_message(msg_block(block_291))
+
+        # At this point we've sent an obviously-bogus block, wait for full processing
+        # without assuming whether we will be disconnected or not
+        try:
+            # Only wait a short while so the test doesn't take forever if we do get
+            # disconnected
+            test_node.sync_with_ping(timeout=1)
+        except AssertionError:
+            test_node.wait_for_disconnect()
+
+            self.nodes[0].disconnect_p2ps()
+            test_node = self.nodes[0].add_p2p_connection(P2PInterface())
+
+            network_thread_start()
+            test_node.wait_for_verack()
+
+        # We should have failed reorg and switched back to 290 (but have block 291)
+        assert_equal(self.nodes[0].getblockcount(), 290)
+        assert_equal(self.nodes[0].getbestblockhash(), all_blocks[286].hash)
+        assert_equal(self.nodes[0].getblock(block_291.hash)["confirmations"], -1)
+
+        # Now send a new header on the invalid chain, indicating we're forked off, and expect to get disconnected
+        block_293 = create_block(block_292.sha256, create_coinbase(293), block_292.nTime+1)
+        block_293.solve()
+        headers_message = msg_headers()
+        headers_message.headers.append(CBlockHeader(block_293))
+        test_node.send_message(headers_message)
+        test_node.wait_for_disconnect()
+
+        # 9. Connect node1 to node0 and ensure it is able to sync
+        connect_nodes(self.nodes[0], 1)
+        sync_blocks([self.nodes[0], self.nodes[1]])
+        self.log.info("Successfully synced nodes 1 and 0")
+
+if __name__ == '__main__':
+    AcceptBlockTest().main()

test/functional/p2p_zpos_fakestake.py

@@ -0,0 +1,106 @@
+#!/usr/bin/env python3
+# Copyright (c) 2019 The PIVX developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+'''
+Covers the scenario of a zPoS block where the coinstake input is a zerocoin spend
+of an already spent coin.
+'''
+
+from time import sleep
+
+from test_framework.authproxy import JSONRPCException
+
+from fake_stake.base_test import Agrarian_FakeStakeTest
+
+class zPoSFakeStake(Agrarian_FakeStakeTest):
+
+    def run_test(self):
+        self.description = "Covers the scenario of a zPoS block where the coinstake input is a zerocoin spend of an already spent coin."
+        self.init_test()
+
+        DENOM_TO_USE = 5000         # zc denomination
+        INITAL_MINED_BLOCKS = 321   # First mined blocks (rewards collected to mint)
+        MORE_MINED_BLOCKS = 301     # More blocks mined before spending zerocoins
+        self.NUM_BLOCKS = 2         # Number of spammed blocks
+
+        # 1) Starting mining blocks
+        self.log.info("Mining %d blocks to get to zPOS activation...." % INITAL_MINED_BLOCKS)
+        self.node.generate(INITAL_MINED_BLOCKS)
+        sleep(2)
+
+        # 2) Collect the possible prevouts and mint zerocoins with those
+        self.log.info("Collecting all unspent coins which we generated from mining...")
+        balance = self.node.getbalance("*", 100)
+        self.log.info("Minting zerocoins...")
+        initial_mints = 0
+        while balance > DENOM_TO_USE:
+            try:
+                self.node.mintzerocoin(DENOM_TO_USE)
+            except JSONRPCException:
+                break
+            sleep(1)
+            initial_mints += 1
+            self.node.generate(1)
+            sleep(1)
+
+            if initial_mints % 5 == 0:
+                self.log.info("Minted %d coins" % initial_mints)
+            if initial_mints >= 70:
+                break
+            balance = self.node.getbalance("*", 100)
+        self.log.info("Minted %d coins in the %d-denom, remaining balance %d", initial_mints, DENOM_TO_USE, balance)
+        sleep(2)
+
+        # 3) mine more blocks
+        self.log.info("Mining %d more blocks ... and getting spendable zerocoins" % MORE_MINED_BLOCKS)
+        self.node.generate(MORE_MINED_BLOCKS)
+        sleep(2)
+        mints = self.node.listmintedzerocoins(True, True)
+        mints_hashes = [x["serial hash"] for x in mints]
+
+        # This mints are not ready spendable, only few of them.
+        self.log.info("Got %d confirmed mints" % len(mints_hashes))
+
+        # 4) spend mints
+        self.log.info("Spending mints in block %d..." % self.node.getblockcount())
+        spends = 0
+        spent_mints = []
+        for mint in mints_hashes:
+            # create a single element list to pass to RPC spendzerocoinmints
+            mint_arg = []
+            mint_arg.append(mint)
+            try:
+                self.node.spendzerocoinmints(mint_arg)
+                sleep(1)
+                spends += 1
+                spent_mints.append(mint)
+            except JSONRPCException as e:
+                self.log.warning(str(e))
+                continue
+        sleep(1)
+        self.log.info("Successfully spent %d mints" % spends)
+
+        # 5) Start mining again so that spends get confirmed in a block.
+        self.log.info("Mining 5 more blocks...")
+        self.node.generate(5)
+        sleep(2)
+
+        # 6) Collect some prevouts for random txes
+        self.log.info("Collecting inputs for txes...")
+        spending_utxo_list = self.node.listunspent()
+        sleep(1)
+
+        # 7) Create "Fake Stake" blocks and send them
+        self.log.info("Creating Fake stake zPoS blocks...")
+        err_msgs = self.test_spam("Main", mints, spending_utxo_list=spending_utxo_list, fZPoS=True)
+
+        if not len(err_msgs) == 0:
+            self.log.error("result: " + " | ".join(err_msgs))
+            raise AssertionError("TEST FAILED")
+
+        self.log.info("%s PASSED" % self.__class__.__name__)
+
+if __name__ == '__main__':
+    zPoSFakeStake().main()

test/functional/p2p_zpos_fakestake_accepted.py

@@ -0,0 +1,117 @@
+#!/usr/bin/env python3
+# Copyright (c) 2019 The PIVX developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+'''
+Performs the same check as in Test_02 verifying that zPoS forked blocks that stake a zerocoin which is spent on mainchain on an higher block are still accepted.
+'''
+
+from test_framework.authproxy import JSONRPCException
+from fake_stake.base_test import Agrarian_FakeStakeTest
+from time import sleep
+
+class zPoSFakeStakeAccepted(Agrarian_FakeStakeTest):
+
+
+    def set_test_params(self):
+        ''' Setup test environment
+        :param:
+        :return:
+        '''
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+        self.extra_args = [['-staking=1', '-debug=net', '-zagrstake']] * self.num_nodes
+
+    def run_test(self):
+        self.description = "Performs the same check as in Test_02 verifying that zPoS forked blocks that stake a zerocoin which is spent on mainchain on an higher block are still accepted."
+        self.init_test()
+
+        DENOM_TO_USE = 1000  # zc denomination
+        INITAL_MINED_BLOCKS = 321
+        MORE_MINED_BLOCKS = 301
+        FORK_DEPTH = 75
+        self.NUM_BLOCKS = 2
+
+        # 1) Starting mining blocks
+        self.log.info("Mining %d blocks to get to zPOS activation...." % INITAL_MINED_BLOCKS)
+        self.node.generate(INITAL_MINED_BLOCKS)
+        sleep(2)
+
+        # 2) Collect the possible prevouts and mint zerocoins with those
+        self.log.info("Collecting all unspent coins which we generated from mining...")
+        balance = self.node.getbalance("*", 100)
+        self.log.info("Minting zerocoins...")
+        initial_mints = 0
+        while balance > DENOM_TO_USE:
+            try:
+                self.node.mintzerocoin(DENOM_TO_USE)
+            except JSONRPCException:
+                break
+            sleep(1)
+            initial_mints += 1
+            self.node.generate(1)
+            sleep(1)
+
+            if initial_mints % 5 == 0:
+                self.log.info("Minted %d coins" % initial_mints)
+            if initial_mints >= 20:
+                break
+            balance = self.node.getbalance("*", 100)
+        self.log.info("Minted %d coins in the %d-denom, remaining balance %d", initial_mints, DENOM_TO_USE, balance)
+        sleep(2)
+
+        # 3) mine more blocks
+        self.log.info("Mining %d more blocks ... and getting spendable zerocoins" % MORE_MINED_BLOCKS)
+        self.node.generate(MORE_MINED_BLOCKS)
+        sleep(2)
+        mints = self.node.listmintedzerocoins(True, True)
+        sleep(1)
+        mints_hashes = [x["serial hash"] for x in mints]
+
+        # This mints are not ready spendable, only few of them.
+        self.log.info("Got %d confirmed mints" % len(mints_hashes))
+
+        # 4) Start mining again so that spends get confirmed in a block.
+        self.log.info("Mining 200 more blocks...")
+        self.node.generate(200)
+        sleep(2)
+
+        # 5) spend mints
+        self.log.info("Spending mints in block %d..." % self.node.getblockcount())
+        spends = 0
+        for mint in mints_hashes:
+            # create a single element list to pass to RPC spendzerocoinmints
+            mint_arg = []
+            mint_arg.append(mint)
+            try:
+                self.node.spendzerocoinmints(mint_arg)
+                sleep(1)
+                spends += 1
+            except JSONRPCException as e:
+                self.log.warning(str(e))
+                continue
+        sleep(1)
+        self.log.info("Successfully spent %d mints" % spends)
+
+        self.log.info("Mining 6 more blocks...")
+        self.node.generate(6)
+        sleep(2)
+
+        # 6) Collect some prevouts for random txes
+        self.log.info("Collecting inputs for txes...")
+        utxo_list = self.node.listunspent()
+        sleep(1)
+
+        # 7) Create valid forked zPoS blocks and send them
+        self.log.info("Creating stake zPoS blocks...")
+        err_msgs = self.test_spam("Fork", mints, spending_utxo_list=utxo_list, fZPoS=True, fRandomHeight=True, randomRange=FORK_DEPTH, randomRange2=50, fMustPass=True)
+
+        if not len(err_msgs) == 0:
+            self.log.error("result: " + " | ".join(err_msgs))
+            raise AssertionError("TEST FAILED")
+
+        self.log.info("%s PASSED" % self.__class__.__name__)
+
+if __name__ == '__main__':
+    zPoSFakeStakeAccepted().main()

test/functional/rpc_bind.py

@@ -0,0 +1,107 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test running bitcoind with the -rpcbind and -rpcallowip options."""
+
+import socket
+import sys
+
+from test_framework.test_framework import BitcoinTestFramework, SkipTest
+from test_framework.util import *
+from test_framework.netutil import *
+
+class RPCBindTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+
+    def setup_network(self):
+        self.add_nodes(self.num_nodes, None)
+
+    def run_bind_test(self, allow_ips, connect_to, addresses, expected):
+        '''
+        Start a node with requested rpcallowip and rpcbind parameters,
+        then try to connect, and check if the set of bound addresses
+        matches the expected set.
+        '''
+        self.log.info("Bind test for %s" % str(addresses))
+        expected = [(addr_to_hex(addr), port) for (addr, port) in expected]
+        base_args = ['-disablewallet', '-nolisten']
+        if allow_ips:
+            base_args += ['-rpcallowip=' + x for x in allow_ips]
+        binds = ['-rpcbind='+addr for addr in addresses]
+        self.nodes[0].rpchost = connect_to
+        self.start_node(0, base_args + binds)
+        pid = self.nodes[0].process.pid
+        assert_equal(set(get_bind_addrs(pid)), set(expected))
+        self.stop_nodes()
+
+    def run_allowip_test(self, allow_ips, rpchost, rpcport):
+        '''
+        Start a node with rpcallow IP, and request getnetworkinfo
+        at a non-localhost IP.
+        '''
+        self.log.info("Allow IP test for %s:%d" % (rpchost, rpcport))
+        base_args = ['-disablewallet', '-nolisten'] + ['-rpcallowip='+x for x in allow_ips]
+        self.nodes[0].rpchost = None
+        self.start_nodes([base_args])
+        # connect to node through non-loopback interface
+        node = get_rpc_proxy(rpc_url(get_datadir_path(self.options.tmpdir, 0), 0, "%s:%d" % (rpchost, rpcport)), 0, coveragedir=self.options.coveragedir)
+        node.getnetworkinfo()
+        self.stop_nodes()
+
+    def run_test(self):
+        # due to OS-specific network stats queries, this test works only on Linux
+        if not sys.platform.startswith('linux'):
+            raise SkipTest("This test can only be run on linux.")
+        # find the first non-loopback interface for testing
+        non_loopback_ip = None
+        for name,ip in all_interfaces():
+            if ip != '127.0.0.1':
+                non_loopback_ip = ip
+                break
+        if non_loopback_ip is None:
+            raise SkipTest("This test requires at least one non-loopback IPv4 interface.")
+        try:
+            s = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
+            s.connect(("::1",1))
+            s.close
+        except OSError:
+            raise SkipTest("This test requires IPv6 support.")
+
+        self.log.info("Using interface %s for testing" % non_loopback_ip)
+
+        defaultport = rpc_port(0)
+
+        # check default without rpcallowip (IPv4 and IPv6 localhost)
+        self.run_bind_test(None, '127.0.0.1', [],
+            [('127.0.0.1', defaultport), ('::1', defaultport)])
+        # check default with rpcallowip (IPv6 any)
+        self.run_bind_test(['127.0.0.1'], '127.0.0.1', [],
+            [('::0', defaultport)])
+        # check only IPv4 localhost (explicit)
+        self.run_bind_test(['127.0.0.1'], '127.0.0.1', ['127.0.0.1'],
+            [('127.0.0.1', defaultport)])
+        # check only IPv4 localhost (explicit) with alternative port
+        self.run_bind_test(['127.0.0.1'], '127.0.0.1:32171', ['127.0.0.1:32171'],
+            [('127.0.0.1', 32171)])
+        # check only IPv4 localhost (explicit) with multiple alternative ports on same host
+        self.run_bind_test(['127.0.0.1'], '127.0.0.1:32171', ['127.0.0.1:32171', '127.0.0.1:32172'],
+            [('127.0.0.1', 32171), ('127.0.0.1', 32172)])
+        # check only IPv6 localhost (explicit)
+        self.run_bind_test(['[::1]'], '[::1]', ['[::1]'],
+            [('::1', defaultport)])
+        # check both IPv4 and IPv6 localhost (explicit)
+        self.run_bind_test(['127.0.0.1'], '127.0.0.1', ['127.0.0.1', '[::1]'],
+            [('127.0.0.1', defaultport), ('::1', defaultport)])
+        # check only non-loopback interface
+        self.run_bind_test([non_loopback_ip], non_loopback_ip, [non_loopback_ip],
+            [(non_loopback_ip, defaultport)])
+
+        # Check that with invalid rpcallowip, we are denied
+        self.run_allowip_test([non_loopback_ip], non_loopback_ip, defaultport)
+        assert_raises_rpc_error(-342, "non-JSON HTTP response with '403 Forbidden' from server", self.run_allowip_test, ['1.1.1.1'], non_loopback_ip, defaultport)
+
+if __name__ == '__main__':
+    RPCBindTest().main()

test/functional/rpc_bip38.py

@@ -0,0 +1,27 @@
+#!/usr/bin/env python3
+# Copyright (c) 2018 The PIVX developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test RPC commands for BIP38 encrypting and decrypting addresses."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import assert_equal
+
+class Bip38Test(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 2
+
+    def run_test(self):
+        password = 'test'
+        address = self.nodes[0].getnewaddress()
+        privkey = self.nodes[0].dumpprivkey(address)
+
+        self.log.info('encrypt address %s' % (address))
+        bip38key = self.nodes[0].bip38encrypt(address, password)['Encrypted Key']
+
+        self.log.info('decrypt bip38 key %s' % (bip38key))
+        assert_equal(self.nodes[1].bip38decrypt(bip38key, password)['Address'], address)
+
+if __name__ == '__main__':
+    Bip38Test().main()

test/functional/rpc_blockchain.py

@@ -0,0 +1,109 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test RPCs related to blockchainstate.
+
+Test the following RPCs:
+    - getblockchaininfo
+    - gettxoutsetinfo
+    - getdifficulty
+    - getbestblockhash
+    - getblockhash
+    - getblockheader
+    - getchaintxstats
+    - getnetworkhashps
+    - verifychain
+
+Tests correspond to code in rpc/blockchain.cpp.
+"""
+
+from decimal import Decimal
+import http.client
+import subprocess
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (
+    assert_equal,
+    assert_greater_than,
+    assert_greater_than_or_equal,
+    assert_raises,
+    assert_raises_rpc_error,
+    assert_is_hex_string,
+    assert_is_hash_string,
+)
+
+class BlockchainTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 1
+        self.extra_args = [['-stopatheight=207', '-prune=1']]
+
+    def run_test(self):
+        #self._test_getblockchaininfo()
+        self._test_gettxoutsetinfo()
+        self._test_getblockheader()
+        #self._test_getdifficulty()
+        self.nodes[0].verifychain(0)
+
+    def _test_getblockchaininfo(self):
+        self.log.info("Test getblockchaininfo")
+
+        keys = [
+            'bestblockhash',
+            'blocks',
+            'chain',
+            'chainwork',
+            'difficulty',
+            'headers',
+            'verificationprogress',
+            'warnings',
+        ]
+        res = self.nodes[0].getblockchaininfo()
+        # result should have these additional pruning keys if manual pruning is enabled
+        assert_equal(sorted(res.keys()), sorted(keys))
+    def _test_gettxoutsetinfo(self):
+        node = self.nodes[0]
+        res = node.gettxoutsetinfo()
+
+        assert_equal(res['total_amount'], Decimal('50000.00000000'))
+        assert_equal(res['transactions'], 200)
+        assert_equal(res['height'], 200)
+        assert_equal(res['txouts'], 200)
+        assert_equal(res['bytes_serialized'], 14073),
+        assert_equal(len(res['bestblock']), 64)
+        assert_equal(len(res['hash_serialized']), 64)
+
+    def _test_getblockheader(self):
+        node = self.nodes[0]
+
+        assert_raises_rpc_error(-5, "Block not found",
+                              node.getblockheader, "nonsense")
+
+        besthash = node.getbestblockhash()
+        secondbesthash = node.getblockhash(199)
+        header = node.getblockheader(besthash)
+
+        assert_equal(header['hash'], besthash)
+        assert_equal(header['height'], 200)
+        assert_equal(header['confirmations'], 1)
+        assert_equal(header['previousblockhash'], secondbesthash)
+        assert_is_hex_string(header['chainwork'])
+        assert_is_hash_string(header['hash'])
+        assert_is_hash_string(header['previousblockhash'])
+        assert_is_hash_string(header['merkleroot'])
+        assert_is_hash_string(header['bits'], length=None)
+        assert isinstance(header['time'], int)
+        #assert isinstance(header['mediantime'], int)
+        assert isinstance(header['nonce'], int)
+        assert isinstance(header['version'], int)
+        #assert isinstance(int(header['versionHex'], 16), int)
+        assert isinstance(header['difficulty'], Decimal)
+
+    def _test_getdifficulty(self):
+        difficulty = self.nodes[0].getdifficulty()
+        # 1 hash in 2 should be valid, so difficulty should be 1/2**31
+        # binary => decimal => binary math is why we do this check
+        assert abs(difficulty * 2**31 - 1) < 0.0001
+
+if __name__ == '__main__':
+    BlockchainTest().main()

test/functional/rpc_decodescript.py

@@ -0,0 +1,179 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test decoding scripts via decodescript RPC command."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+from test_framework.mininode import *
+from io import BytesIO
+
+class DecodeScriptTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+
+    def decodescript_script_sig(self):
+        signature = '304502207fa7a6d1e0ee81132a269ad84e68d695483745cde8b541e3bf630749894e342a022100c1f7ab20e13e22fb95281a870f3dcf38d782e53023ee313d741ad0cfbc0c509001'
+        push_signature = '48' + signature
+        public_key = '03b0da749730dc9b4b1f4a14d6902877a92541f5368778853d9c4a0cb7802dcfb2'
+        push_public_key = '21' + public_key
+
+        # below are test cases for all of the standard transaction types
+
+        # 1) P2PK scriptSig
+        # the scriptSig of a public key scriptPubKey simply pushes a signature onto the stack
+        rpc_result = self.nodes[0].decodescript(push_signature)
+        assert_equal(signature, rpc_result['asm'])
+
+        # 2) P2PKH scriptSig
+        rpc_result = self.nodes[0].decodescript(push_signature + push_public_key)
+        assert_equal(signature + ' ' + public_key, rpc_result['asm'])
+
+        # 3) multisig scriptSig
+        # this also tests the leading portion of a P2SH multisig scriptSig
+        # OP_0 <A sig> <B sig>
+        rpc_result = self.nodes[0].decodescript('00' + push_signature + push_signature)
+        assert_equal('0 ' + signature + ' ' + signature, rpc_result['asm'])
+
+        # 4) P2SH scriptSig
+        # an empty P2SH redeemScript is valid and makes for a very simple test case.
+        # thus, such a spending scriptSig would just need to pass the outer redeemScript
+        # hash test and leave true on the top of the stack.
+        rpc_result = self.nodes[0].decodescript('5100')
+        assert_equal('1 0', rpc_result['asm'])
+
+        # 5) null data scriptSig - no such thing because null data scripts can not be spent.
+        # thus, no test case for that standard transaction type is here.
+
+    def decodescript_script_pub_key(self):
+        public_key = '03b0da749730dc9b4b1f4a14d6902877a92541f5368778853d9c4a0cb7802dcfb2'
+        push_public_key = '21' + public_key
+        public_key_hash = '11695b6cd891484c2d49ec5aa738ec2b2f897777'
+        push_public_key_hash = '14' + public_key_hash
+
+        # below are test cases for all of the standard transaction types
+
+        # 1) P2PK scriptPubKey
+        # <pubkey> OP_CHECKSIG
+        rpc_result = self.nodes[0].decodescript(push_public_key + 'ac')
+        assert_equal(public_key + ' OP_CHECKSIG', rpc_result['asm'])
+
+        # 2) P2PKH scriptPubKey
+        # OP_DUP OP_HASH160 <PubKeyHash> OP_EQUALVERIFY OP_CHECKSIG
+        rpc_result = self.nodes[0].decodescript('76a9' + push_public_key_hash + '88ac')
+        assert_equal('OP_DUP OP_HASH160 ' + public_key_hash + ' OP_EQUALVERIFY OP_CHECKSIG', rpc_result['asm'])
+
+        # 3) multisig scriptPubKey
+        # <m> <A pubkey> <B pubkey> <C pubkey> <n> OP_CHECKMULTISIG
+        # just imagine that the pub keys used below are different.
+        # for our purposes here it does not matter that they are the same even though it is unrealistic.
+        rpc_result = self.nodes[0].decodescript('52' + push_public_key + push_public_key + push_public_key + '53ae')
+        assert_equal('2 ' + public_key + ' ' + public_key + ' ' + public_key +  ' 3 OP_CHECKMULTISIG', rpc_result['asm'])
+
+        # 4) P2SH scriptPubKey
+        # OP_HASH160 <Hash160(redeemScript)> OP_EQUAL.
+        # push_public_key_hash here should actually be the hash of a redeem script.
+        # but this works the same for purposes of this test.
+        rpc_result = self.nodes[0].decodescript('a9' + push_public_key_hash + '87')
+        assert_equal('OP_HASH160 ' + public_key_hash + ' OP_EQUAL', rpc_result['asm'])
+
+        # 5) null data scriptPubKey
+        # use a signature look-alike here to make sure that we do not decode random data as a signature.
+        # this matters if/when signature sighash decoding comes along.
+        # would want to make sure that no such decoding takes place in this case.
+        signature_imposter = '48304502207fa7a6d1e0ee81132a269ad84e68d695483745cde8b541e3bf630749894e342a022100c1f7ab20e13e22fb95281a870f3dcf38d782e53023ee313d741ad0cfbc0c509001'
+        # OP_RETURN <data>
+        rpc_result = self.nodes[0].decodescript('6a' + signature_imposter)
+        assert_equal('OP_RETURN ' + signature_imposter[2:], rpc_result['asm'])
+
+        # 6) a CLTV redeem script. redeem scripts are in-effect scriptPubKey scripts, so adding a test here.
+        # OP_NOP2 is also known as OP_CHECKLOCKTIMEVERIFY.
+        # just imagine that the pub keys used below are different.
+        # for our purposes here it does not matter that they are the same even though it is unrealistic.
+        #
+        # OP_IF
+        #   <receiver-pubkey> OP_CHECKSIGVERIFY
+        # OP_ELSE
+        #   <lock-until> OP_CHECKLOCKTIMEVERIFY OP_DROP
+        # OP_ENDIF
+        # <sender-pubkey> OP_CHECKSIG
+        #
+        # lock until block 500,000
+        rpc_result = self.nodes[0].decodescript('63' + push_public_key + 'ad670320a107b17568' + push_public_key + 'ac')
+        assert_equal('OP_IF ' + public_key + ' OP_CHECKSIGVERIFY OP_ELSE 500000 OP_NOP2 OP_DROP OP_ENDIF ' + public_key + ' OP_CHECKSIG', rpc_result['asm'])
+
+    def decoderawtransaction_asm_sighashtype(self):
+        """Test decoding scripts via RPC command "decoderawtransaction".
+
+        This test is in with the "decodescript" tests because they are testing the same "asm" script decodes.
+        """
+
+        # this test case uses a random plain vanilla mainnet transaction with a single P2PKH input and output
+        tx = '0100000001696a20784a2c70143f634e95227dbdfdf0ecd51647052e70854512235f5986ca010000008a47304402207174775824bec6c2700023309a168231ec80b82c6069282f5133e6f11cbb04460220570edc55c7c5da2ca687ebd0372d3546ebc3f810516a002350cac72dfe192dfb014104d3f898e6487787910a690410b7a917ef198905c27fb9d3b0a42da12aceae0544fc7088d239d9a48f2828a15a09e84043001f27cc80d162cb95404e1210161536ffffffff0100e1f505000000001976a914eb6c6e0cdb2d256a32d97b8df1fc75d1920d9bca88ac00000000'
+        rpc_result = self.nodes[0].decoderawtransaction(tx)
+        assert_equal('304402207174775824bec6c2700023309a168231ec80b82c6069282f5133e6f11cbb04460220570edc55c7c5da2ca687ebd0372d3546ebc3f810516a002350cac72dfe192dfb01 04d3f898e6487787910a690410b7a917ef198905c27fb9d3b0a42da12aceae0544fc7088d239d9a48f2828a15a09e84043001f27cc80d162cb95404e1210161536', rpc_result['vin'][0]['scriptSig']['asm'])
+
+        # this test case uses a mainnet transaction that has a P2SH input and both P2PKH and P2SH outputs.
+        # it's from James D'Angelo's awesome introductory videos about multisig: https://www.youtube.com/watch?v=zIbUSaZBJgU and https://www.youtube.com/watch?v=OSA1pwlaypc
+        # verify that we have not altered scriptPubKey decoding.
+        tx = '01000000018d1f5635abd06e2c7e2ddf58dc85b3de111e4ad6e0ab51bb0dcf5e84126d927300000000fdfe0000483045022100ae3b4e589dfc9d48cb82d41008dc5fa6a86f94d5c54f9935531924602730ab8002202f88cf464414c4ed9fa11b773c5ee944f66e9b05cc1e51d97abc22ce098937ea01483045022100b44883be035600e9328a01b66c7d8439b74db64187e76b99a68f7893b701d5380220225bf286493e4c4adcf928c40f785422572eb232f84a0b83b0dea823c3a19c75014c695221020743d44be989540d27b1b4bbbcfd17721c337cb6bc9af20eb8a32520b393532f2102c0120a1dda9e51a938d39ddd9fe0ebc45ea97e1d27a7cbd671d5431416d3dd87210213820eb3d5f509d7438c9eeecb4157b2f595105e7cd564b3cdbb9ead3da41eed53aeffffffff02611e0000000000001976a914dc863734a218bfe83ef770ee9d41a27f824a6e5688acee2a02000000000017a9142a5edea39971049a540474c6a99edf0aa4074c588700000000'
+        rpc_result = self.nodes[0].decoderawtransaction(tx)
+        assert_equal('8e3730608c3b0bb5df54f09076e196bc292a8e39a78e73b44b6ba08c78f5cbb0', rpc_result['txid'])
+        assert_equal('0 3045022100ae3b4e589dfc9d48cb82d41008dc5fa6a86f94d5c54f9935531924602730ab8002202f88cf464414c4ed9fa11b773c5ee944f66e9b05cc1e51d97abc22ce098937ea01 3045022100b44883be035600e9328a01b66c7d8439b74db64187e76b99a68f7893b701d5380220225bf286493e4c4adcf928c40f785422572eb232f84a0b83b0dea823c3a19c7501 5221020743d44be989540d27b1b4bbbcfd17721c337cb6bc9af20eb8a32520b393532f2102c0120a1dda9e51a938d39ddd9fe0ebc45ea97e1d27a7cbd671d5431416d3dd87210213820eb3d5f509d7438c9eeecb4157b2f595105e7cd564b3cdbb9ead3da41eed53ae', rpc_result['vin'][0]['scriptSig']['asm'])
+        assert_equal('OP_DUP OP_HASH160 dc863734a218bfe83ef770ee9d41a27f824a6e56 OP_EQUALVERIFY OP_CHECKSIG', rpc_result['vout'][0]['scriptPubKey']['asm'])
+        assert_equal('OP_HASH160 2a5edea39971049a540474c6a99edf0aa4074c58 OP_EQUAL', rpc_result['vout'][1]['scriptPubKey']['asm'])
+        txSave = CTransaction()
+        txSave.deserialize(BytesIO(hex_str_to_bytes(tx)))
+
+        # make sure that a specifically crafted op_return value will not pass all the IsDERSignature checks and then get decoded as a sighash type
+        tx = '01000000015ded05872fdbda629c7d3d02b194763ce3b9b1535ea884e3c8e765d42e316724020000006b48304502204c10d4064885c42638cbff3585915b322de33762598321145ba033fc796971e2022100bb153ad3baa8b757e30a2175bd32852d2e1cb9080f84d7e32fcdfd667934ef1b012103163c0ff73511ea1743fb5b98384a2ff09dd06949488028fd819f4d83f56264efffffffff0200000000000000000b6a0930060201000201000180380100000000001976a9141cabd296e753837c086da7a45a6c2fe0d49d7b7b88ac00000000'
+        rpc_result = self.nodes[0].decoderawtransaction(tx)
+        assert_equal('OP_RETURN 300602010002010001', rpc_result['vout'][0]['scriptPubKey']['asm'])
+
+        # verify that we have not altered scriptPubKey processing even of a specially crafted P2PKH pubkeyhash and P2SH redeem script hash that is made to pass the der signature checks
+        tx = '01000000018d1f5635abd06e2c7e2ddf58dc85b3de111e4ad6e0ab51bb0dcf5e84126d927300000000fdfe0000483045022100ae3b4e589dfc9d48cb82d41008dc5fa6a86f94d5c54f9935531924602730ab8002202f88cf464414c4ed9fa11b773c5ee944f66e9b05cc1e51d97abc22ce098937ea01483045022100b44883be035600e9328a01b66c7d8439b74db64187e76b99a68f7893b701d5380220225bf286493e4c4adcf928c40f785422572eb232f84a0b83b0dea823c3a19c75014c695221020743d44be989540d27b1b4bbbcfd17721c337cb6bc9af20eb8a32520b393532f2102c0120a1dda9e51a938d39ddd9fe0ebc45ea97e1d27a7cbd671d5431416d3dd87210213820eb3d5f509d7438c9eeecb4157b2f595105e7cd564b3cdbb9ead3da41eed53aeffffffff02611e0000000000001976a914301102070101010101010102060101010101010188acee2a02000000000017a91430110207010101010101010206010101010101018700000000'
+        rpc_result = self.nodes[0].decoderawtransaction(tx)
+        assert_equal('OP_DUP OP_HASH160 3011020701010101010101020601010101010101 OP_EQUALVERIFY OP_CHECKSIG', rpc_result['vout'][0]['scriptPubKey']['asm'])
+        assert_equal('OP_HASH160 3011020701010101010101020601010101010101 OP_EQUAL', rpc_result['vout'][1]['scriptPubKey']['asm'])
+
+        # some more full transaction tests of varying specific scriptSigs. used instead of
+        # tests in decodescript_script_sig because the decodescript RPC is specifically
+        # for working on scriptPubKeys (argh!).
+        push_signature = bytes_to_hex_str(txSave.vin[0].scriptSig)[2:(0x48*2+4)]
+        signature = push_signature[2:]
+        der_signature = signature[:-2]
+        signature_sighash_decoded = der_signature + '01'
+        signature_2 = der_signature + '82'
+        push_signature_2 = '48' + signature_2
+        signature_2_sighash_decoded = der_signature + '82'
+
+        # 1) P2PK scriptSig
+        txSave.vin[0].scriptSig = hex_str_to_bytes(push_signature)
+        rpc_result = self.nodes[0].decoderawtransaction(bytes_to_hex_str(txSave.serialize()))
+        assert_equal(signature_sighash_decoded, rpc_result['vin'][0]['scriptSig']['asm'])
+
+        # make sure that the sighash decodes come out correctly for a more complex / lesser used case.
+        txSave.vin[0].scriptSig = hex_str_to_bytes(push_signature_2)
+        rpc_result = self.nodes[0].decoderawtransaction(bytes_to_hex_str(txSave.serialize()))
+        assert_equal(signature_2_sighash_decoded, rpc_result['vin'][0]['scriptSig']['asm'])
+
+        # 2) multisig scriptSig
+        txSave.vin[0].scriptSig = hex_str_to_bytes('00' + push_signature + push_signature_2)
+        rpc_result = self.nodes[0].decoderawtransaction(bytes_to_hex_str(txSave.serialize()))
+        assert_equal('0 ' + signature_sighash_decoded + ' ' + signature_2_sighash_decoded, rpc_result['vin'][0]['scriptSig']['asm'])
+
+        # 3) test a scriptSig that contains more than push operations.
+        # in fact, it contains an OP_RETURN with data specially crafted to cause improper decode if the code does not catch it.
+        txSave.vin[0].scriptSig = hex_str_to_bytes('6a143011020701010101010101020601010101010101')
+        rpc_result = self.nodes[0].decoderawtransaction(bytes_to_hex_str(txSave.serialize()))
+        assert_equal('OP_RETURN 3011020701010101010101020601010101010101', rpc_result['vin'][0]['scriptSig']['asm'])
+
+    def run_test(self):
+        self.decodescript_script_sig()
+        self.decodescript_script_pub_key()
+        self.decoderawtransaction_asm_sighashtype()
+
+if __name__ == '__main__':
+    DecodeScriptTest().main()

test/functional/rpc_deprecated.py

@@ -0,0 +1,27 @@
+#!/usr/bin/env python3
+# Copyright (c) 2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test deprecation of RPC calls."""
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import assert_raises_rpc_error
+
+class DeprecatedRpcTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+        self.setup_clean_chain = True
+        self.extra_args = [[], ["-deprecatedrpc=estimatefee", "-deprecatedrpc=createmultisig"]]
+
+    def run_test(self):
+        self.log.info("estimatefee: Shows deprecated message")
+        assert_raises_rpc_error(-32, 'estimatefee is deprecated', self.nodes[0].estimatefee, 1)
+
+        self.log.info("Using -deprecatedrpc=estimatefee bypasses the error")
+        self.nodes[1].estimatefee(1)
+
+        self.log.info("Make sure that -deprecatedrpc=createmultisig allows it to take addresses")
+        assert_raises_rpc_error(-5, "Invalid public key", self.nodes[0].createmultisig, 1, [self.nodes[0].getnewaddress()])
+        self.nodes[1].createmultisig(1, [self.nodes[1].getnewaddress()])
+
+if __name__ == '__main__':
+    DeprecatedRpcTest().main()

test/functional/rpc_fundrawtransaction.py

@@ -0,0 +1,664 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2016 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+
+def get_unspent(listunspent, amount):
+    for utx in listunspent:
+        if utx['amount'] == amount:
+            return utx
+    raise AssertionError('Could not find unspent with amount={}'.format(amount))
+
+
+class RawTransactionsTest(BitcoinTestFramework):
+
+    def __init__(self):
+        super().__init__()
+        self.setup_clean_chain = True
+        self.num_nodes = 4
+
+    def setup_network(self, split=False):
+        self.nodes = start_nodes(self.num_nodes, self.options.tmpdir)
+
+        connect_nodes_bi(self.nodes,0,1)
+        connect_nodes_bi(self.nodes,1,2)
+        connect_nodes_bi(self.nodes,0,2)
+        connect_nodes_bi(self.nodes,0,3)
+
+        self.is_network_split=False
+        self.sync_all()
+
+    def run_test(self):
+        print("Mining blocks...")
+
+        min_relay_tx_fee = self.nodes[0].getnetworkinfo()['relayfee']
+        # This test is not meant to test fee estimation and we'd like
+        # to be sure all txs are sent at a consistent desired feerate
+        for node in self.nodes:
+            node.settxfee(min_relay_tx_fee)
+
+        # if the fee's positive delta is higher than this value tests will fail,
+        # neg. delta always fail the tests.
+        # The size of the signature of every input may be at most 2 bytes larger
+        # than a minimum sized signature.
+
+        #            = 2 bytes * minRelayTxFeePerByte
+        feeTolerance = 2 * min_relay_tx_fee/1000
+
+        self.nodes[2].generate(1)
+        self.sync_all()
+        self.nodes[0].generate(121)
+        self.sync_all()
+
+        watchonly_address = self.nodes[0].getnewaddress()
+        watchonly_pubkey = self.nodes[0].validateaddress(watchonly_address)["pubkey"]
+        watchonly_amount = Decimal(200)
+        self.nodes[3].importpubkey(watchonly_pubkey, "", True)
+        watchonly_txid = self.nodes[0].sendtoaddress(watchonly_address, watchonly_amount)
+        self.nodes[0].sendtoaddress(self.nodes[3].getnewaddress(), watchonly_amount / 10)
+
+        self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), 1.5)
+        self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), 1.0)
+        self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), 5.0)
+
+        self.nodes[0].generate(1)
+        self.sync_all()
+
+        ###############
+        # simple test #
+        ###############
+        inputs  = [ ]
+        outputs = { self.nodes[0].getnewaddress() : 1.0 }
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+        rawtxfund = self.nodes[2].fundrawtransaction(rawtx)
+        fee = rawtxfund['fee']
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtxfund['hex'])
+        assert(len(dec_tx['vin']) > 0) #test if we have enought inputs
+
+        ##############################
+        # simple test with two coins #
+        ##############################
+        inputs  = [ ]
+        outputs = { self.nodes[0].getnewaddress() : 2.2 }
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+
+        rawtxfund = self.nodes[2].fundrawtransaction(rawtx)
+        fee = rawtxfund['fee']
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtxfund['hex'])
+        assert(len(dec_tx['vin']) > 0) #test if we have enough inputs
+
+        ##############################
+        # simple test with two coins #
+        ##############################
+        inputs  = [ ]
+        outputs = { self.nodes[0].getnewaddress() : 2.6 }
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+
+        rawtxfund = self.nodes[2].fundrawtransaction(rawtx)
+        fee = rawtxfund['fee']
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtxfund['hex'])
+        assert(len(dec_tx['vin']) > 0)
+        assert_equal(dec_tx['vin'][0]['scriptSig']['hex'], '')
+
+
+        ################################
+        # simple test with two outputs #
+        ################################
+        inputs  = [ ]
+        outputs = { self.nodes[0].getnewaddress() : 2.6, self.nodes[1].getnewaddress() : 2.5 }
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+
+        rawtxfund = self.nodes[2].fundrawtransaction(rawtx)
+        fee = rawtxfund['fee']
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtxfund['hex'])
+        totalOut = 0
+        for out in dec_tx['vout']:
+            totalOut += out['value']
+
+        assert(len(dec_tx['vin']) > 0)
+        assert_equal(dec_tx['vin'][0]['scriptSig']['hex'], '')
+
+
+        #########################################################################
+        # test a fundrawtransaction with a VIN greater than the required amount #
+        #########################################################################
+        utx = get_unspent(self.nodes[2].listunspent(), 5)
+
+        inputs  = [ {'txid' : utx['txid'], 'vout' : utx['vout']}]
+        outputs = { self.nodes[0].getnewaddress() : 1.0 }
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+        assert_equal(utx['txid'], dec_tx['vin'][0]['txid'])
+
+        rawtxfund = self.nodes[2].fundrawtransaction(rawtx)
+        fee = rawtxfund['fee']
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtxfund['hex'])
+        totalOut = 0
+        for out in dec_tx['vout']:
+            totalOut += out['value']
+
+        assert_equal(fee + totalOut, utx['amount']) #compare vin total and totalout+fee
+
+
+        #####################################################################
+        # test a fundrawtransaction with which will not get a change output #
+        #####################################################################
+        utx = get_unspent(self.nodes[2].listunspent(), 5)
+
+        inputs  = [ {'txid' : utx['txid'], 'vout' : utx['vout']}]
+        outputs = { self.nodes[0].getnewaddress() : Decimal(5.0) - fee - feeTolerance }
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+        assert_equal(utx['txid'], dec_tx['vin'][0]['txid'])
+
+        rawtxfund = self.nodes[2].fundrawtransaction(rawtx)
+        fee = rawtxfund['fee']
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtxfund['hex'])
+        totalOut = 0
+        for out in dec_tx['vout']:
+            totalOut += out['value']
+
+        assert_equal(rawtxfund['changepos'], -1)
+        assert_equal(fee + totalOut, utx['amount']) #compare vin total and totalout+fee
+
+
+        ####################################################
+        # test a fundrawtransaction with an invalid option #
+        ####################################################
+        utx = get_unspent(self.nodes[2].listunspent(), 5)
+
+        inputs  = [ {'txid' : utx['txid'], 'vout' : utx['vout']} ]
+        outputs = { self.nodes[0].getnewaddress() : Decimal(4.0) }
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+        assert_equal(utx['txid'], dec_tx['vin'][0]['txid'])
+
+        try:
+            self.nodes[2].fundrawtransaction(rawtx, {'foo': 'bar'})
+            raise AssertionError("Accepted invalid option foo")
+        except JSONRPCException as e:
+            assert("Unexpected key foo" in e.error['message'])
+
+
+        ############################################################
+        # test a fundrawtransaction with an invalid change address #
+        ############################################################
+        utx = get_unspent(self.nodes[2].listunspent(), 5)
+
+        inputs  = [ {'txid' : utx['txid'], 'vout' : utx['vout']} ]
+        outputs = { self.nodes[0].getnewaddress() : Decimal(4.0) }
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+        assert_equal(utx['txid'], dec_tx['vin'][0]['txid'])
+
+        try:
+            self.nodes[2].fundrawtransaction(rawtx, {'changeAddress': 'foobar'})
+            raise AssertionError("Accepted invalid agrarian address")
+        except JSONRPCException as e:
+            assert("changeAddress must be a valid agrarian address" in e.error['message'])
+
+
+        ############################################################
+        # test a fundrawtransaction with a provided change address #
+        ############################################################
+        utx = get_unspent(self.nodes[2].listunspent(), 5)
+
+        inputs  = [ {'txid' : utx['txid'], 'vout' : utx['vout']} ]
+        outputs = { self.nodes[0].getnewaddress() : Decimal(4.0) }
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+        assert_equal(utx['txid'], dec_tx['vin'][0]['txid'])
+
+        change = self.nodes[2].getnewaddress()
+        try:
+            rawtxfund = self.nodes[2].fundrawtransaction(rawtx, {'changeAddress': change, 'changePosition': 2})
+        except JSONRPCException as e:
+            assert('changePosition out of bounds' == e.error['message'])
+        else:
+            assert(False)
+        rawtxfund = self.nodes[2].fundrawtransaction(rawtx, {'changeAddress': change, 'changePosition': 0})
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtxfund['hex'])
+        out = dec_tx['vout'][0];
+        assert_equal(change, out['scriptPubKey']['addresses'][0])
+
+
+        #########################################################################
+        # test a fundrawtransaction with a VIN smaller than the required amount #
+        #########################################################################
+        utx = get_unspent(self.nodes[2].listunspent(), 1)
+
+        inputs  = [ {'txid' : utx['txid'], 'vout' : utx['vout']}]
+        outputs = { self.nodes[0].getnewaddress() : 1.0 }
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+
+        # 4-byte version + 1-byte vin count + 36-byte prevout then script_len
+        rawtx = rawtx[:82] + "0100" + rawtx[84:]
+
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+        assert_equal(utx['txid'], dec_tx['vin'][0]['txid'])
+        assert_equal("00", dec_tx['vin'][0]['scriptSig']['hex'])
+
+        rawtxfund = self.nodes[2].fundrawtransaction(rawtx)
+        fee = rawtxfund['fee']
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtxfund['hex'])
+        totalOut = 0
+        matchingOuts = 0
+        for i, out in enumerate(dec_tx['vout']):
+            totalOut += out['value']
+            if out['scriptPubKey']['addresses'][0] in outputs:
+                matchingOuts+=1
+            else:
+                assert_equal(i, rawtxfund['changepos'])
+
+        assert_equal(utx['txid'], dec_tx['vin'][0]['txid'])
+        assert_equal("00", dec_tx['vin'][0]['scriptSig']['hex'])
+
+        assert_equal(matchingOuts, 1)
+        assert_equal(len(dec_tx['vout']), 2)
+
+
+        ###########################################
+        # test a fundrawtransaction with two VINs #
+        ###########################################
+        utx = get_unspent(self.nodes[2].listunspent(), 1)
+        utx2 = get_unspent(self.nodes[2].listunspent(), 5)
+
+        inputs  = [ {'txid' : utx['txid'], 'vout' : utx['vout']},{'txid' : utx2['txid'], 'vout' : utx2['vout']} ]
+        outputs = { self.nodes[0].getnewaddress() : 6.0 }
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+        assert_equal(utx['txid'], dec_tx['vin'][0]['txid'])
+
+        rawtxfund = self.nodes[2].fundrawtransaction(rawtx)
+        fee = rawtxfund['fee']
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtxfund['hex'])
+        totalOut = 0
+        matchingOuts = 0
+        for out in dec_tx['vout']:
+            totalOut += out['value']
+            if out['scriptPubKey']['addresses'][0] in outputs:
+                matchingOuts+=1
+
+        assert_equal(matchingOuts, 1)
+        assert_equal(len(dec_tx['vout']), 2)
+
+        matchingIns = 0
+        for vinOut in dec_tx['vin']:
+            for vinIn in inputs:
+                if vinIn['txid'] == vinOut['txid']:
+                    matchingIns+=1
+
+        assert_equal(matchingIns, 2) #we now must see two vins identical to vins given as params
+
+        #########################################################
+        # test a fundrawtransaction with two VINs and two vOUTs #
+        #########################################################
+        utx = get_unspent(self.nodes[2].listunspent(), 1)
+        utx2 = get_unspent(self.nodes[2].listunspent(), 5)
+
+        inputs  = [ {'txid' : utx['txid'], 'vout' : utx['vout']},{'txid' : utx2['txid'], 'vout' : utx2['vout']} ]
+        outputs = { self.nodes[0].getnewaddress() : 6.0, self.nodes[0].getnewaddress() : 1.0 }
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+        assert_equal(utx['txid'], dec_tx['vin'][0]['txid'])
+
+        rawtxfund = self.nodes[2].fundrawtransaction(rawtx)
+        fee = rawtxfund['fee']
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtxfund['hex'])
+        totalOut = 0
+        matchingOuts = 0
+        for out in dec_tx['vout']:
+            totalOut += out['value']
+            if out['scriptPubKey']['addresses'][0] in outputs:
+                matchingOuts+=1
+
+        assert_equal(matchingOuts, 2)
+        assert_equal(len(dec_tx['vout']), 3)
+
+        ##############################################
+        # test a fundrawtransaction with invalid vin #
+        ##############################################
+        listunspent = self.nodes[2].listunspent()
+        inputs  = [ {'txid' : "1c7f966dab21119bac53213a2bc7532bff1fa844c124fd750a7d0b1332440bd1", 'vout' : 0} ] #invalid vin!
+        outputs = { self.nodes[0].getnewaddress() : 1.0}
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+
+        try:
+            rawtxfund = self.nodes[2].fundrawtransaction(rawtx)
+            raise AssertionError("Spent more than available")
+        except JSONRPCException as e:
+            assert("Insufficient" in e.error['message'])
+
+
+        ############################################################
+        #compare fee of a standard pubkeyhash transaction
+        inputs = []
+        outputs = {self.nodes[1].getnewaddress():1.1}
+        rawTx = self.nodes[0].createrawtransaction(inputs, outputs)
+        fundedTx = self.nodes[0].fundrawtransaction(rawTx)
+
+        #create same transaction over sendtoaddress
+        txId = self.nodes[0].sendtoaddress(self.nodes[1].getnewaddress(), 1.1)
+        signedFee = self.nodes[0].getrawmempool(True)[txId]['fee']
+
+        #compare fee
+        feeDelta = Decimal(fundedTx['fee']) - Decimal(signedFee)
+        assert(feeDelta >= 0 and feeDelta <= feeTolerance)
+        ############################################################
+
+        ############################################################
+        #compare fee of a standard pubkeyhash transaction with multiple outputs
+        inputs = []
+        outputs = {self.nodes[1].getnewaddress():1.1,self.nodes[1].getnewaddress():1.2,self.nodes[1].getnewaddress():0.1,self.nodes[1].getnewaddress():1.3,self.nodes[1].getnewaddress():0.2,self.nodes[1].getnewaddress():0.3}
+        rawTx = self.nodes[0].createrawtransaction(inputs, outputs)
+        fundedTx = self.nodes[0].fundrawtransaction(rawTx)
+        #create same transaction over sendtoaddress
+        txId = self.nodes[0].sendmany("", outputs)
+        signedFee = self.nodes[0].getrawmempool(True)[txId]['fee']
+
+        #compare fee
+        feeDelta = Decimal(fundedTx['fee']) - Decimal(signedFee)
+        assert(feeDelta >= 0 and feeDelta <= feeTolerance)
+        ############################################################
+
+
+        ############################################################
+        #compare fee of a 2of2 multisig p2sh transaction
+
+        # create 2of2 addr
+        addr1 = self.nodes[1].getnewaddress()
+        addr2 = self.nodes[1].getnewaddress()
+
+        addr1Obj = self.nodes[1].validateaddress(addr1)
+        addr2Obj = self.nodes[1].validateaddress(addr2)
+
+        mSigObj = self.nodes[1].addmultisigaddress(2, [addr1Obj['pubkey'], addr2Obj['pubkey']])
+
+        inputs = []
+        outputs = {mSigObj:1.1}
+        rawTx = self.nodes[0].createrawtransaction(inputs, outputs)
+        fundedTx = self.nodes[0].fundrawtransaction(rawTx)
+
+        #create same transaction over sendtoaddress
+        txId = self.nodes[0].sendtoaddress(mSigObj, 1.1)
+        signedFee = self.nodes[0].getrawmempool(True)[txId]['fee']
+
+        #compare fee
+        feeDelta = Decimal(fundedTx['fee']) - Decimal(signedFee)
+        assert(feeDelta >= 0 and feeDelta <= feeTolerance)
+        ############################################################
+
+
+        ############################################################
+        #compare fee of a standard pubkeyhash transaction
+
+        # create 4of5 addr
+        addr1 = self.nodes[1].getnewaddress()
+        addr2 = self.nodes[1].getnewaddress()
+        addr3 = self.nodes[1].getnewaddress()
+        addr4 = self.nodes[1].getnewaddress()
+        addr5 = self.nodes[1].getnewaddress()
+
+        addr1Obj = self.nodes[1].validateaddress(addr1)
+        addr2Obj = self.nodes[1].validateaddress(addr2)
+        addr3Obj = self.nodes[1].validateaddress(addr3)
+        addr4Obj = self.nodes[1].validateaddress(addr4)
+        addr5Obj = self.nodes[1].validateaddress(addr5)
+
+        mSigObj = self.nodes[1].addmultisigaddress(4, [addr1Obj['pubkey'], addr2Obj['pubkey'], addr3Obj['pubkey'], addr4Obj['pubkey'], addr5Obj['pubkey']])
+
+        inputs = []
+        outputs = {mSigObj:1.1}
+        rawTx = self.nodes[0].createrawtransaction(inputs, outputs)
+        fundedTx = self.nodes[0].fundrawtransaction(rawTx)
+
+        #create same transaction over sendtoaddress
+        txId = self.nodes[0].sendtoaddress(mSigObj, 1.1)
+        signedFee = self.nodes[0].getrawmempool(True)[txId]['fee']
+
+        #compare fee
+        feeDelta = Decimal(fundedTx['fee']) - Decimal(signedFee)
+        assert(feeDelta >= 0 and feeDelta <= feeTolerance)
+        ############################################################
+
+
+        ############################################################
+        # spend a 2of2 multisig transaction over fundraw
+
+        # create 2of2 addr
+        addr1 = self.nodes[2].getnewaddress()
+        addr2 = self.nodes[2].getnewaddress()
+
+        addr1Obj = self.nodes[2].validateaddress(addr1)
+        addr2Obj = self.nodes[2].validateaddress(addr2)
+
+        mSigObj = self.nodes[2].addmultisigaddress(2, [addr1Obj['pubkey'], addr2Obj['pubkey']])
+
+
+        # send 1.2 BTC to msig addr
+        txId = self.nodes[0].sendtoaddress(mSigObj, 1.2)
+        self.sync_all()
+        self.nodes[1].generate(1)
+        self.sync_all()
+
+        oldBalance = self.nodes[1].getbalance()
+        inputs = []
+        outputs = {self.nodes[1].getnewaddress():1.1}
+        rawTx = self.nodes[2].createrawtransaction(inputs, outputs)
+        fundedTx = self.nodes[2].fundrawtransaction(rawTx)
+
+        signedTx = self.nodes[2].signrawtransaction(fundedTx['hex'])
+        txId = self.nodes[2].sendrawtransaction(signedTx['hex'])
+        self.sync_all()
+        self.nodes[1].generate(1)
+        self.sync_all()
+
+        # make sure funds are received at node1
+        assert_equal(oldBalance+Decimal('1.10000000'), self.nodes[1].getbalance())
+
+        ############################################################
+        # locked wallet test
+        self.nodes[1].encryptwallet("test")
+        self.nodes.pop(1)
+        stop_nodes(self.nodes)
+
+        self.nodes = start_nodes(self.num_nodes, self.options.tmpdir)
+        # This test is not meant to test fee estimation and we'd like
+        # to be sure all txs are sent at a consistent desired feerate
+        for node in self.nodes:
+            node.settxfee(min_relay_tx_fee)
+
+        connect_nodes_bi(self.nodes,0,1)
+        connect_nodes_bi(self.nodes,1,2)
+        connect_nodes_bi(self.nodes,0,2)
+        connect_nodes_bi(self.nodes,0,3)
+        self.is_network_split=False
+        self.sync_all()
+
+        # drain the keypool
+        self.nodes[1].getnewaddress()
+        inputs = []
+        outputs = {self.nodes[0].getnewaddress():1.1}
+        rawTx = self.nodes[1].createrawtransaction(inputs, outputs)
+        # fund a transaction that requires a new key for the change output
+        # creating the key must be impossible because the wallet is locked
+        try:
+            fundedTx = self.nodes[1].fundrawtransaction(rawTx)
+            raise AssertionError("Wallet unlocked without passphrase")
+        except JSONRPCException as e:
+            assert('Keypool ran out' in e.error['message'])
+
+        #refill the keypool
+        self.nodes[1].walletpassphrase("test", 100)
+        self.nodes[1].walletlock()
+
+        try:
+            self.nodes[1].sendtoaddress(self.nodes[0].getnewaddress(), 1.2)
+            raise AssertionError("Wallet unlocked without passphrase")
+        except JSONRPCException as e:
+            assert('walletpassphrase' in e.error['message'])
+
+        oldBalance = self.nodes[0].getbalance()
+
+        inputs = []
+        outputs = {self.nodes[0].getnewaddress():1.1}
+        rawTx = self.nodes[1].createrawtransaction(inputs, outputs)
+        fundedTx = self.nodes[1].fundrawtransaction(rawTx)
+
+        #now we need to unlock
+        self.nodes[1].walletpassphrase("test", 100)
+        signedTx = self.nodes[1].signrawtransaction(fundedTx['hex'])
+        txId = self.nodes[1].sendrawtransaction(signedTx['hex'])
+        self.nodes[1].generate(1)
+        self.sync_all()
+
+        # make sure funds are received at node1
+        assert_equal(oldBalance+Decimal('51.10000000'), self.nodes[0].getbalance())
+
+
+        ###############################################
+        # multiple (~19) inputs tx test | Compare fee #
+        ###############################################
+
+        #empty node1, send some small coins from node0 to node1
+        self.nodes[1].sendtoaddress(self.nodes[0].getnewaddress(), self.nodes[1].getbalance(), "", "", True)
+        self.sync_all()
+        self.nodes[0].generate(1)
+        self.sync_all()
+
+        for i in range(0,20):
+            self.nodes[0].sendtoaddress(self.nodes[1].getnewaddress(), 0.01)
+        self.nodes[0].generate(1)
+        self.sync_all()
+
+        #fund a tx with ~20 small inputs
+        inputs = []
+        outputs = {self.nodes[0].getnewaddress():0.15,self.nodes[0].getnewaddress():0.04}
+        rawTx = self.nodes[1].createrawtransaction(inputs, outputs)
+        fundedTx = self.nodes[1].fundrawtransaction(rawTx)
+
+        #create same transaction over sendtoaddress
+        txId = self.nodes[1].sendmany("", outputs)
+        signedFee = self.nodes[1].getrawmempool(True)[txId]['fee']
+
+        #compare fee
+        feeDelta = Decimal(fundedTx['fee']) - Decimal(signedFee)
+        assert(feeDelta >= 0 and feeDelta <= feeTolerance*19) #~19 inputs
+
+
+        #############################################
+        # multiple (~19) inputs tx test | sign/send #
+        #############################################
+
+        #again, empty node1, send some small coins from node0 to node1
+        self.nodes[1].sendtoaddress(self.nodes[0].getnewaddress(), self.nodes[1].getbalance(), "", "", True)
+        self.sync_all()
+        self.nodes[0].generate(1)
+        self.sync_all()
+
+        for i in range(0,20):
+            self.nodes[0].sendtoaddress(self.nodes[1].getnewaddress(), 0.01)
+        self.nodes[0].generate(1)
+        self.sync_all()
+
+        #fund a tx with ~20 small inputs
+        oldBalance = self.nodes[0].getbalance()
+
+        inputs = []
+        outputs = {self.nodes[0].getnewaddress():0.15,self.nodes[0].getnewaddress():0.04}
+        rawTx = self.nodes[1].createrawtransaction(inputs, outputs)
+        fundedTx = self.nodes[1].fundrawtransaction(rawTx)
+        fundedAndSignedTx = self.nodes[1].signrawtransaction(fundedTx['hex'])
+        txId = self.nodes[1].sendrawtransaction(fundedAndSignedTx['hex'])
+        self.sync_all()
+        self.nodes[0].generate(1)
+        self.sync_all()
+        assert_equal(oldBalance+Decimal('50.19000000'), self.nodes[0].getbalance()) #0.19+block reward
+
+        #####################################################
+        # test fundrawtransaction with OP_RETURN and no vin #
+        #####################################################
+
+        rawtx   = "0100000000010000000000000000066a047465737400000000"
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtx)
+
+        assert_equal(len(dec_tx['vin']), 0)
+        assert_equal(len(dec_tx['vout']), 1)
+
+        rawtxfund = self.nodes[2].fundrawtransaction(rawtx)
+        dec_tx  = self.nodes[2].decoderawtransaction(rawtxfund['hex'])
+
+        assert_greater_than(len(dec_tx['vin']), 0) # at least one vin
+        assert_equal(len(dec_tx['vout']), 2) # one change output added
+
+
+        ##################################################
+        # test a fundrawtransaction using only watchonly #
+        ##################################################
+
+        inputs = []
+        outputs = {self.nodes[2].getnewaddress() : watchonly_amount / 2}
+        rawtx = self.nodes[3].createrawtransaction(inputs, outputs)
+
+        result = self.nodes[3].fundrawtransaction(rawtx, {'includeWatching': True })
+        res_dec = self.nodes[0].decoderawtransaction(result["hex"])
+        assert_equal(len(res_dec["vin"]), 1)
+        assert_equal(res_dec["vin"][0]["txid"], watchonly_txid)
+
+        assert("fee" in result.keys())
+        assert_greater_than(result["changepos"], -1)
+
+        ###############################################################
+        # test fundrawtransaction using the entirety of watched funds #
+        ###############################################################
+
+        inputs = []
+        outputs = {self.nodes[2].getnewaddress() : watchonly_amount}
+        rawtx = self.nodes[3].createrawtransaction(inputs, outputs)
+
+        # Backward compatibility test (2nd param is includeWatching)
+        result = self.nodes[3].fundrawtransaction(rawtx, True)
+        res_dec = self.nodes[0].decoderawtransaction(result["hex"])
+        assert_equal(len(res_dec["vin"]), 2)
+        assert(res_dec["vin"][0]["txid"] == watchonly_txid or res_dec["vin"][1]["txid"] == watchonly_txid)
+
+        assert_greater_than(result["fee"], 0)
+        assert_greater_than(result["changepos"], -1)
+        assert_equal(result["fee"] + res_dec["vout"][result["changepos"]]["value"], watchonly_amount / 10)
+
+        signedtx = self.nodes[3].signrawtransaction(result["hex"])
+        assert(not signedtx["complete"])
+        signedtx = self.nodes[0].signrawtransaction(signedtx["hex"])
+        assert(signedtx["complete"])
+        self.nodes[0].sendrawtransaction(signedtx["hex"])
+        self.nodes[0].generate(1)
+        self.sync_all()
+
+        #######################
+        # Test feeRate option #
+        #######################
+
+        # Make sure there is exactly one input so coin selection can't skew the result
+        assert_equal(len(self.nodes[3].listunspent(1)), 1)
+
+        inputs = []
+        outputs = {self.nodes[2].getnewaddress() : 1}
+        rawtx = self.nodes[3].createrawtransaction(inputs, outputs)
+        result = self.nodes[3].fundrawtransaction(rawtx) # uses min_relay_tx_fee (set by settxfee)
+        result2 = self.nodes[3].fundrawtransaction(rawtx, {"feeRate": 2*min_relay_tx_fee})
+        result3 = self.nodes[3].fundrawtransaction(rawtx, {"feeRate": 10*min_relay_tx_fee})
+        result_fee_rate = result['fee'] * 1000 / count_bytes(result['hex'])
+        assert_fee_amount(result2['fee'], count_bytes(result2['hex']), 2 * result_fee_rate)
+        assert_fee_amount(result3['fee'], count_bytes(result3['hex']), 10 * result_fee_rate)
+
+if __name__ == '__main__':
+    RawTransactionsTest().main()

test/functional/rpc_getchaintips.py

@@ -0,0 +1,62 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the getchaintips RPC.
+
+- introduce a network split
+- work on chains of different lengths
+- join the network together again
+- verify that getchaintips now returns two chain tips.
+"""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import assert_equal
+
+class GetChainTipsTest (BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 4
+
+    def run_test (self):
+        tips = self.nodes[0].getchaintips ()
+        assert_equal (len (tips), 1)
+        assert_equal (tips[0]['branchlen'], 0)
+        assert_equal (tips[0]['height'], 200)
+        assert_equal (tips[0]['status'], 'active')
+
+        # Split the network and build two chains of different lengths.
+        self.split_network ()
+        self.nodes[0].generate(10)
+        self.nodes[2].generate(20)
+        self.sync_all([self.nodes[:2], self.nodes[2:]])
+
+        tips = self.nodes[1].getchaintips ()
+        assert_equal (len (tips), 1)
+        shortTip = tips[0]
+        assert_equal (shortTip['branchlen'], 0)
+        assert_equal (shortTip['height'], 210)
+        assert_equal (tips[0]['status'], 'active')
+
+        tips = self.nodes[3].getchaintips ()
+        assert_equal (len (tips), 1)
+        longTip = tips[0]
+        assert_equal (longTip['branchlen'], 0)
+        assert_equal (longTip['height'], 220)
+        assert_equal (tips[0]['status'], 'active')
+
+        # Join the network halves and check that we now have two tips
+        # (at least at the nodes that previously had the short chain).
+        self.join_network ()
+
+        tips = self.nodes[0].getchaintips ()
+        assert_equal (len (tips), 2)
+        assert_equal (tips[0], longTip)
+
+        assert_equal (tips[1]['branchlen'], 10)
+        assert_equal (tips[1]['status'], 'valid-fork')
+        tips[1]['branchlen'] = 0
+        tips[1]['status'] = 'active'
+        assert_equal (tips[1], shortTip)
+
+if __name__ == '__main__':
+    GetChainTipsTest ().main ()

test/functional/rpc_invalidateblock.py

@@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the invalidateblock RPC."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+class InvalidateTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 3
+
+    def setup_network(self):
+        self.setup_nodes()
+
+    def run_test(self):
+        self.log.info("Make sure we repopulate setBlockIndexCandidates after InvalidateBlock:")
+        self.log.info("Mine 4 blocks on Node 0")
+        self.nodes[0].generate(4)
+        assert(self.nodes[0].getblockcount() == 4)
+        besthash = self.nodes[0].getbestblockhash()
+
+        self.log.info("Mine competing 6 blocks on Node 1")
+        self.nodes[1].generate(6)
+        assert(self.nodes[1].getblockcount() == 6)
+
+        self.log.info("Connect nodes to force a reorg")
+        connect_nodes_bi(self.nodes,0,1)
+        sync_blocks(self.nodes[0:2])
+        assert(self.nodes[0].getblockcount() == 6)
+        badhash = self.nodes[1].getblockhash(2)
+
+        self.log.info("Invalidate block 2 on node 0 and verify we reorg to node 0's original chain")
+        self.nodes[0].invalidateblock(badhash)
+        newheight = self.nodes[0].getblockcount()
+        newhash = self.nodes[0].getbestblockhash()
+        if (newheight != 4 or newhash != besthash):
+            raise AssertionError("Wrong tip for node0, hash %s, height %d"%(newhash,newheight))
+
+        self.log.info("Make sure we won't reorg to a lower work chain:")
+        connect_nodes_bi(self.nodes,1,2)
+        self.log.info("Sync node 2 to node 1 so both have 6 blocks")
+        sync_blocks(self.nodes[1:3])
+        assert(self.nodes[2].getblockcount() == 6)
+        self.log.info("Invalidate block 5 on node 1 so its tip is now at 4")
+        self.nodes[1].invalidateblock(self.nodes[1].getblockhash(5))
+        assert(self.nodes[1].getblockcount() == 4)
+        self.log.info("Invalidate block 3 on node 2, so its tip is now 2")
+        self.nodes[2].invalidateblock(self.nodes[2].getblockhash(3))
+        assert(self.nodes[2].getblockcount() == 2)
+        self.log.info("..and then mine a block")
+        self.nodes[2].generate(1)
+        self.log.info("Verify all nodes are at the right height")
+        time.sleep(5)
+        assert_equal(self.nodes[2].getblockcount(), 3)
+        assert_equal(self.nodes[0].getblockcount(), 4)
+        node1height = self.nodes[1].getblockcount()
+        if node1height < 4:
+            raise AssertionError("Node 1 reorged to a lower height: %d"%node1height)
+
+if __name__ == '__main__':
+    InvalidateTest().main()

test/functional/rpc_listtransactions.py

@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the listtransactions API."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+from test_framework.mininode import CTransaction, COIN
+from io import BytesIO
+
+def txFromHex(hexstring):
+    tx = CTransaction()
+    f = BytesIO(hex_str_to_bytes(hexstring))
+    tx.deserialize(f)
+    return tx
+
+class ListTransactionsTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+        self.enable_mocktime()
+
+    def run_test(self):
+        # Simple send, 0 to 1:
+        txid = self.nodes[0].sendtoaddress(self.nodes[1].getnewaddress(), 0.1)
+        self.sync_all()
+        assert_array_result(self.nodes[0].listtransactions(),
+                           {"txid":txid},
+                           {"category":"send","account":"","amount":Decimal("-0.1"),"confirmations":0})
+        assert_array_result(self.nodes[1].listtransactions(),
+                           {"txid":txid},
+                           {"category":"receive","account":"","amount":Decimal("0.1"),"confirmations":0})
+        # mine a block, confirmations should change:
+        self.nodes[0].generate(1)
+        self.sync_all()
+        assert_array_result(self.nodes[0].listtransactions(),
+                           {"txid":txid},
+                           {"category":"send","account":"","amount":Decimal("-0.1"),"confirmations":1})
+        assert_array_result(self.nodes[1].listtransactions(),
+                           {"txid":txid},
+                           {"category":"receive","account":"","amount":Decimal("0.1"),"confirmations":1})
+
+        # send-to-self:
+        txid = self.nodes[0].sendtoaddress(self.nodes[0].getnewaddress(), 0.2)
+        assert_array_result(self.nodes[0].listtransactions(),
+                           {"txid":txid, "category":"send"},
+                           {"amount":Decimal("-0.2")})
+        assert_array_result(self.nodes[0].listtransactions(),
+                           {"txid":txid, "category":"receive"},
+                           {"amount":Decimal("0.2")})
+
+        # sendmany from node1: twice to self, twice to node2:
+        send_to = { self.nodes[0].getnewaddress() : 0.11,
+                    self.nodes[1].getnewaddress() : 0.22,
+                    self.nodes[0].getaccountaddress("from1") : 0.33,
+                    self.nodes[1].getaccountaddress("toself") : 0.44 }
+        txid = self.nodes[1].sendmany("", send_to)
+        self.sync_all()
+        assert_array_result(self.nodes[1].listtransactions(),
+                           {"category":"send","amount":Decimal("-0.11")},
+                           {"txid":txid} )
+        assert_array_result(self.nodes[0].listtransactions(),
+                           {"category":"receive","amount":Decimal("0.11")},
+                           {"txid":txid} )
+        assert_array_result(self.nodes[1].listtransactions(),
+                           {"category":"send","amount":Decimal("-0.22")},
+                           {"txid":txid} )
+        assert_array_result(self.nodes[1].listtransactions(),
+                           {"category":"receive","amount":Decimal("0.22")},
+                           {"txid":txid} )
+        assert_array_result(self.nodes[1].listtransactions(),
+                           {"category":"send","amount":Decimal("-0.33")},
+                           {"txid":txid} )
+        assert_array_result(self.nodes[0].listtransactions(),
+                           {"category":"receive","amount":Decimal("0.33")},
+                           {"txid":txid, "account" : "from1"} )
+        assert_array_result(self.nodes[1].listtransactions(),
+                           {"category":"send","amount":Decimal("-0.44")},
+                           {"txid":txid, "account" : ""} )
+        assert_array_result(self.nodes[1].listtransactions(),
+                           {"category":"receive","amount":Decimal("0.44")},
+                           {"txid":txid, "account" : "toself"} )
+
+
+if __name__ == '__main__':
+    ListTransactionsTest().main()

test/functional/rpc_net.py

@@ -0,0 +1,94 @@
+#!/usr/bin/env python3
+# Copyright (c) 2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test RPC calls related to net.
+
+Tests correspond to code in rpc/net.cpp.
+"""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (
+    assert_equal,
+    assert_greater_than_or_equal,
+    assert_raises_rpc_error,
+    connect_nodes_bi,
+    disconnect_nodes,
+    p2p_port,
+    wait_until,
+)
+
+class NetTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 2
+
+    def run_test(self):
+        self._test_connection_count()
+        self._test_getnettotals()
+        self._test_getnetworkinginfo()
+        self._test_getaddednodeinfo()
+        #self._test_getpeerinfo()
+
+    def _test_connection_count(self):
+        # connect_nodes_bi connects each node to the other
+        assert_equal(self.nodes[0].getconnectioncount(), 2)
+
+    def _test_getnettotals(self):
+        # getnettotals totalbytesrecv and totalbytessent should be
+        # consistent with getpeerinfo. Since the RPC calls are not atomic,
+        # and messages might have been recvd or sent between RPC calls, call
+        # getnettotals before and after and verify that the returned values
+        # from getpeerinfo are bounded by those values.
+        net_totals_before = self.nodes[0].getnettotals()
+        peer_info = self.nodes[0].getpeerinfo()
+        net_totals_after = self.nodes[0].getnettotals()
+        assert_equal(len(peer_info), 2)
+        peers_recv = sum([peer['bytesrecv'] for peer in peer_info])
+        peers_sent = sum([peer['bytessent'] for peer in peer_info])
+
+        assert_greater_than_or_equal(peers_recv, net_totals_before['totalbytesrecv'])
+        assert_greater_than_or_equal(net_totals_after['totalbytesrecv'], peers_recv)
+        assert_greater_than_or_equal(peers_sent, net_totals_before['totalbytessent'])
+        assert_greater_than_or_equal(net_totals_after['totalbytessent'], peers_sent)
+
+        # test getnettotals and getpeerinfo by doing a ping
+        # the bytes sent/received should change
+        # note ping and pong are 32 bytes each
+        self.nodes[0].ping()
+        wait_until(lambda: (self.nodes[0].getnettotals()['totalbytessent'] >= net_totals_after['totalbytessent'] + 32 * 2), timeout=1)
+        wait_until(lambda: (self.nodes[0].getnettotals()['totalbytesrecv'] >= net_totals_after['totalbytesrecv'] + 32 * 2), timeout=1)
+
+        peer_info_after_ping = self.nodes[0].getpeerinfo()
+
+    def _test_getnetworkinginfo(self):
+        assert_equal(self.nodes[0].getnetworkinfo()['connections'], 2)
+
+        disconnect_nodes(self.nodes[0], 1)
+        # Wait a bit for all sockets to close
+        wait_until(lambda: self.nodes[0].getnetworkinfo()['connections'] == 0, timeout=3)
+
+        connect_nodes_bi(self.nodes, 0, 1)
+        assert_equal(self.nodes[0].getnetworkinfo()['connections'], 2)
+
+    def _test_getaddednodeinfo(self):
+        assert_equal(self.nodes[0].getaddednodeinfo(True), [])
+        # add a node (node2) to node0
+        ip_port = "127.0.0.1:{}".format(p2p_port(2))
+        self.nodes[0].addnode(ip_port, 'add')
+        # check that the node has indeed been added
+        added_nodes = self.nodes[0].getaddednodeinfo(True, ip_port)
+        assert_equal(len(added_nodes), 1)
+        assert_equal(added_nodes[0]['addednode'], ip_port)
+        # check that a non-existent node returns an error
+        assert_raises_rpc_error(-24, "Node has not been added", self.nodes[0].getaddednodeinfo, True, '1.1.1.1')
+
+    def _test_getpeerinfo(self):
+        peer_info = [x.getpeerinfo() for x in self.nodes]
+        # check both sides of bidirectional connection between nodes
+        # the address bound to on one side will be the source address for the other node
+        assert_equal(peer_info[0][0]['addrbind'], peer_info[1][0]['addr'])
+        assert_equal(peer_info[1][0]['addrbind'], peer_info[0][0]['addr'])
+
+if __name__ == '__main__':
+    NetTest().main()

test/functional/rpc_rawtransaction.py

@@ -0,0 +1,346 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the rawtransaction RPCs.
+
+Test the following RPCs:
+   - createrawtransaction
+   - signrawtransaction
+   - sendrawtransaction
+   - decoderawtransaction
+   - getrawtransaction
+"""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+
+class multidict(dict):
+    """Dictionary that allows duplicate keys.
+
+    Constructed with a list of (key, value) tuples. When dumped by the json module,
+    will output invalid json with repeated keys, eg:
+    >>> json.dumps(multidict([(1,2),(1,2)])
+    '{"1": 2, "1": 2}'
+
+    Used to test calls to rpc methods with repeated keys in the json object."""
+
+    def __init__(self, x):
+        dict.__init__(self, x)
+        self.x = x
+
+    def items(self):
+        return self.x
+
+
+# Create one-input, one-output, no-fee transaction:
+class RawTransactionsTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 3
+
+    def setup_network(self, split=False):
+        super().setup_network()
+        connect_nodes_bi(self.nodes,0,2)
+
+    def run_test(self):
+
+        #prepare some coins for multiple *rawtransaction commands
+        self.nodes[2].generate(1)
+        self.sync_all()
+        self.nodes[0].generate(101)
+        self.sync_all()
+        self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(),1.5)
+        self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(),1.0)
+        self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(),5.0)
+        self.sync_all()
+        self.nodes[0].generate(5)
+        self.sync_all()
+
+        # Test `createrawtransaction` required parameters
+        assert_raises_rpc_error(-1, "createrawtransaction", self.nodes[0].createrawtransaction)
+        assert_raises_rpc_error(-1, "createrawtransaction", self.nodes[0].createrawtransaction, [])
+
+        # Test `createrawtransaction` invalid extra parameters
+        assert_raises_rpc_error(-1, "createrawtransaction", self.nodes[0].createrawtransaction, [], {}, 0, False, 'foo')
+
+        # Test `createrawtransaction` invalid `inputs`
+        txid = '1d1d4e24ed99057e84c3f80fd8fbec79ed9e1acee37da269356ecea000000000'
+        assert_raises_rpc_error(-3, "Expected type array", self.nodes[0].createrawtransaction, 'foo', {})
+        assert_raises_rpc_error(-1, "JSON value is not an object as expected", self.nodes[0].createrawtransaction, ['foo'], {})
+        assert_raises_rpc_error(-8, "txid must be hexadecimal string", self.nodes[0].createrawtransaction, [{}], {})
+        assert_raises_rpc_error(-8, "txid must be hexadecimal string", self.nodes[0].createrawtransaction, [{'txid': 'foo'}], {})
+        assert_raises_rpc_error(-8, "Invalid parameter, missing vout key", self.nodes[0].createrawtransaction, [{'txid': txid}], {})
+        assert_raises_rpc_error(-8, "Invalid parameter, missing vout key", self.nodes[0].createrawtransaction, [{'txid': txid, 'vout': 'foo'}], {})
+        assert_raises_rpc_error(-8, "Invalid parameter, vout must be positive", self.nodes[0].createrawtransaction, [{'txid': txid, 'vout': -1}], {})
+        assert_raises_rpc_error(-8, "Invalid parameter, sequence number is out of range", self.nodes[0].createrawtransaction, [{'txid': txid, 'vout': 0, 'sequence': -1}], {})
+
+        # Test `createrawtransaction` invalid `outputs`
+        address = self.nodes[0].getnewaddress()
+        assert_raises_rpc_error(-3, "Expected type object", self.nodes[0].createrawtransaction, [], 'foo')
+        #assert_raises_rpc_error(-8, "Data must be hexadecimal string", self.nodes[0].createrawtransaction, [], {'data': 'foo'})
+        assert_raises_rpc_error(-5, "Invalid Agrarian address", self.nodes[0].createrawtransaction, [], {'foo': 0})
+        #assert_raises_rpc_error(-3, "Amount is not a number", self.nodes[0].createrawtransaction, [], {address: 'foo'})
+        assert_raises_rpc_error(-3, "Invalid amount", self.nodes[0].createrawtransaction, [], {address: -1})
+        assert_raises_rpc_error(-8, "Invalid parameter, duplicated address: %s" % address, self.nodes[0].createrawtransaction, [], multidict([(address, 1), (address, 1)]))
+
+        # Test `createrawtransaction` invalid `locktime`
+        assert_raises_rpc_error(-3, "Expected type number", self.nodes[0].createrawtransaction, [], {}, 'foo')
+        assert_raises_rpc_error(-8, "Invalid parameter, locktime out of range", self.nodes[0].createrawtransaction, [], {}, -1)
+        assert_raises_rpc_error(-8, "Invalid parameter, locktime out of range", self.nodes[0].createrawtransaction, [], {}, 4294967296)
+        addr = self.nodes[0].getnewaddress("")
+        addrinfo = self.nodes[0].validateaddress(addr)
+        pubkey = addrinfo["scriptPubKey"]
+
+        self.log.info('sendrawtransaction with missing prevtx info')
+
+        # Test `signrawtransaction` invalid `prevtxs`
+        inputs  = [ {'txid' : txid, 'vout' : 3, 'sequence' : 1000}]
+        outputs = { self.nodes[0].getnewaddress() : 1 }
+        rawtx   = self.nodes[0].createrawtransaction(inputs, outputs)
+
+        prevtx = dict(txid=txid, scriptPubKey=pubkey, vout=3, amount=1)
+        succ = self.nodes[0].signrawtransaction(rawtx, [prevtx])
+        assert succ["complete"]
+        del prevtx["amount"]
+        succ = self.nodes[0].signrawtransaction(rawtx, [prevtx])
+        assert succ["complete"]
+        assert_raises_rpc_error(-3, "Missing vout", self.nodes[0].signrawtransaction, rawtx, [
+            {
+                "txid": txid,
+                "scriptPubKey": pubkey,
+                "amount": 1,
+            }
+        ])
+        assert_raises_rpc_error(-3, "Missing txid", self.nodes[0].signrawtransaction, rawtx, [
+            {
+                "scriptPubKey": pubkey,
+                "vout": 3,
+                "amount": 1,
+            }
+        ])
+        assert_raises_rpc_error(-3, "Missing scriptPubKey", self.nodes[0].signrawtransaction, rawtx, [
+            {
+                "txid": txid,
+                "vout": 3,
+                "amount": 1
+            }
+        ])
+
+        #########################################
+        # sendrawtransaction with missing input #
+        #########################################
+        inputs  = [ {'txid' : "1d1d4e24ed99057e84c3f80fd8fbec79ed9e1acee37da269356ecea000000000", 'vout' : 1}] #won't exists
+        outputs = { self.nodes[0].getnewaddress() : 4.998 }
+        rawtx   = self.nodes[2].createrawtransaction(inputs, outputs)
+        rawtx   = self.nodes[2].signrawtransaction(rawtx)
+
+        # This will raise an exception since there are missing inputs
+        assert_raises_rpc_error(-25, "Missing inputs", self.nodes[2].sendrawtransaction, rawtx['hex'])
+
+        #####################################
+        # getrawtransaction with block hash #
+        #####################################
+
+        # make a tx by sending then generate 2 blocks; block1 has the tx in it
+        tx = self.nodes[2].sendtoaddress(self.nodes[1].getnewaddress(), 1)
+        block1, block2 = self.nodes[2].generate(2)
+        self.sync_all()
+        # We should be able to get the raw transaction by providing the correct block
+        gottx = self.nodes[0].getrawtransaction(tx, True, block1)
+        assert_equal(gottx['txid'], tx)
+        assert_equal(gottx['in_active_chain'], True)
+        # We should not have the 'in_active_chain' flag when we don't provide a block
+        gottx = self.nodes[0].getrawtransaction(tx, True)
+        assert_equal(gottx['txid'], tx)
+        assert 'in_active_chain' not in gottx
+        # We should not get the tx if we provide an unrelated block
+        assert_raises_rpc_error(-5, "No such transaction found", self.nodes[0].getrawtransaction, tx, True, block2)
+        # An invalid block hash should raise the correct errors
+        assert_raises_rpc_error(-8, "parameter 3 must be hexadecimal", self.nodes[0].getrawtransaction, tx, True, True)
+        assert_raises_rpc_error(-8, "parameter 3 must be hexadecimal", self.nodes[0].getrawtransaction, tx, True, "foobar")
+        assert_raises_rpc_error(-8, "parameter 3 must be of length 64", self.nodes[0].getrawtransaction, tx, True, "abcd1234")
+        assert_raises_rpc_error(-5, "Block hash not found", self.nodes[0].getrawtransaction, tx, True, "0000000000000000000000000000000000000000000000000000000000000000")
+        # Undo the blocks and check in_active_chain
+        self.nodes[0].invalidateblock(block1)
+        gottx = self.nodes[0].getrawtransaction(tx, True, block1)
+        assert_equal(gottx['in_active_chain'], False)
+        self.nodes[0].reconsiderblock(block1)
+        assert_equal(self.nodes[0].getbestblockhash(), block2)
+
+        #########################
+        # RAW TX MULTISIG TESTS #
+        #########################
+        # 2of2 test
+        addr1 = self.nodes[2].getnewaddress()
+        addr2 = self.nodes[2].getnewaddress()
+
+        addr1Obj = self.nodes[2].validateaddress(addr1)
+        addr2Obj = self.nodes[2].validateaddress(addr2)
+
+        # Tests for createmultisig and addmultisigaddress
+        assert_raises_rpc_error(-1, "Invalid public key", self.nodes[0].createmultisig, 1, ["01020304"])
+        # createmultisig can only take public keys
+        self.nodes[0].createmultisig(2, [addr1Obj['pubkey'], addr2Obj['pubkey']])
+        # addmultisigaddress can take both pubkeys and addresses so long as they are in the wallet, which is tested here.
+        assert_raises_rpc_error(-1, "no full public key for address", self.nodes[0].createmultisig, 2, [addr1Obj['pubkey'], addr1])
+
+        mSigObj = self.nodes[2].addmultisigaddress(2, [addr1Obj['pubkey'], addr1])
+
+        #use balance deltas instead of absolute values
+        bal = self.nodes[2].getbalance()
+
+        # send 1.2 BTC to msig adr
+        txId = self.nodes[0].sendtoaddress(mSigObj, 1.2)
+        self.sync_all()
+        self.nodes[0].generate(1)
+        self.sync_all()
+        assert_equal(self.nodes[2].getbalance(), bal+Decimal('1.20000000')) #node2 has both keys of the 2of2 ms addr., tx should affect the balance
+
+
+        # 2of3 test from different nodes
+        bal = self.nodes[2].getbalance()
+        addr1 = self.nodes[1].getnewaddress()
+        addr2 = self.nodes[2].getnewaddress()
+        addr3 = self.nodes[2].getnewaddress()
+
+        addr1Obj = self.nodes[1].validateaddress(addr1)
+        addr2Obj = self.nodes[2].validateaddress(addr2)
+        addr3Obj = self.nodes[2].validateaddress(addr3)
+
+        mSigObj = self.nodes[2].addmultisigaddress(2, [addr1Obj['pubkey'], addr2Obj['pubkey'], addr3Obj['pubkey']])
+
+        txId = self.nodes[0].sendtoaddress(mSigObj, 2.2)
+        decTx = self.nodes[0].gettransaction(txId)
+        rawTx = self.nodes[0].decoderawtransaction(decTx['hex'])
+        self.sync_all()
+        self.nodes[0].generate(1)
+        self.sync_all()
+
+        #THIS IS A INCOMPLETE FEATURE
+        #NODE2 HAS TWO OF THREE KEY AND THE FUNDS SHOULD BE SPENDABLE AND COUNT AT BALANCE CALCULATION
+        assert_equal(self.nodes[2].getbalance(), bal) #for now, assume the funds of a 2of3 multisig tx are not marked as spendable
+
+        txDetails = self.nodes[0].gettransaction(txId, True)
+        rawTx = self.nodes[0].decoderawtransaction(txDetails['hex'])
+        vout = False
+        for outpoint in rawTx['vout']:
+            if outpoint['value'] == Decimal('2.20000000'):
+                vout = outpoint
+                break
+
+        bal = self.nodes[0].getbalance()
+        inputs = [{ "txid" : txId, "vout" : vout['n'], "scriptPubKey" : vout['scriptPubKey']['hex']}]
+        outputs = { self.nodes[0].getnewaddress() : 2.19 }
+        rawTx = self.nodes[2].createrawtransaction(inputs, outputs)
+        rawTxPartialSigned = self.nodes[1].signrawtransaction(rawTx, inputs)
+        assert_equal(rawTxPartialSigned['complete'], False) #node1 only has one key, can't comp. sign the tx
+
+        rawTxSigned = self.nodes[2].signrawtransaction(rawTx, inputs)
+        assert_equal(rawTxSigned['complete'], True) #node2 can sign the tx compl., own two of three keys
+        self.nodes[2].sendrawtransaction(rawTxSigned['hex'])
+        rawTx = self.nodes[0].decoderawtransaction(rawTxSigned['hex'])
+        self.sync_all()
+        self.nodes[0].generate(1)
+        self.sync_all()
+        assert_equal(self.nodes[0].getbalance(), bal+Decimal('250.00000000')+Decimal('2.19000000')) #block reward + tx
+
+        # 2of2 test for combining transactions
+        bal = self.nodes[2].getbalance()
+        addr1 = self.nodes[1].getnewaddress()
+        addr2 = self.nodes[2].getnewaddress()
+
+        addr1Obj = self.nodes[1].validateaddress(addr1)
+        addr2Obj = self.nodes[2].validateaddress(addr2)
+
+        self.nodes[1].addmultisigaddress(2, [addr1Obj['pubkey'], addr2Obj['pubkey']])
+        mSigObj = self.nodes[2].addmultisigaddress(2, [addr1Obj['pubkey'], addr2Obj['pubkey']])
+        mSigObjValid = self.nodes[2].validateaddress(mSigObj)
+
+        txId = self.nodes[0].sendtoaddress(mSigObj, 2.2)
+        decTx = self.nodes[0].gettransaction(txId)
+        rawTx2 = self.nodes[0].decoderawtransaction(decTx['hex'])
+        self.sync_all()
+        self.nodes[0].generate(1)
+        self.sync_all()
+
+        assert_equal(self.nodes[2].getbalance(), bal) # the funds of a 2of2 multisig tx should not be marked as spendable
+
+        txDetails = self.nodes[0].gettransaction(txId, True)
+        rawTx2 = self.nodes[0].decoderawtransaction(txDetails['hex'])
+        vout = False
+        for outpoint in rawTx2['vout']:
+            if outpoint['value'] == Decimal('2.20000000'):
+                vout = outpoint
+                break
+
+        bal = self.nodes[0].getbalance()
+        inputs = [{ "txid" : txId, "vout" : vout['n'], "scriptPubKey" : vout['scriptPubKey']['hex'], "redeemScript" : mSigObjValid['hex'], "amount" : vout['value']}]
+        outputs = { self.nodes[0].getnewaddress() : 2.19 }
+        rawTx2 = self.nodes[2].createrawtransaction(inputs, outputs)
+        rawTxPartialSigned1 = self.nodes[1].signrawtransaction(rawTx2, inputs)
+        self.log.info(rawTxPartialSigned1)
+        assert_equal(rawTxPartialSigned['complete'], False) #node1 only has one key, can't comp. sign the tx
+
+        rawTxPartialSigned2 = self.nodes[2].signrawtransaction(rawTx2, inputs)
+        self.log.info(rawTxPartialSigned2)
+        assert_equal(rawTxPartialSigned2['complete'], False) #node2 only has one key, can't comp. sign the tx
+
+        rawTxSignedComplete = self.nodes[2].signrawtransaction(rawTxPartialSigned1['hex'], inputs)
+        self.log.info(rawTxSignedComplete)
+        assert_equal(rawTxSignedComplete['complete'], True)
+        self.nodes[2].sendrawtransaction(rawTxSignedComplete['hex'])
+        rawTx2 = self.nodes[0].decoderawtransaction(rawTxSignedComplete['hex'])
+        self.sync_all()
+        self.nodes[0].generate(1)
+        self.sync_all()
+        assert_equal(self.nodes[0].getbalance(), bal+Decimal('250.00000000')+Decimal('2.19000000')) #block reward + tx
+
+        # decoderawtransaction tests
+        encrawtx = "01000000010000000000000072c1a6a246ae63f74f931e8365e15a089c68d61900000000000000000000ffffffff0100e1f505000000000000000000"
+        decrawtx = self.nodes[0].decoderawtransaction(encrawtx) # decode as non-witness transaction
+        assert_equal(decrawtx['vout'][0]['value'], Decimal('1.00000000'))
+
+        # getrawtransaction tests
+        # 1. valid parameters - only supply txid
+        txHash = rawTx["txid"]
+        assert_equal(self.nodes[0].getrawtransaction(txHash), rawTxSigned['hex'])
+
+        # 2. valid parameters - supply txid and 0 for non-verbose
+        assert_equal(self.nodes[0].getrawtransaction(txHash, 0), rawTxSigned['hex'])
+
+        # 3. valid parameters - supply txid and False for non-verbose
+        assert_equal(self.nodes[0].getrawtransaction(txHash, False), rawTxSigned['hex'])
+
+        # 4. valid parameters - supply txid and 1 for verbose.
+        # We only check the "hex" field of the output so we don't need to update this test every time the output format changes.
+        assert_equal(self.nodes[0].getrawtransaction(txHash, 1)["hex"], rawTxSigned['hex'])
+
+        # 5. valid parameters - supply txid and True for non-verbose
+        assert_equal(self.nodes[0].getrawtransaction(txHash, True)["hex"], rawTxSigned['hex'])
+
+        inputs  = [ {'txid' : "1d1d4e24ed99057e84c3f80fd8fbec79ed9e1acee37da269356ecea000000000", 'vout' : 1, 'sequence' : 1000}]
+        outputs = { self.nodes[0].getnewaddress() : 1 }
+        rawtx   = self.nodes[0].createrawtransaction(inputs, outputs)
+        decrawtx= self.nodes[0].decoderawtransaction(rawtx)
+        assert_equal(decrawtx['vin'][0]['sequence'], 1000)
+
+        # 9. invalid parameters - sequence number out of range
+        inputs  = [ {'txid' : "1d1d4e24ed99057e84c3f80fd8fbec79ed9e1acee37da269356ecea000000000", 'vout' : 1, 'sequence' : -1}]
+        outputs = { self.nodes[0].getnewaddress() : 1 }
+        assert_raises_rpc_error(-8, 'Invalid parameter, sequence number is out of range', self.nodes[0].createrawtransaction, inputs, outputs)
+
+        # 10. invalid parameters - sequence number out of range
+        inputs  = [ {'txid' : "1d1d4e24ed99057e84c3f80fd8fbec79ed9e1acee37da269356ecea000000000", 'vout' : 1, 'sequence' : 4294967296}]
+        outputs = { self.nodes[0].getnewaddress() : 1 }
+        assert_raises_rpc_error(-8, 'Invalid parameter, sequence number is out of range', self.nodes[0].createrawtransaction, inputs, outputs)
+
+        inputs  = [ {'txid' : "1d1d4e24ed99057e84c3f80fd8fbec79ed9e1acee37da269356ecea000000000", 'vout' : 1, 'sequence' : 4294967294}]
+        outputs = { self.nodes[0].getnewaddress() : 1 }
+        rawtx   = self.nodes[0].createrawtransaction(inputs, outputs)
+        decrawtx= self.nodes[0].decoderawtransaction(rawtx)
+        assert_equal(decrawtx['vin'][0]['sequence'], 4294967294)
+
+if __name__ == '__main__':
+    RawTransactionsTest().main()

test/functional/rpc_signmessage.py

@@ -0,0 +1,38 @@
+#!/usr/bin/env python3
+# Copyright (c) 2016-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test RPC commands for signing and verifying messages."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import assert_equal
+
+class SignMessagesTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+
+    def run_test(self):
+        message = 'This is just a test message'
+
+        #self.log.info('test signing with priv_key')
+        #priv_key = 'cUeKHd5orzT3mz8P9pxyREHfsWtVfgsfDjiZZBcjUBAaGk1BTj7N'
+        #address = 'mpLQjfK79b7CCV4VMJWEWAj5Mpx8Up5zxB'
+        #expected_signature = 'INbVnW4e6PeRmsv2Qgu8NuopvrVjkcxob+sX8OcZG0SALhWybUjzMLPdAsXI46YZGb0KQTRii+wWIQzRpG/U+S0='
+        #signature = self.nodes[0].signmessagewithprivkey(priv_key, message)
+        #assert_equal(expected_signature, signature)
+        #assert(self.nodes[0].verifymessage(address, signature, message))
+
+        self.log.info('test signing with an address with wallet')
+        address = self.nodes[0].getnewaddress()
+        signature = self.nodes[0].signmessage(address, message)
+        assert(self.nodes[0].verifymessage(address, signature, message))
+
+        self.log.info('test verifying with another address should not work')
+        other_address = self.nodes[0].getnewaddress()
+        other_signature = self.nodes[0].signmessage(other_address, message)
+        assert(not self.nodes[0].verifymessage(other_address, signature, message))
+        assert(not self.nodes[0].verifymessage(address, other_signature, message))
+
+if __name__ == '__main__':
+    SignMessagesTest().main()

test/functional/rpc_signrawtransaction.py

@@ -0,0 +1,104 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test transaction signing using the signrawtransaction RPC."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+
+class SignRawTransactionsTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+
+    def successful_signing_test(self):
+        """Create and sign a valid raw transaction with one input.
+
+        Expected results:
+
+        1) The transaction has a complete set of signatures
+        2) No script verification error occurred"""
+        privKeys = ['cUeKHd5orzT3mz8P9pxyREHfsWtVfgsfDjiZZBcjUBAaGk1BTj7N']
+
+        inputs = [
+            # Valid pay-to-pubkey script
+            {'txid': '9b907ef1e3c26fc71fe4a4b3580bc75264112f95050014157059c736f0202e71', 'vout': 0,
+             'scriptPubKey': '76a91460baa0f494b38ce3c940dea67f3804dc52d1fb9488ac'}
+        ]
+
+        outputs = {'xwMWGTnBNUmGxMm8vfAdbL45bWXyVTYctd': 0.1}
+
+        rawTx = self.nodes[0].createrawtransaction(inputs, outputs)
+        rawTxSigned = self.nodes[0].signrawtransaction(rawTx, inputs, privKeys)
+
+        # 1) The transaction has a complete set of signatures
+        assert 'complete' in rawTxSigned
+        assert_equal(rawTxSigned['complete'], True)
+
+        # 2) No script verification error occurred
+        assert 'errors' not in rawTxSigned
+
+    def script_verification_error_test(self):
+        """Create and sign a raw transaction with valid (vin 0), invalid (vin 1) and one missing (vin 2) input script.
+
+        Expected results:
+
+        3) The transaction has no complete set of signatures
+        4) Two script verification errors occurred
+        5) Script verification errors have certain properties ("txid", "vout", "scriptSig", "sequence", "error")
+        6) The verification errors refer to the invalid (vin 1) and missing input (vin 2)"""
+        privKeys = ['cUeKHd5orzT3mz8P9pxyREHfsWtVfgsfDjiZZBcjUBAaGk1BTj7N']
+
+        inputs = [
+            # Valid pay-to-pubkey script
+            {'txid': '9b907ef1e3c26fc71fe4a4b3580bc75264112f95050014157059c736f0202e71', 'vout': 0},
+            # Invalid script
+            {'txid': '5b8673686910442c644b1f4993d8f7753c7c8fcb5c87ee40d56eaeef25204547', 'vout': 7},
+            # Missing scriptPubKey
+            {'txid': '9b907ef1e3c26fc71fe4a4b3580bc75264112f95050014157059c736f0202e71', 'vout': 1},
+        ]
+
+        scripts = [
+            # Valid pay-to-pubkey script
+            {'txid': '9b907ef1e3c26fc71fe4a4b3580bc75264112f95050014157059c736f0202e71', 'vout': 0,
+             'scriptPubKey': '76a91460baa0f494b38ce3c940dea67f3804dc52d1fb9488ac'},
+            # Invalid script
+            {'txid': '5b8673686910442c644b1f4993d8f7753c7c8fcb5c87ee40d56eaeef25204547', 'vout': 7,
+             'scriptPubKey': 'badbadbadbad'}
+        ]
+
+        outputs = {'xwMWGTnBNUmGxMm8vfAdbL45bWXyVTYctd': 0.1}
+
+        rawTx = self.nodes[0].createrawtransaction(inputs, outputs)
+        rawTxSigned = self.nodes[0].signrawtransaction(rawTx, scripts, privKeys)
+
+        # 3) The transaction has no complete set of signatures
+        assert 'complete' in rawTxSigned
+        assert_equal(rawTxSigned['complete'], False)
+
+        # 4) Two script verification errors occurred
+        assert 'errors' in rawTxSigned
+        assert_equal(len(rawTxSigned['errors']), 2)
+
+        # 5) Script verification errors have certain properties
+        assert 'txid' in rawTxSigned['errors'][0]
+        assert 'vout' in rawTxSigned['errors'][0]
+        assert 'scriptSig' in rawTxSigned['errors'][0]
+        assert 'sequence' in rawTxSigned['errors'][0]
+        assert 'error' in rawTxSigned['errors'][0]
+
+        # 6) The verification errors refer to the invalid (vin 1) and missing input (vin 2)
+        assert_equal(rawTxSigned['errors'][0]['txid'], inputs[1]['txid'])
+        assert_equal(rawTxSigned['errors'][0]['vout'], inputs[1]['vout'])
+        assert_equal(rawTxSigned['errors'][1]['txid'], inputs[2]['txid'])
+        assert_equal(rawTxSigned['errors'][1]['vout'], inputs[2]['vout'])
+
+    def run_test(self):
+        self.successful_signing_test()
+        self.script_verification_error_test()
+
+
+if __name__ == '__main__':
+    SignRawTransactionsTest().main()

test/functional/rpc_users.py

@@ -0,0 +1,153 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test multiple RPC users."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import str_to_b64str, assert_equal
+
+import os
+import http.client
+import urllib.parse
+
+class HTTPBasicsTest (BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+
+    def setup_chain(self):
+        super().setup_chain()
+        #Append rpcauth to bitcoin.conf before initialization
+        rpcauth = "rpcauth=rt:93648e835a54c573682c2eb19f882535$7681e9c5b74bdd85e78166031d2058e1069b3ed7ed967c93fc63abba06f31144"
+        rpcauth2 = "rpcauth=rt2:f8607b1a88861fac29dfccf9b52ff9f$ff36a0c23c8c62b4846112e50fa888416e94c17bfd4c42f88fd8f55ec6a3137e"
+        rpcuser = "rpcuser=rpcuser�"
+        rpcpassword = "rpcpassword=rpcpassword�"
+        with open(os.path.join(self.options.tmpdir+"/node0", "agrarian.conf"), 'a', encoding='utf8') as f:
+            f.write(rpcauth+"\n")
+            f.write(rpcauth2+"\n")
+        with open(os.path.join(self.options.tmpdir+"/node1", "agrarian.conf"), 'a', encoding='utf8') as f:
+            f.write(rpcuser+"\n")
+            f.write(rpcpassword+"\n")
+
+    def run_test(self):
+
+        ##################################################
+        # Check correctness of the rpcauth config option #
+        ##################################################
+        url = urllib.parse.urlparse(self.nodes[0].url)
+
+        #Old authpair
+        authpair = url.username + ':' + url.password
+
+        #New authpair generated via share/rpcuser tool
+        password = "cA773lm788buwYe4g4WT+05pKyNruVKjQ25x3n0DQcM="
+
+        #Second authpair with different username
+        password2 = "8/F3uMDw4KSEbw96U3CA1C4X05dkHDN2BPFjTgZW4KI="
+        authpairnew = "rt:"+password
+
+        headers = {"Authorization": "Basic " + str_to_b64str(authpair)}
+
+        conn = http.client.HTTPConnection(url.hostname, url.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        resp = conn.getresponse()
+        assert_equal(resp.status, 200)
+        conn.close()
+
+        #Use new authpair to confirm both work
+        headers = {"Authorization": "Basic " + str_to_b64str(authpairnew)}
+
+        conn = http.client.HTTPConnection(url.hostname, url.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        resp = conn.getresponse()
+        assert_equal(resp.status, 200)
+        conn.close()
+
+        #Wrong login name with rt's password
+        authpairnew = "rtwrong:"+password
+        headers = {"Authorization": "Basic " + str_to_b64str(authpairnew)}
+
+        conn = http.client.HTTPConnection(url.hostname, url.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        resp = conn.getresponse()
+        assert_equal(resp.status, 401)
+        conn.close()
+
+        #Wrong password for rt
+        authpairnew = "rt:"+password+"wrong"
+        headers = {"Authorization": "Basic " + str_to_b64str(authpairnew)}
+
+        conn = http.client.HTTPConnection(url.hostname, url.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        resp = conn.getresponse()
+        assert_equal(resp.status, 401)
+        conn.close()
+
+        #Correct for rt2
+        authpairnew = "rt2:"+password2
+        headers = {"Authorization": "Basic " + str_to_b64str(authpairnew)}
+
+        conn = http.client.HTTPConnection(url.hostname, url.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        resp = conn.getresponse()
+        assert_equal(resp.status, 200)
+        conn.close()
+
+        #Wrong password for rt2
+        authpairnew = "rt2:"+password2+"wrong"
+        headers = {"Authorization": "Basic " + str_to_b64str(authpairnew)}
+
+        conn = http.client.HTTPConnection(url.hostname, url.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        resp = conn.getresponse()
+        assert_equal(resp.status, 401)
+        conn.close()
+
+        ###############################################################
+        # Check correctness of the rpcuser/rpcpassword config options #
+        ###############################################################
+        url = urllib.parse.urlparse(self.nodes[1].url)
+
+        # rpcuser and rpcpassword authpair
+        rpcuserauthpair = "rpcuser�:rpcpassword�"
+
+        headers = {"Authorization": "Basic " + str_to_b64str(rpcuserauthpair)}
+
+        conn = http.client.HTTPConnection(url.hostname, url.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        resp = conn.getresponse()
+        assert_equal(resp.status, 200)
+        conn.close()
+
+        #Wrong login name with rpcuser's password
+        rpcuserauthpair = "rpcuserwrong:rpcpassword"
+        headers = {"Authorization": "Basic " + str_to_b64str(rpcuserauthpair)}
+
+        conn = http.client.HTTPConnection(url.hostname, url.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        resp = conn.getresponse()
+        assert_equal(resp.status, 401)
+        conn.close()
+
+        #Wrong password for rpcuser
+        rpcuserauthpair = "rpcuser:rpcpasswordwrong"
+        headers = {"Authorization": "Basic " + str_to_b64str(rpcuserauthpair)}
+
+        conn = http.client.HTTPConnection(url.hostname, url.port)
+        conn.connect()
+        conn.request('POST', '/', '{"method": "getbestblockhash"}', headers)
+        resp = conn.getresponse()
+        assert_equal(resp.status, 401)
+        conn.close()
+
+
+if __name__ == '__main__':
+    HTTPBasicsTest ().main ()

test/functional/test_framework/address.py

@@ -0,0 +1,69 @@
+#!/usr/bin/env python3
+# Copyright (c) 2016 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Encode and decode BASE58, P2PKH and P2SH addresses."""
+
+from .script import hash256, hash160, sha256, CScript, OP_0
+from .util import bytes_to_hex_str, hex_str_to_bytes
+
+chars = '123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz'
+
+def byte_to_base58(b, version):
+    result = ''
+    str = bytes_to_hex_str(b)
+    str = bytes_to_hex_str(chr(version).encode('latin-1')) + str
+    checksum = bytes_to_hex_str(hash256(hex_str_to_bytes(str)))
+    str += checksum[:8]
+    value = int('0x'+str,0)
+    while value > 0:
+        result = chars[value % 58] + result
+        value //= 58
+    while (str[:2] == '00'):
+        result = chars[0] + result
+        str = str[2:]
+    return result
+
+# TODO: def base58_decode
+
+def keyhash_to_p2pkh(hash, main = False):
+    assert (len(hash) == 20)
+    version = 0 if main else 111
+    return byte_to_base58(hash, version)
+
+def scripthash_to_p2sh(hash, main = False):
+    assert (len(hash) == 20)
+    version = 5 if main else 196
+    return byte_to_base58(hash, version)
+
+def key_to_p2pkh(key, main = False):
+    key = check_key(key)
+    return keyhash_to_p2pkh(hash160(key), main)
+
+def script_to_p2sh(script, main = False):
+    script = check_script(script)
+    return scripthash_to_p2sh(hash160(script), main)
+
+def key_to_p2sh_p2wpkh(key, main = False):
+    key = check_key(key)
+    p2shscript = CScript([OP_0, hash160(key)])
+    return script_to_p2sh(p2shscript, main)
+
+def script_to_p2sh_p2wsh(script, main = False):
+    script = check_script(script)
+    p2shscript = CScript([OP_0, sha256(script)])
+    return script_to_p2sh(p2shscript, main)
+
+def check_key(key):
+    if (type(key) is str):
+        key = hex_str_to_bytes(key) # Assuming this is hex string
+    if (type(key) is bytes and (len(key) == 33 or len(key) == 65)):
+        return key
+    assert(False)
+
+def check_script(script):
+    if (type(script) is str):
+        script = hex_str_to_bytes(script) # Assuming this is hex string
+    if (type(script) is bytes or type(script) is CScript):
+        return script
+    assert(False)

test/functional/test_framework/authproxy.py

@@ -0,0 +1,178 @@
+# Copyright (c) 2011 Jeff Garzik
+#
+# Previous copyright, from python-jsonrpc/jsonrpc/proxy.py:
+#
+# Copyright (c) 2007 Jan-Klaas Kollhof
+#
+# This file is part of jsonrpc.
+#
+# jsonrpc is free software; you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License as published by
+# the Free Software Foundation; either version 2.1 of the License, or
+# (at your option) any later version.
+#
+# This software is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this software; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+"""HTTP proxy for opening RPC connection to agrariand.
+
+AuthServiceProxy has the following improvements over python-jsonrpc's
+ServiceProxy class:
+
+- HTTP connections persist for the life of the AuthServiceProxy object
+  (if server supports HTTP/1.1)
+- sends protocol 'version', per JSON-RPC 1.1
+- sends proper, incrementing 'id'
+- sends Basic HTTP authentication headers
+- parses all JSON numbers that look like floats as Decimal
+- uses standard Python json lib
+"""
+
+import base64
+import decimal
+import http.client
+import json
+import logging
+import socket
+import time
+import urllib.parse
+
+HTTP_TIMEOUT = 300
+USER_AGENT = "AuthServiceProxy/0.1"
+
+log = logging.getLogger("BitcoinRPC")
+
+class JSONRPCException(Exception):
+    def __init__(self, rpc_error):
+        try:
+            errmsg = '%(message)s (%(code)i)' % rpc_error
+        except (KeyError, TypeError):
+            errmsg = ''
+        super().__init__(errmsg)
+        self.error = rpc_error
+
+
+def EncodeDecimal(o):
+    if isinstance(o, decimal.Decimal):
+        return str(o)
+    raise TypeError(repr(o) + " is not JSON serializable")
+
+class AuthServiceProxy():
+    __id_count = 0
+
+    # ensure_ascii: escape unicode as \uXXXX, passed to json.dumps
+    def __init__(self, service_url, service_name=None, timeout=HTTP_TIMEOUT, connection=None, ensure_ascii=True):
+        self.__service_url = service_url
+        self._service_name = service_name
+        self.ensure_ascii = ensure_ascii  # can be toggled on the fly by tests
+        self.__url = urllib.parse.urlparse(service_url)
+        port = 80 if self.__url.port is None else self.__url.port
+        user = None if self.__url.username is None else self.__url.username.encode('utf8')
+        passwd = None if self.__url.password is None else self.__url.password.encode('utf8')
+        authpair = user + b':' + passwd
+        self.__auth_header = b'Basic ' + base64.b64encode(authpair)
+
+        if connection:
+            # Callables re-use the connection of the original proxy
+            self.__conn = connection
+        elif self.__url.scheme == 'https':
+            self.__conn = http.client.HTTPSConnection(self.__url.hostname, port, timeout=timeout)
+        else:
+            self.__conn = http.client.HTTPConnection(self.__url.hostname, port, timeout=timeout)
+
+    def __getattr__(self, name):
+        if name.startswith('__') and name.endswith('__'):
+            # Python internal stuff
+            raise AttributeError
+        if self._service_name is not None:
+            name = "%s.%s" % (self._service_name, name)
+        return AuthServiceProxy(self.__service_url, name, connection=self.__conn)
+
+    def _request(self, method, path, postdata):
+        '''
+        Do a HTTP request, with retry if we get disconnected (e.g. due to a timeout).
+        This is a workaround for https://bugs.python.org/issue3566 which is fixed in Python 3.5.
+        '''
+        headers = {'Host': self.__url.hostname,
+                   'User-Agent': USER_AGENT,
+                   'Authorization': self.__auth_header,
+                   'Content-type': 'application/json'}
+        try:
+            self.__conn.request(method, path, postdata, headers)
+            return self._get_response()
+        except http.client.BadStatusLine as e:
+            if e.line == "''":  # if connection was closed, try again
+                self.__conn.close()
+                self.__conn.request(method, path, postdata, headers)
+                return self._get_response()
+            else:
+                raise
+        except (BrokenPipeError, ConnectionResetError):
+            # Python 3.5+ raises BrokenPipeError instead of BadStatusLine when the connection was reset
+            # ConnectionResetError happens on FreeBSD with Python 3.4
+            self.__conn.close()
+            self.__conn.request(method, path, postdata, headers)
+            return self._get_response()
+
+    def get_request(self, *args):
+        AuthServiceProxy.__id_count += 1
+
+        log.debug("-%s-> %s %s" % (AuthServiceProxy.__id_count, self._service_name,
+                                   json.dumps(args, default=EncodeDecimal, ensure_ascii=self.ensure_ascii)))
+        return {'version': '1.1',
+                'method': self._service_name,
+                'params': args,
+                'id': AuthServiceProxy.__id_count}
+
+    def __call__(self, *args, **argsn):
+        postdata = json.dumps(self.get_request(*args), default=EncodeDecimal, ensure_ascii=self.ensure_ascii)
+        response = self._request('POST', self.__url.path, postdata.encode('utf-8'))
+        if response['error'] is not None:
+            raise JSONRPCException(response['error'])
+        elif 'result' not in response:
+            raise JSONRPCException({
+                'code': -343, 'message': 'missing JSON-RPC result'})
+        else:
+            return response['result']
+
+    def batch(self, rpc_call_list):
+        postdata = json.dumps(list(rpc_call_list), default=EncodeDecimal, ensure_ascii=self.ensure_ascii)
+        log.debug("--> " + postdata)
+        return self._request('POST', self.__url.path, postdata.encode('utf-8'))
+
+    def _get_response(self):
+        req_start_time = time.time()
+        try:
+            http_response = self.__conn.getresponse()
+        except socket.timeout as e:
+            raise JSONRPCException({
+                'code': -344,
+                'message': '%r RPC took longer than %f seconds. Consider '
+                           'using larger timeout for calls that take '
+                           'longer to return.' % (self._service_name,
+                                                  self.__conn.timeout)})
+        if http_response is None:
+            raise JSONRPCException({
+                'code': -342, 'message': 'missing HTTP response from server'})
+
+        content_type = http_response.getheader('Content-Type')
+        if content_type != 'application/json':
+            raise JSONRPCException({
+                'code': -342, 'message': 'non-JSON HTTP response with \'%i %s\' from server' % (http_response.status, http_response.reason)})
+
+        responsedata = http_response.read().decode('utf8')
+        response = json.loads(responsedata, parse_float=decimal.Decimal)
+        elapsed = time.time() - req_start_time
+        if "error" in response and response["error"] is None:
+            log.debug("<-%s- [%.6f] %s" % (response["id"], elapsed, json.dumps(response["result"], default=EncodeDecimal, ensure_ascii=self.ensure_ascii)))
+        else:
+            log.debug("<-- [%.6f] %s" % (elapsed, responsedata))
+        return response
+
+    def __truediv__(self, relative_uri):
+        return AuthServiceProxy("{}/{}".format(self.__service_url, relative_uri), self._service_name, connection=self.__conn)

test/functional/test_framework/bignum.py

@@ -0,0 +1,97 @@
+#!/usr/bin/env python3
+#
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Big number routines.
+
+This file is copied from python-bitcoinlib.
+"""
+
+import struct
+
+
+# generic big endian MPI format
+
+def bn_bytes(v, have_ext=False):
+    ext = 0
+    if have_ext:
+        ext = 1
+    return ((v.bit_length()+7)//8) + ext
+
+def bn2bin(v):
+    s = bytearray()
+    i = bn_bytes(v)
+    while i > 0:
+        s.append((v >> ((i-1) * 8)) & 0xff)
+        i -= 1
+    return s
+
+def bin2bn(s):
+    l = 0
+    for ch in s:
+        l = (l << 8) | ch
+    return l
+
+def bn2mpi(v):
+    have_ext = False
+    if v.bit_length() > 0:
+        have_ext = (v.bit_length() & 0x07) == 0
+
+    neg = False
+    if v < 0:
+        neg = True
+        v = -v
+
+    s = struct.pack(b">I", bn_bytes(v, have_ext))
+    ext = bytearray()
+    if have_ext:
+        ext.append(0)
+    v_bin = bn2bin(v)
+    if neg:
+        if have_ext:
+            ext[0] |= 0x80
+        else:
+            v_bin[0] |= 0x80
+    return s + ext + v_bin
+
+def mpi2bn(s):
+    if len(s) < 4:
+        return None
+    s_size = bytes(s[:4])
+    v_len = struct.unpack(b">I", s_size)[0]
+    if len(s) != (v_len + 4):
+        return None
+    if v_len == 0:
+        return 0
+
+    v_str = bytearray(s[4:])
+    neg = False
+    i = v_str[0]
+    if i & 0x80:
+        neg = True
+        i &= ~0x80
+        v_str[0] = i
+
+    v = bin2bn(v_str)
+
+    if neg:
+        return -v
+    return v
+
+# agrarian-specific little endian format, with implicit size
+def mpi2vch(s):
+    r = s[4:]           # strip size
+    r = r[::-1]         # reverse string, converting BE->LE
+    return r
+
+def bn2vch(v):
+    return bytes(mpi2vch(bn2mpi(v)))
+
+def vch2mpi(s):
+    r = struct.pack(b">I", len(s))   # size
+    r += s[::-1]            # reverse string, converting LE->BE
+    return r
+
+def vch2bn(s):
+    return mpi2bn(vch2mpi(s))
+

test/functional/test_framework/blockstore.py

@@ -0,0 +1,170 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2016 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""BlockStore and TxStore helper classes."""
+
+from .mininode import *
+from io import BytesIO
+import dbm.dumb as dbmd
+
+logger = logging.getLogger("TestFramework.blockstore")
+
+class BlockStore():
+    """BlockStore helper class.
+
+    BlockStore keeps a map of blocks and implements helper functions for
+    responding to getheaders and getdata, and for constructing a getheaders
+    message.
+    """
+
+    def __init__(self, datadir):
+        self.blockDB = dbmd.open(datadir + "/blocks", 'c')
+        self.currentBlock = 0
+        self.headers_map = dict()
+
+    def close(self):
+        self.blockDB.close()
+
+    def erase(self, blockhash):
+        del self.blockDB[repr(blockhash)]
+
+    # lookup an entry and return the item as raw bytes
+    def get(self, blockhash):
+        value = None
+        try:
+            value = self.blockDB[repr(blockhash)]
+        except KeyError:
+            return None
+        return value
+
+    # lookup an entry and return it as a CBlock
+    def get_block(self, blockhash):
+        ret = None
+        serialized_block = self.get(blockhash)
+        if serialized_block is not None:
+            f = BytesIO(serialized_block)
+            ret = CBlock()
+            ret.deserialize(f)
+            ret.calc_sha256()
+        return ret
+
+    def get_header(self, blockhash):
+        try:
+            return self.headers_map[blockhash]
+        except KeyError:
+            return None
+
+    # Note: this pulls full blocks out of the database just to retrieve
+    # the headers -- perhaps we could keep a separate data structure
+    # to avoid this overhead.
+    def headers_for(self, locator, hash_stop, current_tip=None):
+        if current_tip is None:
+            current_tip = self.currentBlock
+        current_block_header = self.get_header(current_tip)
+        if current_block_header is None:
+            return None
+
+        response = msg_headers()
+        headersList = [ current_block_header ]
+        maxheaders = 2000
+        while (headersList[0].sha256 not in locator.vHave):
+            prevBlockHash = headersList[0].hashPrevBlock
+            prevBlockHeader = self.get_header(prevBlockHash)
+            if prevBlockHeader is not None:
+                headersList.insert(0, prevBlockHeader)
+            else:
+                break
+        headersList = headersList[:maxheaders] # truncate if we have too many
+        hashList = [x.sha256 for x in headersList]
+        index = len(headersList)
+        if (hash_stop in hashList):
+            index = hashList.index(hash_stop)+1
+        response.headers = headersList[:index]
+        return response
+
+    def add_block(self, block):
+        block.calc_sha256()
+        try:
+            self.blockDB[repr(block.sha256)] = bytes(block.serialize())
+        except TypeError as e:
+            logger.exception("Unexpected error")
+        self.currentBlock = block.sha256
+        self.headers_map[block.sha256] = CBlockHeader(block)
+
+    def add_header(self, header):
+        self.headers_map[header.sha256] = header
+
+    # lookup the hashes in "inv", and return p2p messages for delivering
+    # blocks found.
+    def get_blocks(self, inv):
+        responses = []
+        for i in inv:
+            if (i.type == 2): # MSG_BLOCK
+                data = self.get(i.hash)
+                if data is not None:
+                    # Use msg_generic to avoid re-serialization
+                    responses.append(msg_generic(b"block", data))
+        return responses
+
+    def get_locator(self, current_tip=None):
+        if current_tip is None:
+            current_tip = self.currentBlock
+        r = []
+        counter = 0
+        step = 1
+        lastBlock = self.get_block(current_tip)
+        while lastBlock is not None:
+            r.append(lastBlock.hashPrevBlock)
+            for i in range(step):
+                lastBlock = self.get_block(lastBlock.hashPrevBlock)
+                if lastBlock is None:
+                    break
+            counter += 1
+            if counter > 10:
+                step *= 2
+        locator = CBlockLocator()
+        locator.vHave = r
+        return locator
+
+class TxStore():
+    def __init__(self, datadir):
+        self.txDB = dbmd.open(datadir + "/transactions", 'c')
+
+    def close(self):
+        self.txDB.close()
+
+    # lookup an entry and return the item as raw bytes
+    def get(self, txhash):
+        value = None
+        try:
+            value = self.txDB[repr(txhash)]
+        except KeyError:
+            return None
+        return value
+
+    def get_transaction(self, txhash):
+        ret = None
+        serialized_tx = self.get(txhash)
+        if serialized_tx is not None:
+            f = BytesIO(serialized_tx)
+            ret = CTransaction()
+            ret.deserialize(f)
+            ret.calc_sha256()
+        return ret
+
+    def add_transaction(self, tx):
+        tx.calc_sha256()
+        try:
+            self.txDB[repr(tx.sha256)] = bytes(tx.serialize())
+        except TypeError as e:
+            logger.exception("Unexpected error")
+
+    def get_transactions(self, inv):
+        responses = []
+        for i in inv:
+            if (i.type == 1): # MSG_TX
+                tx = self.get(i.hash)
+                if tx is not None:
+                    responses.append(msg_generic(b"tx", tx))
+        return responses

test/functional/test_framework/blocktools.py

@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2016 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Utilities for manipulating blocks and transactions."""
+
+from .mininode import *
+from .script import CScript, OP_TRUE, OP_CHECKSIG, OP_RETURN
+
+# Create a block (with regtest difficulty)
+def create_block(hashprev, coinbase, nTime=None):
+    block = CBlock()
+    if nTime is None:
+        import time
+        block.nTime = int(time.time()+600)
+    else:
+        block.nTime = nTime
+    block.hashPrevBlock = hashprev
+    block.nBits = 0x1e0ffff0 # Will break after a difficulty adjustment...
+    block.nAccumulatorCheckpoint = 0
+    block.vtx.append(coinbase)
+    block.hashMerkleRoot = block.calc_merkle_root()
+    block.calc_sha256()
+    return block
+
+def serialize_script_num(value):
+    r = bytearray(0)
+    if value == 0:
+        return r
+    neg = value < 0
+    absvalue = -value if neg else value
+    while (absvalue):
+        r.append(int(absvalue & 0xff))
+        absvalue >>= 8
+    if r[-1] & 0x80:
+        r.append(0x80 if neg else 0)
+    elif neg:
+        r[-1] |= 0x80
+    return r
+
+# Create a coinbase transaction, assuming no miner fees.
+# If pubkey is passed in, the coinbase output will be a P2PK output;
+# otherwise an anyone-can-spend output.
+def create_coinbase(height, pubkey = None):
+    coinbase = CTransaction()
+    coinbase.vin.append(CTxIn(COutPoint(0, 0xffffffff),
+                ser_string(serialize_script_num(height)), 0xffffffff))
+    coinbaseoutput = CTxOut()
+    coinbaseoutput.nValue = 50 * COIN
+    halvings = int(height/150) # regtest
+    coinbaseoutput.nValue >>= halvings
+    if (pubkey != None):
+        coinbaseoutput.scriptPubKey = CScript([pubkey, OP_CHECKSIG])
+    else:
+        coinbaseoutput.scriptPubKey = CScript([OP_TRUE])
+    coinbase.vout = [ coinbaseoutput ]
+    coinbase.calc_sha256()
+    return coinbase
+
+# Create a transaction.
+# If the scriptPubKey is not specified, make it anyone-can-spend.
+def create_transaction(prevtx, n, sig, value, scriptPubKey=CScript()):
+    tx = CTransaction()
+    assert(n < len(prevtx.vout))
+    tx.vin.append(CTxIn(COutPoint(prevtx.sha256, n), sig, 0xffffffff))
+    tx.vout.append(CTxOut(value, scriptPubKey))
+    tx.calc_sha256()
+    return tx
+
+def get_legacy_sigopcount_block(block, fAccurate=True):
+    count = 0
+    for tx in block.vtx:
+        count += get_legacy_sigopcount_tx(tx, fAccurate)
+    return count
+
+def get_legacy_sigopcount_tx(tx, fAccurate=True):
+    count = 0
+    for i in tx.vout:
+        count += i.scriptPubKey.GetSigOpCount(fAccurate)
+    for j in tx.vin:
+        # scriptSig might be of type bytes, so convert to CScript for the moment
+        count += CScript(j.scriptSig).GetSigOpCount(fAccurate)
+    return count

test/functional/test_framework/comptool.py

@@ -0,0 +1,397 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2016 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Compare two or more agrariands to each other.
+
+To use, create a class that implements get_tests(), and pass it in
+as the test generator to TestManager.  get_tests() should be a python
+generator that returns TestInstance objects.  See below for definition.
+
+TestNode behaves as follows:
+    Configure with a BlockStore and TxStore
+    on_inv: log the message but don't request
+    on_headers: log the chain tip
+    on_pong: update ping response map (for synchronization)
+    on_getheaders: provide headers via BlockStore
+    on_getdata: provide blocks via BlockStore
+"""
+
+from .mininode import *
+from .blockstore import BlockStore, TxStore
+from .util import p2p_port, wait_until
+
+import logging
+
+logger=logging.getLogger("TestFramework.comptool")
+
+global mininode_lock
+
+class RejectResult():
+    """Outcome that expects rejection of a transaction or block."""
+    def __init__(self, code, reason=b''):
+        self.code = code
+        self.reason = reason
+    def match(self, other):
+        if self.code != other.code:
+            return False
+        return other.reason.startswith(self.reason)
+    def __repr__(self):
+        return '%i:%s' % (self.code,self.reason or '*')
+
+class TestNode(P2PInterface):
+
+    def __init__(self, block_store, tx_store):
+        super().__init__()
+        self.bestblockhash = None
+        self.block_store = block_store
+        self.block_request_map = {}
+        self.tx_store = tx_store
+        self.tx_request_map = {}
+        self.block_reject_map = {}
+        self.tx_reject_map = {}
+
+        # When the pingmap is non-empty we're waiting for
+        # a response
+        self.pingMap = {}
+        self.lastInv = []
+        self.closed = False
+
+    def on_close(self):
+        self.closed = True
+
+    def on_headers(self, message):
+        if len(message.headers) > 0:
+            best_header = message.headers[-1]
+            best_header.calc_sha256()
+            self.bestblockhash = best_header.sha256
+
+    def on_getheaders(self, message):
+        response = self.block_store.headers_for(message.locator, message.hashstop)
+        if response is not None:
+            self.send_message(response)
+
+    def on_getdata(self, message):
+        [self.send_message(r) for r in self.block_store.get_blocks(message.inv)]
+        [self.send_message(r) for r in self.tx_store.get_transactions(message.inv)]
+
+        for i in message.inv:
+            if i.type == 1 or i.type == 1 | (1 << 30): # MSG_TX or MSG_WITNESS_TX
+                self.tx_request_map[i.hash] = True
+            elif i.type == 2 or i.type == 2 | (1 << 30): # MSG_BLOCK or MSG_WITNESS_BLOCK
+                self.block_request_map[i.hash] = True
+
+    def on_inv(self, message):
+        self.lastInv = [x.hash for x in message.inv]
+
+    def on_pong(self, message):
+        try:
+            del self.pingMap[message.nonce]
+        except KeyError:
+            raise AssertionError("Got pong for unknown ping [%s]" % repr(message))
+
+    def on_reject(self, message):
+        if message.message == b'tx':
+            self.tx_reject_map[message.data] = RejectResult(message.code, message.reason)
+        if message.message == b'block':
+            self.block_reject_map[message.data] = RejectResult(message.code, message.reason)
+
+    def send_inv(self, obj):
+        mtype = 2 if isinstance(obj, CBlock) else 1
+        self.send_message(msg_inv([CInv(mtype, obj.sha256)]))
+
+    def send_getheaders(self):
+        # We ask for headers from their last tip.
+        m = msg_getheaders()
+        m.locator = self.block_store.get_locator(self.bestblockhash)
+        self.send_message(m)
+
+    def send_header(self, header):
+        m = msg_headers()
+        m.headers.append(header)
+        self.send_message(m)
+
+    # This assumes BIP31
+    def send_ping(self, nonce):
+        self.pingMap[nonce] = True
+        self.send_message(msg_ping(nonce))
+
+    def received_ping_response(self, nonce):
+        return nonce not in self.pingMap
+
+    def send_mempool(self):
+        self.lastInv = []
+        self.send_message(msg_mempool())
+
+# TestInstance:
+#
+# Instances of these are generated by the test generator, and fed into the
+# comptool.
+#
+# "blocks_and_transactions" should be an array of
+#    [obj, True/False/None, hash/None]:
+#  - obj is either a CBlock, CBlockHeader, or a CTransaction, and
+#  - the second value indicates whether the object should be accepted
+#    into the blockchain or mempool (for tests where we expect a certain
+#    answer), or "None" if we don't expect a certain answer and are just
+#    comparing the behavior of the nodes being tested.
+#  - the third value is the hash to test the tip against (if None or omitted,
+#    use the hash of the block)
+#  - NOTE: if a block header, no test is performed; instead the header is
+#    just added to the block_store.  This is to facilitate block delivery
+#    when communicating with headers-first clients (when withholding an
+#    intermediate block).
+# sync_every_block: if True, then each block will be inv'ed, synced, and
+#    nodes will be tested based on the outcome for the block.  If False,
+#    then inv's accumulate until all blocks are processed (or max inv size
+#    is reached) and then sent out in one inv message.  Then the final block
+#    will be synced across all connections, and the outcome of the final
+#    block will be tested.
+# sync_every_tx: analogous to behavior for sync_every_block, except if outcome
+#    on the final tx is None, then contents of entire mempool are compared
+#    across all connections.  (If outcome of final tx is specified as true
+#    or false, then only the last tx is tested against outcome.)
+
+class TestInstance():
+    def __init__(self, objects=None, sync_every_block=True, sync_every_tx=False):
+        self.blocks_and_transactions = objects if objects else []
+        self.sync_every_block = sync_every_block
+        self.sync_every_tx = sync_every_tx
+
+class TestManager():
+
+    def __init__(self, testgen, datadir):
+        self.test_generator = testgen
+        self.p2p_connections= []
+        self.block_store    = BlockStore(datadir)
+        self.tx_store       = TxStore(datadir)
+        self.ping_counter   = 1
+
+    def add_all_connections(self, nodes):
+        for i in range(len(nodes)):
+            # Create a p2p connection to each node
+            node = TestNode(self.block_store, self.tx_store)
+            node.peer_connect('127.0.0.1', p2p_port(i))
+            self.p2p_connections.append(node)
+
+    def clear_all_connections(self):
+        self.p2p_connections = []
+
+    def wait_for_disconnections(self):
+        def disconnected():
+            return all(node.closed for node in self.p2p_connections)
+        wait_until(disconnected, timeout=10, lock=mininode_lock)
+
+    def wait_for_verack(self):
+        return all(node.wait_for_verack() for node in self.p2p_connections)
+
+    def wait_for_pings(self, counter):
+        def received_pongs():
+            return all(node.received_ping_response(counter) for node in self.p2p_connections)
+        wait_until(received_pongs, lock=mininode_lock)
+
+    # sync_blocks: Wait for all connections to request the blockhash given
+    # then send get_headers to find out the tip of each node, and synchronize
+    # the response by using a ping (and waiting for pong with same nonce).
+    def sync_blocks(self, blockhash, num_blocks):
+        def blocks_requested():
+            return all(
+                blockhash in node.block_request_map and node.block_request_map[blockhash]
+                for node in self.p2p_connections
+            )
+
+        # --> error if not requested
+        wait_until(blocks_requested, attempts=20*num_blocks, lock=mininode_lock)
+
+        # Send getheaders message
+        [ c.send_getheaders() for c in self.p2p_connections ]
+
+        # Send ping and wait for response -- synchronization hack
+        [ c.send_ping(self.ping_counter) for c in self.p2p_connections ]
+        self.wait_for_pings(self.ping_counter)
+        self.ping_counter += 1
+
+    # Analogous to sync_block (see above)
+    def sync_transaction(self, txhash, num_events):
+        # Wait for nodes to request transaction (50ms sleep * 20 tries * num_events)
+        def transaction_requested():
+            return all(
+                txhash in node.tx_request_map and node.tx_request_map[txhash]
+                for node in self.p2p_connections
+            )
+
+        # --> error if not requested
+        wait_until(transaction_requested, attempts=20*num_events, lock=mininode_lock)
+
+        # Get the mempool
+        [ c.send_mempool() for c in self.p2p_connections ]
+
+        # Send ping and wait for response -- synchronization hack
+        [ c.send_ping(self.ping_counter) for c in self.p2p_connections ]
+        self.wait_for_pings(self.ping_counter)
+        self.ping_counter += 1
+
+        # Sort inv responses from each node
+        with mininode_lock:
+            [ c.lastInv.sort() for c in self.p2p_connections ]
+
+    # Verify that the tip of each connection all agree with each other, and
+    # with the expected outcome (if given)
+    def check_results(self, blockhash, outcome):
+        with mininode_lock:
+            for c in self.p2p_connections:
+                if outcome is None:
+                    if c.bestblockhash != self.p2p_connections[0].bestblockhash:
+                        return False
+                elif isinstance(outcome, RejectResult): # Check that block was rejected w/ code
+                    if c.bestblockhash == blockhash:
+                        return False
+                    if blockhash not in c.block_reject_map:
+                        logger.error('Block not in reject map: %064x' % (blockhash))
+                        return False
+                    if not outcome.match(c.block_reject_map[blockhash]):
+                        logger.error('Block rejected with %s instead of expected %s: %064x' % (c.block_reject_map[blockhash], outcome, blockhash))
+                        return False
+                elif ((c.bestblockhash == blockhash) != outcome):
+                    return False
+            return True
+
+    # Either check that the mempools all agree with each other, or that
+    # txhash's presence in the mempool matches the outcome specified.
+    # This is somewhat of a strange comparison, in that we're either comparing
+    # a particular tx to an outcome, or the entire mempools altogether;
+    # perhaps it would be useful to add the ability to check explicitly that
+    # a particular tx's existence in the mempool is the same across all nodes.
+    def check_mempool(self, txhash, outcome):
+        with mininode_lock:
+            for c in self.p2p_connections:
+                if outcome is None:
+                    # Make sure the mempools agree with each other
+                    if c.lastInv != self.p2p_connections[0].lastInv:
+                        return False
+                elif isinstance(outcome, RejectResult): # Check that tx was rejected w/ code
+                    if txhash in c.lastInv:
+                        return False
+                    if txhash not in c.tx_reject_map:
+                        logger.error('Tx not in reject map: %064x' % (txhash))
+                        return False
+                    if not outcome.match(c.tx_reject_map[txhash]):
+                        logger.error('Tx rejected with %s instead of expected %s: %064x' % (c.tx_reject_map[txhash], outcome, txhash))
+                        return False
+                elif ((txhash in c.lastInv) != outcome):
+                    return False
+            return True
+
+    def run(self):
+        # Wait until verack is received
+        self.wait_for_verack()
+
+        test_number = 0
+        tests = self.test_generator.get_tests()
+        for test_instance in tests:
+            test_number += 1
+            logger.info("Running test %d: %s line %s" % (test_number, tests.gi_code.co_filename, tests.gi_frame.f_lineno))
+            # We use these variables to keep track of the last block
+            # and last transaction in the tests, which are used
+            # if we're not syncing on every block or every tx.
+            [ block, block_outcome, tip ] = [ None, None, None ]
+            [ tx, tx_outcome ] = [ None, None ]
+            invqueue = []
+
+            for test_obj in test_instance.blocks_and_transactions:
+                b_or_t = test_obj[0]
+                outcome = test_obj[1]
+                # Determine if we're dealing with a block or tx
+                if isinstance(b_or_t, CBlock):  # Block test runner
+                    block = b_or_t
+                    block_outcome = outcome
+                    tip = block.sha256
+                    # each test_obj can have an optional third argument
+                    # to specify the tip we should compare with
+                    # (default is to use the block being tested)
+                    if len(test_obj) >= 3:
+                        tip = test_obj[2]
+
+                    # Add to shared block_store, set as current block
+                    # If there was an open getdata request for the block
+                    # previously, and we didn't have an entry in the
+                    # block_store, then immediately deliver, because the
+                    # node wouldn't send another getdata request while
+                    # the earlier one is outstanding.
+                    first_block_with_hash = True
+                    if self.block_store.get(block.sha256) is not None:
+                        first_block_with_hash = False
+                    with mininode_lock:
+                        self.block_store.add_block(block)
+                        for c in self.p2p_connections:
+                            if first_block_with_hash and block.sha256 in c.block_request_map and c.block_request_map[block.sha256] == True:
+                                # There was a previous request for this block hash
+                                # Most likely, we delivered a header for this block
+                                # but never had the block to respond to the getdata
+                                c.send_message(msg_block(block))
+                            else:
+                                c.block_request_map[block.sha256] = False
+                    # Either send inv's to each node and sync, or add
+                    # to invqueue for later inv'ing.
+                    if (test_instance.sync_every_block):
+                        # if we expect success, send inv and sync every block
+                        # if we expect failure, just push the block and see what happens.
+                        if outcome == True:
+                            [ c.send_inv(block) for c in self.p2p_connections ]
+                            self.sync_blocks(block.sha256, 1)
+                        else:
+                            [ c.send_message(msg_block(block)) for c in self.p2p_connections ]
+                            [ c.send_ping(self.ping_counter) for c in self.p2p_connections ]
+                            self.wait_for_pings(self.ping_counter)
+                            self.ping_counter += 1
+                        if (not self.check_results(tip, outcome)):
+                            raise AssertionError("Test failed at test %d" % test_number)
+                    else:
+                        invqueue.append(CInv(2, block.sha256))
+                elif isinstance(b_or_t, CBlockHeader):
+                    block_header = b_or_t
+                    self.block_store.add_header(block_header)
+                    [ c.send_header(block_header) for c in self.p2p_connections ]
+
+                else:  # Tx test runner
+                    assert(isinstance(b_or_t, CTransaction))
+                    tx = b_or_t
+                    tx_outcome = outcome
+                    # Add to shared tx store and clear map entry
+                    with mininode_lock:
+                        self.tx_store.add_transaction(tx)
+                        for c in self.p2p_connections:
+                            c.tx_request_map[tx.sha256] = False
+                    # Again, either inv to all nodes or save for later
+                    if (test_instance.sync_every_tx):
+                        [ c.send_inv(tx) for c in self.p2p_connections ]
+                        self.sync_transaction(tx.sha256, 1)
+                        if (not self.check_mempool(tx.sha256, outcome)):
+                            raise AssertionError("Test failed at test %d" % test_number)
+                    else:
+                        invqueue.append(CInv(1, tx.sha256))
+                # Ensure we're not overflowing the inv queue
+                if len(invqueue) == MAX_INV_SZ:
+                    [ c.send_message(msg_inv(invqueue)) for c in self.p2p_connections ]
+                    invqueue = []
+
+            # Do final sync if we weren't syncing on every block or every tx.
+            if (not test_instance.sync_every_block and block is not None):
+                if len(invqueue) > 0:
+                    [ c.send_message(msg_inv(invqueue)) for c in self.p2p_connections ]
+                    invqueue = []
+                self.sync_blocks(block.sha256, len(test_instance.blocks_and_transactions))
+                if (not self.check_results(tip, block_outcome)):
+                    raise AssertionError("Block test failed at test %d" % test_number)
+            if (not test_instance.sync_every_tx and tx is not None):
+                if len(invqueue) > 0:
+                    [ c.send_message(msg_inv(invqueue)) for c in self.p2p_connections ]
+                    invqueue = []
+                self.sync_transaction(tx.sha256, len(test_instance.blocks_and_transactions))
+                if (not self.check_mempool(tx.sha256, tx_outcome)):
+                    raise AssertionError("Mempool test failed at test %d" % test_number)
+
+        [ c.disconnect_node() for c in self.p2p_connections ]
+        self.wait_for_disconnections()
+        self.block_store.close()
+        self.tx_store.close()

test/functional/test_framework/coverage.py

@@ -0,0 +1,109 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Utilities for doing coverage analysis on the RPC interface.
+
+Provides a way to track which RPC commands are exercised during
+testing.
+"""
+
+import os
+
+
+REFERENCE_FILENAME = 'rpc_interface.txt'
+
+
+class AuthServiceProxyWrapper():
+    """
+    An object that wraps AuthServiceProxy to record specific RPC calls.
+
+    """
+    def __init__(self, auth_service_proxy_instance, coverage_logfile=None):
+        """
+        Kwargs:
+            auth_service_proxy_instance (AuthServiceProxy): the instance
+                being wrapped.
+            coverage_logfile (str): if specified, write each service_name
+                out to a file when called.
+
+        """
+        self.auth_service_proxy_instance = auth_service_proxy_instance
+        self.coverage_logfile = coverage_logfile
+
+    def __getattr__(self, name):
+        return_val = getattr(self.auth_service_proxy_instance, name)
+        if not isinstance(return_val, type(self.auth_service_proxy_instance)):
+            # If proxy getattr returned an unwrapped value, do the same here.
+            return return_val
+        return AuthServiceProxyWrapper(return_val, self.coverage_logfile)
+
+    def __call__(self, *args, **kwargs):
+        """
+        Delegates to AuthServiceProxy, then writes the particular RPC method
+        called to a file.
+
+        """
+        return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
+        self._log_call()
+        return return_val
+
+    def _log_call(self):
+        rpc_method = self.auth_service_proxy_instance._service_name
+
+        if self.coverage_logfile:
+            with open(self.coverage_logfile, 'a+', encoding='utf8') as f:
+                f.write("%s\n" % rpc_method)
+
+    def __truediv__(self, relative_uri):
+        return AuthServiceProxyWrapper(self.auth_service_proxy_instance / relative_uri,
+                                       self.coverage_logfile)
+
+    def get_request(self, *args, **kwargs):
+        self._log_call()
+        return self.auth_service_proxy_instance.get_request(*args)
+
+def get_filename(dirname, n_node):
+    """
+    Get a filename unique to the test process ID and node.
+
+    This file will contain a list of RPC commands covered.
+    """
+    pid = str(os.getpid())
+    return os.path.join(
+        dirname, "coverage.pid%s.node%s.txt" % (pid, str(n_node)))
+
+
+def write_all_rpc_commands(dirname, node):
+    """
+    Write out a list of all RPC functions available in `agrarian-cli` for
+    coverage comparison. This will only happen once per coverage
+    directory.
+
+    Args:
+        dirname (str): temporary test dir
+        node (AuthServiceProxy): client
+
+    Returns:
+        bool. if the RPC interface file was written.
+
+    """
+    filename = os.path.join(dirname, REFERENCE_FILENAME)
+
+    if os.path.isfile(filename):
+        return False
+
+    help_output = node.help().split('\n')
+    commands = set()
+
+    for line in help_output:
+        line = line.strip()
+
+        # Ignore blanks and headers
+        if line and not line.startswith('='):
+            commands.add("%s\n" % line.split()[0])
+
+    with open(filename, 'w', encoding='utf8') as f:
+        f.writelines(list(commands))
+
+    return True

test/functional/test_framework/key.py

@@ -0,0 +1,232 @@
+# Copyright (c) 2011 Sam Rushing
+"""ECC secp256k1 OpenSSL wrapper.
+
+WARNING: This module does not mlock() secrets; your private keys may end up on
+disk in swap! Use with caution!
+
+This file is modified from python-bitcoinlib.
+"""
+
+import ctypes
+import ctypes.util
+import hashlib
+import sys
+
+ssl = ctypes.cdll.LoadLibrary(ctypes.util.find_library ('ssl') or 'libeay32')
+
+ssl.BN_new.restype = ctypes.c_void_p
+ssl.BN_new.argtypes = []
+
+ssl.BN_bin2bn.restype = ctypes.c_void_p
+ssl.BN_bin2bn.argtypes = [ctypes.c_char_p, ctypes.c_int, ctypes.c_void_p]
+
+ssl.BN_CTX_free.restype = None
+ssl.BN_CTX_free.argtypes = [ctypes.c_void_p]
+
+ssl.BN_CTX_new.restype = ctypes.c_void_p
+ssl.BN_CTX_new.argtypes = []
+
+ssl.ECDH_compute_key.restype = ctypes.c_int
+ssl.ECDH_compute_key.argtypes = [ctypes.c_void_p, ctypes.c_int, ctypes.c_void_p, ctypes.c_void_p]
+
+ssl.ECDSA_sign.restype = ctypes.c_int
+ssl.ECDSA_sign.argtypes = [ctypes.c_int, ctypes.c_void_p, ctypes.c_int, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p]
+
+ssl.ECDSA_verify.restype = ctypes.c_int
+ssl.ECDSA_verify.argtypes = [ctypes.c_int, ctypes.c_void_p, ctypes.c_int, ctypes.c_void_p, ctypes.c_int, ctypes.c_void_p]
+
+ssl.EC_KEY_free.restype = None
+ssl.EC_KEY_free.argtypes = [ctypes.c_void_p]
+
+ssl.EC_KEY_new_by_curve_name.restype = ctypes.c_void_p
+ssl.EC_KEY_new_by_curve_name.argtypes = [ctypes.c_int]
+
+ssl.EC_KEY_get0_group.restype = ctypes.c_void_p
+ssl.EC_KEY_get0_group.argtypes = [ctypes.c_void_p]
+
+ssl.EC_KEY_get0_public_key.restype = ctypes.c_void_p
+ssl.EC_KEY_get0_public_key.argtypes = [ctypes.c_void_p]
+
+ssl.EC_KEY_set_private_key.restype = ctypes.c_int
+ssl.EC_KEY_set_private_key.argtypes = [ctypes.c_void_p, ctypes.c_void_p]
+
+ssl.EC_KEY_set_conv_form.restype = None
+ssl.EC_KEY_set_conv_form.argtypes = [ctypes.c_void_p, ctypes.c_int]
+
+ssl.EC_KEY_set_public_key.restype = ctypes.c_int
+ssl.EC_KEY_set_public_key.argtypes = [ctypes.c_void_p, ctypes.c_void_p]
+
+ssl.i2o_ECPublicKey.restype = ctypes.c_void_p
+ssl.i2o_ECPublicKey.argtypes = [ctypes.c_void_p, ctypes.c_void_p]
+
+ssl.EC_POINT_new.restype = ctypes.c_void_p
+ssl.EC_POINT_new.argtypes = [ctypes.c_void_p]
+
+ssl.EC_POINT_free.restype = None
+ssl.EC_POINT_free.argtypes = [ctypes.c_void_p]
+
+ssl.EC_POINT_mul.restype = ctypes.c_int
+ssl.EC_POINT_mul.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p]
+
+# this specifies the curve used with ECDSA.
+NID_secp256k1 = 714 # from openssl/obj_mac.h
+
+SECP256K1_ORDER = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141
+SECP256K1_ORDER_HALF = SECP256K1_ORDER // 2
+
+# Thx to Sam Devlin for the ctypes magic 64-bit fix.
+def _check_result(val, func, args):
+    if val == 0:
+        raise ValueError
+    else:
+        return ctypes.c_void_p (val)
+
+ssl.EC_KEY_new_by_curve_name.restype = ctypes.c_void_p
+ssl.EC_KEY_new_by_curve_name.errcheck = _check_result
+
+class CECKey():
+    """Wrapper around OpenSSL's EC_KEY"""
+
+    POINT_CONVERSION_COMPRESSED = 2
+    POINT_CONVERSION_UNCOMPRESSED = 4
+
+    def __init__(self):
+        self.k = ssl.EC_KEY_new_by_curve_name(NID_secp256k1)
+
+    def __del__(self):
+        if ssl:
+            ssl.EC_KEY_free(self.k)
+        self.k = None
+
+    def set_secretbytes(self, secret):
+        priv_key = ssl.BN_bin2bn(secret, 32, ssl.BN_new())
+        group = ssl.EC_KEY_get0_group(self.k)
+        pub_key = ssl.EC_POINT_new(group)
+        ctx = ssl.BN_CTX_new()
+        if not ssl.EC_POINT_mul(group, pub_key, priv_key, None, None, ctx):
+            raise ValueError("Could not derive public key from the supplied secret.")
+        ssl.EC_POINT_mul(group, pub_key, priv_key, None, None, ctx)
+        ssl.EC_KEY_set_private_key(self.k, priv_key)
+        ssl.EC_KEY_set_public_key(self.k, pub_key)
+        ssl.EC_POINT_free(pub_key)
+        ssl.BN_CTX_free(ctx)
+        return self.k
+
+    def set_privkey(self, key):
+        self.mb = ctypes.create_string_buffer(key)
+        return ssl.d2i_ECPrivateKey(ctypes.byref(self.k), ctypes.byref(ctypes.pointer(self.mb)), len(key))
+
+    def set_pubkey(self, key):
+        self.mb = ctypes.create_string_buffer(key)
+        return ssl.o2i_ECPublicKey(ctypes.byref(self.k), ctypes.byref(ctypes.pointer(self.mb)), len(key))
+
+    def get_privkey(self):
+        size = ssl.i2d_ECPrivateKey(self.k, 0)
+        mb_pri = ctypes.create_string_buffer(size)
+        ssl.i2d_ECPrivateKey(self.k, ctypes.byref(ctypes.pointer(mb_pri)))
+        return mb_pri.raw
+
+    def get_pubkey(self):
+        size = ssl.i2o_ECPublicKey(self.k, 0)
+        mb = ctypes.create_string_buffer(size)
+        ssl.i2o_ECPublicKey(self.k, ctypes.byref(ctypes.pointer(mb)))
+        return mb.raw
+
+    def get_raw_ecdh_key(self, other_pubkey):
+        ecdh_keybuffer = ctypes.create_string_buffer(32)
+        r = ssl.ECDH_compute_key(ctypes.pointer(ecdh_keybuffer), 32,
+                                 ssl.EC_KEY_get0_public_key(other_pubkey.k),
+                                 self.k, 0)
+        if r != 32:
+            raise Exception('CKey.get_ecdh_key(): ECDH_compute_key() failed')
+        return ecdh_keybuffer.raw
+
+    def get_ecdh_key(self, other_pubkey, kdf=lambda k: hashlib.sha256(k).digest()):
+        # FIXME: be warned it's not clear what the kdf should be as a default
+        r = self.get_raw_ecdh_key(other_pubkey)
+        return kdf(r)
+
+    def sign(self, hash, low_s = True):
+        # FIXME: need unit tests for below cases
+        if not isinstance(hash, bytes):
+            raise TypeError('Hash must be bytes instance; got %r' % hash.__class__)
+        if len(hash) != 32:
+            raise ValueError('Hash must be exactly 32 bytes long')
+
+        sig_size0 = ctypes.c_uint32()
+        sig_size0.value = ssl.ECDSA_size(self.k)
+        mb_sig = ctypes.create_string_buffer(sig_size0.value)
+        result = ssl.ECDSA_sign(0, hash, len(hash), mb_sig, ctypes.byref(sig_size0), self.k)
+        assert 1 == result
+        assert mb_sig.raw[0] == 0x30
+        assert mb_sig.raw[1] == sig_size0.value - 2
+        total_size = mb_sig.raw[1]
+        assert mb_sig.raw[2] == 2
+        r_size = mb_sig.raw[3]
+        assert mb_sig.raw[4 + r_size] == 2
+        s_size = mb_sig.raw[5 + r_size]
+        s_value = int.from_bytes(mb_sig.raw[6+r_size:6+r_size+s_size], byteorder='big')
+        if (not low_s) or s_value <= SECP256K1_ORDER_HALF:
+            return mb_sig.raw[:sig_size0.value]
+        else:
+            low_s_value = SECP256K1_ORDER - s_value
+            low_s_bytes = (low_s_value).to_bytes(33, byteorder='big')
+            while len(low_s_bytes) > 1 and low_s_bytes[0] == 0 and low_s_bytes[1] < 0x80:
+                low_s_bytes = low_s_bytes[1:]
+            new_s_size = len(low_s_bytes)
+            new_total_size_byte = (total_size + new_s_size - s_size).to_bytes(1,byteorder='big')
+            new_s_size_byte = (new_s_size).to_bytes(1,byteorder='big')
+            return b'\x30' + new_total_size_byte + mb_sig.raw[2:5+r_size] + new_s_size_byte + low_s_bytes
+
+    def verify(self, hash, sig):
+        """Verify a DER signature"""
+        return ssl.ECDSA_verify(0, hash, len(hash), sig, len(sig), self.k) == 1
+
+    def set_compressed(self, compressed):
+        if compressed:
+            form = self.POINT_CONVERSION_COMPRESSED
+        else:
+            form = self.POINT_CONVERSION_UNCOMPRESSED
+        ssl.EC_KEY_set_conv_form(self.k, form)
+
+
+class CPubKey(bytes):
+    """An encapsulated public key
+
+    Attributes:
+
+    is_valid      - Corresponds to CPubKey.IsValid()
+    is_fullyvalid - Corresponds to CPubKey.IsFullyValid()
+    is_compressed - Corresponds to CPubKey.IsCompressed()
+    """
+
+    def __new__(cls, buf, _cec_key=None):
+        self = super(CPubKey, cls).__new__(cls, buf)
+        if _cec_key is None:
+            _cec_key = CECKey()
+        self._cec_key = _cec_key
+        self.is_fullyvalid = _cec_key.set_pubkey(self) != 0
+        return self
+
+    @property
+    def is_valid(self):
+        return len(self) > 0
+
+    @property
+    def is_compressed(self):
+        return len(self) == 33
+
+    def verify(self, hash, sig):
+        return self._cec_key.verify(hash, sig)
+
+    def __str__(self):
+        return repr(self)
+
+    def __repr__(self):
+        # Always have represent as b'<secret>' so test cases don't have to
+        # change for py2/3
+        if sys.version > '3':
+            return '%s(%s)' % (self.__class__.__name__, super(CPubKey, self).__repr__())
+        else:
+            return '%s(b%s)' % (self.__class__.__name__, super(CPubKey, self).__repr__())
+

test/functional/test_framework/mininode.py

@@ -0,0 +1,444 @@
+#!/usr/bin/env python3
+# Copyright (c) 2010 ArtForz -- public domain half-a-node
+# Copyright (c) 2012 Jeff Garzik
+# Copyright (c) 2010-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Bitcoin P2P network half-a-node.
+
+This python code was modified from ArtForz' public domain  half-a-node, as
+found in the mini-node branch of http://github.com/jgarzik/pynode.
+
+P2PConnection: A low-level connection object to a node's P2P interface
+P2PInterface: A high-level interface object for communicating to a node over P2P"""
+import asyncore
+from collections import defaultdict
+from io import BytesIO
+import logging
+import socket
+import struct
+import sys
+import threading
+
+from test_framework.messages import *
+from test_framework.util import wait_until
+
+logger = logging.getLogger("TestFramework.mininode")
+
+MESSAGEMAP = {
+    b"addr": msg_addr,
+    b"block": msg_block,
+    b"blocktxn": msg_blocktxn,
+    b"cmpctblock": msg_cmpctblock,
+    b"feefilter": msg_feefilter,
+    b"getaddr": msg_getaddr,
+    b"getblocks": msg_getblocks,
+    b"getblocktxn": msg_getblocktxn,
+    b"getdata": msg_getdata,
+    b"getheaders": msg_getheaders,
+    b"headers": msg_headers,
+    b"inv": msg_inv,
+    b"mempool": msg_mempool,
+    b"ping": msg_ping,
+    b"pong": msg_pong,
+    b"reject": msg_reject,
+    b"sendcmpct": msg_sendcmpct,
+    b"sendheaders": msg_sendheaders,
+    b"tx": msg_tx,
+    b"verack": msg_verack,
+    b"version": msg_version,
+    #b"getsporks": msg_generic,
+}
+
+MAGIC_BYTES = {
+    "mainnet": b"\x90\xc4\xfd\xe9",   # mainnet
+    "testnet3": b"\x45\x76\x65\xba",  # testnet3
+    "regtest": b"\xa1\xcf\x7e\xac",   # regtest
+}
+
+class P2PConnection(asyncore.dispatcher):
+    """A low-level connection object to a node's P2P interface.
+
+    This class is responsible for:
+
+    - opening and closing the TCP connection to the node
+    - reading bytes from and writing bytes to the socket
+    - deserializing and serializing the P2P message header
+    - logging messages as they are sent and received
+
+    This class contains no logic for handing the P2P message payloads. It must be
+    sub-classed and the on_message() callback overridden."""
+
+    def __init__(self):
+        # All P2PConnections must be created before starting the NetworkThread.
+        # assert that the network thread is not running.
+        assert not network_thread_running()
+
+        super().__init__(map=mininode_socket_map)
+
+    def peer_connect(self, dstaddr, dstport, net="regtest"):
+        self.dstaddr = dstaddr
+        self.dstport = dstport
+        self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
+        self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+        self.sendbuf = b""
+        self.recvbuf = b""
+        self.state = "connecting"
+        self.network = net
+        self.disconnect = False
+
+        logger.info('Connecting to Bitcoin Node: %s:%d' % (self.dstaddr, self.dstport))
+
+        try:
+            self.connect((dstaddr, dstport))
+        except:
+            self.handle_close()
+
+    def peer_disconnect(self):
+        # Connection could have already been closed by other end.
+        if self.state == "connected":
+            self.disconnect_node()
+
+    # Connection and disconnection methods
+
+    def handle_connect(self):
+        """asyncore callback when a connection is opened."""
+        if self.state != "connected":
+            logger.debug("Connected & Listening: %s:%d" % (self.dstaddr, self.dstport))
+            self.state = "connected"
+            self.on_open()
+
+    def handle_close(self):
+        """asyncore callback when a connection is closed."""
+        logger.debug("Closing connection to: %s:%d" % (self.dstaddr, self.dstport))
+        self.state = "closed"
+        self.recvbuf = b""
+        self.sendbuf = b""
+        try:
+            self.close()
+        except:
+            pass
+        self.on_close()
+
+    def disconnect_node(self):
+        """Disconnect the p2p connection.
+
+        Called by the test logic thread. Causes the p2p connection
+        to be disconnected on the next iteration of the asyncore loop."""
+        self.disconnect = True
+
+    # Socket read methods
+
+    def handle_read(self):
+        """asyncore callback when data is read from the socket."""
+        t = self.recv(8192)
+        if len(t) > 0:
+            self.recvbuf += t
+            self._on_data()
+
+    def _on_data(self):
+        """Try to read P2P messages from the recv buffer.
+
+        This method reads data from the buffer in a loop. It deserializes,
+        parses and verifies the P2P header, then passes the P2P payload to
+        the on_message callback for processing."""
+        try:
+            while True:
+                if len(self.recvbuf) < 4:
+                    return
+                if self.recvbuf[:4] != MAGIC_BYTES[self.network]:
+                    raise ValueError("got garbage %s" % repr(self.recvbuf))
+                if len(self.recvbuf) < 4 + 12 + 4 + 4:
+                    return
+                command = self.recvbuf[4:4+12].split(b"\x00", 1)[0]
+                msglen = struct.unpack("<i", self.recvbuf[4+12:4+12+4])[0]
+                checksum = self.recvbuf[4+12+4:4+12+4+4]
+                if len(self.recvbuf) < 4 + 12 + 4 + 4 + msglen:
+                    return
+                msg = self.recvbuf[4+12+4+4:4+12+4+4+msglen]
+                th = sha256(msg)
+                h = sha256(th)
+                if checksum != h[:4]:
+                    raise ValueError("got bad checksum " + repr(self.recvbuf))
+                self.recvbuf = self.recvbuf[4+12+4+4+msglen:]
+                if command in MESSAGEMAP:
+                    #raise ValueError("Received unknown command from %s:%d: '%s' %s" % (self.dstaddr, self.dstport, command, repr(msg)))
+                    logger.debug("Command: '" + str(command) + "'")
+                    f = BytesIO(msg)
+                    t = MESSAGEMAP[command]()
+                    t.deserialize(f)
+                    self._log_message("receive", t)
+                    self.on_message(t)
+        except Exception as e:
+            logger.exception('Error reading message:', repr(e))
+            raise
+
+    def on_message(self, message):
+        """Callback for processing a P2P payload. Must be overridden by derived class."""
+        raise NotImplementedError
+
+    # Socket write methods
+
+    def writable(self):
+        """asyncore method to determine whether the handle_write() callback should be called on the next loop."""
+        with mininode_lock:
+            pre_connection = self.state == "connecting"
+            length = len(self.sendbuf)
+        return (length > 0 or pre_connection)
+
+    def handle_write(self):
+        """asyncore callback when data should be written to the socket."""
+        with mininode_lock:
+            # asyncore does not expose socket connection, only the first read/write
+            # event, thus we must check connection manually here to know when we
+            # actually connect
+            if self.state == "connecting":
+                self.handle_connect()
+            if not self.writable():
+                return
+
+            try:
+                sent = self.send(self.sendbuf)
+            except:
+                self.handle_close()
+                return
+            self.sendbuf = self.sendbuf[sent:]
+
+    def send_message(self, message, pushbuf=False):
+        """Send a P2P message over the socket.
+
+        This method takes a P2P payload, builds the P2P header and adds
+        the message to the send buffer to be sent over the socket."""
+        if self.state != "connected" and not pushbuf:
+            raise IOError('Not connected, no pushbuf')
+        self._log_message("send", message)
+        command = message.command
+        data = message.serialize()
+        tmsg = MAGIC_BYTES[self.network]
+        tmsg += command
+        tmsg += b"\x00" * (12 - len(command))
+        tmsg += struct.pack("<I", len(data))
+        th = sha256(data)
+        h = sha256(th)
+        tmsg += h[:4]
+        tmsg += data
+        with mininode_lock:
+            if (len(self.sendbuf) == 0 and not pushbuf):
+                try:
+                    sent = self.send(tmsg)
+                    self.sendbuf = tmsg[sent:]
+                except BlockingIOError:
+                    self.sendbuf = tmsg
+            else:
+                self.sendbuf += tmsg
+
+    # Class utility methods
+
+    def _log_message(self, direction, msg):
+        """Logs a message being sent or received over the connection."""
+        if direction == "send":
+            log_message = "Send message to "
+        elif direction == "receive":
+            log_message = "Received message from "
+        log_message += "%s:%d: %s" % (self.dstaddr, self.dstport, repr(msg)[:500])
+        if len(log_message) > 500:
+            log_message += "... (msg truncated)"
+        logger.debug(log_message)
+
+
+class P2PInterface(P2PConnection):
+    """A high-level P2P interface class for communicating with a Bitcoin node.
+
+    This class provides high-level callbacks for processing P2P message
+    payloads, as well as convenience methods for interacting with the
+    node over P2P.
+
+    Individual testcases should subclass this and override the on_* methods
+    if they want to alter message handling behaviour."""
+    def __init__(self):
+        super().__init__()
+
+        # Track number of messages of each type received and the most recent
+        # message of each type
+        self.message_count = defaultdict(int)
+        self.last_message = {}
+
+        # A count of the number of ping messages we've sent to the node
+        self.ping_counter = 1
+
+        # The network services received from the peer
+        self.nServices = 0
+
+    def peer_connect(self, *args, services=NODE_NETWORK, send_version=True, **kwargs):
+        super().peer_connect(*args, **kwargs)
+
+        if send_version:
+            # Send a version msg
+            vt = msg_version()
+            vt.nServices = services
+            vt.addrTo.ip = self.dstaddr
+            vt.addrTo.port = self.dstport
+            vt.addrFrom.ip = "0.0.0.0"
+            vt.addrFrom.port = 0
+            self.send_message(vt, True)
+
+    # Message receiving methods
+
+    def on_message(self, message):
+        """Receive message and dispatch message to appropriate callback.
+
+        We keep a count of how many of each message type has been received
+        and the most recent message of each type."""
+        with mininode_lock:
+            try:
+                command = message.command.decode('ascii')
+                self.message_count[command] += 1
+                self.last_message[command] = message
+                getattr(self, 'on_' + command)(message)
+            except:
+                print("ERROR delivering %s (%s)" % (repr(message), sys.exc_info()[0]))
+                raise
+
+    # Callback methods. Can be overridden by subclasses in individual test
+    # cases to provide custom message handling behaviour.
+
+    def on_open(self):
+        pass
+
+    def on_close(self):
+        pass
+
+    def on_addr(self, message): pass
+    def on_block(self, message): pass
+    def on_blocktxn(self, message): pass
+    def on_cmpctblock(self, message): pass
+    def on_feefilter(self, message): pass
+    def on_getaddr(self, message): pass
+    def on_getblocks(self, message): pass
+    def on_getblocktxn(self, message): pass
+    def on_getdata(self, message): pass
+    def on_getheaders(self, message): pass
+    def on_headers(self, message): pass
+    def on_mempool(self, message): pass
+    def on_pong(self, message): pass
+    def on_reject(self, message): pass
+    def on_sendcmpct(self, message): pass
+    def on_sendheaders(self, message): pass
+    def on_tx(self, message): pass
+
+    def on_inv(self, message):
+        want = msg_getdata()
+        for i in message.inv:
+            if i.type != 0:
+                want.inv.append(i)
+        if len(want.inv):
+            self.send_message(want)
+
+    def on_ping(self, message):
+        self.send_message(msg_pong(message.nonce))
+
+    def on_verack(self, message):
+        self.verack_received = True
+
+    def on_version(self, message):
+        assert message.nVersion >= MIN_VERSION_SUPPORTED, "Version {} received. Test framework only supports versions greater than {}".format(message.nVersion, MIN_VERSION_SUPPORTED)
+        self.send_message(msg_verack())
+        self.nServices = message.nServices
+
+    # Connection helper methods
+
+    def wait_for_disconnect(self, timeout=60):
+        test_function = lambda: self.state != "connected"
+        wait_until(test_function, timeout=timeout, lock=mininode_lock)
+
+    # Message receiving helper methods
+
+    def wait_for_block(self, blockhash, timeout=60):
+        test_function = lambda: self.last_message.get("block") and self.last_message["block"].block.rehash() == blockhash
+        wait_until(test_function, timeout=timeout, lock=mininode_lock)
+
+    def wait_for_getdata(self, timeout=60):
+        test_function = lambda: self.last_message.get("getdata")
+        wait_until(test_function, timeout=timeout, lock=mininode_lock)
+
+    def wait_for_getheaders(self, timeout=60):
+        test_function = lambda: self.last_message.get("getheaders")
+        wait_until(test_function, timeout=timeout, lock=mininode_lock)
+
+    def wait_for_inv(self, expected_inv, timeout=60):
+        """Waits for an INV message and checks that the first inv object in the message was as expected."""
+        if len(expected_inv) > 1:
+            raise NotImplementedError("wait_for_inv() will only verify the first inv object")
+        test_function = lambda: self.last_message.get("inv") and \
+                                self.last_message["inv"].inv[0].type == expected_inv[0].type and \
+                                self.last_message["inv"].inv[0].hash == expected_inv[0].hash
+        wait_until(test_function, timeout=timeout, lock=mininode_lock)
+
+    def wait_for_verack(self, timeout=60):
+        test_function = lambda: self.message_count["verack"]
+        wait_until(test_function, timeout=timeout, lock=mininode_lock)
+
+    # Message sending helper functions
+
+    def send_and_ping(self, message):
+        self.send_message(message)
+        self.sync_with_ping()
+
+    # Sync up with the node
+    def sync_with_ping(self, timeout=60):
+        self.send_message(msg_ping(nonce=self.ping_counter))
+        test_function = lambda: self.last_message.get("pong") and self.last_message["pong"].nonce == self.ping_counter
+        wait_until(test_function, timeout=timeout, lock=mininode_lock)
+        self.ping_counter += 1
+
+
+# Keep our own socket map for asyncore, so that we can track disconnects
+# ourselves (to workaround an issue with closing an asyncore socket when
+# using select)
+mininode_socket_map = dict()
+
+# One lock for synchronizing all data access between the networking thread (see
+# NetworkThread below) and the thread running the test logic.  For simplicity,
+# P2PConnection acquires this lock whenever delivering a message to a P2PInterface,
+# and whenever adding anything to the send buffer (in send_message()).  This
+# lock should be acquired in the thread running the test logic to synchronize
+# access to any data shared with the P2PInterface or P2PConnection.
+mininode_lock = threading.RLock()
+
+class NetworkThread(threading.Thread):
+    def __init__(self):
+        super().__init__(name="NetworkThread")
+
+    def run(self):
+        while mininode_socket_map:
+            # We check for whether to disconnect outside of the asyncore
+            # loop to workaround the behavior of asyncore when using
+            # select
+            disconnected = []
+            for fd, obj in mininode_socket_map.items():
+                if obj.disconnect:
+                    disconnected.append(obj)
+            [obj.handle_close() for obj in disconnected]
+            asyncore.loop(0.1, use_poll=True, map=mininode_socket_map, count=1)
+        logger.debug("Network thread closing")
+
+def network_thread_start():
+    """Start the network thread."""
+    # Only one network thread may run at a time
+    assert not network_thread_running()
+
+    NetworkThread().start()
+
+def network_thread_running():
+    """Return whether the network thread is running."""
+    return any([thread.name == "NetworkThread" for thread in threading.enumerate()])
+
+def network_thread_join(timeout=10):
+    """Wait timeout seconds for the network thread to terminate.
+
+    Throw if the network thread doesn't terminate in timeout seconds."""
+    network_threads = [thread for thread in threading.enumerate() if thread.name == "NetworkThread"]
+    assert len(network_threads) <= 1
+    for thread in network_threads:
+        thread.join(timeout)
+        assert not thread.is_alive()

test/functional/test_framework/netutil.py

@@ -0,0 +1,156 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Linux network utilities.
+
+Roughly based on http://voorloopnul.com/blog/a-python-netstat-in-less-than-100-lines-of-code/ by Ricardo Pascal
+"""
+
+import sys
+import socket
+import fcntl
+import struct
+import array
+import os
+from binascii import unhexlify, hexlify
+
+# STATE_ESTABLISHED = '01'
+# STATE_SYN_SENT  = '02'
+# STATE_SYN_RECV = '03'
+# STATE_FIN_WAIT1 = '04'
+# STATE_FIN_WAIT2 = '05'
+# STATE_TIME_WAIT = '06'
+# STATE_CLOSE = '07'
+# STATE_CLOSE_WAIT = '08'
+# STATE_LAST_ACK = '09'
+STATE_LISTEN = '0A'
+# STATE_CLOSING = '0B'
+
+def get_socket_inodes(pid):
+    '''
+    Get list of socket inodes for process pid.
+    '''
+    base = '/proc/%i/fd' % pid
+    inodes = []
+    for item in os.listdir(base):
+        target = os.readlink(os.path.join(base, item))
+        if target.startswith('socket:'):
+            inodes.append(int(target[8:-1]))
+    return inodes
+
+def _remove_empty(array):
+    return [x for x in array if x !='']
+
+def _convert_ip_port(array):
+    host,port = array.split(':')
+    # convert host from mangled-per-four-bytes form as used by kernel
+    host = unhexlify(host)
+    host_out = ''
+    for x in range(0, len(host) // 4):
+        (val,) = struct.unpack('=I', host[x*4:(x+1)*4])
+        host_out += '%08x' % val
+
+    return host_out,int(port,16)
+
+def netstat(typ='tcp'):
+    '''
+    Function to return a list with status of tcp connections at linux systems
+    To get pid of all network process running on system, you must run this script
+    as superuser
+    '''
+    with open('/proc/net/'+typ,'r',encoding='utf8') as f:
+        content = f.readlines()
+        content.pop(0)
+    result = []
+    for line in content:
+        line_array = _remove_empty(line.split(' '))     # Split lines and remove empty spaces.
+        tcp_id = line_array[0]
+        l_addr = _convert_ip_port(line_array[1])
+        r_addr = _convert_ip_port(line_array[2])
+        state = line_array[3]
+        inode = int(line_array[9])                      # Need the inode to match with process pid.
+        nline = [tcp_id, l_addr, r_addr, state, inode]
+        result.append(nline)
+    return result
+
+def get_bind_addrs(pid):
+    '''
+    Get bind addresses as (host,port) tuples for process pid.
+    '''
+    inodes = get_socket_inodes(pid)
+    bind_addrs = []
+    for conn in netstat('tcp') + netstat('tcp6'):
+        if conn[3] == STATE_LISTEN and conn[4] in inodes:
+            bind_addrs.append(conn[1])
+    return bind_addrs
+
+# from: http://code.activestate.com/recipes/439093/
+def all_interfaces():
+    '''
+    Return all interfaces that are up
+    '''
+    is_64bits = sys.maxsize > 2**32
+    struct_size = 40 if is_64bits else 32
+    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+    max_possible = 8 # initial value
+    while True:
+        bytes = max_possible * struct_size
+        names = array.array('B', b'\0' * bytes)
+        outbytes = struct.unpack('iL', fcntl.ioctl(
+            s.fileno(),
+            0x8912,  # SIOCGIFCONF
+            struct.pack('iL', bytes, names.buffer_info()[0])
+        ))[0]
+        if outbytes == bytes:
+            max_possible *= 2
+        else:
+            break
+    namestr = names.tostring()
+    return [(namestr[i:i+16].split(b'\0', 1)[0],
+             socket.inet_ntoa(namestr[i+20:i+24]))
+            for i in range(0, outbytes, struct_size)]
+
+def addr_to_hex(addr):
+    '''
+    Convert string IPv4 or IPv6 address to binary address as returned by
+    get_bind_addrs.
+    Very naive implementation that certainly doesn't work for all IPv6 variants.
+    '''
+    if '.' in addr: # IPv4
+        addr = [int(x) for x in addr.split('.')]
+    elif ':' in addr: # IPv6
+        sub = [[], []] # prefix, suffix
+        x = 0
+        addr = addr.split(':')
+        for i,comp in enumerate(addr):
+            if comp == '':
+                if i == 0 or i == (len(addr)-1): # skip empty component at beginning or end
+                    continue
+                x += 1 # :: skips to suffix
+                assert(x < 2)
+            else: # two bytes per component
+                val = int(comp, 16)
+                sub[x].append(val >> 8)
+                sub[x].append(val & 0xff)
+        nullbytes = 16 - len(sub[0]) - len(sub[1])
+        assert((x == 0 and nullbytes == 0) or (x == 1 and nullbytes > 0))
+        addr = sub[0] + ([0] * nullbytes) + sub[1]
+    else:
+        raise ValueError('Could not parse address %s' % addr)
+    return hexlify(bytearray(addr)).decode('ascii')
+
+def test_ipv6_local():
+    '''
+    Check for (local) IPv6 support.
+    '''
+    import socket
+    # By using SOCK_DGRAM this will not actually make a connection, but it will
+    # fail if there is no route to IPv6 localhost.
+    have_ipv6 = True
+    try:
+        s = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
+        s.connect(('::1', 0))
+    except socket.error:
+        have_ipv6 = False
+    return have_ipv6

test/functional/test_framework/script.py

@@ -0,0 +1,647 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Functionality to build scripts, as well as SignatureHash().
+
+This file is modified from python-bitcoinlib.
+"""
+
+from .mininode import CTransaction, CTxOut, sha256, hash256, uint256_from_str, ser_uint256, ser_string
+from binascii import hexlify
+import hashlib
+
+import sys
+bchr = chr
+bord = ord
+if sys.version > '3':
+    long = int
+    bchr = lambda x: bytes([x])
+    bord = lambda x: x
+
+import struct
+
+from .bignum import bn2vch
+
+MAX_SCRIPT_ELEMENT_SIZE = 520
+
+OPCODE_NAMES = {}
+
+def hash160(s):
+    return hashlib.new('ripemd160', sha256(s)).digest()
+
+
+_opcode_instances = []
+class CScriptOp(int):
+    """A single script opcode"""
+    __slots__ = []
+
+    @staticmethod
+    def encode_op_pushdata(d):
+        """Encode a PUSHDATA op, returning bytes"""
+        if len(d) < 0x4c:
+            return b'' + bchr(len(d)) + d # OP_PUSHDATA
+        elif len(d) <= 0xff:
+            return b'\x4c' + bchr(len(d)) + d # OP_PUSHDATA1
+        elif len(d) <= 0xffff:
+            return b'\x4d' + struct.pack(b'<H', len(d)) + d # OP_PUSHDATA2
+        elif len(d) <= 0xffffffff:
+            return b'\x4e' + struct.pack(b'<I', len(d)) + d # OP_PUSHDATA4
+        else:
+            raise ValueError("Data too long to encode in a PUSHDATA op")
+
+    @staticmethod
+    def encode_op_n(n):
+        """Encode a small integer op, returning an opcode"""
+        if not (0 <= n <= 16):
+            raise ValueError('Integer must be in range 0 <= n <= 16, got %d' % n)
+
+        if n == 0:
+            return OP_0
+        else:
+            return CScriptOp(OP_1 + n-1)
+
+    def decode_op_n(self):
+        """Decode a small integer opcode, returning an integer"""
+        if self == OP_0:
+            return 0
+
+        if not (self == OP_0 or OP_1 <= self <= OP_16):
+            raise ValueError('op %r is not an OP_N' % self)
+
+        return int(self - OP_1+1)
+
+    def is_small_int(self):
+        """Return true if the op pushes a small integer to the stack"""
+        if 0x51 <= self <= 0x60 or self == 0:
+            return True
+        else:
+            return False
+
+    def __str__(self):
+        return repr(self)
+
+    def __repr__(self):
+        if self in OPCODE_NAMES:
+            return OPCODE_NAMES[self]
+        else:
+            return 'CScriptOp(0x%x)' % self
+
+    def __new__(cls, n):
+        try:
+            return _opcode_instances[n]
+        except IndexError:
+            assert len(_opcode_instances) == n
+            _opcode_instances.append(super(CScriptOp, cls).__new__(cls, n))
+            return _opcode_instances[n]
+
+# Populate opcode instance table
+for n in range(0xff+1):
+    CScriptOp(n)
+
+
+# push value
+OP_0 = CScriptOp(0x00)
+OP_FALSE = OP_0
+OP_PUSHDATA1 = CScriptOp(0x4c)
+OP_PUSHDATA2 = CScriptOp(0x4d)
+OP_PUSHDATA4 = CScriptOp(0x4e)
+OP_1NEGATE = CScriptOp(0x4f)
+OP_RESERVED = CScriptOp(0x50)
+OP_1 = CScriptOp(0x51)
+OP_TRUE=OP_1
+OP_2 = CScriptOp(0x52)
+OP_3 = CScriptOp(0x53)
+OP_4 = CScriptOp(0x54)
+OP_5 = CScriptOp(0x55)
+OP_6 = CScriptOp(0x56)
+OP_7 = CScriptOp(0x57)
+OP_8 = CScriptOp(0x58)
+OP_9 = CScriptOp(0x59)
+OP_10 = CScriptOp(0x5a)
+OP_11 = CScriptOp(0x5b)
+OP_12 = CScriptOp(0x5c)
+OP_13 = CScriptOp(0x5d)
+OP_14 = CScriptOp(0x5e)
+OP_15 = CScriptOp(0x5f)
+OP_16 = CScriptOp(0x60)
+
+# control
+OP_NOP = CScriptOp(0x61)
+OP_VER = CScriptOp(0x62)
+OP_IF = CScriptOp(0x63)
+OP_NOTIF = CScriptOp(0x64)
+OP_VERIF = CScriptOp(0x65)
+OP_VERNOTIF = CScriptOp(0x66)
+OP_ELSE = CScriptOp(0x67)
+OP_ENDIF = CScriptOp(0x68)
+OP_VERIFY = CScriptOp(0x69)
+OP_RETURN = CScriptOp(0x6a)
+
+# stack ops
+OP_TOALTSTACK = CScriptOp(0x6b)
+OP_FROMALTSTACK = CScriptOp(0x6c)
+OP_2DROP = CScriptOp(0x6d)
+OP_2DUP = CScriptOp(0x6e)
+OP_3DUP = CScriptOp(0x6f)
+OP_2OVER = CScriptOp(0x70)
+OP_2ROT = CScriptOp(0x71)
+OP_2SWAP = CScriptOp(0x72)
+OP_IFDUP = CScriptOp(0x73)
+OP_DEPTH = CScriptOp(0x74)
+OP_DROP = CScriptOp(0x75)
+OP_DUP = CScriptOp(0x76)
+OP_NIP = CScriptOp(0x77)
+OP_OVER = CScriptOp(0x78)
+OP_PICK = CScriptOp(0x79)
+OP_ROLL = CScriptOp(0x7a)
+OP_ROT = CScriptOp(0x7b)
+OP_SWAP = CScriptOp(0x7c)
+OP_TUCK = CScriptOp(0x7d)
+
+# splice ops
+OP_CAT = CScriptOp(0x7e)
+OP_SUBSTR = CScriptOp(0x7f)
+OP_LEFT = CScriptOp(0x80)
+OP_RIGHT = CScriptOp(0x81)
+OP_SIZE = CScriptOp(0x82)
+
+# bit logic
+OP_INVERT = CScriptOp(0x83)
+OP_AND = CScriptOp(0x84)
+OP_OR = CScriptOp(0x85)
+OP_XOR = CScriptOp(0x86)
+OP_EQUAL = CScriptOp(0x87)
+OP_EQUALVERIFY = CScriptOp(0x88)
+OP_RESERVED1 = CScriptOp(0x89)
+OP_RESERVED2 = CScriptOp(0x8a)
+
+# numeric
+OP_1ADD = CScriptOp(0x8b)
+OP_1SUB = CScriptOp(0x8c)
+OP_2MUL = CScriptOp(0x8d)
+OP_2DIV = CScriptOp(0x8e)
+OP_NEGATE = CScriptOp(0x8f)
+OP_ABS = CScriptOp(0x90)
+OP_NOT = CScriptOp(0x91)
+OP_0NOTEQUAL = CScriptOp(0x92)
+
+OP_ADD = CScriptOp(0x93)
+OP_SUB = CScriptOp(0x94)
+OP_MUL = CScriptOp(0x95)
+OP_DIV = CScriptOp(0x96)
+OP_MOD = CScriptOp(0x97)
+OP_LSHIFT = CScriptOp(0x98)
+OP_RSHIFT = CScriptOp(0x99)
+
+OP_BOOLAND = CScriptOp(0x9a)
+OP_BOOLOR = CScriptOp(0x9b)
+OP_NUMEQUAL = CScriptOp(0x9c)
+OP_NUMEQUALVERIFY = CScriptOp(0x9d)
+OP_NUMNOTEQUAL = CScriptOp(0x9e)
+OP_LESSTHAN = CScriptOp(0x9f)
+OP_GREATERTHAN = CScriptOp(0xa0)
+OP_LESSTHANOREQUAL = CScriptOp(0xa1)
+OP_GREATERTHANOREQUAL = CScriptOp(0xa2)
+OP_MIN = CScriptOp(0xa3)
+OP_MAX = CScriptOp(0xa4)
+
+OP_WITHIN = CScriptOp(0xa5)
+
+# crypto
+OP_RIPEMD160 = CScriptOp(0xa6)
+OP_SHA1 = CScriptOp(0xa7)
+OP_SHA256 = CScriptOp(0xa8)
+OP_HASH160 = CScriptOp(0xa9)
+OP_HASH256 = CScriptOp(0xaa)
+OP_CODESEPARATOR = CScriptOp(0xab)
+OP_CHECKSIG = CScriptOp(0xac)
+OP_CHECKSIGVERIFY = CScriptOp(0xad)
+OP_CHECKMULTISIG = CScriptOp(0xae)
+OP_CHECKMULTISIGVERIFY = CScriptOp(0xaf)
+
+# expansion
+OP_NOP1 = CScriptOp(0xb0)
+OP_CHECKLOCKTIMEVERIFY = CScriptOp(0xb1)
+OP_CHECKSEQUENCEVERIFY = CScriptOp(0xb2)
+OP_NOP4 = CScriptOp(0xb3)
+OP_NOP5 = CScriptOp(0xb4)
+OP_NOP6 = CScriptOp(0xb5)
+OP_NOP7 = CScriptOp(0xb6)
+OP_NOP8 = CScriptOp(0xb7)
+OP_NOP9 = CScriptOp(0xb8)
+OP_NOP10 = CScriptOp(0xb9)
+
+# template matching params
+OP_SMALLINTEGER = CScriptOp(0xfa)
+OP_PUBKEYS = CScriptOp(0xfb)
+OP_PUBKEYHASH = CScriptOp(0xfd)
+OP_PUBKEY = CScriptOp(0xfe)
+
+OP_INVALIDOPCODE = CScriptOp(0xff)
+
+OPCODE_NAMES.update({
+    OP_0 : 'OP_0',
+    OP_PUSHDATA1 : 'OP_PUSHDATA1',
+    OP_PUSHDATA2 : 'OP_PUSHDATA2',
+    OP_PUSHDATA4 : 'OP_PUSHDATA4',
+    OP_1NEGATE : 'OP_1NEGATE',
+    OP_RESERVED : 'OP_RESERVED',
+    OP_1 : 'OP_1',
+    OP_2 : 'OP_2',
+    OP_3 : 'OP_3',
+    OP_4 : 'OP_4',
+    OP_5 : 'OP_5',
+    OP_6 : 'OP_6',
+    OP_7 : 'OP_7',
+    OP_8 : 'OP_8',
+    OP_9 : 'OP_9',
+    OP_10 : 'OP_10',
+    OP_11 : 'OP_11',
+    OP_12 : 'OP_12',
+    OP_13 : 'OP_13',
+    OP_14 : 'OP_14',
+    OP_15 : 'OP_15',
+    OP_16 : 'OP_16',
+    OP_NOP : 'OP_NOP',
+    OP_VER : 'OP_VER',
+    OP_IF : 'OP_IF',
+    OP_NOTIF : 'OP_NOTIF',
+    OP_VERIF : 'OP_VERIF',
+    OP_VERNOTIF : 'OP_VERNOTIF',
+    OP_ELSE : 'OP_ELSE',
+    OP_ENDIF : 'OP_ENDIF',
+    OP_VERIFY : 'OP_VERIFY',
+    OP_RETURN : 'OP_RETURN',
+    OP_TOALTSTACK : 'OP_TOALTSTACK',
+    OP_FROMALTSTACK : 'OP_FROMALTSTACK',
+    OP_2DROP : 'OP_2DROP',
+    OP_2DUP : 'OP_2DUP',
+    OP_3DUP : 'OP_3DUP',
+    OP_2OVER : 'OP_2OVER',
+    OP_2ROT : 'OP_2ROT',
+    OP_2SWAP : 'OP_2SWAP',
+    OP_IFDUP : 'OP_IFDUP',
+    OP_DEPTH : 'OP_DEPTH',
+    OP_DROP : 'OP_DROP',
+    OP_DUP : 'OP_DUP',
+    OP_NIP : 'OP_NIP',
+    OP_OVER : 'OP_OVER',
+    OP_PICK : 'OP_PICK',
+    OP_ROLL : 'OP_ROLL',
+    OP_ROT : 'OP_ROT',
+    OP_SWAP : 'OP_SWAP',
+    OP_TUCK : 'OP_TUCK',
+    OP_CAT : 'OP_CAT',
+    OP_SUBSTR : 'OP_SUBSTR',
+    OP_LEFT : 'OP_LEFT',
+    OP_RIGHT : 'OP_RIGHT',
+    OP_SIZE : 'OP_SIZE',
+    OP_INVERT : 'OP_INVERT',
+    OP_AND : 'OP_AND',
+    OP_OR : 'OP_OR',
+    OP_XOR : 'OP_XOR',
+    OP_EQUAL : 'OP_EQUAL',
+    OP_EQUALVERIFY : 'OP_EQUALVERIFY',
+    OP_RESERVED1 : 'OP_RESERVED1',
+    OP_RESERVED2 : 'OP_RESERVED2',
+    OP_1ADD : 'OP_1ADD',
+    OP_1SUB : 'OP_1SUB',
+    OP_2MUL : 'OP_2MUL',
+    OP_2DIV : 'OP_2DIV',
+    OP_NEGATE : 'OP_NEGATE',
+    OP_ABS : 'OP_ABS',
+    OP_NOT : 'OP_NOT',
+    OP_0NOTEQUAL : 'OP_0NOTEQUAL',
+    OP_ADD : 'OP_ADD',
+    OP_SUB : 'OP_SUB',
+    OP_MUL : 'OP_MUL',
+    OP_DIV : 'OP_DIV',
+    OP_MOD : 'OP_MOD',
+    OP_LSHIFT : 'OP_LSHIFT',
+    OP_RSHIFT : 'OP_RSHIFT',
+    OP_BOOLAND : 'OP_BOOLAND',
+    OP_BOOLOR : 'OP_BOOLOR',
+    OP_NUMEQUAL : 'OP_NUMEQUAL',
+    OP_NUMEQUALVERIFY : 'OP_NUMEQUALVERIFY',
+    OP_NUMNOTEQUAL : 'OP_NUMNOTEQUAL',
+    OP_LESSTHAN : 'OP_LESSTHAN',
+    OP_GREATERTHAN : 'OP_GREATERTHAN',
+    OP_LESSTHANOREQUAL : 'OP_LESSTHANOREQUAL',
+    OP_GREATERTHANOREQUAL : 'OP_GREATERTHANOREQUAL',
+    OP_MIN : 'OP_MIN',
+    OP_MAX : 'OP_MAX',
+    OP_WITHIN : 'OP_WITHIN',
+    OP_RIPEMD160 : 'OP_RIPEMD160',
+    OP_SHA1 : 'OP_SHA1',
+    OP_SHA256 : 'OP_SHA256',
+    OP_HASH160 : 'OP_HASH160',
+    OP_HASH256 : 'OP_HASH256',
+    OP_CODESEPARATOR : 'OP_CODESEPARATOR',
+    OP_CHECKSIG : 'OP_CHECKSIG',
+    OP_CHECKSIGVERIFY : 'OP_CHECKSIGVERIFY',
+    OP_CHECKMULTISIG : 'OP_CHECKMULTISIG',
+    OP_CHECKMULTISIGVERIFY : 'OP_CHECKMULTISIGVERIFY',
+    OP_NOP1 : 'OP_NOP1',
+    OP_CHECKLOCKTIMEVERIFY : 'OP_CHECKLOCKTIMEVERIFY',
+    OP_CHECKSEQUENCEVERIFY : 'OP_CHECKSEQUENCEVERIFY',
+    OP_NOP4 : 'OP_NOP4',
+    OP_NOP5 : 'OP_NOP5',
+    OP_NOP6 : 'OP_NOP6',
+    OP_NOP7 : 'OP_NOP7',
+    OP_NOP8 : 'OP_NOP8',
+    OP_NOP9 : 'OP_NOP9',
+    OP_NOP10 : 'OP_NOP10',
+    OP_SMALLINTEGER : 'OP_SMALLINTEGER',
+    OP_PUBKEYS : 'OP_PUBKEYS',
+    OP_PUBKEYHASH : 'OP_PUBKEYHASH',
+    OP_PUBKEY : 'OP_PUBKEY',
+    OP_INVALIDOPCODE : 'OP_INVALIDOPCODE',
+})
+
+class CScriptInvalidError(Exception):
+    """Base class for CScript exceptions"""
+    pass
+
+class CScriptTruncatedPushDataError(CScriptInvalidError):
+    """Invalid pushdata due to truncation"""
+    def __init__(self, msg, data):
+        self.data = data
+        super(CScriptTruncatedPushDataError, self).__init__(msg)
+
+# This is used, eg, for blockchain heights in coinbase scripts (bip34)
+class CScriptNum():
+    def __init__(self, d=0):
+        self.value = d
+
+    @staticmethod
+    def encode(obj):
+        r = bytearray(0)
+        if obj.value == 0:
+            return bytes(r)
+        neg = obj.value < 0
+        absvalue = -obj.value if neg else obj.value
+        while (absvalue):
+            r.append(absvalue & 0xff)
+            absvalue >>= 8
+        if r[-1] & 0x80:
+            r.append(0x80 if neg else 0)
+        elif neg:
+            r[-1] |= 0x80
+        return bytes(bchr(len(r)) + r)
+
+
+class CScript(bytes):
+    """Serialized script
+
+    A bytes subclass, so you can use this directly whenever bytes are accepted.
+    Note that this means that indexing does *not* work - you'll get an index by
+    byte rather than opcode. This format was chosen for efficiency so that the
+    general case would not require creating a lot of little CScriptOP objects.
+
+    iter(script) however does iterate by opcode.
+    """
+    @classmethod
+    def __coerce_instance(cls, other):
+        # Coerce other into bytes
+        if isinstance(other, CScriptOp):
+            other = bchr(other)
+        elif isinstance(other, CScriptNum):
+            if (other.value == 0):
+                other = bchr(CScriptOp(OP_0))
+            else:
+                other = CScriptNum.encode(other)
+        elif isinstance(other, int):
+            if 0 <= other <= 16:
+                other = bytes(bchr(CScriptOp.encode_op_n(other)))
+            elif other == -1:
+                other = bytes(bchr(OP_1NEGATE))
+            else:
+                other = CScriptOp.encode_op_pushdata(bn2vch(other))
+        elif isinstance(other, (bytes, bytearray)):
+            other = CScriptOp.encode_op_pushdata(other)
+        return other
+
+    def __add__(self, other):
+        # Do the coercion outside of the try block so that errors in it are
+        # noticed.
+        other = self.__coerce_instance(other)
+
+        try:
+            # bytes.__add__ always returns bytes instances unfortunately
+            return CScript(super(CScript, self).__add__(other))
+        except TypeError:
+            raise TypeError('Can not add a %r instance to a CScript' % other.__class__)
+
+    def join(self, iterable):
+        # join makes no sense for a CScript()
+        raise NotImplementedError
+
+    def __new__(cls, value=b''):
+        if isinstance(value, bytes) or isinstance(value, bytearray):
+            return super(CScript, cls).__new__(cls, value)
+        else:
+            def coerce_iterable(iterable):
+                for instance in iterable:
+                    yield cls.__coerce_instance(instance)
+            # Annoyingly on both python2 and python3 bytes.join() always
+            # returns a bytes instance even when subclassed.
+            return super(CScript, cls).__new__(cls, b''.join(coerce_iterable(value)))
+
+    def raw_iter(self):
+        """Raw iteration
+
+        Yields tuples of (opcode, data, sop_idx) so that the different possible
+        PUSHDATA encodings can be accurately distinguished, as well as
+        determining the exact opcode byte indexes. (sop_idx)
+        """
+        i = 0
+        while i < len(self):
+            sop_idx = i
+            opcode = bord(self[i])
+            i += 1
+
+            if opcode > OP_PUSHDATA4:
+                yield (opcode, None, sop_idx)
+            else:
+                datasize = None
+                pushdata_type = None
+                if opcode < OP_PUSHDATA1:
+                    pushdata_type = 'PUSHDATA(%d)' % opcode
+                    datasize = opcode
+
+                elif opcode == OP_PUSHDATA1:
+                    pushdata_type = 'PUSHDATA1'
+                    if i >= len(self):
+                        raise CScriptInvalidError('PUSHDATA1: missing data length')
+                    datasize = bord(self[i])
+                    i += 1
+
+                elif opcode == OP_PUSHDATA2:
+                    pushdata_type = 'PUSHDATA2'
+                    if i + 1 >= len(self):
+                        raise CScriptInvalidError('PUSHDATA2: missing data length')
+                    datasize = bord(self[i]) + (bord(self[i+1]) << 8)
+                    i += 2
+
+                elif opcode == OP_PUSHDATA4:
+                    pushdata_type = 'PUSHDATA4'
+                    if i + 3 >= len(self):
+                        raise CScriptInvalidError('PUSHDATA4: missing data length')
+                    datasize = bord(self[i]) + (bord(self[i+1]) << 8) + (bord(self[i+2]) << 16) + (bord(self[i+3]) << 24)
+                    i += 4
+
+                else:
+                    assert False # shouldn't happen
+
+
+                data = bytes(self[i:i+datasize])
+
+                # Check for truncation
+                if len(data) < datasize:
+                    raise CScriptTruncatedPushDataError('%s: truncated data' % pushdata_type, data)
+
+                i += datasize
+
+                yield (opcode, data, sop_idx)
+
+    def __iter__(self):
+        """'Cooked' iteration
+
+        Returns either a CScriptOP instance, an integer, or bytes, as
+        appropriate.
+
+        See raw_iter() if you need to distinguish the different possible
+        PUSHDATA encodings.
+        """
+        for (opcode, data, sop_idx) in self.raw_iter():
+            if data is not None:
+                yield data
+            else:
+                opcode = CScriptOp(opcode)
+
+                if opcode.is_small_int():
+                    yield opcode.decode_op_n()
+                else:
+                    yield CScriptOp(opcode)
+
+    def __repr__(self):
+        def _repr(o):
+            if isinstance(o, bytes):
+                return "x('%s')" % hexlify(o).decode('ascii')
+            else:
+                return repr(o)
+
+        ops = []
+        i = iter(self)
+        while True:
+            op = None
+            try:
+                op = _repr(next(i))
+            except CScriptTruncatedPushDataError as err:
+                op = '%s...<ERROR: %s>' % (_repr(err.data), err)
+                break
+            except CScriptInvalidError as err:
+                op = '<ERROR: %s>' % err
+                break
+            except StopIteration:
+                break
+            finally:
+                if op is not None:
+                    ops.append(op)
+
+        return "CScript([%s])" % ', '.join(ops)
+
+    def GetSigOpCount(self, fAccurate):
+        """Get the SigOp count.
+
+        fAccurate - Accurately count CHECKMULTISIG, see BIP16 for details.
+
+        Note that this is consensus-critical.
+        """
+        n = 0
+        lastOpcode = OP_INVALIDOPCODE
+        for (opcode, data, sop_idx) in self.raw_iter():
+            if opcode in (OP_CHECKSIG, OP_CHECKSIGVERIFY):
+                n += 1
+            elif opcode in (OP_CHECKMULTISIG, OP_CHECKMULTISIGVERIFY):
+                if fAccurate and (OP_1 <= lastOpcode <= OP_16):
+                    n += opcode.decode_op_n()
+                else:
+                    n += 20
+            lastOpcode = opcode
+        return n
+
+
+SIGHASH_ALL = 1
+SIGHASH_NONE = 2
+SIGHASH_SINGLE = 3
+SIGHASH_ANYONECANPAY = 0x80
+
+def FindAndDelete(script, sig):
+    """Consensus critical, see FindAndDelete() in Satoshi codebase"""
+    r = b''
+    last_sop_idx = sop_idx = 0
+    skip = True
+    for (opcode, data, sop_idx) in script.raw_iter():
+        if not skip:
+            r += script[last_sop_idx:sop_idx]
+        last_sop_idx = sop_idx
+        if script[sop_idx:sop_idx + len(sig)] == sig:
+            skip = True
+        else:
+            skip = False
+    if not skip:
+        r += script[last_sop_idx:]
+    return CScript(r)
+
+
+def SignatureHash(script, txTo, inIdx, hashtype):
+    """Consensus-correct SignatureHash
+
+    Returns (hash, err) to precisely match the consensus-critical behavior of
+    the SIGHASH_SINGLE bug. (inIdx is *not* checked for validity)
+    """
+    HASH_ONE = b'\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
+
+    if inIdx >= len(txTo.vin):
+        return (HASH_ONE, "inIdx %d out of range (%d)" % (inIdx, len(txTo.vin)))
+    txtmp = CTransaction(txTo)
+
+    for txin in txtmp.vin:
+        txin.scriptSig = b''
+    txtmp.vin[inIdx].scriptSig = FindAndDelete(script, CScript([OP_CODESEPARATOR]))
+
+    if (hashtype & 0x1f) == SIGHASH_NONE:
+        txtmp.vout = []
+
+        for i in range(len(txtmp.vin)):
+            if i != inIdx:
+                txtmp.vin[i].nSequence = 0
+
+    elif (hashtype & 0x1f) == SIGHASH_SINGLE:
+        outIdx = inIdx
+        if outIdx >= len(txtmp.vout):
+            return (HASH_ONE, "outIdx %d out of range (%d)" % (outIdx, len(txtmp.vout)))
+
+        tmp = txtmp.vout[outIdx]
+        txtmp.vout = []
+        for i in range(outIdx):
+            txtmp.vout.append(CTxOut(-1))
+        txtmp.vout.append(tmp)
+
+        for i in range(len(txtmp.vin)):
+            if i != inIdx:
+                txtmp.vin[i].nSequence = 0
+
+    if hashtype & SIGHASH_ANYONECANPAY:
+        tmp = txtmp.vin[inIdx]
+        txtmp.vin = []
+        txtmp.vin.append(tmp)
+
+    s = txtmp.serialize_without_witness()
+    s += struct.pack(b"<I", hashtype)
+
+    hash = hash256(s)
+
+    return (hash, None)

test/functional/test_framework/siphash.py

@@ -0,0 +1,63 @@
+#!/usr/bin/env python3
+# Copyright (c) 2016-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Specialized SipHash-2-4 implementations.
+
+This implements SipHash-2-4 for 256-bit integers.
+"""
+
+def rotl64(n, b):
+    return n >> (64 - b) | (n & ((1 << (64 - b)) - 1)) << b
+
+def siphash_round(v0, v1, v2, v3):
+    v0 = (v0 + v1) & ((1 << 64) - 1)
+    v1 = rotl64(v1, 13)
+    v1 ^= v0
+    v0 = rotl64(v0, 32)
+    v2 = (v2 + v3) & ((1 << 64) - 1)
+    v3 = rotl64(v3, 16)
+    v3 ^= v2
+    v0 = (v0 + v3) & ((1 << 64) - 1)
+    v3 = rotl64(v3, 21)
+    v3 ^= v0
+    v2 = (v2 + v1) & ((1 << 64) - 1)
+    v1 = rotl64(v1, 17)
+    v1 ^= v2
+    v2 = rotl64(v2, 32)
+    return (v0, v1, v2, v3)
+
+def siphash256(k0, k1, h):
+    n0 = h & ((1 << 64) - 1)
+    n1 = (h >> 64) & ((1 << 64) - 1)
+    n2 = (h >> 128) & ((1 << 64) - 1)
+    n3 = (h >> 192) & ((1 << 64) - 1)
+    v0 = 0x736f6d6570736575 ^ k0
+    v1 = 0x646f72616e646f6d ^ k1
+    v2 = 0x6c7967656e657261 ^ k0
+    v3 = 0x7465646279746573 ^ k1 ^ n0
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    v0 ^= n0
+    v3 ^= n1
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    v0 ^= n1
+    v3 ^= n2
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    v0 ^= n2
+    v3 ^= n3
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    v0 ^= n3
+    v3 ^= 0x2000000000000000
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    v0 ^= 0x2000000000000000
+    v2 ^= 0xFF
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    v0, v1, v2, v3 = siphash_round(v0, v1, v2, v3)
+    return v0 ^ v1 ^ v2 ^ v3

test/functional/test_framework/socks5.py

@@ -0,0 +1,159 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Dummy Socks5 server for testing."""
+
+import socket, threading, queue
+import logging
+
+logger = logging.getLogger("TestFramework.socks5")
+
+### Protocol constants
+class Command:
+    CONNECT = 0x01
+
+class AddressType:
+    IPV4 = 0x01
+    DOMAINNAME = 0x03
+    IPV6 = 0x04
+
+### Utility functions
+def recvall(s, n):
+    """Receive n bytes from a socket, or fail."""
+    rv = bytearray()
+    while n > 0:
+        d = s.recv(n)
+        if not d:
+            raise IOError('Unexpected end of stream')
+        rv.extend(d)
+        n -= len(d)
+    return rv
+
+### Implementation classes
+class Socks5Configuration():
+    """Proxy configuration."""
+    def __init__(self):
+        self.addr = None # Bind address (must be set)
+        self.af = socket.AF_INET # Bind address family
+        self.unauth = False  # Support unauthenticated
+        self.auth = False  # Support authentication
+
+class Socks5Command():
+    """Information about an incoming socks5 command."""
+    def __init__(self, cmd, atyp, addr, port, username, password):
+        self.cmd = cmd # Command (one of Command.*)
+        self.atyp = atyp # Address type (one of AddressType.*)
+        self.addr = addr # Address
+        self.port = port # Port to connect to
+        self.username = username
+        self.password = password
+    def __repr__(self):
+        return 'Socks5Command(%s,%s,%s,%s,%s,%s)' % (self.cmd, self.atyp, self.addr, self.port, self.username, self.password)
+
+class Socks5Connection():
+    def __init__(self, serv, conn, peer):
+        self.serv = serv
+        self.conn = conn
+        self.peer = peer
+
+    def handle(self):
+        """Handle socks5 request according to RFC192."""
+        try:
+            # Verify socks version
+            ver = recvall(self.conn, 1)[0]
+            if ver != 0x05:
+                raise IOError('Invalid socks version %i' % ver)
+            # Choose authentication method
+            nmethods = recvall(self.conn, 1)[0]
+            methods = bytearray(recvall(self.conn, nmethods))
+            method = None
+            if 0x02 in methods and self.serv.conf.auth:
+                method = 0x02 # username/password
+            elif 0x00 in methods and self.serv.conf.unauth:
+                method = 0x00 # unauthenticated
+            if method is None:
+                raise IOError('No supported authentication method was offered')
+            # Send response
+            self.conn.sendall(bytearray([0x05, method]))
+            # Read authentication (optional)
+            username = None
+            password = None
+            if method == 0x02:
+                ver = recvall(self.conn, 1)[0]
+                if ver != 0x01:
+                    raise IOError('Invalid auth packet version %i' % ver)
+                ulen = recvall(self.conn, 1)[0]
+                username = str(recvall(self.conn, ulen))
+                plen = recvall(self.conn, 1)[0]
+                password = str(recvall(self.conn, plen))
+                # Send authentication response
+                self.conn.sendall(bytearray([0x01, 0x00]))
+
+            # Read connect request
+            ver, cmd, _, atyp = recvall(self.conn, 4)
+            if ver != 0x05:
+                raise IOError('Invalid socks version %i in connect request' % ver)
+            if cmd != Command.CONNECT:
+                raise IOError('Unhandled command %i in connect request' % cmd)
+
+            if atyp == AddressType.IPV4:
+                addr = recvall(self.conn, 4)
+            elif atyp == AddressType.DOMAINNAME:
+                n = recvall(self.conn, 1)[0]
+                addr = recvall(self.conn, n)
+            elif atyp == AddressType.IPV6:
+                addr = recvall(self.conn, 16)
+            else:
+                raise IOError('Unknown address type %i' % atyp)
+            port_hi,port_lo = recvall(self.conn, 2)
+            port = (port_hi << 8) | port_lo
+
+            # Send dummy response
+            self.conn.sendall(bytearray([0x05, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]))
+
+            cmdin = Socks5Command(cmd, atyp, addr, port, username, password)
+            self.serv.queue.put(cmdin)
+            logger.info('Proxy: %s', cmdin)
+            # Fall through to disconnect
+        except Exception as e:
+            logger.exception("socks5 request handling failed.")
+            self.serv.queue.put(e)
+        finally:
+            self.conn.close()
+
+class Socks5Server():
+    def __init__(self, conf):
+        self.conf = conf
+        self.s = socket.socket(conf.af)
+        self.s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        self.s.bind(conf.addr)
+        self.s.listen(5)
+        self.running = False
+        self.thread = None
+        self.queue = queue.Queue() # report connections and exceptions to client
+
+    def run(self):
+        while self.running:
+            (sockconn, peer) = self.s.accept()
+            if self.running:
+                conn = Socks5Connection(self, sockconn, peer)
+                thread = threading.Thread(None, conn.handle)
+                thread.daemon = True
+                thread.start()
+
+    def start(self):
+        assert(not self.running)
+        self.running = True
+        self.thread = threading.Thread(None, self.run)
+        self.thread.daemon = True
+        self.thread.start()
+
+    def stop(self):
+        self.running = False
+        # connect to self to end run loop
+        s = socket.socket(self.conf.af)
+        s.connect(self.conf.addr)
+        s.close()
+        self.thread.join()
+

test/functional/test_framework/test_framework.py

@@ -0,0 +1,486 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Base class for RPC testing."""
+
+from enum import Enum
+import logging
+import optparse
+import os
+import pdb
+import shutil
+import sys
+import tempfile
+import time
+
+from .authproxy import JSONRPCException
+from . import coverage
+from .test_node import TestNode
+from .util import (
+    MAX_NODES,
+    PortSeed,
+    assert_equal,
+    check_json_precision,
+    connect_nodes_bi,
+    disconnect_nodes,
+    get_datadir_path,
+    initialize_datadir,
+    p2p_port,
+    set_node_times,
+    sync_blocks,
+    sync_mempools,
+)
+
+class TestStatus(Enum):
+    PASSED = 1
+    FAILED = 2
+    SKIPPED = 3
+
+TEST_EXIT_PASSED = 0
+TEST_EXIT_FAILED = 1
+TEST_EXIT_SKIPPED = 77
+
+class BitcoinTestFramework():
+    """Base class for a agrarian test script.
+
+    Individual agrarian test scripts should subclass this class and override the set_test_params() and run_test() methods.
+
+    Individual tests can also override the following methods to customize the test setup:
+
+    - add_options()
+    - setup_chain()
+    - setup_network()
+    - setup_nodes()
+
+    The __init__() and main() methods should not be overridden.
+
+    This class also contains various public and private helper methods."""
+
+    def __init__(self):
+        """Sets test framework defaults. Do not override this method. Instead, override the set_test_params() method"""
+        self.setup_clean_chain = False
+        self.nodes = []
+        self.mocktime = 0
+        self.supports_cli = False
+        self.set_test_params()
+
+        assert hasattr(self, "num_nodes"), "Test must set self.num_nodes in set_test_params()"
+
+    def main(self):
+        """Main function. This should not be overridden by the subclass test scripts."""
+
+        parser = optparse.OptionParser(usage="%prog [options]")
+        parser.add_option("--nocleanup", dest="nocleanup", default=False, action="store_true",
+                          help="Leave agrariands and test.* datadir on exit or error")
+        parser.add_option("--noshutdown", dest="noshutdown", default=False, action="store_true",
+                          help="Don't stop agrariands after the test execution")
+        parser.add_option("--srcdir", dest="srcdir", default=os.path.normpath(os.path.dirname(os.path.realpath(__file__))+"/../../../src"),
+                          help="Source directory containing agrariand/agrarian-cli (default: %default)")
+        parser.add_option("--cachedir", dest="cachedir", default=os.path.normpath(os.path.dirname(os.path.realpath(__file__)) + "/../../cache"),
+                          help="Directory for caching pregenerated datadirs")
+        parser.add_option("--tmpdir", dest="tmpdir", help="Root directory for datadirs")
+        parser.add_option("-l", "--loglevel", dest="loglevel", default="INFO",
+                          help="log events at this level and higher to the console. Can be set to DEBUG, INFO, WARNING, ERROR or CRITICAL. Passing --loglevel DEBUG will output all logs to console. Note that logs at all levels are always written to the test_framework.log file in the temporary test directory.")
+        parser.add_option("--tracerpc", dest="trace_rpc", default=False, action="store_true",
+                          help="Print out all RPC calls as they are made")
+        parser.add_option("--portseed", dest="port_seed", default=os.getpid(), type='int',
+                          help="The seed to use for assigning port numbers (default: current process id)")
+        parser.add_option("--coveragedir", dest="coveragedir",
+                          help="Write tested RPC commands into this directory")
+        parser.add_option("--configfile", dest="configfile",
+                          help="Location of the test framework config file")
+        parser.add_option("--pdbonfailure", dest="pdbonfailure", default=False, action="store_true",
+                          help="Attach a python debugger if test fails")
+        parser.add_option("--usecli", dest="usecli", default=False, action="store_true",
+                          help="use bitcoin-cli instead of RPC for all commands")
+        self.add_options(parser)
+        (self.options, self.args) = parser.parse_args()
+
+        PortSeed.n = self.options.port_seed
+
+        os.environ['PATH'] = self.options.srcdir + ":" + self.options.srcdir + "/qt:" + os.environ['PATH']
+
+        check_json_precision()
+
+        self.options.cachedir = os.path.abspath(self.options.cachedir)
+
+        # Set up temp directory and start logging
+        if self.options.tmpdir:
+            self.options.tmpdir = os.path.abspath(self.options.tmpdir)
+            os.makedirs(self.options.tmpdir, exist_ok=False)
+        else:
+            self.options.tmpdir = tempfile.mkdtemp(prefix="test")
+        self._start_logging()
+
+        success = TestStatus.FAILED
+
+        try:
+            if self.options.usecli and not self.supports_cli:
+                raise SkipTest("--usecli specified but test does not support using CLI")
+            self.setup_chain()
+            self.setup_network()
+            time.sleep(5)
+            self.run_test()
+            success = TestStatus.PASSED
+        except JSONRPCException as e:
+            self.log.exception("JSONRPC error")
+        except SkipTest as e:
+            self.log.warning("Test Skipped: %s" % e.message)
+            success = TestStatus.SKIPPED
+        except AssertionError as e:
+            self.log.exception("Assertion failed")
+        except KeyError as e:
+            self.log.exception("Key error")
+        except Exception as e:
+            self.log.exception("Unexpected exception caught during testing")
+        except KeyboardInterrupt as e:
+            self.log.warning("Exiting after keyboard interrupt")
+
+        if success == TestStatus.FAILED and self.options.pdbonfailure:
+            print("Testcase failed. Attaching python debugger. Enter ? for help")
+            pdb.set_trace()
+
+        if not self.options.noshutdown:
+            self.log.info("Stopping nodes")
+            if self.nodes:
+                self.stop_nodes()
+        else:
+            for node in self.nodes:
+                node.cleanup_on_exit = False
+            self.log.info("Note: agrariands were not stopped and may still be running")
+
+        if not self.options.nocleanup and not self.options.noshutdown and success != TestStatus.FAILED:
+            self.log.info("Cleaning up")
+            shutil.rmtree(self.options.tmpdir)
+        else:
+            self.log.warning("Not cleaning up dir %s" % self.options.tmpdir)
+
+        if success == TestStatus.PASSED:
+            self.log.info("Tests successful")
+            exit_code = TEST_EXIT_PASSED
+        elif success == TestStatus.SKIPPED:
+            self.log.info("Test skipped")
+            exit_code = TEST_EXIT_SKIPPED
+        else:
+            self.log.error("Test failed. Test logging available at %s/test_framework.log", self.options.tmpdir)
+            self.log.error("Hint: Call {} '{}' to consolidate all logs".format(os.path.normpath(os.path.dirname(os.path.realpath(__file__)) + "/../combine_logs.py"), self.options.tmpdir))
+            exit_code = TEST_EXIT_FAILED
+        logging.shutdown()
+        sys.exit(exit_code)
+
+    # Methods to override in subclass test scripts.
+    def set_test_params(self):
+        """Tests must this method to change default values for number of nodes, topology, etc"""
+        raise NotImplementedError
+
+    def add_options(self, parser):
+        """Override this method to add command-line options to the test"""
+        pass
+
+    def setup_chain(self):
+        """Override this method to customize blockchain setup"""
+        self.log.info("Initializing test directory " + self.options.tmpdir)
+        if self.setup_clean_chain:
+            self._initialize_chain_clean()
+        else:
+            self._initialize_chain()
+
+    def setup_network(self):
+        """Override this method to customize test network topology"""
+        self.setup_nodes()
+
+        # Connect the nodes as a "chain".  This allows us
+        # to split the network between nodes 1 and 2 to get
+        # two halves that can work on competing chains.
+        for i in range(self.num_nodes - 1):
+            connect_nodes_bi(self.nodes, i, i + 1)
+        self.sync_all()
+
+    def setup_nodes(self):
+        """Override this method to customize test node setup"""
+        extra_args = None
+        if hasattr(self, "extra_args"):
+            extra_args = self.extra_args
+        self.add_nodes(self.num_nodes, extra_args)
+        self.start_nodes()
+
+    def run_test(self):
+        """Tests must override this method to define test logic"""
+        raise NotImplementedError
+
+    # Public helper methods. These can be accessed by the subclass test scripts.
+
+    def add_nodes(self, num_nodes, extra_args=None, rpchost=None, timewait=None, binary=None):
+        """Instantiate TestNode objects"""
+
+        if extra_args is None:
+            extra_args = [[]] * num_nodes
+        if binary is None:
+            binary = [None] * num_nodes
+        assert_equal(len(extra_args), num_nodes)
+        assert_equal(len(binary), num_nodes)
+        for i in range(num_nodes):
+            self.nodes.append(TestNode(i, self.options.tmpdir, extra_args[i], rpchost, timewait=timewait, binary=binary[i], stderr=None, mocktime=self.mocktime, coverage_dir=self.options.coveragedir, use_cli=self.options.usecli))
+
+    def start_node(self, i, *args, **kwargs):
+        """Start a agrariand"""
+
+        node = self.nodes[i]
+
+        node.start(*args, **kwargs)
+        node.wait_for_rpc_connection()
+
+        time.sleep(10)
+
+        if self.options.coveragedir is not None:
+            coverage.write_all_rpc_commands(self.options.coveragedir, node.rpc)
+
+    def start_nodes(self, extra_args=None, *args, **kwargs):
+        """Start multiple agrariands"""
+
+        if extra_args is None:
+            extra_args = [None] * self.num_nodes
+        assert_equal(len(extra_args), self.num_nodes)
+        try:
+            for i, node in enumerate(self.nodes):
+                node.start(extra_args[i], *args, **kwargs)
+            for node in self.nodes:
+                node.wait_for_rpc_connection()
+        except:
+            # If one node failed to start, stop the others
+            self.stop_nodes()
+            raise
+
+        time.sleep(10)
+
+        if self.options.coveragedir is not None:
+            for node in self.nodes:
+                coverage.write_all_rpc_commands(self.options.coveragedir, node.rpc)
+
+    def stop_node(self, i):
+        """Stop a agrariand test node"""
+        self.nodes[i].stop_node()
+        self.nodes[i].wait_until_stopped()
+
+    def stop_nodes(self):
+        """Stop multiple agrariand test nodes"""
+        for node in self.nodes:
+            # Issue RPC to stop nodes
+            node.stop_node()
+
+        for node in self.nodes:
+            # Wait for nodes to stop
+            time.sleep(5)
+            node.wait_until_stopped()
+
+    def restart_node(self, i, extra_args=None):
+        """Stop and start a test node"""
+        self.stop_node(i)
+        self.start_node(i, extra_args)
+
+    def assert_start_raises_init_error(self, i, extra_args=None, expected_msg=None, *args, **kwargs):
+        with tempfile.SpooledTemporaryFile(max_size=2**16) as log_stderr:
+            try:
+                self.start_node(i, extra_args, stderr=log_stderr, *args, **kwargs)
+                self.stop_node(i)
+            except Exception as e:
+                assert 'agrariand exited' in str(e)  # node must have shutdown
+                self.nodes[i].running = False
+                self.nodes[i].process = None
+                if expected_msg is not None:
+                    log_stderr.seek(0)
+                    stderr = log_stderr.read().decode('utf-8')
+                    if expected_msg not in stderr:
+                        raise AssertionError("Expected error \"" + expected_msg + "\" not found in:\n" + stderr)
+            else:
+                if expected_msg is None:
+                    assert_msg = "agrariand should have exited with an error"
+                else:
+                    assert_msg = "agrariand should have exited with expected error " + expected_msg
+                raise AssertionError(assert_msg)
+
+    def wait_for_node_exit(self, i, timeout):
+        self.nodes[i].process.wait(timeout)
+
+    def split_network(self):
+        """
+        Split the network of four nodes into nodes 0/1 and 2/3.
+        """
+        disconnect_nodes(self.nodes[1], 2)
+        disconnect_nodes(self.nodes[2], 1)
+        self.sync_all([self.nodes[:2], self.nodes[2:]])
+
+    def join_network(self):
+        """
+        Join the (previously split) network halves together.
+        """
+        connect_nodes_bi(self.nodes, 1, 2)
+        self.sync_all()
+
+    def sync_all(self, node_groups=None):
+        if not node_groups:
+            node_groups = [self.nodes]
+
+        for group in node_groups:
+            sync_blocks(group)
+            sync_mempools(group)
+
+    def enable_mocktime(self):
+        """Enable mocktime for the script.
+
+        mocktime may be needed for scripts that use the cached version of the
+        blockchain.  If the cached version of the blockchain is used without
+        mocktime then the mempools will not sync due to IBD.
+
+        For backwared compatibility of the python scripts with previous
+        versions of the cache, this helper function sets mocktime to Jan 1,
+        2014 + (201 * 10 * 60)"""
+        self.mocktime = 1454124732 + (201 * 10 * 60)
+
+    def disable_mocktime(self):
+        self.mocktime = 0
+
+    # Private helper methods. These should not be accessed by the subclass test scripts.
+
+    def _start_logging(self):
+        # Add logger and logging handlers
+        self.log = logging.getLogger('TestFramework')
+        self.log.setLevel(logging.DEBUG)
+        # Create file handler to log all messages
+        fh = logging.FileHandler(self.options.tmpdir + '/test_framework.log')
+        fh.setLevel(logging.DEBUG)
+        # Create console handler to log messages to stderr. By default this logs only error messages, but can be configured with --loglevel.
+        ch = logging.StreamHandler(sys.stdout)
+        # User can provide log level as a number or string (eg DEBUG). loglevel was caught as a string, so try to convert it to an int
+        ll = int(self.options.loglevel) if self.options.loglevel.isdigit() else self.options.loglevel.upper()
+        ch.setLevel(ll)
+        # Format logs the same as agrariand's debug.log with microprecision (so log files can be concatenated and sorted)
+        formatter = logging.Formatter(fmt='%(asctime)s.%(msecs)03d000 %(name)s (%(levelname)s): %(message)s', datefmt='%Y-%m-%d %H:%M:%S')
+        formatter.converter = time.gmtime
+        fh.setFormatter(formatter)
+        ch.setFormatter(formatter)
+        # add the handlers to the logger
+        self.log.addHandler(fh)
+        self.log.addHandler(ch)
+
+        if self.options.trace_rpc:
+            rpc_logger = logging.getLogger("BitcoinRPC")
+            rpc_logger.setLevel(logging.DEBUG)
+            rpc_handler = logging.StreamHandler(sys.stdout)
+            rpc_handler.setLevel(logging.DEBUG)
+            rpc_logger.addHandler(rpc_handler)
+
+    def _initialize_chain(self):
+        """Initialize a pre-mined blockchain for use by the test.
+
+        Create a cache of a 200-block-long chain (with wallet) for MAX_NODES
+        Afterward, create num_nodes copies from the cache."""
+
+        assert self.num_nodes <= MAX_NODES
+        create_cache = False
+        for i in range(MAX_NODES):
+            if not os.path.isdir(get_datadir_path(self.options.cachedir, i)):
+                create_cache = True
+                break
+
+        if create_cache:
+            self.log.debug("Creating data directories from cached datadir")
+
+            # find and delete old cache directories if any exist
+            for i in range(MAX_NODES):
+                if os.path.isdir(get_datadir_path(self.options.cachedir, i)):
+                    shutil.rmtree(get_datadir_path(self.options.cachedir, i))
+
+            # Create cache directories, run bitcoinds:
+            for i in range(MAX_NODES):
+                datadir = initialize_datadir(self.options.cachedir, i)
+                args = [os.getenv("BITCOIND", "agrariand"), "-spendzeroconfchange=1", "-server", "-keypool=1", "-datadir=" + datadir, "-discover=0"]
+                if i > 0:
+                    args.append("-connect=127.0.0.1:" + str(p2p_port(0)))
+                self.nodes.append(TestNode(i, self.options.cachedir, extra_args=[], rpchost=None, timewait=None, binary=None, stderr=None, mocktime=self.mocktime, coverage_dir=None))
+                self.nodes[i].args = args
+                self.start_node(i)
+
+            # Wait for RPC connections to be ready
+            for node in self.nodes:
+                node.wait_for_rpc_connection()
+
+            # Create a 200-block-long chain; each of the 4 first nodes
+            # gets 25 mature blocks and 25 immature.
+            # Note: To preserve compatibility with older versions of
+            # initialize_chain, only 4 nodes will generate coins.
+            #
+            # blocks are created with timestamps 10 minutes apart
+            # starting from 2010 minutes in the past
+            self.enable_mocktime()
+            block_time = self.mocktime - (201 * 60)
+            for i in range(2):
+                for peer in range(4):
+                    for j in range(25):
+                        set_node_times(self.nodes, block_time)
+                        self.nodes[peer].generate(1)
+                        block_time += 60
+                    # Must sync before next peer starts generating blocks
+                    sync_blocks(self.nodes)
+
+            # Shut them down, and clean up cache directories:
+            self.stop_nodes()
+            self.nodes = []
+            self.disable_mocktime()
+
+            def cache_path(n, *paths):
+                return os.path.join(get_datadir_path(self.options.cachedir, n), "regtest", *paths)
+
+            for i in range(MAX_NODES):
+                for entry in os.listdir(cache_path(i)):
+                    if entry not in ['wallet.dat', 'chainstate', 'blocks', 'sporks', 'zerocoin', 'backups']:
+                        os.remove(cache_path(i, entry))
+
+        for i in range(self.num_nodes):
+            from_dir = get_datadir_path(self.options.cachedir, i)
+            to_dir = get_datadir_path(self.options.tmpdir, i)
+            shutil.copytree(from_dir, to_dir)
+            initialize_datadir(self.options.tmpdir, i)  # Overwrite port/rpcport in bitcoin.conf
+
+    def _initialize_chain_clean(self):
+        """Initialize empty blockchain for use by the test.
+
+        Create an empty blockchain and num_nodes wallets.
+        Useful if a test case wants complete control over initialization."""
+        for i in range(self.num_nodes):
+            initialize_datadir(self.options.tmpdir, i)
+
+class ComparisonTestFramework(BitcoinTestFramework):
+    """Test framework for doing p2p comparison testing
+
+    Sets up some agrariand binaries:
+    - 1 binary: test binary
+    - 2 binaries: 1 test binary, 1 ref binary
+    - n>2 binaries: 1 test binary, n-1 ref binaries"""
+
+    def set_test_params(self):
+        self.num_nodes = 2
+        self.setup_clean_chain = True
+
+    def add_options(self, parser):
+        parser.add_option("--testbinary", dest="testbinary",
+                          default=os.getenv("BITCOIND", "agrariand"),
+                          help="agrariand binary to test")
+        parser.add_option("--refbinary", dest="refbinary",
+                          default=os.getenv("BITCOIND", "agrariand"),
+                          help="agrariand binary to use for reference nodes (if any)")
+
+    def setup_network(self):
+        extra_args = [['-whitelist=127.0.0.1']] * self.num_nodes
+        if hasattr(self, "extra_args"):
+            extra_args = self.extra_args
+        self.add_nodes(self.num_nodes, extra_args,
+                       binary=[self.options.testbinary] +
+                       [self.options.refbinary] * (self.num_nodes - 1))
+        self.start_nodes()
+
+class SkipTest(Exception):
+    """This exception is raised to skip a test"""
+    def __init__(self, message):
+        self.message = message

test/functional/test_framework/test_node.py

@@ -0,0 +1,286 @@
+#!/usr/bin/env python3
+# Copyright (c) 2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Class for agrariand node under test"""
+
+import decimal
+import errno
+import http.client
+import json
+import logging
+import os
+import re
+import subprocess
+import time
+
+from .authproxy import JSONRPCException
+from .util import (
+    assert_equal,
+    delete_cookie_file,
+    get_rpc_proxy,
+    rpc_url,
+    wait_until,
+    p2p_port,
+)
+
+# For Python 3.4 compatibility
+JSONDecodeError = getattr(json, "JSONDecodeError", ValueError)
+
+BITCOIND_PROC_WAIT_TIMEOUT = 600
+
+class TestNode():
+    """A class for representing a agrariand node under test.
+
+    This class contains:
+
+    - state about the node (whether it's running, etc)
+    - a Python subprocess.Popen object representing the running process
+    - an RPC connection to the node
+    - one or more P2P connections to the node
+
+
+    To make things easier for the test writer, any unrecognised messages will
+    be dispatched to the RPC connection."""
+
+    def __init__(self, i, dirname, extra_args, rpchost, timewait, binary, stderr, mocktime, coverage_dir, use_cli=False):
+        self.index = i
+        self.datadir = os.path.join(dirname, "node" + str(i))
+        self.rpchost = rpchost
+        if timewait:
+            self.rpc_timeout = timewait
+        else:
+            # Wait for up to 60 seconds for the RPC server to respond
+            self.rpc_timeout = 600
+        if binary is None:
+            self.binary = os.getenv("BITCOIND", "agrariand")
+        else:
+            self.binary = binary
+        self.stderr = stderr
+        self.coverage_dir = coverage_dir
+        # Most callers will just need to add extra args to the standard list below. For those callers that need more flexibity, they can just set the args property directly.
+        self.extra_args = extra_args
+        self.args = [self.binary, "-datadir=" + self.datadir, "-server", "-keypool=1", "-discover=0", "-rest", "-logtimemicros", "-debug", "-debugexclude=libevent", "-debugexclude=leveldb", "-mocktime=" + str(mocktime), "-uacomment=testnode%d" % i]
+
+        self.cli = TestNodeCLI(os.getenv("BITCOINCLI", "agrarian-cli"), self.datadir)
+        self.use_cli = use_cli
+
+        self.running = False
+        self.process = None
+        self.rpc_connected = False
+        self.rpc = None
+        self.url = None
+        self.log = logging.getLogger('TestFramework.node%d' % i)
+        self.cleanup_on_exit = True # Whether to kill the node when this object goes away
+
+        self.p2ps = []
+
+    def __del__(self):
+        # Ensure that we don't leave any bitcoind processes lying around after
+        # the test ends
+        if self.process and self.cleanup_on_exit:
+            # Should only happen on test failure
+            # Avoid using logger, as that may have already been shutdown when
+            # this destructor is called.
+            print("Cleaning up leftover process")
+            self.process.kill()
+
+    def __getattr__(self, name):
+        """Dispatches any unrecognised messages to the RPC connection or a CLI instance."""
+        if self.use_cli:
+            return getattr(self.cli, name)
+        else:
+            assert self.rpc_connected and self.rpc is not None, "Error: no RPC connection"
+            return getattr(self.rpc, name)
+
+    def start(self, extra_args=None, stderr=None, *args, **kwargs):
+        """Start the node."""
+        if extra_args is None:
+            extra_args = self.extra_args
+        if stderr is None:
+            stderr = self.stderr
+        # Delete any existing cookie file -- if such a file exists (eg due to
+        # unclean shutdown), it will get overwritten anyway by bitcoind, and
+        # potentially interfere with our attempt to authenticate
+        delete_cookie_file(self.datadir)
+        self.process = subprocess.Popen(self.args + extra_args, stderr=stderr, *args, **kwargs)
+        self.running = True
+        self.log.debug("agrariand started, waiting for RPC to come up")
+
+    def wait_for_rpc_connection(self):
+        """Sets up an RPC connection to the agrariand process. Returns False if unable to connect."""
+        # Poll at a rate of four times per second
+        poll_per_s = 4
+        time.sleep(5)
+        for _ in range(poll_per_s * self.rpc_timeout):
+            assert self.process.poll() is None, "agrariand exited with status %i during initialization" % self.process.returncode
+            try:
+                self.rpc = get_rpc_proxy(rpc_url(self.datadir, self.index, self.rpchost), self.index, timeout=self.rpc_timeout, coveragedir=self.coverage_dir)
+                while self.rpc.getblockcount() < 0:
+                    time.sleep(1)
+                # If the call to getblockcount() succeeds then the RPC connection is up
+                self.rpc_connected = True
+                self.url = self.rpc.url
+                self.log.debug("RPC successfully started")
+                return
+            except IOError as e:
+                if e.errno != errno.ECONNREFUSED:  # Port not yet open?
+                    raise  # unknown IO error
+            except JSONRPCException as e:  # Initialization phase
+                if e.error['code'] != -28:  # RPC in warmup?
+                    raise  # unknown JSON RPC exception
+            except ValueError as e:  # cookie file not found and no rpcuser or rpcassword. bitcoind still starting
+                if "No RPC credentials" not in str(e):
+                    raise
+            time.sleep(1.0 / poll_per_s)
+        raise AssertionError("Unable to connect to agrariand")
+
+    def get_wallet_rpc(self, wallet_name):
+        if self.use_cli:
+            return self.cli("-rpcwallet={}".format(wallet_name))
+        else:
+            assert self.rpc_connected
+            assert self.rpc
+            wallet_path = "wallet/%s" % wallet_name
+            return self.rpc / wallet_path
+
+    def stop_node(self):
+        """Stop the node."""
+        if not self.running:
+            return
+        self.log.debug("Stopping node")
+        try:
+            self.stop()
+        except http.client.CannotSendRequest:
+            self.log.exception("Unable to stop node.")
+        del self.p2ps[:]
+
+    def is_node_stopped(self):
+        """Checks whether the node has stopped.
+
+        Returns True if the node has stopped. False otherwise.
+        This method is responsible for freeing resources (self.process)."""
+        time.sleep(20)
+        if not self.running:
+            return True
+        return_code = self.process.poll()
+        if return_code is None:
+            return False
+
+        # process has stopped. Assert that it didn't return an error code.
+        assert_equal(return_code, 0)
+        self.running = False
+        self.process = None
+        self.rpc_connected = False
+        self.rpc = None
+        self.log.debug("Node stopped")
+        return True
+
+    def wait_until_stopped(self, timeout=BITCOIND_PROC_WAIT_TIMEOUT):
+        wait_until(self.is_node_stopped, timeout=timeout)
+
+    def node_encrypt_wallet(self, passphrase):
+        """"Encrypts the wallet.
+
+        This causes agrariand to shutdown, so this method takes
+        care of cleaning up resources."""
+        self.encryptwallet(passphrase)
+        self.wait_until_stopped()
+
+    def add_p2p_connection(self, p2p_conn, *args, **kwargs):
+        """Add a p2p connection to the node.
+
+        This method adds the p2p connection to the self.p2ps list and also
+        returns the connection to the caller."""
+        if 'dstport' not in kwargs:
+            kwargs['dstport'] = p2p_port(self.index)
+        if 'dstaddr' not in kwargs:
+            kwargs['dstaddr'] = '127.0.0.1'
+
+        p2p_conn.peer_connect(*args, **kwargs)
+        self.p2ps.append(p2p_conn)
+
+        return p2p_conn
+
+    @property
+    def p2p(self):
+        """Return the first p2p connection
+
+        Convenience property - most tests only use a single p2p connection to each
+        node, so this saves having to write node.p2ps[0] many times."""
+        assert self.p2ps, "No p2p connection"
+        return self.p2ps[0]
+
+    def disconnect_p2ps(self):
+        """Close all p2p connections to the node."""
+        for p in self.p2ps:
+            p.peer_disconnect()
+        del self.p2ps[:]
+
+class TestNodeCLIAttr:
+    def __init__(self, cli, command):
+        self.cli = cli
+        self.command = command
+
+    def __call__(self, *args, **kwargs):
+        return self.cli.send_cli(self.command, *args, **kwargs)
+
+    def get_request(self, *args, **kwargs):
+        return lambda: self(*args, **kwargs)
+
+class TestNodeCLI():
+    """Interface to agrarian-cli for an individual node"""
+
+    def __init__(self, binary, datadir):
+        self.options = []
+        self.binary = binary
+        self.datadir = datadir
+        self.input = None
+        self.log = logging.getLogger('TestFramework.bitcoincli')
+
+    def __call__(self, *options, input=None):
+        # TestNodeCLI is callable with agrarian-cli command-line options
+        cli = TestNodeCLI(self.binary, self.datadir)
+        cli.options = [str(o) for o in options]
+        cli.input = input
+        return cli
+
+    def __getattr__(self, command):
+        return TestNodeCLIAttr(self, command)
+
+    def batch(self, requests):
+        results = []
+        for request in requests:
+           try:
+               results.append(dict(result=request()))
+           except JSONRPCException as e:
+               results.append(dict(error=e))
+        return results
+
+    def send_cli(self, command=None, *args, **kwargs):
+        """Run agrarian-cli command. Deserializes returned string as python object."""
+
+        pos_args = [str(arg) for arg in args]
+        named_args = [str(key) + "=" + str(value) for (key, value) in kwargs.items()]
+        assert not (pos_args and named_args), "Cannot use positional arguments and named arguments in the same agrarian-cli call"
+        p_args = [self.binary, "-datadir=" + self.datadir] + self.options
+        if named_args:
+            p_args += ["-named"]
+        if command is not None:
+            p_args += [command]
+        p_args += pos_args + named_args
+        self.log.debug("Running bitcoin-cli command: %s" % command)
+        process = subprocess.Popen(p_args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
+        cli_stdout, cli_stderr = process.communicate(input=self.input)
+        returncode = process.poll()
+        if returncode:
+            match = re.match(r'error code: ([-0-9]+)\nerror message:\n(.*)', cli_stderr)
+            if match:
+                code, message = match.groups()
+                raise JSONRPCException(dict(code=int(code), message=message))
+            # Ignore cli_stdout, raise with cli_stderr
+            raise subprocess.CalledProcessError(returncode, self.binary, output=cli_stderr)
+        try:
+            return json.loads(cli_stdout, parse_float=decimal.Decimal)
+        except JSONDecodeError:
+            return cli_stdout.rstrip("\n")

test/functional/test_framework/util.py

@@ -0,0 +1,588 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Helpful routines for regression testing."""
+
+from base64 import b64encode
+from binascii import hexlify, unhexlify
+from decimal import Decimal, ROUND_DOWN
+import hashlib
+import json
+import logging
+import os
+import random
+import re
+from subprocess import CalledProcessError
+import time
+
+from . import coverage
+from .authproxy import AuthServiceProxy, JSONRPCException
+
+logger = logging.getLogger("TestFramework.utils")
+
+# Assert functions
+##################
+
+def assert_fee_amount(fee, tx_size, fee_per_kB):
+    """Assert the fee was in range"""
+    target_fee = round(tx_size * fee_per_kB / 1000, 8)
+    if fee < target_fee:
+        raise AssertionError("Fee of %s BTC too low! (Should be %s BTC)" % (str(fee), str(target_fee)))
+    # allow the wallet's estimation to be at most 2 bytes off
+    if fee > (tx_size + 20) * fee_per_kB / 1000:
+        raise AssertionError("Fee of %s BTC too high! (Should be %s BTC)" % (str(fee), str(target_fee)))
+
+def assert_equal(thing1, thing2, *args):
+    if thing1 != thing2 or any(thing1 != arg for arg in args):
+        raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
+
+def assert_greater_than(thing1, thing2):
+    if thing1 <= thing2:
+        raise AssertionError("%s <= %s" % (str(thing1), str(thing2)))
+
+def assert_greater_than_or_equal(thing1, thing2):
+    if thing1 < thing2:
+        raise AssertionError("%s < %s" % (str(thing1), str(thing2)))
+
+def assert_raises(exc, fun, *args, **kwds):
+    assert_raises_message(exc, None, fun, *args, **kwds)
+
+def assert_raises_message(exc, message, fun, *args, **kwds):
+    try:
+        fun(*args, **kwds)
+    except JSONRPCException:
+        raise AssertionError("Use assert_raises_rpc_error() to test RPC failures")
+    except exc as e:
+        if message is not None and message not in e.error['message']:
+            raise AssertionError("Expected substring not found:" + e.error['message'])
+    except Exception as e:
+        raise AssertionError("Unexpected exception raised: " + type(e).__name__)
+    else:
+        raise AssertionError("No exception raised")
+
+def assert_raises_process_error(returncode, output, fun, *args, **kwds):
+    """Execute a process and asserts the process return code and output.
+
+    Calls function `fun` with arguments `args` and `kwds`. Catches a CalledProcessError
+    and verifies that the return code and output are as expected. Throws AssertionError if
+    no CalledProcessError was raised or if the return code and output are not as expected.
+
+    Args:
+        returncode (int): the process return code.
+        output (string): [a substring of] the process output.
+        fun (function): the function to call. This should execute a process.
+        args*: positional arguments for the function.
+        kwds**: named arguments for the function.
+    """
+    try:
+        fun(*args, **kwds)
+    except CalledProcessError as e:
+        if returncode != e.returncode:
+            raise AssertionError("Unexpected returncode %i" % e.returncode)
+        if output not in e.output:
+            raise AssertionError("Expected substring not found:" + e.output)
+    else:
+        raise AssertionError("No exception raised")
+
+def assert_raises_rpc_error(code, message, fun, *args, **kwds):
+    """Run an RPC and verify that a specific JSONRPC exception code and message is raised.
+
+    Calls function `fun` with arguments `args` and `kwds`. Catches a JSONRPCException
+    and verifies that the error code and message are as expected. Throws AssertionError if
+    no JSONRPCException was raised or if the error code/message are not as expected.
+
+    Args:
+        code (int), optional: the error code returned by the RPC call (defined
+            in src/rpc/protocol.h). Set to None if checking the error code is not required.
+        message (string), optional: [a substring of] the error string returned by the
+            RPC call. Set to None if checking the error string is not required.
+        fun (function): the function to call. This should be the name of an RPC.
+        args*: positional arguments for the function.
+        kwds**: named arguments for the function.
+    """
+    assert try_rpc(code, message, fun, *args, **kwds), "No exception raised"
+
+def try_rpc(code, message, fun, *args, **kwds):
+    """Tries to run an rpc command.
+
+    Test against error code and message if the rpc fails.
+    Returns whether a JSONRPCException was raised."""
+    try:
+        fun(*args, **kwds)
+    except JSONRPCException as e:
+        # JSONRPCException was thrown as expected. Check the code and message values are correct.
+        if (code is not None) and (code != e.error["code"]):
+            raise AssertionError("Unexpected JSONRPC error code %i" % e.error["code"])
+        if (message is not None) and (message not in e.error['message']):
+            raise AssertionError("Expected substring not found:" + e.error['message'])
+        return True
+    except Exception as e:
+        raise AssertionError("Unexpected exception raised: " + type(e).__name__)
+    else:
+        return False
+
+def assert_is_hex_string(string):
+    try:
+        int(string, 16)
+    except Exception as e:
+        raise AssertionError(
+            "Couldn't interpret %r as hexadecimal; raised: %s" % (string, e))
+
+def assert_is_hash_string(string, length=64):
+    if not isinstance(string, str):
+        raise AssertionError("Expected a string, got type %r" % type(string))
+    elif length and len(string) != length:
+        raise AssertionError(
+            "String of length %d expected; got %d" % (length, len(string)))
+    elif not re.match('[abcdef0-9]+$', string):
+        raise AssertionError(
+            "String %r contains invalid characters for a hash." % string)
+
+def assert_array_result(object_array, to_match, expected, should_not_find=False):
+    """
+        Pass in array of JSON objects, a dictionary with key/value pairs
+        to match against, and another dictionary with expected key/value
+        pairs.
+        If the should_not_find flag is true, to_match should not be found
+        in object_array
+        """
+    if should_not_find:
+        assert_equal(expected, {})
+    num_matched = 0
+    for item in object_array:
+        all_match = True
+        for key, value in to_match.items():
+            if item[key] != value:
+                all_match = False
+        if not all_match:
+            continue
+        elif should_not_find:
+            num_matched = num_matched + 1
+        for key, value in expected.items():
+            if item[key] != value:
+                raise AssertionError("%s : expected %s=%s" % (str(item), str(key), str(value)))
+            num_matched = num_matched + 1
+    if num_matched == 0 and not should_not_find:
+        raise AssertionError("No objects matched %s" % (str(to_match)))
+    if num_matched > 0 and should_not_find:
+        raise AssertionError("Objects were found %s" % (str(to_match)))
+
+# Utility functions
+###################
+
+def check_json_precision():
+    """Make sure json library being used does not lose precision converting BTC values"""
+    n = Decimal("20000000.00000003")
+    satoshis = int(json.loads(json.dumps(float(n))) * 1.0e8)
+    if satoshis != 2000000000000003:
+        raise RuntimeError("JSON encode/decode loses precision")
+
+def count_bytes(hex_string):
+    return len(bytearray.fromhex(hex_string))
+
+def bytes_to_hex_str(byte_str):
+    return hexlify(byte_str).decode('ascii')
+
+def hash256(byte_str):
+    sha256 = hashlib.sha256()
+    sha256.update(byte_str)
+    sha256d = hashlib.sha256()
+    sha256d.update(sha256.digest())
+    return sha256d.digest()[::-1]
+
+def hex_str_to_bytes(hex_str):
+    return unhexlify(hex_str.encode('ascii'))
+
+def str_to_b64str(string):
+    return b64encode(string.encode('utf-8')).decode('ascii')
+
+def satoshi_round(amount):
+    return Decimal(amount).quantize(Decimal('0.00000001'), rounding=ROUND_DOWN)
+
+def wait_until(predicate, *, attempts=float('inf'), timeout=float('inf'), lock=None):
+    if attempts == float('inf') and timeout == float('inf'):
+        timeout = 60
+    attempt = 0
+    timeout += time.time()
+
+    while attempt < attempts and time.time() < timeout:
+        if lock:
+            with lock:
+                if predicate():
+                    return
+        else:
+            if predicate():
+                return
+        attempt += 1
+        time.sleep(0.5)
+
+    # Print the cause of the timeout
+    assert_greater_than(attempts, attempt)
+    assert_greater_than(timeout, time.time())
+    raise RuntimeError('Unreachable')
+
+# RPC/P2P connection constants and functions
+############################################
+
+# The maximum number of nodes a single test can spawn
+MAX_NODES = 8
+# Don't assign rpc or p2p ports lower than this
+PORT_MIN = 11000
+# The number of ports to "reserve" for p2p and rpc, each
+PORT_RANGE = 5000
+
+class PortSeed:
+    # Must be initialized with a unique integer for each process
+    n = None
+
+def get_rpc_proxy(url, node_number, timeout=None, coveragedir=None):
+    """
+    Args:
+        url (str): URL of the RPC server to call
+        node_number (int): the node number (or id) that this calls to
+
+    Kwargs:
+        timeout (int): HTTP timeout in seconds
+
+    Returns:
+        AuthServiceProxy. convenience object for making RPC calls.
+
+    """
+    proxy_kwargs = {}
+    if timeout is not None:
+        proxy_kwargs['timeout'] = timeout
+
+    proxy = AuthServiceProxy(url, **proxy_kwargs)
+    proxy.url = url  # store URL on proxy for info
+
+    coverage_logfile = coverage.get_filename(
+        coveragedir, node_number) if coveragedir else None
+
+    return coverage.AuthServiceProxyWrapper(proxy, coverage_logfile)
+
+def p2p_port(n):
+    assert(n <= MAX_NODES)
+    return PORT_MIN + n + (MAX_NODES * PortSeed.n) % (PORT_RANGE - 1 - MAX_NODES)
+
+def rpc_port(n):
+    return PORT_MIN + PORT_RANGE + n + (MAX_NODES * PortSeed.n) % (PORT_RANGE - 1 - MAX_NODES)
+
+def rpc_url(datadir, i, rpchost=None):
+    rpc_u, rpc_p = get_auth_cookie(datadir)
+    host = '127.0.0.1'
+    port = rpc_port(i)
+    if rpchost:
+        parts = rpchost.split(':')
+        if len(parts) == 2:
+            host, port = parts
+        else:
+            host = rpchost
+    return "http://%s:%s@%s:%d" % (rpc_u, rpc_p, host, int(port))
+
+# Node functions
+################
+
+def initialize_datadir(dirname, n):
+    datadir = os.path.join(dirname, "node" + str(n))
+    if not os.path.isdir(datadir):
+        os.makedirs(datadir)
+    rpc_u, rpc_p = rpc_auth_pair(n)
+    with open(os.path.join(datadir, "agrarian.conf"), 'w', encoding='utf8') as f:
+        f.write("regtest=1\n")
+        f.write("rpcuser=" + rpc_u + "\n")
+        f.write("rpcpassword=" + rpc_p + "\n")
+        f.write("port=" + str(p2p_port(n)) + "\n")
+        f.write("rpcport=" + str(rpc_port(n)) + "\n")
+        f.write("listenonion=0\n")
+        f.write("litemode=1\n")
+        f.write("enablezeromint=0\n")
+        f.write("precompute=0\n")
+        f.write("staking=0\n")
+        f.write("spendzeroconfchange=1\n")
+    return datadir
+
+def rpc_auth_pair(n):
+    return 'rpcuser�' + str(n), 'rpcpass�' + str(n)
+
+def get_datadir_path(dirname, n):
+    return os.path.join(dirname, "node" + str(n))
+
+def get_auth_cookie(datadir):
+    user = None
+    password = None
+    if os.path.isfile(os.path.join(datadir, "agrarian.conf")):
+        with open(os.path.join(datadir, "agrarian.conf"), 'r', encoding='utf8') as f:
+            for line in f:
+                if line.startswith("rpcuser="):
+                    assert user is None  # Ensure that there is only one rpcuser line
+                    user = line.split("=")[1].strip("\n")
+                if line.startswith("rpcpassword="):
+                    assert password is None  # Ensure that there is only one rpcpassword line
+                    password = line.split("=")[1].strip("\n")
+    if os.path.isfile(os.path.join(datadir, "regtest", ".cookie")):
+        with open(os.path.join(datadir, "regtest", ".cookie"), 'r') as f:
+            userpass = f.read()
+            split_userpass = userpass.split(':')
+            user = split_userpass[0]
+            password = split_userpass[1]
+    if user is None or password is None:
+        raise ValueError("No RPC credentials")
+    return user, password
+
+# If a cookie file exists in the given datadir, delete it.
+def delete_cookie_file(datadir):
+    if os.path.isfile(os.path.join(datadir, "regtest", ".cookie")):
+        logger.debug("Deleting leftover cookie file")
+        os.remove(os.path.join(datadir, "regtest", ".cookie"))
+
+def get_bip9_status(node, key):
+    info = node.getblockchaininfo()
+    return info['bip9_softforks'][key]
+
+def set_node_times(nodes, t):
+    for node in nodes:
+        node.setmocktime(t)
+
+def disconnect_nodes(from_connection, node_num):
+    for addr in [peer['addr'] for peer in from_connection.getpeerinfo() if "testnode%d" % node_num in peer['subver']]:
+        try:
+            from_connection.disconnectnode(addr)
+        except JSONRPCException as e:
+            # If this node is disconnected between calculating the peer id
+            # and issuing the disconnect, don't worry about it.
+            # This avoids a race condition if we're mass-disconnecting peers.
+            if e.error['code'] != -29: # RPC_CLIENT_NODE_NOT_CONNECTED
+                raise
+
+    # wait to disconnect
+    wait_until(lambda: [peer['addr'] for peer in from_connection.getpeerinfo() if "testnode%d" % node_num in peer['subver']] == [], timeout=5)
+
+def connect_nodes(from_connection, node_num):
+    ip_port = "127.0.0.1:" + str(p2p_port(node_num))
+    from_connection.addnode(ip_port, "onetry")
+    # poll until version handshake complete to avoid race conditions
+    # with transaction relaying
+    wait_until(lambda:  all(peer['version'] != 0 for peer in from_connection.getpeerinfo()))
+
+def connect_nodes_bi(nodes, a, b):
+    connect_nodes(nodes[a], b)
+    connect_nodes(nodes[b], a)
+
+def sync_blocks(rpc_connections, *, wait=1, timeout=60):
+    """
+    Wait until everybody has the same tip.
+
+    sync_blocks needs to be called with an rpc_connections set that has least
+    one node already synced to the latest, stable tip, otherwise there's a
+    chance it might return before all nodes are stably synced.
+    """
+    # Use getblockcount() instead of waitforblockheight() to determine the
+    # initial max height because the two RPCs look at different internal global
+    # variables (chainActive vs latestBlock) and the former gets updated
+    # earlier.
+    time.sleep(5)
+    maxheight = max(x.getblockcount() for x in rpc_connections)
+    start_time = cur_time = time.time()
+    while cur_time <= start_time + timeout:
+        tips = [r.waitforblockheight(maxheight, int(wait * 1000)) for r in rpc_connections]
+        if all(t["height"] == maxheight for t in tips):
+            if all(t["hash"] == tips[0]["hash"] for t in tips):
+                return
+            raise AssertionError("Block sync failed, mismatched block hashes:{}".format(
+                                 "".join("\n  {!r}".format(tip) for tip in tips)))
+        cur_time = time.time()
+    raise AssertionError("Block sync to height {} timed out:{}".format(
+                         maxheight, "".join("\n  {!r}".format(tip) for tip in tips)))
+
+def sync_chain(rpc_connections, *, wait=1, timeout=60):
+    """
+    Wait until everybody has the same best block
+    """
+    while timeout > 0:
+        best_hash = [x.getbestblockhash() for x in rpc_connections]
+        if best_hash == [best_hash[0]] * len(best_hash):
+            return
+        time.sleep(wait)
+        timeout -= wait
+    raise AssertionError("Chain sync failed: Best block hashes don't match")
+
+def sync_mempools(rpc_connections, *, wait=1, timeout=60, flush_scheduler=True):
+    """
+    Wait until everybody has the same transactions in their memory
+    pools
+    """
+    while timeout > 0:
+        pool = set(rpc_connections[0].getrawmempool())
+        num_match = 1
+        for i in range(1, len(rpc_connections)):
+            if set(rpc_connections[i].getrawmempool()) == pool:
+                num_match = num_match + 1
+        if num_match == len(rpc_connections):
+            #if flush_scheduler:
+                #for r in rpc_connections:
+                #    r.syncwithvalidationinterfacequeue()
+            return
+        time.sleep(wait)
+        timeout -= wait
+    raise AssertionError("Mempool sync failed")
+
+# Transaction/Block functions
+#############################
+
+def find_output(node, txid, amount):
+    """
+    Return index to output of txid with value amount
+    Raises exception if there is none.
+    """
+    txdata = node.getrawtransaction(txid, 1)
+    for i in range(len(txdata["vout"])):
+        if txdata["vout"][i]["value"] == amount:
+            return i
+    raise RuntimeError("find_output txid %s : %s not found" % (txid, str(amount)))
+
+def gather_inputs(from_node, amount_needed, confirmations_required=1):
+    """
+    Return a random set of unspent txouts that are enough to pay amount_needed
+    """
+    assert(confirmations_required >= 0)
+    utxo = from_node.listunspent(confirmations_required)
+    random.shuffle(utxo)
+    inputs = []
+    total_in = Decimal("0.00000000")
+    while total_in < amount_needed and len(utxo) > 0:
+        t = utxo.pop()
+        total_in += t["amount"]
+        inputs.append({"txid": t["txid"], "vout": t["vout"], "address": t["address"]})
+    if total_in < amount_needed:
+        raise RuntimeError("Insufficient funds: need %d, have %d" % (amount_needed, total_in))
+    return (total_in, inputs)
+
+def make_change(from_node, amount_in, amount_out, fee):
+    """
+    Create change output(s), return them
+    """
+    outputs = {}
+    amount = amount_out + fee
+    change = amount_in - amount
+    if change > amount * 2:
+        # Create an extra change output to break up big inputs
+        change_address = from_node.getnewaddress()
+        # Split change in two, being careful of rounding:
+        outputs[change_address] = Decimal(change / 2).quantize(Decimal('0.00000001'), rounding=ROUND_DOWN)
+        change = amount_in - amount - outputs[change_address]
+    if change > 0:
+        outputs[from_node.getnewaddress()] = change
+    return outputs
+
+def random_transaction(nodes, amount, min_fee, fee_increment, fee_variants):
+    """
+    Create a random transaction.
+    Returns (txid, hex-encoded-transaction-data, fee)
+    """
+    from_node = random.choice(nodes)
+    to_node = random.choice(nodes)
+    fee = min_fee + fee_increment * random.randint(0, fee_variants)
+
+    (total_in, inputs) = gather_inputs(from_node, amount + fee)
+    outputs = make_change(from_node, total_in, amount, fee)
+    outputs[to_node.getnewaddress()] = float(amount)
+
+    rawtx = from_node.createrawtransaction(inputs, outputs)
+    signresult = from_node.signrawtransaction(rawtx)
+    txid = from_node.sendrawtransaction(signresult["hex"], True)
+
+    return (txid, signresult["hex"], fee)
+
+# Helper to create at least "count" utxos
+# Pass in a fee that is sufficient for relay and mining new transactions.
+def create_confirmed_utxos(fee, node, count):
+    to_generate = int(0.5 * count) + 101
+    while to_generate > 0:
+        node.generate(min(25, to_generate))
+        to_generate -= 25
+    utxos = node.listunspent()
+    iterations = count - len(utxos)
+    addr1 = node.getnewaddress()
+    addr2 = node.getnewaddress()
+    if iterations <= 0:
+        return utxos
+    for i in range(iterations):
+        t = utxos.pop()
+        inputs = []
+        inputs.append({"txid": t["txid"], "vout": t["vout"]})
+        outputs = {}
+        send_value = t['amount'] - fee
+        outputs[addr1] = float(satoshi_round(send_value / 2))
+        outputs[addr2] = float(satoshi_round(send_value / 2))
+        raw_tx = node.createrawtransaction(inputs, outputs)
+        signed_tx = node.signrawtransaction(raw_tx)["hex"]
+        node.sendrawtransaction(signed_tx)
+
+    while (node.getmempoolinfo()['size'] > 0):
+        node.generate(1)
+
+    utxos = node.listunspent()
+    assert(len(utxos) >= count)
+    return utxos
+
+# Create large OP_RETURN txouts that can be appended to a transaction
+# to make it large (helper for constructing large transactions).
+def gen_return_txouts():
+    # Some pre-processing to create a bunch of OP_RETURN txouts to insert into transactions we create
+    # So we have big transactions (and therefore can't fit very many into each block)
+    # create one script_pubkey
+    script_pubkey = "6a4d0200"  # OP_RETURN OP_PUSH2 512 bytes
+    for i in range(512):
+        script_pubkey = script_pubkey + "01"
+    # concatenate 128 txouts of above script_pubkey which we'll insert before the txout for change
+    txouts = "81"
+    for k in range(128):
+        # add txout value
+        txouts = txouts + "0000000000000000"
+        # add length of script_pubkey
+        txouts = txouts + "fd0402"
+        # add script_pubkey
+        txouts = txouts + script_pubkey
+    return txouts
+
+def create_tx(node, coinbase, to_address, amount):
+    inputs = [{"txid": coinbase, "vout": 0}]
+    outputs = {to_address: amount}
+    rawtx = node.createrawtransaction(inputs, outputs)
+    signresult = node.signrawtransaction(rawtx)
+    assert_equal(signresult["complete"], True)
+    return signresult["hex"]
+
+# Create a spend of each passed-in utxo, splicing in "txouts" to each raw
+# transaction to make it large.  See gen_return_txouts() above.
+def create_lots_of_big_transactions(node, txouts, utxos, num, fee):
+    addr = node.getnewaddress()
+    txids = []
+    for _ in range(num):
+        t = utxos.pop()
+        inputs = [{"txid": t["txid"], "vout": t["vout"]}]
+        outputs = {}
+        change = t['amount'] - fee
+        outputs[addr] = float(satoshi_round(change))
+        rawtx = node.createrawtransaction(inputs, outputs)
+        newtx = rawtx[0:92]
+        newtx = newtx + txouts
+        newtx = newtx + rawtx[94:]
+        signresult = node.signrawtransaction(newtx, None, None, "NONE")
+        txid = node.sendrawtransaction(signresult["hex"], True)
+        txids.append(txid)
+    return txids
+
+def mine_large_block(node, utxos=None):
+    # generate a 66k transaction,
+    # and 14 of them is close to the 1MB block limit
+    num = 14
+    txouts = gen_return_txouts()
+    utxos = utxos if utxos is not None else []
+    if len(utxos) < num:
+        utxos.clear()
+        utxos.extend(node.listunspent())
+    fee = 100 * node.getnetworkinfo()["relayfee"]
+    create_lots_of_big_transactions(node, txouts, utxos, num, fee=fee)
+    node.generate(1)

test/functional/test_runner.py

@@ -0,0 +1,551 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Run regression test suite.
+
+This module calls down into individual test cases via subprocess. It will
+forward all unrecognized arguments onto the individual test scripts.
+
+Functional tests are disabled on Windows by default. Use --force to run them anyway.
+
+For a description of arguments recognized by test scripts, see
+`test/functional/test_framework/test_framework.py:BitcoinTestFramework.main`.
+
+"""
+
+import argparse
+from collections import deque
+import configparser
+import datetime
+import os
+import time
+import shutil
+import signal
+import sys
+import subprocess
+import tempfile
+import re
+import logging
+
+# Formatting. Default colors to empty strings.
+BOLD, BLUE, RED, GREY = ("", ""), ("", ""), ("", ""), ("", "")
+try:
+    # Make sure python thinks it can write unicode to its stdout
+    "\u2713".encode("utf_8").decode(sys.stdout.encoding)
+    TICK = "✓ "
+    CROSS = "✖ "
+    CIRCLE = "○ "
+except UnicodeDecodeError:
+    TICK = "P "
+    CROSS = "x "
+    CIRCLE = "o "
+
+if os.name == 'posix':
+    # primitive formatting on supported
+    # terminal via ANSI escape sequences:
+    BOLD = ('\033[0m', '\033[1m')
+    BLUE = ('\033[0m', '\033[0;34m')
+    RED = ('\033[0m', '\033[0;31m')
+    GREY = ('\033[0m', '\033[1;30m')
+
+TEST_EXIT_PASSED = 0
+TEST_EXIT_SKIPPED = 77
+
+BASE_SCRIPTS= [
+    # Scripts that are run by the travis build process.
+    # Longest test should go first, to favor running tests in parallel
+    'wallet_backup.py',
+    'p2p_pos_fakestake.py',
+    'p2p_pos_fakestake_accepted.py',
+    #'p2p_zpos_fakestake.py',
+    #'p2p_zpos_fakestake_accepted.py',
+    #'zerocoin_wrapped_serials.py',
+    # vv Tests less than 5m vv
+    #'feature_block.py',
+    #'rpc_fundrawtransaction.py',
+    # vv Tests less than 2m vv
+    'p2p_pos_doublespend.py',
+    'wallet_basic.py',
+    'wallet_accounts.py',
+    'wallet_dump.py',
+    'rpc_listtransactions.py',
+    # vv Tests less than 60s vv
+    'wallet_zapwallettxes.py',
+    #'wallet_importmulti.py',
+    #'mempool_limit.py', # We currently don't limit our mempool
+    'wallet_listreceivedby.py',
+    #'wallet_abandonconflict.py',
+    'rpc_rawtransaction.py',
+    'feature_reindex.py',
+    'rpc_bip38.py',
+    # vv Tests less than 30s vv
+    'wallet_keypool_topup.py',
+    #'interface_zmq.py',
+    'interface_bitcoin_cli.py',
+    'mempool_resurrect.py',
+    'wallet_txn_doublespend.py --mineblock',
+    'wallet_txn_clone.py --mineblock',
+    #'rpc_getchaintips.py',
+    'interface_rest.py',
+    'mempool_spend_coinbase.py',
+    'mempool_reorg.py',
+    #'mempool_persist.py', # Not yet implemented
+    'interface_http.py',
+    #'rpc_users.py',
+    'feature_proxy.py',
+    'rpc_signrawtransaction.py',
+    'p2p_disconnect_ban.py',
+    'rpc_decodescript.py',
+    'rpc_blockchain.py',
+    #'rpc_deprecated.py',
+    'wallet_disable.py',
+    'rpc_net.py',
+    'wallet_keypool.py',
+    #'p2p_mempool.py',
+    #'mining_prioritisetransaction.py',
+    #'p2p_invalid_block.py',
+    #'p2p_invalid_tx.py',
+    'rpc_signmessage.py',
+    #'wallet_import_rescan.py',
+    #'mining_basic.py',
+    #'wallet_bumpfee.py',
+    #'wallet_listsinceblock.py',
+    #'p2p_leak.py',
+    'wallet_encryption.py',
+    #'feature_cltv.py',
+    #'wallet_resendwallettransactions.py',
+    #'feature_minchainwork.py',
+    #'p2p_fingerprint.py',
+    'feature_uacomment.py',
+    #'p2p_unrequested_blocks.py',
+    #'feature_config_args.py',
+    'feature_help.py',
+    # Don't append tests at the end to avoid merge conflicts
+    # Put them in a random line within the section that fits their approximate run-time
+]
+
+EXTENDED_SCRIPTS = [
+    # These tests are not run by the travis build process.
+    # Longest test should go first, to favor running tests in parallel
+    # vv Tests less than 20m vv
+    #'feature_fee_estimation.py',
+    # vv Tests less than 5m vv
+    # vv Tests less than 2m vv
+    #'p2p_timeouts.py',
+    # vv Tests less than 60s vv
+    #'p2p_feefilter.py',
+    'rpc_bind.py',
+    # vv Tests less than 30s vv
+    #'example_test.py',
+    'feature_notifications.py',
+    'rpc_invalidateblock.py',
+]
+
+# Place EXTENDED_SCRIPTS first since it has the 3 longest running tests
+ALL_SCRIPTS = EXTENDED_SCRIPTS + BASE_SCRIPTS
+
+NON_SCRIPTS = [
+    # These are python files that live in the functional tests directory, but are not test scripts.
+    "combine_logs.py",
+    "create_cache.py",
+    "test_runner.py",
+]
+
+def main():
+    # Parse arguments and pass through unrecognised args
+    parser = argparse.ArgumentParser(add_help=False,
+                                     usage='%(prog)s [test_runner.py options] [script options] [scripts]',
+                                     description=__doc__,
+                                     epilog='''
+    Help text and arguments for individual test script:''',
+                                     formatter_class=argparse.RawTextHelpFormatter)
+    parser.add_argument('--combinedlogslen', '-c', type=int, default=0, help='print a combined log (of length n lines) from all test nodes and test framework to the console on failure.')
+    parser.add_argument('--coverage', action='store_true', help='generate a basic coverage report for the RPC interface')
+    parser.add_argument('--exclude', '-x', help='specify a comma-separated-list of scripts to exclude.')
+    parser.add_argument('--extended', action='store_true', help='run the extended test suite in addition to the basic tests')
+    parser.add_argument('--force', '-f', action='store_true', help='run tests even on platforms where they are disabled by default (e.g. windows).')
+    parser.add_argument('--help', '-h', '-?', action='store_true', help='print help text and exit')
+    parser.add_argument('--jobs', '-j', type=int, default=4, help='how many test scripts to run in parallel. Default=4.')
+    parser.add_argument('--keepcache', '-k', action='store_true', help='the default behavior is to flush the cache directory on startup. --keepcache retains the cache from the previous testrun.')
+    parser.add_argument('--quiet', '-q', action='store_true', help='only print results summary and failure logs')
+    parser.add_argument('--tmpdirprefix', '-t', default=tempfile.gettempdir(), help="Root directory for datadirs")
+    args, unknown_args = parser.parse_known_args()
+
+    # args to be passed on always start with two dashes; tests are the remaining unknown args
+    tests = [arg for arg in unknown_args if arg[:2] != "--"]
+    passon_args = [arg for arg in unknown_args if arg[:2] == "--"]
+
+    # Read config generated by configure.
+    config = configparser.ConfigParser()
+    configfile = os.path.abspath(os.path.dirname(__file__)) + "/../config.ini"
+    config.read_file(open(configfile))
+
+    passon_args.append("--configfile=%s" % configfile)
+
+    # Set up logging
+    logging_level = logging.INFO if args.quiet else logging.DEBUG
+    logging.basicConfig(format='%(message)s', level=logging_level)
+
+    # Create base test directory
+    tmpdir = "%s/agrarian_test_runner_%s" % (args.tmpdirprefix, datetime.datetime.now().strftime("%Y%m%d_%H%M%S"))
+    os.makedirs(tmpdir)
+
+    logging.debug("Temporary test directory at %s" % tmpdir)
+
+    enable_wallet = config["components"].getboolean("ENABLE_WALLET")
+    enable_utils = config["components"].getboolean("ENABLE_UTILS")
+    enable_bitcoind = config["components"].getboolean("ENABLE_BITCOIND")
+
+    if config["environment"]["EXEEXT"] == ".exe" and not args.force:
+        # https://github.com/bitcoin/bitcoin/commit/d52802551752140cf41f0d9a225a43e84404d3e9
+        # https://github.com/bitcoin/bitcoin/pull/5677#issuecomment-136646964
+        print("Tests currently disabled on Windows by default. Use --force option to enable")
+        sys.exit(0)
+
+    if not (enable_wallet and enable_utils and enable_bitcoind):
+        print("No functional tests to run. Wallet, utils, and agrariand must all be enabled")
+        print("Rerun `configure` with -enable-wallet, -with-utils and -with-daemon and rerun make")
+        sys.exit(0)
+
+    # Build list of tests
+    if tests:
+        # Individual tests have been specified. Run specified tests that exist
+        # in the ALL_SCRIPTS list. Accept the name with or without .py extension.
+        tests = [re.sub("\.py$", "", t) + ".py" for t in tests]
+        test_list = []
+        for t in tests:
+            if t in ALL_SCRIPTS:
+                test_list.append(t)
+            else:
+                print("{}WARNING!{} Test '{}' not found in full test list.".format(BOLD[1], BOLD[0], t))
+    else:
+        # No individual tests have been specified.
+        # Run all base tests, and optionally run extended tests.
+        test_list = BASE_SCRIPTS
+        if args.extended:
+            # place the EXTENDED_SCRIPTS first since the three longest ones
+            # are there and the list is shorter
+            test_list = EXTENDED_SCRIPTS + test_list
+
+    # Remove the test cases that the user has explicitly asked to exclude.
+    if args.exclude:
+        tests_excl = [re.sub("\.py$", "", t) + ".py" for t in args.exclude.split(',')]
+        for exclude_test in tests_excl:
+            if exclude_test in test_list:
+                test_list.remove(exclude_test)
+            else:
+                print("{}WARNING!{} Test '{}' not found in current test list.".format(BOLD[1], BOLD[0], exclude_test))
+
+    if not test_list:
+        print("No valid test scripts specified. Check that your test is in one "
+              "of the test lists in test_runner.py, or run test_runner.py with no arguments to run all tests")
+        sys.exit(0)
+
+    if args.help:
+        # Print help for test_runner.py, then print help of the first script (with args removed) and exit.
+        parser.print_help()
+        subprocess.check_call([(config["environment"]["SRCDIR"] + '/test/functional/' + test_list[0].split()[0])] + ['-h'])
+        sys.exit(0)
+
+    check_script_list(config["environment"]["SRCDIR"])
+    check_script_prefixes()
+
+    if not args.keepcache:
+        shutil.rmtree("%s/test/cache" % config["environment"]["BUILDDIR"], ignore_errors=True)
+
+    run_tests(test_list, config["environment"]["SRCDIR"], config["environment"]["BUILDDIR"], config["environment"]["EXEEXT"], tmpdir, args.jobs, args.coverage, passon_args, args.combinedlogslen)
+
+def run_tests(test_list, src_dir, build_dir, exeext, tmpdir, jobs=1, enable_coverage=False, args=[], combined_logs_len=0):
+    # Warn if bitcoind is already running (unix only)
+    try:
+        if subprocess.check_output(["pidof", "agrariand"]) is not None:
+            print("%sWARNING!%s There is already a agrariand process running on this system. Tests may fail unexpectedly due to resource contention!" % (BOLD[1], BOLD[0]))
+    except (OSError, subprocess.SubprocessError):
+        pass
+
+    # Warn if there is a cache directory
+    cache_dir = "%s/test/cache" % build_dir
+    if os.path.isdir(cache_dir):
+        print("%sWARNING!%s There is a cache directory here: %s. If tests fail unexpectedly, try deleting the cache directory." % (BOLD[1], BOLD[0], cache_dir))
+
+    #Set env vars
+    if "BITCOIND" not in os.environ:
+        os.environ["BITCOIND"] = build_dir + '/src/agrariand' + exeext
+        os.environ["BITCOINCLI"] = build_dir + '/src/agrarian-cli' + exeext
+
+    tests_dir = src_dir + '/test/functional/'
+
+    flags = ["--srcdir={}/src".format(build_dir)] + args
+    flags.append("--cachedir=%s" % cache_dir)
+
+    if enable_coverage:
+        coverage = RPCCoverage()
+        flags.append(coverage.flag)
+        logging.debug("Initializing coverage directory at %s" % coverage.dir)
+    else:
+        coverage = None
+
+    if len(test_list) > 1 and jobs > 1:
+        # Populate cache
+        try:
+            subprocess.check_output([tests_dir + 'create_cache.py'] + flags + ["--tmpdir=%s/cache" % tmpdir])
+        except subprocess.CalledProcessError as e:
+            sys.stdout.buffer.write(e.output)
+            raise
+
+    #Run Tests
+    job_queue = TestHandler(jobs, tests_dir, tmpdir, test_list, flags)
+    time0 = time.time()
+    test_results = []
+
+    max_len_name = len(max(test_list, key=len))
+
+    for _ in range(len(test_list)):
+        test_result, testdir, stdout, stderr = job_queue.get_next()
+        test_results.append(test_result)
+
+        if test_result.status == "Passed":
+            logging.debug("\n%s%s%s passed, Duration: %s s" % (BOLD[1], test_result.name, BOLD[0], test_result.time))
+        elif test_result.status == "Skipped":
+            logging.debug("\n%s%s%s skipped" % (BOLD[1], test_result.name, BOLD[0]))
+        else:
+            print("\n%s%s%s failed, Duration: %s s\n" % (BOLD[1], test_result.name, BOLD[0], test_result.time))
+            print(BOLD[1] + 'stdout:\n' + BOLD[0] + stdout + '\n')
+            print(BOLD[1] + 'stderr:\n' + BOLD[0] + stderr + '\n')
+            if combined_logs_len and os.path.isdir(testdir):
+                # Print the final `combinedlogslen` lines of the combined logs
+                print('{}Combine the logs and print the last {} lines ...{}'.format(BOLD[1], combined_logs_len, BOLD[0]))
+                print('\n============')
+                print('{}Combined log for {}:{}'.format(BOLD[1], testdir, BOLD[0]))
+                print('============\n')
+                combined_logs, _ = subprocess.Popen([os.path.join(tests_dir, 'combine_logs.py'), '-c', testdir], universal_newlines=True, stdout=subprocess.PIPE).communicate()
+                print("\n".join(deque(combined_logs.splitlines(), combined_logs_len)))
+
+    print_results(test_results, max_len_name, (int(time.time() - time0)))
+
+    if coverage:
+        coverage.report_rpc_coverage()
+
+        logging.debug("Cleaning up coverage data")
+        coverage.cleanup()
+
+    # Clear up the temp directory if all subdirectories are gone
+    if not os.listdir(tmpdir):
+        os.rmdir(tmpdir)
+
+    all_passed = all(map(lambda test_result: test_result.was_successful, test_results))
+
+    sys.exit(not all_passed)
+
+def print_results(test_results, max_len_name, runtime):
+    results = "\n" + BOLD[1] + "%s | %s | %s\n\n" % ("TEST".ljust(max_len_name), "STATUS   ", "DURATION") + BOLD[0]
+
+    test_results.sort(key=lambda result: result.name.lower())
+    all_passed = True
+    time_sum = 0
+
+    for test_result in test_results:
+        all_passed = all_passed and test_result.was_successful
+        time_sum += test_result.time
+        test_result.padding = max_len_name
+        results += str(test_result)
+
+    status = TICK + "Passed" if all_passed else CROSS + "Failed"
+    results += BOLD[1] + "\n%s | %s | %s s (accumulated) \n" % ("ALL".ljust(max_len_name), status.ljust(9), time_sum) + BOLD[0]
+    results += "Runtime: %s s\n" % (runtime)
+    print(results)
+
+class TestHandler:
+    """
+    Trigger the test scripts passed in via the list.
+    """
+
+    def __init__(self, num_tests_parallel, tests_dir, tmpdir, test_list=None, flags=None):
+        assert(num_tests_parallel >= 1)
+        self.num_jobs = num_tests_parallel
+        self.tests_dir = tests_dir
+        self.tmpdir = tmpdir
+        self.test_list = test_list
+        self.flags = flags
+        self.num_running = 0
+        # In case there is a graveyard of zombie agrariands, we can apply a
+        # pseudorandom offset to hopefully jump over them.
+        # (625 is PORT_RANGE/MAX_NODES)
+        self.portseed_offset = int(time.time() * 1000) % 625
+        self.jobs = []
+
+    def get_next(self):
+        while self.num_running < self.num_jobs and self.test_list:
+            # Add tests
+            self.num_running += 1
+            t = self.test_list.pop(0)
+            portseed = len(self.test_list) + self.portseed_offset
+            portseed_arg = ["--portseed={}".format(portseed)]
+            log_stdout = tempfile.SpooledTemporaryFile(max_size=2**16)
+            log_stderr = tempfile.SpooledTemporaryFile(max_size=2**16)
+            test_argv = t.split()
+            testdir = "{}/{}_{}".format(self.tmpdir, re.sub(".py$", "", test_argv[0]), portseed)
+            tmpdir_arg = ["--tmpdir={}".format(testdir)]
+            self.jobs.append((t,
+                              time.time(),
+                              subprocess.Popen([self.tests_dir + test_argv[0]] + test_argv[1:] + self.flags + portseed_arg + tmpdir_arg,
+                                               universal_newlines=True,
+                                               stdout=log_stdout,
+                                               stderr=log_stderr),
+                              testdir,
+                              log_stdout,
+                              log_stderr))
+        if not self.jobs:
+            raise IndexError('pop from empty list')
+        while True:
+            # Return first proc that finishes
+            time.sleep(.5)
+            for j in self.jobs:
+                (name, time0, proc, testdir, log_out, log_err) = j
+                if os.getenv('TRAVIS') == 'true' and int(time.time() - time0) > 20 * 60:
+                    # In travis, timeout individual tests after 20 minutes (to stop tests hanging and not
+                    # providing useful output.
+                    proc.send_signal(signal.SIGINT)
+                if proc.poll() is not None:
+                    log_out.seek(0), log_err.seek(0)
+                    [stdout, stderr] = [l.read().decode('utf-8') for l in (log_out, log_err)]
+                    log_out.close(), log_err.close()
+                    if proc.returncode == TEST_EXIT_PASSED and stderr == "":
+                        status = "Passed"
+                    elif proc.returncode == TEST_EXIT_SKIPPED:
+                        status = "Skipped"
+                    else:
+                        status = "Failed"
+                    self.num_running -= 1
+                    self.jobs.remove(j)
+
+                    return TestResult(name, status, int(time.time() - time0)), testdir, stdout, stderr
+            print('.', end='', flush=True)
+
+class TestResult():
+    def __init__(self, name, status, time):
+        self.name = name
+        self.status = status
+        self.time = time
+        self.padding = 0
+
+    def __repr__(self):
+        if self.status == "Passed":
+            color = BLUE
+            glyph = TICK
+        elif self.status == "Failed":
+            color = RED
+            glyph = CROSS
+        elif self.status == "Skipped":
+            color = GREY
+            glyph = CIRCLE
+
+        return color[1] + "%s | %s%s | %s s\n" % (self.name.ljust(self.padding), glyph, self.status.ljust(7), self.time) + color[0]
+
+    @property
+    def was_successful(self):
+        return self.status != "Failed"
+
+
+def check_script_prefixes():
+    """Check that at most a handful of the
+       test scripts don't start with one of the allowed name prefixes."""
+
+    # LEEWAY is provided as a transition measure, so that pull-requests
+    # that introduce new tests that don't conform with the naming
+    # convention don't immediately cause the tests to fail.
+    LEEWAY = 10
+
+    good_prefixes_re = re.compile("(example|feature|interface|mempool|mining|p2p|rpc|wallet|zerocoin)_")
+    bad_script_names = [script for script in ALL_SCRIPTS if good_prefixes_re.match(script) is None]
+
+    if len(bad_script_names) > 0:
+        print("INFO: %d tests not meeting naming conventions:" % (len(bad_script_names)))
+        print("  %s" % ("\n  ".join(sorted(bad_script_names))))
+    assert len(bad_script_names) <= LEEWAY, "Too many tests not following naming convention! (%d found, maximum: %d)" % (len(bad_script_names), LEEWAY)
+
+
+def check_script_list(src_dir):
+    """Check scripts directory.
+
+    Check that there are no scripts in the functional tests directory which are
+    not being run by pull-tester.py."""
+    script_dir = src_dir + '/test/functional/'
+    python_files = set([t for t in os.listdir(script_dir) if t[-3:] == ".py"])
+    missed_tests = list(python_files - set(map(lambda x: x.split()[0], ALL_SCRIPTS + NON_SCRIPTS)))
+    if len(missed_tests) != 0:
+        print("%sWARNING!%s The following scripts are not being run: %s. Check the test lists in test_runner.py." % (BOLD[1], BOLD[0], str(missed_tests)))
+        if os.getenv('TRAVIS') == 'true':
+            # On travis this warning is an error to prevent merging incomplete commits into master
+            sys.exit(1)
+
+class RPCCoverage():
+    """
+    Coverage reporting utilities for test_runner.
+
+    Coverage calculation works by having each test script subprocess write
+    coverage files into a particular directory. These files contain the RPC
+    commands invoked during testing, as well as a complete listing of RPC
+    commands per `agrarian-cli help` (`rpc_interface.txt`).
+
+    After all tests complete, the commands run are combined and diff'd against
+    the complete list to calculate uncovered RPC commands.
+
+    See also: test/functional/test_framework/coverage.py
+
+    """
+    def __init__(self):
+        self.dir = tempfile.mkdtemp(prefix="coverage")
+        self.flag = '--coveragedir=%s' % self.dir
+
+    def report_rpc_coverage(self):
+        """
+        Print out RPC commands that were unexercised by tests.
+
+        """
+        uncovered = self._get_uncovered_rpc_commands()
+
+        if uncovered:
+            print("Uncovered RPC commands:")
+            print("".join(("  - %s\n" % i) for i in sorted(uncovered)))
+        else:
+            print("All RPC commands covered.")
+
+    def cleanup(self):
+        return shutil.rmtree(self.dir)
+
+    def _get_uncovered_rpc_commands(self):
+        """
+        Return a set of currently untested RPC commands.
+
+        """
+        # This is shared from `test/functional/test-framework/coverage.py`
+        reference_filename = 'rpc_interface.txt'
+        coverage_file_prefix = 'coverage.'
+
+        coverage_ref_filename = os.path.join(self.dir, reference_filename)
+        coverage_filenames = set()
+        all_cmds = set()
+        covered_cmds = set()
+
+        if not os.path.isfile(coverage_ref_filename):
+            raise RuntimeError("No coverage reference found")
+
+        with open(coverage_ref_filename, 'r') as f:
+            all_cmds.update([i.strip() for i in f.readlines()])
+
+        for root, dirs, files in os.walk(self.dir):
+            for filename in files:
+                if filename.startswith(coverage_file_prefix):
+                    coverage_filenames.add(os.path.join(root, filename))
+
+        for filename in coverage_filenames:
+            with open(filename, 'r') as f:
+                covered_cmds.update([i.strip() for i in f.readlines()])
+
+        return all_cmds - covered_cmds
+
+
+if __name__ == '__main__':
+    main()

test/functional/wallet_abandonconflict.py

@@ -0,0 +1,160 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2016 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+import urllib.parse
+
+class AbandonConflictTest(BitcoinTestFramework):
+    def __init__(self):
+        super().__init__()
+        self.num_nodes = 2
+        self.setup_clean_chain = False
+
+    def setup_network(self):
+        self.nodes = []
+        self.nodes.append(start_node(0, self.options.tmpdir, ["-debug","-logtimemicros","-minrelaytxfee=0.00001"]))
+        self.nodes.append(start_node(1, self.options.tmpdir, ["-debug","-logtimemicros"]))
+        connect_nodes(self.nodes[0], 1)
+
+    def run_test(self):
+        self.nodes[1].generate(100)
+        sync_blocks(self.nodes)
+        balance = self.nodes[0].getbalance()
+        txA = self.nodes[0].sendtoaddress(self.nodes[0].getnewaddress(), Decimal("10"))
+        txB = self.nodes[0].sendtoaddress(self.nodes[0].getnewaddress(), Decimal("10"))
+        txC = self.nodes[0].sendtoaddress(self.nodes[0].getnewaddress(), Decimal("10"))
+        sync_mempools(self.nodes)
+        self.nodes[1].generate(1)
+
+        sync_blocks(self.nodes)
+        newbalance = self.nodes[0].getbalance()
+        assert(balance - newbalance < Decimal("0.001")) #no more than fees lost
+        balance = newbalance
+
+        url = urllib.parse.urlparse(self.nodes[1].url)
+        self.nodes[0].disconnectnode(url.hostname+":"+str(p2p_port(1)))
+
+        # Identify the 10btc outputs
+        nA = next(i for i, vout in enumerate(self.nodes[0].getrawtransaction(txA, 1)["vout"]) if vout["value"] == Decimal("10"))
+        nB = next(i for i, vout in enumerate(self.nodes[0].getrawtransaction(txB, 1)["vout"]) if vout["value"] == Decimal("10"))
+        nC = next(i for i, vout in enumerate(self.nodes[0].getrawtransaction(txC, 1)["vout"]) if vout["value"] == Decimal("10"))
+
+        inputs =[]
+        # spend 10btc outputs from txA and txB
+        inputs.append({"txid":txA, "vout":nA})
+        inputs.append({"txid":txB, "vout":nB})
+        outputs = {}
+
+        outputs[self.nodes[0].getnewaddress()] = Decimal("14.99998")
+        outputs[self.nodes[1].getnewaddress()] = Decimal("5")
+        signed = self.nodes[0].signrawtransaction(self.nodes[0].createrawtransaction(inputs, outputs))
+        txAB1 = self.nodes[0].sendrawtransaction(signed["hex"])
+
+        # Identify the 14.99998btc output
+        nAB = next(i for i, vout in enumerate(self.nodes[0].getrawtransaction(txAB1, 1)["vout"]) if vout["value"] == Decimal("14.99998"))
+
+        #Create a child tx spending AB1 and C
+        inputs = []
+        inputs.append({"txid":txAB1, "vout":nAB})
+        inputs.append({"txid":txC, "vout":nC})
+        outputs = {}
+        outputs[self.nodes[0].getnewaddress()] = Decimal("24.9996")
+        signed2 = self.nodes[0].signrawtransaction(self.nodes[0].createrawtransaction(inputs, outputs))
+        txABC2 = self.nodes[0].sendrawtransaction(signed2["hex"])
+
+        # In mempool txs from self should increase balance from change
+        newbalance = self.nodes[0].getbalance()
+        assert_equal(newbalance, balance - Decimal("30") + Decimal("24.9996"))
+        balance = newbalance
+
+        # Restart the node with a higher min relay fee so the parent tx is no longer in mempool
+        # TODO: redo with eviction
+        # Note had to make sure tx did not have AllowFree priority
+        stop_node(self.nodes[0],0)
+        self.nodes[0]=start_node(0, self.options.tmpdir, ["-debug","-logtimemicros","-minrelaytxfee=0.0001"])
+
+        # Verify txs no longer in mempool
+        assert_equal(len(self.nodes[0].getrawmempool()), 0)
+
+        # Not in mempool txs from self should only reduce balance
+        # inputs are still spent, but change not received
+        newbalance = self.nodes[0].getbalance()
+        assert_equal(newbalance, balance - Decimal("24.9996"))
+        # Unconfirmed received funds that are not in mempool, also shouldn't show
+        # up in unconfirmed balance
+        unconfbalance = self.nodes[0].getunconfirmedbalance() + self.nodes[0].getbalance()
+        assert_equal(unconfbalance, newbalance)
+        # Also shouldn't show up in listunspent
+        assert(not txABC2 in [utxo["txid"] for utxo in self.nodes[0].listunspent(0)])
+        balance = newbalance
+
+        # Abandon original transaction and verify inputs are available again
+        # including that the child tx was also abandoned
+        self.nodes[0].abandontransaction(txAB1)
+        newbalance = self.nodes[0].getbalance()
+        assert_equal(newbalance, balance + Decimal("30"))
+        balance = newbalance
+
+        # Verify that even with a low min relay fee, the tx is not reaccepted from wallet on startup once abandoned
+        stop_node(self.nodes[0],0)
+        self.nodes[0]=start_node(0, self.options.tmpdir, ["-debug","-logtimemicros","-minrelaytxfee=0.00001"])
+        assert_equal(len(self.nodes[0].getrawmempool()), 0)
+        assert_equal(self.nodes[0].getbalance(), balance)
+
+        # But if its received again then it is unabandoned
+        # And since now in mempool, the change is available
+        # But its child tx remains abandoned
+        self.nodes[0].sendrawtransaction(signed["hex"])
+        newbalance = self.nodes[0].getbalance()
+        assert_equal(newbalance, balance - Decimal("20") + Decimal("14.99998"))
+        balance = newbalance
+
+        # Send child tx again so its unabandoned
+        self.nodes[0].sendrawtransaction(signed2["hex"])
+        newbalance = self.nodes[0].getbalance()
+        assert_equal(newbalance, balance - Decimal("10") - Decimal("14.99998") + Decimal("24.9996"))
+        balance = newbalance
+
+        # Remove using high relay fee again
+        stop_node(self.nodes[0],0)
+        self.nodes[0]=start_node(0, self.options.tmpdir, ["-debug","-logtimemicros","-minrelaytxfee=0.0001"])
+        assert_equal(len(self.nodes[0].getrawmempool()), 0)
+        newbalance = self.nodes[0].getbalance()
+        assert_equal(newbalance, balance - Decimal("24.9996"))
+        balance = newbalance
+
+        # Create a double spend of AB1 by spending again from only A's 10 output
+        # Mine double spend from node 1
+        inputs =[]
+        inputs.append({"txid":txA, "vout":nA})
+        outputs = {}
+        outputs[self.nodes[1].getnewaddress()] = Decimal("9.9999")
+        tx = self.nodes[0].createrawtransaction(inputs, outputs)
+        signed = self.nodes[0].signrawtransaction(tx)
+        self.nodes[1].sendrawtransaction(signed["hex"])
+        self.nodes[1].generate(1)
+
+        connect_nodes(self.nodes[0], 1)
+        sync_blocks(self.nodes)
+
+        # Verify that B and C's 10 BTC outputs are available for spending again because AB1 is now conflicted
+        newbalance = self.nodes[0].getbalance()
+        assert_equal(newbalance, balance + Decimal("20"))
+        balance = newbalance
+
+        # There is currently a minor bug around this and so this test doesn't work.  See Issue #7315
+        # Invalidate the block with the double spend and B's 10 BTC output should no longer be available
+        # Don't think C's should either
+        self.nodes[0].invalidateblock(self.nodes[0].getbestblockhash())
+        newbalance = self.nodes[0].getbalance()
+        #assert_equal(newbalance, balance - Decimal("10"))
+        print("If balance has not declined after invalidateblock then out of mempool wallet tx which is no longer")
+        print("conflicted has not resumed causing its inputs to be seen as spent.  See Issue #7315")
+        print(str(balance) + " -> " + str(newbalance) + " ?")
+
+if __name__ == '__main__':
+    AbandonConflictTest().main()

test/functional/wallet_accounts.py

@@ -0,0 +1,202 @@
+#!/usr/bin/env python3
+# Copyright (c) 2016-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test account RPCs.
+
+RPCs tested are:
+    - getaccountaddress
+    - getaddressesbyaccount
+    - listaddressgroupings
+    - setaccount
+    - sendfrom (with account arguments)
+    - move (with account arguments)
+"""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import assert_equal
+
+class WalletAccountsTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+        self.extra_args = [[]]
+
+    def run_test(self):
+        node = self.nodes[0]
+        # Check that there's no UTXO on any of the nodes
+        assert_equal(len(node.listunspent()), 0)
+
+        # Note each time we call generate, all generated coins go into
+        # the same address, so we call twice to get two addresses w/50 each
+        node.generate(1)
+        node.generate(101)
+        assert_equal(node.getbalance(), 500)
+
+        # there should be 2 address groups
+        # each with 1 address with a balance of 50 Bitcoins
+        address_groups = node.listaddressgroupings()
+        assert_equal(len(address_groups), 1)
+        # the addresses aren't linked now, but will be after we send to the
+        # common address
+        linked_addresses = set()
+        #for address_group in address_groups:
+        #    assert_equal(len(address_group), 1)
+        #    assert_equal(len(address_group[0]), 2)
+        #    assert_equal(address_group[0][1], 250)
+        #    linked_addresses.add(address_group[0][0])
+
+        # send 50 from each address to a third address not in this wallet
+        # There's some fee that will come back to us when the miner reward
+        # matures.
+        node.settxfee(0)
+        common_address = "y9B3dwrBGGs3yVkyEHm68Yn36Wp2Rt7Vtd"
+        txid = node.sendmany("", {common_address: 100}, 1)
+        tx_details = node.gettransaction(txid)
+        fee = -tx_details['details'][0]['fee']
+        # there should be 1 address group, with the previously
+        # unlinked addresses now linked (they both have 0 balance)
+        #address_groups = node.listaddressgroupings()
+        #assert_equal(len(address_groups), 1)
+        #assert_equal(len(address_groups[0]), 1)
+        #assert_equal(set([a[0] for a in address_groups[0]]), linked_addresses)
+        #assert_equal([a[1] for a in address_groups[0]], [0, 0])
+
+        node.generate(1)
+
+        # we want to reset so that the "" account has what's expected.
+        # otherwise we're off by exactly the fee amount as that's mined
+        # and matures in the next 100 blocks
+        node.sendfrom("", common_address, float(fee))
+        amount_to_send = 5.0
+
+        # Create accounts and make sure subsequent account API calls
+        # recognize the account/address associations.
+        accounts = [Account(name) for name in ("a", "b", "c", "d", "e")]
+        for account in accounts:
+            account.add_receive_address(node.getaccountaddress(account.name))
+            account.verify(node)
+
+        # Send a transaction to each account, and make sure this forces
+        # getaccountaddress to generate a new receiving address.
+        for account in accounts:
+            node.sendtoaddress(account.receive_address, amount_to_send)
+            account.add_receive_address(node.getaccountaddress(account.name))
+            account.verify(node)
+
+        # Check the amounts received.
+        node.generate(1)
+        for account in accounts:
+            assert_equal(
+                node.getreceivedbyaddress(account.addresses[0]), amount_to_send)
+            assert_equal(node.getreceivedbyaccount(account.name), amount_to_send)
+
+        # Check that sendfrom account reduces listaccounts balances.
+        for i, account in enumerate(accounts):
+            to_account = accounts[(i+1) % len(accounts)]
+            node.sendfrom(account.name, to_account.receive_address, amount_to_send)
+        node.generate(1)
+        for account in accounts:
+            account.add_receive_address(node.getaccountaddress(account.name))
+            account.verify(node)
+            assert_equal(node.getreceivedbyaccount(account.name), 10)
+            node.move(account.name, "", float(node.getbalance(account.name)))
+            account.verify(node)
+        node.generate(101)
+        #expected_account_balances = {"": 26149.99985650}
+        #for account in accounts:
+        #    expected_account_balances[account.name] = 0
+        #assert_equal(node.listaccounts(), expected_account_balances)
+        #assert_equal(node.getbalance(""), 26149.99985650)
+
+        # Check that setaccount can assign an account to a new unused address.
+        for account in accounts:
+            address = node.getaccountaddress("")
+            node.setaccount(address, account.name)
+            account.add_address(address)
+            account.verify(node)
+            assert(address not in node.getaddressesbyaccount(""))
+
+        # Check that addmultisigaddress can assign accounts.
+        for account in accounts:
+            addresses = []
+            for x in range(10):
+                addresses.append(node.getnewaddress())
+            multisig_address = node.addmultisigaddress(5, addresses, account.name)
+            account.add_address(multisig_address)
+            account.verify(node)
+            node.sendfrom("", multisig_address, 50)
+        #node.generate(101)
+        #for account in accounts:
+        #    assert_equal(node.getbalance(account.name), 50)
+
+        # Check that setaccount can change the account of an address from a
+        # different account.
+        change_account(node, accounts[0].addresses[0], accounts[0], accounts[1])
+
+        # Check that setaccount can change the account of an address which
+        # is the receiving address of a different account.
+        change_account(node, accounts[0].receive_address, accounts[0], accounts[1])
+
+        # Check that setaccount can set the account of an address already
+        # in the account. This is a no-op.
+        change_account(node, accounts[2].addresses[0], accounts[2], accounts[2])
+
+        # Check that setaccount can set the account of an address which is
+        # already the receiving address of the account. It would probably make
+        # sense for this to be a no-op, but right now it resets the receiving
+        # address, causing getaccountaddress to return a brand new address.
+        change_account(node, accounts[2].receive_address, accounts[2], accounts[2])
+
+class Account:
+    def __init__(self, name):
+        # Account name
+        self.name = name
+        # Current receiving address associated with this account.
+        self.receive_address = None
+        # List of all addresses assigned with this account
+        self.addresses = []
+
+    def add_address(self, address):
+        assert_equal(address not in self.addresses, True)
+        self.addresses.append(address)
+
+    def add_receive_address(self, address):
+        self.add_address(address)
+        self.receive_address = address
+
+    def verify(self, node):
+        if self.receive_address is not None:
+            assert self.receive_address in self.addresses
+            assert_equal(node.getaccountaddress(self.name), self.receive_address)
+
+        for address in self.addresses:
+            assert_equal(node.getaccount(address), self.name)
+
+        assert_equal(
+            set(node.getaddressesbyaccount(self.name)), set(self.addresses))
+
+
+def change_account(node, address, old_account, new_account):
+    assert_equal(address in old_account.addresses, True)
+    node.setaccount(address, new_account.name)
+
+    old_account.addresses.remove(address)
+    new_account.add_address(address)
+
+    # Calling setaccount on an address which was previously the receiving
+    # address of a different account should reset the receiving address of
+    # the old account, causing getaccountaddress to return a brand new
+    # address.
+    if address == old_account.receive_address:
+        new_address = node.getaccountaddress(old_account.name)
+        assert_equal(new_address not in old_account.addresses, True)
+        assert_equal(new_address not in new_account.addresses, True)
+        old_account.add_receive_address(new_address)
+
+    old_account.verify(node)
+    new_account.verify(node)
+
+
+if __name__ == '__main__':
+    WalletAccountsTest().main()

test/functional/wallet_backup.py

@@ -0,0 +1,205 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the wallet backup features.
+
+Test case is:
+4 nodes. 1 2 and 3 send transactions between each other,
+fourth node is a miner.
+1 2 3 each mine a block to start, then
+Miner creates 100 blocks so 1 2 3 each have 50 mature
+coins to spend.
+Then 5 iterations of 1/2/3 sending coins amongst
+themselves to get transactions in the wallets,
+and the miner mining one block.
+
+Wallets are backed up using dumpwallet/backupwallet.
+Then 5 more iterations of transactions and mining a block.
+
+Miner then generates 101 more blocks, so any
+transaction fees paid mature.
+
+Sanity check:
+  Sum(1,2,3,4 balances) == 114*50
+
+1/2/3 are shutdown, and their wallets erased.
+Then restore using wallet.dat backup. And
+confirm 1/2/3/4 balances are same as before.
+
+Shutdown again, restore using importwallet,
+and confirm again balances are correct.
+"""
+from random import randint
+import shutil
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+class WalletBackupTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 4
+        self.setup_clean_chain = True
+        # nodes 1, 2,3 are spenders, let's give them a keypool=100
+        self.extra_args = [["-keypool=100"], ["-keypool=100"], ["-keypool=100"], []]
+
+    def setup_network(self, split=False):
+        self.setup_nodes()
+        connect_nodes(self.nodes[0], 3)
+        connect_nodes(self.nodes[1], 3)
+        connect_nodes(self.nodes[2], 3)
+        connect_nodes(self.nodes[2], 0)
+        self.sync_all()
+
+    def one_send(self, from_node, to_address):
+        if (randint(1,2) == 1):
+            amount = Decimal(randint(1,10)) / Decimal(10)
+            self.nodes[from_node].sendtoaddress(to_address, float(amount))
+
+    def do_one_round(self):
+        a0 = self.nodes[0].getnewaddress()
+        a1 = self.nodes[1].getnewaddress()
+        a2 = self.nodes[2].getnewaddress()
+
+        self.one_send(0, a1)
+        self.one_send(0, a2)
+        self.one_send(1, a0)
+        self.one_send(1, a2)
+        self.one_send(2, a0)
+        self.one_send(2, a1)
+
+        # Have the miner (node3) mine a block.
+        # Must sync mempools before mining.
+        sync_mempools(self.nodes)
+        self.nodes[3].generate(1)
+        sync_blocks(self.nodes)
+
+    # As above, this mirrors the original bash test.
+    def start_three(self):
+        self.start_node(0)
+        self.start_node(1)
+        self.start_node(2)
+        connect_nodes(self.nodes[0], 3)
+        connect_nodes(self.nodes[1], 3)
+        connect_nodes(self.nodes[2], 3)
+        connect_nodes(self.nodes[2], 0)
+
+    def stop_three(self):
+        self.stop_node(0)
+        self.stop_node(1)
+        self.stop_node(2)
+
+    def erase_three(self):
+        os.remove(self.options.tmpdir + "/node0/regtest/wallet.dat")
+        os.remove(self.options.tmpdir + "/node1/regtest/wallet.dat")
+        os.remove(self.options.tmpdir + "/node2/regtest/wallet.dat")
+
+    def run_test(self):
+        self.log.info("Generating initial blockchain")
+        self.nodes[0].generate(1)
+        sync_blocks(self.nodes)
+        self.nodes[1].generate(1)
+        sync_blocks(self.nodes)
+        self.nodes[2].generate(1)
+        sync_blocks(self.nodes)
+        self.nodes[3].generate(100)
+        sync_blocks(self.nodes)
+
+        assert_equal(self.nodes[0].getbalance(), 250)
+        assert_equal(self.nodes[1].getbalance(), 250)
+        assert_equal(self.nodes[2].getbalance(), 250)
+        assert_equal(self.nodes[3].getbalance(), 0)
+
+        self.log.info("Creating transactions")
+        # Five rounds of sending each other transactions.
+        for i in range(5):
+            self.do_one_round()
+
+        self.log.info("Backing up")
+        tmpdir = self.options.tmpdir
+        self.nodes[0].backupwallet(tmpdir + "/node0/wallet.bak")
+        self.nodes[0].dumpwallet(tmpdir + "/node0/wallet.dump")
+        self.nodes[1].backupwallet(tmpdir + "/node1/wallet.bak")
+        self.nodes[1].dumpwallet(tmpdir + "/node1/wallet.dump")
+        self.nodes[2].backupwallet(tmpdir + "/node2/wallet.bak")
+        self.nodes[2].dumpwallet(tmpdir + "/node2/wallet.dump")
+
+        self.log.info("More transactions")
+        for i in range(5):
+            self.do_one_round()
+
+        # Generate 101 more blocks, so any fees paid mature
+        self.nodes[3].generate(101)
+        self.sync_all()
+
+        balance0 = self.nodes[0].getbalance()
+        balance1 = self.nodes[1].getbalance()
+        balance2 = self.nodes[2].getbalance()
+        balance3 = self.nodes[3].getbalance()
+        total = balance0 + balance1 + balance2 + balance3
+
+        # At this point, there are 214 blocks (103 for setup, then 10 rounds, then 101.)
+        # 114 are mature, so the sum of all wallets should be 114 * 250 = 28500.
+        assert_equal(total, 28500)
+
+        ##
+        # Test restoring spender wallets from backups
+        ##
+        self.log.info("Restoring using wallet.dat")
+        self.stop_three()
+        self.erase_three()
+
+        # Start node2 with no chain
+        shutil.rmtree(self.options.tmpdir + "/node2/regtest/blocks")
+        shutil.rmtree(self.options.tmpdir + "/node2/regtest/chainstate")
+
+        # Restore wallets from backup
+        shutil.copyfile(tmpdir + "/node0/wallet.bak", tmpdir + "/node0/regtest/wallet.dat")
+        shutil.copyfile(tmpdir + "/node1/wallet.bak", tmpdir + "/node1/regtest/wallet.dat")
+        shutil.copyfile(tmpdir + "/node2/wallet.bak", tmpdir + "/node2/regtest/wallet.dat")
+
+        self.log.info("Re-starting nodes")
+        self.start_three()
+        sync_blocks(self.nodes)
+
+        assert_equal(self.nodes[0].getbalance(), balance0)
+        assert_equal(self.nodes[1].getbalance(), balance1)
+        assert_equal(self.nodes[2].getbalance(), balance2)
+
+        self.log.info("Restoring using dumped wallet")
+        self.stop_three()
+        self.erase_three()
+
+        #start node2 with no chain
+        shutil.rmtree(self.options.tmpdir + "/node2/regtest/blocks")
+        shutil.rmtree(self.options.tmpdir + "/node2/regtest/chainstate")
+
+        self.start_three()
+
+        assert_equal(self.nodes[0].getbalance(), 0)
+        assert_equal(self.nodes[1].getbalance(), 0)
+        assert_equal(self.nodes[2].getbalance(), 0)
+
+        self.nodes[0].importwallet(tmpdir + "/node0/wallet.dump")
+        self.nodes[1].importwallet(tmpdir + "/node1/wallet.dump")
+        self.nodes[2].importwallet(tmpdir + "/node2/wallet.dump")
+
+        sync_blocks(self.nodes)
+
+        assert_equal(self.nodes[0].getbalance(), balance0)
+        assert_equal(self.nodes[1].getbalance(), balance1)
+        assert_equal(self.nodes[2].getbalance(), balance2)
+
+        # Backup to source wallet file must fail
+        sourcePaths = [
+            tmpdir + "/node0/regtest/wallet.dat",
+            tmpdir + "/node0/./regtest/wallet.dat",
+            tmpdir + "/node0/regtest/",
+            tmpdir + "/node0/regtest"]
+
+        for sourcePath in sourcePaths:
+            assert_raises_rpc_error(-4, "backup failed", self.nodes[0].backupwallet, sourcePath)
+
+
+if __name__ == '__main__':
+    WalletBackupTest().main()

test/functional/wallet_basic.py

@@ -0,0 +1,319 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the wallet."""
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+class WalletTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 4
+        self.setup_clean_chain = True
+
+    def setup_network(self):
+        self.add_nodes(4)
+        self.start_node(0)
+        self.start_node(1)
+        self.start_node(2)
+        connect_nodes_bi(self.nodes,0,1)
+        connect_nodes_bi(self.nodes,1,2)
+        connect_nodes_bi(self.nodes,0,2)
+        self.sync_all([self.nodes[0:3]])
+
+    def check_fee_amount(self, curr_balance, balance_with_fee, fee_per_byte, tx_size):
+        """Return curr_balance after asserting the fee was in range"""
+        fee = balance_with_fee - curr_balance
+        fee2 = round(tx_size * fee_per_byte / 1000, 8)
+        self.log.info("current: %s, withfee: %s, perByte: %s, size: %s, fee: %s" % (str(curr_balance), str(balance_with_fee), str(fee_per_byte), str(tx_size), str(fee2)))
+        assert_fee_amount(fee, tx_size, fee_per_byte * 1000)
+        return curr_balance
+
+    def get_vsize(self, txn):
+        return self.nodes[0].decoderawtransaction(txn)['size']
+
+    def run_test(self):
+        # Check that there's no UTXO on none of the nodes
+        assert_equal(len(self.nodes[0].listunspent()), 0)
+        assert_equal(len(self.nodes[1].listunspent()), 0)
+        assert_equal(len(self.nodes[2].listunspent()), 0)
+
+        self.log.info("Mining blocks...")
+
+        self.nodes[0].generate(1)
+
+        walletinfo = self.nodes[0].getwalletinfo()
+        assert_equal(walletinfo['immature_balance'], 250)
+        assert_equal(walletinfo['balance'], 0)
+
+        self.sync_all([self.nodes[0:3]])
+        self.nodes[1].generate(101)
+        self.sync_all([self.nodes[0:3]])
+
+        assert_equal(self.nodes[0].getbalance(), 250)
+        assert_equal(self.nodes[1].getbalance(), 250)
+        assert_equal(self.nodes[2].getbalance(), 0)
+
+        # Check that only first and second nodes have UTXOs
+        utxos = self.nodes[0].listunspent()
+        assert_equal(len(utxos), 1)
+        assert_equal(len(self.nodes[1].listunspent()), 1)
+        assert_equal(len(self.nodes[2].listunspent()), 0)
+
+        # Send 21 BTC from 0 to 2 using sendtoaddress call.
+        # Second transaction will be child of first, and will require a fee
+        self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), 21)
+        #self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), 10)
+
+        walletinfo = self.nodes[0].getwalletinfo()
+        assert_equal(walletinfo['immature_balance'], 0)
+
+        # Have node0 mine a block, thus it will collect its own fee.
+        self.nodes[0].generate(1)
+        self.sync_all([self.nodes[0:3]])
+
+        # Exercise locking of unspent outputs
+        unspent_0 = self.nodes[2].listunspent()[0]
+        unspent_0 = {"txid": unspent_0["txid"], "vout": unspent_0["vout"]}
+        self.nodes[2].lockunspent(False, [unspent_0])
+        assert_raises_rpc_error(-4, "Insufficient funds", self.nodes[2].sendtoaddress, self.nodes[2].getnewaddress(), 20)
+        assert_equal([unspent_0], self.nodes[2].listlockunspent())
+        self.nodes[2].lockunspent(True, [unspent_0])
+        assert_equal(len(self.nodes[2].listlockunspent()), 0)
+
+        # Have node1 generate 100 blocks (so node0 can recover the fee)
+        self.nodes[1].generate(100)
+        self.sync_all([self.nodes[0:3]])
+
+        # node0 should end up with 100 btc in block rewards plus fees, but
+        # minus the 21 plus fees sent to node2
+        assert_equal(self.nodes[0].getbalance(), 500-21)
+        assert_equal(self.nodes[2].getbalance(), 21)
+
+        # Node0 should have two unspent outputs.
+        # Create a couple of transactions to send them to node2, submit them through
+        # node1, and make sure both node0 and node2 pick them up properly:
+        node0utxos = self.nodes[0].listunspent(1)
+        assert_equal(len(node0utxos), 2)
+
+        # create both transactions
+        txns_to_send = []
+        for utxo in node0utxos:
+            inputs = []
+            outputs = {}
+            inputs.append({ "txid" : utxo["txid"], "vout" : utxo["vout"]})
+            outputs[self.nodes[2].getnewaddress("from1")] = float(utxo["amount"])
+            raw_tx = self.nodes[0].createrawtransaction(inputs, outputs)
+            txns_to_send.append(self.nodes[0].signrawtransaction(raw_tx))
+
+        # Have node 1 (miner) send the transactions
+        self.nodes[1].sendrawtransaction(txns_to_send[0]["hex"], True)
+        self.nodes[1].sendrawtransaction(txns_to_send[1]["hex"], True)
+
+        # Have node1 mine a block to confirm transactions:
+        self.nodes[1].generate(1)
+        self.sync_all([self.nodes[0:3]])
+
+        assert_equal(self.nodes[0].getbalance(), 0)
+        assert_equal(self.nodes[2].getbalance(), 500)
+        assert_equal(self.nodes[2].getbalance("from1"), 500-21)
+
+        # Send 10 BTC normal
+        address = self.nodes[0].getnewaddress("test")
+        fee_per_byte = Decimal('0.001') / 1000
+        self.nodes[2].settxfee(float(fee_per_byte * 1000))
+        txid = self.nodes[2].sendtoaddress(address, 10, "", "")
+        fee = self.nodes[2].gettransaction(txid)["fee"]
+        self.nodes[2].generate(1)
+        self.sync_all([self.nodes[0:3]])
+        node_2_bal = self.nodes[2].getbalance()
+        #node_2_bal = self.check_fee_amount(balance, Decimal(balance - fee), fee_per_byte, self.get_vsize(self.nodes[2].getrawtransaction(txid)))
+        assert_equal(self.nodes[0].getbalance(), Decimal('10'))
+
+        # Send 10 BTC with subtract fee from amount
+        txid = self.nodes[2].sendtoaddress(address, 10, "", "")
+        self.nodes[2].generate(1)
+        self.sync_all([self.nodes[0:3]])
+        node_2_bal -= Decimal('10')
+        assert_equal(self.nodes[2].getbalance() - fee, node_2_bal)
+        node_0_bal = self.nodes[0].getbalance()
+        assert_equal(node_0_bal, Decimal('20'))
+
+        # Sendmany 10 BTC
+        txid = self.nodes[2].sendmany('from1', {address: 10}, 0, "")
+        self.nodes[2].generate(1)
+        self.sync_all([self.nodes[0:3]])
+        node_0_bal += Decimal('10')
+        node_2_bal -= Decimal('10')
+        #node_2_bal = self.check_fee_amount(self.nodes[2].getbalance(), node_2_bal - Decimal('10'), fee_per_byte, self.get_vsize(self.nodes[2].getrawtransaction(txid)))
+        assert_equal(self.nodes[0].getbalance(), node_0_bal)
+
+        # Sendmany 10 BTC with subtract fee from amount
+        txid = self.nodes[2].sendmany('from1', {address: 10}, 0, "")
+        self.nodes[2].generate(1)
+        self.sync_all([self.nodes[0:3]])
+        node_2_bal -= Decimal('10')
+        assert_equal(self.nodes[2].getbalance(), node_2_bal + (fee * 3))
+        #node_0_bal = self.check_fee_amount(self.nodes[0].getbalance(), node_0_bal + Decimal('10'), fee_per_byte, self.get_vsize(self.nodes[2].getrawtransaction(txid)))
+
+        # Test ResendWalletTransactions:
+        # Create a couple of transactions, then start up a fourth
+        # node (nodes[3]) and ask nodes[0] to rebroadcast.
+        # EXPECT: nodes[3] should have those transactions in its mempool.
+        txid1 = self.nodes[0].sendtoaddress(self.nodes[1].getnewaddress(), 1)
+        txid2 = self.nodes[1].sendtoaddress(self.nodes[0].getnewaddress(), 1)
+        sync_mempools(self.nodes[0:2])
+
+        self.start_node(3)
+        connect_nodes_bi(self.nodes, 0, 3)
+        sync_blocks(self.nodes)
+
+        #relayed = self.nodes[0].resendwallettransactions()
+        #assert_equal(set(relayed), {txid1, txid2})
+        #sync_mempools(self.nodes)
+
+        #assert(txid1 in self.nodes[3].getrawmempool())
+
+        # Exercise balance rpcs
+        assert_equal(self.nodes[0].getwalletinfo()["unconfirmed_balance"], 1)
+        assert_equal(self.nodes[0].getunconfirmedbalance(), 1)
+
+        #check if we can list zero value tx as available coins
+        #1. create rawtx
+        #2. hex-changed one output to 0.0
+        #3. sign and send
+        #4. check if recipient (node0) can list the zero value tx
+        usp = self.nodes[1].listunspent()
+        inputs = [{"txid":usp[0]['txid'], "vout":usp[0]['vout']}]
+        outputs = {self.nodes[1].getnewaddress(): 49.998, self.nodes[0].getnewaddress(): 11.11}
+
+        rawTx = self.nodes[1].createrawtransaction(inputs, outputs).replace("c0833842", "00000000") #replace 11.11 with 0.0 (int32)
+        decRawTx = self.nodes[1].decoderawtransaction(rawTx)
+        signedRawTx = self.nodes[1].signrawtransaction(rawTx)
+        decRawTx = self.nodes[1].decoderawtransaction(signedRawTx['hex'])
+        zeroValueTxid= decRawTx['txid']
+        assert_raises_rpc_error(-25, "", self.nodes[1].sendrawtransaction, signedRawTx['hex'])
+
+        self.sync_all([self.nodes[0:3]])
+        self.nodes[1].generate(1) #mine a block
+        self.sync_all([self.nodes[0:3]])
+
+        #unspentTxs = self.nodes[0].listunspent() #zero value tx must be in listunspents output
+        #found = False
+        #for uTx in unspentTxs:
+        #    if uTx['txid'] == zeroValueTxid:
+        #        found = True
+        #        assert_equal(uTx['amount'], Decimal('0'))
+        #assert(found)
+
+        #do some -walletbroadcast tests
+        self.stop_nodes()
+        self.start_node(0, ["-walletbroadcast=0"])
+        self.start_node(1, ["-walletbroadcast=0"])
+        self.start_node(2, ["-walletbroadcast=0"])
+        connect_nodes_bi(self.nodes,0,1)
+        connect_nodes_bi(self.nodes,1,2)
+        connect_nodes_bi(self.nodes,0,2)
+        self.sync_all([self.nodes[0:3]])
+
+        txIdNotBroadcasted  = self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), 2)
+        txObjNotBroadcasted = self.nodes[0].gettransaction(txIdNotBroadcasted)
+        self.nodes[1].generate(1) #mine a block, tx should not be in there
+        self.sync_all([self.nodes[0:3]])
+        assert_equal(self.nodes[2].getbalance(), node_2_bal + (fee * 3)) #should not be changed because tx was not broadcasted
+
+        #now broadcast from another node, mine a block, sync, and check the balance
+        self.nodes[1].sendrawtransaction(txObjNotBroadcasted['hex'])
+        self.nodes[1].generate(1)
+        self.sync_all([self.nodes[0:3]])
+        node_2_bal += 2
+        txObjNotBroadcasted = self.nodes[0].gettransaction(txIdNotBroadcasted)
+        assert_equal(self.nodes[2].getbalance(), node_2_bal + (fee * 3))
+
+        #create another tx
+        txIdNotBroadcasted  = self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), 2)
+
+        #restart the nodes with -walletbroadcast=1
+        self.stop_nodes()
+        self.start_node(0)
+        self.start_node(1)
+        self.start_node(2)
+        connect_nodes_bi(self.nodes,0,1)
+        connect_nodes_bi(self.nodes,1,2)
+        connect_nodes_bi(self.nodes,0,2)
+        sync_blocks(self.nodes[0:3])
+
+        self.nodes[0].generate(1)
+        sync_blocks(self.nodes[0:3])
+        node_2_bal += 2
+
+        #tx should be added to balance because after restarting the nodes tx should be broadcastet
+        assert_equal(self.nodes[2].getbalance(), node_2_bal + (fee * 3))
+
+        # This will raise an exception since generate does not accept a string
+        assert_raises_rpc_error(-1, "not an integer", self.nodes[0].generate, "2")
+
+        # Import address and private key to check correct behavior of spendable unspents
+        # 1. Send some coins to generate new UTXO
+        address_to_import = self.nodes[2].getnewaddress()
+        txid = self.nodes[0].sendtoaddress(address_to_import, 1)
+        self.nodes[0].generate(1)
+        self.sync_all([self.nodes[0:3]])
+
+        # 2. Import address from node2 to node1
+        self.nodes[1].importaddress(address_to_import)
+
+        # 3. Validate that the imported address is watch-only on node1
+        assert(self.nodes[1].validateaddress(address_to_import)["iswatchonly"])
+
+        # 4. Check that the unspents after import are not spendable
+        listunspent = self.nodes[1].listunspent(1, 9999999, [], 3)
+        assert_array_result(listunspent,
+                           {"address": address_to_import},
+                           {"spendable": False})
+
+        # 5. Import private key of the previously imported address on node1
+        priv_key = self.nodes[2].dumpprivkey(address_to_import)
+        self.nodes[1].importprivkey(priv_key)
+
+        # 6. Check that the unspents are now spendable on node1
+        assert_array_result(self.nodes[1].listunspent(),
+                           {"address": address_to_import},
+                           {"spendable": True})
+
+        #check if wallet or blochchain maintenance changes the balance
+        self.sync_all([self.nodes[0:3]])
+        blocks = self.nodes[0].generate(2)
+        self.sync_all([self.nodes[0:3]])
+        balance_nodes = [self.nodes[i].getbalance() for i in range(3)]
+        block_count = self.nodes[0].getblockcount()
+
+        maintenance = [
+            '-rescan',
+            '-reindex',
+            '-zapwallettxes=1',
+            '-zapwallettxes=2',
+            #'-salvagewallet',
+        ]
+        chainlimit = 6
+        for m in maintenance:
+            self.log.info("check " + m)
+            self.stop_nodes()
+            # set lower ancestor limit for later
+            self.start_node(0, [m, "-limitancestorcount="+str(chainlimit)])
+            self.start_node(1, [m, "-limitancestorcount="+str(chainlimit)])
+            self.start_node(2, [m, "-limitancestorcount="+str(chainlimit)])
+            if m == '-reindex':
+                # reindex will leave rpc warm up "early"; Wait for it to finish
+                wait_until(lambda: [block_count] * 3 == [self.nodes[i].getblockcount() for i in range(3)])
+            assert_equal(balance_nodes, [self.nodes[i].getbalance() for i in range(3)])
+
+        # Exercise listsinceblock with the last two blocks
+        coinbase_tx_1 = self.nodes[0].listsinceblock(blocks[0])
+        assert_equal(coinbase_tx_1["lastblock"], blocks[1])
+        assert_equal(len(coinbase_tx_1["transactions"]), 1)
+        assert_equal(coinbase_tx_1["transactions"][0]["blockhash"], blocks[1])
+        assert_equal(len(self.nodes[0].listsinceblock(blocks[1])["transactions"]), 0)
+
+if __name__ == '__main__':
+    WalletTest().main()

test/functional/wallet_bumpfee.py

@@ -0,0 +1,300 @@
+#!/usr/bin/env python3
+# Copyright (c) 2016-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the bumpfee RPC.
+
+Verifies that the bumpfee RPC creates replacement transactions successfully when
+its preconditions are met, and returns appropriate errors in other cases.
+
+This module consists of around a dozen individual test cases implemented in the
+top-level functions named as test_<test_case_description>. The test functions
+can be disabled or reordered if needed for debugging. If new test cases are
+added in the future, they should try to follow the same convention and not
+make assumptions about execution order.
+"""
+
+from test_framework.blocktools import send_to_witness
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework import blocktools
+from test_framework.mininode import CTransaction
+from test_framework.util import *
+
+import io
+
+# Sequence number that is BIP 125 opt-in and BIP 68-compliant
+BIP125_SEQUENCE_NUMBER = 0xfffffffd
+
+WALLET_PASSPHRASE = "test"
+WALLET_PASSPHRASE_TIMEOUT = 3600
+
+
+class BumpFeeTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+        self.setup_clean_chain = True
+        self.extra_args = [["-prematurewitness", "-walletprematurewitness", "-deprecatedrpc=addwitnessaddress", "-walletrbf={}".format(i)]
+                           for i in range(self.num_nodes)]
+
+    def run_test(self):
+        # Encrypt wallet for test_locked_wallet_fails test
+        self.nodes[1].node_encrypt_wallet(WALLET_PASSPHRASE)
+        self.start_node(1)
+        self.nodes[1].walletpassphrase(WALLET_PASSPHRASE, WALLET_PASSPHRASE_TIMEOUT)
+
+        connect_nodes_bi(self.nodes, 0, 1)
+        self.sync_all()
+
+        peer_node, rbf_node = self.nodes
+        rbf_node_address = rbf_node.getnewaddress()
+
+        # fund rbf node with 10 coins of 0.001 btc (100,000 satoshis)
+        self.log.info("Mining blocks...")
+        peer_node.generate(110)
+        self.sync_all()
+        for i in range(25):
+            peer_node.sendtoaddress(rbf_node_address, 0.001)
+        self.sync_all()
+        peer_node.generate(1)
+        self.sync_all()
+        assert_equal(rbf_node.getbalance(), Decimal("0.025"))
+
+        self.log.info("Running tests")
+        dest_address = peer_node.getnewaddress()
+        test_simple_bumpfee_succeeds(rbf_node, peer_node, dest_address)
+        test_segwit_bumpfee_succeeds(rbf_node, dest_address)
+        test_nonrbf_bumpfee_fails(peer_node, dest_address)
+        test_notmine_bumpfee_fails(rbf_node, peer_node, dest_address)
+        test_bumpfee_with_descendant_fails(rbf_node, rbf_node_address, dest_address)
+        test_small_output_fails(rbf_node, dest_address)
+        test_dust_to_fee(rbf_node, dest_address)
+        test_settxfee(rbf_node, dest_address)
+        test_rebumping(rbf_node, dest_address)
+        test_rebumping_not_replaceable(rbf_node, dest_address)
+        test_unconfirmed_not_spendable(rbf_node, rbf_node_address)
+        test_bumpfee_metadata(rbf_node, dest_address)
+        test_locked_wallet_fails(rbf_node, dest_address)
+        self.log.info("Success")
+
+
+def test_simple_bumpfee_succeeds(rbf_node, peer_node, dest_address):
+    rbfid = spend_one_input(rbf_node, dest_address)
+    rbftx = rbf_node.gettransaction(rbfid)
+    sync_mempools((rbf_node, peer_node))
+    assert rbfid in rbf_node.getrawmempool() and rbfid in peer_node.getrawmempool()
+    bumped_tx = rbf_node.bumpfee(rbfid)
+    assert_equal(bumped_tx["errors"], [])
+    assert bumped_tx["fee"] - abs(rbftx["fee"]) > 0
+    # check that bumped_tx propagates, original tx was evicted and has a wallet conflict
+    sync_mempools((rbf_node, peer_node))
+    assert bumped_tx["txid"] in rbf_node.getrawmempool()
+    assert bumped_tx["txid"] in peer_node.getrawmempool()
+    assert rbfid not in rbf_node.getrawmempool()
+    assert rbfid not in peer_node.getrawmempool()
+    oldwtx = rbf_node.gettransaction(rbfid)
+    assert len(oldwtx["walletconflicts"]) > 0
+    # check wallet transaction replaces and replaced_by values
+    bumpedwtx = rbf_node.gettransaction(bumped_tx["txid"])
+    assert_equal(oldwtx["replaced_by_txid"], bumped_tx["txid"])
+    assert_equal(bumpedwtx["replaces_txid"], rbfid)
+
+
+def test_segwit_bumpfee_succeeds(rbf_node, dest_address):
+    # Create a transaction with segwit output, then create an RBF transaction
+    # which spends it, and make sure bumpfee can be called on it.
+
+    segwit_in = next(u for u in rbf_node.listunspent() if u["amount"] == Decimal("0.001"))
+    segwit_out = rbf_node.validateaddress(rbf_node.getnewaddress())
+    rbf_node.addwitnessaddress(segwit_out["address"])
+    segwitid = send_to_witness(
+        use_p2wsh=False,
+        node=rbf_node,
+        utxo=segwit_in,
+        pubkey=segwit_out["pubkey"],
+        encode_p2sh=False,
+        amount=Decimal("0.0009"),
+        sign=True)
+
+    rbfraw = rbf_node.createrawtransaction([{
+        'txid': segwitid,
+        'vout': 0,
+        "sequence": BIP125_SEQUENCE_NUMBER
+    }], {dest_address: Decimal("0.0005"),
+         rbf_node.getrawchangeaddress(): Decimal("0.0003")})
+    rbfsigned = rbf_node.signrawtransaction(rbfraw)
+    rbfid = rbf_node.sendrawtransaction(rbfsigned["hex"])
+    assert rbfid in rbf_node.getrawmempool()
+
+    bumped_tx = rbf_node.bumpfee(rbfid)
+    assert bumped_tx["txid"] in rbf_node.getrawmempool()
+    assert rbfid not in rbf_node.getrawmempool()
+
+
+def test_nonrbf_bumpfee_fails(peer_node, dest_address):
+    # cannot replace a non RBF transaction (from node which did not enable RBF)
+    not_rbfid = peer_node.sendtoaddress(dest_address, Decimal("0.00090000"))
+    assert_raises_rpc_error(-4, "not BIP 125 replaceable", peer_node.bumpfee, not_rbfid)
+
+
+def test_notmine_bumpfee_fails(rbf_node, peer_node, dest_address):
+    # cannot bump fee unless the tx has only inputs that we own.
+    # here, the rbftx has a peer_node coin and then adds a rbf_node input
+    # Note that this test depends upon the RPC code checking input ownership prior to change outputs
+    # (since it can't use fundrawtransaction, it lacks a proper change output)
+    utxos = [node.listunspent()[-1] for node in (rbf_node, peer_node)]
+    inputs = [{
+        "txid": utxo["txid"],
+        "vout": utxo["vout"],
+        "address": utxo["address"],
+        "sequence": BIP125_SEQUENCE_NUMBER
+    } for utxo in utxos]
+    output_val = sum(utxo["amount"] for utxo in utxos) - Decimal("0.001")
+    rawtx = rbf_node.createrawtransaction(inputs, {dest_address: output_val})
+    signedtx = rbf_node.signrawtransaction(rawtx)
+    signedtx = peer_node.signrawtransaction(signedtx["hex"])
+    rbfid = rbf_node.sendrawtransaction(signedtx["hex"])
+    assert_raises_rpc_error(-4, "Transaction contains inputs that don't belong to this wallet",
+                          rbf_node.bumpfee, rbfid)
+
+
+def test_bumpfee_with_descendant_fails(rbf_node, rbf_node_address, dest_address):
+    # cannot bump fee if the transaction has a descendant
+    # parent is send-to-self, so we don't have to check which output is change when creating the child tx
+    parent_id = spend_one_input(rbf_node, rbf_node_address)
+    tx = rbf_node.createrawtransaction([{"txid": parent_id, "vout": 0}], {dest_address: 0.00020000})
+    tx = rbf_node.signrawtransaction(tx)
+    rbf_node.sendrawtransaction(tx["hex"])
+    assert_raises_rpc_error(-8, "Transaction has descendants in the wallet", rbf_node.bumpfee, parent_id)
+
+
+def test_small_output_fails(rbf_node, dest_address):
+    # cannot bump fee with a too-small output
+    rbfid = spend_one_input(rbf_node, dest_address)
+    rbf_node.bumpfee(rbfid, {"totalFee": 50000})
+
+    rbfid = spend_one_input(rbf_node, dest_address)
+    assert_raises_rpc_error(-4, "Change output is too small", rbf_node.bumpfee, rbfid, {"totalFee": 50001})
+
+
+def test_dust_to_fee(rbf_node, dest_address):
+    # check that if output is reduced to dust, it will be converted to fee
+    # the bumped tx sets fee=49,900, but it converts to 50,000
+    rbfid = spend_one_input(rbf_node, dest_address)
+    fulltx = rbf_node.getrawtransaction(rbfid, 1)
+    bumped_tx = rbf_node.bumpfee(rbfid, {"totalFee": 49900})
+    full_bumped_tx = rbf_node.getrawtransaction(bumped_tx["txid"], 1)
+    assert_equal(bumped_tx["fee"], Decimal("0.00050000"))
+    assert_equal(len(fulltx["vout"]), 2)
+    assert_equal(len(full_bumped_tx["vout"]), 1)  #change output is eliminated
+
+
+def test_settxfee(rbf_node, dest_address):
+    # check that bumpfee reacts correctly to the use of settxfee (paytxfee)
+    rbfid = spend_one_input(rbf_node, dest_address)
+    requested_feerate = Decimal("0.00025000")
+    rbf_node.settxfee(requested_feerate)
+    bumped_tx = rbf_node.bumpfee(rbfid)
+    actual_feerate = bumped_tx["fee"] * 1000 / rbf_node.getrawtransaction(bumped_tx["txid"], True)["vsize"]
+    # Assert that the difference between the requested feerate and the actual
+    # feerate of the bumped transaction is small.
+    assert_greater_than(Decimal("0.00001000"), abs(requested_feerate - actual_feerate))
+    rbf_node.settxfee(Decimal("0.00000000"))  # unset paytxfee
+
+
+def test_rebumping(rbf_node, dest_address):
+    # check that re-bumping the original tx fails, but bumping the bumper succeeds
+    rbfid = spend_one_input(rbf_node, dest_address)
+    bumped = rbf_node.bumpfee(rbfid, {"totalFee": 2000})
+    assert_raises_rpc_error(-4, "already bumped", rbf_node.bumpfee, rbfid, {"totalFee": 3000})
+    rbf_node.bumpfee(bumped["txid"], {"totalFee": 3000})
+
+
+def test_rebumping_not_replaceable(rbf_node, dest_address):
+    # check that re-bumping a non-replaceable bump tx fails
+    rbfid = spend_one_input(rbf_node, dest_address)
+    bumped = rbf_node.bumpfee(rbfid, {"totalFee": 10000, "replaceable": False})
+    assert_raises_rpc_error(-4, "Transaction is not BIP 125 replaceable", rbf_node.bumpfee, bumped["txid"],
+                          {"totalFee": 20000})
+
+
+def test_unconfirmed_not_spendable(rbf_node, rbf_node_address):
+    # check that unconfirmed outputs from bumped transactions are not spendable
+    rbfid = spend_one_input(rbf_node, rbf_node_address)
+    rbftx = rbf_node.gettransaction(rbfid)["hex"]
+    assert rbfid in rbf_node.getrawmempool()
+    bumpid = rbf_node.bumpfee(rbfid)["txid"]
+    assert bumpid in rbf_node.getrawmempool()
+    assert rbfid not in rbf_node.getrawmempool()
+
+    # check that outputs from the bump transaction are not spendable
+    # due to the replaces_txid check in CWallet::AvailableCoins
+    assert_equal([t for t in rbf_node.listunspent(minconf=0, include_unsafe=False) if t["txid"] == bumpid], [])
+
+    # submit a block with the rbf tx to clear the bump tx out of the mempool,
+    # then call abandon to make sure the wallet doesn't attempt to resubmit the
+    # bump tx, then invalidate the block so the rbf tx will be put back in the
+    # mempool. this makes it possible to check whether the rbf tx outputs are
+    # spendable before the rbf tx is confirmed.
+    block = submit_block_with_tx(rbf_node, rbftx)
+    rbf_node.abandontransaction(bumpid)
+    rbf_node.invalidateblock(block.hash)
+    assert bumpid not in rbf_node.getrawmempool()
+    assert rbfid in rbf_node.getrawmempool()
+
+    # check that outputs from the rbf tx are not spendable before the
+    # transaction is confirmed, due to the replaced_by_txid check in
+    # CWallet::AvailableCoins
+    assert_equal([t for t in rbf_node.listunspent(minconf=0, include_unsafe=False) if t["txid"] == rbfid], [])
+
+    # check that the main output from the rbf tx is spendable after confirmed
+    rbf_node.generate(1)
+    assert_equal(
+        sum(1 for t in rbf_node.listunspent(minconf=0, include_unsafe=False)
+            if t["txid"] == rbfid and t["address"] == rbf_node_address and t["spendable"]), 1)
+
+
+def test_bumpfee_metadata(rbf_node, dest_address):
+    rbfid = rbf_node.sendtoaddress(dest_address, Decimal("0.00100000"), "comment value", "to value")
+    bumped_tx = rbf_node.bumpfee(rbfid)
+    bumped_wtx = rbf_node.gettransaction(bumped_tx["txid"])
+    assert_equal(bumped_wtx["comment"], "comment value")
+    assert_equal(bumped_wtx["to"], "to value")
+
+
+def test_locked_wallet_fails(rbf_node, dest_address):
+    rbfid = spend_one_input(rbf_node, dest_address)
+    rbf_node.walletlock()
+    assert_raises_rpc_error(-13, "Please enter the wallet passphrase with walletpassphrase first.",
+                          rbf_node.bumpfee, rbfid)
+
+
+def spend_one_input(node, dest_address):
+    tx_input = dict(
+        sequence=BIP125_SEQUENCE_NUMBER, **next(u for u in node.listunspent() if u["amount"] == Decimal("0.00100000")))
+    rawtx = node.createrawtransaction(
+        [tx_input], {dest_address: Decimal("0.00050000"),
+                     node.getrawchangeaddress(): Decimal("0.00049000")})
+    signedtx = node.signrawtransaction(rawtx)
+    txid = node.sendrawtransaction(signedtx["hex"])
+    return txid
+
+
+def submit_block_with_tx(node, tx):
+    ctx = CTransaction()
+    ctx.deserialize(io.BytesIO(hex_str_to_bytes(tx)))
+
+    tip = node.getbestblockhash()
+    height = node.getblockcount() + 1
+    block_time = node.getblockheader(tip)["mediantime"] + 1
+    block = blocktools.create_block(int(tip, 16), blocktools.create_coinbase(height), block_time)
+    block.vtx.append(ctx)
+    block.rehash()
+    block.hashMerkleRoot = block.calc_merkle_root()
+    blocktools.add_witness_commitment(block)
+    block.solve()
+    node.submitblock(bytes_to_hex_str(block.serialize(True)))
+    return block
+
+
+if __name__ == "__main__":
+    BumpFeeTest().main()

test/functional/wallet_disable.py

@@ -0,0 +1,29 @@
+#!/usr/bin/env python3
+# Copyright (c) 2015-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test a node with the -disablewallet option.
+
+- Test that validateaddress RPC works when running with -disablewallet
+- Test that it is not possible to mine to an invalid address.
+"""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+class DisableWalletTest (BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+        self.extra_args = [["-disablewallet"]]
+
+    def run_test (self):
+        # Make sure wallet is really disabled
+        assert_raises_rpc_error(-32601, 'Method not found', self.nodes[0].getwalletinfo)
+        x = self.nodes[0].validateaddress('3J98t1WpEZ73CNmQviecrnyiWrnqRhWNLy')
+        assert(x['isvalid'] == False)
+        x = self.nodes[0].validateaddress('xwMWGTnBNUmGxMm8vfAdbL45bWXyVTYctd')
+        assert(x['isvalid'] == True)
+
+if __name__ == '__main__':
+    DisableWalletTest ().main ()

test/functional/wallet_dump.py

@@ -0,0 +1,93 @@
+#!/usr/bin/env python3
+# Copyright (c) 2016-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the dumpwallet RPC."""
+
+import os
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (assert_equal, assert_raises_rpc_error)
+
+
+def read_dump(file_name, addrs, hd_master_addr_old):
+    """
+    Read the given dump, count the addrs that match, count change and reserve.
+    Also check that the old hd_master is inactive
+    """
+    with open(file_name, encoding='utf8') as inputfile:
+        found_addr = 0
+        found_addr_chg = 0
+        found_addr_rsv = 0
+        hd_master_addr_ret = None
+        for line in inputfile:
+            # only read non comment lines
+            if line[0] != "#" and len(line) > 10:
+                # split out some data
+                key_label, comment = line.split("#")
+                # key = key_label.split(" ")[0]
+                keytype = key_label.split(" ")[2]
+                addr = comment.split(" addr=")[1].strip()
+
+                # count key types
+                if addr in addrs:
+                    found_addr += 1
+                elif keytype == "change=1":
+                    found_addr_chg += 1
+                elif keytype == "reserve=1":
+                    found_addr_rsv += 1
+        return found_addr, found_addr_chg, found_addr_rsv, hd_master_addr_ret
+
+
+class WalletDumpTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 1
+        self.extra_args = [["-keypool=90"]]
+
+    def setup_network(self, split=False):
+        # Use 1 minute timeout because the initial getnewaddress RPC can take
+        # longer than the default 30 seconds due to an expensive
+        # CWallet::TopUpKeyPool call, and the encryptwallet RPC made later in
+        # the test often takes even longer.
+        self.add_nodes(self.num_nodes, self.extra_args, timewait=60)
+        self.start_nodes()
+
+    def run_test (self):
+        tmpdir = self.options.tmpdir
+
+        # generate 20 addresses to compare against the dump
+        test_addr_count = 20
+        addrs = []
+        for i in range(0,test_addr_count):
+            addr = self.nodes[0].getnewaddress()
+            #vaddr= self.nodes[0].validateaddress(addr) #required to get hd keypath
+            addrs.append(addr)
+        # Should be a no-op:
+        self.nodes[0].keypoolrefill()
+
+        # dump unencrypted wallet
+        result = self.nodes[0].dumpwallet(tmpdir + "/node0/wallet.unencrypted.dump")
+        assert_equal(result['filename'], os.path.abspath(tmpdir + "/node0/wallet.unencrypted.dump"))
+
+        found_addr, found_addr_chg, found_addr_rsv, hd_master_addr_unenc = \
+            read_dump(tmpdir + "/node0/wallet.unencrypted.dump", addrs, None)
+        assert_equal(found_addr, test_addr_count)  # all keys must be in the dump
+        assert_equal(found_addr_chg, 0)  # 0 blocks where mined
+        assert_equal(found_addr_rsv, 90 + 1)  # keypool size (TODO: fix off-by-one)
+
+        #encrypt wallet, restart, unlock and dump
+        self.nodes[0].node_encrypt_wallet('test')
+        self.start_node(0)
+        self.nodes[0].walletpassphrase('test', 10)
+        # Should be a no-op:
+        self.nodes[0].keypoolrefill()
+        self.nodes[0].dumpwallet(tmpdir + "/node0/wallet.encrypted.dump")
+
+        found_addr, found_addr_chg, found_addr_rsv, hd_master_addr_enc = \
+            read_dump(tmpdir + "/node0/wallet.encrypted.dump", addrs, hd_master_addr_unenc)
+        assert_equal(found_addr, test_addr_count)
+        assert_equal(found_addr_chg, 90 + 1)  # old reserve keys are marked as change now
+        assert_equal(found_addr_rsv, 90 + 1)  # keypool size (TODO: fix off-by-one)
+
+if __name__ == '__main__':
+    WalletDumpTest().main ()

test/functional/wallet_encryption.py

@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+# Copyright (c) 2016-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test Wallet encryption"""
+
+import time
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (
+    assert_equal,
+    assert_raises_rpc_error,
+    assert_greater_than,
+    assert_greater_than_or_equal,
+)
+
+class WalletEncryptionTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 1
+
+    def run_test(self):
+        passphrase = "WalletPassphrase"
+        passphrase2 = "SecondWalletPassphrase"
+
+        # Make sure the wallet isn't encrypted first
+        address = self.nodes[0].getnewaddress()
+        privkey = self.nodes[0].dumpprivkey(address)
+        assert_equal(privkey[:1], "c")
+        assert_equal(len(privkey), 52)
+
+        # Encrypt the wallet
+        self.nodes[0].node_encrypt_wallet(passphrase)
+        self.start_node(0)
+
+        # Test that the wallet is encrypted
+        assert_raises_rpc_error(-13, "Please enter the wallet passphrase with walletpassphrase first", self.nodes[0].dumpprivkey, address)
+
+        # Check that walletpassphrase works
+        self.nodes[0].walletpassphrase(passphrase, 2)
+        assert_equal(privkey, self.nodes[0].dumpprivkey(address))
+
+        # Check that the timeout is right
+        time.sleep(2)
+        assert_raises_rpc_error(-13, "Please enter the wallet passphrase with walletpassphrase first", self.nodes[0].dumpprivkey, address)
+
+        # Test wrong passphrase
+        assert_raises_rpc_error(-14, "wallet passphrase entered was incorrect", self.nodes[0].walletpassphrase, passphrase + "wrong", 10)
+
+        # Test walletlock
+        self.nodes[0].walletpassphrase(passphrase, 84600)
+        assert_equal(privkey, self.nodes[0].dumpprivkey(address))
+        self.nodes[0].walletlock()
+        assert_raises_rpc_error(-13, "Please enter the wallet passphrase with walletpassphrase first", self.nodes[0].dumpprivkey, address)
+
+        # Test wallet already unlocked
+        self.nodes[0].walletpassphrase(passphrase, 12000, True)
+        assert_raises_rpc_error(-17, "Wallet is already unlocked", self.nodes[0].walletpassphrase, passphrase, 100, True)
+        self.nodes[0].walletlock()
+
+        # Test passphrase changes
+        self.nodes[0].walletpassphrasechange(passphrase, passphrase2)
+        assert_raises_rpc_error(-14, "wallet passphrase entered was incorrect", self.nodes[0].walletpassphrase, passphrase, 10)
+        self.nodes[0].walletpassphrase(passphrase2, 10)
+        assert_equal(privkey, self.nodes[0].dumpprivkey(address))
+        self.nodes[0].walletlock()
+
+        # Test timeout bounds
+        assert_raises_rpc_error(-8, "Timeout cannot be negative.", self.nodes[0].walletpassphrase, passphrase2, -10)
+        # Check the timeout
+        # Check a time less than the limit
+        MAX_VALUE = 100000000
+        expected_time = int(time.time()) + MAX_VALUE - 600
+        self.nodes[0].walletpassphrase(passphrase2, MAX_VALUE - 600)
+        actual_time = self.nodes[0].getwalletinfo()['unlocked_until']
+        assert_greater_than_or_equal(actual_time, expected_time)
+        assert_greater_than(expected_time + 5, actual_time) # 5 second buffer
+        # Check a time greater than the limit
+        expected_time = int(time.time()) + MAX_VALUE - 1
+        self.nodes[0].walletpassphrase(passphrase2, MAX_VALUE + 1000)
+        actual_time = self.nodes[0].getwalletinfo()['unlocked_until']
+        assert_greater_than_or_equal(actual_time, expected_time)
+        assert_greater_than(expected_time + 5, actual_time) # 5 second buffer
+
+if __name__ == '__main__':
+    WalletEncryptionTest().main()

test/functional/wallet_import_rescan.py

@@ -0,0 +1,187 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test wallet import RPCs.
+
+Test rescan behavior of importaddress, importpubkey, importprivkey, and
+importmulti RPCs with different types of keys and rescan options.
+
+In the first part of the test, node 0 creates an address for each type of
+import RPC call and sends BTC to it. Then other nodes import the addresses,
+and the test makes listtransactions and getbalance calls to confirm that the
+importing node either did or did not execute rescans picking up the send
+transactions.
+
+In the second part of the test, node 0 sends more BTC to each address, and the
+test makes more listtransactions and getbalance calls to confirm that the
+importing nodes pick up the new transactions regardless of whether rescans
+happened previously.
+"""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (assert_raises_rpc_error, connect_nodes, sync_blocks, assert_equal, set_node_times)
+
+import collections
+import enum
+import itertools
+
+Call = enum.Enum("Call", "single multi")
+Data = enum.Enum("Data", "address pub priv")
+Rescan = enum.Enum("Rescan", "no yes late_timestamp")
+
+
+class Variant(collections.namedtuple("Variant", "call data rescan prune")):
+    """Helper for importing one key and verifying scanned transactions."""
+
+    def try_rpc(self, func, *args, **kwargs):
+        if self.expect_disabled:
+            assert_raises_rpc_error(-4, "Rescan is disabled in pruned mode", func, *args, **kwargs)
+        else:
+            return func(*args, **kwargs)
+
+    def do_import(self, timestamp):
+        """Call one key import RPC."""
+
+        if self.call == Call.single:
+            if self.data == Data.address:
+                response = self.try_rpc(self.node.importaddress, self.address["address"], self.label,
+                                               self.rescan == Rescan.yes)
+            elif self.data == Data.pub:
+                response = self.try_rpc(self.node.importpubkey, self.address["pubkey"], self.label,
+                                               self.rescan == Rescan.yes)
+            elif self.data == Data.priv:
+                response = self.try_rpc(self.node.importprivkey, self.key, self.label, self.rescan == Rescan.yes)
+            assert_equal(response, None)
+
+        elif self.call == Call.multi:
+            response = self.node.importmulti([{
+                "scriptPubKey": {
+                    "address": self.address["address"]
+                },
+                "timestamp": timestamp + TIMESTAMP_WINDOW + (1 if self.rescan == Rescan.late_timestamp else 0),
+                "pubkeys": [self.address["pubkey"]] if self.data == Data.pub else [],
+                "keys": [self.key] if self.data == Data.priv else [],
+                "label": self.label,
+                "watchonly": self.data != Data.priv
+            }], {"rescan": self.rescan in (Rescan.yes, Rescan.late_timestamp)})
+            assert_equal(response, [{"success": True}])
+
+    def check(self, txid=None, amount=None, confirmations=None):
+        """Verify that getbalance/listtransactions return expected values."""
+
+        balance = self.node.getbalance(self.label, 0, True)
+        assert_equal(balance, self.expected_balance)
+
+        txs = self.node.listtransactions(self.label, 10000, 0, True)
+        assert_equal(len(txs), self.expected_txs)
+
+        if txid is not None:
+            tx, = [tx for tx in txs if tx["txid"] == txid]
+            assert_equal(tx["account"], self.label)
+            assert_equal(tx["address"], self.address["address"])
+            assert_equal(tx["amount"], amount)
+            assert_equal(tx["category"], "receive")
+            assert_equal(tx["label"], self.label)
+            assert_equal(tx["txid"], txid)
+            assert_equal(tx["confirmations"], confirmations)
+            assert_equal("trusted" not in tx, True)
+            # Verify the transaction is correctly marked watchonly depending on
+            # whether the transaction pays to an imported public key or
+            # imported private key. The test setup ensures that transaction
+            # inputs will not be from watchonly keys (important because
+            # involvesWatchonly will be true if either the transaction output
+            # or inputs are watchonly).
+            if self.data != Data.priv:
+                assert_equal(tx["involvesWatchonly"], True)
+            else:
+                assert_equal("involvesWatchonly" not in tx, True)
+
+
+# List of Variants for each way a key or address could be imported.
+IMPORT_VARIANTS = [Variant(*variants) for variants in itertools.product(Call, Data, Rescan, (False, True))]
+
+# List of nodes to import keys to. Half the nodes will have pruning disabled,
+# half will have it enabled. Different nodes will be used for imports that are
+# expected to cause rescans, and imports that are not expected to cause
+# rescans, in order to prevent rescans during later imports picking up
+# transactions associated with earlier imports. This makes it easier to keep
+# track of expected balances and transactions.
+ImportNode = collections.namedtuple("ImportNode", "prune rescan")
+IMPORT_NODES = [ImportNode(*fields) for fields in itertools.product((False, True), repeat=2)]
+
+# Rescans start at the earliest block up to 2 hours before the key timestamp.
+TIMESTAMP_WINDOW = 2 * 60 * 60
+
+
+class ImportRescanTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2 + len(IMPORT_NODES)
+
+    def setup_network(self):
+        extra_args = [["-addresstype=legacy"] for _ in range(self.num_nodes)]
+        for i, import_node in enumerate(IMPORT_NODES, 2):
+            if import_node.prune:
+                extra_args[i] += ["-prune=1"]
+
+        self.add_nodes(self.num_nodes, extra_args)
+        self.start_nodes()
+        for i in range(1, self.num_nodes):
+            connect_nodes(self.nodes[i], 0)
+
+    def run_test(self):
+        # Create one transaction on node 0 with a unique amount and label for
+        # each possible type of wallet import RPC.
+        for i, variant in enumerate(IMPORT_VARIANTS):
+            variant.label = "label {} {}".format(i, variant)
+            variant.address = self.nodes[1].validateaddress(self.nodes[1].getnewaddress(variant.label))
+            variant.key = self.nodes[1].dumpprivkey(variant.address["address"])
+            variant.initial_amount = 10 - (i + 1) / 4.0
+            variant.initial_txid = self.nodes[0].sendtoaddress(variant.address["address"], variant.initial_amount)
+
+        # Generate a block containing the initial transactions, then another
+        # block further in the future (past the rescan window).
+        self.nodes[0].generate(1)
+        assert_equal(self.nodes[0].getrawmempool(), [])
+        timestamp = self.nodes[0].getblockheader(self.nodes[0].getbestblockhash())["time"]
+        set_node_times(self.nodes, timestamp + TIMESTAMP_WINDOW + 1)
+        self.nodes[0].generate(1)
+        sync_blocks(self.nodes)
+
+        # For each variation of wallet key import, invoke the import RPC and
+        # check the results from getbalance and listtransactions.
+        for variant in IMPORT_VARIANTS:
+            variant.expect_disabled = variant.rescan == Rescan.yes and variant.prune and variant.call == Call.single
+            expect_rescan = variant.rescan == Rescan.yes and not variant.expect_disabled
+            variant.node = self.nodes[2 + IMPORT_NODES.index(ImportNode(variant.prune, expect_rescan))]
+            variant.do_import(timestamp)
+            if expect_rescan:
+                variant.expected_balance = variant.initial_amount
+                variant.expected_txs = 1
+                variant.check(variant.initial_txid, variant.initial_amount, 2)
+            else:
+                variant.expected_balance = 0
+                variant.expected_txs = 0
+                variant.check()
+
+        # Create new transactions sending to each address.
+        for i, variant in enumerate(IMPORT_VARIANTS):
+            variant.sent_amount = 10 - (2 * i + 1) / 8.0
+            variant.sent_txid = self.nodes[0].sendtoaddress(variant.address["address"], variant.sent_amount)
+
+        # Generate a block containing the new transactions.
+        self.nodes[0].generate(1)
+        assert_equal(self.nodes[0].getrawmempool(), [])
+        sync_blocks(self.nodes)
+
+        # Check the latest results from getbalance and listtransactions.
+        for variant in IMPORT_VARIANTS:
+            if not variant.expect_disabled:
+                variant.expected_balance += variant.sent_amount
+                variant.expected_txs += 1
+                variant.check(variant.sent_txid, variant.sent_amount, 1)
+            else:
+                variant.check()
+
+if __name__ == "__main__":
+    ImportRescanTest().main()

test/functional/wallet_importmulti.py

@@ -0,0 +1,451 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the importmulti RPC."""
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+class ImportMultiTest (BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+        self.extra_args = [["-addresstype=legacy"], ["-addresstype=legacy"]]
+        self.setup_clean_chain = True
+
+    def setup_network(self):
+        self.setup_nodes()
+
+    def run_test (self):
+        self.log.info("Mining blocks...")
+        self.nodes[0].generate(1)
+        self.nodes[1].generate(1)
+        timestamp = self.nodes[1].getblock(self.nodes[1].getbestblockhash())['mediantime']
+
+        node0_address1 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+
+        #Check only one address
+        assert_equal(node0_address1['ismine'], True)
+
+        #Node 1 sync test
+        assert_equal(self.nodes[1].getblockcount(),1)
+
+        #Address Test - before import
+        address_info = self.nodes[1].validateaddress(node0_address1['address'])
+        assert_equal(address_info['iswatchonly'], False)
+        assert_equal(address_info['ismine'], False)
+
+
+        # RPC importmulti -----------------------------------------------
+
+        # Bitcoin Address
+        self.log.info("Should import an address")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": {
+                "address": address['address']
+            },
+            "timestamp": "now",
+        }])
+        assert_equal(result[0]['success'], True)
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], True)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal(address_assert['timestamp'], timestamp)
+        watchonly_address = address['address']
+        watchonly_timestamp = timestamp
+
+        self.log.info("Should not import an invalid address")
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": {
+                "address": "not valid address",
+            },
+            "timestamp": "now",
+        }])
+        assert_equal(result[0]['success'], False)
+        assert_equal(result[0]['error']['code'], -5)
+        assert_equal(result[0]['error']['message'], 'Invalid address')
+
+        # ScriptPubKey + internal
+        self.log.info("Should import a scriptPubKey with internal flag")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": address['scriptPubKey'],
+            "timestamp": "now",
+            "internal": True
+        }])
+        assert_equal(result[0]['success'], True)
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], True)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal(address_assert['timestamp'], timestamp)
+
+        # ScriptPubKey + !internal
+        self.log.info("Should not import a scriptPubKey without internal flag")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": address['scriptPubKey'],
+            "timestamp": "now",
+        }])
+        assert_equal(result[0]['success'], False)
+        assert_equal(result[0]['error']['code'], -8)
+        assert_equal(result[0]['error']['message'], 'Internal must be set for hex scriptPubKey')
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], False)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal('timestamp' in address_assert, False)
+
+
+        # Address + Public key + !Internal
+        self.log.info("Should import an address with public key")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": {
+                "address": address['address']
+            },
+            "timestamp": "now",
+            "pubkeys": [ address['pubkey'] ]
+        }])
+        assert_equal(result[0]['success'], True)
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], True)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal(address_assert['timestamp'], timestamp)
+
+
+        # ScriptPubKey + Public key + internal
+        self.log.info("Should import a scriptPubKey with internal and with public key")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        request = [{
+            "scriptPubKey": address['scriptPubKey'],
+            "timestamp": "now",
+            "pubkeys": [ address['pubkey'] ],
+            "internal": True
+        }]
+        result = self.nodes[1].importmulti(request)
+        assert_equal(result[0]['success'], True)
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], True)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal(address_assert['timestamp'], timestamp)
+
+        # ScriptPubKey + Public key + !internal
+        self.log.info("Should not import a scriptPubKey without internal and with public key")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        request = [{
+            "scriptPubKey": address['scriptPubKey'],
+            "timestamp": "now",
+            "pubkeys": [ address['pubkey'] ]
+        }]
+        result = self.nodes[1].importmulti(request)
+        assert_equal(result[0]['success'], False)
+        assert_equal(result[0]['error']['code'], -8)
+        assert_equal(result[0]['error']['message'], 'Internal must be set for hex scriptPubKey')
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], False)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal('timestamp' in address_assert, False)
+
+        # Address + Private key + !watchonly
+        self.log.info("Should import an address with private key")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": {
+                "address": address['address']
+            },
+            "timestamp": "now",
+            "keys": [ self.nodes[0].dumpprivkey(address['address']) ]
+        }])
+        assert_equal(result[0]['success'], True)
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], False)
+        assert_equal(address_assert['ismine'], True)
+        assert_equal(address_assert['timestamp'], timestamp)
+
+        self.log.info("Should not import an address with private key if is already imported")
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": {
+                "address": address['address']
+            },
+            "timestamp": "now",
+            "keys": [ self.nodes[0].dumpprivkey(address['address']) ]
+        }])
+        assert_equal(result[0]['success'], False)
+        assert_equal(result[0]['error']['code'], -4)
+        assert_equal(result[0]['error']['message'], 'The wallet already contains the private key for this address or script')
+
+        # Address + Private key + watchonly
+        self.log.info("Should not import an address with private key and with watchonly")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": {
+                "address": address['address']
+            },
+            "timestamp": "now",
+            "keys": [ self.nodes[0].dumpprivkey(address['address']) ],
+            "watchonly": True
+        }])
+        assert_equal(result[0]['success'], False)
+        assert_equal(result[0]['error']['code'], -8)
+        assert_equal(result[0]['error']['message'], 'Incompatibility found between watchonly and keys')
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], False)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal('timestamp' in address_assert, False)
+
+        # ScriptPubKey + Private key + internal
+        self.log.info("Should import a scriptPubKey with internal and with private key")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": address['scriptPubKey'],
+            "timestamp": "now",
+            "keys": [ self.nodes[0].dumpprivkey(address['address']) ],
+            "internal": True
+        }])
+        assert_equal(result[0]['success'], True)
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], False)
+        assert_equal(address_assert['ismine'], True)
+        assert_equal(address_assert['timestamp'], timestamp)
+
+        # ScriptPubKey + Private key + !internal
+        self.log.info("Should not import a scriptPubKey without internal and with private key")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": address['scriptPubKey'],
+            "timestamp": "now",
+            "keys": [ self.nodes[0].dumpprivkey(address['address']) ]
+        }])
+        assert_equal(result[0]['success'], False)
+        assert_equal(result[0]['error']['code'], -8)
+        assert_equal(result[0]['error']['message'], 'Internal must be set for hex scriptPubKey')
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], False)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal('timestamp' in address_assert, False)
+
+
+        # P2SH address
+        sig_address_1 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        sig_address_2 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        sig_address_3 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        multi_sig_script = self.nodes[0].createmultisig(2, [sig_address_1['pubkey'], sig_address_2['pubkey'], sig_address_3['pubkey']])
+        self.nodes[1].generate(100)
+        transactionid = self.nodes[1].sendtoaddress(multi_sig_script['address'], 10.00)
+        self.nodes[1].generate(1)
+        timestamp = self.nodes[1].getblock(self.nodes[1].getbestblockhash())['mediantime']
+
+        self.log.info("Should import a p2sh")
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": {
+                "address": multi_sig_script['address']
+            },
+            "timestamp": "now",
+        }])
+        assert_equal(result[0]['success'], True)
+        address_assert = self.nodes[1].validateaddress(multi_sig_script['address'])
+        assert_equal(address_assert['isscript'], True)
+        assert_equal(address_assert['iswatchonly'], True)
+        assert_equal(address_assert['timestamp'], timestamp)
+        p2shunspent = self.nodes[1].listunspent(0,999999, [multi_sig_script['address']])[0]
+        assert_equal(p2shunspent['spendable'], False)
+        assert_equal(p2shunspent['solvable'], False)
+
+
+        # P2SH + Redeem script
+        sig_address_1 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        sig_address_2 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        sig_address_3 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        multi_sig_script = self.nodes[0].createmultisig(2, [sig_address_1['pubkey'], sig_address_2['pubkey'], sig_address_3['pubkey']])
+        self.nodes[1].generate(100)
+        transactionid = self.nodes[1].sendtoaddress(multi_sig_script['address'], 10.00)
+        self.nodes[1].generate(1)
+        timestamp = self.nodes[1].getblock(self.nodes[1].getbestblockhash())['mediantime']
+
+        self.log.info("Should import a p2sh with respective redeem script")
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": {
+                "address": multi_sig_script['address']
+            },
+            "timestamp": "now",
+            "redeemscript": multi_sig_script['redeemScript']
+        }])
+        assert_equal(result[0]['success'], True)
+        address_assert = self.nodes[1].validateaddress(multi_sig_script['address'])
+        assert_equal(address_assert['timestamp'], timestamp)
+
+        p2shunspent = self.nodes[1].listunspent(0,999999, [multi_sig_script['address']])[0]
+        assert_equal(p2shunspent['spendable'], False)
+        assert_equal(p2shunspent['solvable'], True)
+
+
+        # P2SH + Redeem script + Private Keys + !Watchonly
+        sig_address_1 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        sig_address_2 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        sig_address_3 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        multi_sig_script = self.nodes[0].createmultisig(2, [sig_address_1['pubkey'], sig_address_2['pubkey'], sig_address_3['pubkey']])
+        self.nodes[1].generate(100)
+        transactionid = self.nodes[1].sendtoaddress(multi_sig_script['address'], 10.00)
+        self.nodes[1].generate(1)
+        timestamp = self.nodes[1].getblock(self.nodes[1].getbestblockhash())['mediantime']
+
+        self.log.info("Should import a p2sh with respective redeem script and private keys")
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": {
+                "address": multi_sig_script['address']
+            },
+            "timestamp": "now",
+            "redeemscript": multi_sig_script['redeemScript'],
+            "keys": [ self.nodes[0].dumpprivkey(sig_address_1['address']), self.nodes[0].dumpprivkey(sig_address_2['address'])]
+        }])
+        assert_equal(result[0]['success'], True)
+        address_assert = self.nodes[1].validateaddress(multi_sig_script['address'])
+        assert_equal(address_assert['timestamp'], timestamp)
+
+        p2shunspent = self.nodes[1].listunspent(0,999999, [multi_sig_script['address']])[0]
+        assert_equal(p2shunspent['spendable'], False)
+        assert_equal(p2shunspent['solvable'], True)
+
+        # P2SH + Redeem script + Private Keys + Watchonly
+        sig_address_1 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        sig_address_2 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        sig_address_3 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        multi_sig_script = self.nodes[0].createmultisig(2, [sig_address_1['pubkey'], sig_address_2['pubkey'], sig_address_3['pubkey']])
+        self.nodes[1].generate(100)
+        transactionid = self.nodes[1].sendtoaddress(multi_sig_script['address'], 10.00)
+        self.nodes[1].generate(1)
+        timestamp = self.nodes[1].getblock(self.nodes[1].getbestblockhash())['mediantime']
+
+        self.log.info("Should import a p2sh with respective redeem script and private keys")
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": {
+                "address": multi_sig_script['address']
+            },
+            "timestamp": "now",
+            "redeemscript": multi_sig_script['redeemScript'],
+            "keys": [ self.nodes[0].dumpprivkey(sig_address_1['address']), self.nodes[0].dumpprivkey(sig_address_2['address'])],
+            "watchonly": True
+        }])
+        assert_equal(result[0]['success'], False)
+        assert_equal(result[0]['error']['code'], -8)
+        assert_equal(result[0]['error']['message'], 'Incompatibility found between watchonly and keys')
+
+
+        # Address + Public key + !Internal + Wrong pubkey
+        self.log.info("Should not import an address with a wrong public key")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        address2 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": {
+                "address": address['address']
+            },
+            "timestamp": "now",
+            "pubkeys": [ address2['pubkey'] ]
+        }])
+        assert_equal(result[0]['success'], False)
+        assert_equal(result[0]['error']['code'], -5)
+        assert_equal(result[0]['error']['message'], 'Consistency check failed')
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], False)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal('timestamp' in address_assert, False)
+
+
+        # ScriptPubKey + Public key + internal + Wrong pubkey
+        self.log.info("Should not import a scriptPubKey with internal and with a wrong public key")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        address2 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        request = [{
+            "scriptPubKey": address['scriptPubKey'],
+            "timestamp": "now",
+            "pubkeys": [ address2['pubkey'] ],
+            "internal": True
+        }]
+        result = self.nodes[1].importmulti(request)
+        assert_equal(result[0]['success'], False)
+        assert_equal(result[0]['error']['code'], -5)
+        assert_equal(result[0]['error']['message'], 'Consistency check failed')
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], False)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal('timestamp' in address_assert, False)
+
+
+        # Address + Private key + !watchonly + Wrong private key
+        self.log.info("Should not import an address with a wrong private key")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        address2 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": {
+                "address": address['address']
+            },
+            "timestamp": "now",
+            "keys": [ self.nodes[0].dumpprivkey(address2['address']) ]
+        }])
+        assert_equal(result[0]['success'], False)
+        assert_equal(result[0]['error']['code'], -5)
+        assert_equal(result[0]['error']['message'], 'Consistency check failed')
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], False)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal('timestamp' in address_assert, False)
+
+
+        # ScriptPubKey + Private key + internal + Wrong private key
+        self.log.info("Should not import a scriptPubKey with internal and with a wrong private key")
+        address = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        address2 = self.nodes[0].validateaddress(self.nodes[0].getnewaddress())
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": address['scriptPubKey'],
+            "timestamp": "now",
+            "keys": [ self.nodes[0].dumpprivkey(address2['address']) ],
+            "internal": True
+        }])
+        assert_equal(result[0]['success'], False)
+        assert_equal(result[0]['error']['code'], -5)
+        assert_equal(result[0]['error']['message'], 'Consistency check failed')
+        address_assert = self.nodes[1].validateaddress(address['address'])
+        assert_equal(address_assert['iswatchonly'], False)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal('timestamp' in address_assert, False)
+
+
+        # Importing existing watch only address with new timestamp should replace saved timestamp.
+        assert_greater_than(timestamp, watchonly_timestamp)
+        self.log.info("Should replace previously saved watch only timestamp.")
+        result = self.nodes[1].importmulti([{
+            "scriptPubKey": {
+                "address": watchonly_address,
+            },
+            "timestamp": "now",
+        }])
+        assert_equal(result[0]['success'], True)
+        address_assert = self.nodes[1].validateaddress(watchonly_address)
+        assert_equal(address_assert['iswatchonly'], True)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal(address_assert['timestamp'], timestamp)
+        watchonly_timestamp = timestamp
+
+
+        # restart nodes to check for proper serialization/deserialization of watch only address
+        self.stop_nodes()
+        self.start_nodes()
+        address_assert = self.nodes[1].validateaddress(watchonly_address)
+        assert_equal(address_assert['iswatchonly'], True)
+        assert_equal(address_assert['ismine'], False)
+        assert_equal(address_assert['timestamp'], watchonly_timestamp)
+
+        # Bad or missing timestamps
+        self.log.info("Should throw on invalid or missing timestamp values")
+        assert_raises_rpc_error(-3, 'Missing required timestamp field for key',
+            self.nodes[1].importmulti, [{
+                "scriptPubKey": address['scriptPubKey'],
+            }])
+        assert_raises_rpc_error(-3, 'Expected number or "now" timestamp value for key. got type string',
+            self.nodes[1].importmulti, [{
+                "scriptPubKey": address['scriptPubKey'],
+                "timestamp": "",
+            }])
+
+
+if __name__ == '__main__':
+    ImportMultiTest ().main ()

test/functional/wallet_keypool.py

@@ -0,0 +1,79 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the wallet keypool and interaction with wallet encryption/locking."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+class KeyPoolTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 1
+
+    def run_test(self):
+        nodes = self.nodes
+        addr_before_encrypting = nodes[0].getnewaddress()
+        addr_before_encrypting_data = nodes[0].validateaddress(addr_before_encrypting)
+
+        # Encrypt wallet and wait to terminate
+        nodes[0].node_encrypt_wallet('test')
+        # Restart node 0
+        self.start_node(0)
+        # Keep creating keys
+        addr = nodes[0].getnewaddress()
+        addr_data = nodes[0].validateaddress(addr)
+        assert_raises_rpc_error(-12, "Error: Keypool ran out, please call keypoolrefill first", nodes[0].getnewaddress)
+
+        # put six (plus 2) new keys in the keypool (100% external-, +100% internal-keys, 1 in min)
+        nodes[0].walletpassphrase('test', 12000)
+        nodes[0].keypoolrefill(6)
+        nodes[0].walletlock()
+        wi = nodes[0].getwalletinfo()
+        assert_equal(wi['keypoolsize'], 7)
+
+        # drain the internal keys
+        nodes[0].getrawchangeaddress()
+        nodes[0].getrawchangeaddress()
+        nodes[0].getrawchangeaddress()
+        nodes[0].getrawchangeaddress()
+        nodes[0].getrawchangeaddress()
+        nodes[0].getrawchangeaddress()
+        nodes[0].getrawchangeaddress()
+        addr = set()
+        # the next one should fail
+        assert_raises_rpc_error(-12, "Keypool ran out", nodes[0].getrawchangeaddress)
+
+        # drain the external keys
+        #addr.add(nodes[0].getnewaddress())
+        #addr.add(nodes[0].getnewaddress())
+        #addr.add(nodes[0].getnewaddress())
+        #addr.add(nodes[0].getnewaddress())
+        #addr.add(nodes[0].getnewaddress())
+        #addr.add(nodes[0].getnewaddress())
+        #addr.add(nodes[0].getnewaddress())
+        #assert(len(addr) == 7)
+        # the next one should fail
+        #assert_raises_rpc_error(-12, "Error: Keypool ran out, please call keypoolrefill first", nodes[0].getnewaddress)
+
+        # refill keypool with three new addresses
+        nodes[0].walletpassphrase('test', 1)
+        nodes[0].keypoolrefill(3)
+
+        # test walletpassphrase timeout
+        time.sleep(1.1)
+        assert_equal(nodes[0].getwalletinfo()["unlocked_until"], 0)
+
+        # drain them by mining
+        #nodes[0].generate(1)
+        #nodes[0].generate(1)
+        #nodes[0].generate(1)
+        #assert_raises_rpc_error(-12, "Keypool ran out", nodes[0].generate, 1)
+
+        nodes[0].walletpassphrase('test', 100)
+        nodes[0].keypoolrefill(100)
+        wi = nodes[0].getwalletinfo()
+        assert_equal(wi['keypoolsize'], 101)
+
+if __name__ == '__main__':
+    KeyPoolTest().main()

test/functional/wallet_keypool_topup.py

@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+# Copyright (c) 2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test HD Wallet keypool restore function.
+
+Two nodes. Node1 is under test. Node0 is providing transactions and generating blocks.
+
+- Start node1, shutdown and backup wallet.
+- Generate 110 keys (enough to drain the keypool). Store key 90 (in the initial keypool) and key 110 (beyond the initial keypool). Send funds to key 90 and key 110.
+- Stop node1, clear the datadir, move wallet file back into the datadir and restart node1.
+- connect node1 to node0. Verify that they sync and node1 receives its funds."""
+import shutil
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (
+    assert_equal,
+    connect_nodes_bi,
+    sync_blocks,
+)
+
+class KeypoolRestoreTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 2
+        self.extra_args = [[], ['-keypool=100', '-keypoolmin=20']]
+
+    def run_test(self):
+        self.tmpdir = self.options.tmpdir
+        self.nodes[0].generate(101)
+
+        self.log.info("Make backup of wallet")
+
+        self.stop_node(1)
+
+        shutil.copyfile(self.tmpdir + "/node1/regtest/wallet.dat", self.tmpdir + "/wallet.bak")
+        self.start_node(1, self.extra_args[1])
+        connect_nodes_bi(self.nodes, 0, 1)
+
+        self.log.info("Generate keys for wallet")
+
+        for _ in range(90):
+            addr_oldpool = self.nodes[1].getnewaddress()
+        for _ in range(20):
+            addr_extpool = self.nodes[1].getnewaddress()
+
+        self.log.info("Send funds to wallet")
+
+        self.nodes[0].sendtoaddress(addr_oldpool, 10)
+        self.nodes[0].generate(1)
+        self.nodes[0].sendtoaddress(addr_extpool, 5)
+        self.nodes[0].generate(1)
+        sync_blocks(self.nodes)
+
+        self.log.info("Restart node with wallet backup")
+
+        self.stop_node(1)
+
+        shutil.copyfile(self.tmpdir + "/wallet.bak", self.tmpdir + "/node1/regtest/wallet.dat")
+
+        self.log.info("Verify keypool is restored and balance is correct")
+
+        self.start_node(1, self.extra_args[1])
+        connect_nodes_bi(self.nodes, 0, 1)
+        self.sync_all()
+
+        assert_equal(self.nodes[1].getbalance(), 10)
+        assert_equal(self.nodes[1].listtransactions()[0]['category'], "receive")
+
+        # Check that we have marked all keys up to the used keypool key as used
+        #assert_equal(self.nodes[1].validateaddress(self.nodes[1].getnewaddress())['hdkeypath'], "m/0'/0'/110'")
+
+if __name__ == '__main__':
+    KeypoolRestoreTest().main()

test/functional/wallet_listreceivedby.py

@@ -0,0 +1,124 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the listreceivedbyaddress RPC."""
+from decimal import Decimal
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (
+    assert_array_result,
+    assert_equal,
+    assert_raises_rpc_error,
+    sync_blocks,
+)
+
+
+class ReceivedByTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 2
+
+    def run_test(self):
+        # Generate block to get out of IBD
+        self.nodes[0].generate(1)
+        sync_blocks(self.nodes)
+
+        self.log.info("listreceivedbyaddress Test")
+
+        # Send from node 0 to 1
+        addr = self.nodes[1].getnewaddress()
+        txid = self.nodes[0].sendtoaddress(addr, 0.1)
+        self.sync_all()
+
+        # Check not listed in listreceivedbyaddress because has 0 confirmations
+        assert_array_result(self.nodes[1].listreceivedbyaddress(),
+                            {"address": addr},
+                            {},
+                            True)
+        # Bury Tx under 10 block so it will be returned by listreceivedbyaddress
+        self.nodes[1].generate(10)
+        self.sync_all()
+        assert_array_result(self.nodes[1].listreceivedbyaddress(),
+                            {"address": addr},
+                            {"address": addr, "account": "", "amount": Decimal("0.1"), "confirmations": 10, "txids": [txid, ]})
+        # With min confidence < 10
+        assert_array_result(self.nodes[1].listreceivedbyaddress(5),
+                            {"address": addr},
+                            {"address": addr, "account": "", "amount": Decimal("0.1"), "confirmations": 10, "txids": [txid, ]})
+        # With min confidence > 10, should not find Tx
+        assert_array_result(self.nodes[1].listreceivedbyaddress(11), {"address": addr}, {}, True)
+
+        # Empty Tx
+        addr = self.nodes[1].getnewaddress()
+        assert_array_result(self.nodes[1].listreceivedbyaddress(0, True),
+                            {"address": addr},
+                            {"address": addr, "account": "", "amount": 0, "confirmations": 0, "txids": []})
+
+        self.log.info("getreceivedbyaddress Test")
+
+        # Send from node 0 to 1
+        addr = self.nodes[1].getnewaddress()
+        txid = self.nodes[0].sendtoaddress(addr, 0.1)
+        self.sync_all()
+
+        # Check balance is 0 because of 0 confirmations
+        balance = self.nodes[1].getreceivedbyaddress(addr)
+        assert_equal(balance, Decimal("0.0"))
+
+        # Check balance is 0.1
+        balance = self.nodes[1].getreceivedbyaddress(addr, 0)
+        assert_equal(balance, Decimal("0.1"))
+
+        # Bury Tx under 10 block so it will be returned by the default getreceivedbyaddress
+        self.nodes[1].generate(10)
+        self.sync_all()
+        balance = self.nodes[1].getreceivedbyaddress(addr)
+        assert_equal(balance, Decimal("0.1"))
+
+        # Trying to getreceivedby for an address the wallet doesn't own should return an error
+        assert_raises_rpc_error(-4, "Address not found in wallet", self.nodes[0].getreceivedbyaddress, addr)
+
+        self.log.info("listreceivedbyaccount + getreceivedbyaccount Test")
+
+        # set pre-state
+        addrArr = self.nodes[1].getnewaddress()
+        account = self.nodes[1].getaccount(addrArr)
+        received_by_account_json = [r for r in self.nodes[1].listreceivedbyaccount() if r["account"] == account][0]
+        balance_by_account = self.nodes[1].getreceivedbyaccount(account)
+
+        txid = self.nodes[0].sendtoaddress(addr, 0.1)
+        self.sync_all()
+
+        # listreceivedbyaccount should return received_by_account_json because of 0 confirmations
+        assert_array_result(self.nodes[1].listreceivedbyaccount(),
+                            {"account": account},
+                            received_by_account_json)
+
+        # getreceivedbyaddress should return same balance because of 0 confirmations
+        balance = self.nodes[1].getreceivedbyaccount(account)
+        assert_equal(balance, balance_by_account)
+
+        self.nodes[1].generate(10)
+        self.sync_all()
+        # listreceivedbyaccount should return updated account balance
+        assert_array_result(self.nodes[1].listreceivedbyaccount(),
+                            {"account": account},
+                            {"account": received_by_account_json["account"], "amount": (received_by_account_json["amount"] + Decimal("0.1"))})
+
+        # getreceivedbyaddress should return updates balance
+        balance = self.nodes[1].getreceivedbyaccount(account)
+        assert_equal(balance, balance_by_account + Decimal("0.1"))
+
+        # Create a new account named "mynewaccount" that has a 0 balance
+        self.nodes[1].getaccountaddress("mynewaccount")
+        received_by_account_json = [r for r in self.nodes[1].listreceivedbyaccount(0, True) if r["account"] == "mynewaccount"][0]
+
+        # Test includeempty of listreceivedbyaccount
+        assert_equal(received_by_account_json["amount"], Decimal("0.0"))
+
+        # Test getreceivedbyaccount for 0 amount accounts
+        balance = self.nodes[1].getreceivedbyaccount("mynewaccount")
+        assert_equal(balance, Decimal("0.0"))
+
+if __name__ == '__main__':
+    ReceivedByTest().main()

test/functional/wallet_listsinceblock.py

@@ -0,0 +1,278 @@
+#!/usr/bin/env python3
+# Copyright (c) 2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the listsincelast RPC."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import assert_equal, assert_array_result, assert_raises_rpc_error
+
+class ListSinceBlockTest (BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 4
+        self.setup_clean_chain = True
+
+    def run_test(self):
+        self.nodes[2].generate(101)
+        self.sync_all()
+
+        self.test_no_blockhash()
+        self.test_invalid_blockhash()
+        self.test_reorg()
+        self.test_double_spend()
+        self.test_double_send()
+
+    def test_no_blockhash(self):
+        txid = self.nodes[2].sendtoaddress(self.nodes[0].getnewaddress(), 1)
+        blockhash, = self.nodes[2].generate(1)
+        self.sync_all()
+
+        txs = self.nodes[0].listtransactions()
+        assert_array_result(txs, {"txid": txid}, {
+            "category": "receive",
+            "amount": 1,
+            "blockhash": blockhash,
+            "confirmations": 1,
+        })
+        assert_equal(
+            self.nodes[0].listsinceblock(),
+            {"lastblock": blockhash,
+             "transactions": txs})
+        assert_equal(
+            self.nodes[0].listsinceblock(""),
+            {"lastblock": blockhash,
+             "transactions": txs})
+
+    def test_invalid_blockhash(self):
+        assert_raises_rpc_error(-5, "Block not found", self.nodes[0].listsinceblock,
+                                "42759cde25462784395a337460bde75f58e73d3f08bd31fdc3507cbac856a2c4")
+        assert_raises_rpc_error(-5, "Block not found", self.nodes[0].listsinceblock,
+                                "0000000000000000000000000000000000000000000000000000000000000000")
+        assert_raises_rpc_error(-5, "Block not found", self.nodes[0].listsinceblock,
+                                "invalid-hex")
+
+    def test_reorg(self):
+        '''
+        `listsinceblock` did not behave correctly when handed a block that was
+        no longer in the main chain:
+
+             ab0
+          /       \
+        aa1 [tx0]   bb1
+         |           |
+        aa2         bb2
+         |           |
+        aa3         bb3
+                     |
+                    bb4
+
+        Consider a client that has only seen block `aa3` above. It asks the node
+        to `listsinceblock aa3`. But at some point prior the main chain switched
+        to the bb chain.
+
+        Previously: listsinceblock would find height=4 for block aa3 and compare
+        this to height=5 for the tip of the chain (bb4). It would then return
+        results restricted to bb3-bb4.
+
+        Now: listsinceblock finds the fork at ab0 and returns results in the
+        range bb1-bb4.
+
+        This test only checks that [tx0] is present.
+        '''
+
+        # Split network into two
+        self.split_network()
+
+        # send to nodes[0] from nodes[2]
+        senttx = self.nodes[2].sendtoaddress(self.nodes[0].getnewaddress(), 1)
+
+        # generate on both sides
+        lastblockhash = self.nodes[1].generate(6)[5]
+        self.nodes[2].generate(7)
+        self.log.info('lastblockhash=%s' % (lastblockhash))
+
+        self.sync_all([self.nodes[:2], self.nodes[2:]])
+
+        self.join_network()
+
+        # listsinceblock(lastblockhash) should now include tx, as seen from nodes[0]
+        lsbres = self.nodes[0].listsinceblock(lastblockhash)
+        found = False
+        for tx in lsbres['transactions']:
+            if tx['txid'] == senttx:
+                found = True
+                break
+        assert found
+
+    def test_double_spend(self):
+        '''
+        This tests the case where the same UTXO is spent twice on two separate
+        blocks as part of a reorg.
+
+             ab0
+          /       \
+        aa1 [tx1]   bb1 [tx2]
+         |           |
+        aa2         bb2
+         |           |
+        aa3         bb3
+                     |
+                    bb4
+
+        Problematic case:
+
+        1. User 1 receives BTC in tx1 from utxo1 in block aa1.
+        2. User 2 receives BTC in tx2 from utxo1 (same) in block bb1
+        3. User 1 sees 2 confirmations at block aa3.
+        4. Reorg into bb chain.
+        5. User 1 asks `listsinceblock aa3` and does not see that tx1 is now
+           invalidated.
+
+        Currently the solution to this is to detect that a reorg'd block is
+        asked for in listsinceblock, and to iterate back over existing blocks up
+        until the fork point, and to include all transactions that relate to the
+        node wallet.
+        '''
+
+        self.sync_all()
+
+        # Split network into two
+        self.split_network()
+
+        # share utxo between nodes[1] and nodes[2]
+        utxos = self.nodes[2].listunspent()
+        utxo = utxos[0]
+        privkey = self.nodes[2].dumpprivkey(utxo['address'])
+        self.nodes[1].importprivkey(privkey)
+
+        # send from nodes[1] using utxo to nodes[0]
+        change = '%.8f' % (float(utxo['amount']) - 1.0003)
+        recipientDict = {
+            self.nodes[0].getnewaddress(): 1,
+            self.nodes[1].getnewaddress(): change,
+        }
+        utxoDicts = [{
+            'txid': utxo['txid'],
+            'vout': utxo['vout'],
+        }]
+        txid1 = self.nodes[1].sendrawtransaction(
+            self.nodes[1].signrawtransaction(
+                self.nodes[1].createrawtransaction(utxoDicts, recipientDict))['hex'])
+
+        # send from nodes[2] using utxo to nodes[3]
+        recipientDict2 = {
+            self.nodes[3].getnewaddress(): 1,
+            self.nodes[2].getnewaddress(): change,
+        }
+        self.nodes[2].sendrawtransaction(
+            self.nodes[2].signrawtransaction(
+                self.nodes[2].createrawtransaction(utxoDicts, recipientDict2))['hex'])
+
+        # generate on both sides
+        lastblockhash = self.nodes[1].generate(3)[2]
+        self.nodes[2].generate(4)
+
+        self.join_network()
+
+        self.sync_all()
+
+        # gettransaction should work for txid1
+        assert self.nodes[0].gettransaction(txid1)['txid'] == txid1, "gettransaction failed to find txid1"
+
+        # listsinceblock(lastblockhash) should now include txid1, as seen from nodes[0]
+        lsbres = self.nodes[0].listsinceblock(lastblockhash)
+        assert any(tx['txid'] == txid1 for tx in lsbres['removed'])
+
+        # but it should not include 'removed' if include_removed=false
+        lsbres2 = self.nodes[0].listsinceblock(blockhash=lastblockhash, include_removed=False)
+        assert 'removed' not in lsbres2
+
+    def test_double_send(self):
+        '''
+        This tests the case where the same transaction is submitted twice on two
+        separate blocks as part of a reorg. The former will vanish and the
+        latter will appear as the true transaction (with confirmations dropping
+        as a result).
+
+             ab0
+          /       \
+        aa1 [tx1]   bb1
+         |           |
+        aa2         bb2
+         |           |
+        aa3         bb3 [tx1]
+                     |
+                    bb4
+
+        Asserted:
+
+        1. tx1 is listed in listsinceblock.
+        2. It is included in 'removed' as it was removed, even though it is now
+           present in a different block.
+        3. It is listed with a confirmations count of 2 (bb3, bb4), not
+           3 (aa1, aa2, aa3).
+        '''
+
+        self.sync_all()
+
+        # Split network into two
+        self.split_network()
+
+        # create and sign a transaction
+        utxos = self.nodes[2].listunspent()
+        utxo = utxos[0]
+        change = '%.8f' % (float(utxo['amount']) - 1.0003)
+        recipientDict = {
+            self.nodes[0].getnewaddress(): 1,
+            self.nodes[2].getnewaddress(): change,
+        }
+        utxoDicts = [{
+            'txid': utxo['txid'],
+            'vout': utxo['vout'],
+        }]
+        signedtxres = self.nodes[2].signrawtransaction(
+                self.nodes[2].createrawtransaction(utxoDicts, recipientDict))
+        assert signedtxres['complete']
+
+        signedtx = signedtxres['hex']
+
+        # send from nodes[1]; this will end up in aa1
+        txid1 = self.nodes[1].sendrawtransaction(signedtx)
+
+        # generate bb1-bb2 on right side
+        self.nodes[2].generate(2)
+
+        # send from nodes[2]; this will end up in bb3
+        txid2 = self.nodes[2].sendrawtransaction(signedtx)
+
+        assert_equal(txid1, txid2)
+
+        # generate on both sides
+        lastblockhash = self.nodes[1].generate(3)[2]
+        self.nodes[2].generate(2)
+
+        self.join_network()
+
+        self.sync_all()
+
+        # gettransaction should work for txid1
+        self.nodes[0].gettransaction(txid1)
+
+        # listsinceblock(lastblockhash) should now include txid1 in transactions
+        # as well as in removed
+        lsbres = self.nodes[0].listsinceblock(lastblockhash)
+        assert any(tx['txid'] == txid1 for tx in lsbres['transactions'])
+        assert any(tx['txid'] == txid1 for tx in lsbres['removed'])
+
+        # find transaction and ensure confirmations is valid
+        for tx in lsbres['transactions']:
+            if tx['txid'] == txid1:
+                assert_equal(tx['confirmations'], 2)
+
+        # the same check for the removed array; confirmations should STILL be 2
+        for tx in lsbres['removed']:
+            if tx['txid'] == txid1:
+                assert_equal(tx['confirmations'], 2)
+
+if __name__ == '__main__':
+    ListSinceBlockTest().main()

test/functional/wallet_resendwallettransactions.py

@@ -0,0 +1,29 @@
+#!/usr/bin/env python3
+# Copyright (c) 2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test resendwallettransactions RPC."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import assert_equal, assert_raises_rpc_error
+
+class ResendWalletTransactionsTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 1
+        self.extra_args = [['--walletbroadcast=false']]
+
+    def run_test(self):
+        # Should raise RPC_WALLET_ERROR (-4) if walletbroadcast is disabled.
+        assert_raises_rpc_error(-4, "Error: Wallet transaction broadcasting is disabled with -walletbroadcast", self.nodes[0].resendwallettransactions)
+
+        # Should return an empty array if there aren't unconfirmed wallet transactions.
+        self.stop_node(0)
+        self.start_node(0, extra_args=[])
+        assert_equal(self.nodes[0].resendwallettransactions(), [])
+
+        # Should return an array with the unconfirmed wallet transaction.
+        txid = self.nodes[0].sendtoaddress(self.nodes[0].getnewaddress(), 1)
+        assert_equal(self.nodes[0].resendwallettransactions(), [txid])
+
+if __name__ == '__main__':
+    ResendWalletTransactionsTest().main()

test/functional/wallet_txn_clone.py

@@ -0,0 +1,159 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2016 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the wallet accounts properly when there are cloned transactions with malleated scriptsigs."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+
+class TxnMallTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 4
+
+    def add_options(self, parser):
+        parser.add_option("--mineblock", dest="mine_block", default=False, action="store_true",
+                          help="Test double-spend of 1-confirmed transaction")
+
+    def setup_network(self):
+        # Start with split network:
+        super(TxnMallTest, self).setup_network()
+        disconnect_nodes(self.nodes[1], 2)
+        disconnect_nodes(self.nodes[2], 1)
+
+    def run_test(self):
+        # All nodes should start with 1,250 BTC:
+        starting_balance = 6250
+        for i in range(4):
+            assert_equal(self.nodes[i].getbalance(), starting_balance)
+            self.nodes[i].getnewaddress("")  # bug workaround, coins generated assigned to first getnewaddress!
+
+        # Assign coins to foo and bar accounts:
+        self.nodes[0].settxfee(.001)
+
+        node0_address_foo = self.nodes[0].getnewaddress("foo")
+        fund_foo_txid = self.nodes[0].sendfrom("", node0_address_foo, (1219 *5))
+        fund_foo_tx = self.nodes[0].gettransaction(fund_foo_txid)
+
+        node0_address_bar = self.nodes[0].getnewaddress("bar")
+        fund_bar_txid = self.nodes[0].sendfrom("", node0_address_bar, (29 *5))
+        fund_bar_tx = self.nodes[0].gettransaction(fund_bar_txid)
+
+        assert_equal(self.nodes[0].getbalance(""),
+                     starting_balance - (1219 * 5) - (29 * 5) + fund_foo_tx["fee"] + fund_bar_tx["fee"])
+
+        # Coins are sent to node1_address
+        node1_address = self.nodes[1].getnewaddress("from0")
+
+        # Send tx1, and another transaction tx2 that won't be cloned
+        txid1 = self.nodes[0].sendfrom("foo", node1_address, (40 * 5), 0)
+        txid2 = self.nodes[0].sendfrom("bar", node1_address, (20 * 5), 0)
+
+        # Construct a clone of tx1, to be malleated
+        rawtx1 = self.nodes[0].getrawtransaction(txid1,1)
+        clone_inputs = [{"txid":rawtx1["vin"][0]["txid"],"vout":rawtx1["vin"][0]["vout"]}]
+        clone_outputs = {rawtx1["vout"][0]["scriptPubKey"]["addresses"][0]:float(rawtx1["vout"][0]["value"]),
+                         rawtx1["vout"][1]["scriptPubKey"]["addresses"][0]:float(rawtx1["vout"][1]["value"])}
+        clone_locktime = rawtx1["locktime"]
+        clone_raw = self.nodes[0].createrawtransaction(clone_inputs, clone_outputs, clone_locktime)
+
+        # createrawtransaction randomizes the order of its outputs, so swap them if necessary.
+        # output 0 is at version+#inputs+input+sigstub+sequence+#outputs
+        # 40 BTC serialized is 00286bee00000000
+        pos0 = 2*(4+1+36+1+4+1)
+        hex40 = "00286bee00000000"
+        output_len = 16 + 2 + 2 * int("0x" + clone_raw[pos0 + 16 : pos0 + 16 + 2], 0)
+        if (rawtx1["vout"][0]["value"] == 40 and clone_raw[pos0 : pos0 + 16] != hex40 or
+            rawtx1["vout"][0]["value"] != 40 and clone_raw[pos0 : pos0 + 16] == hex40):
+            output0 = clone_raw[pos0 : pos0 + output_len]
+            output1 = clone_raw[pos0 + output_len : pos0 + 2 * output_len]
+            clone_raw = clone_raw[:pos0] + output1 + output0 + clone_raw[pos0 + 2 * output_len:]
+
+        # Use a different signature hash type to sign.  This creates an equivalent but malleated clone.
+        # Don't send the clone anywhere yet
+        tx1_clone = self.nodes[0].signrawtransaction(clone_raw, None, None, "ALL|ANYONECANPAY")
+        assert_equal(tx1_clone["complete"], True)
+
+        # Have node0 mine a block, if requested:
+        if (self.options.mine_block):
+            self.nodes[0].generate(1)
+            sync_blocks(self.nodes[0:2])
+
+        tx1 = self.nodes[0].gettransaction(txid1)
+        tx2 = self.nodes[0].gettransaction(txid2)
+
+        # Node0's balance should be starting balance, plus 50BTC for another
+        # matured block, minus tx1 and tx2 amounts, and minus transaction fees:
+        expected = starting_balance + fund_foo_tx["fee"] + fund_bar_tx["fee"]
+        if self.options.mine_block: expected += 250
+        expected += tx1["amount"] + tx1["fee"]
+        expected += tx2["amount"] + tx2["fee"]
+        assert_equal(self.nodes[0].getbalance(), expected)
+
+        # foo and bar accounts should be debited:
+        assert_equal(self.nodes[0].getbalance("foo", 0), (1219 * 5) + tx1["amount"] + tx1["fee"])
+        assert_equal(self.nodes[0].getbalance("bar", 0), (29 * 5) + tx2["amount"] + tx2["fee"])
+
+        if self.options.mine_block:
+            assert_equal(tx1["confirmations"], 1)
+            assert_equal(tx2["confirmations"], 1)
+            # Node1's "from0" balance should be both transaction amounts:
+            assert_equal(self.nodes[1].getbalance("from0"), -(tx1["amount"] + tx2["amount"]))
+        else:
+            assert_equal(tx1["confirmations"], 0)
+            assert_equal(tx2["confirmations"], 0)
+
+        # Send clone and its parent to miner
+        self.nodes[2].sendrawtransaction(fund_foo_tx["hex"])
+        txid1_clone = self.nodes[2].sendrawtransaction(tx1_clone["hex"])
+        # mine a block...
+        self.nodes[2].generate(1)
+
+        # Reconnect the split network, and sync chain:
+        connect_nodes(self.nodes[1], 2)
+        connect_nodes(self.nodes[0], 2)
+        connect_nodes(self.nodes[2], 0)
+        connect_nodes(self.nodes[2], 1)
+
+        self.nodes[2].sendrawtransaction(fund_bar_tx["hex"])
+        self.nodes[2].sendrawtransaction(tx2["hex"])
+        self.nodes[2].generate(1)  # Mine another block to make sure we sync
+        sync_blocks(self.nodes)
+
+        # Re-fetch transaction info:
+        tx1 = self.nodes[0].gettransaction(txid1)
+        tx1_clone = self.nodes[0].gettransaction(txid1_clone)
+        tx2 = self.nodes[0].gettransaction(txid2)
+
+        # Verify expected confirmations
+        assert_equal(tx1["confirmations"], -1)
+        assert_equal(tx1_clone["confirmations"], 2)
+        assert_equal(tx2["confirmations"], 1)
+
+        # Check node0's total balance; should be same as before the clone, + 100 BTC for 2 matured,
+        # less possible orphaned matured subsidy
+        expected += 500
+        if (self.options.mine_block):
+            expected -= 250
+        assert_equal(self.nodes[0].getbalance(), expected)
+        assert_equal(self.nodes[0].getbalance("*", 0), expected)
+
+        # Check node0's individual account balances.
+        # "foo" should have been debited by the equivalent clone of tx1
+        assert_equal(self.nodes[0].getbalance("foo"), (1219 * 5) + tx1["amount"] + tx1["fee"])
+        # "bar" should have been debited by (possibly unconfirmed) tx2
+        assert_equal(self.nodes[0].getbalance("bar", 0), (29 * 5) + tx2["amount"] + tx2["fee"])
+        # "" should have starting balance, less funding txes, plus subsidies
+        assert_equal(self.nodes[0].getbalance("", 0), starting_balance
+                                                                - (1219 * 5)
+                                                                + fund_foo_tx["fee"]
+                                                                -   (29 * 5)
+                                                                + fund_bar_tx["fee"]
+                                                                +  500)
+
+        # Node1's "from0" account balance
+        assert_equal(self.nodes[1].getbalance("from0", 0), -(tx1["amount"] + tx2["amount"]))
+
+if __name__ == '__main__':
+    TxnMallTest().main()
+

test/functional/wallet_txn_doublespend.py

@@ -0,0 +1,147 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the wallet accounts properly when there is a double-spend conflict."""
+
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import *
+import time
+
+class TxnMallTest(BitcoinTestFramework):
+    def set_test_params(self):
+        self.num_nodes = 4
+
+    def add_options(self, parser):
+        parser.add_option("--mineblock", dest="mine_block", default=False, action="store_true",
+                          help="Test double-spend of 1-confirmed transaction")
+
+    def setup_network(self):
+        # Start with split network:
+        super().setup_network()
+        disconnect_nodes(self.nodes[1], 2)
+        disconnect_nodes(self.nodes[2], 1)
+
+    def run_test(self):
+        # All nodes should start with 6,250 AGR:
+        starting_balance = 6250
+        for i in range(4):
+            assert_equal(self.nodes[i].getbalance(), starting_balance)
+            self.nodes[i].getnewaddress("")  # bug workaround, coins generated assigned to first getnewaddress!
+
+        # Assign coins to foo and bar accounts:
+        node0_address_foo = self.nodes[0].getnewaddress("foo")
+        fund_foo_txid = self.nodes[0].sendfrom("", node0_address_foo, (1219 * 5))
+        fund_foo_tx = self.nodes[0].gettransaction(fund_foo_txid)
+
+        node0_address_bar = self.nodes[0].getnewaddress("bar")
+        fund_bar_txid = self.nodes[0].sendfrom("", node0_address_bar, (29 * 5))
+        fund_bar_tx = self.nodes[0].gettransaction(fund_bar_txid)
+
+        assert_equal(self.nodes[0].getbalance(""),
+                     starting_balance - (1219 * 5) - (29 * 5) + fund_foo_tx["fee"] + fund_bar_tx["fee"])
+
+        # Coins are sent to node1_address
+        node1_address = self.nodes[1].getnewaddress("from0")
+
+        # First: use raw transaction API to send 1240 BTC to node1_address,
+        # but don't broadcast:
+        doublespend_fee = Decimal('-.02')
+        rawtx_input_0 = {}
+        rawtx_input_0["txid"] = fund_foo_txid
+        rawtx_input_0["vout"] = find_output(self.nodes[0], fund_foo_txid, (1219 * 5))
+        rawtx_input_1 = {}
+        rawtx_input_1["txid"] = fund_bar_txid
+        rawtx_input_1["vout"] = find_output(self.nodes[0], fund_bar_txid, (29 * 5))
+        inputs = [rawtx_input_0, rawtx_input_1]
+        change_address = self.nodes[0].getnewaddress()
+        outputs = {}
+        outputs[node1_address] = float(1240 * 5)
+        outputs[change_address] = float((1248 * 5) - (1240 * 5) + doublespend_fee)
+        rawtx = self.nodes[0].createrawtransaction(inputs, outputs)
+        doublespend = self.nodes[0].signrawtransaction(rawtx)
+        assert_equal(doublespend["complete"], True)
+
+        # Create two spends using 1 50 BTC coin each
+        txid1 = self.nodes[0].sendfrom("foo", node1_address, (40 * 5), 0)
+        txid2 = self.nodes[0].sendfrom("bar", node1_address, (20 * 5), 0)
+
+        # Have node0 mine a block:
+        if (self.options.mine_block):
+            self.nodes[0].generate(1)
+            sync_blocks(self.nodes[0:2])
+
+        tx1 = self.nodes[0].gettransaction(txid1)
+        tx2 = self.nodes[0].gettransaction(txid2)
+
+        # Node0's balance should be starting balance, plus 50BTC for another
+        # matured block, minus 40, minus 20, and minus transaction fees:
+        expected = starting_balance + fund_foo_tx["fee"] + fund_bar_tx["fee"]
+        if self.options.mine_block: expected += 250
+        expected += tx1["amount"] + tx1["fee"]
+        expected += tx2["amount"] + tx2["fee"]
+        assert_equal(self.nodes[0].getbalance(), expected)
+
+        # foo and bar accounts should be debited:
+        assert_equal(self.nodes[0].getbalance("foo", 0), (1219 * 5)+tx1["amount"]+tx1["fee"])
+        assert_equal(self.nodes[0].getbalance("bar", 0), (29 * 5)+tx2["amount"]+tx2["fee"])
+
+        if self.options.mine_block:
+            assert_equal(tx1["bcconfirmations"], 1)
+            assert_equal(tx2["bcconfirmations"], 1)
+            # Node1's "from0" balance should be both transaction amounts:
+            assert_equal(self.nodes[1].getbalance("from0"), -(tx1["amount"]+tx2["amount"]))
+        else:
+            assert_equal(tx1["bcconfirmations"], 0)
+            assert_equal(tx2["bcconfirmations"], 0)
+
+        # Now give doublespend and its parents to miner:
+        self.nodes[2].sendrawtransaction(fund_foo_tx["hex"])
+        self.nodes[2].sendrawtransaction(fund_bar_tx["hex"])
+        doublespend_txid = self.nodes[2].sendrawtransaction(doublespend["hex"])
+
+        # mine a block...
+        self.nodes[2].generate(1)
+
+        # Reconnect the split network, and sync chain:
+        connect_nodes(self.nodes[1], 2)
+        connect_nodes(self.nodes[0], 2)
+        connect_nodes(self.nodes[2], 0)
+        connect_nodes(self.nodes[2], 1)
+        self.nodes[2].generate(1)  # Mine another block to make sure we sync
+        sync_blocks(self.nodes)
+        assert_equal(self.nodes[0].gettransaction(doublespend_txid)["confirmations"], 2)
+
+        # Re-fetch transaction info:
+        tx1 = self.nodes[0].gettransaction(txid1)
+        tx2 = self.nodes[0].gettransaction(txid2)
+
+        # Both transactions should be conflicted
+        assert_equal(tx1["bcconfirmations"], -1)
+        assert_equal(tx2["bcconfirmations"], -1)
+
+        # Node0's total balance should be starting balance, plus 100BTC for
+        # two more matured blocks, minus 1240 for the double-spend, plus fees (which are
+        # negative):
+        expected = starting_balance + 500 - (1240 * 5) + fund_foo_tx["fee"] + fund_bar_tx["fee"] + doublespend_fee
+        assert_equal(self.nodes[0].getbalance(), expected)
+        assert_equal(self.nodes[0].getbalance("*"), expected)
+
+        # Final "" balance is starting_balance - amount moved to accounts - doublespend + subsidies +
+        # fees (which are negative)
+        assert_equal(self.nodes[0].getbalance("foo"), (1219 * 5))
+        #assert_equal(self.nodes[0].getbalance("bar"), (29 * 5))
+        #assert_equal(self.nodes[0].getbalance(""), starting_balance
+        #                                                      -(1219 * 5)
+        #                                                      -  (29 * 5)
+        #                                                      -(1240 * 5)
+        #                                                      + 500
+        #                                                      + fund_foo_tx["fee"]
+        #                                                      + fund_bar_tx["fee"]
+        #                                                      + doublespend_fee)
+
+        # Node1's "from0" account balance should be just the doublespend:
+        assert_equal(self.nodes[1].getbalance("from0"), (1240 * 5))
+
+if __name__ == '__main__':
+    TxnMallTest().main()

test/functional/wallet_zapwallettxes.py

@@ -0,0 +1,70 @@
+#!/usr/bin/env python3
+# Copyright (c) 2014-2017 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+"""Test the zapwallettxes functionality.
+
+- start two bitcoind nodes
+- create two transactions on node 0 - one is confirmed and one is unconfirmed.
+- restart node 0 and verify that both the confirmed and the unconfirmed
+  transactions are still available.
+- restart node 0 with zapwallettxes and persistmempool, and verify that both
+  the confirmed and the unconfirmed transactions are still available.
+- restart node 0 with just zapwallettxes and verify that the confirmed
+  transactions are still available, but that the unconfirmed transaction has
+  been zapped.
+"""
+from test_framework.test_framework import BitcoinTestFramework
+from test_framework.util import (
+    assert_equal,
+    assert_raises_rpc_error,
+    wait_until,
+)
+
+class ZapWalletTXesTest (BitcoinTestFramework):
+    def set_test_params(self):
+        self.setup_clean_chain = True
+        self.num_nodes = 2
+
+    def run_test(self):
+        self.log.info("Mining blocks...")
+        self.nodes[0].generate(1)
+        self.sync_all()
+        self.nodes[1].generate(101)
+        self.sync_all()
+
+        assert_equal(self.nodes[0].getbalance(), 250)
+        # This transaction will be confirmed
+        txid1 = self.nodes[0].sendtoaddress(self.nodes[1].getnewaddress(), 10)
+
+        self.sync_all()
+        self.nodes[0].generate(1)
+        self.sync_all()
+
+        # This transaction will not be confirmed
+        txid2 = self.nodes[0].sendtoaddress(self.nodes[1].getnewaddress(), 20)
+
+        # Confirmed and unconfirmed transactions are now in the wallet.
+        assert_equal(self.nodes[0].gettransaction(txid1)['txid'], txid1)
+        assert_equal(self.nodes[0].gettransaction(txid2)['txid'], txid2)
+
+        # Stop-start node0. Both confirmed and unconfirmed transactions remain in the wallet.
+        self.stop_node(0)
+        self.start_node(0)
+
+        assert_equal(self.nodes[0].gettransaction(txid1)['txid'], txid1)
+        assert_equal(self.nodes[0].gettransaction(txid2)['txid'], txid2)
+
+        # Stop node0 and restart with zapwallettxes and persistmempool. The unconfirmed
+        # transaction is zapped from the wallet, but is re-added when the mempool is reloaded.
+        self.stop_node(0)
+        self.start_node(0, ["-zapwallettxes=2"])
+
+        # tx1 is still be available because it was confirmed
+        assert_equal(self.nodes[0].gettransaction(txid1)['txid'], txid1)
+
+        # This will raise an exception because the unconfirmed transaction has been zapped
+        assert_raises_rpc_error(-5, 'Invalid or non-wallet transaction id', self.nodes[0].gettransaction, txid2)
+
+if __name__ == '__main__':
+    ZapWalletTXesTest().main()

test/functional/zerocoin_publicSpend_reorg.py

@@ -0,0 +1,208 @@
+# !/usr/bin/env python3
+# Copyright (c) 2019 The PIVX developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+'''
+Covers the scenario of two valid PoS blocks with same height
+and same coinstake input.
+'''
+
+from copy import deepcopy
+from io import BytesIO
+import time
+from test_framework.messages import CTransaction, CBlock
+from test_framework.util import bytes_to_hex_str, hex_str_to_bytes, assert_equal
+from fake_stake.base_test import Agrarian_FakeStakeTest
+
+
+class ZerocoinPublicSpendReorg(Agrarian_FakeStakeTest):
+
+    def run_test(self):
+        self.description = "Covers the reorg with a zc public spend in vtx"
+        self.init_test()
+        DENOM_TO_USE = 10           # zc denomination
+        INITAL_MINED_BLOCKS = 321   # First mined blocks (rewards collected to mint)
+        MORE_MINED_BLOCKS = 105     # More blocks mined before spending zerocoins
+
+        # 1) Starting mining blocks
+        self.log.info("Mining %d blocks.." % INITAL_MINED_BLOCKS)
+        self.node.generate(INITAL_MINED_BLOCKS)
+
+        # 2) Mint 2 zerocoins
+        self.node.mintzerocoin(DENOM_TO_USE)
+        self.node.generate(1)
+        self.node.mintzerocoin(DENOM_TO_USE)
+        self.node.generate(1)
+
+        # 3) Mine additional blocks and collect the mints
+        self.log.info("Mining %d blocks.." % MORE_MINED_BLOCKS)
+        self.node.generate(MORE_MINED_BLOCKS)
+        self.log.info("Collecting mints...")
+        mints = self.node.listmintedzerocoins(True, False)
+        assert len(mints) == 2, "mints list has len %d (!= 2)" % len(mints)
+
+        # 4) Get unspent coins and chain tip
+        self.unspent = self.node.listunspent()
+        block_count = self.node.getblockcount()
+        pastBlockHash = self.node.getblockhash(block_count)
+        self.log.info("Block count: %d - Current best: %s..." % (self.node.getblockcount(), self.node.getbestblockhash()[:5]))
+        pastBlock = CBlock()
+        pastBlock.deserialize(BytesIO(hex_str_to_bytes(self.node.getblock(pastBlockHash, False))))
+        checkpoint = pastBlock.nAccumulatorCheckpoint
+
+        # 5) get the raw zerocoin spend txes
+        self.log.info("Getting the raw zerocoin public spends...")
+        public_spend_A = self.node.createrawzerocoinpublicspend(mints[0].get("serial hash"))
+        tx_A = CTransaction()
+        tx_A.deserialize(BytesIO(hex_str_to_bytes(public_spend_A)))
+        tx_A.rehash()
+        public_spend_B = self.node.createrawzerocoinpublicspend(mints[1].get("serial hash"))
+        tx_B = CTransaction()
+        tx_B.deserialize(BytesIO(hex_str_to_bytes(public_spend_B)))
+        tx_B.rehash()
+        # Spending same coins to different recipients to get different txids
+        my_addy = "yAVWM5urwaTyhiuFQHP2aP47rdZsLUG5PH"
+        public_spend_A2 = self.node.createrawzerocoinpublicspend(mints[0].get("serial hash"), my_addy)
+        tx_A2 = CTransaction()
+        tx_A2.deserialize(BytesIO(hex_str_to_bytes(public_spend_A2)))
+        tx_A2.rehash()
+        public_spend_B2 = self.node.createrawzerocoinpublicspend(mints[1].get("serial hash"), my_addy)
+        tx_B2 = CTransaction()
+        tx_B2.deserialize(BytesIO(hex_str_to_bytes(public_spend_B2)))
+        tx_B2.rehash()
+        self.log.info("tx_A id: %s" % str(tx_A.hash))
+        self.log.info("tx_B id: %s" % str(tx_B.hash))
+        self.log.info("tx_A2 id: %s" % str(tx_A2.hash))
+        self.log.info("tx_B2 id: %s" % str(tx_B2.hash))
+
+
+        self.test_nodes[0].handle_connect()
+
+        # 6) create block_A --> main chain
+        self.log.info("")
+        self.log.info("*** block_A ***")
+        self.log.info("Creating block_A [%d] with public spend tx_A in it." % (block_count + 1))
+        block_A = self.new_block(block_count, pastBlock, checkpoint, tx_A)
+        self.log.info("Hash of block_A: %s..." % block_A.hash[:5])
+        self.log.info("sending block_A...")
+        var = self.node.submitblock(bytes_to_hex_str(block_A.serialize()))
+        if var is not None:
+            self.log.info("result: %s" % str(var))
+            raise Exception("block_A not accepted")
+        time.sleep(2)
+        assert_equal(self.node.getblockcount(), block_count+1)
+        assert_equal(self.node.getbestblockhash(), block_A.hash)
+        self.log.info("  >>  block_A connected  <<")
+        self.log.info("Current chain: ... --> block_0 [%d] --> block_A [%d]\n" % (block_count, block_count+1))
+
+        # 7) create block_B --> forked chain
+        self.log.info("*** block_B ***")
+        self.log.info("Creating block_B [%d] with public spend tx_B in it." % (block_count + 1))
+        block_B = self.new_block(block_count, pastBlock, checkpoint, tx_B)
+        self.log.info("Hash of block_B: %s..." % block_B.hash[:5])
+        self.log.info("sending block_B...")
+        var = self.node.submitblock(bytes_to_hex_str(block_B.serialize()))
+        self.log.info("result of block_B submission: %s" % str(var))
+        time.sleep(2)
+        assert_equal(self.node.getblockcount(), block_count+1)
+        assert_equal(self.node.getbestblockhash(), block_A.hash)
+        # block_B is not added. Chain remains the same
+        self.log.info("  >>  block_B not connected  <<")
+        self.log.info("Current chain: ... --> block_0 [%d] --> block_A [%d]\n" % (block_count, block_count+1))
+
+        # 8) Create new block block_C on the forked chain (block_B)
+        block_count += 1
+        self.log.info("*** block_C ***")
+        self.log.info("Creating block_C [%d] on top of block_B triggering the reorg" % (block_count + 1))
+        block_C = self.new_block(block_count, block_B, checkpoint)
+        self.log.info("Hash of block_C: %s..." % block_C.hash[:5])
+        self.log.info("sending block_C...")
+        var = self.node.submitblock(bytes_to_hex_str(block_C.serialize()))
+        if var is not None:
+            self.log.info("result: %s" % str(var))
+            raise Exception("block_C not accepted")
+        time.sleep(2)
+        assert_equal(self.node.getblockcount(), block_count+1)
+        assert_equal(self.node.getbestblockhash(), block_C.hash)
+        self.log.info("  >>  block_A disconnected / block_B and block_C connected  <<")
+        self.log.info("Current chain: ... --> block_0 [%d] --> block_B [%d] --> block_C [%d]\n" % (
+                block_count - 1, block_count, block_count+1
+        ))
+
+        # 7) Now create block_D which tries to spend same coin of tx_B again on the (new) main chain
+        # (this block will be rejected)
+        block_count += 1
+        self.log.info("*** block_D ***")
+        self.log.info("Creating block_D [%d] trying to double spend the coin of tx_B" % (block_count + 1))
+        block_D = self.new_block(block_count, block_C, checkpoint, tx_B2)
+        self.log.info("Hash of block_D: %s..." % block_D.hash[:5])
+        self.log.info("sending block_D...")
+        var = self.node.submitblock(bytes_to_hex_str(block_D.serialize()))
+        self.log.info("result of block_D submission: %s" % str(var))
+        time.sleep(2)
+        assert_equal(self.node.getblockcount(), block_count)
+        assert_equal(self.node.getbestblockhash(), block_C.hash)
+        # block_D is not added. Chain remains the same
+        self.log.info("  >>  block_D rejected  <<")
+        self.log.info("Current chain: ... --> block_0 [%d] --> block_B [%d] --> block_C [%d]\n" % (
+                block_count - 2, block_count - 1, block_count
+        ))
+
+        # 8) Now create block_E which spends tx_A again on main chain
+        # (this block will be accepted and connected since tx_A was spent on block_A now disconnected)
+        self.log.info("*** block_E ***")
+        self.log.info("Creating block_E [%d] trying spend tx_A on main chain" % (block_count + 1))
+        block_E = self.new_block(block_count, block_C, checkpoint, tx_A)
+        self.log.info("Hash of block_E: %s..." % block_E.hash[:5])
+        self.log.info("sending block_E...")
+        var = self.node.submitblock(bytes_to_hex_str(block_E.serialize()))
+        if var is not None:
+            self.log.info("result: %s" % str(var))
+            raise Exception("block_E not accepted")
+        time.sleep(2)
+        assert_equal(self.node.getblockcount(), block_count+1)
+        assert_equal(self.node.getbestblockhash(), block_E.hash)
+        self.log.info("  >>  block_E connected <<")
+        self.log.info("Current chain: ... --> block_0 [%d] --> block_B [%d] --> block_C [%d] --> block_E [%d]\n" % (
+                block_count - 2, block_count - 1, block_count, block_count+1
+        ))
+
+        # 9) Now create block_F which tries to double spend the coin in tx_A
+        # # (this block will be rejected)
+        block_count += 1
+        self.log.info("*** block_F ***")
+        self.log.info("Creating block_F [%d] trying to double spend the coin in tx_A" % (block_count + 1))
+        block_F = self.new_block(block_count, block_E, checkpoint, tx_A2)
+        self.log.info("Hash of block_F: %s..." % block_F.hash[:5])
+        self.log.info("sending block_F...")
+        var = self.node.submitblock(bytes_to_hex_str(block_F.serialize()))
+        self.log.info("result of block_F submission: %s" % str(var))
+        time.sleep(2)
+        assert_equal(self.node.getblockcount(), block_count)
+        assert_equal(self.node.getbestblockhash(), block_E.hash)
+        self.log.info("  >>  block_F rejected <<")
+        self.log.info("Current chain: ... --> block_0 [%d] --> block_B [%d] --> block_C [%d] --> block_E [%d]\n" % (
+                block_count - 3, block_count - 2, block_count - 1, block_count
+        ))
+        self.log.info("All good.")
+
+
+
+    def new_block(self, block_count, prev_block, checkpoint, zcspend = None):
+        if prev_block.hash is None:
+            prev_block.rehash()
+        staking_utxo_list = [self.unspent.pop()]
+        pastBlockHash = prev_block.hash
+        stakingPrevOuts = self.get_prevouts(staking_utxo_list, block_count)
+        block = self.create_spam_block(pastBlockHash, stakingPrevOuts, block_count + 1)
+        if zcspend is not None:
+            block.vtx.append(zcspend)
+            block.hashMerkleRoot = block.calc_merkle_root()
+        block.nAccumulatorCheckpoint = checkpoint
+        block.rehash()
+        block.sign_block(self.block_sig_key)
+        return block
+
+if __name__ == '__main__':
+    ZerocoinPublicSpendReorg().main()

test/functional/zerocoin_valid_public_spend.py

@@ -0,0 +1,102 @@
+#!/usr/bin/env python3
+# Copyright (c) 2019 The PIVX developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+'''
+Covers the 'Wrapped Serials Attack' scenario
+'''
+
+import random
+from time import sleep
+
+from test_framework.authproxy import JSONRPCException
+from test_framework.util import assert_equal, assert_greater_than
+
+from fake_stake.base_test import Agrarian_FakeStakeTest
+
+class zAGRValidCoinSpendTest(Agrarian_FakeStakeTest):
+
+    def run_test(self):
+        self.description = "Covers the 'valid publicCoinSpend spend' scenario."
+        self.init_test()
+
+        INITAL_MINED_BLOCKS = 301   # Blocks mined before minting
+        MORE_MINED_BLOCKS = 52      # Blocks mined after minting (before spending)
+        DENOM_TO_USE = 1         # zc denomination used for double spending attack
+
+        # 1) Start mining blocks
+        self.log.info("Mining %d first blocks..." % INITAL_MINED_BLOCKS)
+        self.node.generate(INITAL_MINED_BLOCKS)
+        sleep(2)
+
+        # 2) Mint zerocoins
+        self.log.info("Minting %d-denom zAGRs..." % DENOM_TO_USE)
+        self.node.mintzerocoin(DENOM_TO_USE)
+        self.node.generate(1)
+        sleep(2)
+        self.node.mintzerocoin(DENOM_TO_USE)
+        sleep(2)
+
+        # 3) Mine more blocks and collect the mint
+        self.log.info("Mining %d more blocks..." % MORE_MINED_BLOCKS)
+        self.node.generate(MORE_MINED_BLOCKS)
+        sleep(2)
+        list = self.node.listmintedzerocoins(True, True)
+        mint = list[0]
+
+        # 4) Get the raw zerocoin data
+        exported_zerocoins = self.node.exportzerocoins(False)
+        zc = [x for x in exported_zerocoins if mint["serial hash"] == x["id"]]
+        if len(zc) == 0:
+            raise AssertionError("mint not found")
+
+        # 5) Spend the minted coin (mine six more blocks)
+        self.log.info("Spending the minted coin with serial %s and mining six more blocks..." % zc[0]["s"])
+        txid = self.node.spendzerocoinmints([mint["serial hash"]])['txid']
+        self.log.info("Spent on tx %s" % txid)
+        self.node.generate(6)
+        sleep(2)
+
+        rawTx = self.node.getrawtransaction(txid, 1)
+        if rawTx is None:
+            self.log.warning("rawTx is: %s" % rawTx)
+            raise AssertionError("TEST FAILED")
+        else:
+            assert (rawTx["confirmations"] == 6)
+
+        self.log.info("%s VALID PUBLIC COIN SPEND PASSED" % self.__class__.__name__)
+
+        self.log.info("%s Trying to spend the serial twice now" % self.__class__.__name__)
+
+        serial = zc[0]["s"]
+        randomness = zc[0]["r"]
+        privkey = zc[0]["k"]
+
+        tx = None
+        try:
+            tx = self.node.spendrawzerocoin(serial, randomness, DENOM_TO_USE, privkey)
+        except JSONRPCException as e:
+            self.log.info("GOOD: Transaction did not verify")
+
+        if tx is not None:
+            self.log.warning("Tx is: %s" % tx)
+            raise AssertionError("TEST FAILED")
+
+        self.log.info("%s DOUBLE SPENT SERIAL NOT VERIFIED, TEST PASSED" % self.__class__.__name__)
+
+        self.log.info("%s Trying to spend using the old coin spend method.." % self.__class__.__name__)
+
+        tx = None
+        try:
+            tx = self.node.spendzerocoin(DENOM_TO_USE, False, False, "", False)
+            raise AssertionError("TEST FAILED, old coinSpend spent")
+        except JSONRPCException as e:
+            self.log.info("GOOD: spendzerocoin old spend did not verify")
+
+
+        self.log.info("%s OLD COIN SPEND NON USABLE ANYMORE, TEST PASSED" % self.__class__.__name__)
+
+
+if __name__ == '__main__':
+    zAGRValidCoinSpendTest().main()

test/functional/zerocoin_wrapped_serials.py

@@ -0,0 +1,108 @@
+#!/usr/bin/env python3
+# Copyright (c) 2019 The PIVX developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+'''
+Covers the 'Wrapped Serials Attack' scenario
+'''
+
+import random
+from time import sleep
+
+from test_framework.authproxy import JSONRPCException
+from test_framework.util import assert_equal, assert_greater_than
+
+from fake_stake.base_test import Agrarian_FakeStakeTest
+
+class zAGRwrappedSerialsTest(Agrarian_FakeStakeTest):
+
+    def run_test(self):
+        q = 73829871667027927151400291810255409637272593023945445234219354687881008052707
+        pow2 = 2**256
+        self.description = "Covers the 'Wrapped Serials Attack' scenario."
+        self.init_test()
+
+        INITAL_MINED_BLOCKS = 351   # Blocks mined before minting
+        MORE_MINED_BLOCKS = 31      # Blocks mined after minting (before spending)
+        DENOM_TO_USE = 1000         # zc denomination used for double spending attack
+        K_BITSIZE = 128             # bitsize of the range for random K
+        NUM_OF_K = 5                # number of wrapping serials to try
+
+        # 1) Start mining blocks
+        self.log.info("Mining %d first blocks..." % INITAL_MINED_BLOCKS)
+        self.node.generate(INITAL_MINED_BLOCKS)
+        sleep(2)
+
+        # 2) Mint zerocoins
+        self.log.info("Minting %d-denom zAGRs..." % DENOM_TO_USE)
+        balance = self.node.getbalance("*", 100)
+        assert_greater_than(balance, DENOM_TO_USE)
+        total_mints = 0
+        while balance > DENOM_TO_USE:
+            try:
+                self.node.mintzerocoin(DENOM_TO_USE)
+            except JSONRPCException:
+                break
+            sleep(1)
+            total_mints += 1
+            self.node.generate(1)
+            sleep(1)
+            if total_mints % 5 == 0:
+                self.log.info("Minted %d coins" % total_mints)
+            if total_mints >= 20:
+                break
+            balance = self.node.getbalance("*", 100)
+        sleep(2)
+
+        # 3) Mine more blocks and collect the mint
+        self.log.info("Mining %d more blocks..." % MORE_MINED_BLOCKS)
+        self.node.generate(MORE_MINED_BLOCKS)
+        sleep(2)
+        mint = self.node.listmintedzerocoins(True, True)[0]
+
+        # 4) Get the raw zerocoin data
+        exported_zerocoins = self.node.exportzerocoins(False)
+        zc = [x for x in exported_zerocoins if mint["serial hash"] == x["id"]]
+        if len(zc) == 0:
+            raise AssertionError("mint not found")
+
+        # 5) Spend the minted coin (mine two more blocks)
+        self.log.info("Spending the minted coin with serial %s and mining two more blocks..." % zc[0]["s"])
+        txid = self.node.spendzerocoinmints([mint["serial hash"]])['txid']
+        self.log.info("Spent on tx %s" % txid)
+        self.node.generate(2)
+        sleep(2)
+
+        # 6) create the new serials
+        new_serials = []
+        for i in range(NUM_OF_K):
+            K = random.getrandbits(K_BITSIZE)
+            new_serials.append(hex(int(zc[0]["s"], 16) + K*q*pow2)[2:])
+
+        randomness = zc[0]["r"]
+        privkey = zc[0]["k"]
+
+        # 7) Spend the new zerocoins
+        for serial in new_serials:
+            self.log.info("Spending the wrapping serial %s" % serial)
+            tx = None
+            try:
+                tx = self.node.spendrawzerocoin(serial, randomness, DENOM_TO_USE, privkey)
+            except JSONRPCException as e:
+                exc_msg = str(e)
+                if exc_msg == "CoinSpend: failed check (-4)":
+                    self.log.info("GOOD: Transaction did not verify")
+                else:
+                    raise e
+
+            if tx is not None:
+                self.log.warning("Tx is: %s" % tx)
+                raise AssertionError("TEST FAILED")
+
+        self.log.info("%s PASSED" % self.__class__.__name__)
+
+
+
+if __name__ == '__main__':
+    zAGRwrappedSerialsTest().main()