Motivation

Maintaining a blockchain network node or validator is a lot of work to initially set up. Depending on the network there are varying levels of system administration work that needs to be done. Early in the life of a network, you will usually have to deploy the same infrastructure over and over again. At the start of my journey validating blockchain protocols, Luke Youngblood had a quote from Elon Musk that he mentioned in a talk that stuck with me "Build the machine that builds the machines". Over the years I have toyed with various different infrastructure provisioning automation from pure shell scripts to complex Terraform deployments.

With the release of a new network - Sui and the start of their testnet it seems like a good idea to build out the infrastructure deployment from scratch to help validators and full node operators while learning a new tool - Ansible.

This tutorial is to show the base structure to set up ansible to build the Sui program and configure a machine while showing the reader how the pieces intertwine. It will also act as an explainer of the key concepts of Ansible.

PR

https://github.com/0xzoz/sui-node-automation/pull/1

Tech/Product

Sui

Sui is a blockchain network that is being developed by Mysten Labs. Mysten Labs consists of some of the core engineers from Meta's defunct Diem/Move project, including the creator of the Move language. It is some of the most exciting tech in the blockchain space and is being built from the ground up factoring in the learnings from the development of Diem/Move. It also sets out to address some of the issues with the existing protocols and languages within the industry at present.

The platform claims to offer unprecedented low latency and scalability for straightforward use cases. This is achievable since the platform enables the capability of parallel transactions. Consequently, Sui can use processing resources more efficiently. With over $300 million in funding since its inception under a year ago, it is one of the most anticipated L1's to come out since Ethereum.

Ansible

Ansible is a software tool that provides simple but powerful automation for cross-platform computer support. It is primarily intended for IT professionals, who use it for application deployment, updates on workstations and servers, cloud provisioning, configuration management, intra-service orchestration, and nearly anything a systems administrator does on a weekly or daily basis. Ansible doesn't depend on agent software and has no additional security infrastructure, so it's easy to deploy.

It works by having a control node and managed nodes. The control node runs ansible and provisions the managed nodes. the control node sends modules to the managed nodes to execute via an ssh connection.

The best part is the composability of the platform allowing the user to package up multiple modules to be used in different applications. With the ability to connect your previously built legos, it drastically speeds up the development process and leaves little room for human error.

Scope

Because there is little information about validator operation and workflow in the Sui ecosystem at this stage, this PR is intended to be a base that can be built upon. Eventually, this will be a repository that any user wishing to run a full node, operate a validator, perform validator operations safely and monitor the node. All from a control machine with one-line commands.

We want this particular PR to allow a user to clone the repository, make a few minor changes and be able to deploy a full node with only one command. It will provision the target machine, build the Sui program and configure it as a systemd service and then start the service.

Subsequent articles will go into detail about running a validator, adding monitoring etc.

Commits

1. Add folder structure

   We want to add a folder structure that relates to the ansible framework. Below will show the outlined structure to help give context as empty folders do not show on git commits.

   ├── group_vars (keep the variables for items that will be used multiple times) 
   │   └── all
   ├── roles  (An ansible role is way to keep related files for a playbook together)
   │   └── sui_fullnode (our first role we will be working with)
   │       └── tasks (a place to store tasks, usually a file in tasks bundles related sub tasks)
   │       └── templates (a place to store templates of files to add to your target machine)
   ├── inventory.yml.sample (a basic inventory template with the key/values we need for our instance) 
   ├── LICENSE
   └── README.md

2. Add docs, sample inventory and license

   We want to add documentation to guide users cloning this repo on how to do a couple of items:

   • Configure their control machine

   • Set up ssh keys to connect from the control machine to the managed nodes

   • Create their inventory

   • Provision and start their node.

   We also want to add the sample inventory key/values that the user will want to copy and configure.

   Lastly, we will want to add a license to our repo that will allow this code to be forked and utilized by others. We choose the MIT open source license because we ❤️ OS and believe that software is a public good.

3. Add variables

   We want to add any common variables that we will be using across our sui_fullnode role(and eventually other roles) to a file within the group vars folder. We have added the all subfolder as these will be used across all the eventual roles we will create.

4. Add templates

   Ansible uses templates to create new files on the nodes using predefined models based on the Jinja2 templating system. Templates are commonly used to configure services based on variable values that can be set up on the playbook itself, in included variable files, or obtained via facts. This enables you to create more versatile setups that adapt behaviour based on dynamic information.
   
   Here we create two templates:

   • Sui fullnode yaml: We want to have the fullnode yaml file stored here to populate with our preferred config

   • Sui systemd file: We want to have a systemd that our playbook can use to create a systemd service for our node.

5. Add build task

   Now to the juicy stuff! These task folders bundle a specific set of sub-tasks(modules) related to the parent task. In this case, it is setting up our Sui environment and building the source(Sui is a spaceship so be prepared to wait while it builds 🚀). Ansible has two types of modules you can use for tasks - builtin and community. BuiltIn are maintained by ansible and you can expect them to be in working condition, community modules can be developed by anyone and should be reviewed before using them. Modules are a way to complete specific actions, an example would be downloading apt packages which would look like the following:

   - name: install apt packages required for building sui binary
     apt:
       install_recommends: no
       force: yes
       name:
         - curl
         - git-all
         - libssl-dev
         - libclang-dev
         - tzdata
         - ca-certificates
         - build-essential
         - pkg-config
         - cmake
         - acl   # needed to prevent errors when building as unprivileged user via become
       state: present
       update_cache: yes

   Here is the order of tasks we will be doing in this build document and the modules they are utilizing:

   • Create an unprivileged sui user - (user module)

   • Check if cargo is installed - (shell module)

   • Download rust Installer - (get_url module)

   • Install rust/cargo - (shell module)

   • Update bashrc for rust/cargo for sui user - (lineinfile module)

   • Update apt-get repo and cache - (apt module)

   • install apt packages required for building sui binary - (apt module)

   • Check if sui is installed - (shell module) Note: notice the two different ways of using shell

   • Install sui - (shell module) Note: notice the two different ways of using shell

6. Add sui configuration task

   This file pulls the most recent sui repository from the devnet branch(not necessary but I added it anyhow) and configures your node with a fullnode.yaml file and the genesis.blob.

   Here is the order of tasks we will be doing in this build document and the modules they are utilizing:

   • Clone the SUI repo - (git module)

   • Creates fullnode directory- (file module)

   • Copy fullnode yaml file - (template module)

   • Get genesis blob - (command module)

   • Allow all access to tcp port 9000 - (ufw module)

   • Allow all access to tcp port 9184 for metrics - (ufw module)

7. Add systemd configuration task

   This will set up and start a systemd service to run the Sui fullnode

   Here is the order of tasks we will be doing in this build document and the modules they are utilizing:

   • Deploy /etc/systemd/system/sui.service - (template module)

   • Enable and start sui service - (systemd module)

8. Package in sui_fullnode role, configure tasks and populate root file

   We now want to tie all this together. We add the previous three tasks to a parent YAML file in the sui-fullnode role folder and we create the entry point for ansible in the root as main.yml. We should note that there are a couple of commented-out lines - Stay tuned for subsequent tutorials securing a server and adding a monitoring stack 👩🏽‍💻

Conclusion

This post should help you get familiarized with ansible and see how easy it is to create reproducible infrastructure. It will be a primer for more complex operations as the repository grows and we add validator functionality, add server security and implement a monitoring stack.

It may also act as an entry point to learning about Sui as a protocol or allow you as a node operator to have a more streamlined workflow. It's still very early days in the blockchain/web3/crypto space. Sui is likely to play a large part in developing the technology and scaling it to billions of users.

Stay tuned to hear about the next articles in the pull-requests-with-zoz series

This is one of many articles based on the concept of pull-requests-with-zoz

https://paragraph.xyz/@0xZOZ/pull-requests-with-zoz

Cover Image

The image above(like my PP) is created with abraham.ai - an AI artwork generator

SEED: Automated Sui node

About

Twitter: @0xzoz

GitHub: @0xzoz

Mirror: @0xzoz

Discord: @zoz.eth#6952

ENS: zoz.eth