Introducing Andaime: Rapidly Prototype, Build and Manage Private Bacalhau Networks

Discover how to prototype, build and manage private Bacalhau networks quickly and efficiently. This open-source tool streamlines AWS deployments, making multi cloud management straightforward.

, and

Jul 25, 2024

Today, we're thrilled to announce the alpha release of Andaime, an open-source command-line interface (CLI) tool that makes it incredibly easy to create, manage, and teardown private Bacalhau networks. Built with Go, Andaime allows you to deploy scalable networks with hundreds of nodes in just minutes.

What is Andaime?

"Andaime" means "scaffolding" in Portuguese, and that’s exactly what it does. It sets up all the essential resources and configurations needed to launch Bacalhau networks with a single CLI command.

Why did we build it?

While we've provided developers with tools and documentation to streamline Bacalhau network creation, provisioning and managing the underlying cloud infrastructure remains time-consuming. Andaime eliminates this overhead by automating the entire lifecycle of cloud resources. Currently, the alpha release supports AWS, but we're working on adding compatibility with GCP and Azure. Notably, Andaime already supports every AWS region.

When you create a network with Andaime, it automatically instantiates and configures the necessary VPCs, Security Groups, Route Tables, Volumes and EC2 instances to ensure smooth Bacalhau network operations.

A common issue with multi-region cloud deployments is the risk of forgetting to terminate resources, which can lead to unnecessary costs. Andaime tackles this by meticulously tagging each resource, making management, updates, and clean-up straightforward and efficient.

How to use Andaime

If you’re on macOS or Linux, getting started is as simple as downloading the latest release binary from our GitHub repository.

For other platforms, or if you prefer building Andaime from scratch, you can follow these instructions:

# Clone the repository and navigate into the directory
git clone https://github.com/bacalhau-project/andaime.git

cd andaime

# Verify dependencies (Go Modules)
go mod tidy
go mod vendor
go mod verify

# Build Andaime
go build .

# Test the build
./andaime version

Deploying a Bacalhau Network

Credentials

Andaime works natively with the AWS API, so you don’t need the AWS CLI installed or configured on your system. You do, however, need some valid AWS credentials.

If you have the AWS CLI installed and configured on your machine, then Andaime will use the default credentials that the AWS CLI is configured with to authenticate against your AWS account.

If you don’t have the AWS CLI installed, you can use environment variables to pass them through to Andaime as you run commands with:

export AWS_ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID>

export AWS_SECRET_ACCESS_KEY=<YOUR_SECRET_ACCESS_KEY>

Creating your first network

Let’s take things for a spin!

We'll start by creating a simple network with one orchestrator node and one compute node, both of which will be `t2.medium` instances running Ubuntu 22.04 with Bacalhau configured on startup.

Run:

./andaime create \
    –orchestrator-nodes=1 \
    –compute-nodes=1 \
    –target-regions=eu-west-2,us-east-1 
    –aws-profile="<OPTIONALLY SET YOUR AWS PROFILE FOR CREDS>"

This will create:

1 orchestrator node
1 compute node

The --target-regions flag enables you to select the regions you would like to deploy Bacalhau nodes to. They are selected sequentially in a round-robin fashion. While all AWS regions are supported, some instance types might be unavailable in certain Availability Zones, potentially leading to occasional failures. In such cases, Andaime will retry in another region to try and ensure your network reaches the desired size.

If you wish to see the resources as they are being created, you can add a --verbose flag to your commands to see what’s being created as Andaime builds your network for you.

Once that command is running you should see something like:

Connecting to your network

After creation, you can connect to the orchestrator node and list all connected compute nodes. Locate the line in the output that shows the orchestrator's IP address:

Orchestrator nodes created with IPs: [52.56.218.249]

Copy the IP address in the output and then run:

bacalhau config set node.clientapi.host <COPIED_IP_ADDR>

This will configure your bacalhau client to run against your newly deployed cluster.

If you run a command like:

bacalhau node list

You will now see a list containing one orchestrator node and one compute node.

Hurrah! We’ve spun up a Bacalhau network in next to no time at all!

Listing Your Resources

Now that we have our network up and running, we might like to know exactly what infrastructure has been provisioned to run it.

You can get a report of all resources created by Andaime across AWS regions:

./andaime list

Which should output something like:

Destroying your resources

Just as easily as you created the network, you can dismantle it and its associated resources using:

./andaime destroy

And that’s it!

Get involved with Andaime!

Andaime is open source, and we’d love your help in building it out for other platforms.

Andaime is a brand new project written from the ground up with lots of new features and ideas to come. This makes it perfect as a first project to start your open source contribution journey with.

Right now, we’re particularly interested in people who may want to help us deploy on Azure and GCP, but if there is another Cloud Platform provider that you’d like to work on supporting, we’ll be more than happy to have your support!

We’re also eager to hear your ideas. What features would make Andaime even better for building and experimenting with Bacalhau? Join us in the Bacalhau Slack and let us know!

If you have questions about contributing to Andaime, or find any bugs, create an issue on GitHub.

Get Involved with Bacalhau!

We welcome your involvement in Bacalhau. There are many ways to contribute, and we’d love to hear from you. Please reach out to us at any of the following locations.

Commercial Support

While Bacalhau is open-source software, the Bacalhau binaries go through the security, verification, and signing build process lovingly crafted by Expanso. You can read more about the difference between open-source Bacalhau and commercially supported Bacalhau in our FAQ. If you would like to use our pre-built binaries and receive commercial support, please contact us!