Bacalhau now available on AWS Marketplace

and

Dec 03, 2024

We’re thrilled to announce that Bacalhau is now available on AWS Marketplace, making it easier than ever to deploy and manage your Bacalhau Networks in the cloud. With just a few clicks, you can harness the power of Bacalhau for distributed computation, benefiting from AWS's robust infrastructure. Here’s everything you need to know about this launch, why it’s a game-changer, and how to get started.

Check it out on AWS Marketplace here:

Graviton instances: https://aws.amazon.com/marketplace/pp/prodview-7ejoqbstr63ds
Intel instances: https://aws.amazon.com/marketplace/pp/prodview-36yd5772rxrxs

What does this mean for you?

With Bacalhau on AWS Marketplace, deployment, configuration, and scaling, are vastly simplified offering a production-ready distributed compute cluster that integrates seamlessly with AWS services.

Whether you're processing large datasets, running distributed simulations, or managing complex workflows, Bacalhau on AWS enables:

Ease of Use: Go from zero to fully functioning clusters in minutes.
Scalability: Auto-scaling compute nodes to meet your workload demands.
Enhanced Security: Integrations with AWS Secrets Manager and IAM enable secure operations within your existing infrastructure.
Flexibility: Support for Intel (x86_64) and ARM (Graviton) architectures.
Cost Efficiency: Dynamic resource management ensures you're only paying for what you need.

Walkthrough

Getting started with Bacalhau on AWS Marketplace couldn’t be simpler. To get you started, we’ve put together a short guide to walk you through the options.

Step-by-Step

If you prefer to read instead of watching the above video, you can follow the same steps here. After deployment is successful, you will have a Bacalhau Network with the following architecture:

Find Bacalhau on AWS Marketplace

Visit AWS Marketplace.
Search for "Bacalhau" and select your preferred architecture (Intel or Graviton), or click the links at the top of the article.
Click “Subscribe” and then “Launch.”

Configure Your Deployment
Before deployment, you can customize the following parameters:

Version: Select your Bacalhau version (e.g., v1.5.0) or keep it as latest to install the most recent, stable version of Bacalhau
Secret Key: Set a secure cluster authentication key.
Instance Types: Choose instance sizes for orchestrator and compute nodes.
Node Counts: Define min, max, and desired compute nodes.

Access and Start Computing

Open the Web UI at http://<orchestrator-ip>:8438.
Use the Orchestrator IP to connect using the Bacalhau CLI.
Access job results - backed by a dedicated S3 bucket, so your storage can grow as your usage does.

And that’s it!

The Best of Bacalhau with the Best of AWS

When deploying Bacalhau through AWS, you don’t just get the best-in-class distributing software, you also benefit from the vast array of AWS Services to make sure that your network remains secure, reliable, performant and cost effective.

Bacalhau on AWS grabs these advantages with both hands by integrating nodes tightly with AWS AppConfig, AWS Secrets Manager, encrypted EBS volumes, IAM, and S3 encryption, all while maintaining the flexibility and portability that Bacalhau has at its heart.

AWS AppConfig

With AWS AppConfig, dynamic configuration management is streamlined, enabling instant propagation of configuration changes across your cluster. Updates take effect in under 60 seconds without requiring a restart, ensuring minimal disruption and maximum responsiveness in dynamic environments.

AWS Secrets Manager

Hybrid cloud compatibility is easily achieved using AWS Secrets Manager, which securely connects on-premises compute nodes. This capability allows seamless integration of on-premises resources with cloud-based workflows, enabling efficient hybrid cloud operations.

Autoscaling

Resource management is optimized through automatic scaling and storage management. Auto-scaling dynamically adjusts resources to meet workload demands, ensuring optimal performance and cost-efficiency.

Security and reliability are foundational to Bacalhau. Encrypted EBS volumes, IAM roles, and server-side S3 encryption provide robust data protection. Additionally, cluster authentication is made seamless and secure with AWS Secrets Manager, ensuring both simplicity and adherence to security best practices.

What’s Inside the Box?

We’ve worked hard to make setting up a Bacalhau Network in AWS as simple and hands-off as possible, but if you want to take a look inside the box (well, instances), you can SSH into each node that the Marketplace template has provisioned for you.

Configuration files can be found at /etc/bacalhau. Specifically:

config.yaml
aws-config.yaml
secrets.yaml

Each of these files are managed by either Bacalhau, or an AWS service. They should not be modified, but if you need change something or are looking for help, please do reach out to us.

Additionally, Bacalhau will have been set up as a service on the host operating system. This means you can view logs using journalctl.

For the main service logs, you can run:

journalctl -u bacalhau

For the logs of the Bacalhau update service, you can run:

journalctl -u bacalhau-config-watch

And finally, if you want to check your AppConfig logs, you can run:

journalctl -u aws-appconfig-agent

Conclusion

By leveraging AWS infrastructure, Bacalhau makes high-performance distributed computing accessible to everyone. With automated setup, dynamic scaling, and robust security, Bacalhau ensures your compute clusters are always ready to handle your workload.

With the AWS Marketplace you can have a production-ready Bacalhau cluster with automated scaling, secure configuration management, and integrated AWS services in minutes, unlocking the full potential of your compute and data.

Deploy Bacalhau on AWS today and experience the future of distributed computation.

Graviton instances: https://aws.amazon.com/marketplace/pp/prodview-7ejoqbstr63ds
Intel instances: https://aws.amazon.com/marketplace/pp/prodview-36yd5772rxrxs

Get Involved!

We welcome your involvement in Bacalhau. There are many ways to contribute, and we’d love to hear from you. Please reach out to us at any of the following locations.

Commercial Support

While Bacalhau is open-source software, the Bacalhau binaries go through the security, verification, and signing build process lovingly crafted by Expanso. You can read more about the difference between open-source Bacalhau and commercially supported Bacalhau in our FAQ. If you would like to use our pre-built binaries and receive commercial support, please contact us!