Bacalhau and the Bluesky API

(04:22) We've built a bot which lets you deploy Bacalhau jobs with the Bluesky API - because it's fun!

Quick Key Points

• We’ve deployed a Bacalhau Bot on Bluesky that you can use to deploy distributed workloads

• You can find it at @jobs.bacalhau.org on Bluesky

The What, Why, and How

Last Saturday, I took myself along to BarCamp London and I brought the Expanso Football along for people to tinker with.

I learned 2 things while I was there…

1. People were super-curious about Bacalhau, which is always nice to hear.

2. Everyone is on Bluesky now.

More than a handful of talks that day were about Bluesky and their API. The Expanso team loves tinkering with APIs - it’s why we build Bacalhau from the API-up. Every Bacalhau feature starts with the API and it stands as the foundation of everything else that comes afterwards.

And so, while hearing all of these great things about the Bluesky API, I thought to myself “I wonder if I can deploy Bacalhau Jobs with a Bluesky post?”

Well, dear reader, you can, and I have, and now you can too!

Bacalhau, CLIs, APIs, and Bluesky

We’ve had a Bluesky profile for a little while now, and we’re just starting to figure out what to do with it (you should definitely follow us if you haven’t already).

One of the most common ways that people run Bacalhau Jobs is using the Bacalhau CLI with something like…

bacalhau job run job.yaml

And it occurred to me that it’s not a million miles away to instead have something like…

@jobs.bacalhau.org job run <SOME_JOB_FILE_SOMEWHERE>

When running a command on the CLI, you’re actively executing an application on your computer, and you likely have everything you need for it to run locally. When posting to an account on a social network, you’re just sending text from place A to place B. We would need to have some code sitting somewhere that could:

1. Detect when someone’s @mentioned our Bluesky account

2. Parse the message and check that it’s asking us to run a job

3. Get the job file from somewhere

4. Send that job to the Bacalhau public network

5. Wait for the results, and respond with them to the original poster.

So, let’s do it!

Building a Bacalhau Bluesky Bot

(If you want to just check out the code we used to build this, the repo is publicly accessible. Check it out!)

With our Bot, we’re gonna shoot for something like the following architecture:

We wrote our Bot in Go, and we built a few small libraries to keep our logic simple. Here’s the main() function of our application with some key points highlighted that we’ll go through super-quick:

Point of Interest 1: Getting Authenticated

The very first thing we need to do is authenticate with the Bluesky API. We can do this with our Bluesky username and password to get a JSON Web Token which we can then use to access our feed and post replies to people

Once we have our session, we can then interact with Bluesky.

Point of Interest 2: Getting our Notifications

The next thing our bot needs to do is check whether or not someone has mentioned our Bluesky account. At the moment, Bluesky doesn’t have a “Mentions” API, but it does have a Notifications API which gives us a list of interactions (likes, reposts, mentions etc.) that we can filter by the “Reason” for each post. In our case, the “Reason“ we’re looking for is a “mention”.

Point of Interest 3: Filtering our Notifications

As we iterate through our notifications, we run a couple of checks to see if we want to do anything about a post. For example, we don’t want to respond to old posts, we don’t want to respond to ourselves, and we need to keep a track of things we have responded to so we don’t respond ad infinitum - so we check:

1. The the notification is from after our bot started

2. That it contains a Bacalhau command with a valid URL

3. That we haven’t already responded to the post

Point of Interest 4: Sending the Job to Bacalhau and Getting the Results

If a post meets all of the above criteria, then we do the following:

1. Get the linked Job YAML

2. Download it to the bot

3. Parse and restructure it to work with the Bacalhau API

4. Submit the Job to the bot’s Bacalhau Network

5. Wait for 20 seconds, the get the results

6. Send the results to the original poster

…and if everything works, you’ll get the job ID, execution ID and STDOUT of the job posted back to you - just like the following:

Or, if things are a little slow, you’ll get some instructions to help you follow up:

Under the Hood

We’re super-fortunate that we already had a couple of things set-up that made creating this bot super simple.

As I mentioned earlier, Bacalhau is designed API-first, so no matter how you want to create a Job or get updates, so long as you can work with HTTP, you’ve got everything you need to get going!

We also have the benefit of a powerful Bacalhau Network running that lets people experiment with the power of Distributed Computing. You can’t run absolutely anything on our bot’s network, but you can definitely run a lot - so give it a go!

As for the bot, it has a few limitations built in:

1. The URL that links to a Job file must be both publicly accessible, and a YAML file adhering to the Job Spec.

2. The Bacalhau Bluesky Bot will only wait for 20 seconds before getting results. If your job takes longer than that to execute, you’ll get instructions on how to get your results with the Bacalhau CLI.

3. The Bacalhau Network that our bot accesses doesn’t have networking enabled, so you won’t be able to deploy jobs which make external calls to the internet

But we think it’s still possible to have a lot of fun with this, even with these limitations.

Give it a go!

Don’t just take my word for it, head on over to your Bluesky account and submit a job! To get you started you can run something like…

@jobs.bacalhau.org job run https://smt.codes/job.yaml

…which is just a “Hello, world” - but as we all know, a “Hello, world.” is often just the first step to great things!

Some things to be mindful of:

1. Jobs that run for 60 seconds will be stopped by the bot, so if you’re looking to run a workload longer than that check out our docs on how to run your own network.

2. If a Job finishes in less than 20 seconds, the STDOUT of that Job will be posted to you on Bluesky - but you can retrieve way more information about the executions with the Bacalhau CLI

Conclusion

APIs give developers the ability to use existing technologies and services together in new and creative ways, and that’s why we built one into Bacalhau from Day 1. We’re thoroughly enjoying our time on Bluesky these days, and we hope this little bit of code inspires new and fun ideas to use both Bacalhau and Bluesky. We’re going to be building more withing the Bluesky ecosystem, so keep your eyes on upcoming blogs over the next few weeks!

Get Involved!

We welcome your involvement in Bacalhau. There are many ways to contribute, and we’d love to hear from you. Please reach out to us at any of the following locations.

• Website Expanso

• Website Bacalhau

• Bluesky Bacalhau

• Twitter Bacalhau

• Twitter Expanso

• Slack

• LinkedIn

• Careers Page

Commercial Support

While Bacalhau is open-source software, the Bacalhau binaries go through the security, verification, and signing build process lovingly crafted by Expanso. You can read more about the difference between open-source Bacalhau and commercially supported Bacalhau in our FAQ. If you would like to use our pre-built binaries and receive commercial support, please contact us!