Hotdog? Not Hotdog? The Bacalhau Bluesky Bot Knows!
(03:02) Sometimes, you just want to use cutting-edge AI to know whether or not you're looking at a hotdog. Y'know?
You may have noticed recently that we’ve been doing a little bit of work with the Bluesky API and Bacalhau:
First, we launched the Bacalhau Bluesky Bot as a helpful introduction to distributed computing through a platform people are familiar with. Then, we added the ability to run distributed ML classification jobs through that same interface!
And naturally, we shared with people! But of course, wherever someone uses AI to detect things in images, invariably the question is asked: “Can it do hotdog/Not hotdog?”
In fact, that was one of the first messages we got sent in a group chat!
Wait… What?
For those of you not familiar with the challenge of hotdog/not hotdog, it’s a reference to a scene from the HBO show “Silicon Valley” where the team have been trying to build an AI model that could detect a food item in a photo, and then based on that classification, estimate the nutritional value of the items in a meal.
This kind of model is what’s known as a “multiclass” classification model - and it’s used where you want to be able to classify something from a set of classes that has 3 or more items. Perfect if you’re trying to figure out what kind of food item is in a photo out of all the possible food types in the world!
What’s really funny about the scene is that Jian Yang instead creates what’s known as a “binary” classification model - a model that’s vastly simpler to train, and can tell whether an image is or isn’t of something. Super-useful for detecting whether there’s a fault in a production line - but not exactly useful if you want to know more about the nutritional value of your food than “this is/is not a hotdog”.
Technically, Jian Yang met the brief - just not in the way that was needed!
So… that’s what we’ve built!
The Bacalhau Bluesky Bot has a multiclass classification model built into it: YOLOv5. YOLO is an incredibly powerful and simple image classification model that can detect 1000s of different kinds of items - and using it is as simple as posting an image to the bot with the word “classify”
And after a couple of seconds, you get back a post of everything the YOLOv5 detected in the image, and all of those items highlighted in said image.
Pretty neat! We live in a time where AI is as simple to get started with as writing a single word and posting an image!
Buuuuttttt… one of the classes in all of the classes that YOLO has available to it is “hotdog” - so it’s not a huge change to tell our ML job to look specifically for hotdogs in an image and tell us whether or not there’s one there.
When you’re presented with an opportunity to have a little fun whilst also being educational, you take it. And so from today, you can now post an image to the Bacalhau Bluesky Bot and ask whether or not there’s a hotdog in there.
Just head over to Bluesky and post the following:
@jobs.bacalhau.org hotdog?
… with an attached image and just as with our existing image classification flow, after a few seconds, you’ll get back an image of whether or not the photo is one that contains a hotdog - but in the same style as Jian Yang’s app in Silicon Valley.
Cool! How does it work?
Very little has changed between our original classification job workload and hotdog/not hotdog.
The only real difference this time is that if you post an image to the bot with the text “hotdog?” it’ll slightly change the job it dispatches to the Bacalhau network to include an environment variable HOTDOG_DETECTION which will is then to true if we do, in fact, want to detect a hotdog.
Once Bacalhau schedules that job and starts running it somewhere on our network, the code reads that environment variable - and if we’re trying to detect a hotdog it’ll run the same classification job, but filters down the results to only detect whether there’s a hotdog.
If a hotdog is detected - huzzah! We paint the hotdog overlay on top of the image and send it back to the poster. If not, we do the inverse.
And that’s how you hotdog/not hotdog!
Try it out for yourself!
Giving it a go is simple as sending a single Bluesky post the the Bacalhau Bluesky Bot. Just head on over to Bluesky and post…
@jobs.bacalhau.org hotdog?
… with the image you’d like to detect the hotdog in, and we’ll let you know!
Get Involved!
We welcome your involvement in Bacalhau. There are many ways to contribute, and we’d love to hear from you. Please reach out to us at any of the following locations.
Commercial Support
While Bacalhau is open-source software, the Bacalhau binaries go through the security, verification, and signing build process lovingly crafted by Expanso. You can read more about the difference between open-source Bacalhau and commercially supported Bacalhau in our FAQ. If you would like to use our pre-built binaries and receive commercial support, please contact us!
Subscribed
