Building a Distributed ETL Pipeline Using Bacalhau and MongoDB Atlas
(1:35)
Here at the Bacalhau Project, we relish every opportunity to share knowledge and showcase what can be built with Bacalhau. We believe that putting knowledge and tools to quickly build amazing things in the hands of developers unlocks the full range of creativity and expertise that the Open Source community has to offer.
To that end, we're pleased to share a brand new case study, workshop, and tutorial video that will teach you how to build and run a distributed Extract-Transform-Load (ETL) Pipeline using Bacalhau, MongoDB, and MongoDB Atlas with Terraform on Google Cloud.
That might sound like quite a bit of stuff to build, but we've worked hard to keep things as straightforward and simple as possible. If you follow the workshop, it should take no more than 20 minutes from start to finish to get your own ETL pipeline up and running.
Once spun up, you'll have a Bacalhau-powered distributed ETL pipeline which records and filters key CPU and Memory usage events from across your infrastructure and stores them in an M0 MongoDB Atlas database where you can explore the events in depth.
MongoDB Atlas is a managed version of the enormously popular MongoDB database which powers countless applications across the world, enabling developers to interface with, and scale their database usage without having to change their applications. This makes it a perfect coupling for experimenting with Bacalhau jobs, and with the M0 tier, you can build and experiment for free.
This will the first in a continuing series of work we'll be producing over the coming weeks and months to help developers get to grips with the power of Bacalhau when coupled with other best-in-class technologies. So, watch this space!
How to Get Involved
We're looking for help in various areas. If you're interested in helping, there are several ways to contribute. Please reach out to us at any of the following locations.
Commercial Support
While Bacalhau is open-source software, the Bacalhau binaries go through the security, verification, and signing build process lovingly crafted by Expanso. You can read more about the difference between open-source Bacalhau and commercially supported Bacalhau in our FAQ. If you would like to use our pre-built binaries and receive commercial support, please contact us!
