Better Monitoring in Bacalhau v1.3.0

5 Days of Bacalhau (4 min)

Keeping tabs on Bacalhau just got simpler. We've updated Bacalhau with new tools and metrics to help you monitor your nodes more effectively. Here’s what’s new:

A Toolkit for Easy Monitoring

To give you a better understanding of your node's performance, we've added new metrics. These include tracking job submissions, publishing times, upload durations, and more. These metrics are designed to offer a deeper insight into the operational aspects of your nodes, helping you to optimize performance and troubleshoot issues efficiently. We're also introducing a telemetry suite that uses docker-compose to simplify setup. It bundles open-telemetry, Prometheus, Grafana, and Jaeger into containers for comprehensive data collection and analysis from your Bacalhau nodes. You'll find all you need to get started in our GitHub README or in the following section.

• job_submitted: Tracks the number of jobs submitted.

• job_publish_duration_milliseconds: Time to publish a job in milliseconds.

• bacalhau_node_info: Static metric with details about a Bacalhau node.

• node_network_transport: The node network transport type.

And many more. Read more about all the new metrics in our release notes.

Example Usage

Step 1 - First of you need to start the containers:

docker-compose up

Step 2 - Export collection endpoint for bacalhau

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

Step 3 - Start Bacalhau

bacalhau serve --node-type=compute,requester

Step 4 - Open Browser

Grafana: http://localhost:3000

• Username: admin

• Password: admin

Jaeger: http://localhost:16686

Step 5 - Clean up

Remove volumes associated with containers to reset state.

Step 6 - Saving Changes to a Grafana Dashboard

Export dashboard data from Grafana as json and save it to file

./grafana/provisioning/dashboards/dashboard.json

Step 7 - Check out Best Practices for Telemetry

OpenTelemetry In Bacalhau will give you an insight into the best practices of telemetry.

---

Start Exploring

With these updates, monitoring and optimizing your Bacalhau nodes is now more straightforward. Check out our release notes for an in-depth review of the new metrics added.

The introduction of better monitoring will enable more users to track jobs deployed to Bacalhau. If you’re interested in learning more about distributed computing and how it can benefit your work, there are several ways to connect with us. Visit our website, sign up to our bi-weekly office hour, join our Slack or send us a message.

How to Get Involved

We're looking for help in various areas. If you're interested in helping, there are several ways to contribute. Please reach out to us at any of the following locations.

• Website Expanso

• Website Bacalhau

• Twitter Bacalhau

• Twitter Expanso

• Slack

• LinkedIn

• Careers Page

Commercial Support

While Bacalhau is open-source software, the Bacalhau binaries go through the security, verification, and signing build process lovingly crafted by Expanso. You can read more about the difference between open-source Bacalhau and commercially supported Bacalhau in our FAQ. If you would like to use our pre-built binaries and receive commercial support, please contact us!