Introducing Bacalhau 1.3.2
(02:51)
As part of our ongoing efforts to continually improve the Bacalhau project, we are proud to usher in Bacalhau 1.3.2.
Continuing on from the advancements brought by Bacalhau 1.3.0 and 1.3.1, this release brings a host of significant improvements, setting us up fanatastically for 1.4.0 in the near future:
With 1.3.2, we’re releasing:
Better Node filtering based on capacity
Enhanced Job Queueing
Improved Configuration Management
A Refined Networking Client with split producer and consumer concerns
Job Queueing with Bacalhau
In earlier releases, jobs will fail immediately if no nodes are found within the network that match the defined specifications of the job. Here, these jobs can now have a specific wait time assigned to queue for available nodes.
This feature starts up disabled by default. You would need to enable it in your Job specification or within your requester node, using the ‘QueueTimeout’ parameter.
Here’s an example of a job spec that waits an hour for available nodes
Type: batch
Count: 1
Tasks:
- Name: main
Engine:
Type: docker
Params:
Image: ubuntu:latest
Entrypoint:
- /bin/bash
Parameters:
- -c
- sleep 90
Timeouts:
QueueTimeout: 3600The same parameter can be set within the requester node to set defaults for any requested jobs without a defined queue time
Node:
Requester:
JobDefaults:
QueueTimeout: 3600s
Scheduler:
QueueBackoff: 1m0sPlease note: Job Queueing does not apply to Ops Jobs, as these jobs are designed to only utilize currently available and matching nodes for urgent use cases.
We’re planning on writing a piece on job queuing and the different kinds of timeouts in the near future, so watch this space to learn more!
Node Filtering
Along with the new job queueing feature, we’ve implemented management improvements allowing you to filter nodes that have no immediate available capacity or already have a large queue. To date, nodes were ranked based on their capacity, but with this new implementation, nodes can be excluded from consideration based on their 'NodeOverSubscriptionFactor' property, ensuring a more efficient job distribution.
The NodeOverSubscriptionFactor parameter is a numerical value representing the sum of a node’s total active capacity and queue capacity. With a default value of 1.5, your node can handle 50% more of its total capacity before being excluded from job queueing consideration. This ensures that jobs are only assigned to nodes that can realistically manage additional workload, optimizing the overall efficiency of the network.
Configuration Management
We have implemented better configuration management, enabling more efficient control over how your nodes are implemented across environments and instances. Moving away from a global configuration, you can define how your Bacalhau node should be set up with each instance with localized and isolated configuration management. This enables more predictable and precise executions for your specialized use cases.
In Summary
As we are with each of our releases, we’re super proud to get this enhanced version of Bacalhau into the hands of developers. We think this latest round of features will greatly simplify how developers interact with Bacalhau, whilst giving them a greater number of controls on exactly how they want jobs to execute, and when.
A jam-packed 1.4.0 is just around the corner too, so watch this space!
How to Get Involved
We're looking for help in various areas. If you're interested in helping, there are several ways to contribute. Please reach out to us at any of the following locations.
Commercial Support
While Bacalhau is open-source software, the Bacalhau binaries go through the security, verification, and signing build process lovingly crafted by Expanso. You can read more about the difference between open-source Bacalhau and commercially supported Bacalhau in our FAQ. If you would like to use our pre-built binaries and receive commercial support, please contact us!
