Bacalhau Project Report - Dec 5, 2022
Networking, SQLite, Direct Node Communication, IPLD Invocation Specs, Simulator wallets.
A nice week for design docs and several designs starting to see prototype implementations in code!
Networking 💻🌐🖥️
We now have a rather excellent design doc for Networking. Bacalhau jobs currently are not able to dynamically access network resources. They can access network resources up front by declaring them as IPFS or HTTP(S) downloads that are shared amongst all nodes.
There are a few reasons for this:
Security. No networking makes it easy to protect against botnets and Bac jobs running DOS attacks.
Determinism. Endpoints can return different results to the same request from different nodes. (Atm we don’t actually do anything about this even with our up-front download, but this is at least a solvable problem for us currently.)
There are lots of use cases we currently can’t meet because of this restriction. The design doc explores how to lift these restrictions, which is exciting because it will unlock many more use cases for the network!
SQLite 💾💿💾
We now have the SQLite localdb driver passing basic tests, that is, we can now record job and event information on disk rather than only in-memory! This is exciting, since it means we’ll eventually be able to have the network stop being so damn forgetful — currently every time we release a new version of the software or a node reboots for whatever reason, it forgets all that came before it! It’s like our nodes constantly have amnesia.
Of course, we won’t be able to turn on the SQLite localdb driver until our work on API stability & backward compatibility is complete, but it’s also forming a valuable component of the FIL+ dashboard project.
Direct Node Communication 🎯🏹💘
We are taking steps to avoid nodes having to broadcast to all other nodes every time they want to say something to just one other node. This will significantly improve the scalability of the network. It’s exciting to see the first steps towards this landing in code form now as a direct libp2p protocol rather than over GossipSub.
Simulator is alive (͡ ° ͜ʖ ͡ °) 💊🤑
We now have the first signs of life in terms of wallet balances in the Simulator, with the first evidence that it’s processing real (imaginary) wallet balances:
Ethereum example ❤️
We have a really nice video showing how you can use Bacalhau to compute over Ethereum data:
What’s next? ⏭️
Simulator able to detect “bad people”, at least for one definition of “bad”
Python SDK and schema wrangling
First pass on new CoD WG aligned invocation spec
Questions/comments? Let us know!
Thanks for reading!
Your Humble Bacalhau Team