Say hello to the barrel weekly 03!
It was 5am in the morning. I was contemplating the code I did during the last week. A huge amount of code actually. Like in the good old days. Lot of improvements over the current source code, some new features, a new couchapp engine, … But this code was built around the current source base inherited from Apache CouchDB. Actually a pile of hacks over another pile created along the years. With some parts quoted as examples of what you should not do in Erlang in some slides around.
It was 5 am in the morning, and I quietly pointed my mouse to the folder and moved it to the trash. I emptied the trash. I simply decided that adding a new pile of hack and calling it 2.0 was not the way I wanted to follow.
The good parts of this work are not lost though. I won’t start from scratch. Barrel goals are still the same. Like I said in my last talk at Erlang Factory Berlin, Barrel is an effort to maintain the couch api but focusing on data locality (put/match the data next to you) and P2P. The code I deleted this morning was about that, but it was still suffering its heritage. It was also painful to work on it.
Instead I decided to take the good parts of Apache CouchDB in Barrel, cut, rewrite some parts and simplify the code in a deeper way than I did until now. For those who follow me I already started such work in the happy and naive days with opencouch. A project that was promptly attacked by some. Which created a lot of frustration and mostly killed the idea of having a stupid and simple data node handling documents and replication tasks, able to communicate with other nodes and device in a peer-to-peer manner. Until now.
The new barrel core will mainly focus on:
- storage improvements: concurrency and performance
- replicator improvements: improve the management of the replication tasks and the stability of the engine (a lot of work in the current source have been done already to make sure the replication can’t freeze, but more could be done)
- simplify the view engine. The view engine aka Map/Reduce indexer in the current source is a lot complicated to make sure it can be used as a generic interface for other indexers. In the facts only geocouch is using that feature (and maybe mango).
- support the parts we need from the CouchDB API to allow the replication
- add missing features for a true P2P data node able to work with others
- user and developer documentation
I will come back on each of these points along the time.
The new barrel core is based on these ideas and some of the brand new code already committed in the current. To not interfere with the actual source code a new repository is created. First bits should land by tomorrow. Current code is moved to a new “barrel-platform” repository where the release will be maintained. Once the new barrel core is stable, the barrel-platform release will point to it.
On a user point of view the new release won’t change anything, regression tests will make sure you will be able to use your databases without losing any data. It will just be simpler to use.
In the meantime the current source code code will be maintained, with new fixes and one new coming feature. The barrel-platform versions 013-014 will be maintained during one year in the branch “couch-1.x”.
Keeps posted. Feedback is welcome