slash dev slash null

stuff about puters

Month: May, 2017

Corfu: Copy Compaction (Part 5)

In the last post we discussed striping a log across many mirrored sets of disks for write concurrency. In this post we will discuss deletes from a global log and use copy collection techniques to free the disk space.  Read the rest of this entry »

Corfu: Stripe & Mirror (Part 4)

The last post introduced a global sequencer to reduce contention on writes to the end of a log file. The idea is to be able to distribute the log file across many machines. In this post we will scale up or fictional example using by striping and mirroring data much like a RAID 10 disk configuration.  Read the rest of this entry »

 Corfu: Global Sequencer (Part 3)

In the last post we outlined how to linearizable consistency without a global buffer. This introduced a write contention challenge. In this post we will get into Corfu’s simple fix for this issue.  Read the rest of this entry »

Corfu: Linearizable Consistency (Part 2)

This the second post in a series about Corfu. In this post I will outline the basic technique it uses to achieve linearizable consistency. We will then look at a simple adaption that we will later see can be used to scale the system which introduces an additional challenge to solve.  Read the rest of this entry »

Corfu: Scaling log replication (Part 1)

Corfu: A distributed shared log is a paper running to 25 pages brought to my attention by The Morning Paper. Corfu takes log replication to a whole new level. Corfu achieves linearizable consistency of a fault tolerate distributed log using very simple write-once techniques. The real eye open is that it implements a distributed log interface without an IO bottleneck. The paper proudly points out that after 40 years of research this is a game changer. This series of posts will build up a simple fictional example that demonstrates the key techniques. Then I will glue it all together to describe how Corfu itself works and demystify the role Paxos plays. Read the rest of this entry »