slash dev slash null

stuff about puters

Corfu: Copy Compaction (Part 5)

In the last post we discussed striping a log across many mirrored sets of disks for write concurrency. In this post we will discuss deletes from a global log and use copy collection techniques to free the disk space.  Read the rest of this entry »

Corfu: Stripe & Mirror (Part 4)

The last post introduced a global sequencer to reduce contention on writes to the end of a log file. The idea is to be able to distribute the log file across many machines. In this post we will scale up or fictional example using by striping and mirroring data much like a RAID 10 disk configuration.  Read the rest of this entry »

 Corfu: Global Sequencer (Part 3)

In the last post we outlined how to linearizable consistency without a global buffer. This introduced a write contention challenge. In this post we will get into Corfu’s simple fix for this issue.  Read the rest of this entry »

Corfu: Linearizable Consistency (Part 2)

This the second post in a series about Corfu. In this post I will outline the basic technique it uses to achieve linearizable consistency. We will then look at a simple adaption that we will later see can be used to scale the system which introduces an additional challenge to solve.  Read the rest of this entry »

Corfu: Scaling log replication (Part 1)

Corfu: A distributed shared log is a paper running to 25 pages brought to my attention by The Morning Paper. Corfu takes log replication to a whole new level. Corfu achieves linearizable consistency of a fault tolerate distributed log using very simple write-once techniques. The real eye open is that it implements a distributed log interface without an IO bottleneck. The paper proudly points out that after 40 years of research this is a game changer. This series of posts will build up a simple fictional example that demonstrates the key techniques. Then I will glue it all together to describe how Corfu itself works and demystify the role Paxos plays. Read the rest of this entry »

C# ASP.NET with SQLServer on Docker and Linux is now “A Thing™”

Those of you who have been following along will know that Scala on Linux is my preferred ecosystem. This past few weeks in the office I have been tinkering with the opensource C# ASP.NET ecosystem. What I came across shocked me to the dotnet core. I have posted some evidence of my findings up on GitHubRead the rest of this entry »

TRex UPaxos Experimental Code

There is now a sketch (hack!) demoing the happy path flow of a UPaxos reconfiguration written up on the TRex wiki.

Paxos Voting Weights

The last UPaxos post took a run through the reconfiguration stall avoiding aspects of the UPaxos paper. In this post, we will take a quick look at how the paper describes hot swapping of nodes using voting weights. Read the rest of this entry »

Cassandra For Shared Media Libraries

A good friend of mine is working on a project which hosts media libraries in his cloud service. At the end of 2015, I integrated Cassandra into a big financial services platform. Cassandra is a great fit for my friend’s service. In this post, I will outline an appropriate Cassandra data model and along the way outline some of the killer features of Cassandra. Read the rest of this entry »

The End Of Agile Estimation: Lies, Damn Lies, And Burndown Charts

In my last couple of posts, I gave an example of a successful agile team that didn’t do estimates, then made the case that estimation is aligned to “wagile” ways of working. This post is going to make the case that the approach is fundamentally flawed when viewed as a measurement activity.

Read the rest of this entry »