slash dev slash null

stuff about puters

Category: Uncategorized

Corfu: Copy Compaction (Part 5)

In the last post we discussed striping a log across many mirrored sets of disks for write concurrency. In this post we will discuss deletes from a global log and use copy collection techniques to free the disk space.  Read the rest of this entry »

Corfu: Stripe & Mirror (Part 4)

The last post introduced a global sequencer to reduce contention on writes to the end of a log file. The idea is to be able to distribute the log file across many machines. In this post we will scale up or fictional example using by striping and mirroring data much like a RAID 10 disk configuration.  Read the rest of this entry »

 Corfu: Global Sequencer (Part 3)

In the last post we outlined how to linearizable consistency without a global buffer. This introduced a write contention challenge. In this post we will get into Corfu’s simple fix for this issue.  Read the rest of this entry »

Corfu: Linearizable Consistency (Part 2)

This the second post in a series about Corfu. In this post I will outline the basic technique it uses to achieve linearizable consistency. We will then look at a simple adaption that we will later see can be used to scale the system which introduces an additional challenge to solve.  Read the rest of this entry »

Corfu: Scaling log replication (Part 1)

Corfu: A distributed shared log is a paper running to 25 pages brought to my attention by The Morning Paper. Corfu takes log replication to a whole new level. Corfu achieves linearizable consistency of a fault tolerate distributed log using very simple write-once techniques. The real eye open is that it implements a distributed log interface without an IO bottleneck. The paper proudly points out that after 40 years of research this is a game changer. This series of posts will build up a simple fictional example that demonstrates the key techniques. Then I will glue it all together to describe how Corfu itself works and demystify the role Paxos plays. Read the rest of this entry »

C# ASP.NET with SQLServer on Docker and Linux is now “A Thing™”

Those of you who have been following along will know that Scala on Linux is my preferred ecosystem. This past few weeks in the office I have been tinkering with the opensource C# ASP.NET ecosystem. What I came across shocked me to the dotnet core. I have posted some evidence of my findings up on GitHubRead the rest of this entry »

Cassandra For Shared Media Libraries

A good friend of mine is working on a project which hosts media libraries in his cloud service. At the end of 2015, I integrated Cassandra into a big financial services platform. Cassandra is a great fit for my friend’s service. In this post, I will outline an appropriate Cassandra data model and along the way outline some of the killer features of Cassandra. Read the rest of this entry »

Designing software for ease of extension and contraction

Today’s Morning Paper post is a must read for software engineers: “Designing software for ease of extension and contraction Parnas, IEEE Transactions on Software Engineering, 1979″

Domain Driven Design: Entities, Value Objects, Aggregates and Roots with JPA (Part 5)

This is the last article in the series which discusses a sample app that does DDD using JPA. I would hesitate to recommend using JPA for DDD unless you are very familiar with JPA. Why? Some of the issues you can hit using JPA are written up on the Scabl blog on Advanced Enterprise DDD. In my own code and this blog I explain how to dodge some of the bullets but you need to be quite aware of the pitfalls of JPA to dodge them all. So why did I write the code? Read the rest of this entry »

Domain Driven Design: Entities, Value Objects, Aggregates and Roots with JPA (Part 4)

Don’t abuse the `public` keyword in Java. The source code has very few public classes or methods. This is unusual for Java projects. Typically Java projects have package layouts that model the solution; “this package has all the entities, that package has all database related code, that package is all the services code”. That approach forces you to make almost everything public. In the long term on a big project brittle connections are made across business responsibility boundaries. There is no way the compiler can enforce boundaries that align to the business domain.  Read the rest of this entry »