The network is faster than the disk

In distributed systems, contrary to popular belief, the local disk may not always be faster than the network. While we do use SSD/Flash drives that are considered faster, networks can also offer high speeds and low latency. It is important to carefully test the performance characteristics of both local disk and network resources when designing and deploying distributed systems. I recently came across two articles that tested latencies on AWS.

Frontend Platforms II: Single-Spa microfrontends with Apollo Core Backends-for-Frontends

In the first post I outlined a platform for customer experiences that hosts many webapps that each serve a community. Many webapps are a single business solution that bring multiple communities together to collaborate via common domain micro services. Each webapp has many screens. Yet how those screens can be composed is infinite and arbitrary. In this post I will introduce hosts, “BFFs” (backends-for-frontends) and single-spa “applications” (actually microfrontends) that serve capabilities to a community that are composed into an webapp. These allow us to scale how teams deliver use cases in a way that is independent to how the customer experience needs to constantly evolve.

Blockchain for Finance is Bollocks

Blockchain was invented in 2008 and folks have claimed that it might be the most important invention since the internet. I have worked at a number of companies and organizations and seen them throw millions of dollars at ”doing Blockchain”. I am familiar with multiple attempts to use block in banking and insurance. All of the time and money invested were wasted. I am going to call it: Blockchain for Finance is bollocks.

How not to structure your database-backed web applications: a study of performance bugs in the wild

The case against ORM continues in this study of performance bugs in the wild.

the morning paper

How not to structure your database-backed web applications: a study of performance bugs in the wild Yang et al., ICSE’18

This is a fascinating study of the problems people get into when using ORMs to handle persistence concerns in their web applications. The authors study real-world applications and distil a catalogue of common performance anti-patterns. There are a bunch of familiar things in the list, and a few that surprised me with the amount of difference they can make. By fixing many of the issues that they find, Yang et al., are able to quantify how many lines of code it takes to address the issue, and what performance improvement the fix delivers.

To prove our point, we manually fix 64 performance issues in [the latest versions of the applications under study] and obtain a median speed-up of 2x (and up to 39x max) with fewer than 5 lines of…

View original post 1,507 more words

File Corruption And Consensus

The Morning Paper blog continues to deliver with on overview of how file corruption causes data loss on consensus systems such as Zookeeper and etcd:

Protocol aware recovery for consensus-based storage

etcd is used by Kubernetes (which is eating the cloud), and Zookeeper is a banks best friend for managing distributed systems configuration, so this is a major problem.

Better yet the paper retrofits a solution called CTRL onto those popular open source work horses with only a 4% overhead. It seems highly likely that CTRL will be coming to your part of the cloud any day soon.

Rust and Embedded Databases for Paxos

So far on my spike into Rust we have been on a roll. Next on the bucket list is an embedded disk backed B-tree database to act as the Paxos journal for TRex. This is where I have hit my first bump in the road.

C# ASP.NET with SQLServer on Docker and Linux is now “A Thing™”

Those of you who have been following along will know that Scala on Linux is my preferred ecosystem. This past few weeks in the office I have been tinkering with the opensource C# ASP.NET ecosystem. What I came across shocked me to the dotnet core. I have posted some evidence of my findings up on GitHub. 

Cassandra For Shared Media Libraries

A good friend of mine is working on a project which hosts media libraries in his cloud service. At the end of 2015, I integrated Cassandra into a big financial services platform. Cassandra is a great fit for my friend’s service. In this post, I will outline an appropriate Cassandra data model and along the way outline some of the killer features of Cassandra.

Designing software for ease of extension and contraction

Today’s Morning Paper post is a must read for software engineers: “Designing software for ease of extension and contraction Parnas, IEEE Transactions on Software Engineering, 1979″

Domain Driven Design: Entities, Value Objects, Aggregates and Roots with JPA (Part 2)

Detour: Why use  JPA in this demo?

For the purposes of this demo, JPA is an officially supported part of the Java ecosystem and is a mature and well documented Java-to-relational mapping tool. Yes, it has quite a few quirks. If you fight it you will probably lose (your mind). If you learn how to do the basics and don’t deviate from that it can be a used as a rapid application tool to support an agile TDD build on Java against a relational database.