55 posts
Most of the problems that people mention with their Kafka implementations is that they don’t have the complete visibility required over the configuration and the API usage. Having more visibility into the commonly required tweaks can allow admins and developers to use Kafka as comfortably as a MySQL or PostgreSQL cluster...
Each kafka topic defines the number of partitions and replication factors when it’s created. However, once a topic is created, the partition count cannot be changed without affecting the ordering guarantees of the kafka partitions since kafka uses the following formula to calculate which partition a record should go to: partition\_id = partition\_key % number of partitions Kafka partitions are the gateway to concurrency and scalability...
I was recently looking at a managed Kafka service and came across services like AWS MSK and Kafka on Confluent Cloud. While comparing these services, I saw that there were limitations on the number of partitions allowed in a cluster...
Monarch is a planet-scale in-memory time series database developed by Google. It is mainly used by as a reliable monitoring system by most of Google’s internal systems like Spanner, BigTable, Colossus, BlobStore...
What is virtual time and why do we need it. As distributed systems have progressed and been adopted over the last decade, there have been numerous technologies in different segments like databases, caches, message queues, etc which are built on top of other frameworks which abstract away the difficulty of managing distributed systems...
I was exploring Neo4j and came upon this [video](https://www. youtube...
Suppose we have a C lib where we have defined various data structures and methods. Due to some reason or constraint, there needs to be a Golang process which has to reuse the structures mentioned in the C lib...
I had worked on Elasticsearch back in 2015, when it was more known for its text searching capabilities using inverted indexes. As I looked to pick it up again last year for another project, I saw that Elasticsearch had added core support for other data types from text like numbers, IP addresses, geospatial data types, etc...
I recently heard about Maglev, the load balancer that Google uses in front of most of its services. I wanted to get a short gist on the matter to understand the reason why Google had to create its own load balancer and the optimizations that they took in order to actually run a load balancer at Google’s scale...
Kubernetes is the de-facto container management system for all sorts of distributed workloads. Known for its extensibility and community support, there are numerous plugins for multiple use cases...