OpenHFT/Chronicle-Map

tech

Replicate your Key Value Store across your network, with consistency, persistance and performance.

— OpenHFT Chronicle-Map
github.com/OpenHFT/Chronicle-Map

Features

  • Ultra low latency: Chronicle Map targets median latency of both read and write queries of less than 1 microsecond in certain tests.

  • High concurrency: Write queries scale well up to the number of hardware execution threads in the server. Read queries never block each other.

  • Persistence to disk - (Optional)

  • Multi-master replication - (Optional, commercial functionality) - Eventually-consistent, fully-redundant, asynchronous replication across servers, "last write wins" strategy by default, allows to implement custom state-based CRDT(conflict-free replicated data type) strategy.The writer can optionally wait for replication to occur across nodes or regions.

FeatureAvailability

In-memory off-heap Map

Open source

Persistence to disk

Open source

Remote calls

Commercially available

Eventually-consistent replication (100% redundancy

Commercially available

Synchronous replication

Commercially available

Partially-redundant replication

On demand

Entry expiration timeouts

On demand

What guarantees does Chronicle Map provide in ACID terms?

  • Atomicity - single-key queries are atomic if Chronicle Map is properly configured, multi-key queries are not atomic.

  • Consistency - doesn’t make sense for key-value stores

  • Isolation - yes; for both single- and multi-key queries.

  • Durability - no; Chronicle Map can be persisted to disk, but with no guarantee as to how frequently this happens. This is under the control of the OS. All data is guaranteed to be written to disk when the Map is closed.

  • Clustering and replication for Chronicle Map is provided by Chronicle Map Enterprise.

What is the data structure of Chronicle Map?

Simply put, a Chronicle Map data store is a big chunk of shared memory (optionally mapped to disk).
It is split into independent segments; each segment has:

  • an independent memory allocation for storing the entries

  • a hash table for search

  • a lock in shared memory (implemented via CAS loops) for managing concurrent access.

See Chronicle Map data store design overview for more information.

Chronicle Map does not support:

  • range queries.

  • iteration over the entries in alphabetical order.

  • sorting of keys.

  • LRU entry eviction.

Unique features[?]

  • Multiple processes can access a Chronicle Map concurrently. At the same time, the data store is in-process for each of the accessing processes. Out-of-process approach to IPC is simply incompatible with Chronicle Map’s median latency target of < 1 μs.

  • Replication without logs, with constant footprint cost, guarantees progress even if the network doesn’t sustain write rates.

Other key points

  • size not limited by memory The size of a Chronicle Map is not limited by memory (RAM), but rather by the available disk capacity.

  • Eventually-consistent replication (100% redundancy) Commercially available

  • Clustering and replication Commercially available

Overview

CM Overview

Conclusion

  • Do not use it in your production application.

  • If you have more time, you can clone and read the source code or take exercise.