Backend Engineer - Emerging Talent

Key-Value File Store

Build a High-Performance, Scalable Key-Value Data Store

We are looking for engineers who love coding, problem-solving, and building systems that scale efficiently and are easy to maintain. To test your skills, we’ve designed a coding challenge that mirrors real-world engineering tasks. It's not just about solving the problem, but about writing clean, maintainable code that shows a deep understanding of systems architecture, performance, and usability (in this case, for a fellow developer).

In addition to solving the challenge, we’ll be looking at how you approach system design, code clarity, and testing—qualities that reflect the mentorship you’ll receive here.

What to expect:

• Effort: It should take a total of 2-4 hours to cover all requirements.
• Relevance: The skills applied will be required for this role.
• Feedback: Thorough code review and design feedback for each submission.
• Iterations: The chance to improve your submission based on feedback.
• Discussion: An exploration of how this was built and the choices made.

The Challenge:

Develop a key-value (kv) data store with advanced functionality, supporting Create, Read, and Delete (CRD) operations.

The data store must be exposed as a library, allowing clients to instantiate a class and interact with it programmatically. Your implementation would ideally demonstrate a level of comfort with both functionality and performance.

Functional Requirements:

1. Local File-based Data Store:
   • Develop a data store that will act as a local storage solution for a single process on one device. You may use a file as the persistence for the data store.
2. Initialization:
   • The data store can be initialized with an optional file path parameter. If not provided, it should create itself in an optimal location. If the file already exists, it will attempt to use it as a valid, pre-created kv store.
3. Create Operation:
   • Support adding a key-value pair, where the key is a string (max 32 chars) and the value is a JSON object (max 16KB).
   • If the key already exists, return an appropriate error.
4. Read Operation:
   • Retrieve the JSON value corresponding to a key.
5. Delete Operation:
   • Remove a key-value pair using the key.
6. Time-to-Live (TTL):
   • Each key supports an optional TTL (in seconds). After this TTL expires, the key should automatically become unavailable for read or delete operations.
7. Batch Operations:
   • Implement batch Create operations that handle multiple key-value pairs in a single call, showcasing the efficiency of your design.
   • Feel free to decide your own limits for a batch and explain using a comment why you chose the limit.
8. Exception handling:
   • Appropriate error responses must always be returned to a client if it uses the data store in unexpected ways or breaches any limits.

Non-Functional Requirements:

1. File Size:
   • The data file should never exceed 1GB in size, even as the data store grows.
2. Concurrency and Thread-Safety:
   • Your data store must be thread-safe, allowing concurrent access from multiple threads.
   • Additionally, it must handle scenarios where only one client process can access the file-based store at any given time, gracefully rejecting concurrent accesses by other processes.
3. Memory Efficiency:
   • Minimize the memory footprint while maximizing performance.

Quality and Testing Expectations:

• Submit your code with comprehensive unit tests. These should cover all edge cases, error scenarios, and concurrent access patterns.
• Include instructions or scripts to run the tests and verify the correctness of your solution.
• We expect the tests to validate TTL expirations, error handling (e.g., duplicate keys), and concurrent access correctness.

Languages:

While you’re free to use a language of your choice, our preferred stack includes:

• NodeJS
• Java
• Python
• GoLang
• Ruby
• C/C++

This is an opportunity to showcase your creativity, design thinking, and problem-solving skills. We're excited to see how you tackle this challenge and look forward to discussing your approach!