Why Databases and Not Files
Why this matters: starting from files rather than from "we'll use Postgres" is what lets you explain why a database earns its place in a design — and which of its guarantees you are actually relying on.
Key takeaway
A database is an organized collection of data designed for efficient management. It facilitates the storage, retrieval, modification, and deletion of data — and it exists because flat files fail at exactly the things multi-user systems need.
The problem
Consider a messaging application similar to WhatsApp. To operate reliably it must store and retrieve user data — contact lists, message history. That data could live in flat files. Four practical limitations make that a bad idea:
| Limitation | What goes wrong with files |
|---|---|
| Concurrency | Managing concurrent access by multiple users is difficult |
| Access control | Granting granular access rights to different users is complex |
| Scalability | Performance and availability degrade as the number of entries increases |
| Search speed | Searching content becomes inefficient as file size grows |
Notice these are the same non-functional concerns from the Foundations module, appearing at the storage layer: concurrency is a correctness problem, access control is security, and the other two are scalability and performance.
The solution
Databases address all four. They power systems from banking to e-commerce, scaling to meet each organization's needs.
The advantages you're actually buying
| Advantage | What it gives you |
|---|---|
| Managing large data | Handles massive datasets more efficiently than file systems |
| Data consistency | Enforces constraints so retrieval is accurate |
| Efficient updates | Easy modification through a Data Manipulation Language (DML) |
| Security | Restricts access to authorized users |
| Data integrity | Maintains accuracy through defined constraints |
| Availability | Supports replication across servers to ensure uptime |
| Scalability | Supports partitioning to distribute load across nodes |
The last two are what the rest of this chapter is about. Replication buys availability; partitioning buys scalability. They are separate mechanisms solving separate problems, and production systems use both together.
The two families
Databases generally fall into two categories:
- SQL — relational databases
- NoSQL — non-relational databases
They differ in structure, storage methods, and intended use cases.
A useful analogy: relational databases resemble phone books with a predetermined schema — names and numbers, every entry the same shape. Non-relational databases are like file directories — unstructured, able to hold diverse data types with dynamic schemas.
What this chapter covers
Types and their use cases, replication models and their trade-offs, partitioning strategies and their costs, and finally a cost-benefit analysis of sharding approaches.
Key takeaway
A database is a set of guarantees — concurrency control, access control, integrity, durability — that you would otherwise have to build yourself. Knowing which of those guarantees your design depends on is what makes the choice defensible.
Interview signal by level
| Level | What a strong answer sounds like |
|---|---|
| L4 | "We'll store it in a database rather than files." |
| L5 | Names the reason: "we need concurrent writes and transactional integrity, which files can't give us." |
| Staff+ | Separates the mechanisms: "replication for availability, partitioning for scale — different problems, and I'll need both. Let me establish which guarantees this data actually requires before picking a system." |
Next: the family that dominates, and the four letters behind it.