What it is
Choosing a database is choosing a set of promises about your data: what a transaction means, what gracefully degraded looks like, how it scales, and who can fix it at three in the morning. Most systems are best served by one relational database with a schema somebody understands, plus a cache or a search index where the workload genuinely needs one - and the interesting work is knowing which of those three things a given piece of data actually is.
Below are the stores we work with, gathered by what they are for. Each section says what we do on that one; the list at the end is the work that is common to all of them.
Relational
MySQL
The default open-source relational database, and the one most existing web applications already run. We design schemas that survive being changed, index them for the queries that actually run, move them between servers and versions with as little downtime as the platform allows, and set up replication or a cluster for availability that does not depend on one machine. When an application is slow, we find out whether it is the query, the index or the server before anybody buys a bigger one.
PostgreSQL
The relational database we reach for by default on new work: strict about correctness, comfortable with JSON, and extensible when a problem needs something else - PostGIS for geography, partitioning for volume, time-series extensions for sensors. Logical replication makes moving to a managed service a copy rather than an outage, and we take full-text search, row-level security and stored procedures as far as they usefully go before adding another store to the estate.
Oracle
Still where a lot of serious enterprise data sits, and usually no longer by choice. We work with it as it is - PL/SQL, schemas older than the people maintaining them, and licensing that makes every change a cost question - and we are equally happy planning the way off it, in stages that keep the business running.
SQL Server
Common wherever the rest of the estate is Microsoft, and a good fit for it: T-SQL, Always On availability groups, and the reporting tooling around it. We work on performance and indexing, migrations between versions and platforms including to Linux and to Azure SQL, partitioning, and the reporting queries that must not slow the transactional workload down. When a reporting demand is really a data problem, we will say that too, rather than tuning the same server a fourth time.
Document and key-value
MongoDB
A document store for data whose shape genuinely varies - catalogues, profiles, content, events - where a fixed schema would mean a table nobody can change. We model documents around how they are read rather than how they arrive, get the indexes right, which is most of what makes Mongo fast or slow, and design the sharding and replication story before a collection is too big to change.
DynamoDB
AWS's managed key-value database: effectively unlimited scale, single-digit millisecond reads, and no server to run - at the price of designing for the access patterns up front, because in DynamoDB the queries are the schema. We work on partition and sort key design, secondary indexes, hot partitions, capacity mode and cost, and the single-table patterns that are powerful when they are understood and painful when they are copied.
Redis
An in-memory store doing four different jobs that are often the same server: a cache, a session store, a rate limiter, and a simple queue. We decide which one a piece of state actually is, set eviction and expiry deliberately rather than by default, and use the data structures - sorted sets, streams, hashes - for the work they are good at instead of pushing everything through strings.
Memcached
The narrower, older cache: multi-threaded, simple, and very good at being a cache and nothing else, which for a lot of workloads is exactly the right answer. We are as happy putting it in front of a database as we are replacing it with Redis where sessions or data structures are needed, and in either case we make the invalidation rules explicit - a cache nobody can explain is worse than no cache at all.
Search
Elasticsearch
A search engine and log store built on Lucene: relevance, analysis, aggregations, and text queries at a scale a relational database cannot answer. We design mappings and analysers rather than accepting the defaults, use index templates and lifecycle policies so that old data leaves on its own, keep it in step with the source of truth through Kafka Connect or a pipeline of its own, and treat it as a derived copy of the data rather than as the record itself - because a search index you cannot rebuild is a search index you are afraid of.
The work we do on all of them
- Schema and data modelling. The shapes that will still make sense in three years, the keys and indexes the workload needs, and a written reason for the store it lives in.
- Migration, with a way back. Between versions, between engines, or onto a managed service - planned in steps, rehearsed, and reversible until the last one.
- Performance. The query plan rather than the guess: indexes, statistics, connection pooling, and the queries that only misbehave at the size of the real table.
- Availability. Replication, failover, cluster topology and connection handling, sized to what the business actually needs rather than to the whitepaper.
- Backups that have been restored. A backup nobody has restored is a hope, so we test one on every system we touch and write down how long the recovery took.
- Upgrades, patching and end of life. Versions that are still supported, security patches applied on a schedule, and a plan for the engine the vendor is retiring.
- Monitoring and cost. The metrics that predict a problem, alerts that somebody acts on, and the storage, capacity and licences that show up on the bill.
- A second opinion. An audit of what you have: what is sized wrongly, what is duplicated, what nobody is backing up, and what could be deleted.
How it starts
Tell us which systems hold the data that matters and what goes wrong with them. For a migration we start with a rehearsal on a copy; for a slow system, with the query that is slowest.
