Serverless SQLite on K8s Using Litestream
Introduction
Typically if you are running any application that is useful, that application must have a database (Ask yourself is there any "stateless" application by itself is useful without a database?). For a database backed application you need to hook the application up with an existing database. The database either needs to be managed by yourself (run your own database server) or someone else (using managed service such as AWS RDS or Cloud SQL from GCP). either way comes with an operational cost in the sense that it's a matter of whether you outsource the database management or not.
However what if I told you that you can have a fully replicated database with little operational costs? In this post we will look at how to run a serverless SQLite database on Kubernetes using Litestream.
Prerequisites
- A Kubernetes cluster
- Access to a Cloud Storage bucket. In the example I will use Google Cloud Storage (GCS) bucket.
- You are perfectly comfortable with running your application on 1 replica. In other words a few seconds of downtime in between the time of single pod rescheduling is acceptable to you.
What is Litestream?
Here is the official definition of Litestream:
Litestream is a standalone disaster recovery tool for SQLite. It runs as a background process and safely replicates changes incrementally to another file or S3. Litestream only communicates with SQLite through the SQLite API so it will not corrupt your database.
In fact you can backup your SQLite database to any storage that Litestream supports. In this post we will use Google Cloud Storage (GCS) bucket.
In this document it also explains how it works:
Litestream is a streaming replication tool for SQLite databases. It runs as a separate background process and continuously copies write-ahead log pages from disk to one or more replicas. This asynchronous replication provides disaster recovery similar to what is available with database servers like Postgres or MySQL.
If you are unfamiliar with the concept of write-ahead logging (WAL), it's simply a journal mode in SQLite that allows you to have concurrent read and write operations to the database. I have a separate post that covers it in detail.
Preparing the GCS bucket
This is the gcloud command to create a GCS bucket:
export PROJECT_ID=$(gcloud config get-value project)
export PROJECT_NUMBER=$(gcloud projects list --filter="name:${PROJECT_ID}" --format="value(PROJECT_NUMBER)")
gcloud storage buckets create \
gs://litestream-demo/ \
--uniform-bucket-level-access \
--project=${PROJECT_ID} \
--public-access-prevention \
--location=europe-west1
Assuming you are going to use a kubernetes service account to access the given GCS bucket, here is the bucket IAM member you need to grant:
export LITESTREAM_NAMESPACE=litestream-demo
export LITESTREAM_SERVICE_ACCOUNT=litestream-sa
gcloud storage buckets add-iam-policy-binding \
gs://litestream-demo/ \
--member=principal://iam.googleapis.com/projects/${PROJECT_NUMBER}/locations/global/workloadIdentityPools/${PROJECT_ID}.svc.id.goog/subject/ns/${LITESTREAM_NAMESPACE}/sa/${LITESTREAM_SERVICE_ACCOUNT} \
--role=roles/storage.objectAdmin
Deploying application
As promised we don't need to deploy any operator or spin up any managed database. We can run the database directly in the Kubernetes deployment. The database will be replicated to the GCS bucket with the help from Litestream.
The Kubernetes manifest is posted at the end of this post. To summarize the setup, we have:
A namespace litestream-demo
A service account litestream-sa that are granted with access to the litestream-demo bucket via workload identity federation.
A configmap litestream-config that contains the Litestream configuration and the application code. The litestream.yml file looks like this:
dbs:
- path: /var/db/demo.db
replicas:
- url: "gcs://litestream-demo/litestream"
It makes sure that the data are replicated to the litestream folder in the litestream-demo bucket. And when the backup is restored, it will be restored to the /var/db/demo.db file, along with its WAL and SHM files.
The main functionality of the deployment litestream-demo is to run the FastAPI application. Notes the deployment strategy is set to Recreate to make sure that only 1 pod is running at any time.
The deployment is consisted of 3 containers:
init-litestream: This is an init container that will restore the database from the GCS bucket on the application startup. Notes that the /var/db/demo.db is restored into the /var/db directory, which is a volume that is shared with the fast-app and litestream containers.
initContainers:
- name: init-litestream
image: litestream/litestream:0.3
resources:
requests:
cpu: 200m
memory: 200Mi
args:
- restore
- -if-db-not-exists
- -if-replica-exists
- /var/db/demo.db
volumeMounts:
- name: db
mountPath: /var/db
- name: litestream-config
mountPath: /etc/litestream.yml
subPath: litestream.yml
fast-app: The application container that actually runs the FastAPI application. It uses the sqlite3 database restored by the
- name: fast-app
image: python:3.13
resources:
requests:
cpu: 200m
memory: 200Mi
command:
- /bin/bash
- -c
- |
pip install fastapi sqlmodel uvicorn;
uvicorn app:app --host 0.0.0.0 --port 8000
workingDir: /fast-app
volumeMounts:
- name: db
mountPath: /var/db
- name: litestream-config
mountPath: /fast-app/app.py
subPath: app.py
livenessProbe:
httpGet:
path: /healthz
port: 8000
initialDelaySeconds: 10
periodSeconds: 10
readinessProbe:
httpGet:
path: /healthz
port: 8000
initialDelaySeconds: 10
periodSeconds: 10
litestream: This is the Litestream container that is responsible for replicating the database to the GCS bucket on every transaction.
- name: litestream
image: litestream/litestream:0.3
args: ["replicate"]
resources:
requests:
cpu: 200m
memory: 200Mi
volumeMounts:
- name: db
mountPath: /var/db
- name: litestream-config
mountPath: /etc/litestream.yml
subPath: litestream.yml
The logs of the litestream container looks like below:
time=2025-02-24T16:03:36.909Z level=INFO msg=litestream version=v0.3.13
time=2025-02-24T16:03:36.913Z level=INFO msg="initialized db" path=/var/db/demo.db
time=2025-02-24T16:03:36.913Z level=INFO msg="replicating to" name=gcs type=gcs sync-interval=1s bucket=litestream-demo path=litestream
time=2025-02-24T16:03:37.975Z level=INFO msg="sync: new generation" db=/var/db/demo.db generation=1e800ba753b3b606 reason="no generation exists"
time=2025-02-24T16:03:38.050Z level=INFO msg="write snapshot" db=/var/db/demo.db replica=gcs position=1e800ba753b3b606/00000000:4152
time=2025-02-24T16:03:38.124Z level=INFO msg="snapshot written" db=/var/db/demo.db replica=gcs position=1e800ba753b3b606/00000000:4152 elapsed=73.395367ms sz=693
time=2025-02-24T16:03:38.167Z level=INFO msg="write wal segment" db=/var/db/demo.db replica=gcs position=1e800ba753b3b606/00000000:0
time=2025-02-24T16:03:38.222Z level=INFO msg="wal segment written" db=/var/db/demo.db replica=gcs position=1e800ba753b3b606/00000000:0 elapsed=54.698238ms sz=4152
The logs are pretty much replicated remotely realtime. On pod start/restart the init-litestream will restore the database from the latest generation in the GCS bucket. The process looks like below:
# kubectl -n litestream-demo logs -f deploy/litestream-demo -c init-litestream
time=2025-02-24T16:03:36.231Z level=INFO msg="restoring snapshot" db=/var/db/demo.db replica=gcs generation=4c5b00c905e47e50 index=0 path=/var/db/demo.db.tmp
time=2025-02-24T16:03:36.308Z level=INFO msg="restoring wal files" db=/var/db/demo.db replica=gcs generation=4c5b00c905e47e50 index_min=0 index_max=1
time=2025-02-24T16:03:36.363Z level=INFO msg="downloaded wal" db=/var/db/demo.db replica=gcs generation=4c5b00c905e47e50 index=1 elapsed=54.662808ms
time=2025-02-24T16:03:36.376Z level=INFO msg="downloaded wal" db=/var/db/demo.db replica=gcs generation=4c5b00c905e47e50 index=0 elapsed=67.442598ms
time=2025-02-24T16:03:36.403Z level=INFO msg="applied wal" db=/var/db/demo.db replica=gcs generation=4c5b00c905e47e50 index=0 elapsed=27.486258ms
time=2025-02-24T16:03:36.409Z level=INFO msg="applied wal" db=/var/db/demo.db replica=gcs generation=4c5b00c905e47e50 index=1 elapsed=6.03625ms
time=2025-02-24T16:03:36.409Z level=INFO msg="renaming database from temporary location" db=/var/db/demo.db replica=gcs
Conclusion
Voila! We have a fully replicated SQLite database on Kubernetes at the cost of a GCS bucket. IMO it's a viable alternative to managed database services if:
- Running on 1 replica can fulfill your availability and reliability requirements, which IMO is the case for many applications as long as you have plenty of cores and RAMs. The vendor might tell you otherwise though.
- You are ok with a few seconds of downtime in between the time of single pod rescheduling.
- There is no live-schema migration in SQLite so you will need to roll your own if it is ever needed.
Multi-replica flavoured HA isn't achievable with SQLite, the LiteFS project provide a viable path to it, however currently it uses consul as a hard dependency for leader election, that being said I think it should be relatively easy to replace it with a kubernetes or Cloud storage based solution.
Complete Kubernetes manifest
apiVersion: v1
kind: Namespace
metadata:
name: litestream-demo
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: litestream-sa
namespace: litestream-demo
---
apiVersion: v1
kind: ConfigMap
metadata:
name: litestream-config
namespace: litestream-demo
data:
litestream.yml: |
dbs:
- path: /var/db/demo.db
replicas:
- url: "gcs://litestream-demo/litestream"
app.py: |
from fastapi import FastAPI
from sqlmodel import Field, Session, SQLModel, create_engine, select
class Hero(SQLModel, table=True):
id: int | None = Field(default=None, primary_key=True)
name: str = Field(index=True)
secret_name: str
age: int | None = Field(default=None, index=True)
sqlite_file_name = "/var/db/demo.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"
connect_args = {"check_same_thread": False}
engine = create_engine(sqlite_url, echo=True, connect_args=connect_args)
def create_db_and_tables():
SQLModel.metadata.create_all(engine)
app = FastAPI()
@app.on_event("startup")
def on_startup():
create_db_and_tables()
@app.post("/heroes/")
def create_hero(hero: Hero):
with Session(engine) as session:
session.add(hero)
session.commit()
session.refresh(hero)
return hero
@app.get("/heroes/")
def read_heroes():
with Session(engine) as session:
heroes = session.exec(select(Hero)).all()
return heroes
@app.get("/healthz")
def healthz():
return {"status": "ok"}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: litestream-demo
namespace: litestream-demo
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: litestream-demo
template:
metadata:
labels:
app: litestream-demo
spec:
serviceAccountName: litestream-sa
initContainers:
- name: init-litestream
image: litestream/litestream:0.3
resources:
requests:
cpu: 200m
memory: 200Mi
args:
- restore
- -if-db-not-exists
- -if-replica-exists
- /var/db/demo.db
volumeMounts:
- name: db
mountPath: /var/db
- name: litestream-config
mountPath: /etc/litestream.yml
subPath: litestream.yml
containers:
- name: litestream
image: litestream/litestream:0.3
args: ["replicate"]
resources:
requests:
cpu: 200m
memory: 200Mi
volumeMounts:
- name: db
mountPath: /var/db
- name: litestream-config
mountPath: /etc/litestream.yml
subPath: litestream.yml
- name: fast-app
image: python:3.13
resources:
requests:
cpu: 200m
memory: 200Mi
command:
- /bin/bash
- -c
- |
pip install fastapi sqlmodel uvicorn;
uvicorn app:app --host 0.0.0.0 --port 8000
workingDir: /fast-app
volumeMounts:
- name: db
mountPath: /var/db
- name: litestream-config
mountPath: /fast-app/app.py
subPath: app.py
livenessProbe:
httpGet:
path: /healthz
port: 8000
initialDelaySeconds: 10
periodSeconds: 10
readinessProbe:
httpGet:
path: /healthz
port: 8000
initialDelaySeconds: 10
periodSeconds: 10
volumes:
- name: db
emptyDir:
sizeLimit: 500Mi
- name: litestream-config
configMap:
name: litestream-config