+ - 0:00:00
Notes for current slide
Notes for next slide

Containerized Clustered PostgreSQL

Josh Berkus

Red Hat OSAS

SCALE 15x 2017

1 / 35

github.com/jberkus/atomicdb

2 / 35

yak shaving

A Long Strange Journey

3 / 35

automated elephant

4 / 35

docker logo

5 / 35

The way containers are designed, and particularly the way Docker is designed, the assumption is that the container is stateless. -- Mark Davis, ClusterHQ

6 / 35

incomplete escalator

7 / 35

kubernetes logo

8 / 35

project atomic

9 / 35
kubernetes logo
elephant plus pets
10 / 35

StatefulSet

Kubernetes API for stateful applications, introduced in 1.4 (still beta) (was PetSet)

  • storage
  • routing
  • identity
  • persistence
11 / 35

Why Containerize?

12 / 35

Pro-container

  • easy install/configure
  • service encapsulation
  • prevent "hot-fixes"
  • ease of dev and testing
  • orchestration
13 / 35

Pro-orchestration

  • auto-vivification
  • deploy & configure in groups
  • manage routing
  • built-in HA/DR
  • self-service for devs
  • manage everything in one place
14 / 35

Pro-Orchestration

easiest way to test the new clustered versions of PostgreSQL

  • CitusDB
  • MMSync
  • BDR
  • More
15 / 35

manage PostgreSQL
as a cluster ecosystem,
instead of as
a bunch of servers

16 / 35

Anti-container

  • extra setup
  • change admin, config, monitorning tools
  • possible performance issues
  • no efficiency benefit for databases
  • everything is alpha
17 / 35

This is your future infrastructure.

18 / 35

Enough Talking,
Let's See Some Code!

19 / 35

micro cluster

20 / 35

Prototypeishness

  • still working on some of the alpha features
  • some productions examples exist already (Spilo)
  • others this year
21 / 35

robot

Bots Not Cattle

22 / 35

autopilot

Autopilot Pattern

23 / 35

Bots/Autopilots

Each containerized application is programmed using a simple state machine to be an intelligent part of a clustered ecosystem.

24 / 35

Container Considerations

  • log to stdout/stderr
  • need bootstrap script
  • install config, hba
  • set up initial users
25 / 35

PV or Not PV?

  • Big Databases: want persistent volumes
  • Small Databases: use ephemeral storage, make lots of replicas.
26 / 35

Productionizing

27 / 35

Add to Patroni/Citus

  • Persistent Volume configs
  • Vault support
  • Sidecar containers (backup, admin)
  • Failover for query node
  • Failover for shards
28 / 35

More Clusters!

  • Patroni-RAFT
  • Postgres Pro MM Clustering
  • New Governor
  • BDR (with Federation?)
  • PostgresXL
29 / 35

Waiting for Kube

  • Union ConfigMap
  • Federated StatefulSets
  • Node Anti-Affinity (1.6?)
  • Fix issues (PVs, etc.)
  • Vault plugin
30 / 35

Pull Requests Welcome!

github.com/jberkus/atomicdb

31 / 35

Resources

  • This Presentation:
    jberkus.github.io/container_cluster
  • Simplified Configuration Scripts:
    github.com/jberkus/atomicdb/
  • Crunchy Data Containers:
    github.com/CrunchyData/crunchy-containers
32 / 35

Spilo Resources

  • Patroni Project:
    github.com/zalando/patroni
  • Spilo Project:
    github.com/zalando/spilo
  • Helm Chart:
    github.com/kubernetes
    /charts/incubator/patroni
33 / 35

valentin from zalando

Thanks, Zalando!

34 / 35

¿questions?

more
jberkus:

project atomic:

 

@fuzzychef
www.databasesoup.com
jberkus.github.io

www.projectatomic.io

 

35 / 35

github.com/jberkus/atomicdb

2 / 35
Paused

Help

Keyboard shortcuts

↑, ←, Pg Up, k Go to previous slide
↓, →, Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
Esc Back to slideshow