Hackish CEPH cluster on Docker

ceph/README.md

@@ -0,0 +1,19 @@
+# CEPH on Docker
+
+Note: this doesn't quite work yet.
+
+The OSD containers need to be started twice (the first time, they fail
+initializing; second time is a champ).
+
+Also, it looks like you need at least two OSD containers (or the OSD
+container should have two disks/directories, whatever).
+
+RadosGw is listening on port 8080.
+
+The `admin` container will create a `docker` user using `radosgw-admin`.
+If you run it multiple times, that's OK: further invocations are idempotent.
+
+Last but not least: it looks like AWS CLI uses a new signature format
+that doesn't work with RadosGW. After almost two hours trying to figure
+out what was wrong, I tried the S3 credentials directly with boto and
+it worked immediately (I was able to create a bucket).

ceph/docker-compose.yml

@@ -0,0 +1,53 @@
+version: "2"
+
+services:
+  mon:
+    image: ceph/daemon
+    command: mon
+    environment:
+      CEPH_PUBLIC_NETWORK: 10.33.0.0/16
+      MON_IP: 10.33.0.2
+  osd:
+    image: ceph/daemon
+    command: osd_directory
+    depends_on:
+      - mon
+    volumes_from: 
+      - mon
+    volumes:
+      - /var/lib/ceph/osd
+  mds:
+    image: ceph/daemon
+    command: mds
+    environment:
+      CEPHFS_CREATE: 1
+    depends_on:
+      - mon
+    volumes_from: 
+      - mon
+  rgw:
+    image: ceph/daemon
+    command: rgw
+    depends_on:
+      - mon
+    volumes_from:
+      - mon
+    environment:
+      CEPH_OPTS: --verbose
+  admin:
+    image: ceph/daemon
+    entrypoint: radosgw-admin
+    depends_on:
+      - mon
+    volumes_from:
+      - mon
+    command: user create --uid=docker --display-name=docker
+
+networks:
+  default:
+    ipam:
+      driver: default
+      config:
+        - subnet: 10.33.0.0/16
+          gateway: 10.33.0.1
+