Raspberry Pi Cluster Computing: K3s, Docker Swarm & Hadoop

1. Pengenalan Cluster Computing

Cluster computing adalah teknik menggabungkan beberapa komputer (node) menjadi satu unit komputasi yang bekerja sama. Raspberry Pi sangat populer untuk membangun cluster karena harganya yang terjangkau, konsumsi daya rendah, dan komunitas yang besar.

Membangun cluster Raspberry Pi adalah cara terbaik untuk belajar konsep distributed computing, container orchestration, dan parallel processing tanpa menghabiskan budget besar. Sebuah cluster 4 node Raspberry Pi 4 bisa dijadikan laboratorium K3s/Hadoop/Docker yang fungsional.

Mengapa Raspberry Pi untuk Cluster?

Teknologi Cluster yang Dibahas

2. Hardware & Setup Cluster

Daftar Komponen

Total estimasi: ~Rp 6.750.000 (harga bervariasi tergantung supplier)

Setup Awal Semua Node

# ============================================
# Setup Raspberry Pi OS di semua node
# Jalankan di setiap Pi (atau gunakan Raspberry Pi Imager)
# ============================================

# 1. Flash Raspberry Pi OS Lite (64-bit) ke microSD
#    Gunakan Raspberry Pi Imager:
#    https://www.raspberrypi.com/software/

# 2. Konfigurasi SSH dan WiFi headless
#    Setelah flash, mount boot partition dan buat file:
touch /boot/ssh                      # Enable SSH
# (opsional) buat wpa_supplicant.conf untuk WiFi

# 3. Boot dan update sistem
sudo apt update && sudo apt upgrade -y

# 4. Set hostname unik untuk setiap node
# Node 1 (Master):
sudo hostnamectl set-hostname pi-master
# Node 2 (Worker 1):
sudo hostnamectl set-hostname pi-worker1
# Node 3 (Worker 2):
sudo hostnamectl set-hostname pi-worker2
# Node 4 (Worker 3):
sudo hostnamectl set-hostname pi-worker3

# 5. Edit /etc/hosts di semua node
sudo tee -a /etc/hosts <<'EOF'
192.168.1.100  pi-master
192.168.1.101  pi-worker1
192.168.1.102  pi-worker2
192.168.1.103  pi-worker3
EOF

# 6. Enable cgroups (untuk Kubernetes)
# Edit /boot/cmdline.txt, tambahkan:
# cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory

# 7. Setup SSH key-based auth (dari master)
ssh-keygen -t ed25519 -C "cluster-key"
ssh-copy-id pi@pi-worker1
ssh-copy-id pi@pi-worker2
ssh-copy-id pi@pi-worker3

# 8. Disable swap (penting untuk K8s)
sudo dphys-swapfile swapoff
sudo dphys-swapfile uninstall
sudo systemctl disable dphys-swapfile

# 9. Reboot semua node
sudo reboot

3. Konfigurasi Jaringan Cluster

Jaringan yang stabil dan cepat adalah fondasi cluster yang baik. Gunakan Gigabit Ethernet (bukan WiFi) untuk koneksi antar node agar latency minimal dan bandwidth konsisten.

Skema IP Cluster

┌───────────────────────────────────────────────────────────┐
│              CLUSTER NETWORK TOPOLOGY                      │
│                                                            │
│   Internet                                                  │
│      │                                                      │
│   ┌──┴──┐                                                   │
│   │Router│  192.168.1.1                                     │
│   └──┬──┘                                                   │
│      │                                                      │
│   ┌──┴──────────────┐                                       │
│   │  GbE Switch     │                                       │
│   │  (5-port)       │                                       │
│   └─┬───┬───┬───┬───┘                                       │
│     │   │   │   │                                           │
│   ┌─┴┐┌─┴┐┌─┴┐┌─┴┐                                         │
│   │M ││W1││W2││W3│  Raspberry Pi 4 nodes                   │
│   └──┘└──┘└──┘└──┘                                          │
│                                                            │
│   M  = pi-master   192.168.1.100 (Master / Control Plane)  │
│   W1 = pi-worker1  192.168.1.101 (Worker Node)             │
│   W2 = pi-worker2  192.168.1.102 (Worker Node)             │
│   W3 = pi-worker3  192.168.1.103 (Worker Node)             │
└───────────────────────────────────────────────────────────┘

Konfigurasi Static IP

# ============================================
# Konfigurasi Static IP (di setiap node)
# Edit /etc/dhcpcd.conf
# ============================================

# Di pi-master:
interface eth0
static ip_address=192.168.1.100/24
static routers=192.168.1.1
static domain_name_servers=8.8.8.8 1.1.1.1

# Di pi-worker1:
interface eth0
static ip_address=192.168.1.101/24
static routers=192.168.1.1
static domain_name_servers=8.8.8.8 1.1.1.1

# (dan seterusnya untuk worker2 = .102, worker3 = .103)

# Restart networking
sudo systemctl restart dhcpcd

# Test connectivity antar node
ping -c 3 pi-worker1
ping -c 3 pi-worker2
ping -c 3 pi-worker3

# Test bandwidth antar node (install iperf3)
sudo apt install iperf3 -y

# Di master (server):
iperf3 -s

# Di worker (client):
iperf3 -c pi-master
# Harapkan ~940 Mbps untuk Gigabit Ethernet

4. Docker Swarm Cluster

Docker Swarm adalah container orchestration tool bawaan Docker yang paling mudah untuk memulai cluster. Swarm mengubah beberapa mesin Docker menjadi satu virtual Docker host yang terdistribusi.

Install Docker di Semua Node

# ============================================
# Install Docker di semua Raspberry Pi
# Jalankan di setiap node
# ============================================

# Install Docker
curl -fsSL https://get.docker.com | sh

# Tambahkan user pi ke grup docker
sudo usermod -aG docker pi

# Aktifkan Docker service
sudo systemctl enable docker
sudo systemctl start docker

# Verifikasi
docker --version
docker run hello-world

# Install docker-compose (opsional)
sudo apt install docker-compose -y

Inisialisasi Docker Swarm

# ============================================
# Inisialisasi Docker Swarm
# ============================================

# Di MASTER node (pi-master):
docker swarm init --advertise-addr 192.168.1.100

# Output akan memberikan token join, contoh:
# docker swarm join --token SWMTKN-1-xxxx 192.168.1.100:2377

# Di WORKER nodes (pi-worker1, pi-worker2, pi-worker3):
docker swarm join --token SWMTKN-1-xxxx 192.168.1.100:2377

# Verifikasi cluster (di master):
docker node ls
# ID            HOSTNAME     STATUS    AVAILABILITY   MANAGER STATUS
# abc123 *     pi-master    Ready     Active         Leader
# def456       pi-worker1   Ready     Active
# ghi789       pi-worker2   Ready     Active
# jkl012       pi-worker3   Ready     Active

Deploy Service di Swarm

# ============================================
# Deploy layanan di Docker Swarm
# ============================================

# 1. Buat overlay network
docker network create --driver overlay cluster-net

# 2. Deploy NGINX web server dengan 3 replica
docker service create \
  --name web-server \
  --replicas 3 \
  --network cluster-net \
  --publish 80:80 \
  nginx:alpine

# 3. Cek status service
docker service ls
docker service ps web-server

# 4. Deploy dengan docker-compose.yml
cat <<'EOF' > docker-compose.yml
version: '3.8'

services:
  web:
    image: nginx:alpine
    ports:
      - "80:80"
    deploy:
      replicas: 3
      restart_policy:
        condition: on-failure
      resources:
        limits:
          cpus: '0.50'
          memory: 256M
    networks:
      - cluster-net

  redis:
    image: redis:alpine
    ports:
      - "6379:6379"
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.role == manager
    volumes:
      - redis-data:/data
    networks:
      - cluster-net

networks:
  cluster-net:
    driver: overlay

volumes:
  redis-data:
EOF

# Deploy stack
docker stack deploy -c docker-compose.yml mystack

# Cek status stack
docker stack ls
docker stack services mystack

# 5. Scale service
docker service scale web-server=5

# 6. Rolling update
docker service update --image nginx:1.25-alpine web-server

5. K3s Kubernetes Cluster

K3s adalah distribusi Kubernetes ringan yang dikembangkan oleh Rancher (SUSE). K3s dioptimasi untuk ARM dan IoT, dengan footprint hanya ~100MB. Ini adalah pilihan terbaik untuk menjalankan Kubernetes di Raspberry Pi.

Install K3s

# ============================================
# Install K3s Kubernetes di Raspberry Pi Cluster
# ============================================

# PASTIKAN cgroup sudah diaktifkan di /boot/cmdline.txt
# cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory

# ========= MASTER NODE (pi-master) =========
# Install K3s server
curl -sfL https://get.k3s.io | sh -s - server \
  --advertise-address 192.168.1.100 \
  --disable servicelb \
  --disable traefik \
  --write-kubeconfig-mode 644

# Ambil node token untuk worker
sudo cat /var/lib/rancher/k3s/server/node-token
# Contoh output: K1081c8f0bced...::server:abc123

# ========= WORKER NODES =========
# Di pi-worker1, pi-worker2, pi-worker3:
curl -sfL https://get.k3s.io | K3S_URL=https://192.168.1.100:6443 \
  K3S_TOKEN=K1081c8f0bced...::server:abc123 \
  sh -s - agent

# ========= VERIFIKASI (di master) =========
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

kubectl get nodes
# NAME          STATUS   ROLES                  AGE   VERSION
# pi-master     Ready    control-plane,master   5m    v1.28+k3s1
# pi-worker1    Ready                     3m    v1.28+k3s1
# pi-worker2    Ready                     3m    v1.28+k3s1
# pi-worker3    Ready                     3m    v1.28+k3s1

kubectl get pods -A

Deploy Aplikasi di K3s

# ============================================
# Deploy aplikasi di K3s cluster
# ============================================

# 1. Install MetalLB (Load Balancer untuk bare-metal)
# Karena K3s di Pi tidak punya cloud load balancer
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.13.10/config/manifests/metallb-native.yaml

# Konfigurasi MetalLB IP range
cat <<'EOF' | kubectl apply -f -
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: default-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.200-192.168.1.250
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: default
  namespace: metallb-system
EOF

# 2. Install Nginx Ingress Controller
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.2/deploy/static/provider/baremetal/deploy.yaml

# 3. Deploy contoh aplikasi (nginx + service)
cat <<'EOF' | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80
        resources:
          requests:
            memory: "64Mi"
            cpu: "100m"
          limits:
            memory: "128Mi"
            cpu: "200m"
---
apiVersion: v1
kind: Service
metadata:
  name: web-app-svc
spec:
  type: LoadBalancer
  selector:
    app: web-app
  ports:
  - port: 80
    targetPort: 80
EOF

# 4. Cek deployment
kubectl get deployments
kubectl get pods -o wide   # Lihat pod tersebar di node mana
kubectl get svc             # Cek external IP dari MetalLB

# 5. Horizontal Pod Autoscaler
kubectl autoscale deployment web-app --cpu-percent=50 --min=2 --max=6

# 6. Monitoring resource usage
kubectl top nodes
kubectl top pods

6. Hadoop HDFS Cluster

Apache Hadoop adalah framework untuk distributed storage (HDFS) dan processing (MapReduce). Meskipun Raspberry Pi bukan mesin produksi untuk Hadoop, cluster ini sangat baik untuk belajar konsep big data.

Install Java dan Hadoop

# ============================================
# Install Hadoop di Raspberry Pi Cluster
# Jalankan di SEMUA node
# ============================================

# 1. Install Java
sudo apt install openjdk-11-jdk -y

# Set JAVA_HOME
echo 'export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-arm64' >> ~/.bashrc
source ~/.bashrc

# 2. Buat user hadoop
sudo adduser hadoop
sudo usermod -aG sudo hadoop

# 3. Setup SSH key untuk user hadoop (di master)
su - hadoop
ssh-keygen -t ed25519
ssh-copy-id hadoop@pi-master
ssh-copy-id hadoop@pi-worker1
ssh-copy-id hadoop@pi-worker2
ssh-copy-id hadoop@pi-worker3

# 4. Download dan install Hadoop (di semua node)
wget https://dlcdn.apache.org/hadoop/common/hadoop-3.3.6/hadoop-3.3.6-aarch64.tar.gz
tar -xzf hadoop-3.3.6-aarch64.tar.gz
sudo mv hadoop-3.3.6 /opt/hadoop

# 5. Konfigurasi environment (tambahkan ke ~/.bashrc)
cat <<'EOF' >> ~/.bashrc
export HADOOP_HOME=/opt/hadoop
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
EOF
source ~/.bashrc

# 6. Konfigurasi Hadoop
cd $HADOOP_CONF_DIR

# core-site.xml
cat <<'EOF' > core-site.xml
<configuration>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://pi-master:9000</value>
  </property>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>/tmp/hadoop-data</value>
  </property>
</configuration>
EOF

# hdfs-site.xml
cat <<'EOF' > hdfs-site.xml
<configuration>
  <property>
    <name>dfs.replication</name>
    <value>3</value>
  </property>
  <property>
    <name>dfs.namenode.name.dir</name>
    <value>/tmp/hadoop/namenode</value>
  </property>
  <property>
    <name>dfs.datanode.data.dir</name>
    <value>/tmp/hadoop/datanode</value>
  </property>
</configuration>
EOF

# workers file
echo -e "pi-worker1\npi-worker2\npi-worker3" > workers

# 7. Format HDFS (hanya di master, sekali saja!)
hdfs namenode -format

# 8. Start HDFS
start-dfs.sh

# 9. Verifikasi
hdfs dfsadmin -report
# Harus menunjukkan 3 datanodes

# 10. Upload test file
hdfs dfs -mkdir /user/hadoop
hdfs dfs -put /etc/hosts /user/hadoop/test.txt
hdfs dfs -cat /user/hadoop/test.txt

7. MPI Parallel Computing

MPI (Message Passing Interface) adalah standar untuk komputasi paralel di cluster. Dengan MPI, kita bisa membagi tugas komputasi ke beberapa node dan menggabungkan hasilnya. Raspberry Pi cluster cocok untuk belajar konsep parallel computing.

Install MPI

# ============================================
# Install MPI di semua node
# ============================================

# Install OpenMPI
sudo apt install openmpi-bin libopenmpi-dev -y

# Buat file host list
cat <<'EOF' > ~/mpi_hosts
pi-master slots=1
pi-worker1 slots=1
pi-worker2 slots=1
pi-worker3 slots=1
EOF

# Test MPI dengan hello world
cat <<'EOF' > mpi_hello.c
#include <mpi.h>
#include <stdio.h>

int main(int argc, char** argv) {
    MPI_Init(&argc, &argv);

    int rank, size;
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);

    char hostname[256];
    gethostname(hostname, 256);

    printf("Hello from rank %d of %d on %s\n",
           rank, size, hostname);

    MPI_Finalize();
    return 0;
}
EOF

mpicc -o mpi_hello mpi_hello.c
mpirun --hostfile ~/mpi_hosts -np 4 ./mpi_hello

# Contoh: Parallel Pi Calculation
cat <<'EOF' > mpi_pi.c
#include <mpi.h>
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char** argv) {
    MPI_Init(&argc, &argv);

    int rank, size;
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);

    long num_steps = 10000000;
    double step = 1.0 / (double)num_steps;
    double local_sum = 0.0;

    // Setiap proses menghitung sebagian
    for (long i = rank; i < num_steps; i += size) {
        double x = (i + 0.5) * step;
        local_sum += 4.0 / (1.0 + x * x);
    }
    local_sum *= step;

    // Kumpulkan hasil dari semua proses
    double pi = 0.0;
    MPI_Reduce(&local_sum, &pi, 1, MPI_DOUBLE,
               MPI_SUM, 0, MPI_COMM_WORLD);

    if (rank == 0) {
        printf("Pi = %.15f\n", pi);
    }

    MPI_Finalize();
    return 0;
}
EOF

mpicc -O2 -o mpi_pi mpi_pi.c
time mpirun --hostfile ~/mpi_hosts -np 4 ./mpi_pi
# Bandingkan dengan single node:
time mpirun -np 1 ./mpi_pi

8. Monitoring Cluster

Monitoring sangat penting untuk cluster production. Berikut setup monitoring menggunakan Prometheus dan Grafana yang ringan untuk Raspberry Pi.

# ============================================
# Install Prometheus + Grafana (di master)
# ============================================

# Gunakan Docker untuk deploy monitoring stack
cat <<'EOF' > monitoring-stack.yml
version: '3.8'

services:
  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus-data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.retention.time=30d'
    restart: unless-stopped

  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    volumes:
      - grafana-data:/var/lib/grafana
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin123
    restart: unless-stopped

  node-exporter:
    image: prom/node-exporter:latest
    ports:
      - "9100:9100"
    pid: host
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
    restart: unless-stopped

volumes:
  prometheus-data:
  grafana-data:
EOF

# Prometheus config
cat <<'EOF' > prometheus.yml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'node-exporter'
    static_configs:
      - targets:
        - 'pi-master:9100'
        - 'pi-worker1:9100'
        - 'pi-worker2:9100'
        - 'pi-worker3:9100'
EOF

# Deploy monitoring stack
docker stack deploy -c monitoring-stack.yml monitoring

# Akses Grafana di http://pi-master:3000
# Login: admin / admin123
# Import dashboard ID 1860 (Node Exporter Full)

9. Troubleshooting

10. Quiz: Uji Pemahamanmu!

Setelah membaca tutorial di atas, jawablah 5 pertanyaan berikut untuk menguji pemahamanmu tentang Raspberry Pi Cluster Computing: