Node-RED Advanced Flow Programming

1. Node-RED: Refresher & Advanced Concepts

Node-RED adalah tool visual programming berbasis Node.js yang memungkinkan kamu membuat flow (alur kerja) dengan cara drag-and-drop node. Meskipun sering digunakan untuk IoT, Node-RED sangat fleksibel untuk automasi apapun — API integration, data processing, dashboard, dan banyak lagi.

Tutorial ini fokus pada fitur advanced yang membuat Node-RED layak digunakan di production: subflows untuk reusable components, context store untuk state management, custom nodes untuk logika khusus, clustering untuk high availability, dan authentication untuk keamanan.

Arsitektur Node-RED

# Komponen Node-RED Runtime:
# - Editor (browser): Visual flow editor berbasis web
# - Runtime (Node.js): Mengeksekusi flow di server
# - Flows: JSON file berisi definisi nodes dan connections
# - Registry: Pendaftaran node types dan credentials

# Flow execution model:
# Event-driven: Setiap node dipicu oleh message (msg)
# Non-blocking: Node berjalan secara async
# Message-passing: Node berkomunikasi via msg object

# Struktur msg object:
# msg.payload   — data utama (bebas tipe)
# msg.topic     — metadata topic (untuk MQTT/routing)
# msg._msgid    — message ID unik
# msg._event    — event context (untuk catch/error)

Ringkasan Node Types

Kategori	Contoh Node	Kegunaan
Input	inject, mqtt in, http in, websocket in	Sumber data/trigger
Output	debug, mqtt out, http response	Kirim data keluar
Function	function, switch, change, json	Transformasi data
Storage	file, file in, csv, html	I/O file
Dashboard	ui_gauge, ui_chart, ui_switch	Visualisasi dashboard
Advanced	link, catch, status, complete	Flow control

2. Subflows & Reusable Components

Subflow adalah flow yang dibungkus menjadi satu node reusable. Ini sangat penting untuk menjaga flow tetap terorganisir dan menghindari duplikasi.

Membuat Subflow

# Cara membuat subflow:
# 1. Menu → Subflows → Create Subflow
# 2. Tambahkan input/output node di subflow
# 3. Bangun logika di dalam subflow
# 4. Subflow muncul di palette sebagai node baru

# Contoh: Subflow "MQTT Alert Processor"
# Input: msg.payload (raw MQTT message)
# Processing:
#   - Parse JSON
#   - Cek threshold
#   - Format alert message
# Output: msg.payload (formatted alert atau null)

# Subflow properties (dapat dikonfigurasi per instance):
# - environment variables (typed: str, num, bool, json, flow, global)
# - input/output count
# - icon dan category

Subflow Environment Variables

// Contoh subflow: Threshold Alert
// Environment variables:
// - threshold (number): Batas nilai
// - alertType (string): "email" atau "sms" atau "mqtt"
// - mqttTopic (string): Topic untuk publish alert

// Di dalam function node subflow:
let value = msg.payload.value;
let threshold = env.get("threshold");  // akses env var
let alertType = env.get("alertType");

if (value > threshold) {
    msg.payload = {
        alert: true,
        type: alertType,
        value: value,
        threshold: threshold,
        message: `Alert: ${msg.payload.device} = ${value} (threshold: ${threshold})`,
        timestamp: Date.now()
    };
    return [msg, null]; // output 1: alert, output 2: null
} else {
    return [null, msg]; // output 1: null, output 2: normal
}

Nested Subflows

# Subflow bisa berisi subflow lain (nested):
# Main Flow
#   ├── Subflow: Data Validator
#   │     ├── Subflow: Type Checker
#   │     └── Subflow: Range Checker
#   ├── Subflow: Alert Processor
#   │     ├── Subflow: Email Sender
#   │     └── Subflow: MQTT Publisher
#   └── Subflow: Data Storage
#         ├── Subflow: InfluxDB Writer
#         └── Subflow: File Archiver

# Tips: Hindari nesting lebih dari 3 level untuk readability

3. Context Store & State Management

Node-RED memiliki sistem context yang memungkinkan penyimpanan data antara message. Ada tiga level context: node (per-node), flow (per-flow/tab), dan global (seluruh runtime).

Context Levels

// Node context — hanya bisa diakses oleh node yang sama
// Berguna untuk: state lokal node (counter, last value, accumulator)
let count = node.get("count") || 0;
count++;
node.set("count", count);
msg.payload = { count: count };

// Flow context — bisa diakses oleh semua node dalam flow yang sama
// Berguna untuk: shared state antar node dalam satu flow
flow.set("lastSensorValue", msg.payload.temperature);
let lastValue = flow.get("lastSensorValue");

// Global context — bisa diakses oleh semua node di semua flow
// Berguna untuk: konfigurasi shared, global state
global.set("config.alertThreshold", 40);
let threshold = global.get("config.alertThreshold");

// Async context access (v3+):
let value = flow.get("key", storeName, (err, val) => {
    if (err) node.error(err);
    else node.log("Value: " + val);
});

Persistent Context Store

# Konfigurasi persistent context di settings.js
# Default: in-memory (hilang saat restart)
# Tambahkan persistent store:

contextStorage:
  default:
    module: memory
  persistent:
    module: localfilesystem
    config:
      base: "context-data"
      dir: "context"
      cache: true  # cache di memory untuk performa

# Usage:
// Simpan ke persistent store
flow.set("lastCalibration", calibrationData, "persistent");

// Baca dari persistent store
let cal = flow.get("lastCalibration", "persistent");

# Alternatif modules:
# - @node-red-contrib/thredis (Redis-backed)
# - node-red-contrib-postgresql-context (PostgreSQL-backed)

4. Membuat Custom Nodes

Untuk logika yang kompleks atau reusable, kamu bisa membuat custom nodes. Custom node terdiri dari file HTML (editor definition) dan file JS (runtime logic).

Struktur Custom Node

# Package structure:
# node-red-contrib-iot-sensor/
# ├── package.json
# ├── README.md
# ├── nodes/
# │   ├── sensor-validator.js    # Runtime logic
# │   └── sensor-validator.html  # Editor definition
# └── icons/
#     └── sensor.svg

Node Runtime (sensor-validator.js)

// nodes/sensor-validator.js
module.exports = function(RED) {
    function SensorValidatorNode(config) {
        RED.nodes.createNode(this, config);
        const node = this;
        
        // Baca konfigurasi dari editor
        node.sensorType = config.sensorType;
        node.minValue = parseFloat(config.minValue);
        node.maxValue = parseFloat(config.maxValue);
        node.name = config.name;

        // Handler saat menerima message
        node.on('input', function(msg, send, done) {
            // send() = mengirim message ke output
            // done() = signal selesai processing
            
            try {
                const value = parseFloat(msg.payload.value);
                
                if (isNaN(value)) {
                    node.warn("Invalid sensor value: " + msg.payload.value);
                    done();
                    return;
                }
                
                // Validasi range
                if (value < node.minValue || value > node.maxValue) {
                    msg.payload = {
                        valid: false,
                        value: value,
                        reason: `Value ${value} out of range [${node.minValue}, ${node.maxValue}]`,
                        timestamp: Date.now()
                    };
                    // Output 2: invalid
                    send([null, msg]);
                } else {
                    msg.payload.valid = true;
                    // Output 1: valid
                    send([msg, null]);
                }
                
                done();
            } catch (err) {
                // Error handling
                done(err);
            }
        });

        // Cleanup saat node dihapus/deploy
        node.on('close', function(done) {
            done();
        });
    }

    // Register node type
    RED.nodes.registerType("sensor-validator", SensorValidatorNode);
};

Node Editor (sensor-validator.html)

<!-- nodes/sensor-validator.html -->
<!-- Node definition untuk editor -->
<script type="text/javascript">
    RED.nodes.registerType('sensor-validator', {
        category: 'IoT',                          // Palette category
        color: '#4CAF50',                          // Warna node
        defaults: {
            name: { value: "" },                   // Nama node (opsional)
            sensorType: { value: "temperature" },  // Tipe sensor
            minValue: { value: "-40", required: true },
            maxValue: { value: "80", required: true }
        },
        inputs: 1,                                 // 1 input port
        outputs: 2,                                // 2 output ports
        icon: "sensor.svg",
        label: function() { return this.name || "sensor-validator"; },
        paletteLabel: "sensor validator"
    });
</script>

<!-- Edit dialog -->
<script type="text/html" data-template-name="sensor-validator">
    <div class="form-row">
        <label for="node-input-name"><i class="fa fa-tag"></i> Name</label>
        <input type="text" id="node-input-name" placeholder="Name">
    </div>
    <div class="form-row">
        <label for="node-input-sensorType">Sensor Type</label>
        <select id="node-input-sensorType">
            <option value="temperature">Temperature</option>
            <option value="humidity">Humidity</option>
            <option value="pressure">Pressure</option>
            <option value="custom">Custom</option>
        </select>
    </div>
    <div class="form-row">
        <label for="node-input-minValue">Min Value</label>
        <input type="text" id="node-input-minValue" placeholder="-40">
    </div>
    <div class="form-row">
        <label for="node-input-maxValue">Max Value</label>
        <input type="text" id="node-input-maxValue" placeholder="80">
    </div>
</script>

<!-- Help text -->
<script type="text/html" data-help-name="sensor-validator">
    <p>Validates sensor data against configurable min/max range.</p>
    <h3>Outputs</h3>
    <ol><li>Valid data</li><li>Invalid data (out of range)</li></ol>
</script>

5. Clustering & High Availability

Untuk production, Node-RED bisa dijalankan dalam mode cluster menggunakan Node.js cluster module atau external load balancer.

// settings.js — konfigurasi untuk production
module.exports = {
    flowFile: 'flows.json',
    credentialSecret: process.env.NODE_RED_CREDENTIAL_SECRET || "my-secret-key",
    
    // UI & Editor
    uiPort: process.env.PORT || 1880,
    httpAdminRoot: '/admin',      // Editor di /admin
    httpNodeRoot: '/api',         // API nodes di /api
    ui: { path: "ui" },           // Dashboard di /ui
    
    // Security — basicAuth untuk editor
    adminAuth: {
        type: "credentials",
        users: [{
            username: "admin",
            password: "$2a$08$...",  // bcrypt hashed
            permissions: "*"
        }]
    },
    
    // Node-level API auth
    httpNodeAuth: { user: "api", pass: "$2a$08$..." },
    
    // Logging
    logging: {
        console: {
            level: "info",
            metrics: false,
            audit: true
        }
    },
    
    // Context storage
    contextStorage: {
        default: { module: "memory" },
        persistent: {
            module: "localfilesystem",
            config: { base: "/data/context", cache: true }
        }
    },
    
    // Function timeout (prevent infinite loops)
    functionTimeout: 30,
    
    // Debug length limit
    debugMaxLength: 1000,
    
    // Diagnostics
    diagnostics: { enabled: true, ui: true }
};

Docker Deployment

# docker-compose.yml — production
version: "3.8"
services:
  nodered:
    image: nodered/node-red:3.1
    ports:
      - "1880:1880"
    volumes:
      - nodered-data:/data
      - ./settings.js:/data/settings.js
    environment:
      - NODE_RED_CREDENTIAL_SECRET=${CRED_SECRET}
      - TZ=Asia/Jakarta
    restart: unless-stopped
    deploy:
      resources:
        limits:
          memory: 512M
          cpus: "0.5"

  mosquitto:
    image: eclipse-mosquitto:2
    ports:
      - "1883:1883"
    volumes:
      - ./mosquitto.conf:/mosquitto/config/mosquitto.conf
      - mosquitto-data:/mosquitto/data
    restart: unless-stopped

volumes:
  nodered-data:
  mosquitto-data:

6. Authentication & Authorization

Node-RED mendukung beberapa metode autentikasi: basic authentication (built-in), credentials file, OAuth, dan custom authentication plugin.

// settings.js — Authentication configurations

// 1. Basic Auth (paling sederhana)
adminAuth: {
    type: "credentials",
    users: [{
        username: "admin",
        password: bcrypt.hashSync("admin-password", 8),
        permissions: "*"
    }],
    default: {
        permissions: "read"  // anonymous user bisa lihat tapi tidak edit
    }
}

// 2. Multiple users dengan permission berbeda
adminAuth: {
    type: "credentials",
    users: [
        { username: "admin", password: "$2a$...", permissions: "*" },
        { username: "developer", password: "$2a$...", permissions: ["read", "write"] },
        { username: "viewer", password: "$2a$...", permissions: "read" }
    ]
}

// 3. API Key authentication untuk HTTP nodes
httpNodeAuth: {
    user: "api-user",
    pass: bcrypt.hashSync("api-password", 8)
}

// 4. Token-based auth (via custom plugin)
// npm install node-red-contrib-auth-oidc
adminAuth: require("node-red-contrib-auth-oidc")({
    issuer: "https://auth.example.com",
    client_id: "nodered",
    client_secret: "secret",
    allowed_users: ["admin@example.com"]
})

7. Error Handling & Debugging

Error handling yang baik sangat penting untuk production flow. Node-RED menyediakan beberapa mekanisme: catch node, status node, dan error reporting.

// Flow-level error handling dengan Catch node
// Setiap flow tab harus punya Catch node:
// - Catch: menangkap error dari semua node di tab
// - Catch (specific): menangkap error dari node tertentu

// Di function node, trigger error:
try {
    let result = JSON.parse(msg.payload);
    msg.payload = result;
    return msg;
} catch (err) {
    // Cara 1: node.error() — ditangkap oleh Catch node
    node.error("Parse error: " + err.message, msg);
    return null;
}

// Cara 2: Mengirim ke catch dengan msg._error
msg._error = {
    message: "Custom error",
    source: { id: node.id, type: node.type },
    timestamp: Date.now()
};
node.send(msg);  // Kirim ke output, bukan error

// Status indicator (icon di bawah node):
node.status({ fill: "green", shape: "dot", text: "connected" });
node.status({ fill: "yellow", shape: "ring", text: "reconnecting" });
node.status({ fill: "red", shape: "dot", text: "error: timeout" });
node.status({}); // Clear status

8. Best Practices & Production Tips

Praktik	Deskripsi
Gunakan Subflows	Modularisasi flow untuk reuse dan maintainability
Error Handling	Selalu tambahkan Catch node di setiap flow tab
Rate Limiting	Gunakan delay node untuk menghindari message flood
Environment Variables	Gunakan env vars, bukan hardcode nilai
Version Control	Export flow JSON dan commit ke Git
Monitoring	Pasang status node untuk monitoring koneksi
Backup	Backup /data directory secara berkala
Resource Limits	Set functionTimeout dan debugMaxLength

9. Quiz: Uji Pemahamanmu!

Setelah membaca tutorial di atas, jawablah 5 pertanyaan berikut untuk menguji pemahamanmu tentang Node-RED Advanced: