What is a broker topology and how does it work?
Broker Topology Explained
Audience: This document is intended for stakeholders without a technical background. It explains how a broker-based architecture works, what the key concepts mean, and how it differs from more familiar approaches such as APIs and GraphQL.
In Mantsu, both the MQTT broker (OT machine data) and the Redpanda streaming broker (business events) follow the broker topology described on this page.
The Core Idea: A Shared Noticeboard
The easiest way to understand a broker topology is to think of a noticeboard in a shared office. Anyone can pin a note to the board. Anyone else can walk up and read the notes that interest them. The person who pinned the note does not need to know who will read it, and the reader does not need to know who wrote it. The noticeboard is the broker.
This is fundamentally different from a phone call (an API), where one party dials another directly, waits for an answer, and the conversation is private between the two.
Publishers, Subscribers, and Messages
Three roles make up a broker topology:
Publisher A publisher is any system or device that sends data to the broker. On the shop floor this is typically a machine, a sensor, or a controller. In an enterprise context it might be an ERP system, a production planning tool, or a quality management application. The publisher does not know who will receive its data. It simply sends a message and moves on.
Broker The broker sits in the middle. It receives messages from publishers, organizes them by topic, and delivers them to any subscriber that has expressed interest in that topic. The broker handles routing so that publishers and subscribers never need to know about each other.
Subscriber A subscriber is any system or application that has told the broker: "I want to receive messages on this topic." When a new message arrives on that topic, the broker delivers it. A subscriber can follow many topics at once, and many subscribers can follow the same topic simultaneously.
graph LR
subgraph Publishers
P1[Machine sensor]
P2[Production controller]
P3[Quality system]
end
subgraph Broker["📡 Broker"]
T1(Topic: machine/temperature)
T2(Topic: production/output)
T3(Topic: quality/rejects)
end
subgraph Subscribers
S1[MES application]
S2[Reporting dashboard]
S3[Maintenance alert]
end
P1 -->|publishes| T1
P2 -->|publishes| T2
P3 -->|publishes| T3
T1 -->|delivers| S1
T1 -->|delivers| S3
T2 -->|delivers| S1
T2 -->|delivers| S2
T3 -->|delivers| S2What is a message? A message is a small, self-contained packet of information. It has a topic (the address it belongs to) and a payload (the actual content). A payload might be a temperature reading, a production count, an alarm event, or a quality measurement. Messages are typically lightweight and structured so that any system can read them without needing to know the internals of the sender.
Do Messages Stay on the Broker?
This depends on the type of broker. The answer is one of the most important practical differences between the two main broker technologies used in manufacturing: MQTT and Kafka.
MQTT: live data, last known value
MQTT is designed for lightweight, real-time communication, making it ideal for sensors and machines that send frequent updates. By default, a message sent over MQTT is delivered to whoever is currently subscribed and then discarded. If no one is subscribed at that moment, the message is lost.
MQTT does support a concept called a retained message: the broker keeps the single most recent message on a topic, so a new subscriber immediately receives the latest known value when it connects. Think of it as a whiteboard that always shows the current reading, overwritten each time a new value arrives.
MQTT is therefore well suited to answering: "What is the current value right now?"
Kafka: a durable, replayable log
Kafka (and its equivalent Redpanda) works differently. Every message is written to a persistent log on disk and kept for a configurable period: hours, days, or indefinitely. Subscribers do not just receive new messages; they can also go back and read messages from the past.
Think of Kafka as a journal rather than a whiteboard. The journal keeps every entry ever made. A new reader can start from the beginning and read the full history, or start from any point in time, or simply follow along from the current moment.
This makes Kafka well suited to answering: "What happened, and in what order?" It also means that if a downstream application goes offline temporarily, it can catch up on everything it missed when it comes back.
graph TD
subgraph MQTT["📡 MQTT Broker"]
direction LR
MW[Whiteboard\nshows latest value only]
end
subgraph Kafka["📋 Kafka / Redpanda"]
direction LR
KL[Persistent log\nfull history retained]
end
P1[Machine sensor] -->|publishes| MW
P1 -->|publishes| KL
MW -->|delivers current value| S1[Live dashboard]
KL -->|delivers from any point in history| S2[Reporting application]
KL -->|replays missed messages| S3[System that was offline]Summary: MQTT vs Kafka
| MQTT | Kafka / Redpanda |
|---|---|---|
Primary use | Real-time sensor data, OT layer | Event streaming, IT layer |
Message retention | Last value only (optional) | Full history, configurable duration |
Can replay past messages? | No, but MQTT5 does allow storage for specific topics | Yes |
Designed for | Lightweight devices, low bandwidth | High-throughput enterprise systems |
Best question it answers | What is the value right now? | What happened, and when? |
In a well-designed manufacturing architecture, both are used together: MQTT at plant level for machine communication, Kafka at enterprise level for durable event streaming and integration.
How Is This Different from an API?
An API (Application Programming Interface) works on a request-response model. One application asks another for data, the second application responds, and the exchange is complete. Neither party is involved until one of them initiates a new request.
sequenceDiagram
participant App as Application
participant API as API / Server
App->>API: Request: give me the current temperature
API-->>App: Response: 74.2°C
Note over App,API: Nothing happens until the next requestA broker topology works on a publish-subscribe model. Data flows continuously. The subscriber does not ask; it simply receives new data the moment the publisher sends it.
sequenceDiagram
participant Sensor as Machine sensor
participant Broker as Broker
participant App as Application
Sensor->>Broker: 74.2°C
Broker->>App: 74.2°C
Sensor->>Broker: 74.5°C
Broker->>App: 74.5°C
Sensor->>Broker: 75.1°C
Broker->>App: 75.1°C
Note over Sensor,App: Data flows continuously without requestsIn practice, APIs come in different forms. The two most common are REST and SOAP. A REST API communicates over standard web protocols and is widely used in modern applications and cloud services. A SOAP API is an older, more formal standard that is still common in enterprise systems such as ERP and LIMS platforms. Despite their differences in technical style, both follow the same fundamental principle: one party sends a request, the other responds, and the exchange ends there.
The practical difference is coupling and timing. With an API, the caller and the provider are directly connected: the caller must know the address of the provider, both must be available at the same time, and if the provider is slow the caller waits. With a broker, the publisher and subscriber are completely independent. They do not know each other, they do not need to be online at the same time, and either side can change without affecting the other.
****** | API (request-response) | Broker (publish-subscribe) |
|---|---|---|
Who initiates? | The consumer (caller) | The producer (publisher) |
When is data received? | Only when requested | Whenever new data arrives |
Are sender and receiver coupled? | Yes, directly connected | No, fully independent |
What if the receiver is offline? | Request fails | Kafka: message is stored; MQTT: message may be lost |
Best suited for | Retrieving specific data on demand | Reacting to events as they happen |
How Is This Different from GraphQL?
GraphQL is a type of API. Like a standard API it follows the request-response model, but it gives the consumer much more control over exactly what data is returned. Instead of receiving a fixed set of fields, the caller writes a precise query describing exactly which fields it needs, and nothing more.
sequenceDiagram
participant App as Dashboard
participant GQL as GraphQL API
App->>GQL: Query: give me machine ID, temperature, and shift output for line 3
GQL-->>App: Exactly those three fields for line 3
Note over App,GQL: Data shape is defined by the caller, not the providerGraphQL is excellent for building dashboards and reporting interfaces that need to pull specific, structured data on demand. It is efficient because callers only receive what they asked for, and flexible because different callers can request different shapes of data from the same API.
Where GraphQL ends and broker topology begins
GraphQL answers the question: "Show me a specific set of data as it stands right now."
A broker answers the question: "Tell me whenever something changes or happens."
The difference is querying versus streaming.
A GraphQL query is a snapshot. You ask, you receive, you move on. If you want to know whether the temperature changed, you must ask again. You can automate this by polling repeatedly, but this is wasteful and introduces a delay between the real event and your awareness of it.
A broker subscription is a continuous feed. The moment a machine temperature changes, the broker delivers that change to every interested subscriber, instantly and without any polling. If an alarm threshold is breached, a broker-connected system can react within milliseconds.
graph TD
subgraph Polling["GraphQL: polling for changes"]
Q1[Query at 09:00] --> R1[74.2°C]
Q2[Query at 09:01] --> R2[74.2°C]
Q3[Query at 09:02] --> R3[75.8°C ⚠️]
note1[Change happened at 09:01:30
but was only discovered at 09:02]
end
subgraph Streaming["Broker: event-driven"]
E1[09:01:30: sensor publishes 75.8°C]
E2[Broker delivers instantly]
E3[System reacts at 09:01:30 ⚡]
E1 --> E2 --> E3
endWhen GraphQL and APIs are the right tool
GraphQL and APIs shine whenever the consumer needs to ask a specific question and receive a precise, structured answer. The consumer is in control: it decides what it wants, when it wants it, and what shape the response should take.
Good examples in a manufacturing context:
Reporting and analytics. A shift report needs to show total output, downtime, and scrap rate for a specific line over a specific eight-hour window. The reporting tool queries the database at the end of the shift with exactly those parameters and receives exactly those numbers. There is no need for a continuous stream; the data already exists and the question is well-defined.
Retrieving results from a LIMS. A Laboratory Information Management System (LIMS) stores test results, certifications, and quality verdicts. When a production operator or an ERP system needs to know whether a batch has passed quality release, it asks the LIMS a direct question: "Give me the test results for batch 2024-B-0471." The LIMS responds with the full result record. This is a point-in-time lookup against a system of record. The consumer cares about a specific batch, at a specific moment, and wants a complete and authoritative answer. An API or GraphQL query is the natural fit.
Building flexible dashboards. A management dashboard may need to combine data from multiple sources: production counts from the MES, energy consumption from a utility system, and quality metrics from the LIMS. GraphQL allows the dashboard to request precisely the fields it needs from each source in a single query, without receiving large amounts of irrelevant data.
sequenceDiagram
participant ERP as ERP System
participant LIMS as LIMS API
ERP->>LIMS: Query: release status for batch 2024-B-0471
LIMS-->>ERP: Batch passed. Certificate number: QC-8812. Released by: J. Peeters. Date: 2024-06-09.
Note over ERP,LIMS: A specific question, a specific answer, at a specific momentIn all of these cases the consumer knows exactly what it is looking for. The data already exists somewhere. The exchange is intentional and bounded.
When a broker is the right tool
A broker works on a fundamentally different principle. The publisher sends a message to the broker and immediately moves on. It does not wait for a response. It does not know who will receive the message. It does not care what the receiver will do with the data. Its only responsibility is to publish accurately and consistently.
This is what makes a broker powerful in a manufacturing environment. A temperature sensor does not know whether its data will be used by a dashboard, a maintenance alert, an energy optimization algorithm, or a compliance logging system. It does not need to know. It simply publishes the reading, and the broker takes care of delivery to whoever is interested.
graph LR
subgraph Publisher
S[Temperature sensor
publishes 74.2°C]
end
subgraph Broker["📡 Broker"]
T(Topic: line3/temperature)
end
subgraph Subscribers
D[Live dashboard
displays the value]
A[Alerting system
checks against threshold]
E[Energy monitor
logs for efficiency analysis]
H[Historian
writes to long-term storage]
end
S -->|publishes once| T
T --> D
T --> A
T --> E
T --> H
style S fill:#f0f0f0
style T fill:#e8f4f8The sensor publishes once. Four completely different systems receive and use the data for four completely different purposes. Adding a fifth subscriber, say a new AI-based anomaly detection system, requires no changes to the sensor, no changes to the broker configuration, and no changes to any of the existing subscribers. The new system simply subscribes to the topic and starts receiving data.
This decoupling is what separates a broker from an API. With an API, the provider must know about every consumer and build an interface for each use case. With a broker, the provider publishes data once and the organization decides independently how to use it.
A broker is therefore the right tool when:
- Something is happening now and multiple systems need to react to it
- The producer cannot or should not need to know who consumes its data
- New consumers may be added in the future without changing the source system
- Speed of delivery matters more than the precision of the question being answered
Using both together
GraphQL and broker topologies are not competing approaches. They address different moments in the data lifecycle and are most effective when used in combination.
The broker handles the flow of live events. Something happens on the shop floor, the message travels through the broker to every interested system, and each system acts on it in its own way. A historian subscribes and writes every event to a database. An alerting system subscribes and checks whether thresholds have been crossed.
Once the data is in the database, a GraphQL API can expose it for structured queries. A reporting tool, a LIMS integration, or an ERP system can then ask precise questions about historical data without needing to connect to the broker directly.
graph TD
subgraph Live["Real-time layer (broker)"]
Sensor[Machine or sensor]
Broker[Broker]
Alert[Alerting system]
Historian[Historian]
end
subgraph Historical["Historical layer (API)"]
DB[(Database)]
GQL[GraphQL API]
Report[Reporting tool]
LIMS[LIMS / ERP query]
end
Sensor -->|publishes event| Broker
Broker -->|delivers instantly| Alert
Broker -->|delivers instantly| Historian
Historian -->|writes to| DB
DB -->|exposed via| GQL
GQL -->|answers specific questions| Report
GQL -->|answers specific questions| LIMSThe broker answers: "What is happening right now, and who needs to know?" The API answers: "What happened, what does it mean, and what exactly do I need to know about it?"
Both questions are valid. Both need to be answered. The skill lies in knowing which tool is responsible for which question.
| GraphQL / API | Broker (pub-sub) |
|---|---|---|
Model | Query on demand | Continuous event stream |
Who initiates the exchange? | The consumer | The producer |
Data freshness | As of the moment of the query | Real-time, pushed immediately |
Does the producer know its consumers? | Yes, the API is built for specific use cases | No, the publisher does not know or care |
Best for | Reporting, lab results, point-in-time lookups | Automation, alerts, live monitoring |
Can new consumers be added easily? | Only if the API supports their use case | Yes, any system can subscribe without changes |
Handles history? | Yes, queries the database directly | Kafka: yes; MQTT: last value only |
Can they work together? | Yes: the broker feeds data and GraphQL exposes it for queries | |
Putting It All Together
A modern manufacturing data architecture typically combines all of these approaches, each used where it fits best:
graph TD
subgraph OT["🏭 Plant floor"]
Machines[Machines & sensors]
MQTT[MQTT Broker\nreal-time OT data]
end
subgraph IT["🏢 Enterprise layer"]
Kafka[Kafka / Redpanda\ndurable event log]
DB[(Database\nhistorical storage)]
GQL[GraphQL API\nquery interface]
end
subgraph Consumers["📊 Applications"]
Live[Live monitoring\ndashboard]
Report[Reporting &\nanalytics]
Alert[Alerting &\nautomation]
end
Machines -->|publishes live data| MQTT
MQTT -->|selected data promoted| Kafka
Kafka -->|streams events| Alert
Kafka -->|writes history| DB
DB -->|queried via| GQL
GQL -->|serves structured data| Report
Kafka -->|streams live| LiveEach layer plays a different role:
- MQTT handles fast, lightweight communication on the plant floor
- Kafka provides a durable, replayable log of everything that matters at enterprise level
- A database stores historical data in a queryable form
- GraphQL gives dashboards and reports a flexible, efficient way to retrieve exactly what they need
This combination means that real-time alerts fire the moment something happens, reporting tools can query history at any time, and every layer remains independent and maintainable on its own.
Frequently asked questions
Where does Mantsu use the broker topology? For both tiers of the UNS: an MQTT broker for OT machine data and the Redpanda (Kafka) broker for IT business events.
In Mantsu, does a publisher know who consumes its data? No. Publishers and subscribers are fully decoupled; new consumers subscribe without any change to the publisher.
Do messages stay on the broker in Mantsu? It depends on the tier: the MQTT broker typically retains the latest value per topic, while the Redpanda (Kafka) topics keep a configurable, replayable history.
Why does Mantsu use brokers instead of only APIs? A broker lets data be published once and read by many, decoupling systems; Mantsu still uses REST or OData APIs for transactional calls such as ERP exchanges where request/response fits.
Summary: A broker topology works like a shared noticeboard: publishers post messages to topics, subscribers read what they need, and neither knows about the other. Mantsu applies this pattern for machine data (MQTT) and business events (Redpanda).