Six systems that all need to talk to each other directly add up to 15 possible integrations. Add a seventh and you're at 21. That is usually when IT managers start looking into middleware, often without knowing exactly what it covers or whether they really need it.
Middleware is software that sits between your applications and passes messages, data, and processes from one system to another, translating along the way. Each system talks only to the middleware instead of to every other system. That keeps the number of integrations down and puts translation, error handling, and logging in one place.
Below, you'll read why point-to-point integrations grow into spaghetti, which types of middleware exist, when a single API integration is enough, and what to get right once you choose. It's written for IT managers, so every technical term gets a plain-language explanation.
Why middleware: point-to-point integrations grow into spaghetti
Software integration starts out tidy: the web store sends orders to the ERP, done. An ERP, or enterprise resource planning system, is the central package that runs your orders, stock, purchasing, and finance. The second and third integrations usually go fine too. The trouble starts when every system talks directly to every other system, because integrations then multiply faster than systems.
The math is simple. Three systems give you three possible integrations, four give you six, and six give you 15. Each integration has its own scripts, error handling, and assumptions about the data. Change one field in the ERP and you have to walk through every integration, one by one.
Mark, IT manager at a technical wholesaler with 120 employees, knows this well. His organization runs on an ERP, a web store, a WMS, and an accounting package. A WMS, or warehouse management system, tells the warehouse what to pick, pack, and ship. In five years, 11 integrations have grown up around those systems, built by three vendors and one colleague who has since left. When the ERP switched to a new product number format, three integrations stopped and nobody knew which script ran where. It took four days before orders flowed to the warehouse again.
With middleware, that picture looks different. Each system delivers its data to the integration layer and picks up what it needs there. Product numbers are translated in one place, logging lives in one place, and when something breaks, one overview shows which message got stuck. Our software integration page explains how we connect ERP, web store, and warehouse systems.
Types of middleware in plain language
The term middleware covers a family of solutions that all tackle the same problem, each at a different scale. These are the five types you'll meet in practice.
- API gateway. One central front door for all requests from outside, which checks who is knocking and forwards each request to the right system.
- Message queue. A waiting line where systems drop off messages, so the receiver picks them up at its own pace and nothing gets lost when a system is briefly offline.
- ESB (enterprise service bus). A central bus that every system plugs into and that routes, translates, and monitors messages, mostly used by large organizations with dozens of applications.
- iPaaS (integration platform as a service). A cloud-based integration platform, such as Alumio or Boomi, where you connect common packages through ready-made connectors instead of programming everything yourself. A connector is a prebuilt link to one specific package, such as Shopify.
- Your own integration layer. Custom software that contains exactly the translations, rules, and checks your organization needs, and that you own outright.
These types don't exclude each other. An iPaaS often uses a message queue under the hood, and a custom layer can sit behind an API gateway. The question is which combination fits your systems, volumes, and reliability requirements.
Under all of these sits the same building block: the API. An API, or application programming interface, is the agreed way a system answers questions and hands over data, and Wikipedia explains it in detail. Middleware uses those APIs and adds the layer on top that translates, monitors, and recovers. Our article on API integrations explains how a single integration works.
When an API integration is enough and when you need middleware
Middleware is an investment in structure, and it has to pay for itself. With two systems exchanging one type of message, a direct API integration is almost always the better choice. It's simpler, faster to build, and easier for the next administrator to understand.
The balance tips as soon as one of these four factors starts to pinch:
- Number of systems. From four or five systems sharing data, keeping an overview becomes a bigger problem than the technology.
- Volumes. Thousands of messages per hour call for a queue and buffering, which means holding messages temporarily so the slowest system doesn't set the pace for everyone.
- Error handling. When a failed order silently disappears until the customer calls, you need one central place that catches errors and resubmits them.
- Monitoring. Once finding out whether a message arrived means searching through four log files, central monitoring pays for itself quickly.
Aging systems play a role too. Joris, operations director at a machine builder with 80 employees, works with a production system from 2009 that nobody dares to touch. The new planning tool has to talk to it anyway. Instead of prying it open, Joris puts an integration layer around it that translates the outdated file export into messages the planning tool understands. The production system stays untouched, the rest of the organization works with current data, and the replacement can follow step by step.
We follow the same order in legacy modernization: first the interfaces, then the modules. Our article on legacy systems explains how to recognize and replace them. The Dutch National Cyber Security Centre (NCSC) describes the risks of legacy systems from a security perspective.
For Deron, we connected RFID scanners to the product catalog and the ERP, so production data lands in the ERP without manual retyping. RFID uses small radio tags to identify products automatically at a scanner. That case and other integration projects are on our client cases page.
Standard platform or your own integration layer
Once you've decided on middleware, the next choice is between a standard integration platform and a custom layer built for you.
| Situation | Usually the better choice |
|---|---|
| Mostly standard packages (Exact Online and AFAS, the Dutch accounting packages, plus Shopify and Salesforce) with existing connectors | iPaaS |
| Your own business rules decide how data is translated and checked | Custom integration layer |
| High volumes or strict turnaround requirements | Custom layer with a message queue |
| Little in-house development capacity and a need for quick results | iPaaS |
| Data may not leave your own environment (healthcare, government) | Custom layer on your own infrastructure |
An iPaaS usually charges per connector or per message volume, so the subscription grows with your message counts. A custom integration layer takes more build time up front and is yours afterward. The combination is often strongest: standard connectors for the common packages and custom APIs for the parts where your process differs from the market.
What you need to get right: logging, retries, versioning, and ownership
Middleware only solves the spaghetti problem if you get a few things right from day one. Otherwise you just move the chaos to one central place.
- Logging. Every message gets a unique identifier, so you can trace when an order came in, where it went, and where it got stuck.
- Retries. A failed message is resubmitted automatically, with a growing wait between attempts, and after a fixed number of tries it lands in a separate list for a person to review.
- Versioning. Systems change. A new version of the ERP API may only replace the old integration once it has been tested, which means running versions side by side for a while.
- Ownership. One person or team is responsible for the integration layer, knows the translation rules, and decides on changes. Without an owner, middleware turns back into a black box within two years.
In our experience, that last point is the most underestimated. Across 100+ projects in more than 30 years, the technology was rarely the problem. Things go wrong when knowledge about the integrations sits with one person, or with a vendor that has since disappeared. That's why the team that builds an integration layer also maintains it: 30+ engineers in Nijmegen and Sarajevo, ISO 9001 and ISO 27001 certified.
Frequently asked questions
What is middleware in simple terms?
Middleware is the layer in between that lets your systems talk to each other. Each system drops its messages off at that layer and picks up what it needs, instead of connecting directly to every other system. The layer translates the data, checks that messages arrive, and resubmits them when something goes wrong.
What is the difference between middleware and an API?
An API is the agreed entry point of a single system: the way that system answers questions and hands over data. Middleware uses the APIs of several systems and adds a layer on top that translates, routes, and handles errors. An API integration connects two systems, while middleware manages the connections between many systems at once.
When do you need middleware?
As soon as four or more systems share data, volumes climb, or a lost message causes direct damage. With two systems and one type of message, a direct API integration is usually enough. The tipping point is where maintaining separate integrations costs more time than setting up one central layer.
What is system integration?
System integration is everything that makes separate applications work as one coherent whole: integrations, data translation, process agreements, and maintenance. Middleware is one of the tools for that, alongside API integrations, file exchange, and cleaning up master data. In practice, the terms software integration and system integration are used interchangeably.
In summary
- Middleware sits between your applications and passes on messages, data, and processes, with translation and error handling in one place.
- Direct integrations grow faster than the number of systems: six systems mean 15 possible integrations.
- API gateway, message queue, ESB, iPaaS, and a custom integration layer are different answers to the same problem, and you often combine them.
- With two systems, an API integration is enough. From four or five systems, high volumes, or strict error handling requirements, middleware pays for itself.
- Logging, retries, versioning, and a clear owner determine whether the integration layer is still manageable three years from now.
The next step is an inventory: which systems you have, which messages flow between them, and where things go wrong today. With that overview, the choice between an API integration, an integration platform, or a custom layer is usually quick. Send us an overview of your systems and we'll think along on an approach that fits, with no obligation. Get in touch for an exploratory conversation. We'll also tell you honestly if a single API integration is all you need.
