middleware

What is middleware?

Middleware is software that bridges the gap between applications and operating systems (OSes) by providing a method for communication and data management. This capability is useful for applications that otherwise do not have any way to exchange data with other software tools or databases.

Why is it called middleware?

Middleware is so named because it sits between an OS and the applications that run on it in the software stack. Specifically, it sits between the client-side requests on the front end and the back-end resource being requested.

A client -- typically an application residing on the front end, i.e., where the user interacts with the software -- can make a request with network-based interactions. Resources such as databases, message queues, NoSQL data stores and file servers are part of the back end. Middleware sits between these ends to facilitate communication and data sharing between them.

How middleware works

Middleware abstracts communication processes between software components with the help of services. In doing so, it enables applications, even those written in different programming languages, to communicate.

Communication is established with a common messaging service to transfer data -- for example, Simple Object Access Protocol, representational state transfer (REST) or JavaScript Object Notation. Other communication technologies used include object request broker (ORB) and Extensible Markup Language.

Middleware components

Middleware typically consists of these components:

• Management console. Provides an overview of the middleware's activities and configurations.
• Client interface. Communicates with the involved applications, databases or other services.
• Internal interface. Ensures that components function cohesively.
• Platform interface. Ensures compatibility between software.
• Contract manager. Defines the procedures that applications must follow for data exchange.
• Session manager. Creates secure communication channels between the middleware and applications.
• Database manager. Integrates with different databases.
• Runtime monitor. Continuously monitors middleware data movements and generates reports for developers.

Why use middleware?

Early on, middleware was used mainly as a mediator between an application front end and back-end resources. Today, middleware serves many more functions, including the following.

Application development

Middleware enables organizations and developers to build applications more efficiently. For example, it is used in application integration. Organizations using multi-cloud and containerized environments also use middleware to cost-effectively develop and scale applications. In addition, middleware is used for data handling and application programming interface (API) management and in many authentication and messaging services.

Manage connectivity to various back-end resources

A middleware component might create a connection pool to provide fast and efficient access to a back-end database. It can also create connections to message queues and topics and manage connections to cloud-based resources, like Amazon Simple Storage Service.

Implement logic based on client requests

Middleware can take a user request, implement logic and customize the results. For example, it might recognize that a client browser making a request has the language header set to English, so the queries it makes to the back end should return nothing but English-based results. Another use case: A server could identify the geographical location of the client based on its IP address and return data to the client that prioritizes results that are located close by.

Handle concurrent processing, load balancing, transaction management

Middleware can typically scale vertically and horizontally to distribute incoming client requests over multiple cloud servers, virtual machines or availability zones. This is known as load balancing. It can also handle concurrency and transaction management issues, such as problems that arise when multiple clients attempt to simultaneously access or update a back-end resource.

Secure access to back-end resources

Middleware servers usually require both a secure connection using a technology like Secure Sockets Layer and authentication using either a username and password combination or a digital certificate. Once the client provides this security information, the middleware uses it to check if the client has the authority to access the data. Only after the rights are affirmed is the data sent from the middleware server to the client using a secure and encrypted connection.

Middleware categories

Most middleware falls into one of two categories. Both are available in on premises and cloud-based versions.

Enterprise application integration middleware

This type of middleware enables programmers to create business applications without having to customize integrations for each new application. It provides a layer of functionality for data consistency and business to business (B2B) integration so the two software applications can work together.

Platform middleware

Platform middleware provides a runtime hosting environment for software development and delivery. It may provide a container for application program logic, as well as tools like web servers, application servers and content management systems.

Generally, embedded or external communications platform middleware enables different communications tools to work together for application and service interaction. Resource management services, such as Microsoft Azure Resource Manager, host application program logic at runtime. Other components include Trusted Platform Modules and in-memory data grids.

Platform middleware products are also available on premises or through a cloud platform suite.

Types of middleware

There are many types of middleware, each with specific functions and uses. Common types include the following:

• Messaging middleware, also known as message-oriented middleware, facilitates communications between distributed applications and services.
• Object or ORB middleware enables software components or objects to communicate and interact with a program, such as containers, across distributed systems.
• Remote Procedure Call middleware provides a protocol that enables a program to request a service from another program located on a separate computer or network.
• Data or database middleware enables direct access to, and interaction with, databases; it typically includes Structured Query Language database software and its own data store.
• Transaction or transactional middleware ensures transactions move from one phase to the next via transaction process monitoring.
• Content-centric middleware enables client-side requests for specific content and abstracts and delivers it.
• Embedded middleware facilitates communication and integration between embedded apps and real-time OSes.
• API middleware enables developers to create and manage APIs and, in some cases, to monetize APIs.
• Asynchronous data streaming middleware enables data sharing between applications.

Other types of middleware are also available, such as device middleware to develop OS-specific apps, portal middleware to integrate related applications and ORB middleware to broker requests between multiple application objects or components on a distributed network.

Middleware products and providers

Most middleware products are available in both on-premises and cloud-based versions. They can be used independently or together, depending on the use case. Businesses may purchase middleware that's bundled into cloud services suites, such as middleware as a service or integration platform as a service, or choose independent middleware products for their specific needs.

Popular middleware vendors and products include the following:

• IBM Worklight. IBM Worklight is a mobile enterprise application platform to build, implement, connect, secure, integrate and manage mobile applications.
• Microsoft BizTalk Server. Microsoft BizTalk Server is an on-premises, stable middleware platform that connects various systems or applications via adapters.
• Red Hat JBoss Enterprise Application Platform. Red Hat JBoss EAP enables developers to build secure, scalable, high-performance apps for on-premises and cloud environments. It is part of Red Hat's Runtimes group of middleware products that facilitate the creation, integration and automation of enterprise applications.
• Oracle Fusion Middleware. Oracle Fusion Middleware is a suite of products to develop nimble enterprise applications in client-server, web and cloud environments.
• Tibco Connected Intelligence. Tibco Connected Intelligence is a middleware platform for connecting software; it also provides an analytics data fabric for analyzing data in motion and at rest.
• SAP Process Orchestration. SAP Process Orchestration is an on-premises middleware solution to automate and optimize business processes by developing custom applications and integrations.
• Apache Kafka and Apache Carmel. Apache Kafka is distributed middleware for developing event-driven applications and user experiences that require real-time data streaming. Apache Camel, meanwhile, provides an open source middleware for B2B and microservices-based environments.

Other middleware vendors include Flow, Temenos and WSO2 Carbon.

How to choose a middleware platform

Organizations should choose middleware based on what they are trying to achieve: data management, application development, business process automation and so on. It's also useful to assess the tools included in the middleware for their requirements. For example, some middleware includes specific tools to help with application development, such as IBM Worklight, Oracle Fusion Middleware and Red Hat JBoss EAP, while others include tools for connecting various systems, such as Microsoft BizTalk Server.

Additionally, the reliability, complexity and performance of a middleware product should be compared with other middleware products.

Another consideration is interoperability. Middleware should connect with a wide range of applications and enable interoperability. It should also include APIs and prebuilt integrations.

It's important to check that the middleware can be deployed in the organization's environment, whether it is on premises or cloud-based. A final factor to consider is community support as this enables developers to use open source tools to optimize their applications.

The evolution of middleware

In the 1980s, middleware was mainly used to integrate newer and legacy programs without needing to rewrite the latter's code. It was also used in server-based and shared library instances.

As its popularity grew, middleware was used to mediate an application's front end with a back-end resource. From 2000 until approximately 2010, middleware was synonymous with application servers. Since then, the applications of middleware started to grow.

Today, developers can use middleware to create smaller, microservices-based applications. These applications can connect to the required back-end resources, which may be distributed across multiple clouds, and then be deployed within a lightweight, container-based system, like Docker, and then on to a cloud-based service.

Some programming languages also include middleware capabilities. One example is Ballerina. Some infrastructure-as-a-service models have also enabled middleware capabilities.

Learn more about the middleware tool Oracle Fusion and how it integrates customer data. Follow this comprehensive guide to build a successful enterprise API strategy. Also, check out REST API design best practices and guidelines and this rundown of multi-runtime microservices architecture.