Middleware

Modern software operates within the Cloud or internet. Software requires its own software infrastructure. Some code runs on the Cloud or internet servers. Some code runs within the user’s browser. These languages are different. The back-end software tends to be structured procedural languages. The client-side software tends to be written using object oriented languages. And what runs between the client and the server?

Show Notes

The first time I heard the phrase middleware, I was writing software for FedEx with a terrific team of Oracle developers in the late 1990s. I had to ask for a definition. In short, middleware is software that run between two things. Middleware is software that is invisible to the user. Middleware is software but seems to fall outside the accepted definitions of an application or app. It is software’s own software-based infrastructure. Does that make any sense? Probably not.
I wrote my first lines of code on a PDP-11 microcomputer from Digital Electronics Corporation (DEC) during high school. I attended a school in a wealthy area surrounded by the massively burgeoning IT industry.  In my high school, four years before the IBM-PC came to the market, I learned BASIC. The name literally meant “Beginner’s All-purpose Symbolic Instruction Code”, and no does not relate to the American slang basic, which seems to be a bit of an insult. 

10 BEGIN 

20 PRINT “HELLO WORLD”

30 END

Hard to imagine how these building blocks gave us the world of modern digital commerce. Those roots go back even farther. That trip requires tipping our hat in honor of Ada Lovelace, Alen Turing.
My educational progression followed the global development of software and software tools. In 1982, heading to university, I bought an IBM-PC for $2,500. That was the discount a university professor got. Adjusted for 2022 dollars, the cost would approximate $7,500. 
As a college first year, I thought I would study computer science. Regrettably, my university’s systems were older than my high school’s systems. No way was I going to learn how to program in assembly language on a huge mainframe computer. Mainframes, in 1982, were already doomed, so I thought. I never wanted to write an operating system, which was one of the capstone assignments. 
I finally took CS-111 as an independent study. In a class by myself, I read Cooper and Clancy’s “Oh Pascal” learning the software language Pascal. That book remains on the shelf behind me in my office today. 
At the age of 28, I had already been a contributing author and technical editor for books about computer programming. Two evenings a week, I stood in front of a group of adults teaching programming skills at a community college. One high school elective and single independent study course for one semester started me on my career. While an autodidact, nearly all I learned fell beyond the reach of classrooms. School never came easily to me, likely because of my learning differences. 
I watch colleagues like Dimitri in awe. In direct comparison, I see that I fraud and the idiot, even today. We’re all like that though, aren’t we? We get good at something, or we get recognition then we tell ourselves: No, I don’t deserve this. From my perspective, he is better skilled that I, smarter than I. I admire he jumps between programming languages and environments.
I mentioned Pascal on purpose, not just as the rambling digression (I do love my digressions though). The programming language I use for work today had been built from Pascal. Oracle, when needing to create a procedural programming language borrowed heavily from Pascal. Pascal is an imperative and procedural programming language, a natural progression for me from BASIC. The language had been designed by Niklaus Wirth. Mr. Wirth, who is 88 years old in 2022, won the Turing Prize in 1984, roughly the same year I learned Pascal. 
Stevie and I write code in Oracle’s PL/SQL language. The language derived from an earlier structured procedural language called Pascal, that I learned at university after learning BASIC in high school. We declare variables at the top of a subroutine. We write code in logical subroutines called procedures or functions. These get compiled by Oracle into the database as something that is no longer intelligible to human readers. These routines / subroutines form building blocks within the database to perform tasks, typically with data. One might need a routine that calculates the total value of an invoice. That routine must spin through each line of data for an invoice. The routine identifies the quantity of items ordered. It then multiplies quantity by the unit price to generate the value of that invoice line. Then we tally the total of the invoice lines to get the invoice total. We, the developers, must control this process precisely, due to the variations needed. When do you round the numbers to two decimal places? When do you calculate the taxes? How do you handle items that are not taxed. How to you calculate discounts? Each of these steps must follow the client’s instructions and we must do it precisely. The process and math must be consistent and transparent. Someone will run a calculator through the numbers to confirm the math. Later, an auditor will verify every bit of it. Nothing can be hidden. 
That’s the land of managing the boring business data: customer contacts, invoice information, inspection data, reporting, document management, bank balances, reservations, etc. My world.
Our software depends on a user running the application within a browser. The user updates a customer’s profile or generates a service order. Those tasks such as presenting the data entry web pages to the user’s browser is accomplished by Oracle APEX. I do not have to write the JavaScript and HTML and CSS required to make an application operate within a browser. 98% web-based applications rely on JavaScript. 100% of web-based applications depend on HTML and CSS. We tell the browser to paint a region blue by setting the region’s color property to blue. We do not tell the browser how to do the painting. We only specify the color. We don’t tell a browser how to draw a line, we specify the thickness and style of the line. I won’t present a sample of object-oriented code to read. It doesn’t read well, maybe it is hard to hear: onclick= “void(0);”
When I look at our applications written in Oracle APEX, the apps can look different on the different browsers. Occasionally, in Firefox, I see fonts with serif instead of the sans-serif font specified by our team. These variations are reminders of a key difference between procedural languages and object-oriented languages. With object-oriented languages, such as JavaScript, the browser decides how to execute a process. The browser’s own personality expresses itself a bit. 

One might argue that the process of formatting and presenting data to users via a browser could be called “edge-ware”. Browsers and JavaScript is software sitting at the edge of an application. Edge-ware, a term absolutely nobody uses, sits at the edge of a software application formatting and presenting our pages to the user. Ok, if you do an internet search on Edge-ware, you will find a village in New Zealand and a couple of companies. Let’s also quietly acknowledge that JavaScript is used on servers too. That is a more recent development. 
The user logs into our Oracle APEX application via browser. The data travels back and forth between the user and the Oracle server via the internet. For illustrative purposes, the Oracle Server sits in the middle of the internet, the “Cloud”. Users sit at the edge of the cloud with their phones, tablets, laptops, and desktop computers. And let’s add cars, refrigerators, doorbell cameras, modern televisions – these all sit at the edge of the cloud with you the user.  
The spirit of “edge-ware” tells a story. 
If we have lovely object-oriented software running on the client-side, or edge, of our applications. And we have lovely software written in a procedural language compiled within the Oracle database, then how does the client/edge talk to the server and the database? Say “Hello” to middleware.

Middleware is like a Babelfish (for fans of Hitchhiker’s Guide books) or the Universal Translator (for fans of Star Trek). Middleware does this cool job of shuttling instructions and data between two very different environments that use different languages and different technologies. 

I think I defined middleware. It required 2000 words to accomplish that goal. Through the journey I did a bit to differentiate structured programming languages from object-oriented languages.

In December and January of 2021, our project with Electrotest had been defined as middleware. With a careful stretch of my definition above, our software could, maybe, possibly be called middleware. The project leaders used the term as a means of evaluating the replacement of multiple legacy systems. Our software would be a central repository for data from several external and legacy systems. Our software would sit in the middle of existing systems. Fair enough, close to the definition above, without all the discussion about babelfish or universal translators. I suspect that the project leaders used the term as a way of exploring writing new software without committing to the project. Likely also as way of softening the discussions with folks who may be resistant to change. 

As a team, we had been given the opportunity to solve some long term and systemic issues. A CEO rolling through announcing massive project to all software that helps the company do its thing would be unpopular. These projects occasionally fail or people get resistant to the change. 
The project leadership used the term “middleware” to help others accept these early steps. Middleware sounds less threatening than: “Our software is crap, we’re replacing our crappy software for the unknown crap written by someone else who doesn’t now crap about our wonderful business.” That’s good political thinking and good leadership. 
We’re going to stick this reliable, awesome Oracle database into the middle of our systems to help solve problems our other software doesn’t solve. 
To the user community at Electrotest, this approach appeared friendly and hopeful. Everyone recognizes problems in systems. We’ll just fix those in a targeted manner.
To the new developers, we get the message, if this goes well maybe we can take more on later. Let’s get one thing working and evaluate.
I need to remember this trick. I have been in several situations during the last decade during which the big boss stands up in front of the users to mandate immediate and instantaneous adoption of our software. The big boss writes a memo reinforcing the message. Within months, the resistance from management overwhelms all work. Resistance to change kills custom software development. And please don’t ask us programmers to manage that for you. 
Electrotest’s approach likely generated an environment of collaboration and success for both the user community and the developers. This definition of “middleware” became more political than technical. It worked. Well done to the team. I’d never had conceived of that approach. 
We benefitted from viewing ourselves as this fake-middleware. We reach into the other systems to review their data whilst exploring past mistakes. We did not need a lot of input from user for this analysis. Hundreds of customers had been named with a dot, or two dots, or a slash, or two slashes, or three slashes. What can I infer from that? 
First inference is that the legacy system, DAX, required the user to type in a name for the client. But the client’s name data entry process had nothing behind the scenes controlling it. Second, I inferred that most users were right handed. Their favorite keys results from movements of their right hand. Both the dot and slash sit below the right-hand pinkie finger. Why were they doing this? Clearly, I can’t know without interviewing the users. I make more inference:

• The user perceives no value in entering the data
• The user does not possess the data and therefore can’t do the data entry
•  There is something about the system causes the user to curtail their efforts

One observation I made is that their unique identifiers for service orders were limited to six numeric characters. Meaning that each time they got to their millionth order, the order numbering process started over from 1. When you do that, the order number is no longer unique. It appeared to me that they cycled through their order numbers every two to four years. When unique identifiers fail at being unique, the data is junk. 
Another observation about the data is that some orders had a value of one cent. Some had a value of negative one cent: zero point zero one euros. Again, I inferred the users need to perform operational tasks that the system did not understand. The system required a price on each line, therefore zero could not be permitted. The users had to create order lines that had zero prices.
The users were not wrong, the software was wrong. This happens. We call this opportunity. Business owners likely call it something else.
What did we have to do to prove we could make a few improvements? I believe we had to demonstrate our ability to write software that met their specifications, solve problems, and do it at a reasonable cost while executing quickly. When managing corporate data and building an ERP (Enterprise Resource Planning) and or CRM (Customer/Contact Resource Management), customer data forms the center of the data picture. One cannot create a service order nor an invoice without knowing who the customer is and their preferences. 
We had hundreds of pages and menus in two applications that were linked. We included over one hundred thousand rows of existing corporate data. We had a reliable and proven login process for user authentication and authorization. When the client looked at the system roughly six weeks after telling us to execute, they saw something that they recognized as their system: their colors, their data, their flow. We wrapped it neatly in corporate branding with their logos and security. It worked reliably on Windows computers, Apple computers, tablets, phones, and desktops. It worked on Safari, Chrome, Edge, and Firefox. 
Some of the menu items took you to page 1, the empty dashboard page. Actually, most of the menu items took you to page 1, the empty dashboard page. What do you want in six weeks? The customer profile stuff looked good enough to earn credibility with a new client. We did put the server’s time on the dashboard. The server runs in UTC, the coordinated universal time sometimes called Greenwich Mean Time or Zulu (that a NATO and military term).
We wrote almost no database code with PL/SQL, our procedural language. We limited our effort with client-side languages. We designated the desired fonts and corporate colors using HTML and CSS as appropriate. We did not have to do this for hundreds of new pages. We did it once and we were done (for now – we kept coming back to the look-n-feel, corporate colors, and logos). We never touched the middleware between the client’s browser and the server’s Oracle environment. 
For the dozens of lookup tables, we made one, the copied it over and over and over. All of the lookup tables had roughly the same structure with the same fieldnames. With the exception of the primary key, which was unique for each table, all of the look up tables were the same. 
We did not hit perfect. We could not achieve perfection. We did not have formal user requirements. We used our experiences and expertise to build a system we thought would meet their needs. We did not then have direct access to the client to ask questions. We did our work based on our professional standards. That’s what we presented. I am certain someone said: “Oh that’s perfect” or “Hey terrific”, but within days we got requests to make changes. Not perfect, but good enough. Sometimes good enough is good enough. When the mission is to build something fast, grab a big hammer and start swinging. There is time later to do the finish work, efforts we call “painting with a small brush”. 

What is The Soul of an Internet Machine?

"The Soul of an Internet Machine". This show explores the intersection of business and technology and the internet.

4 | MIDDLEWARE
The first time I heard the phrase middleware, I was writing software for FedEx with a terrific team of Oracle developers in the late 1990s. I had to ask for a definition. In short, middleware is software that runs between two things. Middleware is software that is invisible to the user. Middleware is software but seems to fall outside the accepted definitions of an application or app. It is software’s own software-based infrastructure. Does that make any sense? Probably not.
I mentioned in prior episodes that I got recruited in 2000 to worked for Cisco System. I worked there for five years. Through that relationship I got shipped overseas as a civilian member of a military unit. Nearly everything Cisco does is also software. We route traffic through the internet using software. Obviously, the bits travel over copper wire, optical fiber, radio waves, or free-space lasers. The how and why of it all occurs with software. None of the software applications I write today would exist without the underlying software that makes the internet go.
A long walk for a bad pun right? If we have middleware running in between software applications and systems, then exactly what do we call the software that underpins the entire global exchange of data? Underware ?
I attempted to define the word “Application” in Episode 3 “Framework”. It is a machine that does something with data such as updating, displaying, or saving. It is a machine into which a user logs into and does stuff with. That might be navigating a car, playing Candy Crush, or writing a podcast episode, or generating invoices. I fussed a bit about the user-login process. Some how, the user-login get performed behind the scenes and automatically using a Google account, a Facebook account, an Apple Account, an Amazon account, or a Microsoft account. An exchange of credentials occurs.
Is software different from an application?
Let’s say yes, although some may disagree. If we do disagree that disagreement may likely stem from the ambiguity in our words: software, hardware, application, middleware, etc. Some software is not an application. To draw a Venn diagram, the set called “software” includes the set called “applications”.
Can you have an application without software? No, not really in this context.
Returning to my Venn diagram, the term middleware falls within the set called “Software” and typically outside of the set called “Applications”. One approach to the definition of middleware is software that is not an application used directly by a user. This definition fails to sustain for long. The definition is more right than wrong. And it will fall apart before the end of this episode.
The concept of software in the middle remains important. I like my definition that middleware is software that operates as infrastructure to other software. That is a definition I can cling to.
Do we know what software is? I am on Episode 4 of Series 2023 for a podcast show called “The Soul of an Internet Machine”, a name honoring Tracey Kidder and his book, “The Soul of a New Machine” from the 1980s. Without a working definition of software in your head, you likely dropped out by now (or maybe like some friends, you just use my podcasts as a sleep aid in the evenings). If you’ve made it thus far, then I can skip the definition of software. I don’t know that I could define the term. My farm truck is loaded with software. Even my kitchen stove runs on bloody software, otherwise that stupid clock wouldn’t blink at me.
I wrote my first lines of code on a PDP-11 microcomputer from Digital Electronics Corporation (DEC) during high school. I attended a school in a wealthy area surrounded by the massively burgeoning IT industry. In my high school, four years before the IBM-PC came to the market, I learned BASIC. The name literally meant “Beginner’s All-purpose Symbolic Instruction Code”, and no does not relate to the American slang basic, which seems to be a bit of an insult.
10 BEGIN
20 PRINT “HELLO WORLD”
30 END
Hard to imagine how these building blocks gave us the world of modern digital commerce. Those roots go back even farther. That trip requires tipping our hat in honor of Ada Lovelace, Alen Turing (which Microsoft Word corrected as “During” – the world treated him so badly during his 42-year life, now my word processor tried to remove his name from this episode – stupid software).
My educational progression followed the global development of software and software tools. In 1982, heading to university, I bought an IBM-PC for $2,500. That was the discount a university professor got. Adjusted for 2022 dollars, the cost would approximate $7,500. It was all I had. My father gave me a small portable typewriter as a high school graduation present. Honor to the man who man his living writing words on typewriters. His writing kept the heat on, food on the table, clothes on our backs, and paid for my degree.
As a college first year, I thought I would study computer science. Regrettably, my university’s systems were older than my high school’s systems. No way was I going to learn how to program in assembly language on a huge mainframe computer. Mainframes, in 1982, were already doomed, so I thought. I never wanted to write an operating system, which was one of the capstone assignments. There were two IBM PCs on campus that first year, maybe three. One belonged to me. One belonged to the great-grandson of Herman Hollerith, the founder of IBM. An athlete named Lynn may have had a third one. Or maybe she bought it during her second year. Having personal computers in 1982/83 was rare.
Finally, this left-handed dyslexic who can’t spell anything well while scribbling with the penmanship of an immature child could submit a paper that earned good grade. I finally had something that fixed my spelling and brought consistency to the shape of my letters. When teachers could read my work, I did slightly better.
I finally took CS-111 as an independent study. In a class by myself, I read Cooper and Clancy’s “Oh Pascal” learning the software language Pascal. That book remains on the shelf behind me in my office today.
At the age of 28, I had already been a contributing author and technical editor for books about computer programming. Two evenings a week, I stood in front of a group of adults teaching programming skills at a community college. One high school elective and single independent study course for one semester started me on my career. While not an autodidact, nearly all I learned fell beyond the reach of classrooms. School never came easily to me, likely because of my learning differences.
I watch colleagues like Dimitri Gielis in awe. In direct comparison, I see that I am a fraud and the idiot, even today. We’re all like that though, aren’t we? We get good at something, or we get recognition then we tell ourselves: No, I don’t deserve this. From my perspective, he is better skilled than I, smarter than I. I admire he jumps between programming languages and environments.
I mentioned Pascal on purpose, not just as a rambling digression (I do love my digressions though). The programming language I use for work today had been built from Pascal. Oracle, when needing to create a procedural programming language borrowed heavily from Pascal. Pascal is an imperative and procedural programming language, a natural progression for me from BASIC. The language had been designed by Niklaus Wirth. Mr. Wirth, who is 88 years old in 2022, won the Turing Prize in 1984, roughly the same year I learned Pascal.
If you are a person in your twenties with your fingers on your phone continuously, or maybe you are that guy I talked to at the phone company about my backup DSL line, you probably don’t need to teach me how the internet works or grab my phone from my hand to show me that little cog icon. Back down and shut up. Now back to programming and middleware.
As an imperative language, we write statements that control and manipulate data directly. We provide detailed instructions to computer systems on how a task is to be completed. Imperative language typically contrasts with “declarative programming”. With declarative programming, developers describe what is required whilst focusing less on how a task is completed.
Pascal and Oracle’s PL/SQL are both imperative procedural languages. We build one or more subroutines also called procedures or functions. We write the detailed building blocks of how we manage data. Procedural languages tend to contrast with object-oriented languages. Object-oriented developers change the state of an object. I need this box to be blue. I need this field to display, or to hide. I need this field to be set to all capital letters/uppercase. You state what is to be done, leaving the process of how to execute to the programming language.
During the half-century since the invention and revision of computer programming languages, structured procedural languages such as PL/SQL incorporate object-oriented elements and techniques. And object-oriented languages share structural elements of procedural languages. They are not converging as languages or techniques, they borrow from each other’s strengths. JavaScript is an object-oriented language. 98% of websites use JavaScript on the client-side to control the behavior of web browsers. JavaScript had been created at Netscape in the mid-1990s. Our friends at Microsoft, in tribute to Bill Gate’s early history, reverse engineered Netscape’s language to create their own version of Netscape’s browser language. Reverse engineering is such a pretty word for theft, isn’t.
During those years of the mid-late 1990s, during the “Browser Wars”, I wrote software in Oracle for FedEx. Our team’s goal focused on catching bad guys who shipped contraband on international routes. Rather cool stuff. I spent a lot of time with U.S. Customs officers opening packages to find drugs, counterfeit computer chips, counterfeit watches, and the like. Their success rate at finding bad stuff came from our ability to identify risks from the data provided by the vendors who shipped material across the globe. I endeavored to ignore the Browser Wars.
I have wandered far from Electrotest and the start up of a long-term software development project. In short, Stevie and I write code in Oracle’s PL/SQL language. The language derived from an earlier structured procedural language called Pascal, that I learned at university after learning BASIC in high school. We declare variables at the top of a subroutine. We write code in logical subroutines called procedures or functions. These get compiled by Oracle into the database as something that is no longer intelligible to human readers. These routines / subroutines form building blocks within the database to perform tasks, typically with data. One might need a routine that calculates the total value of an invoice. That routine must spin through each line of data for an invoice. The routine identifies the quantity of items ordered. It then multiplies quantity by the unit price to generate the value of that invoice line. Then we tally the total of the invoice lines to get the invoice total. We, the developers, must control this process precisely, due to the variations needed. When do you round the numbers to two decimal places? When do you calculate the taxes? How do you handle items that are not taxed. How to you calculate discounts? Each of these steps must follow the client’s instructions and we must do it precisely. The process and math must be consistent and transparent. Someone will run a calculator through the numbers to confirm the math. Later, an auditor will verify every bit of it. Nothing can be hidden.
That’s the land of managing the boring business data: customer contacts, invoice information, inspection data, reporting, document management, bank balances, reservations, etc. My world.
Our software depends on a user running the application within a browser. The user updates a customer’s profile or generates a service order. Those tasks such as presenting the data entry web pages to the user’s browser is accomplished by Oracle APEX. I do not have to write the JavaScript and HTML and CSS required to make an application operate within a browser. 98% web-based applications rely on JavaScript. 100% of web-based applications depend on HTML and CSS. We tell the browser to paint a region blue by setting the region’s color property to blue. We do not tell the browser how to do the painting. We only specify the color. We don’t tell a browser how to draw a line, we specify the thickness and style of the line. I won’t present a sample of object-oriented code to read. It doesn’t read well, maybe it is hard to hear: onclick= “void(0);”
When I look at our applications written in Oracle APEX, the apps can look different on the different browsers. Occasionally, in Firefox, I see fonts with serif instead of the sans-serif font specified by our team. These variations are reminders of a key difference between procedural languages and object-oriented languages. With object-oriented languages, such as JavaScript, the browser decides how to execute a process. The browser’s own personality expresses itself a bit.
One might argue that the process of formatting and presenting data to users via a browser could be called “edge-ware”. Browsers and JavaScript is software sitting at the edge of an application. Edge-ware, a term absolutely nobody uses, sits at the edge of a software application formatting and presenting our pages to the user. Ok, if you do an internet search on Edge-ware, you will find a village in New Zealand and a couple of companies. Let’s also quietly acknowledge that JavaScript is used on servers too. That is a more recent development.
The user logs into our Oracle APEX application via browser. The data travels back and forth between the user and the Oracle server via the internet. For illustrative purposes, the Oracle Server sits in the middle of the internet, the “Cloud”. Users sit at the edge of the cloud with their phones, tablets, laptops, and desktop computers. And let’s add cars, refrigerators, doorbell cameras, modern televisions – these all sit at the edge of the cloud with you, the user.
The spirit of “edge-ware” tells a story.
If we have lovely object-oriented software running on the client-side, or edge, of our applications. And we have lovely software written in a procedural language compiled within the Oracle database, then how does the client/edge talk to the server and the database? Say “Hello” to middleware.
That took me a while. Middleware is the software that runs in between two environments. In this example, it helps transport the data and instructions from the Oracle database server (which knows nothing about a web browser) to the client-side browser at the edge. The client-side browser such as Chrome, Safari, and Microsoft’s Edge know nothing about databases and procedural languages. You are not going to teach a browser how to calculate the total for an invoice then apply local and federal taxes to that sum. And you are not going to have Oracle’s PL/SQL software accurately and easily describe how to build HTML webpages. (yes, exceptions exist, I have written HTML within my Oracle code. Certainly, it is possible to do sums with JavaScript. In general, each have their specialization.)
Middleware is like a Babelfish (for fans of Hitchhiker’s Guide books) or the Universal Translator (for fans of Star Trek). Middleware does this cool job of shuttling instructions and data between two very different environments that use different languages and different technologies.
I think I defined middleware. It required 2000 words to accomplish that goal. Through the journey I did a bit to differentiate structured programming languages from object-oriented languages.
What I love about Oracle APEX is that it permits me and others like me to specialize, almost hyperspecialize. I avoid middleware. I avoid the object-oriented stuff running at the edge. I focus on my specialization. While the look-and-feel shall always be very important, we get a terrific looking product that we can customize without requiring expertise in multiple languages, and multiple disciplines.
In Episode 2, I introduced you to data tables and provided a working definition. I fussed some about how projects do start and how they should start getting Step 1 before Step 2, etc. I stated that data tables create the structural elements from which our software is built. In Episode 3, I discussed the framework of applications endeavoring to define “application” . Furthermore, I advocated for engaging a seasoned team for developing software instead of a random group of individuals recruited from LinkedIn or Indeed. In Episode 3, I briefly discussed creating hundreds of pages from hundreds of data tables using Oracle APEX without writing much, or any code. This work got presented to the client in early January of 2021.
In this episode, number 4, I described the hidden infrastructure of web-based software. Internet technology such as TCP/IP transports data between a server and a client-side browser. Client-side software tends to be written in object-oriented languages such as JavaScript where developers change the properties of objects. On the server, we run other languages, often tending towards imperative procedural languages or structured programming languages such as Oracle’s PL/SQL. Between the client’s browser and the server, developers depend on middleware to translate between the client’s environment and the server’s environment.
Early in this episode, I hinted that the definition of middleware would fall apart soon. Please do cling to the definition I have already offered and supported.
In December and January of 2021, our project with Electrotest had been defined as middleware. With a careful stretch of my definition above, our software could, maybe, possibly be called middleware. The project leaders used the term as a means of evaluating the replacement of multiple legacy systems. Our software would be a central repository for data from several external and legacy systems. Our software would sit in the middle of existing systems. Fair enough, close to the definition above, without all the discussion about babelfish or universal translators. I suspect that the project leaders used the term as a way of exploring writing new software without committing to the project. Likely also as way of softening the discussions with folks who may be resistant to change.
As a team, we had been given the opportunity to solve some long term and systemic issues. A CEO rolling through the office announcing massive project impacting all software would be unpopular. These projects occasionally fail or people get resistant to the change.
The project leadership used the term “middleware” to help others accept these early steps. Middleware sounds less threatening than: “Our software is crap, we’re replacing our crappy software for the unknown crap written by someone else who doesn’t now crap about our wonderful business.” That’s good political thinking and good leadership.
We’re going to stick this reliable, awesome Oracle database into the middle of our systems to help solve problems our other software doesn’t solve.
To the user community at Electrotest, this approach appeared friendly and hopeful. Everyone recognizes problems in systems. We’ll just fix those in a targeted manner.
To the new developers, we get the message, if this goes well maybe we can take more on later. Let’s get one thing working and evaluate.
I need to remember this trick. I have been in several situations during the last decade during which the big boss stands up in front of the users to mandate immediate and instantaneous adoption of our software. The big boss writes a memo reinforcing the message. Within months, the resistance from management overwhelms all work. Resistance to change kills custom software development. And please don’t ask us programmers to manage that for you.
Electrotest’s approach likely generated an environment of collaboration and success for both the user community and the developers. This definition of “middleware” became more political than technical. It worked. Well done to the team. I’d never had conceived of that approach.
We benefitted from viewing ourselves as this fake-middleware. We reach into the other systems to review their data whilst exploring past mistakes. We did not need a lot of input from user for this analysis. Hundreds of customers had been named with a dot, or two dots, or a slash, or two slashes, or three slashes. What can I infer from that?
First inference is that the legacy system, DAX, required the user to type in a name for the client. But the client’s name data entry process had nothing behind the scenes controlling it. Second, I inferred that most users were right handed. Their favorite keys results from movements of their right hand. Both the dot and slash sit below the right-hand pinkie finger. Why were they doing this? Clearly, I can’t know without interviewing the users. I make more inference:
• The user perceives no value in entering the data
• The user does not possess the data and therefore can’t do the data entry
• There is something about the system causes the user to curtail their efforts
One observation I made is that their unique identifiers for service orders were limited to six numeric characters. Meaning that each time they got to their millionth order, the order numbering process started over from 1. When you do that, the order number is no longer unique. It appeared to me that they cycled through their order numbers every two to four years. When unique identifiers fail at being unique, the data are junk.
Another observation about the data is that some orders had a value of one cent. Some had a value of negative one cent: zero point zero one euros. Again, I inferred the users need to perform operational tasks that the system did not understand. The system required a price on each line, therefore zero could not be permitted. The users had to create order lines that had zero prices.
The users were not wrong, the software was wrong. This happens. We call this opportunity. Business owners likely call it something else.
What did we have to do to prove we could make a few improvements? I believe we had to demonstrate our ability to write software that met their specifications, solve problems, and do it at a reasonable cost while executing quickly. When managing corporate data and building an ERP (Enterprise Resource Planning) and or CRM (Customer/Contact Resource Management), customer data forms the center of the data picture. One cannot create a service order nor an invoice without knowing who the customer is and their preferences.
Dirk, who had been consulting with the client for a while, provided 5000 lines of data table definitions. Many of these tables served as “look ups”. We use the data in “look up” tables to validate other data such as invoice method (email or post), invoice frequency (end of month, start of month, immediately), or a client’s language preference (Dutch or French).
Let’s digress for a minute to the Belgium national football team. I read recently that the team, made up of native French speakers and native Dutch speakers adopted English as their linqua franca. Without stepping into cultural and language politics of Belgium, I observed that the phrase “linqua franca” is Latin meaning French Language. The lingua franca became English. We use a Latin term meaning French Language to describe what is often English. English is the lingua franca for software developers, likely due to the origins of so much of the post-World War II development of early computer systems. We Americans, plus post-war allies, used English when created programming language, building rockets, and the like.
Our first mission involved creating scores of pages that allows authorized users to modify the look up tables. The second mission involved creating a means of managing and updating customer profiles.
Stevie and I reviewed the data tables organizing them into modules such as invoices, service orders, planning, clients, inspectors, inspections, training, equipment, and so on down the list. In the first demo, the project leadership and client saw a robust and seemingly complete menu system. The colors matched their corporate brand book. The font was very close, but as I was to learn later, failed on two points. We separated the routine day-to-day operations from the behind-the-scenes administration of the application. We created a dynamic and operational page showing customers and their profiles.
We had hundreds of pages and menus in two applications that were linked. We included over one hundred thousand rows of existing corporate data. We had a reliable and proven login process for user authentication and authorization. When the client looked at the system roughly six weeks after telling us to execute, they saw something that they recognized as their system: their colors, their data, their flow. We wrapped it neatly in corporate branding with their logos and security. It worked reliably on Windows computers, Apple computers, tablets, phones, and desktops. It worked on Safari, Chrome, Edge, and Firefox.
Some of the menu items took you to page 1, the empty dashboard page. Actually, most of the menu items took you to page 1, the empty dashboard page. What do you want in six weeks? The customer profile stuff looked good enough to earn credibility with a new client, Electrotest . We did put the server’s time on the dashboard. The server runs in UTC, the coordinated universal time sometimes called Greenwich Mean Time or Zulu (that a NATO and military term).
We wrote almost no database code with PL/SQL, our procedural language. We limited our effort with client-side languages. We designated the desired fonts and corporate colors using HTML and CSS as appropriate. We did not have to do this for hundreds of new pages. We did it once and we were done (for now – we kept coming back to the look-n-feel, corporate colors, and logos). We never touched the middleware between the client’s browser and the server’s Oracle environment.
For the dozens of lookup tables, we made one, the copied it over and over and over. All of the lookup tables had roughly the same structure with the same fieldnames. With the exception of the primary key, which was unique for each table, all of the look up tables were the same.
We did not hit perfect. We could not achieve perfection. We did not have formal user requirements. Using our experiences and expertise, we built a system we thought would meet their needs. We did not then have direct access to the client to ask questions. We did our work based on our professional standards. That’s what we presented. I am certain someone said: “Oh that’s perfect” or “Hey terrific”, but within days we got requests to make changes. Not perfect, but good enough. Sometimes good enough is good enough. When the mission is to build something fast, grab a big hammer and start swinging. There is time later to do the finish work, efforts we call “painting with a small brush”.
In the next episode, we will explore the process of writing an application that must be presented in multiple languages simultaneously. I will talk about European data laws, colors, and fonts. My font selection, while visually close to the corporate font, failed twice. First, my technique violated European data privacy laws. Second, it was actually the wrong font, close is still wrong sometimes. Until next time, be well and do good and have fun.