Integration Architecture and the Baggage (mis-) Handlers

This is a bit of a stream of consciousness post. No agenda; it’s just something I observed and extrapolated into (near-) relevancy.

There’s the plane.

Then, there’s the conveyor belt with a guy at each end; one who loads the incoming bags on the belt, and the other who grabs them from the belt and swings them onto the waiting trailer.

When the trailer fills (or the plane empties), the truck comes over and hauls it off to Baggage Claim.

It’s a dance. Can be amusing to watch, though. Beneath the grins, it’s a system rife with opportunity for error. Consider:

• There’s only one door, so the physical world requires the guy on the plane to move further away from the door to fetch more bags.
• If the belt gets ahead of the guy on the ground, the bags get to the ground, too. Sure, he can stop the belt, but if he does, the guy in the plane gets held up.
• If the trailer is full, the belt must stop.
• If the truck is busy, the trailer sits.

Once unloaded, it all happens in reverse.

The airline needs to turn (unload, reload and go) a plane quickly. You may assume outgoing bags are loaded onto a trailer at the terminal and delivered to the plane, even while incoming bags are still being handled. Consider:

• What if you’re short handlers? Bags don’t move on / off planes, onto belts or into / out of trailers.
• What if you’re short belts, or a belt is out of service? Bags don’t move on / off planes or in / out of trailers.
• What if you’re short trailers? Bags get handled twice, and are left on the ground, either coming or going.
• What if you’re short trucks? Bags sit on loaded trailers. Empty trailers sit where they don’t need to be.

Then, there’s the endpoints:

• The plane is early: resources (handlers, trailers and trucks) are redirected, putting other arrivals / departures at risk.
• The plane is late: resources (handlers, trailers and trucks) wait, or are redirected. Outgoing bags wait somewhere, along with the handlers, trailers and trucks.
• The guys in Baggage Claim are behind; the loaded trailer sits, which holds up loading bags that need to be on an outgoing flight.

Each connection represents a potential choke point: a place where the process runs the risk of coming to a halt. Failure at any point will impact other parts of the system, causing delays.

I’m just so glad I pack light enough to carry on.

I didn’t write this to complain about baggage handlers. Rather, how about we imagine modeling this a data workflow? What would you do to protect against delays in the system?

Note there are components in place in this system to mitigate some of the risks of delaying the process:

• Belts reduce the distance a handler must travel, increasing capacity and saving time. Time = money.
• Using trailers rather than trucks (trailers can be left at endpoints) creates a buffer in which bags can be stored, for short term intervals.
• Using trailers also reduces the number of trucks and drivers while increasing truck utilization. The accountants will enjoy maximum utilization of a capital expense.
• Proper staffing of handlers at both the ends of belts and in Baggage Claim keeps things moving.

Some rough equivalents for a workflow architecture:

• Bags: data, packaged in a mostly standardized form (the real world just sucks sometimes).
• The plane baggage compartment: data / application silo in which users can create, manipulate and store data. The plane baggage compartment has a finite capacity.
• Belts: FIFO queue (first-in-first-out queue) with finite capacity and known duration to offload data packets from the data / application silo.
• Trailers: LIFO stack with finite capacity. A stack is essentially a LIFO queue, accepting data packets from the belt queue, but rather inefficiently: the last bag in is the first bag out by default. However, there can be exceptions: see Handlers, next.
• Handlers: processes with finite capacity and the power to evaluate data packages (i.e., reading certain baggage tags for expedited handling). Premium tags are placed in a place in the trailer so they can be retrieved first by the handlers in Baggage Claim, rather than the standard LIFO treatment.
• Baggage Claim: data / application silo that serves processed data back to users (or to other systems).

Use of components like queues and stacks can enable your workflow architecture to scale to Internet capacity. You should establish service-level agreements (SLAs) at each touchpoint to ensure you’re sending / receiving data in an acceptable timeframe and in the proper formats.

In past lives, I worked extensively as an Integration Architect. This was in the days before Windows Workflow Foundation and BizTalk. BizTalk 2004 was a godsend: in fact, I still have a current BizTalk 2006 implementation on a VPC with which I tinker when I feel the need to code. It’s not as sexy as a hot web application built in Silverlight or WPF, but it keeps me thinking of ways to add business value by integrating data trapped within cranky silos with other applications and end users.

A data model, application architecture or process workflow works the best when it’s modeled as close to the real-life process it represents. Once modeled, you can look closely at ways to improve it in an iterative fashion.

Original Post: October 2007