Extreme automation: FedEx Ground hubs speed deliveries

FedEx Ground's hubs sort 3.5 million packages a day without human intervention, helping the courier win market share.

To someone standing outside the FedEx Ground distribution hub in Hagerstown, Md., the package delivery business appears deceptively simple.

On one side of the building, workers unload trucks full of boxes and parcels that have been picked up from businesses and various regional drop-off points. On the opposite side of the building, they load the same packages -- now sorted by ZIP code -- onto more trucks. The outbound vehicles then head for points north, south, east and west to deliver their time-sensitive loads of small packages on a specified date -- FedEx Ground's specialty -- to destinations anywhere in the continental United States and Canada.

But if you venture inside the cavernous, warehouse-like facility, it's a different story. First, there's the noise -- a loud and steady hum from the network of whizzing conveyor belts that speed up to 7,500 packages an hour through a highly complex and thoroughly automated categorizing and tracking process that involves a primary sort, a secondary sort and no fewer than 12 scans before packages reach their destinations.

The scans record each package's physical dimensions, weight and destination ZIP code, plus other information, such as where the package is at any given moment, whether in the hub or en route to its destination.

At the hub, "after a package is unloaded from a trailer, the next time it is touched by human hands is at the loading dock," says Ken Spangler, a senior vice president and CIO at FedEx Ground, a $7 billion unit of FedEx Corp. "Everything in between is automated."

And "everything in between" -- such as the routing of the package -- can also be changed on the fly. But it all happens at lightning speed, and that's critical because winning market share in the ultra-competitive $31 billion domestic ground package-delivery business comes down to a matter of milliseconds.

"Over the last five years, we have been on a mission to get faster and faster," Spangler says. "We re-engineer and speed up our lanes an average of twice a year, and sometimes more frequently than that. Over 80,000 ZIP-code-to-ZIP-code lanes are at least a day faster now." As a result, the company's ground service is able to deliver in a single business day almost a quarter of the 3.5 million packages it handles daily. Most packages -- about 85% -- are delivered within three business days.

In the past six years, FedEx Ground has fully automated 53 of its terminals and opened nine new hubs, such as the 600,000-plus-square-foot Hagerstown facility, which has been expanded twice since it opened in 2005. The business result: FedEx Ground has gained market share for 25 consecutive quarters, notes Spangler.

The much larger, $49.5 billion United Parcel Service, which is FedEx Ground's chief competitor, has "a similar level of automation," says Satish Jindel, president of SJ Consulting Group, a transportation and logistics consultancy in Sewickley, Pa.

But there's one big factor that differentiates FedEx Ground in the automation wars: the level of vendor independence and control it has in choosing the sorting and scanning systems and the other pieces of material-handling equipment that make up its proprietary Integrated Sortation System, or ISS.

Typically, vendors of sorting equipment and camera-based scanning hardware provide turnkey systems (including sorting software) to the delivery industry. In contrast, FedEx Ground requires vendors to provide a standard interface so that it can integrate the sorting and scanning hardware into its proprietary and customized process -- similar to the way Wal-Mart requires its suppliers to ship products according to its specifications.

"This decouples us from vendor systems and provides what we believe to be a distinct competitive advantage," Spangler says. "We had vendors before who controlled the actual sortation systems. That really held us captive to how good they were as an IT shop. If they lost some of their engineers or gained some, it made us better or worse."

Now that FedEx Ground has built its own system to control and manage sorting, "we need them to provide just the material-handling specifics [and] the interface to us in a standard format," Spangler explains.

ISS itself is FedEx Ground's other big differentiator. Based on a service-oriented architecture, the system is made up of a series of business process components, or rules, which can be swapped out or modified as business conditions or customer specifications dictate. For example, a large shipper may require that all of its packages, regardless of weight or size, be shipped together. The rules engine can be configured on the fly to quickly accommodate the requirement.

How It Works

Remarkably, given the complexity of the processes it handles in milliseconds, ISS was built on the principle of simplicity, says Shawn Weis, technical principal and the chief architect behind ISS.

"We tried to keep it as simple as possible by breaking everything down to functional components," Weis says. "We applied all of the principles of adaptability and scalability. By isolating all business rules into components, we created a very configurable business rules engine. As business rules change, we can add configurations or change configurations without changing lines of code to fulfill business needs." This is also what enables FedEx Ground to continually re-engineer and improve its overland routing efficiency.

Back on the distribution floor, ISS controls where and when to divert each moving package, based on data recorded by the scanners situated above the conveyor belts. This is where system performance is absolutely critical.

"We have a second, maybe two, to divert packages as they should be diverted based on the overhead scanner information," Weis notes. "Obviously, that makes performance a huge deal."

Toward that end, FedEx Ground has designed and implemented "contingencies for contingencies" for virtually anything and everything that can happen at a hub, says Spangler.

First, redundancy is required for everything. "There are dual ISS servers, routers and core switches, and there are remote network closets beside each sorter," Spangler notes. Separate fiber networks run to each closet from the central server room.

Also, all scanners and sorting equipment send package data directly to two ISS servers, rather than taking the conventional approach of sending data to a single server with a second server playing backup. But it wasn't always this way. The change was made "because anything that takes longer than 450 milliseconds can cause a potential issue," Spangler says. "So now, both servers have all of the transactions all of the time. We changed the architecture to bring it up another level."

FedEx Ground also has contracts for telecommunications services with two carriers and has a backup for each. If there's a massive outage on the wired network, FedEx moves to its satellite network.

"The cost of our network implementation was driven up significantly with the redundancy, but the [payoff] is zero outages in our environment," Spangler says.

In the big picture, this automation enables FedEx to keep customers continually informed of their packages' whereabouts.

FedEx Ground commits to publishing package tracking and location data for customer viewing within 15 minutes of taking custody of a package. But in reality, Spangler says, "it's near-real-time: We generally see publishing on an average of five minutes."

The process also works in reverse. Many of FedEx Ground's largest customers, like Hallmark, electronically feed shipping data to the carrier before their packages are picked up. It takes one to three minutes to configure ISS with the customer's shipping specifications and be ready to sort and ship accordingly, Spangler says.

The bottom line: Where FedEx Ground was once nimble, "we're now extremely nimble," Spangler says.

And in a competition measured in milliseconds, that makes a big difference.

Copyright © 2011 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon