Disney takes a ride on Ethernet

Most network administrators will do just about anything to keep streaming audio from clogging their enterprise Ethernet networks. But Tokyo DisneySea is a different kettle of fish. There, Disney's network architects designed the new theme park's network to mix audio and data packets over standard Fast Ethernet and Gigabit Ethernet connections. The new park, slated to open later this year adjacent to Tokyo Disneyland, will contain 34 new rides, many with water-based themes relating to Disney standards such as the Little Mermaid and Captain Nemo. With these new rides came new challenges, including designing new story lines and visitor experiences, not to mention hooking the network to a 500-room hotel on park grounds.

All of the networking is plain old Ethernet, making Tokyo DisneySea the first Disney theme park to make exclusive use of Ethernet for all the communications and control of park systems, including music, lighting, point-of-sale cash registers, ride control and management, and overall network management. Other Disney parks have been designed around ATM or use multiple network topologies.

Disney chose Ethernet "because it has absolutely the best price/performance of any network topology," says Tony Tamalunas, principal engineer of network systems for Disney. "There isn't anything with a lower cost. We like to say that we put our money onstage, not backstage. If we did the Tokyo park using OC-12 ATM, it would cost twice to three times what Gigabit Ethernet cost."

Disney engineers started working on the network design for the new park four years ago, when Gigabit Ethernet was just emerging. Teams of specialists from various Disney departments would meet weekly for several hours to design the gear that would go into the new park. Early on, the IS department recognized that Ethernet could deliver the needed control and connectivity, but there were plenty of hurdles along the way.

For example, Disney engineers conducted their own interoperability tests, inviting multiple network equipment vendors to prove that their gear worked together before Disney would buy anything for deployment. They also needed superior global support because Disney IS staffers work in many different countries and cities.

What made using Ethernet possible at the Tokyo DisneySea was a combination of advanced protocols and equipment, called CobraNet, designed by Peak Audio. CobraNet allows real-time, uncompressed audio to be distributed over standard Ethernet cabling, so signal-processing equipment can be located at a distance from the audio amplifiers and speakers. This means music sources can be located in remote-control rooms and piped to amplifiers at the attraction, without loss of sound quality.

CobraNet works by setting up an isochronous service on an Ethernet network, with all the audio sent inside ordinary Ethernet packets. According to Kevin Gross, director of network technology at Peak Audio, the system uses one megabit of bandwidth per audio channel, which means a 100M bit/sec Fast Ethernet connection can "very comfortably handle up to 64 different audio channels." If an audio engineer tried to string enough analog cabling to handle these 64 separate channels, the "resulting cabling would be an inch or two in diameter, and you would get lots of interference in the signals," he says.

"To make this work, we required a bulletproof backbone with no dropouts, no jitter and some network delays," Tamalunas says. "But the delays have to be deterministic, which isn't something you normally think of when it comes to running audio over an Ethernet network."

CobraNet is installed at Seattle's Experience Music Project museum, the Sydney Olympics and Opera House, and Wembley's soccer stadium north of London. But some at Disney doubted that sending digital audio over Ethernet could work.

"In order to get corporate approval for this system, we had to pass our plans by people at Disney who we call the 'golden ears.' These are our own audio specialists who have to approve the quality of our audio systems. They listened to the music being carried over the CobraNet systems, and they couldn't tell the difference, so it was a big achievement for us," says John Noonan, director of Show Control for Disney.

Noonan says the Tokyo DisneySea park is using 48-kHz sampling at 24-bit resolution, while older parks had only 18-bit resolution on their audio systems. CD sound quality typically uses 16-bit resolution and a 44.1-kHz sampling rate.

CobraNet differs from Internet streaming technologies from RealNetworks and Microsoft in that it doesn't compress the audio signal at all. CobraNet also offers end-to-end network latency of only a few milliseconds, according to Gross. Of course, the trade-off is that CobraNet is limited to LAN and campus environments, whereas RealAudio and Windows Media run over the Web.

This real-time nature of the audio signal is important to Disney. In the new Tokyo park, staffers had to develop a ride that went through a simulated volcano, equipped with 30 5,000-watt amplifiers driving various subwoofers and other large speakers. All of this is controlled over the park Ethernet system, including transmission of the sound effects themselves.

"When we first demonstrated this in our California lot, we kept setting off car alarms all over the neighborhood," Tamalunas says.

The trick to making use of CobraNet across the 300-acre Disney theme park is to have precise control over the number of individual network hops and end-to-end latencies - something that is possible when you are building an entire switched network from scratch. In Disney's case, it makes extensive use of Extreme Network's Black Diamond Gigabit Ethernet network switches with about 45 separate dual runs of fiber from the central control area to outlying buildings. At the end of each run is a switched hub that connects to local equipment in that building.

"We have 48-port edge devices and most of them are full. Some are 10 megabit, some are 100 megabit out from the hubs," Tamalunas says. He designed the network to separate the park systems into 15 virtual LANs (VLAN), which segments the CobraNet audio traffic from data traffic and keeps the individual applications separated within the park, without the need to install separate physical networks.

But even with the VLANs, there is still a complex Ethernet network to manage. The overall network has close to 2,000 devices on it, split between embedded computers and Windows NT-based computers. However, the NT computers aren't your usual office applications. "No one is using these NT machines to run Word or Excel," Tamalunas says. "These are all custom applications for our attractions that we wrote ourselves."

Some of the Disney entertainment productions are live shows that feature complex choreography, such as the timing of a parade along one of the park's streets. To pull these events off, a show manager has to be in control of the various sound and lighting systems along the parade route and have a precise means of timing each part of the production, as well as match the performance with the size of the crowd along the parade route. The network architects used two things to help the show producers. First, the entire park runs at 30 frames per second, so it is possible to synchronize events no matter where in the park they occur.

"If you could fly over our park during one of these events in a helicopter, you would see lights going on and off in sequence all over the park," Tamalunas says. "The timing has to be that precise or the shows don't work."

"Timing is important," Noonan says. "We have music playing synchronously over all 300 acres. This means as you walk down the street, you are hearing the same music coming at your right and left ears at the same time, even though the sound may be traveling over two different physical network paths to get to these different sets of speakers."

Second, the show manager has to be able to take control over the various networks from a vantage point along the parade route. "Our network control center is in a building off in a corner without any windows. The show managers need to be able to see their shows taking place in front of them, which meant we needed to be able to have remote control stations around the park," Tamalunas says. At their central network operations center, the IS staff can easily grant control to particular places at specific times and for authorized users.

Not all the network applications are fun and games, however. The Disney network makes extensive use of standard protocols, including SNMP. "Our lighting-control systems make use of SNMP for higher-level reporting, which we developed ourselves because we didn't want to deal with a bunch of proprietary lighting-control protocols from different vendors," he says.

They also make use of quality of service, and over-provision their network bandwidth to provide the deterministic quality needed to run the CobraNet protocols and deliver the high audio quality required.

While it's too soon to say whether the park will be a hit with the public, the network architecture is a winner. And for Disney, combining the various applications together over one Ethernet doesn't belong in its Futurama section, but very much in the present day.

This story, "Disney takes a ride on Ethernet" was originally published by Network World.

Related:

Copyright © 2001 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon