Researchers find bugs in archive file formats

IDG News Service - Researchers have found ways to hide malicious software in commonly used archival formats that went undetected until recently by most antivirus programs.

Most antivirus vendors have patched their applications in order to detect the tampered archive file formats, such as ".rar," and ".zip," said Tomislav Pericin, founder of the commercial software protection project RLPack.

Perison gave a presentation at the Black Hat security conference today with Mario Vuksan, an independent security researcher, and Brian Karney, the COO of AccessData.

The three researchers showed how it is possible to tamper with the archive formats and insert malicious code such as the Conficker worm, which is then executed on a person's computer.

Many corporations use so-called "gateway" security products that analyze file attachments to see if they're malicious. Hackers have found that compressing malicious files -- also known as "packing" -- can sometimes trip up security products, although those products are much better now at that kind of detection.

But the researchers showed that by tampering with different archival formats, it is still possible to evade those gateway products. That's dangerous, since an end user may then open an attachment that could allow a hacker to have remote access to a computer.

"The problem is the AV vendors and the archive vendors have two different solutions. If they don't work in sync, the user can extract an archive on their PC, but the AV won't be able to, and that's a problem," Pericin said.

Most end users do run antivirus software, which means that an executable such as Conficker would be detected when it runs. But the researchers found at least eight vulnerabilities in which security products didn't catch the bad files. Most of the affected vendors have deployed patches, Pericin said.

They've also found at least 30 other potential vulnerabilities in security products, but they're waiting for all vendors to roll out patches to see if those problems persist, Pericin said.

There are no intrinsic problems in the archival file formats themselves, and it will always be possible to modify those files, Pericin said.

The researchers also showed how it is possible to embed secret content within an archive file. The technique is generally known as stenography, or a way to write hidden messages known only to the sender and recipient. There are at least two software tools that can put hidden messages into ".zip" files.

Stenography "has a lot of implications," Pericin said. "If you are a terrorist, for example, and you want to hide communication, if you use encryption, the government can detect that. But if you hide stuff and no one is looking at it, you have this obscure channel of communication that works."

However, the researchers released a free, open-source tool that can spot both malicious software and hidden content in archival formats. Called NyxEngine, it preprocesses archive formats, breaking inside and inspecting what's inside. It supports the ".zip," ".rar," ".gz" and ".cab" formats.