Dry Rot And The Internet

A termite infestation is one of the most insidious and destructive predicaments that a wood-framed structure can face. Infestations typically start in some obscure corner, well out of sight, and spread silently, inch-by-inch over the course of years. Colonies can number into the millions, using a decentralized swarm intelligence that's self-organizing. By the time that the owner becomes aware of the problem it's often too late, the integrity of the entire building has been compromised.

Now imagine this scenario played out by a state-sponsored botnet that's employing a bare-metal rootkit to fly below radar level; perhaps the result of a hardware vendor cooperating with an intelligence agency to embed stealth technology at the circuit level. The infestation could occur over the span of several years, as the botnet spreads to hundreds of millions of hosts using a decentralized peer-to-peer swarm intelligence that relies on a carefully designed covert channel. The botnet could sit dormant (in a manner similar to Conficker), a massive sleeper cell that exists only to propagate, waiting for the order to wake up in the event of Wold War III. Or it could work to progressively corrupt data, instituting alterations until even the backups of backups are bad.

What would happen if the circuit-level backdoor was discovered by other nation state players and unleashed against its maker? According to researchers that I've spoken with, these are cyber-war scenarios that the DoD has examined.

But is this really what we need to be worried about on a day-to-day basis? Bruce Schneier says cyber-crime is the real threat (and I would agree with this). Though, he also pointed out in a 2005 essay that:

"The countermeasures aimed at preventing both cyberwar and cyberterrorist attacks will also defend against cybercrime and cybervandalism. So even if organizations secure their networks for the wrong reasons, they'll do the right thing."

This is akin to NASA's Apollo program, which yielded a number of technological advances as a byproduct of our ultimate goal of landing on the moon. So, even if we never actually made it to the moon, the effort would have been worth it in the long run. -BB (2009-12-30)

Open Source Anti-Virus as the Public Option

Yesterday afternoon, over lunch, a colleague of mine who was born in Hungary pointed out that the United States is the only industrialized country that doesn't provide universal health care to its citizens. Then he went on to explain how medical care was a basic human right and that society, as a whole, benefits from keeping its population in good health.

Could the same argument be made with regard to computers? Should there be a state-funded alternative (e.g. open source anti-virus) so that users could take steps to maintain the health of their systems? After all, decreasing the number of compromised machines has its benefits, right? Or would this approach just provide attackers with a better way to implement instance-specific attacks, leaving users with a false sense of security? This is one of those "dangerous ideas" that I'd encourage people to think about. -BB (2009-12-23)

Black Hat Vertical Integration

While bulletproof hosting services have proven valuable to online criminals, some groups are moving up the food chain by directly allocating blocks of IP addresses from Regional Internet Registries (RIR) and Local Internet Registries (LIR). According to a posting by Kasperksy:

"Attackers who own their own large blocks of IP space have a much easier time hiding their activities than do criminals who have to go through legitimate ISPs or hosting providers. There's no abuse desk to complain to, no recourse for people who find themselves being attacked by a given range of IP addresses."

In theory, this sort of thing shouldn't happen. The problem is that in certain parts of Europe the record-keeping and oversight facilities necessary to verify applicant organizations are lacking (again, this is an infrastructure issue). A couple of years back, the Russian Business Network was able leverage this aspect of address allocation to score a large block of IP addresses from RIPE, essentially becoming a rogue ISP.

Fear and Loathing at CEIC 2010

In May of 2010, our fearless leader (Bill Blunden) will head back to Vegas to speak at the Computer and Enterprise Investigations Conference. Anti-forensics and rootkits will likely be on the menu. Presentation date TBA. -R. James (Dec. 12, 2009)

Why Isn't China Throttling Its Malware?

Anyone who has done business in Hong Kong knows that, despite the rapid growth of mainland China, this region still has one ace up its sleeve: infrastructure, thanks to the British colonialists. Specifically, I'm talking about the legal and regulatory oversight necessary to support economic activity.

For example, if you want to buy or sell gold, it's generally less risky to do so in Hong Kong because there's a significant amount of checks and balances in place to safeguard buyers and sellers. In fact, it's fairly common for merchants from the mainland to travel to Hong Kong to deal in gold for this very reason. Simply put, the infrastructure is better.

This reality points to basic underlying flaws in China's system. Perhaps this is to be expected, given that the current system evolved as a result of thousands of years of rule by dictatorship, in one form or another. China simply doesn't have the tradition of checks and balances that are the hallmark of a democratic society. This, in turn, may explain why the vast majority of bullet-proof internet hosting services operate out of China. -BB (2009/11/29)

U.S.-China Economic and Security Review Commission, 2009 Report

This congressional committee report, in Section 4 of Chapter 2, concludes that:

"The direct attribution of such activities targeting the United States presents challenges due to hackers' ability to conceal their locations. Nonetheless, a significant and increasing body of circumstantial and forensic evidence strongly indicates the involvement of Chinese state and state-supported entities."

The report doesn't go into the details of exactly how we know who's attacking us. In so many words, they're saying "we just know, trust us." Boy, that sounds like a slam dunk to me! I can't help but wonder if the actual perpetrator is simply making effective use of anti-forensics to place the blame on somebody else?

Regardless of who's culpable. The existence of state-sponsored hacking isn't necessarily earth-shaking news. As the recent 60 Minutes piece demonstrated, we're probably one of the more active players in this field. So, when other countries discover the existence of advanced persistent threats in their networks, some of the binaries that they recover probably can be attributed to us.

Fear and Loathing at Black Hat DC 2010

In late January, Bill will be navigating the beltway to speak at Black Hat DC 2010. Hopefully life in Northern California hasn't softened him up so much that he can't handle winter on the east coast. -R.James (Nov. 12, 2009)

Wired Magazine on the 60 Minutes Report

One side claims the 2007 power outage in Brazil was due to hackers and the other side dismisses it as the result of poorly maintained high voltage insulators. Who do you believe? This story from Wired reminds me of an observation that Bruce Schneier made recently.

"We tend to be poor judges of risk. We overact to rare risks, we ignore long-term risks, we magnify risks that are also morally offensive. We get risks wrong -- threats, probabilities, and costs -- all the time. When we're afraid, really afraid, we'll do almost anything to make that fear go away. Both politicians and marketers have learned to push that fear button to get us to do what they want."

As an experiment, read through the news stories that I've collected over the past year and ask yourself which threat seems more immediate: cyberwar or cybercrime. Naturally, some people would argue that the actual threat that cyberwar represents can't be properly evaluated because much of the truly substantive evidence must be kept secret for the sake of national security... -BB (2009/11/11)

60 Minutes: Sabotaging the System

This evening I watched a piece by 60 Minutes that focused on threats to our infrastructure from computer-based attacks. While some aspects of the broadcast verged on sensationalism (which is only natural, given that 60 Minutes is trying to attract viewers on behalf of their advertisers), I was encouraged by the inclusion of points that are typically neglected when it comes to news stories like this.

For example, take the following observation made by Jim Lewis, director of the Center for Strategic and International Studies:

"We're in the top of the league. We are really good. And if you talk to the Russians or the Chinese, they say, 'How can you complain about us, when you do exactly the same thing?' It's a fair point with one exception: we have more to steal. We have more to lose. We're the place that depends on the Internet. We've done the most to take advantage of it. We're the ones who've woven it into our economy, into our national security, in ways that they haven't. So, we are more vulnerable."

Sure, our networks have been penetrated and data has been stolen. But we're not an innocent bystander here. Heck, we break into networks in other countries too, all of the time. In fact, we're pretty damn good at it. So should do we, as a country, have the right to be indignant when intruders breach our security? Personally I think embarrassment might be a better response. Obviously our offense is much better than our defense. But why does this state of affairs exist? The 60 Minutes report hinted that part of the problem has to do with the financial prerogatives of the corporations that create high-tech products. Specifically, Congressman Jim Langevin noted that:

"The private sector has different priorities than we do in providing security. Their, in a sense bottom line, is about profits. We need to change that. We need to change their motivation so that when we see a vulnerability like this we can require them to fix it."

In my opinion, instituting meaningful change is going to be difficult, as legislators will be forced to bite the hand that feeds. Don't think for a minute that all of those hi-tech lobbyists will roll over and purr if our representatives start talking about measures that might adversely impact the bottom line. Offshore outsourcing, for instance, represents a long-term threat to the technical leadership that the United States has maintained since World War II. Yet, our legislators are woefully silent when it comes to actually doing anything about it. Guess what happens when most of our hardware is manufactured in other countries because it's cheaper? According to Jim Gosler:

"We have found microelectronics and electronics embedded in applications that shouldn't be there. And it's very clear that a foreign intelligence service put them there."

Would you like some fries with that? -BB (2009-11-08)

Peter Kleissner: It's Just Technology

After presenting the "Stoned Again" bootkit at Black Hat USA 2009, Peter's then employer (Ikarus Software) asked him to resign. This is ridiculous. As Professor George Ledin of Sonoma State has pointed out, it's probably more dangerous not to have an open discussion of malware technology. It seems the AV industry would rather gag everyone and stifle external research.

Reading this Washington Post article made me think of Colonel Kurtz from the movie Apocalypse Now.

"I've seen horrors... horrors that you've seen. But you have no right to call me a murderer... you have no right to judge me."

Microsoft's (Lack of) Forensic Tools - Continued

A reader contacted us this morning to let us know that Microsoft does actually offer a forensic tool. It's a custom USB drive that ships with a suite of 150 commands. Unfortunately, Microsoft seems to limit distribution of its forensic thumb drive to law enforcement personnel.

The tool's public announcement, from 2008, can be viewed here. Microsoft's official page for this product is here.

Can You Believe It? They're Spying on Us!

Yet another vague story from the Wall Street Journal about an unnamed company that had its machines compromised by intruders who were "likely supported, if not orchestrated," by the Chinese government. Note that attribution is one of the primary issues when it comes to cyber-attacks. Recall the news stories that came out earlier this year that had legislators clamoring for retaliation. As it turned out, the reported attacks didn't come from North Korea, but from somewhere in Miami (or who knows where).

Keep in mind, dear reader, that the art of starting wars has been honed for thousands of years. Whenever I read this sort of story, I'm reminded of a particularly chilling quote from Gilbert's Nuremberg Diary that's attributed to Hermann Goering:

"Voice or no voice, the people can always be brought to the bidding of the leaders. That is easy. All you have to do is to tell them they are being attacked, and denounce the pacifists for lack of patriotism and exposing the country to danger."

Finally, just to be fair, even if this actually is the work of attackers backed by China, I'm pretty sure we're spying on China also. It's just that we're not as noisy or conspicuous when we do. -BB (2009/10/23)

The Invisible Giants

In the early 1900s, the city of Cleveland established itself as a center of economic activity. Its status was reflected by the fact that, in the wake of the Federal Reserve Act, Cleveland was chosen to host one of the Fed's twelve regional banks. The driving force behind Cleveland's ascent during this period can be traced back to the Van Sweringen brothers, who developed a railroad empire that was based in the city. The Van Sweringen brothers were elusive, low key, billionaires. One might even go so far as to say that discretion was their hallmark. They literally had a man on their payroll whose sole job it was to keep their name out of the papers. The economic equivalent of a rootkit, they preferred to exercise their power indirectly from behind the scenes, with subtlety. Hence, cynics who scoff at the notion of hidden rulers and their intermediaries in the power structure might be well advised to recall a statement made by then President Woodrow Wilson:

"A great industrial nation is controlled by its system of credit. Our system of credit is privately concentrated. The growth of the Nation, therefore, and all our activities are in the hands of a few men... We have come to be one of the worst ruled, one of the most completely controlled and dominated, governments in the civilized world, no longer a government by free opinion, no longer a government by conviction and the vote of the majority, but a government by the opinion and the duress of small groups of dominant men."

Related: Thought control in economics. A professor at Wellesley observes that "supply and demand curves only determine prices in perfectly competitive markets, which don't exist. I considered this key to my students' education, especially since mainstream economists apply the framework inappropriately so often."

We're Number 1 (Well, Sort Of)

As of 7:27am PST (2009-09-17), The Rootkit Arsenal is the #1 selling book in the Security category of the "Business & Culture" sub-section of the "Computers & Internet" section at amazon.com. Though, strictly speaking I think I should point out that with its overall sales ranking of 8,399 the book is hardly the most popular technical book at amazon.com. My suspicion is that books are assigned to these carefully delineated groups for marketing purposes. Ahem. Anyway, having put this into context, I'd like to extend my thanks to everyone who's read the book and also to my cohorts here at Below Gotham Labs. Keep those e-mails coming. -BB

State-Sponsored Rootkits

Recently, a professional malware developer who worked for ERA IT Solutions (a commercial software company that supplies security tools to the Swiss government) released VoIP monitoring code to the public. That's right, you heard correct, there are professional software engineers actively designing malware on behalf of national governments.

Security through obscurity may not be an impenetrable shield but it is a barrier, and not always a trivial one. Results that might take an independent lab several months of excruciating reverse engineering might only take a few days for a lone engineer who happens to possess the necessary design documents and specifications. Having the cooperation of OEMs and software vendors can make the difference between a buggy proof of concept and a robust, production-quality, implementation with all the bells and whistles. This is because effort that otherwise would be spent isolating magic numbers and decomposing obscure protocols can be directed towards actual software development.

I'll probably never know exactly how far ahead state of the art rootkits are from what we see at conferences like Black Hat. I don't have the requisite security clearance. But if my instincts are correct, the things that show up in the public sector are relatively basic instruments that merely hint at what's been done by the intelligence agencies. To see what I'm talking about, check out the rootkit described in this article. -BB (2009-08-30)

Microsoft's (Lack of) Forensic Tools

For many years, I wondered why Microsoft couldn't release a set of utilities that were as serviceable as those offered by the researchers at Winternals. Then, on July 18th of 2006, Microsoft announced they were acquiring Winternals. Will we have to wait for a similar event to occur in order to have access to robust, native, forensic tools?

After all, if anyone possesses the information necessary to build a stable and comprehensive suite of forensic tools for Windows it would be, well, Microsoft. Perhaps they're worried that such apps would be used by reversers to peek at things that they're not supposed to? Who knows? I just wish that I could sidestep the process of having to deal with freeware that randomly crashes or shelling out big bucks for overpriced third-party software. -BB (2009-08-19)

Sun Tzu and Cyber War in Georgia

"A wise general makes a point of foraging on the enemy. One cartload of the enemy's provisions is equivalent to twenty of one's own, and likewise a single pound of his provender is equivalent to twenty from one's own store" -Sun Tzu, The Art of War

While reading the Wall Street Journal's article on the DDoS that took place last year in Georgia, I couldn't help but think of the above quote. The perpetrators used our infrastructure to support their attack. They used U.S.-based social-networking sites, stolen American identities, and modified code that Microsoft provides for free.

As the article observed: "cyber-warfare has outpaced military and international agreements, which don't take into account the possibility of American resources and civilian technology being turned into weapons."

Encryption Keys and Plausible Deniability

Recently an article appeared in the Register about two people who were convicted for failing to reveal their encryption keys to authorities. If you're using an encryption package that allows you to create, and encode, a virtual file system (i.e. a large file that the software mounts and treats as a logical disk), one way you could protect yourself would be to create a secondary encrypted file system within another. This way, if you're coerced into providing an encryption key you could offer the key to the outer file system (which you might want to populate with a smattering of decoy files) while concealing the inner file system somehow. This is the motivation behind TrueCrypt's "hidden volume" feature.

I suppose that if you really wanted to be paranoid, you could create yet another encrypted file system within the secondary file system...

Computer Security Meets Ulam's Dilemma

Stanislaw Ulam was a Mathematician from Poland who came to the United States at the outbreak of World War II and subsequently was involved in the Manhattan Project. He observed that, over time, mathematics had grown into such a vast discipline that making progress required focusing on a narrow area of specialization. The problem with this tendency is that it makes it much more difficult to grasp, and appreciate, developments in other sub-domains.

Having walked the halls at Black Hat, I can see the same thing happening to computer security. Fields like web-based attacks and firmware exploits are so rich with ideas and technical minutiae that specialization is becoming a matter of necessity. The emerging ecosystem that supports the creation and deployment of malware reflects this fact. One engineer builds a rootkit that gets bundled as a payload in an exploit used by a worm that's written by another engineer, who then sells it to someone else who uses it to seed the internet and grow a botnet, that gets rented out by a front man from somewhere else...

Like an Eskimo stuck on an iceberg that's breaking apart, it gets harder and harder to keep a foothold on every field until finally it becomes impossible. Eventually, you have to choose your own little plot of conceptual real estate and try to keep an eye on related subjects. In the worst case, you choose an area that dwindles into obscurity (remember Trusted Xenix?), and then, well, it helps if you can swim.

Black Hat USA 2009 Material Posted

Here's the white paper and slide deck that I presented at Black Hat USA 2009. My comments on the event follow below.

Black Hat USA 2009: Postgame Wrap-Up

Looking back over the two-day event, the first thing that struck me was the sheer scale of the conference and how well they were able to manage the flow of people. Caesar's Palace was definitely a suitable venue for this conference.

I started off the first day with the keynote address by Douglas Merrill, whose talk revolved around psychological acceptance (i.e. security measures are futile unless users are willing to actually use them). Next, I sat in on Peter Kleissner's presentation on the Stoned Again Bootkit, which detailed a framework for loading arbitrary payloads into the kernel during system startup.

The highlight of the morning session was the talk led by Peter Silberman and Steve Davis , from Mandiant, who demonstrated how to re-construct Metasploit intrusions using a custom tool in conjunction with Memoryze to scan the address space of a compromised process.

In the afternoon I stayed primarily on the rootkit track. I sat through Erez Metula's discussion of user-mode rootkits, which embed themselves in virtual machine runtime environments (e.g. the JRE, or .NET) by altering the bytecode libraries that they rely upon. This talk was particularly well organized and easy to follow, though the emphasis in this case appeared to be on data exfiltration and manipulation. Metula observed that absolute stealth would probably require the assistance of a system-level rootkit.

I ended the first day with the presentation on "Ring -3" rootkits from the Invisible Things Lab (ITL), which focused on firmware-related subversion that targeted a special region of memory reserved for Intel's Active Management Technology. This time, Joanna sat with the audience while her two colleagues (Alexander Tereshkin and Rafal Wojtczuk) did most of the talking. The trend that the speakers touched upon is that vendors often try to protect against malware by putting special management code in remote locations that the operating system (and any malware that it might be hosting) cannot access. This is all nice and well until malware somehow loads itself into these specially protected regions...

On the second day of Black Hat, I started with a presentation by ITL and then sat in on Nick Harbour's discussion. Nick, a reputed Ninja, examined API tracing via detour patching as a way to reverse engineer malware. He also demonstrated a novel technique for unwrapping packed binaries using a customized version of kernel32.dll.

Being a native of the Bay Area, I couldn't resist the talk on smart parking meters given by Joe Grand, Jacob Appelbaum, and Chris Tarnovsky. I can't speak for everyone, but the photograph of the meter with $999.99 worth of parking time brought many people to a standing ovation. Over the next few months I'm going to be eagerly watching the Mission District for hacked parking meters. Let's hear it for a truly great presentation!

I also sat in on the Feds versus Ex-Feds panel for a bit. Man, those feds are a cheeky bunch. I suspect they were overcompensating as they may have expected the same from us. One audience member commented that he was essentially asked to: "step up to the microphone, sir, and be shot."

Around the mid-point of the discussion panel, I left to go prep for my own talk. During my presentation on anti-forensics I looked down into the audience and recognized a couple of well-known people whose work I truly respect: Richard Bejtlich and Jamie Butler. Whoa. That was cool. Thanks so much, Richard and Jamie, for taking the time to sit through my talk!

Fear and Loathing at Black Hat USA 2009

Bill Blunden will be joining the pilgrimage to Vegas this July to speak at Black Hat USA 2009. The title of his presentation is Anti-Forensics: The Rootkit Connection. The speaker schedule is available here. It looks like Bill will be speaking on July 30th from 16:45-18:00 in the Augustus Ballroom on the Fourth Floor.

Fear and Loathing in San Francisco

On May 15th, 2009, at San Francisco State University I'll be giving an encore performance of the rootkit presentation that I gave at Sonoma State back on April 9th. The talk will be given in the HSS building, room 362, from noon to 1:30pm.

The Rootkit Arsenal: Approach versus Intent

"If you know the enemy and know yourself, you need not fear the result of a hundred battles."
-Sun Tzu

Recently a number of people have raised the issue of whether an open discussion of Black Hat tradecraft is a dubious proposition. The general concern being that a book like The Rootkit Arsenal poses a threat because it will show bad people how to do bad things. In response to the e-mails that I've received, I'd like to take a moment and directly address this topic.

The Rootkit Arsenal offers both concepts and source code. Ultimately, I'm a broker. I can't control what the reader does with what they read. However, I might add that the bad guys already know this stuff. In fact, many of the book's tactics were excavated from Black Hat sites. It's the average system administrator who needs to appreciate just how potent this technology can be.

Hence, though the approach of my book is obviously from the vantage point of a Black Hat, my intent is to offer insights which normal, law-abiding, IT professionals might find useful. Trying to secure the Internet by limiting access to potentially dangerous information is a recipe for disaster. Security through obscurity is not the answer. As Mark Ludwig put it in his seminal book The Giant Black Book of Computer Viruses, "No intellectual battle was ever won by retreat. No nation has ever become great by putting its citizens' eyes out."

Malware Research at American Universities

Why is the obscure art of malware so, well, obscure? Why aren't students at MIT, Princeton, Caltech, and Stanford actively studying this relevant topic? According to George Ledin of the Anti-Conficker Project, "The AV industry has kept everything under wraps, most university professors are busy with their cozy niche and don't want the aggravation, and the topic is dangerous, unchartered territory."

But this answer begs the question: why is this dangerous territory? Heck, software is just software. Right? Ledin presents his case, quite well, in the January 2005 issue of the CACM.

Here's what Niccolo Machiavelli would say: "And it ought to be remembered that there is nothing more difficult to take in hand, more perilous to conduct, or more uncertain in its success, then to take the lead in the introduction of a new order of things. Because the innovator has for enemies all those who have done well under the old conditions and lukewarm defenders in those who may do well under the new. This coolness arises partly from fear of the opponents, who have the laws on their side, and partly from the incredulity of men, who do not readily believe in new things until they have had a long experience of them"

Fear and Loathing in Sonoma

At the request of George Ledin, the Spring 2009 Computer Science Colloquium organized by Sonoma State University will be hosting a presentation by Bill Blunden in April. The hour-long talk, entitled The Rootkit Primer, will provide an overview that examines the core services that rootkits provide, how they provide these services, and who's using this technology.

Powerpoint slides of the talk can be found here.

The Rootkit Arsenal

In late April, Wordware Publishing will be sending my book The Rootkit Arsenal to press. The manuscript was several years in the making and the book investigates a broad range of related topics (e.g. system-level code, anti-forensics, reversing, etc.). Unlike the vast majority of computer security books The Rootkit Arsenal does not attempt to veil itself with ethical window dressing. My book approaches its material, without apologies, from the standpoint of a Black Hat. No doubt this publication will ruffle a few feathers.

Greetings and Welcome

This entry marks the launch of the web site for Below Gotham Labs. We'd like to thank everyone involved and encourage our visitors check out the latest news, events, and publications.