Innovating Threat Detection Methodologies in Email Security

Security systems rely on continuously updated threat signatures to combat ever evolving threats. Database updates are implemented every so often, so new attacks are correctly identified and blocked. 

That's why security firms see to it that information about the latest threats are collected and analyzed promptly in order to prevent bigger problems. The prevailing idea in email security is that you can beat it if you know it. You can't respond to attacks blindly.

Things may need to change soon as a recently published study by security firm BitDam reveals the dangers of dealing with unknown attacks. Security systems may be good at gathering information on the most recent threats (to update their signature databases and detection capabilities), but their defenses are notably inadequate when they first encounter malicious software that doesn’t have the corresponding threat signature in their database.

Exploring the Unknown

BitDam’s email security study zeroes in on the weaknesses of leading enterprise email platforms in addressing unknown threats or those they encounter for the first time. Modern systems have been quite effective in blocking attacks that have already been identified. The more important question, however, is how they deal with new attack methodologies or variants thereof.

The study’s process can be summed up as follows: 

1. The researchers collected malware samples,
2. Verified that these are indeed harmful or malicious,
3. Modified the verified malicious software,
4. Sent the verified malware to target email accounts,
5. Monitored the performance of the email security systems protecting the targeted email accounts, and
6. Collected and analyzed the monitored data.

Verified malware that managed to pass through the email security systems is re-sent with diminishing frequency over the duration of the study. In the first four hours, the malware-laced files are re-sent every 30 minutes. For the next 20 hours, the frequency of re-sending is reduced to once every 2 hours. The frequency is further reduced to once every 6 hours for the next seven days and eventually stopped after the seventh day. This is done to simulate 

The study initially focused on Microsoft’s Office365 ATP and Google’s G Suite. It is a continuing study, and the plan is to eventually include Proofpoint and other major email security systems.

So How Did BitDam Obtain Thousands of ‘Unknown’ Malware?

This is certainly an important question. BitDam used thousands of verified malware for the study. If it has access to this many unknown malware, it’s only reasonable for the security community to suspect the firm. However, BitDam did not actually obtain thousands of malware that are not yet registered in the threat databases of Microsoft and Google. They had to be creative and resourceful to proceed with the study.

The sourcing of threats that would be considered unknown to Office365 and G Suite is one of the critical challenges in doing the study. BitDam does not have abundant sources of threats that have not been identified yet by major security systems. The solution: modifying the threats to make them appear as new and unknown.

The alteration of recent-but-known threats into unknowns was made possible using two methods. The first was by changing the hash of the files containing the malware with the introduction of benign data to them. The second method required the modification of the static signature of a macro by adding comments consisting of random words and converting the code of every macro function to base64 string.

In other words, the unknown threats used for the tests are variants of existing recent ones. Using these variants solved two major problems for the researchers: the problem of attaching of the infected files to the test emails and the outright filtering of the malware (since they are already in the database of Office365 and G Suite). Microsoft and Google’s email services automatically check the files being attached to emails in the same way they scan the attachments that attempt to enter the inboxes.

This entire process presented the following eureka moments to the researchers:

• Email systems tend to misidentify variants of existing threats even if the original threats already have their signatures registered.
• It is easy to produce variants of malware that appear as unknown to security systems. With the help of artificial intelligence, myriad variants of malware can be generated and used for more attacks.

This explains why BitDam’s researchers had to do step 3 in the process mentioned above. The modification is necessary to come up with viable unknown threats and to enable the attachment of the malware-laden files in the test emails.

The Problem of High Detection Miss Rates and TTD

After resolving the problems of attaching the malware-laced files and the immediate detection by Office365 and G Suite, the researchers proceeded with the tests and were confronted with worrisome results.

After several weeks of doing the tests, Office365 showed an average first encounter miss rate of 23%. The miss rate was at its highest in the first week (31%). G Suite performed even worse, registering an average first encounter miss rate of 35.5%. Just like Office365, it recorded the highest rate in the first week at a stunning 45%.

Also alarming is the time to detect (TTD) numbers. Office365 had an average TTD of 48 hours after the first encounter. For G Suite, it is 26.4 hours.

To clarify, the first encounter miss rate refers to the rate by which the email security systems failed to detect the verified malware sent to them. TTD, on the other hand, refers to the time it takes for the security systems to detect the malware after the first time it was introduced to them.

Detection failures create weak points that allow threats to penetrate. With long TTDs, the risks are aggravated. A TTD of 48 hours means email accounts are vulnerable for a period of two days. The security system only learns that the threat it allowed to pass earlier should have been blocked. By then, email users may have already downloaded the attached files or clicked on the harmful links.

Using the prevailing threat detection approach, it would be necessary for security systems to update their databases with the signature of a threat at the very moment it is released. This is just impossible.

Identification Is Not the Only Solution

Knowing the unknown is not the only way to deal with the problem of new and yet-to-be-identified attacks. After all, it’s virtually impossible to identify threats and update threat signature databases at the very moment they are released.

As such, BitDam suggests a rethinking of how threat detection operates. Instead of relying heavily on updated data (data-driven) to identify attacks, the idea is to adopt a model-driven approach.

BitDam has developed an ATP solution that uses a threat-agnostic engine. It presents a detection approach that does not require information about attacks to determine if something is harmful and should be blocked. It focuses on the way apps interact with files.

Models of “clean” execution flows are created to have a benchmark of how apps operate when they are working with safe, unadulterated, or benign files. If the ATP engine observes execution flows that deviate from how clean flows go about, the logical decision would be to block the suspicious file.

BitDam’s model-driven threat detection engine has been highly effective as evidenced by how it detected the threats missed by Office365 and G Suite on the first encounter. It shows that it’s not necessary to know the unknown to correctly appraise it as malicious or harmful.

In Conclusion

Being unknown makes threats riskier and scarier. Fortunately, the solution does not always have to be the opposite of what is unknown. BitDam has introduced a model-based threat detection approach that has been proven in tests to be highly effective. It can even reduce TTD to zero. This method is not aimed at replacing data-driven strategies, though. It can boost the effectiveness of current email security systems, but it will likely need updated threat information to address the possibility of excessive false positives if it becomes overly aggressive.