Threat Intelligence and SIEM (Part 1) Reactive Security

Foreword

This blog post is the first part in a series about reactive versus proactive security with security

information and event management (SIEM) and threat intelligence (TI). In this post we will present an

overview of reactive SIEM, what it does, how it works, and its limitations. Please note this post will

cover a significant amount of background information and details, but will not cover how to deploy and

configure SIEM, what vendor to choose, or how to implement use cases.

SIEM Overview

Security information and event management (SIEM) is a solution that provides a birds eye view of

an IT infrastructure. It fulfills two main objectives: (1) detecting in (near) real-time security incidents,

and (2) efficiently managing logs. These objectives were respectively called security event

management (SEM) and security information management (SIM), but nowadays these functions have

been merged into a single capability known as SIEM. From a high-level point of view, a SIEM collects

information (e.g., logs, events, flows) from various devices on a network, correlates and analyzes the

data to detect incidents and abnormal patterns of activity, and, finally, stores the information for later

use (reporting, behavior profiling, etc.). When successfully deployed and configured, a SIEM helps

organizations:

Discover internal/external threats.

Monitor (privileged) user activity and access to resources.
Provide compliance reporting.
Support incident response.

Typical Architecture

As previously mentioned, the SIEM will gather logs and events from a heterogeneous collection of

data sources which can be grouped into four categories:

1. Network devices (routers, switches, etc.)

2. Security devices (IDP/IPS, firewalls, etc.)
3. Servers (Web, mail, etc.)
4. Applications

For each device, a collector will be used to gather and normalize its information before forwarding the

logs to the central engine the heart of the SIEM where correlations and analyses take place.
Finally, the logs will be stored in a database for a certain amount of time depending on the

organizations retention policy. The typical architecture described above can be depicted as follows:

Figure 1: Typical Environment: Devices send their logs directly to the SIEM where collectors, central
engine, and database are located in a single appliance. In a distributed environment, the collectors
and database can be physically located at a different place.

Depending on the SIEM vendor, the term collector can be changed to agent or connector. Some

vendors also offer smart connectors which automatically detect the type of device they are

connected to, simply by recognizing the logs they receive. To remain as generic as possible, we will

keep the term collector.

The Real Value of Logs, Events, and Flows

A SIEM system can make use of diverse information types. The primary type is log data, usually

meant for several purposes such as debugging, system administration, and security audits. The most

commonly used standard for logging is Syslog.1 Depending on the device, the standard can change.
 For instance in the case of Web servers, most of them will use the Common Log Format2 or other

proprietary formats.

Another type of information that can be retrieved by a SIEM is events. Events are usually produced by

security devices or controls such as IDPS or Identity and Access Management (IAM) systems. It can

be, for example, input validation failures (e.g., invalid parameter names/value, protocol violations) or

application errors and systems events (e.g., runtime errors, connectivity problems, performance

issues, etc.). Events can be correlated together with other information to provide higher intelligence

into log management.

Compared to an ordinary logger, a SIEM system can use various conditions to check whether certain

events are matching a rule, and depending on the latter, an alarm can be triggered. For example, let

us consider a port scan: when a firewall receives a single packet on port 20, it can send a log

connection attempt to ftp service, nothing dramatic. But when it receives packets with the destination

port set from 20 to 100 in less than two seconds, all these events sent to the SIEM can match the rule

port scan, which will trigger an alert to the security team: some malicious activities are undergoing!

We can list four categories of conditions:

Event Based: An IDS reports a signature X targeted at host Y and vulnerability scanner knows that Y
is vulnerable. It triggers an alert.
Rule Based: If X + Y + Z then do A, or If X repeats more than three times in interval Y then do Z.
Anomaly Based: If the traffic on port X exceeds the standard deviation of historic traffic patterns then
trigger an alert (e.g., new worm, bot communicating with C&C).
Risk Based: If attack type is destructive (e.g., buffer overflow versus SYN scan), and target is a
critical asset (production server versus workstation), then trigger an alert.

Some event standards have been proposed to improve interoperability and simplify integration of

devices. For example, ArcSight came up with the Common Event Format3 and IBM proposed Log

Event Extended Format and Splunk Common Information Model.

Finally the last type of data a SIEM can use is traffic flow,4 providing a better overview of the network

activity. The two main standards are NetFlow (RFC 3954) and IPFIX (RFC 5101 and 5102). The

problem with these two formats is that they do not provide information above the Layer 4 of the OSI

model. To remedy this issue, some vendors propose application-aware flows which help detect

threats through the analysis of the packet content, using Deep Packet Inspection (e.g., IBMs QFlow).
The central engine will correlate all of the gathered information by using diverse algorithms and data-

mining techniques. These techniques will identify suspicious patterns and behaviors, and provide

great help for intrusion detection and auditing.

Limitations

Deploying a SIEM solution can be quite complex and expensive: the price of appliances, the time for

configuration and tuning, and the expertise required for daily use/maintenance can discourage

customers. After purchase and deployment, the recurrent question is now what do we do? and

enterprises tend to answer by using a monitor-and-respond strategy.5 By using the SIEM in a

signature-based defense approach, the security team (or the security operation center [SOC] team)

will monitor activities and regularly update the security devices with signatures of known threats.

Figure 2: Reacting upon incident flow chart.

Upon detection, the team will investigate the alert, escalate it to the Incident Response (IR) team for

remediation if they cannot resolve the issue directly themselves (e.g., stop the attack if still ongoing,

re-image compromised systems, etc.), and finally report to the board. This overall process can take

quite some time, and time is money, especially when it comes to security.

Moreover, since the signature-based approach only protects from known threats, the anomaly-based

approach, which focuses on detecting abnormal behaviors, should in theory help detect unknown

threats, but practically it significantly increases the false-positive rate. The direct consequence is the
time required by the security team to investigate them, increasing the chance of missing true-

positives. In a survey of the Ponemon Institute, they discovered that on average a company will have

up to 17,000 alerts per weeks, but only 700 will be investigated!

A recent Ponemon Institute study6 found that companies spend $1.27 million annually on average by

wasting time responding to inaccurate and erroneous intelligence. In addition, the rise of targeted

attacks, so-called advanced persistent threats (APTs), makes it clear that this traditional reactive

security posture is no longer sufficient. With shrinking security budgets,7 companies need to find new

efficient ways to defend themselves. The question we have to answer is: How do we get ahead of

threats? It is time to become proactive.

Threat Intelligence and SIEM (Part 2) Understanding Threat
Intelligence
In part one of the series we addressed the limitation of the reactive security posture of traditional

security information and event management (SIEM) solutions. As Irving Lachow wrote, passive

defenses are a necessary component of a well-designed cyber defense program, but they are no

longer sufficient to address increasingly sophisticated threats.1

To prepare their (sophisticated) attacks, hackers can buy the same security devices used by the

targeted company and craft their tools and/or methodologies to make sure the attack will be

undetected and successful. The overall visibility provided by SIEM is a good start, but we need to add

a key element to thwart such attacks in a proactive manner: threat intelligence (TI).

Before defining TI, let us understand its essence with a little analogy in physical security: consider

your network infrastructure as a bank.

You have customers coming in and out to access their accounts in a certain part of your building

(DMZ), and there are other areas which are restricted to employees only (internal network). To secure

your bank you have several security measures in place, such as guards (firewalls) and cameras

(IDPS) to help you protect your assets (e.g., money, client information). Finally, theres a security

manager, a person who coordinates all the efforts of the security team and gets regular updates from

the guards, that can see the camera feeds and other security precautions.

This manager (SIEM) can also retrieve external information from other parties (e.g., other partners,

police) about the current state of security in the financial industry. This external information can

include info on recent attacks: why did they succeed/fail, what group was involved, what were the

exploited weak points (vault, security mechanisms, etc.), how did they proceed (fraud/social

engineering vs raid at night), and what actions they took.

This is what threat intelligence is all about.

Once retrieved, the security manager can not only act upon and update all employees and security

personnel to watch for possible future threats, but also warn the banks executive team and the board

of directors to help stakeholders make the right decisions.

 Threat Intelligence Definitions

To have a better understanding of what threat intelligence is, let us first have a look at a couple of

definitions.

Cyber threat intelligence was defined by Jon Friedman and Mark Bouchard2 as follows:

It is the knowledge about adversaries and their motivations, intentions, and methods that is collected,

analyzed, and disseminated in ways that help security and business staff at all levels protect the

critical assets of the enterprise.

Levi Gundert, Vice President of Information Security Strategy at Recorded Future, defines threat

intelligence this way:

Threat Intelligence is the act of formulating an analysis based on the identification, collection, and

enrichment of relevant information. The goal of threat intelligence is to reduce operational risk, which

in turn maintains or increases business profitability. In some cases, threat intelligence contributes to

an information security program that creates a competitive advantage; strong security becomes a

market differentiator.

In summary, it is important to understand that TI is more than just information: it gives us an analysis

of adversaries and their motivations and methods, based on the collection of data that is enriched by

using context. Once integrated in a security lifecycle, it enhances the security level and awareness of

an enterprise and helps business continuity.

Why Use Threat Intelligence?

TI gives insights on attackers and their capabilities, providing invaluable information to enhance the

security level. When companies use such intelligence, they can focus their actions on several crucial

points to efficiently protect themselves:

Who is attacking: TI helps defenders attribute attacks/malicious activities to certain groups (cyber
criminals, hacktivists, government/national agencies, etc.)
Why they are doing it: knowing who is behind an attack helps defenders understand their
adversarys motivations, how much effort they would put into an attack (advanced persistent threat
[APT] vs opportunistic attacks), and how business/industry-specific such attacks could be.
 What they are after: with this information defenders can prioritize their actions based on the
importance of the asset or assets the attackers are targeting.
How they are proceeding: the so-called tactics, techniques, and procedures (TTPs) give insight
about the way adversaries typically proceed, the tools and infrastructures they use, and more.
Where they come from: correlating an adversarys country of origin with its geopolitical situation can
help defenders understand their enemies.
How to recognize them: also dubbed indicators of compromise (IOC) or artifacts, these technical
telltales (IP addresses, hashes, etc.) provide clear information that can be used to detect and signal a
malicious presence.
How to mitigate them: information about the steps a company can take to protect itself.

All of these questions are directly connected to each other. By correlating TI provided by external

parties with internal information collected by a SIEM solution, defenders have a better vision of

attacks in their context and can proactively defend themselves against emerging threats to the

business. Thanks to the efforts of companies such as the MITRE and Facebook, standards have

been created to help people retrieve, use, and create TI. We will discuss the standards later in

another post. TI can be retrieved from various sources, such as cyber security vendors, independent

labs and researchers, open source projects, and government and industry groups.

Threat intelligence can be presented at two different levels, depending on the intended audience. On

the one hand it can be at a strategic-level: it is human-readable, not too technical, and is meant to be

solely processed by humans (e.g., C-suite personnel) to give them insight into the threat impact on

business continuity, helping them make the right decisions. Typical formats of strategic intelligence

are reports or newsletters for instance.

Alternatively, intelligence can also be at the operational-level: once retrieved by SOC analysts, this

machine-readable data is consumed by devices to make them able to act upon threats. Usually

operational intelligence is XML-formatted data to ease the processing. Due to its broad variety and

level of use, TI can benefit people with different roles within the same organization. For example, in

addition to the board and SOC analysts, incident response teams can also glean valuable information

to remediate security events.

For more information on the different types of TI, such as strategic and operational, see the Aim

Small, Miss Small: Producing a World-Class Threat Intelligence Capability white paper by Levi

Gundert.

Dismissing Irrelevant Information

There are numbers of reasons companies should use TI. One of the most useful is the ability to

decrease the amount of time spent on analyzing alerts concerning irrelevant threats (e.g.,

vulnerabilities targeting applications not even present on the defended network; we saw in the

previous post that this can happen when a company is flooded with alerts). With good intelligence, an

enterprise can easily dismiss invalid indicators to weed out false positives and therefore focus on the

actual threats it is facing.

If you are working for a bank, for example, you could make sure to use TI tailormade for financial

institutions from security vendors or partners (e.g., other banks which may have seen new threats),

giving you the right keys to defend against business threats. Moreover, you could retrieve generic TI

(applicable to any business) to help thwart leaks of intellectual properties or employees personally

identified information (PII). TI also enhances vulnerability management, as it provides a way to

prioritize indicators and patches, helping IT staff fix the most dangerous vulnerabilities first.

Due to the different types and broad applications, there are several ways to consume TI. Like we

mentioned before, from a strategic to an operational level, several groups can benefit from such

intelligence and help keep the business running. TI becomes an integrated part of the security

lifecycle involving many actors within the enterprise.

In the next post in this series, we will review what tools are required to leverage TI, and discuss why

combining SIEM and TI could be a perfect match.

1Irving Lachow, Active Cyber Defense: A Framework for Policymakers (Washington, DC, February
2013).

2Jon Friedman and Mark Bouchard, Definitive Guide to Cyber Threat Intelligence (CyberEdge Press,
Annapolis, MD, 2015).
Threat Intelligence and SIEM (Part 3) Combining for Better
Security
Previously, in part one and part two of this series, I explained how threat intelligence (TI) provides

defenders better insight into the type of malware, delivery mechanisms, exploits, and overall

situational awareness of threats and attack strategies faced by other companies.

Using such intelligence with security products enhances the threat landscape visibility, and as a result

security teams are able to respond more quickly and accurately.

How Companies Are Leveraging Threat Intelligence

In a SANS Institute survey,1 Dave Shackleford asked companies around the world how they were

using TI. A total of 326 companies of all sizes and from various industries, such as government,

banking/finance, and IT, participated, resulting in the following findings:

Figure 1: Tools for aggregating and using threat intelligence. (Source: SANS Institute)

Up to 55% of the surveys respondents are using SIEM to aggregate and analyze TI data from diverse

sources. According to the way SIEM solutions work as we saw in the first part of this series, TI and

SIEM are indeed meant to be combined.

Let us consider the following flowchart to understand how TI can be combined with a SIEM:

Figure 2: Security monitoring workflow. (Source: Securosis.com)

We already explained the upper right-side of the above diagram in the first part of this series when we

discussed reactive security with SIEM. Lets now focus on the remaining boxes.

The upper left side depicts the collection of TI. Before collecting threat intelligence, we first must

define and profile our adversaries who represent business risks. Then we can select the right TI that

will truly leverage our security posture. We can then start collecting and analyzing intelligence,

which means reviewing the intelligence to make sure it is actionable. Finally we can proceed to

integrating TI with the SIEM.

In the above figure, in the Security Analytics portion, we can start analyzing our environment like it

is done in the standard SIEM approach, but this time correlating our detection mechanisms (signature

and anomaly based) with the TI relevant to our business.

 Upon detection, we prioritize alerts based on the number, frequency, and types of indicators which

triggered them.2 According to their priority, we may want to collect deeper which is collecting

more information from the devices involved by querying the databases; this makes it easier for further

forensics investigations.

Finally, as shown in the Action box, once alerts are reviewed by the security operations team, they

can be validated if its a true positive or discarded in case of a false positive. In the latter case, the

intelligence can be evaluated and the policies tuned to avoid further false alerts. After validation, the

alerts can be escalated to the IR team via a ticketing mechanism as described in the first blog post.

4 Qualities of Actionable Threat Intelligence

The advantages of using TI with SIEM is indisputable: nowadays a good SIEM solution ships with

support for threat intelligence integration. All vendors are well aware that, to be taken seriously, they

have to answer the growing demand for easier and better TI integration. Gartners security analyst

Lawrence Pingree says, by 2017, at least 50% of technology providers will use intelligence-sharing

capabilities between disparate technologies and across different vendors to support orchestrated

security policy responses across protected environments.3

As shown in the flowchart above, the collected TI will have to be assessed in terms of quality and

effectiveness. Sergio Caltagirone, Chief Scientist of the Center for Cyber Intelligence Analysis and

Threat Research, identified four qualities that intelligence has to meet to be actionable: 4

1. Relevance: TI teams must ensure the intelligence is pertinent to the business; measurable by positive
hits or real alerts once deployed in the environment.
2. Completeness: Threat intelligence must be detailed enough, and provide sufficient context, to
guarantee effective detection.
3. Timeliness: To have a valuable impact, intelligence must be retrieved and processed as fast as
possible.
4. Accuracy: To ensure effective detection, the TI must have few false positives and be highly accurate.

All these aspects can be assessed at different levels by parties involved in a complete security

lifecycle, as detailed according to the Active Cyber Defense Cycle. These qualities can be used as

criteria to acknowledge the efficiency and usefulness of the retrieved TI.

Finally, by combining internal and external threat intelligence, defenders have a way to empower real-

time threat identification; for instance, it now becomes easier to detect malware that is communicating
with C&C servers belonging to a certain campaign, to match past/historical internal log data with

current threat intelligence, or even to validate correlation rules and improve baselining alerts,

therefore reducing false positive and the waste of time and money.

In the next part of this blog series, well focus on the various threat intelligence standards, and how to

share threat intelligence.

1Dave Shackleford, Whos Using Cyberthreat Intelligence and How? (Technical Report, February
2015).

2 Securosis LLC., Leveraging Threat Intelligence in Security Monitoring (2014).

3 Lawrence Pingree, Context-Aware Security and Intelligence-Sharing Concepts Merge to Create

Intelligence-Aware Security Controls (Technical Report, 2014).

4Sergio Caltagirone, The 4 Qualities of Good Threat Intelligence (July 2015;

www.activeresponse.org).
 Threat Intelligence and SIEM (Part 4) An Active Cyber
Defense Approach
In the previous posts of this blog series, we saw how threat intelligence can significantly help

organizations move from a reactive to a proactive security approach. By having real-time insight and

context on the threat landscape, we can move left in the Cyber Kill Chain (see image below) and

thwart an attack at an earlier stage, limiting its impact and cost.

However, even with fine-tuned security devices and actionable intelligence, theres still a risk that

determined hackers will be able to break through your defenses. Nothing prevents them from buying

the same IDS/IPS and firewalls you have to find an unknown hole and craft a novel attack (otherwise

known as a zero-day exploit).

In this post well describe how to use active defense mechanisms as another layer of security to

improve your chances against adversaries.

Using Deception as an Active Defense

Active defense (AD) can be defined as any measures originated by the defender against the

attacker, and can be divided into three categories:

1. Counterattack
2. Preemptive attack
3. Active deception1

The first two categories, respectively, involve hacking back as an act of retribution, and engaging an

adversary upon its first hostile move for prevention purposes. In this post, we will focus on the non-

offensive part the last category of active deception.

Its main goal is to thwart attacks and provides the defender with attribution capabilities, helping him or

her to define the opponent; collect information about its tactics, techniques, and procedures (TTPs);

and create threat intelligence based on this information.

The idea is to increase the complexity of an attack (e.g., make it more difficult for the attacker) using

tools such as honeypots and honeytokens, forcing the attacker to make more moves that will

eventually be noticed. We can deploy decoys and monitor their activity with our security information

and event management (SIEM); every time the attacker interacts with one of them, we will be aware

of his or her presence and will be able to gather information about possible next moves.

Based on this data well create our own internal threat intelligence (TI) which can be correlated with

the external TI within the SIEM.

Lets look at these tools in details.

Honeypots

Lance Spitzner, security researcher at Symantec, defined a honeypot as an information system

resource whose value lies in unauthorized or illicit use of that resource.

Its goal is to mimic a legitimate system running vulnerable services on the network, with all activities

monitored. Once spotted, an attacker will very likely try to compromised this easy-looking target.

However, no valid information will be found, and no further damage can be done to harm the other

legitimate instances running on the same network.

An important characteristic is that honeypots cannot be accessed by normal users. By monitoring

their activities, security staff know that all events are genuinely malicious and can gain insight into

opponents methodologies to better protect their infrastructure. Moreover, they can extract forensics

information to provide law enforcement with details to prosecute. Honeypot systems can be deployed

at diverse locations on the infrastructure.

 Figure 1: Possible honeypot deployment. (Source: SANS)

The sole purpose of honeypots is to be mistaken for a production resource and undergo attacks.

From a hackers perspective, there should be no difference between honeypots and regular

machines. The ENISA identified two fundamental criteria for honeypot classification 2:

1. Type of attacked resources

2. Level of interaction

Attacked Resources

The types of attacked resources describes the running service(s) to be exploited. On a server-side

honeypot, running services such as SSH or NetBIOS is a good idea and will likely tempt attackers to

misuse them. On a client-side honeypot (or honeyclient), we can deploy specific applications, such

as a Web browser that actively connects itself to remote services and analyzes the behavior of the

server or its delivered content to detect malicious attempts.

Level of Interaction

Depending on the level of mimetic we want to achieve, we can use honeypots with three different

levels of interaction: low, high, or hybrid. Low-interaction honeypots emulate a resource, providing

limited functionality compared to the real instance. The success of tricking the opponent depends on

the degree of accuracy of emulation. Easy to deploy, emulation (in this context) reduces the risk from
 a machine to be compromised. However, some emulated resources might behave differently than the

real ones and experienced hackers may notice and abort their attempts at an early stage.

High-interaction honeypots do not use emulation and instead employ real operating systems and

resources. These are less likely to be noticed. However, the resource consumption and complexity

may affect scalability and performance.

Finally hybrid honeypots are a combination of low and high interaction, providing the best features of

both types.

In their report, the ENISA and the CERT researchers from Poland evaluated a broad range of

honeypots depending on criteria such as the detection scope, accuracy of emulation, quality of

collected data, and even the scalability. They covered and tested up to 30 different standalone

honeypots with various levels of interaction and purposes (general purpose, Web, SSH, SCADA,

VoIP, USB, etc.).

Honeytokens

The concept of honeypots mimicking a legitimate machine has been extended to files. In the same

approach, a honeytoken (also called decoy document) is information or a digital file that no

legitimate user should use.

Each access to that token is monitored and considered malicious. A typical use case is to disseminate

honeytokens in a database or a system, entitled with a juicy name such as passwordlist or

medicalrecords. It can be a text file, a spreadsheet, bogus social security numbers, or even a (fake)

user account.

Once an attacker (external or internal) accesses that file or tries to log himself in using the user

account, an alert is triggered. CERT researchers advise three different techniques to detect

honeytokens misuse:

1. System or application logs

2. IDS signatures
3. Usage of specialized data loss prevention solutions*
 *Which may make use of the previous two methods.

Another technique proposed by Matthew Toussain is to use CybOX to automatically retrieve extra

information and create an indicator of compromise (IOC). In fact CybOX supports the logging of data

and could allow us to not only identify that access, but also log information concerning what process

on the system opened its file handle, what network ports it is currently utilizing, and what other system

files it has accessed.3

Before creating honeytokens, there are few requirements identified by Brian M. Bowen et al. 4:

Must appear to be true/valid/valuable/desirable

Should not interfere with normal system operations; not pollute authentic data
Should be obvious for legitimate users that the honeytoken is a decoy for an attacker
Has to be monitored; access has to be detected
Must be unique to minimize false-positive rates

Regarding the location of decoy documents, some places should be privileged such as databases,

file-sharing services (FTP servers, Windows shares SMB), users email inbox, and corporate cloud.

The location in which tokens are deployed can be chosen according to the requirements above, but

also according to external TI such as TTPs data to increase detection: by having insights on relevant

APT groups for the business, we can use this intelligence to deploy decoys at locations the attackers

will most likely look, enhancing our detection and deception capabilities.

The most important point to consider is to make sure that the token will not interfere with the

production systems and files.

Conclusion

Active defense provides a mechanism to add an extra layer of security. It enables defenders to better

protect their network against determined attackers. Some tools such as honeypots and honeytokens

can be deployed: they increase the effort an attacker has to put in to succeed. Every interaction hell

have with a decoy will be monitored and the security team can collect this information and act.
Security teams can not only harden their infrastructure to prevent further instance of attack, but also

formalize the collected information with certain standards such as STIX and CybOX to create internal

TI and feed it into their SIEM solution.

In addition, external TI retrieved from diverse sources including the open, deep, and dark Web

can help enrich the newly created TI, providing contextual awareness to increase detection and

efficient SIEM correlation. For example, Recorded Futures real-time threat intelligence can enrich

information in Splunk and ArcSight. Finally, security teams can share their homemade TI with the

security information community to help others protecting their environment.