Hacking Book | Free Online Hacking Learning

 Home

threat intelligence in situation awareness

Posted by truschel at 2020-02-27
all

This paper is a text version of the speech of the same name at the 2017 ISC conference, focusing on the content that was not launched due to time constraints.

Situation awareness is a big data analysis problem

Situation awareness is a kind of security capacity building, which focuses on the detection and analysis of threats. It just adapts to the current situation that only investing in defense can no longer effectively deal with the current threat. Therefore, under the catalysis of 4.19 speech, it quickly became the most important safety hotspot.

Situation awareness is based on the collection of alarms and metadata. In order to achieve the goal of "all-round and all-weather", it needs to use real-time data and historical data for detection in three aspects: flow, content and terminal. But the visual display of simple alarm is not the real "state". In order to present the current "state", it is necessary to conduct false alarm screening, qualitative analysis (identification of directional attack or random attack), understand the impact scope and harm of attack, determine the methods and difficulties of mitigation or removal, etc. for the alarm or exception. Under this premise, it can be called "state" if the organization is faced with a comprehensive presentation of security events. And "potential" is a possible security event or state in the future. This prediction can be based on the intention of known attackers, technical and tactical characteristics and killchain analysis. If we can obtain information sharing from related industries or organizations, we can undoubtedly master "potential" more comprehensively. So we can think that situation awareness is a problem of big data security analysis in a sense (any security topic seems to be able to say so nowadays).

When it comes to big data analysis, another word that is most directly related is machine learning. But at present, the role of machine learning or artificial intelligence in security is not as big as the general imagination. Palantir, a famous big data analysis company, believes that if the data comes from many different sources, it is difficult to ensure the completeness of the data, or the opponent is aware of confrontation detection and other conditions, it is difficult to completely rely on machine learning, more security analysts are needed to intervene. On the other hand, the exceptions found by machine learning have no context. If we want to deal with such exceptions, we also need the intervention of analysts. Take the most mature discovery of DGA domain name by machine learning as an example. It can't tell the security operator that this DGA domain name corresponds to a blackmail software, a worm Trojan, or an apt attack. Is it a DNS transmission tunnel? Without this information, the event cannot be handled effectively. Finally, the false alarm rate of machine learning can not be well controlled. According to the data provided by trend company this year, the false alarm rate of machine learning is more than 100 times higher than the traditional rule detection. In short, with the development of machine learning technology, we can rely on it to provide the efficiency of analysts, but it can not be completely replaced. Security analysts remain the most important resource.

Security analysis capacity needs to be built gradually

Mature security analysts are not available to most organizations. Under this condition, how to build situation awareness? The following provides a gradual development path for reference:

Based on the start of machine readable Threat Intelligence: Taking IOC as an example, timely detection of the lost host is the key to prevent the actual occurrence of major losses. IOC intelligence can be detected and analyzed by relying on DNS logs or online behavior logs. And provide context information such as attack type, gang, attack mode, harm and disposal suggestions. The requirements for internal data, external intelligence and human are simple. As a starting point, it is very suitable for security personnel to have practical operation and understanding of the whole process of detection, analysis, research and disposal.

Advanced analysis models provided by existing products: good big data analysis products (open source or commercial) will provide some typical security analysis scenarios, how to find exceptions, how to quickly check through intelligence, how to use internal data for in-depth investigation and analysis. It is undoubtedly the most convenient and feasible way to use these existing products to complete more internal data collection and master further safety analysis skills.

Through TTP intelligence (attacker's tactics, technology and attack process), the ability of anomaly analysis is further enhanced: if the organization has a relatively solid foundation in data collection ability, intelligence collection ability and personal ability of security analyst in the first two stages, then it can focus on collecting threat intelligence related to its own industry and organization to understand the attack The author tries to find the corresponding clues in the data analysis platform and study them.

The mature analysis mode is solidified through the automation tools: the internal data and external intelligence in the final organization are relatively stable, and the analysis mode and process are relatively stable and clear. At this time, the parts that can be completed automatically need to be realized in different ways. In order to achieve efficient operation and maintenance and avoid too many safety analysts falling into simple and repetitive labor.

The whole process is undoubtedly centered on Threat Intelligence. It involves a variety of different threat intelligence. Let's introduce them one by one.

Mrti: machine readable Threat Intelligence

Machine readable intelligence is more about empowering security products, enabling them to detect and discover more critical threats, and providing necessary content such as priority, context and other event response for alarms, making devices smarter and people more responsive. The most common machine readable information is divided into three categories:

Lost detection IOC Intelligence: it is used to find the lost hosts controlled by apt gangs, Trojan backdoors and Botnet, which are usually domain names and URLs. From the trend, the proportion of deploying CNC architecture by attacking some legal websites is increasing, so the detection necessity of URL type is increasing.

File reputation: the detection ability of a single virus engine for files is difficult to meet the needs of reality. Based on a large number of sample databases in the cloud, through multiple engines, sandbox, Yara rules and other detection and analysis methods, we can get more comprehensive and accurate types, families and other information of malicious files. Querying file reputation through file hash is an effective enhancement mode for AV or sandbox detection without affecting the stability and performance of the local system.

IP Intelligence: IP intelligence is the intelligence data under the protection scenario of Internet service server. Using IP intelligence data, we can intercept the known Black IP, judge the priority of Web attack alarm (screen a large number of automatic attacks in the network) and increase the context information related to the attack.

Threat Intelligence Analysis Platform

Once there is an alarm that needs to be handled, the security analyst needs to have the corresponding tools to identify the false alarm, identify the type of attack and be able to further analyze the intention of attack and the background of the gang. Threat intelligence analysis platform is a special tool for this purpose.

Taking the 360 Threat Intelligence Analysis Platform (Ti. 360. Net) as an example, the following information can be provided for a domain name:

Identification information of domain names from different security information sources;

Sample and malicious link information of domain name association;

The number of visits, the earliest existence time and the latest visit time of the domain name itself;

Known attack families or groups related to domain name, and corresponding details;

Once mentioned the security blog or analysis report of this domain name;

What IP addresses did the domain name refer to;

Domain name registrant information;

...

Using this information, we can master the information of the main information sources on the domain name query in the world, judge whether the domain name is black or white, what kind of attackers are using it, the time period and impact of using it, and can mine more content through association analysis.

When it comes to association analysis, we often equate with the attacker's traceability, but there are many practical scenarios, such as the IP found in an attack. We want to know whether it will be a directed attack. Through platform query, we find that these IP have been used by the Mafia in recent years to do spam attacks, which can preliminarily exclude the possibility of directed attacks. We also want to know whether several IPS in an attack belong to the same gang. We can refer to the attack history information through the intelligence platform, and also pay attention to the geographic location, host type (gateway, IDC host, terminal, etc.), operating system and other information of IP, which are powerful basis for quick judgment.

Association analysis needs certain knowledge and experience basis. Visualization is an effective way to reduce the analysis threshold, especially the visual analysis of built-in automatic analysis model, which can automatically display other elements related to query objects, and identify special objects in a certain way. As the following Fortinet recently announced an IOC, although there is no special multi context information through the domain name itself, in the visual analysis, the associated attack resources and virtual identity can be seen clearly at a glance.

Human readable intelligence (TTP intelligence, strategic intelligence)

TTP intelligence (tactics, technology, attack process) is the information provided to security analysts and security operators, focusing on malware, vulnerabilities, attacks, focusing on the analysis of its purpose, harm, mechanism, impact range and detection and prevention mechanism. These intelligence can help enterprises in daily security work face attacks against their own organizations or industries, prevent the occurrence of threats in advance, or obtain handling methods when threats occur. Analysts can also expand and accumulate means and methods of safe hunting through TTP intelligence. Here's a recent example: on August 7, 2017, the xshell software manufacturer announced that it solved the security problem of a software version with the cooperation of Kaspersky, but did not disclose the technical details and hazards, so it did not receive enough attention. 360 Threat Intelligence Center tracked and found that its version component contains backdoor code, and there are a large number of lost hosts in the current network, and information is collected and may be implanted into more malicious components, so it took the lead in issuing the analysis report on August 14, after which other security manufacturers have confirmed the problem, attracted attention and began to deal with this major security risk. (Reference: https://ti.360.net/blog/articles/analysis-of-xshell-ghost/).

Strategic intelligence is provided to CSO level managers and security operators. Compared with tactical intelligence, it is more comprehensive. Its content is not limited to specific attack events, but a certain attack area or a certain kind of threat faced by a specific industry. It can help managers determine the direction or focus of security investment, including resources, strategies, etc. On September 10, 360 Threat Intelligence Center released the analysis report on source attack of software supply chain (reference link: https://ti.360.net/blog/articles/supply-chain-attacks-of-software/), which is aimed at the supply chain attack mode with frequent incidents, great impact and serious harm. It provides security management with attack mode, harm, typical cases, protection suggestions and other aspects Detailed intelligence information to support the understanding of how to establish appropriate protection in this area, but also give the security operators the necessary clues to understand how to find and detect the corresponding threat.

Summary

Threat Intelligence is still a relatively new security technology in China, but it is the best choice to help organizations improve detection, analysis, prevention and prediction capabilities rapidly at this stage, especially in the process of building the comprehensive capacity of situation awareness. At the same time, relying on different types of information related products, so that the ability of analysts gradually grow, improve efficiency, is also an effective way to solve the talent gap.

© 2023