Hacking Book | Free Online Hacking Learning


threat intelligence in security analysis (1)

Posted by agaran at 2020-03-08

0x00 background

I've been too busy in the past two months. This blog has been delayed until now. A series of safety analysis was planned, but it was put on hold for work reasons. In the Dragon Boat Festival, I went to Chengdu to eat hot pot and wrote this piece of Threat Intelligence in security analysis on the shuttle. As the beginning of the Security Analysis series.

0x01 what is Threat Intelligence

I define Threat Intelligence as security information that has been studied and judged.

There are three entities: research, security information, threat intelligence. The relationship among them is as follows:

No untrained security information can be called "Threat Intelligence". Threat Intelligence is used to support decision-making or security analysis. Security information of unknown source and authenticity will affect the accuracy of decision-making and analysis results.

Threat Intelligence is a description of the current (and past) state of the target for a period of time; it is affected by the quantity and quality of security information and the rationality of the research and judgment process, so that threat intelligence is not 100% correct.

0x02 safety information collection

Security analysis relies on the ability to obtain and use data, so the first problem to be overcome is "how to collect security information".

I wrote a brief discussion on the life cycle of safety analysis before. It expresses a view that the starting point of safety analysis should be "setting goals". It is also applicable here. Before the safety information collection plan, the objectives and scope shall be clear. The information collection plan formulated shall include "the type of information to be processed", "the feasible entry point for information research and judgment", "the widest possible source channel" and "when to collect information".

For the information itself, the following points must be paid attention to when collecting:

I've talked about so many requirements for information collection. I'd like to talk about the sources of information: osint, closed and confidential.

Security analysts should be based on full source analysis, rather than limited to easily accessible information. In any way, the starting point is to get the desired information, and the goal is to export the high-quality information needed for decision-making. From the cost point of view, the cost of open-source data collection is far lower than the deployment of private assets; the difficulty of open-source data acquisition is low, but the amount of processing is huge, so a more reasonable information acquisition structure is that the three complement each other.

0x03 information research and judgment

In the stage of information collection, it is only to build the access to information, aiming at the source, not the information itself. Only by studying and judging information itself can it be transformed into Threat Intelligence.

Information research and judgment is a very important link in the life cycle of threat intelligence. The mainstream way of research and judgment: human flesh to see or run some machine learning algorithms.

People's subjective judgment is very accurate. Since they are human beings, they must have their own fields of expertise and knowledge blind areas. It is difficult for people to judge the information in the fields that they are not good at. At the same time, people's energy is limited and they are weak in the face of massive information.

In order to solve the problem of subjective judgment, some manufacturers have introduced machine learning to the information judgment. There is no denying that this is the trend in the era of information explosion. Limited by the current bottleneck of machine learning development, it is difficult to have an algorithm that can fully automatically carry out information research and judgment, and then pat me on the face with the processing results of four nines accuracy rate, and directly tell me that this is accurate Threat Intelligence. It's impossible.

Threat Intelligence is used to make decisions and support our analysis. Any threat intelligence that can not reach the accuracy of 99.99% cannot be directly used in actual production. In the case of incomplete credibility, someone must intervene. It also indirectly shows the necessity of safety analysis and safety operation. In the field of security analysis, human-computer collaboration is still the mainstream in the future.

0x04 information research and judgment model

On the osint information research and judgment, I will talk about the specific methods of information research and judgment.

There are three basic principles in the research and judgment process:

Many people do research and judgment, NLP, thesaurus, or even supervised / unsupervised machine learning, which are not to the point. I don't deny the necessity of these jobs. But the thinking is more or less crooked.

Therefore, I think there are two aspects of information research:

Many people only pay attention to information itself, but ignore the dimension of "information source". By adding the source reliability judgment, the accuracy rate of information research will be greatly improved.

Size determination

Based on these two dimensions, is it easier to divide some fine particles? Here are some judging sizes:


Information itself:

After the size is marked, the information source can be y-axis, the information itself is x-axis, and the unknown state is the origin to establish the coordinate system:

In this way, the information processed by the machine can be divided into three levels:

To put it in words: the security information with reliable information source and high quality of information itself is valuable threat information.

Value description

Of course, there are 369 Threat Intelligence, and the information that needs to be studied and judged has priorities. In fact, there are ways to quantify "value".

Although the above description of size is divided into six levels, human language is actually represented in the digital world by numbers. The standard can be quantified by algorithm.

The value of threat intelligence can be identified by the module of its coordinates:

The value of a, which is automatically determined as threat intelligence, is:

(x > 0 and Y > 0)

Similarly, the priority to be determined for mapping B and C is:

(x > 0 | y > 0)

(x > 0 | y > 0)

In this way, the level of threat intelligence can be distinguished. This is only limited to OSINT information processing. Different types of safety information, although the size is different, but the general idea is the same.

0x05 practice

I've talked about so many methodologies. To help better understand them, here's an example.

The scene is set as: enterprise security construction, collection of vulnerability Threat Intelligence for security operation.

0x06 postscript

First of all, this film only describes the perspective of security analysis, how to produce Threat Intelligence, and provides an engineering solution.

Because the problem of security analysis is too big, the general method abstracted out is always dry, hoping to combine the content of "0x05 practice" for readers to gain.

PS: blog synchronization update, http://pi4net.com needs to turn over the wall.

If you are interested in security analysis, please comment. It may be the next blog content.