Hacking Book | Free Online Hacking Learning


apt organization tracking governance based on knowledge map

Posted by chiappelli at 2020-03-28

Apt refers to the concept of advanced persistent threat, which was formally proposed by the United States in 2006. It is specially used to describe the persistent attacks with advanced technical means discovered by the United States in the government and military intranet. At NIST sp 800-137, information security continuity monitoring (ICSM) for federal information systems and organizations, this defines apt (advanced persistent threat): an adversary with top-notch expertise and effective resources that allows it to achieve its goals by attacking opportunities generated by a variety of different attack media (e.g., network, physical and spoofing). It is typical to establish and expand strongholds in the organization's information system technology infrastructure, to achieve continuous information disclosure, or to destroy and hinder the operation plan or the organization to achieve mission critical tasks, or to place them where they can play a role in the future. In addition, in order to achieve its goal, the high-level persistent threat will continue for a long time many times and resist the efforts of the defenders through various ways (automatic upgrade). At the same time, it maintains a certain degree of interaction with the outside world to implement its goals.

Apt detection usually faces the following problems:

(1) Advanced intrusion detection

Apt attack has a strong ability of single point concealment. The main IP used and the related malicious code carried are often variable, which makes the single feature detection method invalid.

(2) Long term hidden problems

The attack time of apt attack is often very long, and it does not have special abnormal characteristics on a single time node, which makes the traditional detection methods often miss the actual apt attack events.

Therefore, the discovery of apt attacks and the tracking of apt organizations have become a research hotspot.

Since December 2009, when the aurora attack on Google was exposed, apt attack has been well known. At present, the famous apt attacks in the world include: in July 2010, the earthquake network attack against the industrial successful destruction of Bushehr nuclear power plant equipment in Iran; on December 23, 2015, the Christmas blackout in Ivano frankovsk region of Ukraine; on March 20, 2013, the Bank of Korea suffered the apt attack, and in 2016, the Bank of Bangladesh lost 81 million overseas deposits, etc.

In recent years, the activities of Lazarus group in North Korea in East Asia began to ease as the situation in North Korea gradually eased, but it still frequently attacked South Korea and European and American countries to obtain intelligence. In Southeast Asia, the Hailian organization mainly targets the Chinese government, scientific research and other important departments; in Europe, apt28, apt29 and turla have been widely active as the most famous apt organizations in Eastern Europe; among them, the apt28 organization supported by Russia has influenced the American general election for many times, giving birth to a new word of "election security". In the United States, combined with publicly disclosed information, as a powerful country in cyberspace capability, equation organization is considered to be related to North American intelligence agencies. Since 2013, many leaks of relevant intelligence agencies have demonstrated their complete cyberspace attack system and automatic attack weapons, and exposed relevant evidence that they regard China as one of their important targets for global cyber espionage.

A nationwide blackout occurred in Venezuela. Guri hydropower station, the third largest hydropower station in the world, was attacked. The capital Caracas and most other regions were in darkness. The power supply of 18 states in the country was interrupted, and only 5 states survived. The blackout brought heavy losses to Venezuela. The national traffic was paralyzed, the subway system was shut down, the hospital operation was interrupted, and all communication lines were interrupted The road is interrupted, the flight cannot take off and land normally. President Maduro criticized the power outage as a "power war" directed by the United States against Venezuela in a social media article on August 8.

The New York Times said the United States is currently upgrading its cyber attacks on Russia's power grid and trying to plant potential malware in Russian systems. The U.S. is doing so in preparation for cyber attacks on its power system in the event of a conflict with Russia

At the instigation of President trump of the United States, the U.S. cyber command, in cooperation with the U.S. central command, paralyzed Iran's rocket launch system and retaliated against the Iranian Islamic Revolutionary Guard for shooting down an American made reconnaissance drone near the Strait of Hormuz. According to the New York Times, the U.S. operation is also aimed at temporarily destroying computers and networks used by Iranian intelligence services, as well as planning more cyber attacks.

Iran Revolutionary Guard information warfare forces successfully broke through the containment of American information warfare forces, broke into the control center of more than 30 substations in New York City, and damaged the information station of the control center, resulting in a power outage of about 4 hours in New York City, the first time in decades, a large-scale power outage caused chaos.

The U.S. Department of defense emphasizes "cyberspace wins" and re attacks, at the same time, it requires stopping and defeating attacks against key infrastructure in advance, emphasizing intelligence sharing and joint operations.

The national network strategy inherits the national security strategy report and Presidential Decree No. 13800 signed by trump in May 2017, "strengthening the network security of federal networks and key infrastructure", which refers to four pillars: 1. Protecting the American people, land and American life style; 2. Promoting the prosperity of the United States; 3. Maintaining peace based on strength; 4. Enhancing the influence of the United States 。 It emphasizes that the network should be the priority and key part of the U.S. government's governance; it emphasizes the supporting role of private enterprises, and focuses on building a network security system that integrates government, military and enterprise, and integrates military and civilian; it explicitly proposes to prevent apt attacks from China, Russia, North Korea and other countries.

On September 5, 2018, the U.S. House of Representatives voted to pass the 2018 cyber deterrence and Response Act [2], which aims to prevent and sanction future state supported cyber attacks against the United States, so as to protect the political, economic and critical infrastructure of the United States from infringement. The bill requires the president of the United States to confirm the list of advanced persistent threat (APT) organizations and publish it in the Federal Register and update it regularly. The act also requires the U.S. government to sanction the participants who launch state supported cyber attacks against the United States, including individuals, organizational entities and even the country. The sanctions program includes or can last for up to one year's financial sanctions, export and entry restrictions, etc. The U.S. government and private enterprises have a large number of security teams or companies in the focus, sniping and in-depth study of apt attacks.

3.1 plan X of DARPA uses knowledge map to describe battlefield map to support VR operation

Plan x is a project announced by DARPA in 2012. Its main goal is to develop revolutionary technologies to understand, plan and manage network warfare in a real-time, large-scale and dynamic network environment. Based on a general map, it helps the military network operators to carry out the task of network intrusion in the battlefield in a visual way.

Plan x uses the technology of automatic construction of battlefield network map to transform the network map, combat unit and capability set in the battlefield into the set of nodes and edges in the map. Based on the tactical objectives set in advance by the warfighter, it carries out automatic map search to find the best intrusion path and scheme, which is provided to the warfighter.

Plan x is a strategic project for the U.S. military to build an automated battlefield network topology and support the U.S. soldiers to conduct intuitive and visual network intrusion attacks. By integrating the concepts of network map, combat unit and capability set, the concept of network combat space is formed. On the basis of this concept, through the research of five technical fields, the network combat space is displayed to the military personnel, so that the ordinary soldiers can also use technology to fight against the computer attacks launched by the enemy on the intuitive interface, simplifying the operation process, so that the soldiers can carry out network operations without using the keyboard.

3.2 mitre's cygraph prototype supports network operations

Mitre company is the pioneer of knowledge engineering, the famous CVE, Stix intelligence and so on. It has built the largest knowledge engineering in the United States. Cygraph is a prototype system of mitre in graph model research [3]. Cygraph uses hierarchical graph structure, including four levels of graph data: network infrastructure, security post, network threats and mission dependencies, which are used to support attack surface identification and attack situation understanding for critical asset protection. Cygraph has a deep understanding of the impact of network activities on tasks. In the form of cygraph attribute graph, layers define a subset of the whole model space, and the relationships within and across each layer.

According to mitre analysts, in the actual operation environment of the network, the key problem is not the lack of information, but how to combine different information parts into an overall analysis picture, so that it can be used to support situation awareness and the formulation of the best action plan [4]. Cygraph integrates isolated data and events into a continuous overall situation. Combined with its multi-level battlefield map structure, cygraph can dynamically provide appropriate context information to respond to attacks and protect mission critical assets.

Cygraph combines the structure of multi-layer knowledge map, stores the knowledge of each level in layers, defines the relationship within each layer and across each layer, so as to provide multiple levels of context semantics supplement for network intrusion events, and provides the best response strategy based on multi-layer graph Association.

Att & CK (advanced statistics, techniques, and common knowledge) is a model and knowledge base reflecting the attack behavior in each attack life cycle. Derived from a project for enumeration and classification for Microsoft Windows Gamma The tactics, technology and process (TTP) of the system to improve the detection of malicious activities. Compared with the traditional attack mode, ATT & CK emphasizes landing, for example, modeling and Simulation of apt3 / Gothic Panda (foreign security enterprises generally think that it is a Chinese hacker organization) attack skills are carried out, and evaluation focuses on the technical ability of products to detect the behavior of opponents. Att & CK uses the knowledge base to analyze the attack method of adversary and evaluate the existing protection system. At the same time, it can be combined with the shooting range to carry out attack simulation test and automatic verification. Mitre carried out the assessment of various protection capabilities in combination with ATT & CK, and fireeye, RSA, Microsoft, crowdstrike and other famous sandbox and EDR products participated in the assessment. The actual effect of tracking apt tissue was detected.

The practice of apt tracking based on knowledge map takes threat meta language model as the core. By analyzing the published apt analysis report, the description information and analysis logic relationship of apt organization in the report are extracted, and the apt knowledge map is constructed from top to bottom. In combination with the ontology structure of knowledge map, apt organization is tracked and depicted.

1.1 knowledge type of apt knowledge map

The definition of apt knowledge type refers to all kinds of current security standards and specifications, such as common attack mode enumeration and classification (capec) for attack mechanism, malware attribute enumeration and characteristic (MAEC) for describing malicious behavior characteristics, and public vulnerability and disclosure (CVE) for studying vulnerability formation mechanism, etc. Secondly, we analyze the open apt organizational report of Stix, and extract twelve knowledge types involved in the structured report: attack mode, campaign, defense measures, identity, threat indicator, intrusion set, malicious code, observable entity, report, attacker, tool, vulnerability.

The apt knowledge map constructed in this paper defines the following ten kinds of knowledge:

Attack mode: commonly used tactics, techniques and specific process mechanism of attackers;

Malicious code: software or code fragments that carry out malicious activities;

Hidden danger: unsafe configuration and software vulnerabilities that can be exploited by attackers;

Target object: victims' specific asset types and related attributes;

Threat subject: the initiator of the attack, which can be individuals, groups and organizations;

Report: analysis results for specific attack organizations;

Campaign: a series of threats against specific victims;

Defense strategy: means of defense and response to attack;

Threat indicator: in detection or forensics, threat object or characteristic information with high confidence;

Attack tools: attackers exploit legitimate / illegal software collections.

As a description constraint of different types of knowledge attributes, dictionary specification is convenient for the unified expression and understanding of knowledge, and it is also a standard for external data fusion and disambiguation. The design of this dictionary specification aims at the attribute description specification of ten knowledge types (attack mode, malicious code, hidden danger, target object, threat subject, report, campaign, defense strategy, threat indicator and attack tool). The dictionary design also refers to Stix and all kinds of security standards and specifications. The threat subjects include identity, role, technical level, resource level and motivation, with a total of 40 description specifications; malicious code includes 280 description specifications of malicious code actions, such as creation process; hidden danger includes 1037 vulnerability description specifications, such as input verification and representation, use of inappropriate API, etc Target object includes industry, geographical attribute and association standard, totaling 3458 description specifications; attack mode includes technical mechanism, such as 519 description mechanisms such as trusted certificate and authentication abuse; campaign includes 42 kinds of event specification definitions such as information collection and availability destruction; threat indicator includes 10 types of indicator, such as IP, domain name and file hash Observable data definition; attack tools are classified into 9 categories according to the type of tools, such as backdoor, Trojan horse, agent, etc.; defense strategies are classified into 6 categories according to the protection mode of attack chain model, including detection, rejection, interruption, degradation, deception and destruction; the attribute description of report knowledge is not defined in dictionary due to the uncertainty of its description content. Tables 4.1 and 4.2 are examples of the threat subject and malicious code part dictionaries.

Table 4.1 Dictionary of threat subject




Inner ghost




Criminal organization


Internal misuse

Self fired author




Person in charge

Malware author


Lone ranger

Infrastructure Architect

Infrastructure operators

Table 4.2 Dictionary of malicious code

Malicious code actions

network connections

create a file

Create process

Modify registry key

Exception instruction occurred

Kill process


Taking apt28: at the center of the store released by fireeye as an example [6], it is analyzed in the report that apt 28 uses vulnerabilities such as cve-2015-1701, cve-2015-2424, cve-2015-2590 and cve-2015-3043 to affect flash, Java and windows. The logical relationship used in the analysis is derived from the expert knowledge CVE ﹣ ID (vulnerability) "affects" CPE ﹣ ID (asset) contained in the U.S. national vulnerability database (NVD). In addition, NVD also contains the logical semantics of expert knowledge such as capec ﹣ ID (attack mechanism) "exploits" CWE ﹣ ID (vulnerability) ", CVE ﹣ ID (vulnerability)" belongs to "CWE ﹣ ID (vulnerability)" [7].

Secondly, seven types of relationships are defined in Stix's open structured apt report: targets, uses, indications, mitigates, attributed to, variant of, impersonates, which can connect 12 object domains. The overview of stix2.0 object relationships is shown in Figure 4.1 below.

Figure 4.1 stix2.0 structure

Summarize and summarize the various semantic relationships involved in APT report, including semantic relationships such as "instruction", "utilization", "belonging to", "including", "Action Association", "module similarity", and build the ontology structure of apt knowledge map as shown in Figure 4.2.

In this paper, we build the apt knowledge base from the top to the bottom. Firstly, we do the operation of information extraction and alignment. Based on the apt knowledge map ontology, we extract the relevant knowledge entities, attributes and knowledge relationships from the mass data. Then, according to the knowledge attributes defined in the apt knowledge ontology, the attribute disambiguation fusion supplement is carried out, and the apt knowledge base is output.

3.1 information extraction of multi-source heterogeneous and trusted apt

Relevant information sources of apt organizations include structured data (such as structured intelligence database, Stix Intelligence), semi-structured data (such as open source intelligence community websites such as alienvault [8], IBM X-Force intelligence community websites [9], MISP, ATT & CK), unstructured data (such as Talos security blog, GitHub apt report).

The structured intelligence database usually identifies the data type, so the information extraction method uses field mapping to map different fields of the same kind of data to specific attributes representing the same content.

Web site semi-structured data extraction uses web crawler technology, combined with the manual analysis of web pages. First, the data within the page is classified and aligned to the knowledge type in APT ontology, and then the data description is transformed into structured knowledge representation. In addition, the knowledge relationships such as "belonging", "using", "including" and "module similarity" are extracted by jumping the links of websites.

Unstructured data such as apt report and security blog mainly use regular expression to extract threat indicators (IP, domain name, file hash, etc.), and then use keyword matching to extract the relationship between report and organization.

3.2 knowledge disambiguation and fusion

There is still a problem after extracting knowledge to form a unified representation and aligning knowledge entities: redundant knowledge extraction and lack of attributes. In this paper, different disambiguation fusion methods are used for the knowledge types in APT ontology structure, such as threat subject. Because apt organization name uses different nicknames in the analysis reports of different manufacturers, apt organization name attribute is subject to the name of the first discovery attack organization, and other aliases are included in the alias attribute.

Figure 4.3 apt organization overview

(1) Apt organization portrait

Combined with the ontology structure of the constructed knowledge map, the attack organization of apt32 is depicted through semantic search, and the results are shown in Figure 4.4 and figure 4.5.

Figure 4.4 apt32 Diamond Model

Figure 4.5 apt 32 portrait

Image information includes infrastructure (IP, domain name, e-mail) controlled by apt32 organization, technical means (mastered loopholes, attack technical means) and attack tools (agent tools, backdoor tools, etc.). Image information provides rich compound semantics for apt event analysis, which can be used to support and promote the tracking of apt related events.

(2) Apt organization tracking

Combined with apt image knowledge, the real-time monitoring and statistics of the active situation of apt organization are realized by comparing the attribute characteristics of real-time monitoring threat events with the organizational characteristics of apt.

This paper is based on a large data analysis cluster experiment environment which consists of 600 IDS, Sandbox probe equipment and 4 servers in a regulatory department, and combines 225 apt organization portrait features provided by knowledge map to carry out feature association,

In the period from June 2, 2019 to June 9, 2019, five apt organizations were found to be active. The results are shown in Figure 4.6.

Figure 4.6 apt organization tracking

In the "419" speech, it was clearly pointed out: "first, establish a correct concept of network security. Second, accelerate the construction of key information infrastructure security system. Third, all-weather and all-round awareness of network security situation. " It is pointed out that "the key information infrastructure in the fields of finance, energy, electricity, communication, transportation, etc. is the nerve center of economic and social operation, the top priority of network security and the target that may be attacked. "If you know yourself and your enemy, you can win every battle." Not realizing that risk is the biggest risk. The network security has a strong concealment. A technical loophole and security risk may not be found for several years. As a result, "who comes in and doesn't know, whether the enemy is a friend or not, and what they do don't know". For a long time, "Lurk" in it, once something happens, it will happen

Apt attacks are usually highly secretive and persistent, and their targets are often targeted at the important assets and key infrastructure of enterprises and governments, so apt attacks also have great harm.

The U.S. government and enterprises have attached great importance to the detection of apt attacks for a long time, and their apt detection and protection capabilities are still the most advanced in the world. Based on the achievements of American government and enterprises in APT detection and protection, as well as some experience of this paper, the following suggestions are given.

At present, our government has not yet issued policies and regulations specifically for apt attacks, which is very unfavorable for promoting, standardizing and guiding the analysis and detection of domestic apt attacks. Therefore, the state should issue relevant laws and policies as soon as possible to clarify the importance and urgency of apt attack detection, so as to lay a foundation for the government and security enterprises to detect and Research on apt attacks.

The U.S. government has always attached great importance to the cooperation with enterprises in network security technology research and model construction. Typical meta language models, such as Stix framework, ATT & CK framework, capec, MAEC, CWE and CVE built by mitre company, have been greatly supported by the U.S. government and widely used in various standard frameworks of the U.S. government and military. At present, China's government is also gradually strengthening cooperation with enterprises, which needs to be further deepened. The state and enterprises need to further cooperate to build technical solutions and model standards that can be used at the industry and national level.

Because of the high secrecy of apt attacks, the sharing of threat intelligence about apt attack organizations is of great significance to the discovery and monitoring of apt attacks. Since 9 / 11, the United States has begun to rectify the domestic chaotic information sharing mechanism. Many governments and government designated security companies have begun to establish cross departmental and even cross industry information sharing formats, and build a series of threat information sharing models, including Stix built by mitre company. In the model of threat information sharing, China did not refer to Stix structure until 2018, and launched GB / T 36643-2018 national standard of information security technology network security threat information format specification. Since its launch, Stix has been quickly promoted as a standard format for Threat Intelligence description by governments and enterprises in the United States. The vast majority of apt reports released in the United States will provide sitx format for all governments and enterprises to share. However, since the implementation of the national standard in China, it has not been widely used in domestic government and enterprises. The wide use of Stix is of great significance for US government and enterprises to share and analyze network intrusion events, including apt attacks. China should constantly improve the specification of general intelligence description, focus on training and promoting the sharing mechanism of general threat intelligence, and lay a good foundation for detection and defense of apt attacks.

The Stix framework can be widely used as a general threat information description specification in the vast majority of government and enterprise departments in the United States. One of the main reasons is that under the Stix framework, mitre company has gradually improved a set of threat meta language models to support, including general attack pattern enumeration and classification (capec), malware attribute enumeration and classification (MAEC), general vulnerability Sex enumeration (CWE), general vulnerability and disclosure (CVE), etc. At present, China has not built a complete set of universal threat meta language to support unified threat information expression format and apt related threat information and knowledge sharing.

This paper combs the changes of us cyberspace strategy from the aspects of strategy, regulations and specific technical models, as well as the technologies and models related to apt detection and tracking; and through the construction of a complete apt thematic knowledge map to support the tracking of apt organizations, the feasibility of this technical scheme is verified through practical exercises and tests. In addition, some suggestions are put forward to promote the detection and tracking ability of apt in China.


[1].Binde B E,MccRee R,O'Connor TJ.Assessing Outbound Traffic to Uncover Advanced Persistent Threat [R]. Maryland:SANS Technology Institute,2011.

[2].H.R.5576 - Cyber Deterrence and Response Act of 2018[EB/OL].https://www.congress.gov/bill/115th-congress/house-bill/5576/,2018

[3].Noel S, Bodeau D, McQuaid R. Big-data graph knowledge bases for cyber resilience[C]//NATO IST-153 Workshop on Cyber Resilience. 2017: 23-25.

[4].CyGraph: Cybersecurity Situational Awareness[EB/OL]. https://neo4j.com/blog/cygraph-cybersecurity-situational-awareness/,2015

[5].STIX.Introduction to STIX[EB/OL]. https://oasis-open.github.io/cti-documentation/

[6].APT28:AT THE CENTER OF THE STORM[EB/OL]. https://www2.fireeye.com/rs/848-DID-242/images/APT28-Center-of-Storm-2017.pdf


[8].AlienVault[EB/OL]. https://otx.alienvault.com/preview

[9].IBM X-Force[EB/OL]. https://exchange.xforce.ibmcloud.com/

Past review

The right way to open machine learning in network security, have you failed to learn it?

Review after five years -- Analysis of the use of the back door of Leike router

Similarities and differences between vessel safety and EDR

[recruitment] recruitment announcement of interns of Lvmeng science and Technology Innovation Center (long term effective)

The original article of the official account only represents the author's viewpoint and does not represent the position of the Green League. All original content copyright belongs to green alliance technology research communication. Without authorization, no media, WeChat official account is allowed to be copied, reproduced, excerpts or otherwise used. The reprint should be marked from the Green Alliance Technology Research Newsletter and attached to the link.

About us

Lvmeng technology research communication is operated by Lvmeng technology innovation center, which is the leading technology research department of Lvmeng technology. It includes Cloud Security Lab, security big data analysis lab and Internet of things Security Lab. The team members are composed of doctors and masters from Tsinghua University, Peking University, Harbin Institute of technology, Chinese Academy of Sciences, Beijing post and other key universities.

As one of the important training units of "post doctoral workstation sub station of Haidian Park of Zhongguancun Science and Technology Park", Lvmeng science and technology innovation center has carried out post doctoral joint training with Tsinghua University. The scientific research achievements have covered all kinds of national projects, national patents, national standards, high-level academic papers, professional books, etc.

We continue to explore the cutting-edge academic direction in the field of information security, starting from practice, combining the company's resources and advanced technology, to achieve a concept level prototype system, and then deliver product line incubation products and create huge economic value.

Long press the QR code above to follow us