Artificial intelligence (AI) has never been a concept that emerged in recent two years. It has gone through three waves since 1956. At the Dartmouth conference in 1956, four Turing prize winners, one Nobel Prize winner and Shannon, the founder of information theory, were regarded as the first people to propose artificial intelligence. The picture below is a group photo of several people living in Dartmouth in 2006, which is the 50th anniversary of the advent of artificial intelligence.

From left: Moore, McCarthy, Minsky, Selfridge, salomonov

However, artificial intelligence has experienced three waves. In 1990, it ushered in the second golden age. At that time, Japan intended to build the legendary "fifth generation computer". At that time, Japan claimed that the ability of the fifth generation computer was to learn independently. With the failure of the development of the fifth generation computer, artificial intelligence was put on hold again; by 2006 The emergence of in-depth learning has become another wave of artificial intelligence so far. These are the contents that iFLYTEK's general director of voice cloud technology introduced to us on the basis of CSS 2017 Third China leaders Internet Security Conference and the sub forum of artificial intelligence and security ethics held by Beijing National Conference Center.

The main theme of this year's CSS conference is artificial intelligence, so we heard Professor Song Xiaodong of UCB on the main forum about the relationship between artificial intelligence and security. Not only that, cloud service providers have been promoting big data, cloud computing and artificial intelligence in the past two years, and Tencent cloud is no exception. Li Wei, vice president of Tencent cloud and head of Tencent social network and cloud security, said that all parts of Tencent cloud's security products, including Tianyu, Yunjing, legu, and Shudun, have used the "security AI brain" in the branch venue of "big data and cloud security" on the next day of yesterday's conference.

In fact, it is inevitable for cloud service providers to hold high the banner of artificial intelligence to engage in security. On the one hand, at present, they are in an era where single point protection has little effect, and traditional security tools have great limitations; on the other hand, in this era, the so-called artificial intelligence, the main body is machine learning, and machine learning needs data the most - data is the natural advantage of cloud service providers Besides, Tencent is not only an enterprise with cloud data. But is this really a good thing?

CSS 2017's two sub forums expanded the attack and defense of artificial intelligence in a more detailed way. One of the most frequently asked questions Li Wei received in an interview with the media in the afternoon was song Xiaodong's comment on the main forum about how security manufacturers feel about attackers abusing artificial intelligence or attacking artificial intelligence.

Data is the cornerstone of AI, especially security

Interestingly, sun Chaohui, CEO of Beijing paiwang Software Co., Ltd., has not been invited to the security forum for the first time. As usual, paiwang is not a security enterprise, but an enterprise that does flow control and optimization solutions. At last year's Alibaba cloud security summit, we also saw sun Zhaohui. This time, he explained that panabit observed the abnormal traffic during the outbreak of dark cloud DDoS attacks - students who have been paying close attention to freebuf for a long time should be clear that Tencent has sent several waves of analysis articles on dark cloud III. In fact, in late May, Pai net software has found that the attacks focused on some IP addresses are scattered, and they do not generate targeted large-scale attacks at one time. They belong to "pulse pressure DDoS attacks", so many anti DDoS and firewall security devices have not been detected at first.

Tencent released the warning and analysis of dark cloud III until June 9. As sun Zhaohui said last year, enterprises such as panabit hold data, but they are unable to act because they are not security enterprises. But Sun Zhaohui often appears in the vision of security conference, which is the embodiment of data value.

Shao Jiangning, chief security officer of Microsoft China and chief architect of the government and public utilities department, directly defined Microsoft as "one of the strongest enterprises" of artificial intelligence in the keynote speech. His definition of machine learning model is as follows:

Model = data + algorithm

"What security companies lack most is data", and data may also become the key to the ability game between enterprises in the era of artificial intelligence. That's probably the key to Microsoft's daring to say that. Who makes people cloud service providers and system manufacturers? This is the aggregator of handheld scale data. Data analysis and statistics such as malicious programs are almost handy. Microsoft claims to have "cosmic security alert data", such as tens of petabytes of logs, 300 million active Microsoft account users, etc.

The topic shared by Dr. Cheng Jiefeng, an expert engineer of Tencent cloud, is "in depth learning in graph Computing: Security AI chapter". In his speech, he shared the difficulties Tencent cloud encountered in using machine learning to crack down on malicious accounts. Perhaps we can conclude that the application difficulty of machine learning in the security field lies in the data volume.

Tencent cloud expert engineer Cheng Jiefeng

Even though we know that the Imagenet challenge launched by Stanford University, Princeton University and Columbia University in the early years makes the recognition accuracy of computer vision surpass that of human eyes, it is more like people's "pie", and more like a pie for the security community. According to Cheng, the success of computer vision at this stage is due to the fact that we have a large number of pictures that can be used as material sources, such as the pictures of cats and dogs. Alphago's success is similar in that it has a large number of ready-made samples.

But in the field of security, it is difficult to obtain samples of this magnitude. On the one hand, a large amount of data that can be used for machine learning is actually the data after the event has occurred; on the other hand, security itself is a process of dynamic balance. When we set up a protocol rule, after the attacker bypasses it, the defense party will update it and keep changing. Anomaly recognition has a high demand for machine learning, and it also faces more complex problems.

From the analysis of hosts, services, terminals and networks, malicious accounts are often a major source of attacks and fraud. We need to use machine learning to identify malicious accounts from billions of accounts, for example, to identify illegal pornographic accounts from malicious account avatars. Some of those pornographic accounts are very sexy, some are directly exposed, some are with text content. But in fact, there is only a small proportion of all kinds of pornographic images. For example, the proportion of "black frame" bad images is only less than 0.005%. How can we talk about machine learning? If we only take "sexy" as the research dimension of the head image, or even the head image with text as the feature, there will be a large-scale miscarriage. Therefore, it is necessary to extract the information related to the black production finally, and combine the malicious account number, black mobile phone number, black device, IP address, "sub graph embedding depth calculation" to complete the final pornographic Gang signal recognition bad image. Finally, according to Cheng Jiefeng, the recognition accuracy can reach 99%.

This is actually an example of Tencent's cloud integration of artificial intelligence. In the final analysis, it is still data that makes up for its shortcomings. On the first day of the main forum, Dong Zhiqiang, head of Yunding laboratory, mentioned dark cloud III in his speech "Yunding viewpoint: Interpretation of the trend of public cloud malicious code", which controls wannacry in advance, and shows the current analysis of server security software use, trojan virus trend statistics, and even the characteristics of webshell, which are the embodiment of data advantages of cloud service providers.

Is AI really the future of the security industry?

When the data becomes extremely massive, automation is a necessary strategy, and because of the "polymorphism" of malicious code, if the defense party wants to disrupt the attack chain of the attacker, it needs to be faster than the attacker, so automation is also necessary. However, Wang Haibing, director of geekpwn lab, in the topic "Ai security in the eyes of geekpwn hackers", elaborated the way to fight against AI from the perspective of attackers. This issue is in fact the same as Professor Song Xiaodong of the University of California, Berkeley, who mentioned the concerns about AI security in the main forum.

From the conventional point of view of hackers, the methods for different devices are basically the same. For example, Tencent Cohen lab seems to be a brand-new field to crack Tesla. In fact, it is to conduct in-depth research on its firmware based on Linux system; for example, for many router attacks, it is only the vulnerability of components like web server that is accurately identified. From this point of view, AI is not so mysterious. It needs to run in the information system environment first. For example, many face recognition algorithms can be considered as the application of deep learning, while the face recognition of hackers bypassing access control mainly uses the system loopholes, rather than the problems of AI itself. This in itself is what AI should consider.

What's more, the development of artificial intelligence for today's practitioners does not need to start from scratch, but the ready-made libraries and frameworks can be used. For example, opencv, a computer vision library, and Google's famous tensorflow framework have found a considerable number of security vulnerabilities.

The most wonderful part is that the algorithm is attacked. The classic example is the panda picture above. After the careful interference of the attacker, the so-called counter sample finally produced is difficult to see the difference with the naked eye. However, the deviation of the infection has been carefully designed, and this picture can allow the neural network to misclassify it. In October last year's excellent silicon valley station, Ian goodsell gave a speech entitled physical advanced examples. When we put a picture of a bird through anti-interference, we print it out with a printer, and then identify it with a device, we can still recognize it as an airplane.

This study is related to Song Xiaodong's involvement in traffic signs being tampered with, which can have a malicious impact on the identification system of autonomous driving vehicles. However, the study of University of Illinois, Urbana-Champaign believes that, for traffic signs, the identification of automatic driving vehicles may change because of the relationship between distance and angle. For example, identification at a certain distance will not affect the sample, but soon, OpenAI Posted a cat, even with the distance and angle difference of observation recognition, the cat will be recognized as a desktop computer in most cases due to careful interference.

From the above three aspects, the security problems existing in AI are also worthy of further study in future technology development. But in fact, so far, we haven't heard of any AI research companies that have worked hard on the security of AI itself. So Wang Haibing said:

"A lot of AI people don't have the safety string."

However, in this scenario, development is serving for security, but its own security problems are once again ignored. This is actually only part of what song Xiaodong said about attacks on AI (attacks on AI and abuse of attack AI in AI)

Nowadays, there are more and more security products of application machine learning, such as many "next generation terminal" products, such as cyclone, crowdstrike, sentinelone and so on. However, Lin Yichen, who is in charge of security research of didi labs, thinks that the iterative efficiency of machine learning model alone can't catch up with the attack speed in fact, and its iterative frequency is still at least weekly. Therefore, machine learning and deep learning are applied to scenes like offline learning, and it is valuable to find association and exception from existing data. This also determines that at present, machine learning can only be used as a dimension of multi-dimensional security, such as wind control. It is almost impossible to use face recognition alone, and it must be supplemented by user portraits and strategy models.

But at the end of the speech, Lin Yilin said one thing: is ai the right way to safety? I don't know

Li Wei, vice president of Tencent cloud, head of Tencent social network and Tencent Cloud Security (middle)

Li Wei, vice president of Tencent cloud and head of Tencent social network and Tencent cloud security, said in an interview with the media group that Tencent cloud once encountered a case where a criminal gang did not hire people to identify the verification code, but rented multiple hosts of a cloud manufacturer, applied artificial intelligence to identify them. This case is enough to show that the attacker has also introduced the application of artificial intelligence in attack technology. As mentioned above, the scenarios set by the researchers were never arbitrary.

What are the criteria for AI success?

Elon Musk had earlier put forward a view that artificial intelligence might be strong enough to dominate human beings in the future. In order to avoid this tragedy, musk set up a new company neurolink to integrate human brain and computer, which he thought was the only way for human beings. He even mentioned that "brain computer interface" can translate human consciousness in real time, and transform it into electronic signals that can be output, so as to connect or even control external devices. But this idea was completely denied by Professor Zhai Zhenming in the field of AI and safety ethics branch.

Zhai Zhenming, a professor of human-computer interconnection laboratory at Sun Yat sen University, graduated from Beijing Iron and Steel Institute (in his own words, he made steel), went to the United States to do philosophy research and became a doctor of philosophy, and later began to study VR and artificial intelligence. Now he has his own VR laboratory, which integrates the Internet of things and VR to form the so-called Er (extended reality).

Zhai Zhenming, Professor of human computer interconnection laboratory, Sun Yat sen University

According to his description, if you wear VR glasses in the laboratory, the first thing you can see from the glasses is still the real environment in the laboratory, but the wearer doesn't know when to start and enter the virtual reality environment, so he can't understand whether he is in the reality or in the virtual reality. Because during the whole process, the wearer and IOT The device has a real interaction, which is easy to cheat people.

Zhai Zhenming denies that Musk's foothold lies in, first of all, that based on his discussion with American quantum physicist Henry Stapp, the so-called artificial intelligence can't generate real self-consciousness, or that AI generated by Turing model and von Neumann model can't have self-consciousness. He put forward an interesting thought: what is the difference between putting VR glasses on human beings and a machine? Whether alphago or alpha cat, even if the convolution layer is 5 million layers, the artificial intelligence behind VR glasses is fundamentally different from people's self-consciousness.

This picture is worth pondering

The core of this is that human beings receive external cognitive information by natural senses rather than external control signals, which is the basis for human individuals to maintain their dominant position. If we start to instill control signals directly into the human brain, then the self-consciousness of human may be out of the question, and even the self-consciousness of human may be completely erased, because Musk's Neuralink intends to stimulate the brain center directly around the natural senses of human, which is the collapse of self-consciousness itself, and eventually leads to the destruction of human beings.

But more interestingly, as Vladimir dashchenko, a security expert at Kaspersky Lab, said on the main forum, this bypass method provides more convenience for attackers to control human beings. However, if Er (extended reality) can succeed in a large scale, it will confuse whether the wearer is in the real world or the digital world, and its harm is about the same as Musk's brain computer fusion. Looking back at the differences between VR glasses worn by machines and VR glasses worn by humans, can you feel the gap? It's a little far away, or it's a little ahead of the time. Here's a simple share of recent works. Interested students can search Professor Zhai Zhenming's research by themselves (three principles of human-computer interaction can be searched).

We have said so many AI problems and problems, but it is still the main theme to use AI to help security, because AI is indeed effective, even if not completely effective. At this stage, Shao Jiangning of Microsoft summarizes whether AI is successfully applied in the security field:

Adaptable, explainable, actionable

Among them, adaptability can be listed as a new variety that can adapt to malicious programs; explainable means that human beings can understand, not be unable to attribute and explain; executable means to improve response speed.

Successful data detection requires a combination of discrete data sets and rules with security expertise. The final success of AI application lies in real-time detection, reduce false positions and fast triage. At least Shao Jiangning said that Microsoft's internal ai r & D is based on this standard.

However, we still can't see from these test standards the specification of AI's own security, so AI is still full of various uncertainties. There are many problems to be solved, such as personal privacy protection in big data environment mentioned by Zhang Jian, director of the system and Service Certification Department of China Information Security Certification Center. Most countries' guidance documents for big data are just issued in the early stage in the direction of how to develop this technology, and personal privacy seems to be the second problem.

Maybe before a certain technology really develops, the focus of attention is only what it can achieve, not how many problems it has, security is one of them. Li Wei and Cheng Jiefeng both said in the interview that it is possible for the black industry to make use of artificial intelligence, but we have invested in research first, which is the advantage. Before the villains have made spears, we need to develop a shield with strong resistance. Whether AI can be so optimistic is unknown in the hands of attackers.

More trivia:

Tencent cloud announces the establishment of DDoS Protection Alliance

Fang Yong, technical director of Tencent cloud security, talks about the way to deal with public cloud DDoS challenges

Zhai Zhenming, a professor at the human computer interconnection Laboratory of Sun Yat sen University, came to the stage and said that he had saved about 700 bitcoins earlier, sold 300 of them, and caught up with Mt. GOx's bankruptcy. He lost 100 of them. 300 bitcoins were stolen on the same day

Women chairpersons of sub Forums

Intelligent hardware and Internet of things security branch, see snow seizing the control right of doorbell again

*Official freebuf report by Ouyang onion, reprinted without permission