[foreword]

Tencent launched the online vulnerability submission platform and security vulnerability reward plan of the security emergency response center (TSRC) in May 2012, and it has been nearly three years since its bumps. I have long wanted to write an article on the construction of SRC (Security Response Center) based on the experience and lessons of TSRC. Now I have time.

I remember that at the first TSRC salon in 2013, superhei and cnhawk had a live PK on the "rule of law" and "rule of man" issues of enterprise SRC (see the actual record of TSRC 4-19 salon). They were not addicted and then sent an article PK. Iceyes, sweeping monk and dragonegg all wrote articles on vulnerability reporting platform (see Appendix 1).

It's easy to know, but it's hard to do. I don't think all of you have any practical experience in building SRC, so I'm finally on the stage (things are changing, now iceyes works in ASRC).

[background]

    There are many business models and products involved in large-scale enterprises, especially the Internet business pays attention to "small steps, fast running, agile iteration", which often ignores the security inspection or fails to carry out detailed security inspection. At the same time, the security system itself is a program, and there are various omissions, because the online characteristics of the Internet business make the Internet accessible to all people, compared with Traditional businesses are more vulnerable to attacks, so they are more likely to be found by people who are friendly, malicious or unintentional. If the vulnerability is exploited or traded in the black market, it is extremely harmful to enterprises and users.

Since the vulnerability is inevitable to be found by the outside, how to attract the external security forces to participate in it?

The "responsible security vulnerability disclosure process" of the Internet working group (refer to the Chinese version translated by Han Haiyuan, and the similar approach proposed by Microsoft is called "collaborative vulnerability disclosure") is basically the consensus of the industry.

The specific methods can be roughly divided into Microsoft mode and Google mode.

IT enterprises represented by Microsoft mainly adopt the way of announcement and acknowledgment. As long as the vulnerability discoverer reports the vulnerability to Microsoft first, Microsoft will issue a security notice on the patch release day of each month to thank the vulnerability reporter. As Microsoft is a world-class enterprise, it can find its loopholes and get thanks. This honor makes the loophole reporter all over the world enjoy different things. Of course, the times are improving, and now Microsoft has a vulnerability incentive program.

Internet companies represented by Google have adopted cash incentives. As long as the bug finder reports the bug to the authorities without directly disclosing it, he will get a valuable cash reward. This mode has both announcement and money, which greatly mobilizes the enthusiasm of the reporter.

Tencent's security team has submitted loopholes for both models of enterprises. From the perspective of human nature, of course, the latter has a better effect.

So, what mode is Tencent prepared to adopt for vulnerability collection? Of course, it's the SRC with Tencent characteristics in line with China's national conditions.

A landmark proposal

All the friends in the circle know that before the TSRC era (before May 2012), they fed back the loopholes of Tencent's business to us. No matter how serious the loopholes were, they only presented QQ dolls to express their thanks. When they issued the security announcement, I looked up the historical records and there were only three announcements (tx07040201, tx07092701, tx09040901).

Although the feedback is totally unequal to the contribution of the vulnerability reporter, the whole industry was like this at that time, and hackers always have a kind of chivalrous spirit. It seems to be a chivalrous act to notify the manufacturer of the vulnerability, and the simple thanks of the manufacturer seem reasonable.

Until the end of March 2012, after the security emergency response team finished handling a serious security vulnerability reported externally, our CTO and senior brother Tony sent me an email: "this vulnerability is very serious, and the doll is not enough to express our thanks. I have a suggestion, please add my wechat chat.

So, after a discussion in wechat group, I, blazing angel, COOLC, LS and Tony witnessed the origin of Tencent's vulnerability reward plan.

[growing up through bumps]

Then there are a series of special approvals.

At the same time, technically, we have also begun to build. There is no development, no product, no design, no operation. We are holding these positions ourselves (know everything, life will be better). The system development of TSRC phase I and the docking with the internal security work order system are all completed by harite. Most of the products, web design, copywriting, processing flow, microblog, blog articles are done by harite and I.

The following two figures are the vulnerability report processing flow of TSRC. The review procedure is the function added by later optimization iteration.

The core of the comparison is that the TSRC vulnerability reporting system has opened the internal vulnerability work order system, once the identified vulnerability is automatically synchronized to the work order system to drive the business repair, and after the repair, it will automatically synchronize the status to the TSRC vulnerability reporting system to feed back to the reporter for review.

To be honest, at that time, I didn't have the bottom of my heart. After all, TSRC is the first enterprise in the industry to build a vulnerability collection platform. What if no one reports the vulnerability when the platform is built? What if there are countless loopholes at the same time? What should I do if I get hit by a competitor? In case of...

After a series of quick development, Tencent's vulnerability reporting platform invited some white hat internal tests on May 20 and officially launched on May 31.

It is gratifying that the project has achieved good results as soon as it goes online, and most white hats agree with the mode of the enterprise's self built vulnerability reporting platform. In June 2012, 38 white hats reported hundreds of loopholes, including some serious loopholes. Thank you very much for your support at that time. Without your support, TSRC will be hard to get to today. Thank!

In the next few months, the number of vulnerabilities also climbed, 176 in July, 280 in August It's also the number of vulnerabilities that are ultimately identified - and more reports that confirm that they are not vulnerabilities (several times as many). The vulnerability report is directly related to SMS and wechat. Every day, we hear several people's mobile phones ticking. There is a serious shortage of manpower. We have to deal with many loopholes every day except for other jobs. What's worse is that when sending gifts every month, the delivery list needs to be filled in by hand. There are about 50 of them every month, and the hand is almost useless (later, it realized one key direct connection to EMS, and no need to write the delivery list again).

Like all products, the completion of construction is just the beginning, and the key lies in operation. The 16 word motto of TSRC construction and operation is: small step, fast running, bold trial and error, communication as the king, serving people with virtue.

Because there are no rules in the early stage, there are often disputes about vulnerability rating. Change! So we soon released Tencent external report vulnerability handling process, which is constantly updated, and is still maintaining and updating.

Then it's because of the poor communication with the white hat, resulting in friction. Change! The comment function is added. The discovery is not enough. Change again! Added a key QQ connection function.

Then it is found that after some vulnerabilities are pushed to the business for repair, if they are not completely repaired, they will be discovered externally. Change! The process of "reporter confirms repair" is added.

       ……

It is in this iterative operation process that TSRC forms the current framework.

The figure below shows the number of vulnerabilities of Tencent and several large-scale friends on the cloud vulnerability reporting platform in recent years. It can be seen that Tencent ranked the "king of vulnerabilities" in 2012 and got rid of the "laurel crown" in 2013.

Take a look at Tencent's trend in the number of externally discovered vulnerabilities (mainly from TSRC and the black cloud vulnerability reporting platform).

We can see that after the launch of TSRC's vulnerability reward program, the number of reported vulnerabilities soared (which shocked us at one time). The more vulnerabilities SRC collects, the more that the enterprise's security system needs to be optimized). In 2013, the number reached its peak, but in 2014, the number of vulnerabilities finally converged. In recent years, the team has made a lot of efforts, and finally saw the effect.

In January 2013, Netease also launched a vulnerability reporting platform. Then jd.com, baidu.com, alibaba.com, etc. launched one after another, and Src mode was basically accepted by large Internet enterprises and carried out in full swing. This year can be called "SRC first year".

Now we can see that with the development of the security industry, traditional enterprises such as Suncom, State Grid, ZTE (ZTE specifically referred to TSRC, let's be a little proud of it), Lenovo and so on have carried out "vulnerability incentive plan" (see Appendix), and we believe that there will be more SRC enterprises in the future.

[thinking: communication is the king, people-oriented]

The biggest difference between the reward model of loopholes and the chivalrous model in the past is that the reporter of loopholes is driven by interests (this is not wrong, Wang Yangming's mind has long said that "natural principle is human desire"), and the loopholes score will directly affect the income, so the loopholes score should be very careful, otherwise it will cause controversy.

In the early stage, I analyzed all the problems that TSRC caused disputes and found that more than 80% of them were communication problems with the reporter. In other words, as long as the communication is proper, these disputes will not appear.

It is normal for the manufacturer and the vulnerability reporter to have different ratings for the same vulnerability, so we should fully communicate at this time. Therefore, TSRC added the comment function in vulnerability handling, but later found that it was still a problem to communicate too slowly through comments, so it added the one click QQ connection function. For those who don't use instant messaging, find a way to get a phone number.

If the reporter disagrees with the handling process, he / she can bypass the current vulnerability handling personnel's one key QQ to contact the chief operating officer of TSRC for communication; if he / she still disagrees, he / she can contact the chief executive officer (haha, it is the CEO of TSRC); if he / she still cannot reach a consensus, he / she can invite the industry bulls recognized by both parties to make a judgment.

So, I think communication is the top priority of SRC's operation process - the contradiction with people has been solved, and others are easy to say.

The core of vulnerability reporting platform is the above vulnerability reporter. No one reports the vulnerability to you. What's the significance of your platform? So platform stickiness is very important.

How to improve the platform viscosity? On the one hand, you should make the reporter happy to come to your platform. In fact, it can be seen as a community product in a vertical subdivision field, and this product operation mode can be explored more.

In the early days, we launched a virtual Penguin medal to reward the loophole reporter who made outstanding contributions. At that time, we boasted that we would also like to substantialize it. Now we have made a real Penguin medal. The figure below is one of the designs.

[thinking: not just vulnerability collection platform]

After the establishment of SRC of some enterprises, I saw that SRC of some enterprises only collects external vulnerabilities (or even a device), which I think is not enough (know why my title is called the construction of emergency response center).

SRC itself is a window of the enterprise, which conveys the enterprise's attitude towards security. Under this framework, SRC should undertake more work.

A very direct function of SRC is the touchstone of internal security system. Enterprises with a little emphasis on security must have their own security teams and systems, so why are vulnerabilities still found outside? It must be caused by the omission of relevant teams and systems.

The next step is to analyze and improve. Every vulnerability of TSRC will be recovered to find and fix the corresponding team and system omissions. At the same time, the vulnerability reporter helps Tencent discover the vulnerability and optimizes Tencent's security system.

The figure below shows the problems and optimization scheme of Tencent vulnerability scanner system found after the web vulnerability recovery, as well as the number of optimization points obtained from TSRC vulnerabilities over the years (this data trend is basically consistent with the trend of external vulnerability data).

In addition, the vulnerability reporter on the platform can test our security system from a relatively third-party perspective. For example, before the WAF researched by Tencent went online, the vulnerability reporter made a special bypass test, and found some problems. The effect was very good. This article "the real record of WAF SQL injection bypassing challenges" is a summary of WAF bypassing challenges at that time - such activities not only increase user stickiness, but also optimize the system.

Now the safety industry has begun to receive attention. Senior safety talents are in urgent need. SRC is an excellent recruitment channel. Some of our team's interns, fresh graduates and social recruiters have been excellent vulnerability reporters on TSRC, and flyh4t, the current head of TSRC, has also been the vulnerability reporter of TSRC.

SRC will also be responsible for the output of the enterprise's security influence, and TSRC will also share some Tencent's experience and tools in security construction on the official website. But at present, it's not enough (some people in the group say Tencent doesn't share security, but it's gratifying that another non Tencent person immediately says Tencent has shares. Thank you for your support. There will be more in the future. In the follow-up, flyh4t also plans to refine the ideas in the high-quality vulnerability report and send them to the blog for discussion and common improvement.

In addition to its own corporate vulnerabilities, SRC can do more. Taking OpenSSL heart bleeding and Bash shell breaking vulnerability as typical cases, the security vulnerabilities of some basic components have a great impact on Internet enterprises, so TSRC has also launched the "the security hero project" to expand the scope of vulnerability awards to basic components, hoping to help Internet security.

[thinking: ranking system vs. exchange system]

In the early months, considering that there is a risk of insufficient funds if awards are given according to the number of loopholes, we adopted the "ranking system", that is, ranking the monthly white hat points and giving gifts (iPad, mobile phone, QQ doll and other physical objects) according to the level. The following figure shows the rules for getting gifts under the ranking system in October 2012:

After a few months, the drawbacks of the ranking system became apparent: the reporter could not get what he liked. This is a serious blow to the initiative of vulnerability reporter. That can't be done. It has to be changed!

So the "exchange system" was born. That is to say, the reporter can get the corresponding amount of gold coins through the score of the loophole, which can be used by the reporter to exchange items freely in the integral mall. In order to prevent bankruptcy, we use simple economic means to control the "inflation" of gifts by using the exchange coefficient of gold coin and RMB. However, after a period of time, it was found that the funds were enough. At the same time, the white hat generally reflected that it was more difficult to dig Tencent loopholes, so we gradually increased the coefficient.

This is to let the market decide the value of the loopholes. If fewer vulnerabilities are more difficult to find, the reward for a single vulnerability is more expensive. In an internal discussion, we joked that Tencent's reflective XSS vulnerabilities will be rewarded $500 in the future, which means Tencent has done a good job in security. Many friends make complaints about Tencent's award less than Google and Facebook. My answer is still the market to determine the value of the vulnerability. At this stage, there is no time when a Tencent leak is 500 dollars.

Before, we were worried about the negative impact of direct reward cash on the industry, but we saw that other platforms also use cash, which was accepted by the industry. Later, TSRC also had direct cash reward, and this year, it also added the function of direct exchange of gold coins for cash.

[thinking: dangerous in the Jianghu, beware of speculation]

Since PR has been involved in the security industry, ordinary security issues may be hyped by competitors. SRC, as the official team dealing with security issues directly, must be careful.

A friend of mine has been able to make complaints about Tencent's official reply to safety. We have confirmed that we are communicating with the business department to develop solutions to this problem. If there is any new progress, we will synchronize in time. This reply is the official standard reply caliber for internal discussion and improvement of multiple versions of security issues, but it is not because of perfunctory, but to avoid being hyped.

Earlier, when our staff replied to the security breach on the black cloud vulnerability reporting platform, they were hyped by their competitors because of their arbitrary scripts.

In 2013, TSRC issued an annual report, which announced the loopholes and rewards handled by TSRC in the previous year. Then, in the evening of that day, there was a soft article that used the data in the report to blame Tencent for more loopholes and less rewards.

TSRC email once received an email from a company saying that it would cooperate. TSRC replied that it was very welcome, but it was not our responsibility, so we transferred it to a certain department. As a result, the next day there was a soft article accusing Tencent of repeatedly urging and prevaricating on security issues.

There are several similar lessons. In a word, the SRC of an enterprise must be careful in its external language and avoid being used by competitors.

[thinking: cloud SRC]

We can think of SRC as a security capability or product. An SRC service provider provides SRC systems for enterprises without SRC resources. This is cloud Src. It's the same as our ordinary users who don't have the resources to build their own personal blog and use Sina blog.

Cloud SRC is to provide free vulnerability collection services and platforms for enterprises, and the profit point is to provide security value-added services for enterprises on the platform (such as vulnerability repair scheme consultation, security reinforcement and even security intelligence, and specific value-added services can be found again).

I understand that cloud SRC is a free private service for vulnerability information provided to all manufacturers in need. Therefore, neither the traditional third-party vulnerability reporting platform (dark cloud or sky mending) nor the emerging security crowd testing platform (dark cloud crowd testing, vulnerability box, sobug, and wechat crowd testing) can be regarded as cloud Src. The manufacturer resident mode of some crowd testing platforms is a little bit like that.

With the attention paid to information security, all kinds of enterprises in the future, especially the traditional enterprises trying to enter the Internet business, must have their own SRC, but most enterprises may not have the resources to do it by themselves, so I think the cloud SRC service is worth trying.

[thinking: xsrc alliance]

Xsrc refers to all SRC, where "X" is a wildcard, representing one or more characters, equivalent to "*" in the regular. Remember when the first few SRC came out, someone joked that there were too many xsrc and 26 letters were not enough, so we had to expand the number of X.

The so-called xsrc alliance is a win-win alliance in which the SRC of an enterprise has a common need to exchange resources and cooperate.

There are many things to cooperate here: unified account system, vulnerability scoring standard and experience sharing related to vulnerability collection platform; security intelligence sharing in the industry (somewhat similar to MAPP); security problem communication between enterprises (vulnerability notification / botnet notification / attack coordination); security system sharing

Of course, the security teams of various companies have cooperated, so most of the above ideas are a little difficult to implement.

[postscript]

With the development of information technology to the era of the Internet of things, information security is closely related to our lives. With the increasing attention of enterprises to security, there may be SRC needs. This is a trend of era evolution, so how should SRC go in the future? This article will discuss with colleagues in the industry.

[Appendix 1: several articles on SRC construction]

Superhei, some suggestions for the platform construction of Party A, such as TSRC, http://hi.baidu.com/hi'heige/item/937357b12bed3a1ebba9316

Cnhawk, on the construction of enterprise vulnerability collection platform, http://www.freebuf.com/articles/others-articles/8963.html

Superhei, on the construction of enterprise vulnerability collection platform, http://hi.baidu.com/hi_heige/item/b619f44cc111996af61d7b9dd

Cnhawk, three discussion on the construction of enterprise vulnerability collection platform, http://www.freebuf.com/articles/others-articles/9059.html

Iceyes, about vulnerability reporting platform, http://www.freebuf.com/news/special/16302.html

Sweeping monks, make complaints about the big companies in the Tucao platform, http://www.freebuf.com/articles/others-articles/10813.html

DragonEgg, who is the most awesome? Reward analysis of major domestic emergency response centers, http://www.freebuf.com/articles/others-articles/13953.html

[Appendix 2: SRC address of domestic enterprises]

Tencent, TSRC, http://security.tencent.com

Baidu, BSRC, http://sec.baidu.com

Alibaba, ASRC, http://security.alibaba.com

360，360SRC，http://security.360.cn

Netease, NSC, http://aq.163.com

Store 1, 1src, http://security.yhd.com

JD, jsrc, http://security.jd.com

Sina, SSRC, http://sec.sina.com.cn

Kingsoft, ksrc, http://sec.kingsoft.com

Xunlei, xsrc, http://safe.xunlei.com

Ctrip, CSRC, http://sec.ctrip.com

Where to go, qsrc, http://security.qunar.com

Sogou, sgsrc, http://sec.sogou.com

Xiaomi, MISRC, https://sec.xiaomi.com

Express, qsrc, http://security.kuaibo.com

Meizusrc, http://sec.meizu.com

Suntrust, Suntrust Security Response Center, http://security.sangfor.com.cn

State Grid, information system security laboratory vulnerability submission platform, http://124.205.52.38

ZTE, ZTE psirt, http://www.zte.com.cn/cn/about/corporate/citizenship/security/201405/t2014530.html

Lenovo, lsrc, http://lsrc.lenovo.com

[aggregation of domestic vulnerability incentive program] http://security.tencent.com/index.php/xsrc