Hacking Book | Free Online Hacking Learning


internet of things data analysis system

Posted by forbes at 2020-03-24

Timely article updates can focus on wechat company's "digital future" (zycnb1)

1. Cloud edge end architecture

In the architecture of Internet of things for the new generation of information infrastructure, data processing and data-based intelligent services become more and more important. In the first two years, a relatively hot word, called "edge computing", refers to putting the simple and real-time computing and analysis process closer to the terminal equipment to ensure the real-time data processing and reduce the risk of data transmission. Recently, a new buzzword called "cloud edge collaboration" has emerged, which is not very different from edge computing. It just emphasizes the architecture of "cloud edge end". The terminal is responsible for overall perception, while the edge is responsible for partial data analysis and reasoning. The cloud collects all edge perception data, business data and Internet data to complete industry and cross industry Situation awareness and analysis.

AI based intelligent service runs through the whole architecture of "cloud edge end". At the sensing terminal, AI technology aims to improve the sensitivity and accuracy of comprehensive perception, as well as the real-time nature of human-computer interaction and object-to-object interaction. At the same time, it can also carry out simple logic reasoning through the chip.

At the edge, AI technology is mainly responsible for collecting local data and relevant business data in the domain, completing the analysis and reasoning of perception data, and transmitting the relevant analysis results or models to the perception terminal to achieve the cooperation between the perception terminal and the edge cloud. At the same time, the edge cloud and the edge cloud can also share data, resources, algorithms, etc. through network sharing, Complete the coordination between edge clouds.

In the cloud, not only cloud computing related storage, computing, network and security resources similar to edge cloud need to be provided, but also all data need to be collected and integrated to provide intelligent services based on global data, including intelligent scheduling, operation and maintenance, macro decision-making, etc.

Cloud center is good at global, non real-time, long cycle big data processing and analysis, and can play an advantage in long cycle maintenance, business decision support and other fields. Edge computing is more suitable for local, real-time, short period data processing and analysis, and can better support the real-time intelligent decision-making and execution of local business. Edge computing and cloud center are complementary and collaborative. Edge cloud collaboration will enlarge the application value of edge computing and cloud computing: edge computing is not only close to the execution unit, but also the collection and preliminary processing unit of high-value data required by the cloud, which can better support the cloud application; on the contrary, cloud computing can distribute business rules or models optimized by big data analysis and processing to On the edge side, edge computing is based on new business rules or models.

In addition to edge cloud collaboration, cloud interconnection and cloud network integration have also become an important trend, which refers to the interconnection and sharing between cloud centers, including dynamic adjustment of service resources, rational allocation of computing resources and customized business interoperability.

2. Knowledge map in IOT

As mentioned above, AI runs through the whole architecture of "cloud edge end". As for AI and knowledge map in IOT, I think there are several points as follows:

The so-called semantic IOT is to establish a set of standard IOT semantics through semantic modeling, which has a standard specification for the attributes, States, actions and capabilities of objects, and can realize the description, analysis, registration, access and cognition of objects, so as to solve the problems of what objects are, where objects are and how objects are used.

The so-called map enabling is to build semantic network and capability map similar to knowledge map, to realize the fusion of multiple perceptual data and the standard integration of object capabilities, to achieve "object discovery" and integration of objects and capabilities on the basis of object description.

The establishment of knowledge rules and capability map constitutes a complete knowledge map model to complete the integration of perception data and business data. The rule engine with business knowledge as the core can complete the analysis and reasoning of simple logic in real time.

On the basis of rule engine, using the learning deepening thought of connectionism to learn and reason hidden knowledge, the existing rule engine is improved to achieve an automatic learning.

3. Data analysis of Internet of things

With regard to the common data, applications, models and models of the Internet of things data, this paper uses the description on the Internet to summarize:

From the new generation architecture, we describe several kinds of data, including Internet of things data, business data and Internet data. Here we focus on Internet of things data.

(1) Internet of things data

Static data and dynamic data

Single from the change of data, IOT data can be divided into static data and dynamic data. Static data are mostly label data, address data, RFID data are mostly static data, static data are mostly stored in structural and relational databases. Dynamic data is data in time series, and the characteristics of IOT dynamic data are that each data is one-to-one with time Should relation, and this kind of relation is particularly important in data processing, this kind of data storage usually uses the time series database way to store.

Static data will increase with the increase of sensors and the number of control equipment; dynamic data will not only increase with the number of devices and sensors, but also increase with the increase of time.

Whether static data or dynamic data, the growth of data in the Internet of things 1.0 stage is linear, not exponential, but because the Internet of things dynamic data is continuous, so the amount of data is also massive. Therefore, the pressure of data in IOT 1.0 is controllable, not as uncountable and uncontrollable as advertised.

Energy / asset attribute / diagnosis / signal

In terms of the original characteristics of data, we can divide IOT data into energy data, asset attribute data, diagnostic data and signal data.

Energy data: it refers to the relevant data that can be used to calculate energy consumption, such as current, voltage, power factor, frequency, harmonic, etc. Energy data is the most critical data type of the Internet of things. One of the ultimate purposes of the Internet of things is to save energy. So obtaining energy data, understanding energy data and analyzing energy data are the necessary functions in the implementation of the Internet of things. Energy collection equipment is also one of the important equipment of the Internet of things.

Asset attribute data: asset data usually refers to hardware asset data, such as equipment specifications, parameters and other attributes, equipment location information, subordination between equipment, etc. Asset data is mainly used for asset management. Asset management is a very important function of the industrial Internet of things and can even be studied as an independent system, because it can interface with almost all systems such as ERP system, MES system, logistics, etc.

Diagnostic data: diagnostic data refers to the data that detects the operation status of the equipment during the operation of the equipment. There are two types of diagnostic data: one is the operation parameters of the equipment, such as the input / output value of the equipment, which is usually the traditional industrial automation data, namely the OT technology related data; the other is the peripheral diagnostic data of the equipment, such as the surface temperature and noise of the equipment Voice, equipment vibration, etc., it is worth mentioning that peripheral diagnosis is the embodiment of Internet of things technology, which includes new sensor technology and Internet of things communication technology. Peripheral diagnostic data is an important metadata for predictive maintenance, and also provides a basis for depth control model. Therefore, diagnostic data is the data type we need to focus on.

Signal data: signal data or alarm data is the most popular data used in the industrial field at present, because it is intuitive, easy to understand and key, and at the same time, it is informed locally and remotely. Signal data is easy to be ignored, but it is one of the data needed by the Internet of things, which can be collected quickly and provide important reference value for the Internet of things system.

The relevance between data is the relationship between different data. The relationship between data has the most direct impact on the understanding of the operation of the whole system. Combing the correct relationship between data is the cornerstone of the effective operation of the system and generating value.

The correlation between data can be analyzed from the following aspects:

Time relevance: Data photography at the same time. Data is generated by the system at the same time. It reflects the state of the system at that time. From the perspective of the data world, the system is the data collection at that time. Data photographing reflects the static display of the system; time stamp is the key factor of this kind of data, so it is required that the time stamp of each data acquisition must be the same. Time stamp is the missing of many data at present, and it is also one of the problems that need to be paid attention to and solved in the implementation of the Internet of things.

Process relevance: that is, the data of one point affects the generation of the data of the second point after a certain period of time, which embodies the dynamic process display of the system. The process relationship between the data needs to be provided by the model and modified in the implementation.

The timeliness of data refers to the time from the generation of data to its removal. The timeliness of data is determined by the implementation and deployment of the system. Data can be used many times or can be cleared after being used once. Generally speaking, whether remote deployment data or edge deployment data affects the timeliness of data. Generally, the timeliness of edge deployment data is short and that of remote data is long. The data needed for edge deployment is usually timely, but the edge storage space and computing power are weak, so it can't be saved for a long time; the remote data is usually historical data display and computing analysis, and the cloud space and computing are highly scalable, so the data is time-effective.

The real-time of data is also a part of the timeliness of data. The real-time is related to the location of data deployment, the importance of data and the way of transmission.

(2) Application mode

After the equipment data is collected through the Internet of things, if the equipment data status exceeds the preset status, it will automatically alarm at the first time, and the administrator will carry out processing at the first time, and can issue commands through remote operation. Solve the problem in the bud.

Through the statistical method, the historical operation data of the equipment is analyzed. Different reports can be analyzed according to different dimensions. Then display it in the form of chart or large screen in front of the administrator. Administrators can quickly and intuitively understand the health of the entire IOT device.

This part needs to dig out valuable things from the data. For example, through continuous tracking and analysis of equipment data for a period of time and combined with the past equipment operation and maintenance experience of human beings, the failure probability of equipment and possible causes after failure are predicted through machine learning, and maintenance scheme is given. The example just mentioned is the tip of the iceberg in the advanced application of the Internet of things. By introducing the current hot AI technology. The Internet of things can become an intelligent Internet of things. Perhaps in the near future, people and equipment can talk freely, and equipment and equipment can also talk and make optimal decisions automatically.

(3) Analysis mode

Data analysis of the Internet of things can be divided into the following four categories:

In each category, there are two levels of analysis:

It can be seen that the data analysis of the Internet of things is based on the physical mechanism, that is, the understanding of professional knowledge, rather than the methods and capabilities of data analysis. Without sufficient physical mechanism and professional knowledge, blindly analyzing industrial data with some big data and artificial intelligence tools will be counterproductive

(4) Analytical model