Analysis: * Construction methods and difficulties of big data applications

* The industry has always been a frontier market for security technology applications. In the field of security, the current application of big data can be divided into the following three levels:

1. Statistical inquiry: This is the most basic application of big data. It is mainly oriented to the history and the status quo and answers what has already happened, such as the statistics of floating population in different regions, the statistics of actual vehicle ownership, and the distribution of various types of cases. trend.

2. Data mining: It is the core application mode of big data at present, and its key point is not to find cause and effect, but to discover the relationship between data. This relationship may be intuitively explained, or it may not immediately be able to find the underlying causes, but it may have a guiding significance for the work, such as the relationship between the seasonal climate and certain types of cases, the scope of vehicle activities, the habits of activities and the relationship between black cars. .

3. Forecasting and predicting: It is the future development direction of big data applications. Based on data statistics, analysis, and mining, an appropriate data model is established. Starting from the data relationship, derivation of causality can be achieved for a certain period of time. The trend is to make predictions, give early warnings of danger signals, and guide the direction of prevention work.

These three levels are specific to the actual business systems, including map detection, vehicle characteristics analysis systems, personnel characteristics analysis systems, video detection systems, and so on. These systems are based on video, pictures, and structured descriptions acquired by surveillance front-ends such as general video surveillance, vehicle/personnel bayonet, and smart IPC. Through intelligent analysis of big data platforms, such as map search, semantic search, and vehicles can be realized. / Personnel deployment, comparison of suspected cases, detailed analysis of features, etc. In-depth big data applications can help * solve cases quickly and scientifically.

* Big data applied to different types of police, due to the difference in the actual application requirements, the problems resolved are also different. For example, in the field of intelligent transportation, big data is currently mainly used for vehicle grooming. For example, based on statistics of traffic volumes at different roads and junctions (hourly, daily, and monthly statistics), these statistics can be used to analyze the traffic density of a road in real time at different times. Development direction and trends. These applications have now landed in many major cities. For example, everyone on the bus sees real-time images of the commuting peaks on mobile TV. This is based on technical analysis of big data.

* Construction methods and difficulties of big data applications

Take the vehicle analysis system as an example to introduce how to build applications on the Safe City Big Data platform:

1, the source and composition of data

Based on the big data platform vehicle analysis system, its data can be divided into static data and dynamic data. The static data mainly come from resource information databases such as vehicle management database, stolen robber library, dispatched vehicle library, and involved vehicle library. These data constitute the core database of vehicle data warehouse. The dynamic data mainly comes from the bayonet networking platform. Its data can be divided into structured bayonet traffic data and unstructured bayonet passing pictures. These data will continue to grow over time and constitute a vehicle data warehouse. The central repository. Images of vehicles captured or intercepted by video surveillance equipment from other devices such as bolts, domes, etc., come from vehicles outside the system and constitute the vehicle's data warehouse.

2, the storage of data

The static data for the core library is usually stored in a relational database. For the central bank's bayonet traffic data, it is stored in the column-oriented high-reliability and high-performance distributed database HBase. The real-time passing records, because of their large query volume and fast update rate, are placed in memory to optimize the throughput. , Reduce system I/O load. The vehicle image data of the peripheral library is stored in an ordinary storage space such as an IPSAN.

3, data structure and search query

For the massive unstructured data such as the clip-on image of the bayonet, in order to realize the data retrieval, it must be structured and incorporated into the library through smart analysis technology, and the license plate color, car body color, and car must be extracted from the bayonet picture. Structured information not available on the front end of traditional bayonet mounts, subdivisions, etc. is stored in HBase.

After structuring the data, the system-designed big data search engine can provide simple retrieval and multiple retrieval of multiple conditions. These conditions include time, location, logo, subdivision model library, etc.; and, based on the vehicle number plate. The fuzzy search, confusion search (such as "B" and "8", "V" and "U", "2" and "Z", etc.) lays the foundation for the subsequent application of the vehicle analysis system; in addition, through the core The docking of library data can automatically call out vehicle owner information, driver information, accident/illegal information, etc. in the query process.

4, data mining analysis and application

After the structuring of the data, combined with the GIS engine provided by the platform, we can easily carry out various types of statistics for the traffic, criminal investigation and other departments. These statistical reports include traffic statistics at road junctions, vehicle ownership statistics, average travel time statistics of road sections, traffic statistics of road network traffic, statistics of vehicle trip regulations, and so on.

Using the results of the bayonet picture structuring and docking with the vehicle's core library, the system can provide a set of pseudo-jacket applications that can reach the actual combat level. The application can be used for analysis of illegal activities such as counterfeit cards, decks, and rotation of license plates. At the same time, it can distinguish between deck vehicles and decked vehicles to more accurately combat illegal vehicles.

The use of bayonet traffic data mining analysis, combined with the wisdom of many years of work experience of the police officers, the system provides a series of bayonet tactics for users to use in different scenarios. These techniques include: trailing tracking analysis of vehicles, analysis of gang vehicles, analysis of night and day vehicles, regional vehicle analysis, regional vehicle frequency analysis, vehicle activity area analysis, route matching analysis, frequency change analysis, and more.

5, the data display

Using the display framework provided by the Ping An City Big Data Platform, the application capabilities of the vehicle analysis system can be integrated into the platform framework, form a complete set of safe city solutions with other systems, provide a unified access interface and interface, and can also be used as The function enhancement module of the existing bayonet networking application system is separately deployed and provides a separate access interface and interface.

Accuracy and applicability, * Market competition for big data applications

It is true that big data technologies such as face recognition and vehicle identification have already been applied in the security market. Through intelligent analysis of video and structured data extraction of effective information, users of video surveillance are really sued by others for security. The new era of automatic security is also a common pursuit of the security industry. However, the application of big data is far from mature, and accuracy and applicability will determine who will stand out in the near future.

The first is accuracy, taking face analysis techniques as an example. The so-called "accuracy rate" of face recognition refers to the results of comparison testing based on the world's most authoritative face database LFW. LFW is managed by the University of Massachusetts, Amherst, and it can be considered as a “question bank” to examine the face recognition capabilities of the deep learning system. It extracts 6,000 face photos under different orientations, expressions, and lighting conditions from the Internet as exam questions. , you can let any system "running" inside. The running process is as follows: LFW gives a set of photos asking if the system in the test is the same person. The system gives the answer of yes or no. The accuracy rate of 99% means that 99% of the questions were answered correctly by the face recognition system.

The key issue is that LFW and similar databases FDDB are just purely laboratory-level and academic testing tools. In the actual commercial scenario with a sample size of 100,000 or megabytes, a system with a high test score may not be able to If you have maintained your achievements, the misrecognition rate will go up straight and may not even work at all. In the test of some real complex scenes, the recognition rate of 98% of face recognition will linearly drop to about 70% under the false recognition rate of one in 100,000. Using intelligent analysis technologies such as face recognition to convert unstructured data into structured data is the basis for subsequent big data applications. Therefore, from the current level of smart analysis, the accuracy rate will still be a long time security industry. Common pursuit.

The second is applicability. The applicability mainly refers to the level of understanding of security vendors for users. It refers to all aspects of the design and implementation of application construction. Whether each function module is the real concern of the user or whether the system operation mode is really convenient for the user, etc. Such as, directly determines the user experience of the application system, which mainly depends on the security industry's industry, the actual project accumulation. Of course, neither accuracy nor applicability can change the direction in which big data applications become the future of business applications.