What should public opinion analysis do?

Due to the huge amount of information on the Internet, it is difficult to deal with the collection and processing of Shanghai information on the Internet only by manual methods. It is necessary to strengthen the research on relevant information technologies, form an automatic analysis system for online public opinion, respond to online public opinion in time, and shift from passive blocking prevention to active combing and guidance. This system should have the following functions:

The first is the public opinion analysis engine. This is the core function of public opinion analysis system, including: 1, hot topics and sensitive topics identification, which can identify hot topics in a given time period according to the parameters such as the authority of news sources, the number of comments and the intensity of speech time. Use keyword control and semantic analysis to identify sensitive topics. 2. Tendency analysis: for each topic, analyze and count the opinions and tendencies of each sender's published articles. 3. Topic tracking, analyzing whether the topics of newly published articles and posts are the same as the existing topics. 4, automatic summary, for a variety of topics, various trends can form an automatic summary. 5, trend analysis, analysis of a theme in different time periods, people's attention. 6, emergency analysis, comprehensive analysis of emergencies across time and space, get the whole picture of the event, predict the development trend of the event. 7, alarm system, for emergencies involving content security, sensitive topics in a timely manner to find and alarm. 8. Statistical reports are generated according to the result base processed by the public opinion analysis engine. Users can browse through the browser to provide information retrieval function, query hot topics and trends according to specified conditions, browse the specific content of information and provide decision support.

The second is the automatic information collection function. The existing information collection technology mainly obtains page information automatically from the Internet through the link relationship between web pages, and expands to the whole network with the link. At present, some search engines use this technology to search web pages around the world. The public opinion monitoring system should be able to set subject objectives according to users' information needs, and complete the information collection task by combining manual participation with automatic information collection.

The third is the data cleaning function. Preprocessing the collected information, such as format conversion, data cleaning and data statistics. For news comments, it is necessary to filter out irrelevant information and save the title, source, release time, content, click volume, commentator, comment content and comment number. For BBS, you need to record the title, speaker, release time, content, reply content, reply quantity, etc. Post, and finally form formatted information. If conditions permit, it can be operated directly on the database of the server.

The core technology of public opinion analysis system lies in public opinion analysis engine, and the most important technologies involved include text classification, clustering, opinion orientation identification, topic detection and tracking, automatic summarization and other computer text information content identification technologies. These technologies have always been the focus of information workers at home and abroad. Text Retrieval Conference (TREC), Information Retrieval Professional Group Conference (SIGIR) and Text Detection and Tracking Conference (TDT) are the most important international conferences and forums to show the latest research results of this technology. Among them, the technology based on keyword statistical analysis method is relatively mature, but its effectiveness still has a lot of room for improvement.

In recent years, experts and scholars have been studying more effective methods, among which semantic-based content recognition method is the most important one. Although great progress has been made, there are still many problems and difficulties to be overcome. In order to improve the ability of network information control in China, we must face up to the difficulties and strengthen the research on related technologies.