How to build an enterprise big data application R&D system

1. Data basic platform

Basic data platform construction work, including data platform construction, data specifications, data warehouse, product data specifications, product ID, user ID, unified SDK, etc. .

The data of many companies cannot be effectively utilized because of the lack of unified standards. Product data reporting is left to developers according to their own understanding and habits. There is no standardized SDK and reporting protocol, and the data is scattered among the products of various departments. The server cannot build a structured data warehouse.

Many people will understand the architecture of a data platform as a high-level technical task. In fact, the realization of the value of the entire data platform requires the cooperation of all departments of the company. For example, the establishment of a key data indicator system requires the cooperation of various departments. Department business indicators are refined and recognized by the business department. Common key indicators include: DAU, PCU, WAU, MAU, daily retention rate (1-30 day retention), cumulative retention rate (7-day, 14-day, 30-day cumulative retention rate), new users, effective new additions Users, active conversion rate, paid conversion rate, revenue indicators, ARPU per capita revenue, channel performance data, etc.

The Internet is a magical network, and big data development and software customization are also models. The most detailed quotation is provided here. If you really want to do it, you can come here. The starting number for this mobile phone is one. The one in the middle of eight and seven is San Er Zero and the last one is One Four Two Five Zero. You can find it by combining them in order. What I want to say is, unless you want to do it or understand this aspect, if you are just joining in the fun, don’t come. .

2. Data reporting and visualization

In the first level, it is very convenient to standardize the data indicator system, unify definitions, and unify dimension distinctions. Standardized configurable data report design and intuitive visual output design, including multiple data categories such as behavior, revenue, performance, quality, etc.

In the PPT, the data report system of Umeng, Xunlei, Baidu, Tencent and other companies will be explained in detail.

?

3. Product and Operation Analysis

On the basis of establishing a data platform and visualization, conduct various analyzes on existing user behavior, income data, etc. Analyze and output daily, weekly, monthly and various special analysis reports. Common data analysis work is as follows:

1. A/B TEST for product analysis and optimization;

2. Use funnel model to conduct user reach analysis, such as TIPS, advertising, etc. Active conversion;

3. Revenue effect monitoring and analysis, including paid conversion rate, channel effect data, etc.;

4. Long-term business health analysis, such as user flow model, Product life cycle analysis of product growth and health;

5. Real-time feedback on marketing promotion activities;

User portraits are also a common data analysis method, including users such as gender, age, Behavior, income, interests and hobbies, consumption behavior, online behavior, channel preferences, behavioral preferences, life trajectory and location, etc., reflect various characteristics of users in order to achieve a comprehensive understanding of users and provide users with personalized services. Usually a thematic analysis of user portraits is done every six months.

Commonly used analysis tools: EXCLE, SPSS, SAS, Enterprise Miner, Clementine, STATISTICA. The ones I personally use more often are: EXCEL and SPSS.

4. Refined operation platform

A refined operation platform built on the basis of data. Most of the main platform logic is to segment users and products. And service segmentation, through the combination optimization of multiple recommendation algorithms, personalized recommendations of goods and services are made. In addition, there are product data operation systems built for different product life cycles and user life cycles.

5. Data products

There are many generalized data products, such as search, weather forecast, etc. Here we mainly talk about data products in a narrow sense, taking the data products of three BAT companies as examples to share.

Tencent: Guangdiantong, Xingge

Alibaba: Data Cube, Taobao Intelligence, Taobao Index, in the cloud

Baidu: Baidu Prediction, Baidu Statistics, Baidu Index, Baidu Sinan, Baidu Actuarial

6. Strategic analysis and decision-making

Strategic analysis and decision-making are more similar to many traditional strategic analyses. , the methodologies at the business analysis level are similar, the biggest difference is that the data comes from big data.

Many companies mistakenly place the tasks of the "business operation monitoring layer" and "user/customer experience optimization layer" in the business analysis or strategic analysis layer. Fu Zhihua believes that the "business operation monitoring layer" and "user/customer experience optimization layer" are more realized through machines, algorithms and data products, while "strategic analysis" and "operation analysis" are mostly realized by people. Many companies leave things that machines can do to people, which results in low efficiency in finding problems.

The suggestion is to use machines as much as possible to do the "business operation monitoring layer" and "user/customer experience optimization layer" for things that can be done with machines, and on this basis, let people do what humans are better at. Empirical analysis and strategic judgment.

In the rapidly changing Internet field, it is difficult to predict the general development direction of the business with data. If someone says that the general direction of WeChat is researched through data mining and analysis, it is estimated that Product managers will laugh. In essence, data can play a relatively good role in refined marketing and operations, but it plays a smaller role in creative matters such as product planning and advertising creativity. But once the product idea is created, it can pass grayscale testing and data to verify the effect.