Give a few examples of data applications from Meitu. As shown in Figure 1, the first one from the left is DataFace, Meitu's self-developed data visualization platform, which supports the business side to drag and drop freely to generate visual reports for efficient data reporting and subsequent analysis; the second one is the home page of Meitai APP, with popular personalized recommendation, based on the behavioral data used to recommend the list of videos that users may like and be interested in; the third one is based on the data of users' cheating, which can effectively judge and filter the cheating behavior of users according to certain models and strategies for anti-cheating. In addition to this, there are a wide range of applications including search, a/b experiments, channel tracking, advertising, etc.
Currently, Meitu has 500 million active users per month, and these users generate close to 20 billion behavioral data per day, so the overall volume is relatively large, with thousands of cluster machines, as well as PBs of total historical data.
With a relatively large number of business lines and the relatively extensive use of data in each line of business, as well as a relatively large overall user base, all of these factors have driven us to build corresponding data platforms to drive growth in these businesses and use data more efficiently.
/ Overall architecture of the Mito data platform /
The overall architecture of our data platform is shown in Figure 2. In this part of data collection, we build a collection server-side logging system Arachnia, which supports client-side SDK for each app integration and is responsible for collecting app client-side data; we also have data integration (import and export) based on DataX implementation; Mor crawler platform supports configurable task development for crawling public network data.