I. Project Source Code
Project source code URL: (This project is not open source at present.)
II. Introduction
The feature learning is to recognize and use features automatically. There are many successful cases with modern deep learning in the field of feature learning, which learn the description of features automatically in the unsupervised or semi-supervised way and in compressed form. The results are used to support advanced achievements in such fields as speech recognition, image classification, object recognition and others. The abstract feature expression can be acquired automatically, but only according to this, the feature results of learning cannot be understood and utilized, i.e. they can only be used in a way of black box. At the same time, how to create features similar to those with good results becomes more difficult.
Generally, the feature engineering goes through the following steps: data collection, data preprocessing, data modeling, feature design, feature selection, model evaluation, model iteration.
The model training in personalized system is mainly aimed at user’s past behavior, such as searching, browsing, click, adding cart and order. The appropriate features are selected to train model and predict user’s future behavior.
III. Data Collection
The user feature logs are collected by parameter server, including some records of basic features such as user behavior location, time, frequency, preference, etc. The time windows are usually multi-day and multi-month, ranging from 7 to 30 days, depending on the model requirements of different experimental recommendation positions.
IV. Data Preprocessing
4.1 Data Washing and Formatting
The data washing is to remove the noise in the data, such as some abnormal data, misreported data, missing data and so on. The data formatting is to organize the data into the input format needed by the model training framework and to facilitate observation.
4.2 Data Sampling
According to the event tracking, the click and order data of each recommendation position are sampled proportionally. The sampling proportion varies according to the model of the different experimental recommendation position. Generally speaking, the click, post-click order and direct order are labeled as positive samples, which are labeled as action = 0, and are subdivided into action = 1, 2, 3 and so on, while the non-click and non-browse are labeled as negative samples, which are labeled as action = 0. Meanwhile, the positive samples need different sampling strategies, for example, a part of the click samples can be used as positive samples, and another part can be used as negative samples, which depends on what strategy used in training. The sample is a tag with a series of features, and it is either labeled order or click.
4.3 Data Conversion
As mentioned above.
V. Model Training
The artificial feature construction and automatic training of feature extraction are both implemented.
5.1 Artificial Feature Construction
The features are mainly used as initialization features, especially for some less active users. For example, the statistics are made in time window of one week or one month, filtering user’s browsing, click, adding cart, order as features. As shown below, each dimension is cross-multiplied to get about tens of basic features.
| sku | cid3 | brand | productWord |
|---|---|---|---|
| sku-7day-browse-total | cid3-7day-browse-total | brand-7day-browse-total | pwd-7day-browse-total |
| sku-7day-browse-avg | cid3-7day-browse-avg | brand-7day-browse-avg | pwd-7day-browse-avg |
| sku-7day-browse-ratio-all-station | cid3-7day-browse-ratio-all_station | brand-7day-browse-ratio-all-station | pwd-7day-browse-ratio-all-station |
| sku-7day-browse-ratio-category | cid3-7day-browse-ratio-category | brand-7day-browse-ratio-category | pwd-7day-browse-ratio-category |
| sku-7day-browse-users-total | cid3-7day-browse-users-total | brand-7day-browse-users-total | pwd-7day-browse-users-total |
| sku-7day-browse-users-avg | cid3-7day-browse-users-avg | brand-7day-browse-users-avg | pwd-7day-browse-users-avg |
| sku-7day-browse-users-ratio-all-station | cid3-7day-browse-users-ratio-all-station | brand-7day-browse-users-ratio-all-station | pwd-7day-browse-users-ratio-all-station |
| sku-7day-browse-users-ratio-category | cid3-7day-browse-users-ratio-category | brand-7day-browse-users-ratio-category | pwd-7day-browse-users-ratio-category |
| sku-7day-sale-total | cid3-7day-sale-total | brand-7day-sale-total | pwd-7day-sale-total |
| sku-7day-sale-avg | cid3-7day-sale-avg | brand-7day-sale-avg | pwd-7day-sale-avg |
| sku-7day-sale-ratio-all-station | cid3-7day-sale-ratio-all-station | brand-7day-sale-ratio-all-station | pwd-7day-sale-ratio-all-station |
| sku-7day-sale-ratio-category | cid3-7day-sale-ratio-category | brand-7day-sale-ratio-category | pwd-7day-sale-ratio-category |
| sku-7day-order-total | cid3-7day-order-total | brand-7day-order-total | pwd-7day-order-total |
| sku-7day-order-avg | cid3-7day-order-avg | brand-7day-order-avg | pwd-7day-order-avg |
| sku-7day-order-ratio-all-station | cid3-7day-order-ratio-all-station | brand-7day-order-ratio-all-station | pwd-7day-order-ratio-all-station |
| sku-7day-order-ratio-category | cid3-7day-order-ratio-category | brand-7day-order-ratio-category | pwd-7day-order-ratio-category |
| sku-7day-conversion-rate-total | cid3-7day-conversion-rate-total | brand-7day-conversion-rate-total | pwd-7day-conversion-rate-total |
| sku-7day-conversion-rate-avg | cid3-7day-conversion-rate-avg | brand-7day-conversion-rate-avg | pwd-7day-conversion-rate-avg |
5.2 Automatic Training of Feature Extraction
For the samples of highly active users, there are many features. The models are trained by Xgboost and TensorFlow directly to extract features automatically.
VI. Model Evaluation
Feature Importance;
Hit rate of prediction (click@4/20, order@4/20);
Online evaluating indicators (CTR/CVR);
There is an example to illustrate the comparison of different models:
| model | Experiments | trainSet | testSet | weight | click/order@N |
|---|---|---|---|---|---|
| model_v1 | 101 | 7 days | 2 days | 50:5:1 | 0.23695 |
| model_v2 | 102 | 30 days | 2 days | 50:5:1 | 0.26951 |
| model_v3 | 103 | 30 days | 2 days | 50:10:1 | 0.25782 |
VII. Model Iteration
To adjust model iteratively, the model adjustment strategies of different experiment positions are also different. From the comparison among these above examples, it can be found that:
(1) The offline indicators can be improved by increasing the number of training sets.
(2) By adjusting the sampling proportion, the noise can be reduced, and it is better to sampling rigorously. The premise is that the training data sample is large enough.
(3) Weight settings, it is setting of training positive and negative samples. The weight should be adjusted to overall positive correlation with each action proportion in the samples, rather than simply increase or decrease the weight of click and order, and it should be in accordance with the sample proportion.
VIII. References
[1]…
[2]…
[3]…