Deep Feature Synthesis

Deep Feature Synthesis (DFS) is an automated method for performing feature engineering on relational and transactional data.

Input Data

Deep Feature Synthesis requires structured datasets in order to perform feature engineering. To demonstrate the capabilities of DFS, we will use a mock customer transactions dataset.

Note

Before using DFS, it is recommended that you prepare your data as an EntitySet. See Representing Data with EntitySets to learn how.

In [1]: import featuretools as ft

In [2]: es = ft.demo.load_mock_customer(return_entityset=True)

In [3]: es
Out[3]: 
Entityset: transactions
  Entities:
    customers [Rows: 5, Columns: 3]
    sessions [Rows: 35, Columns: 4]
    products [Rows: 5, Columns: 2]
    transactions [Rows: 500, Columns: 5]
  Relationships:
    transactions.product_id -> products.product_id
    transactions.session_id -> sessions.session_id
    sessions.customer_id -> customers.customer_id

Once data is prepared as an EntitySet, we are ready to automatically generate features for a target entity - e.g. customers.

Running DFS

Typically, without automated feature engineering, a data scientist would write code to aggregate data for a customer, and apply different statistical functions resulting in features quantifying the customer’s behavior. In this example, an expert might be interested in features such as: total number of sessions or month the customer signed up.

These features can be generated by DFS when we specify the target_entity as customers and "count" and "month" as primitives.

In [4]: feature_matrix, feature_defs = ft.dfs(entityset=es,
   ...:                                       target_entity="customers",
   ...:                                       agg_primitives=["count"],
   ...:                                       trans_primitives=["month"],
   ...:                                       max_depth=1)
   ...: 

In [5]: feature_matrix
Out[5]: 
            zip_code  COUNT(sessions)  MONTH(join_date)
customer_id                                            
1              60091               10                 1
2              02139                8                 2
3              02139                5                 4
4              60091                8                 5
5              02139                4                 7

In the example above, "count" is an aggregation primitive because it computes a single value based on many sessions related to one customer. "month" is called a transform primitive because it takes one value for a customer transforms it to another.

Note

Feature primitives are a fundamental component to Featuretools. To learn more read Feature primitives.

Creating “Deep Features”

The name Deep Feature Synthesis comes from the algorithm’s ability to stack primitives to generate more complex features. Each time we stack a primitive we increase the “depth” of a feature. The max_depth parameter controls the maximum depth of the features returned by DFS. Let us try running DFS with max_depth=2

In [6]: feature_matrix, feature_defs = ft.dfs(entityset=es,
   ...:                                       target_entity="customers",
   ...:                                       agg_primitives=["mean", "sum", "mode"],
   ...:                                       trans_primitives=["month", "hour"],
   ...:                                       max_depth=2)
   ...: 

In [7]: feature_matrix
Out[7]: 
            zip_code MODE(sessions.device)  HOUR(join_date)  SUM(transactions.amount)  MEAN(transactions.amount)  MONTH(join_date)  MODE(transactions.product_id)  MODE(sessions.MONTH(session_start))  MODE(sessions.HOUR(session_start))  MODE(sessions.MODE(transactions.product_id))  SUM(sessions.MEAN(transactions.amount))  MEAN(sessions.SUM(transactions.amount))  MEAN(sessions.MEAN(transactions.amount))
customer_id                                                                                                                                                                                                                                                                                                                                                                                                         
1              60091               desktop                0                  10236.77                  78.143282                 1                              3                                    1                                   0                                             1                               791.976505                               1023.67700                                 79.197651
2              02139                mobile                0                   9118.81                  74.744344                 2                              1                                    1                                   1                                             1                               596.243506                               1139.85125                                 74.530438
3              02139               desktop                0                   5758.24                  73.823590                 4                              5                                    1                                   8                                             3                               369.770121                               1151.64800                                 73.954024
4              60091               desktop                0                   8205.28                  73.921441                 5                              4                                    1                                   3                                             1                               584.673126                               1025.66000                                 73.084141
5              02139                tablet                0                   4571.37                  78.816724                 7                              2                                    1                                   0                                             2                               313.448942                               1142.84250                                 78.362236

With a depth of 2, a number of features are generated using the supplied primitives. The algorithm to synthesize these definitions is described in this paper. In the returned feature matrix, let us understand one of the depth 2 features

In [8]: feature_matrix[['MEAN(sessions.SUM(transactions.amount))']]
Out[8]: 
             MEAN(sessions.SUM(transactions.amount))
customer_id                                         
1                                         1023.67700
2                                         1139.85125
3                                         1151.64800
4                                         1025.66000
5                                         1142.84250

For each customer this feature

  1. calculates the sum of all transaction amounts per session to get total amount per session,
  2. then applies the mean to the total amounts across multiple sessions to identify the average amount spent per session

We call this feature a “deep feature” with a depth of 2.

Let’s look at another depth 2 feature that calculates for every customer the most common hour of the day when they start a session

In [9]: feature_matrix[['MODE(sessions.HOUR(session_start))']]
Out[9]: 
             MODE(sessions.HOUR(session_start))
customer_id                                    
1                                             0
2                                             1
3                                             8
4                                             3
5                                             0

For each customer this feature calculates

  1. The hour of the day each of his or her sessions started, then
  2. uses the statistical function mode to identify the most common hour he or she started a session

Stacking results in features that are more expressive than individual primitives themselves. This enables the automatic creation of complex patterns for machine learning.

Changing Target Entity

DFS is powerful because we can create a feature matrix for any entity in our dataset. If we switch our target entity to “sessions”, we can synthesize features for each session instead of each customer. Now, we can use these features to predict the outcome of a session.

In [10]: feature_matrix, feature_defs = ft.dfs(entityset=es,
   ....:                                       target_entity="sessions",
   ....:                                       agg_primitives=["mean", "sum", "mode"],
   ....:                                       trans_primitives=["month", "hour"],
   ....:                                       max_depth=2)
   ....: 

In [11]: feature_matrix.head(5)
Out[11]: 
            customer_id   device  SUM(transactions.amount)  MONTH(session_start)  MODE(transactions.product_id)  MEAN(transactions.amount)  HOUR(session_start) customers.zip_code  MODE(transactions.HOUR(transaction_time))  customers.MEAN(transactions.amount) customers.MODE(sessions.device)  customers.SUM(transactions.amount)  customers.MODE(transactions.product_id)  MODE(transactions.MONTH(transaction_time))  customers.HOUR(join_date) MODE(transactions.products.brand)  customers.MONTH(join_date)
session_id                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
1                     1  desktop                   1245.54                     1                              2                  77.846250                    0              60091                                          0                            78.143282                         desktop                            10236.77                                        3                                           1                          0                                 C                           1
2                     1  desktop                    895.33                     1                              3                  89.533000                    0              60091                                          0                            78.143282                         desktop                            10236.77                                        3                                           1                          0                                 C                           1
3                     5   mobile                    939.82                     1                              5                  67.130000                    0              02139                                          0                            78.816724                          tablet                             4571.37                                        2                                           1                          0                                 C                           7
4                     3   mobile                   2054.32                     1                              1                  82.172800                    0              02139                                          0                            73.823590                         desktop                             5758.24                                        5                                           1                          0                                 C                           4
5                     2   tablet                    715.35                     1                              1                  65.031818                    1              02139                                          1                            74.744344                          mobile                             9118.81                                        1                                           1                          0                                 B                           2

As we can see, DFS will also build deep features based on a parent entity, in this case the customer of a particular session. For example, the feature below calculates the mean transaction amount of the customer of the session.

In [12]: feature_matrix[['customers.MEAN(transactions.amount)']].head(5)
Out[12]: 
            customers.MEAN(transactions.amount)
session_id                                     
1                                     78.143282
2                                     78.143282
3                                     78.816724
4                                     73.823590
5                                     74.744344

Improve feature output

To learn about the parameters to change in DFS read Tuning Deep Feature Synthesis.