Exporting Feature Matrix

In this example, we’re working with a mock customer behavior dataset

In [1]: import featuretools as ft

In [2]: es = ft.demo.load_mock_customer(return_entityset=True)

In [3]: es
Out[3]: 
Entityset: transactions
  Entities:
    transactions [Rows: 500, Columns: 5]
    products [Rows: 5, Columns: 2]
    sessions [Rows: 35, Columns: 4]
    customers [Rows: 5, Columns: 4]
  Relationships:
    transactions.product_id -> products.product_id
    transactions.session_id -> sessions.session_id
    sessions.customer_id -> customers.customer_id

Run Deep Feature Synthesis

A minimal input to DFS is a set of entities and a list of relationships and the “target_entity” to calculate features for. The output of DFS is a feature matrix and the corresponding list of feature definitions

In [4]: feature_matrix, features = ft.dfs(entityset=es,
   ...:                                   target_entity="customers",
   ...:                                   verbose=True)
   ...: 
Built 73 features

Elapsed: 00:00 | Remaining: ? | Progress:   0%|          | Calculated: 0/1 chunks
Elapsed: 00:00 | Remaining: 00:00 | Progress: 100%|##########| Calculated: 1/1 chunks

In [5]: feature_matrix
Out[5]: 
            zip_code  COUNT(sessions)  NUM_UNIQUE(sessions.device) MODE(sessions.device)  SUM(transactions.amount)  STD(transactions.amount)  MAX(transactions.amount)  SKEW(transactions.amount)  MIN(transactions.amount)  MEAN(transactions.amount)  COUNT(transactions)  NUM_UNIQUE(transactions.product_id)  MODE(transactions.product_id)  DAY(join_date)  DAY(date_of_birth)  YEAR(join_date)  YEAR(date_of_birth)  MONTH(join_date)  MONTH(date_of_birth)  WEEKDAY(join_date)  WEEKDAY(date_of_birth)  SUM(sessions.STD(transactions.amount))  SUM(sessions.MAX(transactions.amount))  SUM(sessions.SKEW(transactions.amount))  SUM(sessions.MIN(transactions.amount))  SUM(sessions.MEAN(transactions.amount))  SUM(sessions.NUM_UNIQUE(transactions.product_id))  STD(sessions.SUM(transactions.amount))  STD(sessions.MAX(transactions.amount))  STD(sessions.SKEW(transactions.amount))  STD(sessions.MIN(transactions.amount))  STD(sessions.MEAN(transactions.amount))  STD(sessions.COUNT(transactions))  STD(sessions.NUM_UNIQUE(transactions.product_id))  MAX(sessions.SUM(transactions.amount))  MAX(sessions.STD(transactions.amount))  MAX(sessions.SKEW(transactions.amount))  MAX(sessions.MIN(transactions.amount))  MAX(sessions.MEAN(transactions.amount))  MAX(sessions.COUNT(transactions))  MAX(sessions.NUM_UNIQUE(transactions.product_id))  SKEW(sessions.SUM(transactions.amount))  SKEW(sessions.STD(transactions.amount))  SKEW(sessions.MAX(transactions.amount))  SKEW(sessions.MIN(transactions.amount))  SKEW(sessions.MEAN(transactions.amount))  SKEW(sessions.COUNT(transactions))  SKEW(sessions.NUM_UNIQUE(transactions.product_id))  MIN(sessions.SUM(transactions.amount))  MIN(sessions.STD(transactions.amount))  MIN(sessions.MAX(transactions.amount))  MIN(sessions.SKEW(transactions.amount))  MIN(sessions.MEAN(transactions.amount))  MIN(sessions.COUNT(transactions))  MIN(sessions.NUM_UNIQUE(transactions.product_id))  MEAN(sessions.SUM(transactions.amount))  MEAN(sessions.STD(transactions.amount))  MEAN(sessions.MAX(transactions.amount))  MEAN(sessions.SKEW(transactions.amount))  MEAN(sessions.MIN(transactions.amount))  MEAN(sessions.MEAN(transactions.amount))  MEAN(sessions.COUNT(transactions))  MEAN(sessions.NUM_UNIQUE(transactions.product_id))  NUM_UNIQUE(sessions.MODE(transactions.product_id))  NUM_UNIQUE(sessions.DAY(session_start))  NUM_UNIQUE(sessions.YEAR(session_start))  NUM_UNIQUE(sessions.MONTH(session_start))  NUM_UNIQUE(sessions.WEEKDAY(session_start))  MODE(sessions.MODE(transactions.product_id))  MODE(sessions.DAY(session_start))  MODE(sessions.YEAR(session_start))  MODE(sessions.MONTH(session_start))  MODE(sessions.WEEKDAY(session_start))
customer_id                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
1              60091                8                            3                mobile                   9025.62                 40.442059                    139.43                   0.019698                      5.81                  71.631905                  126                                    5                              4              17                  18             2011                 1994                 4                     7                   6                       0                              312.745952                                 1057.97                                -0.476122                                   78.59                               582.193117                                                 40                              279.510713                                7.322191                                 0.589386                                6.954507                                13.759314                           4.062019                                           0.000000                                 1613.93                               46.905665                                 0.640252                                   26.36                                88.755625                                 25                                                  5                                 0.778170                                -0.312355                                -0.780493                                 2.440005                                 -0.424949                            1.946018                                           0.000000                                   809.97                               30.450261                                  118.90                                -1.038434                                50.623125                                 12                                                  5                              1128.202500                                39.093244                               132.246250                                 -0.059515                                 9.823750                                 72.774140                           15.750000                                           5.000000                                                   4                                         1                                         1                                          1                                            1                                             4                                  1                                2014                                    1                                      2
2              13244                7                            3               desktop                   7200.28                 37.705178                    146.81                   0.098259                      8.73                  77.422366                   93                                    5                              4              15                  18             2012                 1986                 4                     8                   6                       0                              258.700528                                  931.63                                -0.277640                                  154.60                               548.905851                                                 35                              251.609234                               17.221593                                 0.509798                               15.874374                                11.477071                           3.450328                                           0.000000                                 1320.64                               47.935920                                 0.755711                                   56.46                                96.581000                                 18                                                  5                                -0.440929                                 0.013087                                -1.539467                                 2.154929                                  0.235296                           -0.303276                                           0.000000                                   634.84                               27.839228                                  100.04                                -0.763603                                61.910000                                  8                                                  5                              1028.611429                                36.957218                               133.090000                                 -0.039663                                22.085714                                 78.415122                           13.285714                                           5.000000                                                   4                                         1                                         1                                          1                                            1                                             3                                  1                                2014                                    1                                      2
3              13244                6                            3               desktop                   6236.62                 43.683296                    149.15                   0.418230                      5.89                  67.060430                   93                                    5                              1              13                  21             2011                 2003                 8                    11                   5                       4                              257.299895                                  847.63                                 2.286086                                   66.21                               405.237462                                                 29                              219.021420                               10.724241                                 0.429374                                5.424407                                11.174282                           2.428992                                           0.408248                                 1477.97                               50.110120                                 0.854976                                   20.06                                82.109444                                 18                                                  5                                 2.246479                                -0.245703                                -0.941078                                 1.000771                                  0.678544                           -1.507217                                          -2.449490                                   889.21                               35.704680                                  126.74                                -0.289466                                55.579412                                 11                                                  4                              1039.436667                                42.883316                               141.271667                                  0.381014                                11.035000                                 67.539577                           15.500000                                           4.833333                                                   4                                         1                                         1                                          1                                            1                                             1                                  1                                2014                                    1                                      2
4              60091                8                            3                mobile                   8727.68                 45.068765                    149.95                  -0.036348                      5.73                  80.070459                  109                                    5                              2               8                  15             2011                 2006                 4                     8                   4                       1                              356.125829                                 1157.99                                 0.002764                                  131.51                               649.657515                                                 37                              235.992478                                3.514421                                 0.387884                               16.960575                                13.027258                           3.335416                                           0.517549                                 1351.46                               54.293903                                 0.382868                                   54.83                               110.450000                                 18                                                  5                                -0.391805                                -1.065663                                 0.027256                                 2.103510                                  1.980948                            0.282488                                          -0.644061                                   771.68                               29.026424                                  139.20                                -0.711744                                70.638182                                 10                                                  4                              1090.960000                                44.515729                               144.748750                                  0.000346                                16.438750                                 81.207189                           13.625000                                           4.625000                                                   5                                         1                                         1                                          1                                            1                                             1                                  1                                2014                                    1                                      2
5              60091                6                            3                mobile                   6349.66                 44.095630                    149.02                  -0.025941                      7.55                  80.375443                   79                                    5                              5              17                  28             2010                 1984                 7                     7                   5                       5                              259.873954                                  839.76                                 0.014384                                   86.49                               472.231119                                                 30                              402.775486                                7.928001                                 0.415426                                4.961414                                11.007471                           3.600926                                           0.000000                                 1700.67                               51.149250                                 0.602209                                   20.65                                94.481667                                 18                                                  5                                 0.472342                                 0.204548                                -0.333796                                -0.470410                                  0.335175                           -0.317685                                           0.000000                                   543.18                               36.734681                                  128.51                                -0.539060                                66.666667                                  8                                                  5                              1058.276667                                43.312326                               139.960000                                  0.002397                                14.415000                                 78.705187                           13.166667                                           5.000000                                                   5                                         1                                         1                                          1                                            1                                             3                                  1                                2014                                    1                                      2

Save as csv

The feature matrix is a pandas dataframe that we can save to disk

In [6]: feature_matrix.to_csv("feature_matrix.csv")

We can also read it back in as follows:

In [7]: saved_fm = pd.read_csv("feature_matrix.csv", index_col="customer_id")

In [8]: saved_fm
Out[8]: 
             zip_code  COUNT(sessions)  NUM_UNIQUE(sessions.device) MODE(sessions.device)  SUM(transactions.amount)  STD(transactions.amount)  MAX(transactions.amount)  SKEW(transactions.amount)  MIN(transactions.amount)  MEAN(transactions.amount)  COUNT(transactions)  NUM_UNIQUE(transactions.product_id)  MODE(transactions.product_id)  DAY(join_date)  DAY(date_of_birth)  YEAR(join_date)  YEAR(date_of_birth)  MONTH(join_date)  MONTH(date_of_birth)  WEEKDAY(join_date)  WEEKDAY(date_of_birth)  SUM(sessions.STD(transactions.amount))  SUM(sessions.MAX(transactions.amount))  SUM(sessions.SKEW(transactions.amount))  SUM(sessions.MIN(transactions.amount))  SUM(sessions.MEAN(transactions.amount))  SUM(sessions.NUM_UNIQUE(transactions.product_id))  STD(sessions.SUM(transactions.amount))  STD(sessions.MAX(transactions.amount))  STD(sessions.SKEW(transactions.amount))  STD(sessions.MIN(transactions.amount))  STD(sessions.MEAN(transactions.amount))  STD(sessions.COUNT(transactions))  STD(sessions.NUM_UNIQUE(transactions.product_id))  MAX(sessions.SUM(transactions.amount))  MAX(sessions.STD(transactions.amount))  MAX(sessions.SKEW(transactions.amount))  MAX(sessions.MIN(transactions.amount))  MAX(sessions.MEAN(transactions.amount))  MAX(sessions.COUNT(transactions))  MAX(sessions.NUM_UNIQUE(transactions.product_id))  SKEW(sessions.SUM(transactions.amount))  SKEW(sessions.STD(transactions.amount))  SKEW(sessions.MAX(transactions.amount))  SKEW(sessions.MIN(transactions.amount))  SKEW(sessions.MEAN(transactions.amount))  SKEW(sessions.COUNT(transactions))  SKEW(sessions.NUM_UNIQUE(transactions.product_id))  MIN(sessions.SUM(transactions.amount))  MIN(sessions.STD(transactions.amount))  MIN(sessions.MAX(transactions.amount))  MIN(sessions.SKEW(transactions.amount))  MIN(sessions.MEAN(transactions.amount))  MIN(sessions.COUNT(transactions))  MIN(sessions.NUM_UNIQUE(transactions.product_id))  MEAN(sessions.SUM(transactions.amount))  MEAN(sessions.STD(transactions.amount))  MEAN(sessions.MAX(transactions.amount))  MEAN(sessions.SKEW(transactions.amount))  MEAN(sessions.MIN(transactions.amount))  MEAN(sessions.MEAN(transactions.amount))  MEAN(sessions.COUNT(transactions))  MEAN(sessions.NUM_UNIQUE(transactions.product_id))  NUM_UNIQUE(sessions.MODE(transactions.product_id))  NUM_UNIQUE(sessions.DAY(session_start))  NUM_UNIQUE(sessions.YEAR(session_start))  NUM_UNIQUE(sessions.MONTH(session_start))  NUM_UNIQUE(sessions.WEEKDAY(session_start))  MODE(sessions.MODE(transactions.product_id))  MODE(sessions.DAY(session_start))  MODE(sessions.YEAR(session_start))  MODE(sessions.MONTH(session_start))  MODE(sessions.WEEKDAY(session_start))
customer_id                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      
1               60091                8                            3                mobile                   9025.62                 40.442059                    139.43                   0.019698                      5.81                  71.631905                  126                                    5                              4              17                  18             2011                 1994                 4                     7                   6                       0                              312.745952                                 1057.97                                -0.476122                                   78.59                               582.193117                                                 40                              279.510713                                7.322191                                 0.589386                                6.954507                                13.759314                           4.062019                                           0.000000                                 1613.93                               46.905665                                 0.640252                                   26.36                                88.755625                                 25                                                  5                                 0.778170                                -0.312355                                -0.780493                                 2.440005                                 -0.424949                            1.946018                                           0.000000                                   809.97                               30.450261                                  118.90                                -1.038434                                50.623125                                 12                                                  5                              1128.202500                                39.093244                               132.246250                                 -0.059515                                 9.823750                                 72.774140                           15.750000                                           5.000000                                                   4                                         1                                         1                                          1                                            1                                             4                                  1                                2014                                    1                                      2
2               13244                7                            3               desktop                   7200.28                 37.705178                    146.81                   0.098259                      8.73                  77.422366                   93                                    5                              4              15                  18             2012                 1986                 4                     8                   6                       0                              258.700528                                  931.63                                -0.277640                                  154.60                               548.905851                                                 35                              251.609234                               17.221593                                 0.509798                               15.874374                                11.477071                           3.450328                                           0.000000                                 1320.64                               47.935920                                 0.755711                                   56.46                                96.581000                                 18                                                  5                                -0.440929                                 0.013087                                -1.539467                                 2.154929                                  0.235296                           -0.303276                                           0.000000                                   634.84                               27.839228                                  100.04                                -0.763603                                61.910000                                  8                                                  5                              1028.611429                                36.957218                               133.090000                                 -0.039663                                22.085714                                 78.415122                           13.285714                                           5.000000                                                   4                                         1                                         1                                          1                                            1                                             3                                  1                                2014                                    1                                      2
3               13244                6                            3               desktop                   6236.62                 43.683296                    149.15                   0.418230                      5.89                  67.060430                   93                                    5                              1              13                  21             2011                 2003                 8                    11                   5                       4                              257.299895                                  847.63                                 2.286086                                   66.21                               405.237462                                                 29                              219.021420                               10.724241                                 0.429374                                5.424407                                11.174282                           2.428992                                           0.408248                                 1477.97                               50.110120                                 0.854976                                   20.06                                82.109444                                 18                                                  5                                 2.246479                                -0.245703                                -0.941078                                 1.000771                                  0.678544                           -1.507217                                          -2.449490                                   889.21                               35.704680                                  126.74                                -0.289466                                55.579412                                 11                                                  4                              1039.436667                                42.883316                               141.271667                                  0.381014                                11.035000                                 67.539577                           15.500000                                           4.833333                                                   4                                         1                                         1                                          1                                            1                                             1                                  1                                2014                                    1                                      2
4               60091                8                            3                mobile                   8727.68                 45.068765                    149.95                  -0.036348                      5.73                  80.070459                  109                                    5                              2               8                  15             2011                 2006                 4                     8                   4                       1                              356.125829                                 1157.99                                 0.002764                                  131.51                               649.657515                                                 37                              235.992478                                3.514421                                 0.387884                               16.960575                                13.027258                           3.335416                                           0.517549                                 1351.46                               54.293903                                 0.382868                                   54.83                               110.450000                                 18                                                  5                                -0.391805                                -1.065663                                 0.027256                                 2.103510                                  1.980948                            0.282488                                          -0.644061                                   771.68                               29.026424                                  139.20                                -0.711744                                70.638182                                 10                                                  4                              1090.960000                                44.515729                               144.748750                                  0.000346                                16.438750                                 81.207189                           13.625000                                           4.625000                                                   5                                         1                                         1                                          1                                            1                                             1                                  1                                2014                                    1                                      2
5               60091                6                            3                mobile                   6349.66                 44.095630                    149.02                  -0.025941                      7.55                  80.375443                   79                                    5                              5              17                  28             2010                 1984                 7                     7                   5                       5                              259.873954                                  839.76                                 0.014384                                   86.49                               472.231119                                                 30                              402.775486                                7.928001                                 0.415426                                4.961414                                11.007471                           3.600926                                           0.000000                                 1700.67                               51.149250                                 0.602209                                   20.65                                94.481667                                 18                                                  5                                 0.472342                                 0.204548                                -0.333796                                -0.470410                                  0.335175                           -0.317685                                           0.000000                                   543.18                               36.734681                                  128.51                                -0.539060                                66.666667                                  8                                                  5                              1058.276667                                43.312326                               139.960000                                  0.002397                                14.415000                                 78.705187                           13.166667                                           5.000000                                                   5                                         1                                         1                                          1                                            1                                             3                                  1                                2014                                    1                                      2