Feature types¶
Featuretools groups features into four general types:
Identity Features¶
In Featuretools, each feature is defined as a combination of other features. At the lowest level are IdentityFeature features which are equal to the value of a single variable.
Most of the time, identity features will be defined transparently for you, such as in the transform feature example below. They may also be defined explicitly:
In [1]: time_feature = ft.Feature(es["transactions"]["transaction_time"])
In [2]: time_feature
Out[2]: <Feature: transaction_time>
Direct Features¶
Direct features are used to “inherit” feature values from a parent to a child entity. Suppose each event is associated with a single instance of the entity products.
This entity has metadata about different products, such as brand, price, etc. We can pull the brand of the product into a feature of the event entity by including the event entity as an argument to Feature.
In this case, Feature is an alias for primitives.DirectFeature:
In [3]: brand = ft.Feature(es["products"]["brand"], entity=es["transactions"])
In [4]: brand
Out[4]: <Feature: products.brand>
Transform Features¶
Transform features take one or more features on an Entity and create a single new feature for that same entity. For example, we may want to take a fine-grained “timestamp” feature and convert it into the hour of the day in which it occurred.
In [5]: from featuretools.primitives import Hour
In [6]: ft.Feature(time_feature, primitive=Hour)
Out[6]: <Feature: HOUR(transaction_time)>
Using algebraic and boolean operations, transform features can combine other features into arbitrary expressions. For example, to determine if a given event event happened in the afternoon, we can write:
In [7]: hour_feature = ft.Feature(time_feature, primitive=Hour)
In [8]: after_twelve = hour_feature > 12
In [9]: after_twelve
Out[9]: <Feature: HOUR(transaction_time) > 12>
In [10]: at_twelve = hour_feature == 12
In [11]: before_five = hour_feature <= 17
In [12]: is_afternoon = after_twelve & before_five
In [13]: is_afternoon
Out[13]: <Feature: AND(HOUR(transaction_time) > 12, HOUR(transaction_time) <= 17)>
Aggregation Features¶
Aggregation features are used to create features for a parent entity by summarizing data from a child entity. For example, we can create a Count feature which counts the total number of events for each customer:
In [14]: from featuretools.primitives import Count
In [15]: total_events = ft.Feature(es["transactions"]["transaction_id"], parent_entity=es["customers"], primitive=Count)
In [16]: fm = ft.calculate_feature_matrix([total_events], es)
In [17]: fm.head()
Out[17]:
COUNT(transactions)
customer_id
5 79
4 109
1 126
3 93
2 93
Note
For users who have written aggregations in SQL, this concept will be familiar. One key difference in featuretools is that GROUP BY and JOIN are implicit. Since the parent and child entities are specified, featuretools can infer how to group the child entity and then join the resulting aggregation back to the parent entity.
Often times, we only want to aggregate using a certain amount of previous data. For example, we might only want to count events from the past 30 days. In this case, we can provide the use_previous parameter:
In [18]: total_events_last_30_days = ft.Feature(es["transactions"]["transaction_id"],
....: parent_entity=es["customers"],
....: use_previous="30 days",
....: primitive=Count)
....:
In [19]: fm = ft.calculate_feature_matrix([total_events_last_30_days], es)
In [20]: fm.head()
Out[20]:
COUNT(transactions, Last 30 Days)
customer_id
5 0.0
4 0.0
1 0.0
3 0.0
2 0.0
Unlike with cumulative transform features, the use_previous parameter here is evaluated relative to instances of the parent entity, not the child entity. The above feature translates roughly to the following: “For each customer, count the events which occurred in the 30 days preceding the customer’s timestamp.”
Find the list of the supported aggregation features here.
Where clauses¶
When defining aggregation or cumulative transform features, we can provide a where parameter to filter the instances we are aggregating over. Using the is_afternoon feature from earlier, we can count the total number of events which occurred in the afternoon:
In [21]: afternoon_events = ft.Feature(es["transactions"]["transaction_id"],
....: parent_entity=es["customers"],
....: where=is_afternoon,
....: primitive=Count).rename("afternoon_events")
....:
In [22]: fm = ft.calculate_feature_matrix([afternoon_events], es)
In [23]: fm.head()
Out[23]:
afternoon_events
customer_id
5 0.0
4 0.0
1 0.0
3 0.0
2 0.0
The where argument can be any previously-defined boolean feature. Only instances for which the where feature is True are included in the final calculation.
Aggregations of Direct Feature¶
Composing multiple feature types is an extremely powerful abstraction that Featuretools makes simple. For instance, we can aggregate direct features on a child entity from a different parent entity. For example, to calculate the most common brand a customer interacted with:
In [24]: from featuretools.primitives import Mode
In [25]: brand = ft.Feature(es["products"]["brand"], entity=es["transactions"])
In [26]: favorite_brand = ft.Feature(brand, parent_entity=es["customers"], primitive=Mode)
In [27]: fm = ft.calculate_feature_matrix([favorite_brand], es)
In [28]: fm.head()
Out[28]:
MODE(transactions.products.brand)
customer_id
5 B
4 B
1 B
3 B
2 B
Side note: Feature equality overrides default equality¶
Because we can check if two features are equal (or a feature is equal to a value), we override Python’s equals (==) operator. This means to check if two feature objects are equal (instead of their computed values in the feature matrix), we need to compare their hashes:
In [29]: hour_feature.hash() == hour_feature.hash()
Out[29]: True
In [30]: hour_feature.hash() != hour_feature.hash()
Out[30]: False
dictionaries and sets use equality underneath, so those keys need to be hashes as well
In [31]: myset = set()
In [32]: myset.add(hour_feature.hash())
In [33]: hour_feature.hash() in myset
Out[33]: True
In [34]: mydict = dict()
In [35]: mydict[hour_feature.hash()] = hour_feature
In [36]: hour_feature.hash() in mydict
Out[36]: True