We are delighted to announce the release of Feast 0.14, which introduces a new feature and several important improvements:
- [Experimental] AWS Lambda feature servers, which allow you to quickly deploy an HTTP server to serve online features on AWS Lambda. GCP Cloud Run and Java feature servers are coming soon!
- Bug fixes around performance. The core online serving path is now significantly faster.
- Improvements for developer experience. The integration tests are now faster, and temporary tables created during integration tests are immediately dropped after the test.
Experimental features are subject to API changes in the near future as we collect feedback. If you have thoughts, please don’t hesitate to reach out to the Feast team!
[Experimental] AWS Lambda feature servers
Prior to Feast 0.13, the only way for users to retrieve online features was to use the Python SDK. This was restrictive, so Feast 0.13 introduced local Python feature servers, allowing users to deploy a local HTTP server to serve their online features. Feast 0.14 now allows users to deploy a feature server on AWS Lambda to quickly serve features at scale.
The new AWS Lambda feature servers are available for feature stores using the AWS provider. To deploy a feature server to AWS Lambda, they must be enabled and be given the appropriate permissions:
project: dev
registry: s3://feast/registries/dev
provider: aws
online_store:
region: us-west-2
offline_store:
cluster_id: feast
region: us-west-2
user: admin
database: feast
s3_staging_location: s3://feast/redshift/tests/staging_location
iam_role: arn:aws:iam::{aws_account}:role/redshift_s3_access_role
flags:
alpha_features: true
aws_lambda_feature_server: true
feature_server:
enabled: True
execution_role_name: arn:aws:iam::{aws_account}:role/lambda_execution_role
Calling `feast apply` will then deploy the feature server. The precise endpoint can be determined with by calling `feast endpoint`, and the endpoint can then be queried as follows:
$ feast apply
10/07/2021 03:57:26 PM INFO:Pulling remote image feastdev/feature-server:aws:
10/07/2021 03:57:28 PM INFO:Creating remote ECR repository feast-python-server-key_shark-0_13_1_dev23_gb3c08320:
10/07/2021 03:57:29 PM INFO:Pushing local image to remote {aws_account}.dkr.ecr.us-west-2.amazonaws.com/feast-python-server-key_shark-0_13_1_dev23_gb3c08320:0_13_1_dev23_gb3c08320:
10/07/2021 03:58:44 PM INFO:Deploying feature server...
10/07/2021 03:58:45 PM INFO: Creating AWS Lambda...
10/07/2021 03:58:46 PM INFO: Creating AWS API Gateway...
Registered entity driver_id
Registered feature view driver_hourly_stats
Deploying infrastructure for driver_hourly_stats
$ feast endpoint
10/07/2021 03:59:01 PM INFO:Feature server endpoint: https://hkosgmz4m2.execute-api.us-west-2.amazonaws.com
$ feast materialize-incremental $(date +%Y-%m-%d)
Materializing 1 feature views to 2021-10-06 17:00:00-07:00 into the dynamodb online store.
driver_hourly_stats from 2020-10-08 23:01:34-07:00 to 2021-10-06 17:00:00-07:00:
100%|█████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 16.89it/s]
$ curl -X POST \
"https://hkosgmz4m2.execute-api.us-west-2.amazonaws.com/get-online-features" \
-H "Content-type: application/json" \
-H "Accept: application/json" \
-d '{
"features": [
"driver_hourly_stats:conv_rate",
"driver_hourly_stats:acc_rate",
"driver_hourly_stats:avg_daily_trips"
],
"entities": {
"driver_id": [1001, 1002, 1003]
},
"full_feature_names": true
}' | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1346 100 1055 100 291 3436 947 --:--:-- --:--:-- --:--:-- 4370
{
"field_values": [
{
"fields": {
"driver_id": 1001,
"driver_hourly_stats__conv_rate": 0.025330161675810814,
"driver_hourly_stats__avg_daily_trips": 785,
"driver_hourly_stats__acc_rate": 0.835975170135498
},
"statuses": {
"driver_hourly_stats__avg_daily_trips": "PRESENT",
"driver_id": "PRESENT",
"driver_hourly_stats__conv_rate": "PRESENT",
"driver_hourly_stats__acc_rate": "PRESENT"
}
},
{
"fields": {
"driver_hourly_stats__conv_rate": 0.7595187425613403,
"driver_hourly_stats__acc_rate": 0.1740121990442276,
"driver_id": 1002,
"driver_hourly_stats__avg_daily_trips": 875
},
"statuses": {
"driver_hourly_stats__acc_rate": "PRESENT",
"driver_id": "PRESENT",
"driver_hourly_stats__avg_daily_trips": "PRESENT",
"driver_hourly_stats__conv_rate": "PRESENT"
}
},
{
"fields": {
"driver_hourly_stats__acc_rate": 0.7785481214523315,
"driver_hourly_stats__conv_rate": 0.33832859992980957,
"driver_hourly_stats__avg_daily_trips": 846,
"driver_id": 1003
},
"statuses": {
"driver_id": "PRESENT",
"driver_hourly_stats__conv_rate": "PRESENT",
"driver_hourly_stats__acc_rate": "PRESENT",
"driver_hourly_stats__avg_daily_trips": "PRESENT"
}
}
]
}
See AWS Lambda feature server for detailed info on how to use this functionality.
Performance bug fixes and developer experience improvements
The provider for a feature store is now cached instead of being instantiated repeatedly, making the core online serving path 30% faster.
Integration tests now run significantly faster on Github Actions due to caching. Also, tables created during integration tests were previously not always cleaned up properly; now they are always deleted immediately after the integration tests finish.
What’s next
We are collaborating with the community on supporting streaming sources, low latency serving, a Python feature transformation server for on demand transforms, improved support for Kubernetes deployments, and more.
In addition, there is active community work on building Hive, Snowflake, Azure, Astra, Presto, and Alibaba Cloud connectors. If you have thoughts on what to build next in Feast, please fill out this form.
Download Feast 0.14 today from PyPI (or pip install feast) and try it out! Let us know on our slack channel what you think. Also join the Google group here to check out our newsletter and stay up to date with Feast!
Credits
We want to extend our gratitude and acknowledgement to all Feast community contributors, @achals, @adchia, @Agent007, @DvirDukhan, @felixwang9817, @loftiskg, @mavysavydav, @samuel100, @tsotnet, and @ysk24ok, who helped contribute to this release.
To see all the features, enhancements, and bug fixes from the Feast community contributors, check the changelog for this release.
