We are delighted to announce the release of Feast 0.18, which introduces several new features and other improvements:
- Snowflake offline store, which allows you to define and use features stored in Snowflake.
- [Experimental] Saved Datasets, which allow training datasets to be persisted in an offline store.
- [Experimental] Data quality monitoring, which allows you to validate your training data with Great Expectations. Future work will allow you to detect issues with upstream data pipelines and check for training-serving skew.
- Python feature server graduation from alpha status.
- Performance improvements to on demand feature views, protobuf serialization and deserialization, and the Python feature server.
Experimental features are subject to API changes in the near future as we collect feedback. If you have thoughts, please don’t hesitate to reach out to the Feast team through our Slack!
Snowflake offline store
Prior to Feast 0.18, Feast had first-class support for Google BigQuery and AWS Redshift as offline stores. In addition, there were various plugins for Snowflake, Azure, Postgres, and Hive. Feast 0.18 introduces first-class support for Snowflake as an offline store, so users can more easily leverage features defined in Snowflake. The Snowflake offline store can be used with the AWS, GCP, and Azure providers.
[Experimental] Saved Datasets
Training datasets generated via get_historical_features can now be persisted in an offline store and reused later. This functionality will be primarily needed to generate reference datasets for validation purposes (see next section) but also could be useful in other use cases like caching results of a computationally intensive point-in-time join.
[Experimental] Data quality monitoring
Feast 0.18 includes the first milestone of our data quality monitoring work. Many users have requested ways to validate their training and serving data, as well as monitor for training-serving skew. Feast 0.18 allows users to validate their training data through an integration with Great Expectations. Users can declare one of the previously generated training datasets as a reference for this validation by persisting it as a “saved dataset” (see previous section). More details about future milestones of data quality monitoring can be found here. There’s also a tutorial on validating historical features that demonstrates all new concepts in action.
Python feature server
The Python feature server was first introduced in Feast 0.13, in alpha status. It provided an HTTP endpoint to enable users to access features through any programming language that can make HTTP requests. Over the last several months, the Python feature server has been extensively tested and improved, and has now graduated from alpha status. We provide Docker images and a Helm chart for easy deployment of the Python feature server.
Performance improvements
The Feast team and community members have made several significant performance improvements. For example, the Python feature server performance was improved by switching to a more efficient serving interface. Improving our protobuf serialization and deserialization logic led to speedups in on demand feature views. The Datastore implementation was also sped up by batching operations. For more details, please see our blog post with detailed benchmarks!
What’s next
We are collaborating with the community on the first milestone of the feast plan command, future milestones of data quality monitoring, and a consolidation of our online serving logic into Golang.
In addition, there is active community work on adding support for Snowflake as an online store, merging the Azure plugin into the main Feast repo, and more. If you have thoughts on what to build next in Feast, please fill out this form.
Download Feast 0.18 today from PyPI (or pip install feast) and try it out! Let us know on our slack channel what you think. Also join the Google group here to check out our newsletter and stay up to date with Feast!
Credits
We want to extend our gratitude and acknowledgement to all Feast community contributors, @achals, @adchia, @corentinmarek, @diogommartins, @dmille, @felixwang9817, @judahrand, @michelle-rascati-sp, @mickey-liu, @mirayyuce, @NalinGHub, @ptoman-pa, @pyalex, @sfc-gh-madkins, and @tsotnet, who helped contribute to this release.To see all the features, enhancements, and bug fixes from the Feast community contributors, check the changelog for this release.
