From 9617d6e649385ef680b2d0ede2b37edcc037a8b5 Mon Sep 17 00:00:00 2001 From: Chaitany Patel Date: Tue, 6 Oct 2026 13:17:48 +0530 Subject: [PATCH 1/2] fix: Use UTC end bound and correct docs for disable_event_timestamp disable_event_timestamp was documented as stamping rows with the current datetime, but no materialization engine reads the flag; rows keep their source event timestamps. Update docstrings, CLI help and docs so the flag only promises full-window materialization. Also compute the full-window bounds in UTC. The CLI and feature server used naive datetime.now(), which make_tzaware() treats as UTC, so hosts west of UTC skipped the newest rows. Fixes #6936 Signed-off-by: Chaitany Patel --- README.md | 2 +- docs/getting-started/components/online-store.md | 2 +- docs/getting-started/components/overview.md | 2 +- docs/getting-started/concepts/data-ingestion.md | 2 +- docs/getting-started/quickstart.md | 2 +- docs/reference/feast-cli-commands.md | 4 ++-- .../feature-servers/python-feature-server.md | 2 +- infra/templates/README.md.jinja2 | 2 +- sdk/python/feast/cli/cli.py | 15 ++++++--------- sdk/python/feast/feature_server.py | 5 ++--- sdk/python/feast/feature_store.py | 2 +- sdk/python/feast/infra/provider.py | 2 +- sdk/python/tests/unit/test_feature_server.py | 16 ++++++++++++++++ 13 files changed, 35 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index 942d6d0856e..16cc33f12de 100644 --- a/README.md +++ b/README.md @@ -127,7 +127,7 @@ feast materialize 2021-04-12T00:00:00 $CURRENT_TIME feast materialize --disable-event-timestamp ``` -The `--disable-event-timestamp` flag allows you to materialize all available feature data using the current datetime as the event timestamp, without needing to specify start and end timestamps. This is useful when your source data lacks proper event timestamp columns. +The `--disable-event-timestamp` flag allows you to materialize all available data (from 1970-01-01 up to the current UTC time) without specifying start and end timestamps. Rows keep the event timestamps from the source. ```commandline Materializing feature view driver_hourly_stats from 2021-04-14 to 2021-04-15 done! diff --git a/docs/getting-started/components/online-store.md b/docs/getting-started/components/online-store.md index 5e45c1c2879..3f0849f8633 100644 --- a/docs/getting-started/components/online-store.md +++ b/docs/getting-started/components/online-store.md @@ -1,7 +1,7 @@ # Online store Feast uses online stores to serve features at low latency. -Feature values are loaded from data sources into the online store through _materialization_, which can be triggered through the `materialize` command (either with specific timestamps or using `--disable-event-timestamp` to materialize all data with current timestamps). +Feature values are loaded from data sources into the online store through _materialization_, which can be triggered through the `materialize` command (either with specific timestamps or using `--disable-event-timestamp` to materialize all available data). The storage schema of features within the online store mirrors that of the original data source. One key difference is that for each [entity key](../concepts/entity.md), only the latest feature values are stored. diff --git a/docs/getting-started/components/overview.md b/docs/getting-started/components/overview.md index 98d5b42b3b7..3f9c87a2439 100644 --- a/docs/getting-started/components/overview.md +++ b/docs/getting-started/components/overview.md @@ -7,7 +7,7 @@ * **Create Batch Features:** ELT/ETL systems like Spark and SQL are used to transform data in the batch store. * **Create Stream Features:** Stream features are created from streaming services such as Kafka or Kinesis, and can be pushed directly into Feast via the [Push API](../../reference/data-sources/push.md). * **Feast Apply:** The user (or CI) publishes versioned controlled feature definitions using `feast apply`. This CLI command updates infrastructure and persists definitions in the object store registry. -* **Feast Materialize:** The user (or scheduler) executes `feast materialize` (with timestamps or `--disable-event-timestamp` to materialize all data with current timestamps) which loads features from the offline store into the online store. +* **Feast Materialize:** The user (or scheduler) executes `feast materialize` (with timestamps or `--disable-event-timestamp` to materialize all available data) which loads features from the offline store into the online store. * **Model Training:** A model training pipeline is launched. It uses the Feast Python SDK to retrieve a training dataset that can be used for training models. * **Get Historical Features:** Feast exports a point-in-time correct training dataset based on the list of features and entity dataframe provided by the model training pipeline. * **Deploy Model:** The trained model binary (and list of features) are deployed into a model serving system. This step is not executed by Feast. diff --git a/docs/getting-started/concepts/data-ingestion.md b/docs/getting-started/concepts/data-ingestion.md index f1c1cc0131a..18e375aa70c 100644 --- a/docs/getting-started/concepts/data-ingestion.md +++ b/docs/getting-started/concepts/data-ingestion.md @@ -70,7 +70,7 @@ CURRENT_TIME=$(date -u +"%Y-%m-%dT%H:%M:%S") feast materialize-incremental $CURRENT_TIME ``` -**Simple materialization (for data without event timestamps):** +**Simple materialization (all available data, no start/end timestamps):** ```bash feast materialize --disable-event-timestamp ``` diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md index 0cda41263d2..851ff47f91c 100644 --- a/docs/getting-started/quickstart.md +++ b/docs/getting-started/quickstart.md @@ -513,7 +513,7 @@ feast materialize-incremental $CURRENT_TIME {% endtab %} {% tab title="Bash (simple)" %} ```bash -# Alternative: Materialize all data using current timestamp (for data without event timestamps) +# Alternative: Materialize all available data without specifying start/end timestamps feast materialize --disable-event-timestamp ``` {% endtab %} diff --git a/docs/reference/feast-cli-commands.md b/docs/reference/feast-cli-commands.md index db1051cb1be..845330fd414 100644 --- a/docs/reference/feast-cli-commands.md +++ b/docs/reference/feast-cli-commands.md @@ -272,7 +272,7 @@ Load data from feature views into the online store. feast materialize 2020-01-01T00:00:00 2022-01-01T00:00:00 ``` -**Without timestamps (uses current datetime):** +**Without timestamps (materializes all available data):** ```bash feast materialize --disable-event-timestamp ``` @@ -287,7 +287,7 @@ feast materialize -v driver_hourly_stats 2020-01-01T00:00:00 2022-01-01T00:00:00 feast materialize --disable-event-timestamp -v driver_hourly_stats ``` -The `--disable-event-timestamp` flag is useful when your source data lacks event timestamp columns, allowing you to materialize all available data using the current datetime as the event timestamp. +The `--disable-event-timestamp` flag lets you materialize all available data (from 1970-01-01 up to the current UTC time) without specifying start and end timestamps. Rows keep the event timestamps from the source. ```text Materializing 1 feature views from 2020-01-01 to 2022-01-01 diff --git a/docs/reference/feature-servers/python-feature-server.md b/docs/reference/feature-servers/python-feature-server.md index 34d62a007d7..bed50cb1cf0 100644 --- a/docs/reference/feature-servers/python-feature-server.md +++ b/docs/reference/feature-servers/python-feature-server.md @@ -286,7 +286,7 @@ curl -X POST "http://localhost:6566/materialize" -d '{ }' | jq ``` -When `disable_event_timestamp` is set to `true`, the `start_ts` and `end_ts` parameters are not required, and all available data is materialized using the current datetime as the event timestamp. This is useful when your source data lacks proper event timestamp columns. +When `disable_event_timestamp` is set to `true`, the `start_ts` and `end_ts` parameters are not required, and all available data (from 1970-01-01 up to the current UTC time) is materialized. Rows keep the event timestamps from the source. Or from Python: ```python diff --git a/infra/templates/README.md.jinja2 b/infra/templates/README.md.jinja2 index 2c92401f83d..c2ec17f8209 100644 --- a/infra/templates/README.md.jinja2 +++ b/infra/templates/README.md.jinja2 @@ -125,7 +125,7 @@ feast materialize 2021-04-12T00:00:00 $CURRENT_TIME feast materialize --disable-event-timestamp ``` -The `--disable-event-timestamp` flag allows you to materialize all available feature data using the current datetime as the event timestamp, without needing to specify start and end timestamps. This is useful when your source data lacks proper event timestamp columns. +The `--disable-event-timestamp` flag allows you to materialize all available data (from 1970-01-01 up to the current UTC time) without specifying start and end timestamps. Rows keep the event timestamps from the source. ```commandline Materializing feature view driver_hourly_stats from 2021-04-14 to 2021-04-15 done! diff --git a/sdk/python/feast/cli/cli.py b/sdk/python/feast/cli/cli.py index f18b670a435..50090fb4701 100644 --- a/sdk/python/feast/cli/cli.py +++ b/sdk/python/feast/cli/cli.py @@ -13,7 +13,7 @@ # limitations under the License. import json import logging -from datetime import datetime +from datetime import datetime, timezone from importlib.metadata import version as importlib_version from pathlib import Path from typing import List, Optional @@ -363,7 +363,7 @@ def registry_dump_command(ctx: click.Context): @click.option( "--disable-event-timestamp", is_flag=True, - help="Materialize all available data using current datetime as event timestamp (useful when source data lacks event timestamps)", + help="Materialize all available data without specifying START_TS and END_TS (rows keep their source event timestamps)", ) @click.option( "--version", @@ -388,7 +388,7 @@ def materialize_command( START_TS and END_TS should be in ISO 8601 format, e.g. '2021-07-16T19:20:01' - If --disable-event-timestamp is used, timestamps are not required and all available data will be materialized using the current datetime as the event timestamp. + If --disable-event-timestamp is used, timestamps are not required and all available data will be materialized. Rows keep their source event timestamps. """ store = create_feature_store(ctx) @@ -397,12 +397,9 @@ def materialize_command( raise click.UsageError( "Cannot specify START_TS or END_TS when --disable-event-timestamp is used" ) - now = datetime.now() - # Query all available data and use current datetime as event timestamp - start_date = datetime( - 1970, 1, 1 - ) # Beginning of time to capture all historical data - end_date = now + # Query all available data, from the beginning of time up to now (UTC) + start_date = datetime(1970, 1, 1, tzinfo=timezone.utc) + end_date = utils._utc_now() else: if not start_ts or not end_ts: raise click.UsageError( diff --git a/sdk/python/feast/feature_server.py b/sdk/python/feast/feature_server.py index 15cc550f745..d408b1b0eb3 100644 --- a/sdk/python/feast/feature_server.py +++ b/sdk/python/feast/feature_server.py @@ -22,7 +22,7 @@ from collections import defaultdict from concurrent.futures import ThreadPoolExecutor from contextlib import asynccontextmanager -from datetime import datetime +from datetime import datetime, timezone from importlib import resources as importlib_resources from types import SimpleNamespace from typing import Any, DefaultDict, Dict, List, NamedTuple, Optional, Set, Union @@ -478,8 +478,7 @@ def _parse_materialize_timestamps( ) -> tuple: """Parse and validate start/end timestamps from a MaterializeRequest.""" if request.disable_event_timestamp: - now = datetime.now() - return datetime(1970, 1, 1), now + return datetime(1970, 1, 1, tzinfo=timezone.utc), utils._utc_now() if not request.start_ts or not request.end_ts: raise ValueError( diff --git a/sdk/python/feast/feature_store.py b/sdk/python/feast/feature_store.py index 54a846733ce..c8be0d067e2 100644 --- a/sdk/python/feast/feature_store.py +++ b/sdk/python/feast/feature_store.py @@ -2855,7 +2855,7 @@ def materialize( end_date (datetime): End date for time range of data to materialize into the online store feature_views (List[str]): Optional list of feature view names. If selected, will only run materialization for the specified feature views. - disable_event_timestamp (bool): If True, materializes all available data using current datetime as event timestamp instead of source event timestamps + disable_event_timestamp (bool): If True, materializes all available data. Rows keep their source event timestamps. full_feature_names (bool): If True, feature names will be prefixed with the corresponding feature view name. version (str): Optional version to materialize (e.g., 'v2'). Requires feature_views diff --git a/sdk/python/feast/infra/provider.py b/sdk/python/feast/infra/provider.py index e0226a10d43..a4c9c9d0616 100644 --- a/sdk/python/feast/infra/provider.py +++ b/sdk/python/feast/infra/provider.py @@ -244,7 +244,7 @@ def materialize_single_feature_view( registry: The registry for the current feature store. project: Feast project to which the objects belong. tqdm_builder: A function to monitor the progress of materialization. - disable_event_timestamp: If True, materializes all available data using current datetime as event timestamp instead of source event timestamps. + disable_event_timestamp: If True, materializes all available data. Rows keep their source event timestamps. """ pass diff --git a/sdk/python/tests/unit/test_feature_server.py b/sdk/python/tests/unit/test_feature_server.py index 309a8935252..fd1b55ea460 100644 --- a/sdk/python/tests/unit/test_feature_server.py +++ b/sdk/python/tests/unit/test_feature_server.py @@ -232,6 +232,22 @@ def test_materialize_request_model(): assert req2.end_ts == "2021-01-02T00:00:00" +def test_parse_materialize_timestamps_disable_event_timestamp_uses_utc(): + """The full-window bounds must be tz-aware UTC, not naive local time.""" + from datetime import datetime, timedelta, timezone + + from feast.feature_server import MaterializeRequest, _parse_materialize_timestamps + + req = MaterializeRequest(feature_views=["test"], disable_event_timestamp=True) + before = datetime.now(timezone.utc) + start_date, end_date = _parse_materialize_timestamps(req) + after = datetime.now(timezone.utc) + + assert start_date == datetime(1970, 1, 1, tzinfo=timezone.utc) + assert end_date.utcoffset() == timedelta(0) + assert before <= end_date <= after + + def _enable_offline_batching_config( fs, enabled: bool = True, batch_size: int = 1, batch_interval_seconds: int = 60 ): From 16e8c4a8c9a490891a53592a3839bc055142e65e Mon Sep 17 00:00:00 2001 From: Chaitany Patel Date: Wed, 7 Oct 2026 11:00:29 +0530 Subject: [PATCH 2/2] docs: State the UTC upper bound for disable_event_timestamp Signed-off-by: Chaitany Patel --- sdk/python/feast/cli/cli.py | 4 ++-- sdk/python/feast/feature_store.py | 2 +- sdk/python/feast/infra/provider.py | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/sdk/python/feast/cli/cli.py b/sdk/python/feast/cli/cli.py index 50090fb4701..810c6a0c9fb 100644 --- a/sdk/python/feast/cli/cli.py +++ b/sdk/python/feast/cli/cli.py @@ -363,7 +363,7 @@ def registry_dump_command(ctx: click.Context): @click.option( "--disable-event-timestamp", is_flag=True, - help="Materialize all available data without specifying START_TS and END_TS (rows keep their source event timestamps)", + help="Materialize all available data, from 1970-01-01 up to the current UTC time, without specifying START_TS and END_TS (rows keep their source event timestamps)", ) @click.option( "--version", @@ -388,7 +388,7 @@ def materialize_command( START_TS and END_TS should be in ISO 8601 format, e.g. '2021-07-16T19:20:01' - If --disable-event-timestamp is used, timestamps are not required and all available data will be materialized. Rows keep their source event timestamps. + If --disable-event-timestamp is used, timestamps are not required and all available data (from 1970-01-01 up to the current UTC time) will be materialized. Rows keep their source event timestamps. """ store = create_feature_store(ctx) diff --git a/sdk/python/feast/feature_store.py b/sdk/python/feast/feature_store.py index c8be0d067e2..409809b9f52 100644 --- a/sdk/python/feast/feature_store.py +++ b/sdk/python/feast/feature_store.py @@ -2855,7 +2855,7 @@ def materialize( end_date (datetime): End date for time range of data to materialize into the online store feature_views (List[str]): Optional list of feature view names. If selected, will only run materialization for the specified feature views. - disable_event_timestamp (bool): If True, materializes all available data. Rows keep their source event timestamps. + disable_event_timestamp (bool): If True, materializes all available data (from 1970-01-01 up to the current UTC time). Rows keep their source event timestamps. full_feature_names (bool): If True, feature names will be prefixed with the corresponding feature view name. version (str): Optional version to materialize (e.g., 'v2'). Requires feature_views diff --git a/sdk/python/feast/infra/provider.py b/sdk/python/feast/infra/provider.py index a4c9c9d0616..2af6f84f10f 100644 --- a/sdk/python/feast/infra/provider.py +++ b/sdk/python/feast/infra/provider.py @@ -244,7 +244,7 @@ def materialize_single_feature_view( registry: The registry for the current feature store. project: Feast project to which the objects belong. tqdm_builder: A function to monitor the progress of materialization. - disable_event_timestamp: If True, materializes all available data. Rows keep their source event timestamps. + disable_event_timestamp: If True, materializes all available data (from 1970-01-01 up to the current UTC time). Rows keep their source event timestamps. """ pass