Infer column dtypes with columns and schema_overrides

[tonyyuyiding]

Summary

Fixes #5036

• I am a human, and I fully understand the code I am submitting.

Motivation

Originally, in pl.read_csv, Pyrefly only infers the column dtypes if schema is provided. However, in the case below, where we have both columns and schema_overrides, there is enough information to infer the precise schema.

from typing import reveal_type
import polars as pl

df = pl.read_csv(
    'data.csv',
    columns=['DATE', 'TT BUY'],
    schema_overrides={'DATE': pl.Datetime, 'TT BUY': pl.String},
)
reveal_type(df) # Opaque dataframe. Desired: DataFrame['DATE': Date, 'TT BUY': String]

With this PR, Pyrefly will try to infer the schema as precisely as possible with columns and schema_overrides, even though schema is not given.

Approach

The function polars_csv_options is used to parse the options of read_csv and scan_csv. In the old implementation, it assumed schema always exists and schema_overrides is either a sequence or absent, because the checker relied on schema to perform inference. To add the new feature, I have extended relevant enums and updated some old functions accordingly to handle more possible options. Then I added a new branch in infer_read_csv_schema to handle the cases where schema is missing, while the old branch was left unchanged.

Test Plan

Tests have been added in pyrefly/lib/test/polars/dataframe.rs.

[meta-codesync]

This pull request has been imported. If you are a Meta employee, you can view this in D123393717. (Because this pull request was imported automatically, there will not be any future comments.)

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅