Cast expressions cannot distinguish inherited metadata from an explicitly empty target #24724

Describe the bug

• Filed from fix: preserve projection metadata during optimization #24670 while fixing Projection field metadata is lost during physical plan optimization #24721

A cast to a data type and a cast to an explicit Arrow field have different metadata:

• A type-only cast should inherit metadata from the source field
• An explicit target field should use its own metadata, including an empty map that clears source metadata

DF currently stores both forms as a target FieldRef, so once the target field has empty metadata later on in planning we cannot tell whether it was synthesized from a DataType or explicitly supplied. Cast target can then be lost during schema derivations

I saw this when using explicit extension metadata, such as arrow.uuid, can disappear from cast output in sqllogic tets

To Reproduce

These two casts have different intended behavior but can produce the same stored target field:

// Type-only: inherit source metadata.
Cast::new(expr.clone(), DataType::FixedSizeBinary(16));

// Explicit field: clear source metadata.
Cast::new_from_field(
    expr,
    Arc::new(Field::new("", DataType::FixedSizeBinary(16), true)),
);

If the source field contains:

metadata = {"ARROW:extension:name": "arrow.uuid"}

the first cast should retain arrow.uuid, while the second should produce an empty metadata map. Inspecting only target_field.metadata().is_empty() cannot distinguish these cases.

This is visible with a custom SQL type planner:

SELECT
    CAST(raw AS UUID),
    arrow_metadata(CAST(raw AS UUID), 'ARROW:extension:name')
FROM (
    VALUES (
        arrow_cast(X'00010203040506070809000102030506', 'FixedSizeBinary(16)')
    )
) AS uuids(raw);

The expected metadata value is arrow.uuid, not NULL.

Expected behavior

Cast target should be represented clearly and preserved:

• Type-only casts inherit source field metadata
• Explicit target fields replace source metadata, even when their metadata map is empty
• Built-in SQL and standard Substrait casts use type-only targets
• Logical/ physical plan rewrites and protobuf round trips preserve distinction