Be able to cast struct to have non-nullable inner field

Is your feature request related to a problem or challenge?

these work:

> select arrow_cast(struct(10), 'Struct("c0": nullable Int64)');
+--------------------------------------------------------------------+
| arrow_cast(struct(Int64(10)),Utf8("Struct("c0": nullable Int64)")) |
+--------------------------------------------------------------------+
| {c0: 10}                                                           |
+--------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.065 seconds.

> select arrow_cast(10, 'List(non-null Int64)');
+----------------------------------------------------+
| arrow_cast(Int64(10),Utf8("List(non-null Int64)")) |
+----------------------------------------------------+
| [10]                                               |
+----------------------------------------------------+
1 row(s) fetched.
Elapsed 0.010 seconds.

this doesn't work, when field is specified to be non-nullable inside a struct:

> select arrow_cast(struct(10), 'Struct("c0": non-null Int64)');
Optimizer rule 'simplify_expressions' failed
caused by
This feature is not implemented: Unsupported CAST from Struct("c0": Int64) to Struct("c0": non-null Int64)

Describe the solution you'd like

see if its possible to implement this support? it seems to work for lists but not structs, so not sure if this is a limitation or theres a valid reason to not allow this

Describe alternatives you've considered

No response

Additional context

i think somewhere around here in the codebase?

datafusion/datafusion/physical-expr/src/expressions/cast.rs

Lines 384 to 416 in 5c92390

    
           pub fn cast_with_target_field( 
        
               expr: Arc<dyn PhysicalExpr>, 
        
               input_schema: &Schema, 
        
               target_field: FieldRef, 
        
               cast_options: Option<CastOptions<'static>>, 
        
           ) -> Result<Arc<dyn PhysicalExpr>> { 
        
               let expr_type = expr.data_type(input_schema)?; 
        
               let cast_type = target_field.data_type(); 
        
               if expr_type == *cast_type && is_default_target_field(&target_field) { 
        
                   return Ok(Arc::clone(&expr)); 
        
               } 
        
               let can_build_cast = if requires_nested_struct_cast(&expr_type, cast_type) { 
        
                   // Allow casts involving structs (including nested inside Lists, Dictionaries, 
        
                   // etc.) that pass name-based compatibility validation. This validation is 
        
                   // applied at planning time (now) to fail fast, rather than deferring errors 
        
                   // to execution time. The name-based casting logic will be executed at runtime 
        
                   // via ColumnarValue::cast_to. 
        
                   can_cast_named_struct_types(&expr_type, cast_type) 
        
               } else { 
        
                   can_cast_types(&expr_type, cast_type) 
        
               }; 
        
               if !can_build_cast { 
        
                   return not_impl_err!("Unsupported CAST from {expr_type} to {cast_type}"); 
        
               } 
        
               Ok(Arc::new(CastExpr::new_with_target_field( 
        
                   expr, 
        
                   target_field, 
        
                   cast_options, 
        
               ))) 
        
           }

take

untake

Navigation Menu

Repository navigation

Be able to cast struct to have non-nullable inner field #22709

Is your feature request related to a problem or challenge?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	pub fn cast_with_target_field(
	expr: Arc<dyn PhysicalExpr>,
	input_schema: &Schema,
	target_field: FieldRef,
	cast_options: Option<CastOptions<'static>>,
	) -> Result<Arc<dyn PhysicalExpr>> {
	let expr_type = expr.data_type(input_schema)?;
	let cast_type = target_field.data_type();
	if expr_type == *cast_type && is_default_target_field(&target_field) {
	return Ok(Arc::clone(&expr));
	}

	let can_build_cast = if requires_nested_struct_cast(&expr_type, cast_type) {
	// Allow casts involving structs (including nested inside Lists, Dictionaries,
	// etc.) that pass name-based compatibility validation. This validation is
	// applied at planning time (now) to fail fast, rather than deferring errors
	// to execution time. The name-based casting logic will be executed at runtime
	// via ColumnarValue::cast_to.
	can_cast_named_struct_types(&expr_type, cast_type)
	} else {
	can_cast_types(&expr_type, cast_type)
	};

	if !can_build_cast {
	return not_impl_err!("Unsupported CAST from {expr_type} to {cast_type}");
	}

	Ok(Arc::new(CastExpr::new_with_target_field(
	expr,
	target_field,
	cast_options,
	)))
	}

Be able to cast struct to have non-nullable inner field #22709

Description

Is your feature request related to a problem or challenge?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Activity

mkleen commented on Jun 3, 2026

mkleen commented on Aug 24, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions