Skip to content

[VL] Exception in GlutenParquetV1QuerySuite decimal read schema evolusion #11219

@jinchengchenghh

Description

@jinchengchenghh

Backend

VL (Velox)

Bug description

Read decimal(2, 1) as decimal(3, 2)

10:51:35.289 ERROR org.apache.spark.task.TaskResources: Task 643 failed by error: 
org.apache.gluten.exception.GlutenException: Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: Converted type DECIMAL(2, 1) is not allowed for requested type DECIMAL(3, 2)
Retriable: False
Expression: isCompatible( requestedType, isRepeated, [&](const TypePtr& type) { return type->isShortDecimal() && type->asShortDecimal().precision() >= schemaElementPrecision && type->asShortDecimal().scale() == schemaElementScale; })
Context: Split [Hive: file:///private/var/folders/63/845y6pk53dx_83hpw8ztdchw0000gn/T/spark-d7f8cf57-bac8-4ed7-a1b3-0f5fef463077/part-00000-21ecad37-29b1-49f0-8028-a6f32211c9e3.snappy.parquet 0 - 1055] Task Gluten_Stage_401_TID_643_VTID_401
Additional Context: Operator: TableScan[0] 0
10:51:35.291 ERROR org.apache.spark.executor.Executor: Exception in task 0.0 in stage 401.0 (TID 643)
org.apache.gluten.exception.GlutenException: Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: Converted type DECIMAL(2, 1) is not allowed for requested type DECIMAL(3, 2)
Retriable: False
Expression: isCompatible( requestedType, isRepeated, [&](const TypePtr& type) { return type->isShortDecimal() && type->asShortDecimal().precision() >= schemaElementPrecision && type->asShortDecimal().scale() == schemaElementScale; })
Context: Split [Hive: file:///private/var/folders/63/845y6pk53dx_83hpw8ztdchw0000gn/T/spark-d7f8cf57-bac8-4ed7-a1b3-0f5fef463077/part-00000-21ecad37-29b1-49f0-8028-a6f32211c9e3.snappy.parquet 0 - 1055] Task Gluten_Stage_401_TID_643_VTID_401
Additional Context: Operator: TableScan[0] 0

Gluten version

No response

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtriage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions