IGNITE-29031 SQL Calcite: Support byte[] in UDF and UDTF parameters and results by tkalkirill · Pull Request #13543 · apache/ignite

tkalkirill · 2026-09-02T10:49:08Z

https://issues.apache.org/jira/browse/IGNITE-29031

zstan · 2026-09-03T06:03:31Z

First of all you need to follow the common process [1] i.e. make a PR from your own ignite mirror, not from origin = https://github.com/apache but from: https://github.com/tkalkirill/ignite

[1] https://cwiki.apache.org/confluence/spaces/IGNITE/pages/177047163/How+to+Contribute#HowtoContribute-GITworkflow

tkalkirill · 2026-09-03T06:11:23Z

@zstan Okay, subsequent tickets will do as described.

zstan · 2026-09-04T13:11:40Z

            if (isA(fromType, Primitive.LONG))
                return Expressions.call(BuiltInMethod.INTERNAL_TO_TIMESTAMP.method, operand);
        }
+        else if (targetType == byte[].class && fromType == ByteString.class)


The reverse branch in fromInternal is reachable through RexImpTable.defineReflective. I verified this with an operator backed by binaryLength(byte[]): with the conversion, a binary literal works; without it, generated code fails to compile because it passes ByteString to a method expecting byte[]. I suggest keeping this branch.

sorry, but still miss it ( do we have a test for it ? suggest it plz ?

Added a org.apache.ignite.internal.processors.query.calcite.integration.OperatorsExtensionIntegrationTest#testByteArrayFunctions that reproduces the issue if this is removed.

zstan · 2026-09-08T08:45:48Z

+                assertNotSame(row, res);
+                assertEquals(1, res.length);
+                assertSame(val, res[0]);
+                assertSame(val, row[0]);


zstan · 2026-09-08T09:05:44Z

            if (isA(fromType, Primitive.LONG))
                return Expressions.call(BuiltInMethod.INTERNAL_TO_TIMESTAMP.method, operand);
        }
+        else if (targetType == byte[].class && fromType == ByteString.class)


sorry, but still miss it ( do we have a test for it ? suggest it plz ?

zstan · 2026-09-08T12:27:20Z

    }

+    /** */
+    static List<Expression> fromInternal(RexToLixTranslator translator,


Look now you have two identical code, differs only with "RexToLixTranslator translator" param
i mean:
ConverterUtils#fromInternal(java.lang.Class[], List) and ConverterUtils#fromInternal(RexToLixTranslator, java.lang.Class[], java.util.List)

seems you can just rewrite your code like :

private static Expression fromInternal(@Nullable Expression root, Expression operand, Type targetType) { if (Types.isAssignableFrom(targetType, operand.getType()) || Types.isAssignableFrom(targetType, Primitive.box(operand.getType()))) return operand; if (!TypeUtils.isConvertableType(targetType)) return targetType == BigDecimal.class ? fromInternal(operand, operand.getType(), targetType) : convert(operand, operand.getType(), targetType); Primitive primitive = Primitive.of(targetType); if (Primitive.is(operand.getType())) operand = Expressions.box(operand); Expression converted = Expressions.call( TypeUtils.class, "fromInternal", root,//translator.getRoot(), operand, Expressions.constant(targetType) ); return primitive == null ? Expressions.convert_(converted, targetType) : Expressions.unbox(Expressions.convert_(converted, primitive.boxClass), primitive); }

probably some assertions check are helpful, it`s just a prototype
I run this approach through all calcite tests and it`s ok
wdyt ? props: more readable code, less code base

Thanks, I did it a bit differently, and the duplication went away.

Copilot

🔵 Needs a closer look

One or more issues must be addressed before approval.

Pull request overview

Adds SQL Calcite support for byte[] UDF/UDTF parameters and results, with broader conversion coverage for temporal, primitive, collection, and custom types.

Changes:

Adds internal/public binary and temporal value conversions.
Updates reflective scalar and table functions to expose SQL-compatible types.
Expands integration and execution tests.

File summaries

File	Description
modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite/integration/UserDefinedFunctionsIntegrationTest.java	Updated as part of this pull request.
modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite/integration/OperatorsExtensionIntegrationTest.java	Updated as part of this pull request.
modules/calcite/src/test/java/org/apache/ignite/internal/processors/query/calcite/exec/rel/ExecutionTest.java	Updated as part of this pull request.
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/util/TypeUtils.java	Updated as part of this pull request.
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/TableFunctionScan.java	Updated as part of this pull request.
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/LogicalRelImplementor.java	Updated as part of this pull request.
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/ReflectiveCallNotNullImplementor.java	Updated as part of this pull request.
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/IgniteTableFunction.java	Updated as part of this pull request.
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/IgniteScalarFunction.java	Updated as part of this pull request.
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/IgniteReflectiveFunctionBase.java	Updated as part of this pull request.
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/IgniteFunctionParameter.java	Updated as part of this pull request.
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/ConverterUtils.java	Updated as part of this pull request.

Review details

Suppressed comments (1)

modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/TableFunctionScan.java:73

Using an exact-class check rejects valid array row holders whose runtime type is a subtype of Object[] (for example String[] or byte[][]). These values satisfy the method's documented Object[] row contract but now fail with IgniteSQLException before column conversion; use an Object[] assignability/instanceof check while still cloning the array.

        if (rowContainer.getClass() != Object[].class && !Collection.class.isAssignableFrom(rowContainer.getClass()))
            throw new IgniteSQLException("Unable to process table function data: row type is neither Collection or Object[].");

        Object[] rowArr = rowContainer.getClass() == Object[].class
            ? ((Object[])rowContainer).clone()

Files reviewed: 12/12 changed files
Comments generated: 0
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

zstan · 2026-09-11T10:19:21Z

+        if (fromType == ByteString.class && toType == byte[].class)
+            return Expressions.call(BuiltInMethod.BYTE_STRING_TO_BYTE_ARRAY.method, operand);


these conversion still has no test coverage, isn`t it ?

zstan · 2026-09-11T10:27:07Z

+ * <p>The SQL representation is required to validate user-defined function arguments and to convert literal arguments
+ * while deriving a table function row type.
+ */
+final class IgniteFunctionParameter implements FunctionParameter {


seems all tests are passed if you completelly remote this class, am i right ?

zstan · 2026-09-11T11:14:34Z

+
+        return primitive == null
+            ? Expressions.convert_(converted, targetType)
+            : Expressions.unbox(Expressions.convert_(converted, primitive.boxClass), primitive);


redundant ?

IGNITE-29031 Wip

e353bca

zstan reviewed Sep 3, 2026

View reviewed changes

Comment thread ...gnite/internal/processors/query/calcite/integration/UserDefinedFunctionsIntegrationTest.java

zstan reviewed Sep 3, 2026

View reviewed changes

Comment thread ...n/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/IgniteTableFunction.java Outdated

IGNITE-29031 After review 1.0

f73b7e7

zstan reviewed Sep 4, 2026

View reviewed changes

Comment thread ...ache/ignite/internal/processors/query/calcite/exec/exp/ReflectiveCallNotNullImplementor.java

zstan reviewed Sep 4, 2026

View reviewed changes

Comment thread ...rc/main/java/org/apache/ignite/internal/processors/query/calcite/exec/TableFunctionScan.java Outdated

zstan reviewed Sep 4, 2026

View reviewed changes

Comment thread ...alcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/util/TypeUtils.java Outdated

zstan reviewed Sep 4, 2026

View reviewed changes

Comment thread ...rc/main/java/org/apache/ignite/internal/processors/query/calcite/exec/TableFunctionScan.java Outdated

tkalkirill added 3 commits September 7, 2026 13:29

Merge branch 'master' into ignite-29031

4578951

IGNITE-29031 After review 2.0

cea8805

IGNITE-29031 After review 2.2

64b0419

zstan reviewed Sep 8, 2026

View reviewed changes

tkalkirill added 4 commits September 8, 2026 16:47

IGNITE-29031 After review 3.0

0ac6b0b

IGNITE-29031 After review 3.1

3937734

IGNITE-29031 After review 3.2

0bc8c0a

Merge branch 'master' into ignite-29031

66bb687

zstan requested a lite review from Copilot September 11, 2026 05:59

Copilot started reviewing on behalf of zstan September 11, 2026 06:00 View session

Copilot AI reviewed Sep 11, 2026

View reviewed changes

zstan reviewed Sep 11, 2026

View reviewed changes

		if (fromType == ByteString.class && toType == byte[].class)
		return Expressions.call(BuiltInMethod.BYTE_STRING_TO_BYTE_ARRAY.method, operand);

Uh oh!

Conversation

tkalkirill commented Sep 2, 2026

Uh oh!

zstan commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tkalkirill commented Sep 3, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zstan Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Suppressed comments (1)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zstan commented Sep 3, 2026 •

edited

Loading

zstan Sep 8, 2026 •

edited

Loading