Skip to content

TOO LONG QUERY error should name the materialized property whose update produced the query #1915

Open

Description

@CrushBy

Description

When the text of a query exceeds queryLengthLimit, the apply fails with

SQLTooLongQueryException TOO LONG QUERY ROWS, LENGTH : 3597686, QUERY : INSERT INTO lsfpool.n0_t_44 (k0,p00) SELECT ... CASE WHEN (... = 2 AND ... = 0 AND ...

The message carries only the length and the first characters of the SQL. When the query is the incremental update of a materialized property that reads non-materialized properties — for example a GROUP MIN ... BY key(x) MATERIALIZED whose grouping key key is a multi-branch OVERRIDE over properties of another aggregate — the definitions of those properties are expanded into the update query, so its length does not depend on the number of changed rows: a write of a few objects fails exactly like a write of thousands, and the developer has to work out from the generated SQL which property's definition made the query long.

key (Event e) = OVERRIDE kind1(e), kind2(e), kind3(e);            // not materialized, several branches
head (STRING[10] k) = GROUP MIN date(Event e) BY key(e) MATERIALIZED; // its update expands key(e) into the query
// a write of 100 new events: TOO LONG QUERY ROWS, LENGTH : 3597686, QUERY : INSERT INTO lsfpool...
// expected: the error names head[STRING[10]] as the property being updated (and, ideally, key[Event] as the expanded one)

The same happens with a materialized sum over a non-materialized condition that contains a multi-branch selection switched by a data property changed in the same session: the module loads and a dry run passes, and the first apply that changes the switch fails with the same message. The server log already has what the message lacks: the execution stack printed under the error lists the apply steps with the properties, for example

Applying changes: 351 of 827
Writing materialized property changes : ProbeMat.oldIn[Catalog.Category,INTEGER] 'Old in' [ProbeMat(8:9)]
Materializing property changes : ProbeMat.totalSpan[] 'Total span' [ProbeMat(19:16)]
Materializing property changes : ProbeMat.span[Catalog.Category] 'Span' [ProbeMat(16:21)]

but through the HTTP API and in the client only the message arrives.

Fix

Add to the error the materialized property whose stored values the failing query updates — at least the property that is being updated, ideally also the non-materialized properties whose definitions were expanded into the query. The apply step from the execution stack (the property being written or materialized) is the minimum.

Reason

The failure broke a scheduled writer, and finding the property to materialize required reasoning about the dependency chain by hand, because the error names none; in the second case the time went into locating the materialized property and its dependency from the SQL prefix, while the server log had the names all along.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions