Skip to content

Incorrect backslash treatment in string literals in DataFusion CLI #13286

Closed
#14844
Closed
Incorrect backslash treatment in string literals in DataFusion CLI#13286
#14844
Assignees
Lordworms
Labels
bugSomething isn't working

Description

@findepi
findepi
opened on Nov 7, 2024

In standard SQL, the \ character has no special meaning in '...' varchar literals.

In PostgreSQL

postgres=# SELECT '\', '\\'
postgres-# ;
 ?column? | ?column?
----------+----------
 \        | \\
(1 row)

in DataFusion sqllogictest the behavior is the same:

query T
SELECT '\'
----
\

query T
SELECT '\\'
----
\\

query T
SELECT '\\\'
----
\\\

query T
SELECT '\\\\'
----
\\\\

However, DataFusion CLI behaves differently

$ cargo run
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.15s
     Running `target/debug/datafusion-cli`
DataFusion CLI v43.0.0
> SELECT '\\';
+-----------+
| Utf8("\") |
+-----------+
| \         |
+-----------+
1 row(s) fetched.
Elapsed 0.049 seconds.

> SELECT '\';  🤔 Invalid statement: SQL error: TokenizerError("unsupported escape char: '\\''")

Given that DataFusion CLI is used a lot to test and verify DataFusion's behavior, it's super important for the CLI to behave correctly with respect to its input.

Activity

  1. alamb commented on Feb 5, 2025

    @alamb
    alamb
    on Feb 5, 2025
    Contributor

    BTW @pmcgleenon hit this when updating ClickBench as well:

    ClickHouse/ClickBench#301

  2. findepi
    added
    bugSomething isn't working
    correctness
    on Feb 5, 2025

  3. Lordworms commented on Feb 22, 2025

    @Lordworms
    Lordworms
    on Feb 22, 2025
    Contributor

    take

  4. github-actions
    assigned
    Lordworms
    on Feb 22, 2025

  5. alamb commented on Feb 25, 2025

    @alamb
    alamb
    on Feb 25, 2025
    Contributor

    @Lordworms has a proposed PR here to fix this:

    However it seems like the way to get consistent behavior with sqllogictest (and psql) is to avoid unescaping in datafusion-cli 🤔

    Any thoughts on this?

  6. alamb
    closed this as completedin #14844on Feb 28, 2025

  7. alamb commented on Feb 28, 2025

    @alamb
    alamb
    on Feb 28, 2025
    Contributor

    @Lordworms has a proposed PR here to fix this:

    However it seems like the way to get consistent behavior with sqllogictest (and psql) is to avoid unescaping in datafusion-cli 🤔

    Any thoughts on this?

    Documentatiion in

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions