Repository navigation
Automate testing / ensuring that string functions get the same answer for String, LargeString, StringView, DictionaryString, etc #12415
Description
Is your feature request related to a problem or challenge?
@2010YOUY01 found a bug where substr behaved differently for StringView than for String: #12383
As we continue to implement special versions of the string functions for different representations this drift may get worse
Describe the solution you'd like
I would like a systematic and SQL level set of tests that ensure the string functions compute the same answer for all different combinations.
Describe alternatives you've considered
I think we could do this in sqllogictest by separating the table definitions into different .slt files and then includeing the same queries file. Here is an example:
So this would look something like
in strings_stringview.slt
strings_stringview.slt defines the table with appropriate type. Note there would be similar files for string and largestring
statement ok
-- Populate the "t" table table with the "standard" values in StringView form
CREATE TABLE temp AS VALUES
('one', 'two'),
('three, 'four)
statement ok
CREATE TABLE t as
SELECT
arrow_cast(column1, 'Utf8View') as column1,
arrow_cast(column2, 'Utf8View') as column2,
FROM temp
-- run queries from string_queries.slt
include string_queries.sltin strings_queries.slt
-- Note this file is intended to be run with a table 't' already defined
-- with standard data with the appropriate data type (StringView, String, etc)
-- Test concat
query
SELECT concat(column1, column2) from t;
...Additional context
No response
Activity
goldmedal commentedon Sep 17, 2024
take
goldmedal commentedon Sep 17, 2024
I plan to create a directory to collect all the string-related tests. The structure would like
The purpose of each file:
init_data.slt.part: To generate the common testing data to ensure every derived test file uses the same data.string_query.slt.part: To collect all the tests for string-like type (string, largestring and StringView)All the tests will be based on a
testtable likeThe original test case in
datafusion/sqllogictest/test_files/string_view.sltis complete for many string scenarios. I'll follow it to implement the common tests and add some scalar function tests.string.slt,string_view.sltandlarge_string.slt: They are the main entrypoint of sqllogictests. The structure would likealamb commentedon Sep 17, 2024
Thank you @goldmedal . Thank you so much.
Note that @2010YOUY01 did an initial version with one function here: #12433
I suggest we try and follow the same model with some other functions and see how it goes
goldmedal commentedon Sep 18, 2024
Thanks for the suggestion. I think my idea is similar to his. The difference is that I tried to group the test cases by data type instead of by function. I'm not sure which one is better. Maybe by function to implement tests is more simple 🤔
goldmedal commentedon Sep 21, 2024
Move the TODO list here:
Some TODO items should be finished in the follow-up PR
string_view.sltLIKE/ILIKESUBSTRplaning||operator~operator (regex match)~*operator (regex match case-insensitive)!~~operator (not like match)!~~*operator (not like match case-insensitive)string/string_view.sltalamb commentedon Oct 25, 2024
I think @goldmedal has basically completed this epic -- and we should close it -- is that the case ?
goldmedal commentedon Oct 26, 2024
Yes, the testing framework has been finished. Other function tests can be enhanced by the related issue. Let's close this issue.