Finalize SIGMOD 2024 paper ~(if accepted)~

@JayjeetAtGithub

UPDATE: Final paper: https://dl.acm.org/doi/10.1145/3626246.3653368 (alternate download)

¯### Is your feature request related to a problem or challenge?

@JayjeetAtGithub @Dandandan @yjshen @ozankabak @sunchao and @viirya and submitted a paper to the SIGMOD 2024 conference, which was tracked by #6782

If our paper is accepted, this ticket tracks follow on work items to complete prior to the final copy

For the Industrial Track the dates are:

All deadlines below are 11:59 PM Pacific Time.
Paper submission: Thursday, November 30, 2023
Notification of accept/reject: Wednesday, January 31, 2024
Camera-ready deadline: Thursday, March 28, 2024

Describe the solution you'd like

Here are the items I know so far:

Fix the (currently) non working email for @JayjeetAtGithub (jchakraborty@influxdata.com currently does not work)
Clean up bibliography into a consistent style (sometimes all authors are listed, sometimes just the first one is -- they should all be the same)

Nice to haves:

Update benchmark scripts: use fetch_arrow_table instead of fetchall for DuckDB JayjeetAtGithub/datafusion-duckdb-benchmark#25
Rerun the benchmarks with versions of datafusion and duckdb that have been released since our initial runs (e.g. datafusion 33 and duckdb 0.9.2) Write DataFusion paper for (SIGMOD / VLDB / ICDE) #6782 (comment))
Update the Results section with the new results, updating the query textual description if needed

Describe alternatives you've considered

No response

Additional context

No response

Here is what we submitted:
DataFusion_Query_Engine___SIGMOD_2024.pdf

I can check results and update them, please assign it to me.

FWIW the notification deadline was yesterday but I have not heard anything one way or the other (and the CMT tool doesn't say one way or the other). I will email the chairs tomorrow if we haven't heard by then

I emailed the chairs today and they said the notification will be delayed a few days. Will post updates here as I have them.

Thanks for the update 🚀

@alamb

Thank you @alamb

The paper was accepted to SIGMOD! 🎉

I'll spend some time reviewing the comments later this week and we can organize action items for the final draft

From: Microsoft CMT <email@msr-cmt.org>
Date: Sun, Feb 4, 2024 at 11:28 AM
Subject: SIGMOD 2024 Industry Track decision for Paper 1
To: Andrew Lamb 

Dear Andrew  Lamb,

It is our great pleasure to inform you that your paper #1 "Apache Arrow DataFusion: A Fast, Embeddable, Modular Analytic Query Engine" has been Accepted to the conference. Congratulations!

The papers will be presented at SIGMOD 2024, in Santiago, Chile in June, so please plan for at least one author of the paper to attend the conference. This might require a visa, so please consult the following page at your earliest convenience https://2024.sigmod.org/visa_chile.shtml .

We hope that you will find the reviews helpful in revising accordingly the camera-ready version of the manuscript. Please note that the papers will appear in PACMMOD, like the Research Track papers of SIGMOD 2024. The formatting guidelines for the camera-ready papers are available at: https://dl.acm.org/journal/pacmmod/author-guidelines#formatting , under the "Length and Format for Camera-Ready Papers" section.

Congratulations again and looking forward to seeing you at SIGMOD 2024!

Danica and Ippokratis.

Cool! Congrats to all!

This is great news! congrats all!

Congratulations everyone !

Great news! Congratulations to all involved!

Congratulations everyone !

Here is the reviewer feedback

Reviewer #2
Questions

Is the paper readable and well organized?
Definitely - very clear
Does this paper present a significant addition to the body of work in the area of data management research?
Definitely - a significant addition
Is the paper likely to have a broad impact on the data management community?
SIGMOD attendees will learn something interesting from the paper
The paper is likely to influence research in the community
Overall rating
Accept
Reviewer’s confidence
Expert
Strong points
Good presentation of the Apache Arrow DataFusion open-source project.
DataFusion efficiently implements operators that can be used by various other data systems, avoiding their cumbersome re-implementation.
Good experimental results versus DuckDB (which is an extremely well optimized embeddable analytics database).
I really appreciate how the DataFusion community was involved even in writing this paper. See here: Write DataFusion paper for (SIGMOD / VLDB / ICDE) #6782
Weak points
Minor: although well-engineered, the algorithms behind the supported operators are not new. DataFusion implements well-known techniques.
Overall comments
The paper describes the functionality of DataFusion, a very well-designed and implemented library based on Apache Arrow, which implements a variety of operators used in SQL. Similar to Arrow, DataFusion is an embeddable library (built in Rust), which can easily be embedded in broader data systems that require analytical operations. The paper includes a nice experimental evaluation versus DuckDB, demonstrating good results.

Reviewer #5
Questions

Is the paper readable and well organized?
Definitely - very clear
Does this paper present a significant addition to the body of work in the area of data management research?
Mostly - the contributions are above the bar
Is the paper likely to have a broad impact on the data management community?
SIGMOD attendees will learn something interesting from the paper
Overall rating
Accept
Reviewer’s confidence
Expert
Strong points
The paper is well written.
Extensive evaluation using 3 popular benchmarks.
An active community-driven project.
Weak points
The DataFusion project is a combination and integration of other well-known components/systems; as such, its overall technical novelty is limited.
The experimental evaluation didn't compare against many other popular OLAP systems in the field.
The support for complex analytical queries (e.g., multi-way join as those found in TPC-DS) is limited.
Overall comments
This paper is well written and the DataFusion project has a good momentum in the community. The idea of building an OLAP engine using a decoupled, component-based approach is interesting (versus tightly coupled designs). The paper has described most elements in DataFusion, but didn't offer enough details to demonstrate sufficient technical novelty (that goes beyond integration of various existing componentshe). How to better suit the cloud environment where most OLAP engines are running on nowadays is also not discussed in the paper.

Reviewer #7
Questions

Is the paper readable and well organized?
Mostly - the presentation has minor issues, but is acceptable
Does this paper present a significant addition to the body of work in the area of data management research?
Mostly - the contributions are above the bar
Is the paper likely to have a broad impact on the data management community?
SIGMOD attendees will learn something interesting from the paper
Overall rating
Reject
Reviewer’s confidence
Knowledgeable
Strong points

Presents the technologies that power DataFusion and provides motivating use cases for using DataFusion, making a compelling argument over reuse in analytic systems using commodity OLAP engines and a paradigm shift in that direction.
Provides extensive evaluation of DataFusion's performance.
Presents DataFusion's architecture, extension APIs and features.

Weak points

One of the main claims of the paper is that DataFusion is catalyzing the development of new data systems. The presentation and the evaluation of the paper would benefit from elaborating further on this claim.
Section 5.1 Engine overview and Figure 2 need to be more extensive to be able to follow the rest of Section 5.
The LLVM analogy distracts from the paper.
The paper claims in section 7.4 that "..DataFusion can be customized for these different environments using the MemoryPool trait to control memory allocations, the DiskManager trait for managing temporary files (if any), and a CacheManager for caching information such as directory contents and per-file metadata.". More technical details on this topic would be helpful.
The "Single Core Efficiency" section could benefit from running TPC-H across multiple threads and configuration settings. The authors mention a caveat of restricting duckDB performance for some benchmarks by using single thread.

Overall comments
I would like to thank the authors for their work. Please find some additional minor comments below:

Please move the figure out of the first page, or to the bottom of the first page. It is distracting to read the caption of Figure 1 before the abstract.
Please update the axes in Figure 7 to be legible.
One of the main topics of the paper is that DataFusion catalyzes the development of new data systems. Evaluation in that direction would help support the claims in the paper further. One related angle could be the ease of developing systems (applications) on top of DataFusion, potentially including the overhead in terms of lines of code or engineering hours in developing a simple system/application with DataFusion and using a different stack or being customly built. Similarly, performance evaluation of systems relying on DataFusion could help in this direction as well.
Similarly, the content of the paper would benefit from doing a deep dive into the query engine and a limited set of features based on how they are used by systems developed on DataFusion.

Ok, I did a final read / wordsmith / whitespace engineering on the last few sections. I will plan to do a proofreading pass or two over the next few days, but don't plan to change anything unless there is some grammatical issue.

The current copy of the paper is here:
DataFusion_Query_Engine___SIGMOD_2024 (1).pdf

(getting very close)

I am on the home stretch -- I plan to do a final proofreading of Section 8, 9, 10, and 11 and then submit the draft tomorrow

Thank you Andrew! Sorry I couldn't spent more time on the paper.
I'll try to review the draft this evening.

No problem! I think we are all quite busy -- I also want to submit the paper so I can let it go (my OCD tendencies would be to edit it indefinitely, which is not good :) )

Here is the digital rights form that was submitted: 13731_1_1.pdf

Here is the final draft:
DataFusion_Query_Engine___SIGMOD_2024-FINAL.pdf

Here are the source files:
DataFusion Query Engine - SIGMOD 2024-SOURCE.zip

I am working to upload this to the CMT tool -- once I get confirmation they got it and it is accepted I'll (finally!) close this issue. Thanks all

😅

We needed to make a few tweaks on the final manuscript to conform to the publisher's rules

---------- Forwarded message ---------
From: New Form Needed <submissions@scomminc.com>
Date: Fri, Apr 5, 2024 at 9:39 AM
Subject: ACM Proceedings (sigmodpods NF) - Submission Follow Up, ID No: modIP001
To: <alamb@apache.org>
Cc: <yjshen@apache.org>, <dheres@apache.org>, <jayjeetc@ucsc.edu>, <ozankabak@apache.org>, <viirya@apache.org>, <sunchao@apache.org>, <Jyoti.Leeka@microsoft.com>, <venkatesh.emani@microsoft.com>

Dear Author(s),
For the submission of "Apache Arrow DataFusion: a Fast, Embeddable, Modular Analytic Query Engine" to the proceedings and ACM Digital Library.

Please fix the following on your submission and re-submit on or before April 12th

1. A. The completed ACM rightsreview form was voided, per ACM policy and instructions provided all authors names, accents, initials, authorship order, dept/labs, affiliations, titles must match precisely on the PDF and ACM form: https://www.scomminc.com/pp/acmsig/sigmod-pods2024.htm#AO
https://www.scomminc.com/pp/acmsig/sigmod-pods2024.htm#GA 
The following on your PDF and the rightsreview form do not match (see the following to see the differences). The ACM rightsreview form link will be sent to the designed contact author again.

Form -- Liang-Chi Hsieh: Apple;
Chao Sun: Apple

Files -- Chao Sun: Apple;
Liang-Chi Hsieh: Apple

 B. To update the rightsreview form is best done on a computer by the designated contact author, not a mobile device;  C. Update the rightsreview form data accordingly to match your final version;  D. Then at the bottom click the check box, then click ''Save Paper and Author Details'';  E. Then click ''Proceed to eRights Form'';  F. Then complete the form entirely until the designated contact/corresponding author receives a new ACM confirmation email.

2. Authors 6 & 7. Per ACM policy, we can not allow bundled/bunched or grouped authors in a long string: https://www.scomminc.com/pp/acmsig/sigmod-pods2024.htm#GA and [www.scomminc.com/pp/acmsig/ACM-sample-sigconf-section6.pdf](http://www.scomminc.com/pp/acmsig/ACM-sample-sigconf-section6.pdf)

3. To make the ACM even page headers consistent for this year's conference publication (per the instructions: https://www.scomminc.com/pp/acmsig/sigmod.htm#Le), kindly use the following after authors and affiliation info in your .tex file:

\renewcommand{\shortauthors}{Andrew Lamb et al.}
%% No italics

4. Due to a recent change in the ACM guidelines, the $15.00 within the ACM CR block is no longer needed & not included in the ACM code provided, please comment this out on your new version.
%\acmPrice{15.00}

Then review the new version of your submission for any typos and then submit/upload all the files (required pdf, required source, etc) to update your pdf/submission on or before deadline above to the following link:

<REDACTED>

Thank you,

New Form Needed

Sheridan Communications

I made the requested edits (layouts of author names, remove $15, and update the short authors and submitted a new draft:

Here is the next draft

DataFusion Query Engine - SIGMOD 2024-SOURCE-mk2.zip
DataFusion_Query_Engine___SIGMOD_2024-FINAL-mk2.pdf

(BTW I am sharing this not because I think anyone really cares, but because I thought others might be interested in how this process works)

And apparently I still didn't get it entirely correct:

---------- Forwarded message ---------
From: Tim Pollitt <submissions@scomminc.com>
Date: Tue, Apr 23, 2024 at 8:46 AM
Subject: ACM Proceedings (sigmodpods TP) - Submission Follow Up, ID No: modIP001
To: <alamb@apache.org>
Cc: <yjshen@apache.org>, <dheres@apache.org>, <jayjeetc@ucsc.edu>, <ozankabak@apache.org>, <viirya@apache.org>, <sunchao@apache.org>

Dear Author(s),
For the re-submission of "Apache Arrow DataFusion: a Fast, Embeddable, Modular Analytic Query Engine" to the proceedings and ACM Digital Library.

Please fix the following on your submission and re-submit on or before April 25th

1. Please update the author order on your PDF to match what was listed on your ACM Rightsreview form:

Andrew Lamb, Yijie Shen, Daniël Heres, Jayjeet Chakraborty, Mehmet Ozan Kabak, Liang-Chi Hsieh, & Chao Sun

*Kindly note that per ACM policy, this data must match in both locations.

**Set reminders for the chairs' requested optional presentation video (see original prep instructions link https://www.scomminc.com/pp/acmsig/sigmod.htm).

 Then review your PDF submission for any typos and then submit/upload all the files (required pdf, required source, optional thumbnail etc) to update your pdf/submission on or before deadline provided above to:

<LINK>
Thank you,

Tim Pollitt

Sheridan Communications

"third time's the charm"

DataFusion Query Engine - SIGMOD 2024-FINAL-mk3.zip

DataFusion_Query_Engine___SIGMOD_2024-FINAL-mk3.pdf

Needed to tweak the title to have a A rather than a

---------- Forwarded message ---------
From: Tim Pollitt <submissions@scomminc.com>
Date: Mon, Apr 29, 2024 at 8:47 AM
Subject: ACM Proceedings (sigmodpods TP) - Submission Follow Up, ID No: modIP001
To: <alamb@apache.org>
Cc: <yjshen@apache.org>, <dheres@apache.org>, <jayjeetc@ucsc.edu>, <ozankabak@apache.org>, <viirya@apache.org>, <sunchao@apache.org>

Dear Author(s),
For the re-submission of "Apache Arrow DataFusion: A Fast, Embeddable, Modular Analytic Query Engine" to the proceedings and ACM Digital Library.

Please fix the following on your submission and re-submit on or before May 1st

1.   Kindly make the ''A'' after the colon in the title a capital letter ''A''. This is a valid exception to the rule.

*Set reminders for the chairs' requested optional presentation video (see original prep instructions link https://www.scomminc.com/pp/acmsig/sigmod.htm).

 Then review your PDF submission for any typos and then submit/upload all the files (required pdf, required source, optional thumbnail etc) to update your pdf/submission on or before deadline provided above to:

http://www.scomminc.com/acm/submissions/submission.cfm?grid=sigmodpods&eid=5E5C547F61070302&eid2=6702

Thank you,

DataFusion Query Engine - SIGMOD 2024-FINAL-mk4.zip
DataFusion_Query_Engine___SIGMOD_2024-FINAL-mk4.pdf

Needed to tweak the title to have a A rather than a

I remember we have discussed this A/a issue before in emails. We got the answer now. 😄

Thanks for dealing with the tweak.

Needed to tweak the title to have a A rather than a

I remember we have discussed this A/a issue before in emails. We got the answer now. 😄

Thanks for dealing with the tweak.

No problem -- it seems like this process is still quite manual 😆 -- hopefully we have it all right now 🤞

I think it is finally accepted 🎉

---------- Forwarded message ---------
From: Tim Pollitt <submissions@scomminc.com>
Date: Wed, May 1, 2024 at 2:44 PM
Subject: ACM Proceedings (sigmodpods TP) - Submission Follow Up, ID No: modIP001
To: <alamb@apache.org>
Cc: <yjshen@apache.org>, <dheres@apache.org>, <jayjeetc@ucsc.edu>, <ozankabak@apache.org>, <viirya@apache.org>, <sunchao@apache.org>

Dear Author(s),
For the re-submission of "Apache Arrow DataFusion: A Fast, Embeddable, Modular Analytic Query Engine" to the proceedings and ACM Digital Library.

See below **** for info related to your final pdf.

In addition, authors are requested by the chairs to submit an optional presentation video.

(1) For more info, see instructions for details, and inquiry link to chairs:   https://www.scomminc.com/pp/acmsig/sigmod-video.htm by May 6th.

(2) You will need the DOI assigned to your submission: 10.1145/3626246.3653368

(3) Video descriptions over the 1,024 ACM character count will be deleted/removed by ACM.

*************

 We received notice of your completed ACM form, so everything appears to be in order with this submission, and will be utilized for the ACM DL publication. No revisions are needed, nor will be accepted. This submission is being moved onto the next stages of production (but kindly note the signed ACM forms still go through a double-check by ACM for various permissions).

Thank you,

Tim Pollitt

Sheridan Communications

Navigation Menu

Repository navigation

Finalize SIGMOD 2024 paper ~(if accepted)~ #8373

Describe the solution you'd like

Describe alternatives you've considered

Additional context

28 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Finalize SIGMOD 2024 paper ~(if accepted)~ #8373

Description

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Activity

alamb commented on Jan 9, 2024

vertexclique commented on Jan 9, 2024

alamb commented on Feb 1, 2024

alamb commented on Feb 2, 2024

ozankabak commented on Feb 2, 2024

viirya commented on Feb 2, 2024

alamb commented on Feb 4, 2024

viirya commented on Feb 4, 2024

sunchao commented on Feb 4, 2024

JayjeetAtGithub commented on Feb 4, 2024

matthewmturner commented on Feb 5, 2024

liurenjie1024 commented on Feb 5, 2024

alamb commented on Feb 6, 2024

28 remaining items

alamb commented on Mar 24, 2024

alamb commented on Mar 27, 2024

Dandandan commented on Mar 27, 2024

alamb commented on Mar 27, 2024

alamb commented on Mar 28, 2024

alamb commented on Mar 28, 2024

alamb commented on Apr 6, 2024

alamb commented on Apr 6, 2024

alamb commented on Apr 23, 2024

alamb commented on Apr 23, 2024

alamb commented on Apr 29, 2024

viirya commented on Apr 29, 2024

alamb commented on Apr 30, 2024

alamb commented on May 1, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions