You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
CometTaskMemoryManager logs a warning and a memory dump every time a native reservation is refused #6257
Native operators ask for memory with try_grow until they are refused, and then they spill. A partial grant from Spark is the normal spill signal, not an error. But CometTaskMemoryManager.acquireMemory logs a warning and calls TaskMemoryManager.showMemoryUsage() every time Spark grants less than was asked (CometTaskMemoryManager.java#L74-L85), and showMemoryUsage() logs another line per consumer at INFO. In a four-task aggregate with 96 MB of off-heap memory, that was 257 warnings in about five seconds. A query that spills heavily keeps producing them for its whole run. That buries real warnings and makes a healthy spill look like a failure.
Describe the potential solution
Log the partial grant at DEBUG, or at most once per task at INFO. Nothing is lost when a reservation really does fail: the error the pool returns already says how much was granted and lists the top consumers.
Activity
fix: log partial memory grants at DEBUG and drop the memory usage dump (
fix: log partial memory grants at DEBUG and drop the memory usage dump (
fix: log partial memory grants at DEBUG and drop the memory usage dump (