add auto input type for content based format detection - #61
Conversation
Reading from stdin still needs -i today, which is awkward when you don't know the format up front (the classic case being kubectl -o yaml | qq). This adds an opt-in "auto" input type that sniffs the content and picks json, yaml, toml or xml/html. Detection stays conservative: it only commits to a format when a strict parse confirms it, and errors out otherwise so we never silently guess wrong. Default behaviour is unchanged since it only runs when you pass -i auto. Closes JFryy#34
|
Hi @ChrisJr404, Thanks for submitting this, I definitely like the compromise of the file type set to auto or an auto flag in general. I think some of the pattern matching could be expanded to be more robust but I think it's a good minimal start on cursory glance. Will have a further look on filetype detection patterns and breadth but let's get this implemented. |
|
Thanks for the quick look. Agreed the detection patterns can grow over time; happy to expand the matching in a follow-up once this lands. Let me know if you'd like any tweaks before then. |
JFryy
left a comment
JFryy
left a comment
There was a problem hiding this comment.
Nice work, the conservative approach and the stream guard are a good move. One interaction worth flagging before this ships in a release: -i auto doesn't compose with -s/--slurp for concatenated JSON, which is the main slurp use case:
printf '{"a":1}\n{"b":2}\n' | qq -i auto -s .
# could not detect input format, please specify it with -i/--inputDetect validates the whole buffer with json.Valid, and a stream of JSON values isn't a single valid value, so it falls through and errors. A small fix would be to fall back to a json.NewDecoder loop in the '{', '[' branch that accepts two or more concatenated values, which stays strict while covering the slurp input.
Happy to merge this as-is and take that as a follow-up since I'm planning a broader heuristics pass before the next release, but if you'd rather fold it into this PR that works too.
2 participants
Picks up #34. Reading from stdin still needs
-itoday, so something likekubectl get pod my-pod -o yaml | qqfails because the default assumes json.You mentioned wanting to keep this explicit rather than have qq guess by default, so this is opt-in only: you get detection when you pass
-i auto, and nothing changes otherwise. The detector is deliberately conservative, it only commits to a format when a strict parse confirms it (json, yaml, toml, xml/html) and otherwise returns an error asking you to specify-i, so it will not silently pick the wrong thing. Streaming still needs an explicit type since detection has to buffer the input first.Added unit tests for the detector including the undetectable cases, and the existing tests/test.sh suite still passes.