Skip to content

ndjson-split output artifacts in PowerShell #39

Open

Description

@JaimeStill

Inspired by the Command Line Cartography series, I started out implementing the same sequence for the state of Texas (this issue relates to step 8 in that sequence). Following this, I decided to automate this process by writing a PowerShell script that generates a choropleth based on a provided FIPS code.

With this in mind, I ran into a pretty big roadblock when trying to generate the NDJSON version of the census data. I've created a GitHub Repo to demonstrate the issue, but here is the short of it:

When calling ndjson-cat "{json-file}" | ndjson-split "d.slice(1)", I'm getting odd output artifacts in PowerShell:

resulting .ndjson

E:\Jaime\Desktop\Work\Projects\ps-mapping-issue>node  "C:\Users\jpsti\AppData\Local\Yarn\Data\global\node_modules\.bin\\..\ndjson-cli\ndjson-split" d.slice(1) 
["1201","01","001","021000"]
["1293","01","001","021100"]
/* additional data removed */
["1880","01","133","965800"]
["834","01","133","965900"]

E:\Jaime\Desktop\Work\Projects\ps-mapping-issue>

I initially realized there was an issue when I would pipe this into the ndjson-map call directly after ndjson-split, and it was providing me with this error:

stdin:1

^
SyntaxError: Unexpected end of JSON input

If I run this from a command prompt instead, the desired output is provided.

The following block works just fine in PowerShell:

shp2json "maps/$($shape)/$($shape).shp" | `
      geoproject "d3.$($state.projection).fitSize([$width, $height], d)" | `
      ndjson-split "d.features" | `
      ndjson-map "d.id = d.properties.GEOID.slice(2), d" > "data/$($state.name).ndjson"

The only thing I can think of that would be different in this scenario is that this call to ndjson-split is simply accessing a property of the data, as opposed to calling a JavaScript Array function.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions