I've been profiling genPOI performance (on a single-threaded machine). One of the slow parts is reading/parsing data out of the chunks; I believe much of that data is unnecessary for genPOI.
Profiling result:
genPOI 209.85s (partial)
entities 209.84s (partial)
chunk 4616 * 0.00s = 9.93s (max 0.09s)
get_chunk 4616 * 0.00s = 9.78s (max 0.09s)
load chunk from region 4616 * 0.00s = 4.66s (max 0.08s)
create NBTFileReader 4616 * 0.00s = 4.43s (max 0.08s)
create StringIO for NBT 4617 * 0.00s = 0.04s (max 0.01s)
zlib decompress 4616 * 0.00s = 1.23s (max 0.01s)
remainder 0.68s
seek 4616 * 0.00s = 0.03s (max 0.00s)
biomes 4616 * 0.00s = 0.06s (max 0.01s)
blocks 24181 * 0.00s = 0.43s (max 0.01s)
skylight 24181 * 0.00s = 1.22s (max 0.01s)
blocklight 24181 * 0.00s = 1.17s (max 0.01s)
data 24181 * 0.00s = 1.15s (max 0.01s)
iterate over chunk 4616 * 0.00s = 0.04s (max 0.00s)
remainder 1.40s
Note especially that reading blocks, skylight, and blocklight contribute nontrivially to get_chunk, but only the result of data is used.
I'll prepare a pull request that adds an option to get_chunk to only read data. In my testing that reduces genPOI time by about a third, from about 300s to about 200s.
The other good optimization candidate looks like parsing data out of the NBTFile, which does not look like as easy a win. In that case I assume a C implementation would be faster; and the same approach as here (loading only required data) would probably be an improvement but the approach is not as clear.
I've been profiling genPOI performance (on a single-threaded machine). One of the slow parts is reading/parsing data out of the chunks; I believe much of that data is unnecessary for genPOI.
Profiling result:
Note especially that reading blocks, skylight, and blocklight contribute nontrivially to get_chunk, but only the result of data is used.
I'll prepare a pull request that adds an option to get_chunk to only read data. In my testing that reduces genPOI time by about a third, from about 300s to about 200s.
The other good optimization candidate looks like parsing data out of the NBTFile, which does not look like as easy a win. In that case I assume a C implementation would be faster; and the same approach as here (loading only required data) would probably be an improvement but the approach is not as clear.