[ggcrunchy]

In the lead-up to some previous graphics submissions I often mentioned wanting to do "texture stuff".

A pretty broad umbrella!

This PR is a significant, but nowhere near complete, step in that direction.

It's a "second draft" of FPTexture, which still has some unintegrated bits: GLES3 groundwork; mipmap and linear filtering details; pre-defined extended formats. Those have problems but remain viable future directions to pursue, and still have good parts that can be cannibalized.

(FP = "floating-point", and that branch quickly outgrew its name too.)

This PR's branch subsumes the work from #918 and #920, which were rolled out in stages in the vain hope of not dropping a big monolith. 😃 😃 😃 Some of this material has implications for ANGLE as well.

Some of this work found its way into recent bugfix patches. These represent most, if not all, of the current merge conflicts. (EDIT possibly now addressed.)

---

Tests / R&D: tests.zip

The TinyCC compiler has been invaluable for rapid testing, and I've attached the relevant tests, in their current and often messy forms.

I've lately been making some effort at documenting the new bits belonging to this PR and some other branches, as well as other long-neglected things from older and already-integrated submissions, and leaving stub pages where I haven't gotten to them yet. My very partial effort can be found as Markdown and HTML.

Relative to the docs homepage, the pertinent bits are in /api, though only one or two new things, and then in some of the types, e.g. in graphics.newTexture() there is now some stuff about capture textures. Other entries have new material, like graphics.defineEffect(), system.getInfo() and display.*etDefault(). There are some stubs in /guides, though I haven't mustered the will to write any of that. A huge amount has been added to the rather obscure /native/Cpath, although it still has far to go.

---

WHAT'S NEW

• Reorg of some GL details #918 bits:

  Removed lots of dead code and gathered in stray OpenGL bits. This was originally an organizational thing for ANGLE, and serves a similar need here. There are some "is this OpenGL ES?" assumptions that ES3 will upset, or vice versa; this is meant to make that tractable when the time comes, and in the same spirit to deal with extensions in the meantime.

• Revisions of Texture and PlatformBitmap Format types #920 bits:

  One of the big ideas of this whole effort is to add the ability to use the many additional texture formats, if and when available. There are some occasions where a texture—more specifically, its bitmap data—want to know certain properties, e.g. "do I have alpha?", but there isn't a convenient way to look these up.

  Only a few bits are needed for the texture type itself, so a lot of metadata can be bundled for free, and this also allows a way to pass around a quite reasonable number of extra formats.

  (There has probably been some feature-creep in needed-for-metadata-versus-exhaustive-detail here.)

• GPU resources are given a "render context" during Create() and Update() for some minor insights into the CPU-side world.

• External textures can now use a version 2 callbacks type that takes a linked list of extensions.

  Currently this implements custom formats. Floating-point, compressed DXT, and integer textures, for instance, are each shown in the attachment's examples.

  Texture targets—the "shape" in the branch's name—are planned, and other miscellany like subdata uploads have been considered here. A one-component format will also respect a first getFormat() call indicating that we want to use it as a mask; otherwise custom formats ignore that callback.

  (In the earlier FPTexture work the v2 callbacks struct was quite monolithic, trying to support KTX in the test case, but it became clear too much was optional and it was annoying to dummy so much out.)

  There are some CoronaGraphics.h APIs for adding custom formats, to be used here. Also in that earlier branch, I was adding them to core, but I thought I'd peel that back in order to not make this a huge PR (HA!). In any case, it'll probably be good to grind through them all again, and then we can graduate battle-tested results to core. (In theory these APIs can accommodate other backends, too.)

• In addition to the choices mentioned above (floating-point, integer, compressed), there is some untested work for word-packed formats and plans for depth-stencil ones. Other possibilities like certain video formats are an open question.

• Named extra textures: samplers may be declared in shaders and then paired with paints of the same name. (The paint object may also offer more paints, so long as the textures can be satisfied.)

  At the moment, composite paints provide this support, via an extraPaints table. Perhaps this limit could be relaxed; just a time and effort thing.

  You can declare sampler types other than sampler2D, but they must agree. The Slug-based "text" example uses a usampler2D, for example... or did, though now it employs a "tweak" (see below), but anyhow it tested the feature at one point.

  This entailed some rather significant work in Renderer, for consistency checks and fallbacks in the case of failure.

• Added a classifier for sampler / image uniform types, along with metadata for the same, taking Slang's GL types as authority. There is also a second classifier for the remaining types, and its own corresponding metadata.

  These open the door for other u_UserData* types, but the GL calls are annoyingly type-specific, so it'd be quite a slog.

  The "luts" and "ulut" tests were used to generate and hash out the classifers and lookup tables.

• Added the languageExtensions option to graphics.defineEffect(), to use extension-specific features in a given effect. (TODO Some instancing setup in GLProgram.cpp should be backported to this.)

• Added shellTweaks, to rewrite certain bits of the (GL) shader shell, with a guard against combining them with shell transforms. These probably supersede the useful cases of shell transforms entirely. As one example of them at work, you can also retype the built-in fill samplers, e.g. the "text" example does this: sampler1Type = "usampler2D". The "stuff_3" and "stuff_3D" tests have also been ported to use these.

  (As mentioned, tweaks seem to be outright better than shell transforms, and the latter probably want deprecation. Effect details seem to have been a misfeature as well. The vertex extension C API—currently excepting "undefine"—also doesn't seem to hold any advantage over graphics.defineVertexExtension(). My lack of coverage in the linked docs for those APIs comes out of these considerations.)

• I forget where, but there was a varying being read somewhere in the shell, which is a no-no per spec.

• The effect data type getData()/setData() behavior has been tightened up.

• FormatExtensionList was given an overhaul. Some of this was to make the method names and such clearer. It also now avoids a lot of pointless memory allocations, especially in light of the fairly low strict limits on maximum attribute count, and has a far less scattershot Build() process as well.

  The earlier work would also take a hash of a requested attribute, but blindly assume it was valid; the new version maintains a Base64-ish packed name list (cf. "name_list" in the attachment) to enforce existence.

  The same packing utlities were used in the sampler names / paint work, and have landed in String.

• There were a bunch of queries that passed from Display through Renderer to the command buffer, and it was annoying to add new ones. These are now rebuilt around one enum and function. (Currently these are all static but maybe Vulkan will upset that a little?) This came up when adding system.getInfo( "maxExtraTextureUnits" ).

• Fix for vertex texture image units count.

• At startup, a search is done for GL_RED support, and this is favored for masks, and mask canvases in particular.

FUTURE DIRECTIONS

There is much more to do, and this PR is mostly to get a decent slice of stuff out there.

Some things from FPTexture were mentioned above, and different texture targets ("shapes") would open up more: array textures, buffer samplers, etc. especially with ES3+ support. Multiple render targets and compute shaders are also goals.

The earlier graphics work and the TinyCC "stuff_3D" examples derived from them do use depth, and thus depth textures themselves are obvious follow-ons, as well as stencil support.

I've looked a bit into doing transient texture uploads, where you don't keep the buffer around but maybe at most a 1-bit alpha cutout (if you still want graphics outlines or masked hit tests, say), say via some extra "preload"-type flags to graphics.newTexture(). These would aim to cut down on the giant memory costs of keeping that mostly-useless data around. A further wrinkle would be to allow an operation to be installed and use a default auto-compress operation (e.g. see the STB / SOIL2 tests; or Goofy; or even professional solutions like Spark) before delivering the upload. (ES2 would be annoying here, in that any alpha has to be maintained separately; at this point it's probably safe to just say it doesn't work there.)

KTX / KTX2 and / or Basis as first-class file format(s) is probably also wanted, but presupposes a decent amount of stuff supported.