[feature request] Support lights defined in glTF

[michaliskambi]

There are glTF 2.0 extensions to define lights inside glTF file. It would be very useful for me (and Castle Game Engine :) ) if PasGLTF supported reading such lights.

1. Punctual lights (point, spot, directional) are defined in
   https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Khronos/KHR_lights_punctual .

   Blender can export such lights. I made example in CGE demo-models, https://github.com/castle-engine/demo-models/tree/master/gltf/punctual_lights . The test_lights.gltf inside defines 3 lights (point, spot, directional).

2. Image-based lighting is defined in
   https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Vendor/EXT_lights_image_based .

   Example of it is in
   https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/EnvironmentTest .

[BeRo1985]

KHR_lights_punctual, KHR_TEXTURE_TRANSFORM and KBR_materials_sheen are supported by the PasGLTF example viewer now (see https://github.com/BeRo1985/pasgltf/blob/dc27732d961a5886fde8989172fddc44a09db6c7/src/viewer/UnitGLTFOpenGL.pas and https://github.com/BeRo1985/pasgltf/blob/dc27732d961a5886fde8989172fddc44a09db6c7/src/viewer/UnitOpenGLShadingShader.pas ).

KBR_materials_clearcoat and EXT_lights_image_based are in the work and on my todo-list.

[michaliskambi]

Thank you very much! I'll look over it during the weekend.

One comment I would have is that I see you implement them by extending UnitGLTFOpenGL.pas. From my use-case, extending PasGLTF.pas unit to expose this information would be more immediately useful. To have "lights" available in an easily accessed structure, e.g. like "cameras".

CGE is using PasGLTF unit to read glTF. Then we convert it to X3D and use our own renderer on top of X3D nodes.

[BeRo1985]

In my opinion, the PasGLTF unit itself should remain independent from extensions by simply passing all extras and extensions 1 to 1 as JSON objects, so that the whole PasGLTF unit remains flexible and easier to maintain, so that it parses only the whole stablized/fixed core GLTF 2.0 spec itself, but not the extensions nor the extras .

Because if one reads through the commits in the Khronos-GLTF-Spec-Repo, one sees that extensions can change very quickly. For example KHR_lights_punctal was previously KHR_lights with different definitions etc. The same applies to other rather non-stable/non-fixed extensions like KHR_marterials_clearcoat, KHR_marterials_sheen, KHR_marterials_specular and so on.

In a nutshell and in my observation, GLTF is too much in a state of constant evolution and constant change for this with regard to the extensions.

[michaliskambi]

OK, I see your point.

How about adding a unit like PasGLTFExtensions then? A proposed API would be:

For each extension (like KHR_lights_punctal) it defines a class that takes care of parsing GLTF information, and exposing it in nice Object Pascal-friendly fashion. E.g.

type
  TGLTFKhrLightsPunctual = class
  public
    type
      TLightDataType = ... // move here definition from UnitGLTFOpenGL
      TLight = ...  // move here definition from UnitGLTFOpenGL
      TLights = array of TLight;
    var
      Lights: TLights;
    constructor Create(aDocument: TPasGLTF.TDocument);
  end;

So the TGLTFKhrLightsPunctual constructor would read light information from already-loaded glTF document (very similar to how TGLTFOpenGL.LoadFromDocument is doing) and initialize the Lights array properly.

This way other units (like the viewer in UnitGLTFOpenGL, and Castle Game Engine) could instantiate TGLTFKhrLightsPunctual class to parse the relevant information, and use it.

My reasons for this is that I would very much like to use your code in Castle Game Engine, for glTF handling :) Currently I can happily use the unit PasGLTF. But I cannot really use UnitGLTFOpenGL, since our glTF reader is converting PasGLTF into X3D nodes, and then we have own renderer for X3D nodes.

[BeRo1985]

Well, what would be wrong with copying and adapting the KHR_lights_punctal parsing and loading code from UnitGLTFOpenGL for your usage in CGE? Because then you would be directly also prepared for the future, if you want to implement further GLTF extensions yourself on your own initiative, once you know how to load extensions with PasGLTF. Just my two cents. 😀

[michaliskambi]

I'm going to do that then :)

The reason I instead proposed PasGLTFExtensions unit is that this way both projects would share a common code parsing extensions. Which in practical terms means that Castle Game Engine can immediately benefit from new code on your side. Right now I'll copy + adjust your code from UnitGLTFOpenGL, which means that both our projects will implement similar logic independently.

In any case, you gave me a great headstart, since I can look and see how you did everything in UnitGLTFOpenGL, so I'm not complaining. Thanks! I'll give you a note when I'll implement these extensions in CGE (I may end up adding a unit like PasGLTFExtensions to CGE, so it may be useful then to you.)

[BeRo1985]

I am restructuring the whole PasGLTF viewer example also in the moment, for modern newer OpenGL features, as such as for example for the usage of bindless textures, so that the fragment shader will have direct access to all loaded textures at every time point then, without oldschool glBindTexture calls on CPU-side just per a sampler2D textures[]; array in an uniform buffer object or shader storage buffer object. ( https://www.khronos.org/registry/OpenGL/extensions/ARB/ARB_bindless_texture.txt ) And I've also planned to pack all the materials data and GLTF node transform data then into two large shader storage buffer objects, so that a whole GLTF scene could be rendered then with just two OpenGL draw calls (one for the shadow map and one for the final geometry rendering)

[BeRo1985]

Okay, three to four draw calls, when transparent and alpha-tested materials are also existent then. 😀