-
-
Notifications
You must be signed in to change notification settings - Fork 23.4k
Add KHR_audio_emitter support to the GLTF module #88204
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
modules/gltf/extensions/audio/gltf_document_extension_audio.cpp
Outdated
Show resolved
Hide resolved
modules/gltf/extensions/audio/gltf_document_extension_audio.cpp
Outdated
Show resolved
Hide resolved
b75f5e4
to
22aa3ad
Compare
We are waiting for the ratification of the various gltf extension standards. |
Marking as draft to prevent accidental merging before ratification. However, the code is complete and ready for review. |
22aa3ad
to
7e5871c
Compare
7e5871c
to
df2f275
Compare
df2f275
to
114b80e
Compare
772bb1c
to
8bfbb96
Compare
We're currently looking at adding Is it intentional that currently, embedded audio buffers don't seem to be supported? On playback of such a file, I get an error: ![]() The file in question: 20250404-UnityGLTF-Audio.glb.zip Godot seems to correctly see that there's an audio buffer embedded, its name and settings, but can't play it back. But our export implementation is still WIP – so it could also be that the file isn't valid in some way, I'm not sure if there's another player/implementation that supports the extension in its current form. |
@hybridherbst Thank you for the test file. However, the problem is in the MP3 import code, not in the glTF importer. ![]() As a test, I tried to save the data to a MP3 file and got this: test.mp3.zip. This correctly plays inside of VLC Media Player, but Godot is unable to import it: ![]() The Godot MP3 import code is pretty trivial, it just calls into the MiniMP3 third-party library. The function returns an error code of 0 (so, no error), but the Hertz of the file is zero. I checked every function in MiniMP3 that contains Anyway, it's not a bug in the glTF code. If Unity exports a glTF with an MP3 that Godot could normally read as a standalone file, then that same audio data should work inside of a glTF. |
Thanks for looking at it! – turns out that the actual issue was a wrong mime type, the file specified mp3 but actually was a WAV. This made VLC and others that don't care so much play it back, but Godot (rightfully) rejected it. So the file you extracted would have needed to be renamed to .wav to work... Here's an updated file with an mp3 and correct mime type: |
I have critical concerns that WAV is in the spec and would cause portability issues with the Khronos specification. |
@fire What issues? The corrected model from @hybridherbst loads fine. |
fa63d18
to
e126148
Compare
adce79c
to
95fa474
Compare
@coderofsalvation what you refer to as "the spec" is an unofficial proposed pull request by OMI (which many of us participate in) against a draft KHR spec. Unfortunately Khronos has been extremely slow formally ratifying or publishing this particular extension, but that means while very solid and implementable, the draft extension on a non-khronos repository cannot be treated as canon. I believe @fire is referencing some contention related to certain formats from some of the Khronos meetings, not all of which is visible publicly. I attended one such public meeting over two years ago where the subject of audio formats came up, but I am not sure about the current state so this is speculation on my part. anyway I just wanted to illustrate some of the context here. The KHR audio spec has been in work for over 4 years so there is quite a long history... |
95fa474
to
bf0f0b5
Compare
bf0f0b5
to
399b0d3
Compare
@lyuma thanks for providing context |
Implements this proposal godotengine/godot-proposals#8814
This PR adds support for audio import and export in the GLTF module using the not-yet-finalized KHR_audio_emitter GLTF extension (KhronosGroup/glTF#2137). This allows you to save audio inside of GLTF scenes, and load them back later. In the future this will also allow you to use GLTF as an interchange format between game engines.
You can try out a Three.js implementation here: https://omigroup.github.io/three-omi/
Some example use cases: a fountain that makes water noises, a gun that makes custom sounds when fired, the radio from Portal that plays music on a loop, or a tree that includes bird chirping noises or rustling leaves or something. For more details about the intended use cases, see the proposal.
Freely licensed example file: https://github.com/omigroup/gltf-extensions/tree/main/extensions/2.0/KHR_audio_emitter/examples/boom_box This file contains a boom box, a short looping music clip, OMI physics, and licensing information via
KHR_xmp_json_ld
. The model is CC0, created by The Khronos Group, and the music is CC-BY 3.0, created by Kevin MacLeod.The code in this PR is ready for review, but note that the extension is not yet finalized. Usually my approach is for us to be pioneers with Godot and implement extensions that may not be finalized, like with OMI physics. However, for this extension, since it is using the
KHR_
namespace, we must tread carefully and avoid shipping features in Khronos's namespace into production without Khronos's approval. Don't put words in their mouth, so to speak.Production edit: closes godotengine/internal-team-priorities#44