KEMBAR78
AUDIO: Add support for manually setting sounds' sample rate by fracturehill · Pull Request #4965 · scummvm/scummvm · GitHub
Skip to content

Conversation

@fracturehill
Copy link
Contributor

@fracturehill fracturehill commented May 1, 2023

One of the Nancy Drew games has a puzzle that changes sounds' sample
rates to arbitrary values while they're playing. The initial plan for supporting
this feature was to create a custom AudioStream subclass; however, all
my attempts at doing so resulted in crackling and overly complicated code.

This PR is based around a rewrite of the RateConverter class merging
the previously separate CopyRateConverter, SimpleRateConverter,
and LinearRateConverter subclasses into a single class that supports
setting the input and output rates after creation. The optimized
code paths of those classes have been moved to separate functions,
with the correct one being picked on the fly.

The Mixer class has been upgraded to make use of the new feature,
so all an engine needs to do to change the playback speed of a sound is
to call the newly-added setChannelRate() function.

@ccawley2011
Copy link
Member

I'm a bit worried that this will hurt performance on low-end devices since this PR removes the fast paths for simple conversions. I'm also not sure why the use of templates has been removed, since that doesn't appear to be related to the sample rate, and removing it makes future changes I have planned more complex.

A potential solution would be to have one class that has the functions from each of the old classes as private functions, and have the convert function select the best one based on the configured sample rate.

@fracturehill
Copy link
Contributor Author

@ccawley2011 The templates were removed since they essentially 'lock in' the type of RateConverter at creation. Suppose you have a Copy/SimpleRateConverter whose rate you want to adjust to an arbitrary value: this just wouldn't work, since the interpolation code is in an entirely different class. A solution would be to destroy the old RateConverter and replace it with a new one, but then you also need to carry over the internal buffer and state... which already I tried doing, with poor results.

I'm not at all opposed to bringing the old code paths back in, though. I opted to remove them since the different specializations used the internal variables in slightly different ways that could get tangled up when switching, but with a bit more work they could definitely be brought back.

@ccawley2011
Copy link
Member

@ccawley2011 The templates were removed since they essentially 'lock in' the type of RateConverter at creation.

I was referring to the use of templates for specifying the stereo parameters, rather than the multiple subclasses for different sample rates.

I'm not at all opposed to bringing the old code paths back in, though. I opted to remove them since the different specializations used the internal variables in slightly different ways that could get tangled up when switching, but with a bit more work they could definitely be brought back.

👍

@fracturehill
Copy link
Contributor Author

I was referring to the use of templates for specifying the stereo parameters, rather than the multiple subclasses for different sample rates.

@ccawley2011 I see. There wasn't a particular reason for removing those, they just became victim of overeager refactoring :) I'll work on both of your points as soon as I've got the time, until then I'm marking this as a draft PR.

@fracturehill fracturehill marked this pull request as draft May 2, 2023 10:48
@fracturehill fracturehill force-pushed the variable-rate-audio branch from b6ac595 to 4d8ab46 Compare May 6, 2023 14:52
@fracturehill fracturehill marked this pull request as ready for review May 6, 2023 15:13
@fracturehill
Copy link
Contributor Author

I've re-added the optimized code paths and the templatization of the stereo parameters. From what I've tested, switching between copying and interpolation seems to work fine, without any audible distortion. I'm not entirely sure where simple conversion is useful, so I haven't been able to test it yet.

@hari01584
Copy link
Contributor

This is interesting, however, the changes are huge and this needs a good review and testing before getting merged I believe. Anyways just wondering, why change flow function name to convert?

@ccawley2011
Copy link
Member

I'm not entirely sure where simple conversion is useful, so I haven't been able to test it yet.

It's used when the input sample rate is a multiple of the output sample rate. For example, when converting 44100Hz to 22050Hz or 11025Hz. This should be easy to test with the SDL backend using the --output-rate command line option or by setting the output_rate key in the configuration file.

The RateConverter class has been modified to allow for
variable input and output rates. The optimized code paths
for copy/simple conversions are retained, but have been moved
inside separate functions instead of subclasses. The templatization
of the stereo parameters has been maintained, and is implemented
via the newly-added RateConverter_Impl template class.
Internal variables have been renamed to be more readable. The
flow() function has been renamed to convert(). The drain()
function has been removed, since it was never implemented
or used anywhere.
Extended the Mixer class to support manually setting the
sample rate of a channel.
@fracturehill fracturehill force-pushed the variable-rate-audio branch from 4d8ab46 to 927e2a3 Compare May 8, 2023 21:45
@fracturehill
Copy link
Contributor Author

@ccawley2011 Thanks, I did not know about that flag. Testing with it revealed an error I introduced when renaming variables, which has now been fixed. I've tested switching between all code paths and everything seems to work correctly now :)

@hari01584 The function was named flow() to support the flow/drain dichotomy. However, since I'm removing the drain() function (which was just a nonfunctional stub), leaving flow() named like that would just make it more confusing.

@sev-
Copy link
Member

sev- commented May 14, 2023

Thanks!

@sev- sev- merged commit 0f045a4 into scummvm:master May 14, 2023
hari01584 added a commit to hari01584/scummvm that referenced this pull request Jun 23, 2023
Uses Mixer's setRate() function implemented in pr scummvm#4965 to set
video playback rates of movies that are not 0 or 1.

`undome` of 'mediaband' required this feature where the video can
have playback rates other than 0 or 1.
hari01584 added a commit to hari01584/scummvm that referenced this pull request Jun 24, 2023
Uses Mixer's setRate() function implemented in pr scummvm#4965 to set
video playback rates of movies that are not 0 or 1.

`undome` of 'mediaband' required this feature where the video can
have playback rates other than 0 or 1.
hari01584 added a commit to hari01584/scummvm that referenced this pull request Jun 24, 2023
Uses Mixer's setRate() function implemented in pr scummvm#4965 to set
video playback rates of movies that are not 0 or 1.

`undome` of 'mediaband' required this feature where the video can
have playback rates other than 0 or 1.
hari01584 added a commit to hari01584/scummvm that referenced this pull request Jun 24, 2023
Uses Mixer's setRate() function implemented in pr scummvm#4965 to set
video playback rates of movies that are not 0 or 1.

`undome` of 'mediaband' required this feature where the video can
have playback rates other than 0 or 1.
sev- pushed a commit that referenced this pull request Jun 24, 2023
Uses Mixer's setRate() function implemented in pr #4965 to set
video playback rates of movies that are not 0 or 1.

`undome` of 'mediaband' required this feature where the video can
have playback rates other than 0 or 1.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants