KEMBAR78
[tests] refactor vae tests by sayakpaul · Pull Request #9808 · huggingface/diffusers · GitHub
Skip to content

Conversation

@sayakpaul
Copy link
Member

@sayakpaul sayakpaul commented Oct 30, 2024

What does this PR do?

Internal thread: https://huggingface.slack.com/archives/C065E480NN9/p1730203711189419.

Tears apart test_models_vae.py to break the tests in accordance with the Autoencoder model classes we have under src/diffusers/models/autoencoders.

Didn't include Allegro as it's undergoing some refactoring love from Aryan. Discussed internally.

Some comments inline.

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

else:
z = posterior.mode()
dec = self.decode(z)
dec = self.decode(z).sample
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise we return a tuple of DecoderOutput when return_dict=False.

Comment on lines -231 to -238
sample_size = (
self.config.sample_size[0]
if isinstance(self.config.sample_size, (list, tuple))
else self.config.sample_size
)
self.tile_latent_min_size = int(sample_size / (2 ** (len(self.config.block_out_channels) - 1)))
self.tile_overlap_factor = 0.25
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused.

Comment on lines -313 to +315
output = [self._tiled_decode(x_slice) if self.use_tiling else self.decoder(x) for x_slice in x.split(1)]
output = [
self._tiled_decode(x_slice) if self.use_tiling else self.decoder(x_slice) for x_slice in x.split(1)
]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should use x_slice and not x.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could maybe further refactor this to how the current implementations of Cog/Mochi are with _decode method. A bit easier to understand code flow that way

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah sure feel free to club those in your PR.

@a-r-r-o-w a-r-r-o-w mentioned this pull request Oct 30, 2024
@sayakpaul sayakpaul marked this pull request as ready for review October 30, 2024 12:56
@sayakpaul sayakpaul requested review from DN6 and a-r-r-o-w October 30, 2024 12:56
@sayakpaul
Copy link
Member Author

@DN6 a gentle ping.

@sayakpaul sayakpaul changed the title [WIP] [tests] refactor vae tests [tests] refactor vae tests Nov 11, 2024
temb,
zq,
conv_cache=conv_cache.get(conv_cache_key),
conv_cache.get(conv_cache_key),
Copy link
Member Author

@sayakpaul sayakpaul Nov 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the torch.utils.checkpoint.checkpoint() method doesn't have any conv_cache argument.

Comment on lines -846 to -865
if self.model_class.__name__ in [
"UNetSpatioTemporalConditionModel",
"AutoencoderKLTemporalDecoder",
]:
return
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because these are supported.

@sayakpaul
Copy link
Member Author

@a-r-r-o-w @DN6 a gentle ping.

@sayakpaul sayakpaul merged commit c1926ce into main Dec 4, 2024
18 checks passed
@sayakpaul sayakpaul deleted the vae-tests branch December 4, 2024 10:28
@sayakpaul
Copy link
Member Author

@a-r-r-o-w merging this to unblock you and will let you add any left over tests. Hopefully, that is okay.

sayakpaul added a commit that referenced this pull request Dec 23, 2024
* add: autoencoderkl tests

* autoencodertiny.

* fix

* asymmetric autoencoder.

* more

* integration tests for stable audio decoder.

* consistency decoder vae tests

* remove grad check from consistency decoder.

* cog

* bye test_models_vae.py

* fix

* fix

* remove allegro

* fixes

* fixes

* fixes

---------

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants