Nightly introduced bug for GGUF in comfy? #137800
Description
🐛 Describe the bug
Not sure if you'll be able to help here but trying to move the nightly to test autocast support on MPS for cogvideo, but going to nightly appears to break things, i've tested it with GGUF models in comfyui it worked fine on 2.4.1 but since upgrading to nightly it completely breaks generation...
This is on a 32gb mac, loading the Q8 GGUF flux model, that normally works fine on torch 2.4.1
Using split attention in VAE
INFO:root:Requested to load FluxClipModel_
Requested to load FluxClipModel_
INFO:root:Loading 1 new model
Loading 1 new model
INFO:root:loaded completely 0.0 4777.53759765625 True
loaded completely 0.0 4777.53759765625 True
ggml_sd_loader:
GGMLQuantizationType.F32 466
GGMLQuantizationType.Q8_0 304
GGMLQuantizationType.F16 10
INFO:root:model weight dtype torch.bfloat16, manual cast: None
model weight dtype torch.bfloat16, manual cast: None
INFO:root:model_type FLUX
model_type FLUX
INFO:root:Requested to load Flux
Requested to load Flux
INFO:root:Loading 1 new model
Loading 1 new model
INFO:root:loaded completely 0.0 12125.320556640625 True
loaded completely 0.0 12125.320556640625 True
0%| | 0/8 [00:00<?, ?it/s]
/AppleInternal/Library/BuildRoots/c7c74b64-74b4-11ef-aeda-9635a580fe0d/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:891: failed assertion `[MPSNDArray, initWithBufferImpl:offset:descriptor:isForNDArrayAlias:isUserBuffer:] Error: buffer is not large enough. Must be 120324096 bytes
'
[1] 16334 abort PYTORCH_ENABLE_MPS_FALLBACK=1 python main.py
/opt/homebrew/anaconda3/envs/py10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
Versions
Versions of relevant libraries:
[pip3] came-pytorch==0.1.3
[pip3] facenet-pytorch==2.6.0
[pip3] lion-pytorch==0.2.2
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.26.0
[pip3] onnx==1.14.0
[pip3] onnx2torch==1.5.14
[pip3] onnxruntime==1.18.0
[pip3] onnxruntime-silicon==1.16.3
[pip3] open-clip-torch==2.24.0
[pip3] pytorch-lightning==2.3.0
[pip3] rotary-embedding-torch==0.6.5
[pip3] sensevoice-onnx==1.1.0
[pip3] torch==2.6.0.dev20241011
[pip3] torchaudio==2.5.0.dev20241011
[pip3] torchmetrics==1.3.0.post0
[pip3] torchscale==0.3.0
[pip3] torchsde==0.2.6
[pip3] torchvision==0.20.0.dev20241011
cc @ezyang @gchanan @zou3519 @kadeng @msaroufim @kulinseth @albanD @malfet @DenisVieriu97 @jhavukainen