[dynamo] attempt run only mode when dynamo cache limit is hit #136655
Conversation
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/136655
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 1369a0c with merge base b7a5c7d ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
To summarize Is this N (the cache limit limit) + 1 the cache hit test? |
|
@bhack please clarify what you mean? |
|
Is run only just an extra step for cache hit check after the recompilation limit is exhausted? Or is it something else? |
|
Run only mode is where we do not ever trace bytecode. We try to run previously compiled code and if all guard checks for previously compiled code fail, we run eager. In the case that cache limit is hit and there is no valid previously compiled code, we also disable the eval frame callback so we skip tracing any recursive calls. |
…hit" Implement #135458. Try run-only mode when dynamo cache limit is hit. If no valid cache entries are found, then skip code recursively. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames rec [ghstack-poisoned]
…hit" Implement #135458. Try run-only mode when dynamo cache limit is hit. If no valid cache entries are found, then skip code recursively. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames rec [ghstack-poisoned]
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
Just to understand. Earlier, if we hit cache size limit, we fully go back to eager. |
5 participants
Stack from ghstack (oldest at bottom):
Implement #135458.
Try run-only mode when dynamo cache limit is hit. If no valid cache entries are found, then skip code recursively.
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @amjames @rec