I expect the repro file to contain a trimmed model code which runs the full export-compile-run flow and then reproduces the problem. Loading another .pt2 file is not intuitive here.

I expect the repro file to contain a trimmed model code which runs the full export-compile-run flow and then reproduces the problem. Loading another .pt2 file is not intuitive here.

@desertfire The .pt2 file is a trimmed model. I'm not sure how we can just include the model code instead of using an exported_program (the existing torch.compile minifier does this, but it's buggy). How can we convert the model code into a string and preserve the model safely?

If the trimmed model is as simple as a few nodes, that's possible. But if the trimmed model is more complicated, e.g. contains submodules or parameters, then converting it into a string and load inputs/state_dict is non-trivial anymore.

Another point is that the result of the minifier is a GraphModule object. It seems a bit weird that we first convert the GraphModule object into some nn.Module code, and then export it back to a GraphModule?

When we use TORCH_COMPILE_DEBUG=1, it will generate fx_graph_runnable.py. Maybe we can do something similar here?

When we use TORCH_COMPILE_DEBUG=1, it will generate fx_graph_runnable.py. Maybe we can do something similar here?

pytorch/torch/_inductor/debug.py

Lines 475 to 495 in 91d38a5

	with self.fopen("fx_graph_runnable.py") as fd:
	save_dir = None
	if torch._inductor.config.trace.save_real_tensors:
	inputs = torch._subclasses.fake_utils.try_convert_fake_to_real(inputs)
	save_dir = os.path.dirname(fd.name)
	# dont try to use stable hash torchinductor compilation if saving real tensors
	# and avoid recursively trying to save real tensors inside of the inductor compilation
	# regardless
	stable_hash = torch._inductor.config.trace.save_real_tensors
	with torch._inductor.config.patch(
	{"trace.enabled": False, "trace.save_real_tensors": False}
	):
	save_graph_repro(
	fd,
	gm,
	inputs,
	"inductor",
	save_dir=save_dir,
	stable_hash=stable_hash,
	)

yeah, we can do this, but I think this is not as robust as storing an exported program. This uses torch._dynamo.repro.after_aot.save_graph_repro under the hood, which only works for flattened graphs. It doesn't work out-of-the-box if there are submodules in the graph.

But maybe this is fine since AOTI runs on ep.module() which is already flattened, so the repro are always a flattened graph?

so the repro are always a flattened graph?

I think so.

My main concern with generating .pt2 is for OSS users's ability to see the generated code. Because some OSS users don't want to share their whole model code, but if they see the generated code is small enough, they are more willing to share it as a repro. With .pt2, they can unzip and examine the minimized code in theory, but that just adds an extra step and discourages people to share.

For that reason, I think maybe we can just print the minimized exported graph as a string comment in this file, so users can immediately see what is contained in the graph.

so the repro are always a flattened graph?

I think so.

My main concern with generating .pt2 is for OSS users's ability to see the generated code. Because some OSS users don't want to share their whole model code, but if they see the generated code is small enough, they are more willing to share it as a repro. With .pt2, they can unzip and examine the minimized code in theory, but that just adds an extra step and discourages people to share.

For that reason, I think maybe we can just print the minimized exported graph as a string comment in this file, so users can immediately see what is contained in the graph.

I modified the PR to return the exported graph as a string now. The doc is also updated.

We could compare with eager result here and thus enable accuracy minifier. Ok to do it in a follow-up PR.

We could compare with eager result here and thus enable accuracy minifier. Ok to do it in a follow-up PR.

yeah, I can do it in a follow up

I would recommend moving this under aot_inductor.