KEMBAR78
Object Detection - YoloV3 - Corrupted PredictionResult · Issue #5810 · dotnet/machinelearning · GitHub
Skip to content

Object Detection - YoloV3 - Corrupted PredictionResult #5810

@Pirner

Description

@Pirner

System information

  • OS Windows 10:
  • .NET 5.0:

Issue

  • Trained a yolov3 object detector
  • The prediction output is wrong and also does not change by changing the input image

Source code / logs

`
var context = new MLContext();

        var emptyData = new List<CountInput>();

        var data = context.Data.LoadFromEnumerable(emptyData);

        var pipeline = context.Transforms.ResizeImages(
                resizing: ImageResizingEstimator.ResizingKind.Fill, 
                outputColumnName: "feed_resized", 
                imageWidth: ImageSettings.imageWidth, 
                imageHeight: ImageSettings.imageHeight, 
                inputColumnName: nameof(CountInput.Image)
            )
            .Append(context.Transforms.ConvertToGrayscale(outputColumnName: "grayscale", inputColumnName: "feed_resized"))
            .Append(context.Transforms.ExtractPixels(
                outputColumnName: "input_1:0", inputColumnName: "grayscale",
                interleavePixelColors: false,
                colorsToExtract: Microsoft.ML.Transforms.Image.ImagePixelExtractingEstimator.ColorBits.Alpha,
                outputAsFloatArray: true,
                scaleImage: .00390625f))
            .Append(context.Transforms.ApplyOnnxModel(
                modelFile: _model_path, 
                outputColumnNames: new[] { "tf.concat_3", "tf.concat_5", "tf.concat_7" }, 
                inputColumnNames: new[] { "input_1:0" })
            );

        var model = pipeline.Fit(data);

        _predictionEngine = context.Model.CreatePredictionEngine<CountInput, CountPrediction>(model);

        string image_path = @"D:\ml_dotnet\tests\test.png";
        var image = (Bitmap)Image.FromFile(image_path);

        var prediction = _predictionEngine.Predict(new CountInput { Image = image });

my assumption is that the data is somehow not correctly transported from the ExtractPixels transformation to the ApplyOnnXModel transformation, that would at least explain why the result always stays the same.

The model has been converted from tensorflow2.

Link to my model: https://drive.google.com/drive/folders/10n6J7Z0NzNTcTCV6dJbzGKoaRZDmWfbo?usp=sharing

I already had a look in it with netron, but looks good to me.
image

I am fairly new to this so bear with me in case anything is missing or I did some obvious mistakes.
I also went through the tutorials (especially regarding object detection), the tutorials itself ran fine but I am still missing something.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Doc bugs, questions, minor issues, etc.imageBugs related image datatype tasksquestionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions