I sprayed 4,577 images onto Azure with dfuplus images in 22 groups. Then, I created a superfile to include all of the sprayed images. The training produced an error message, so I will modify the code and try again with a smaller dataset.
Yesterday, the build failed at 41%. To identify what caused this crash, I tested a few different variables/scenarios.
I ran the model on Azure with 600 images (instead of 4,000) to see if the size of the dataset was causing the issue. The workunit still failed. I then tested 400 images which works, telling me that the image sizes are contributing to the issue.
Since HPCC should be able to support more than 700MG, I have to figure out the root cause for the “dataset too large” error so I can still train the model with a large dataset.
