DAY 5: HPCC IN DOCKER DESKTOP

Today I followed the “Machine Learning with HPCC GNN Tutorial Part 1” on my local Docker Desktop. First, I checked terminal to make sure that I had the ML_Core bundle installed, along with the other prerequisites below.

Instead of using the Virtual Machine that the demo used, I’m using Docker Desktop. Instead of using ECL IDE, I’m using VS Code.

Deploying HPCC in Docker Desktop uses the same containerized documentation I used before, but I changed “helm repo add HPCC…” to “git clone https://github.com/hpcc-systems/HPCC-Platform” because “helm-chart” only has gold release, but the latest platform-GNN docker image is in 8.2.0.rc build 

Next, I set up GitHub Pull Requests and Issues in VS Code.

I watched a video on tensors, then ran the code in VS Code and got this result:

DAY 4: DATA AUGMENTATION

After completing the Data Augmentation tutorial on Jupyter Notebook (Keras Preprocessing Layers and tf.images), I concluded that tf.images would be a better fit for my project.

• Keras preprocessing layers
  • Resizing (for consistency) and rescaling pixel values
  • Data augmentation
    • Apply preprocessing layers multiple times to the same image (switch the orientation, rotate it, etc.)
    • 
    • layers.RandomContrast, layers.RandomCrop, layers.RandomZoom
    • *different for my model: instead of flipping the images, it would be best for the variation to be in lighting, shadows, location, background distractions, etc. to simulate a real photo dataset

• Options for using the preprocessing layers
  • First option: make the layers part of the model
  • Second option: apply the preprocessing layers to your dataset
    • I am deciding to use this option so my data augmentation can be separate from the training model and be saved separately
• For the second option:
  • Only augment the training set, not the validation set
  • Train a model using the augmented data
  • Create custom data augmentation layers
• Tf.image
  • Reimport the dataset
  • Retrieve an image to work with
  • Flip the image, grayscale the image, saturate the image, change image brightness, center crop the image, rotate
  • Random transformations (random saturation, brightness, etc)
  • For my purposes, tf.image may be more useful than the basic preprocessing layers because it can emulate real photos more accurately
  • Apply the augmentation to the dataset

Then, I created my own Jupyter Notebook to augment a dataset. The first issue I ran into was that the code in the data_augmentation Jupyter Notebook is changing the images (adjusting brightness, saturation, etc.) but it does not specify code for how to increase the number of images (ex. take 1 image and produce 100 versions of that image).

I went online and researched some other resources that would be able to turn one image into multiple images. I found a tutorial titled “Easy Image Dataset Augmentation with TensorFlow”, but it’s not as detailed. The “Data Augmentation” TensorFlow documentation is very detailed, on the other hand, but not aligned with my purposes. Therefore, I am now splicing together lines of code from both documents to fit my objectives.

In order to augment my dataset, I needed to create a folder of the student images I had, and another folder of images of people who don’t attend my school (I used photos of celebrities). The next step was to make sure that all of the image sizes were the same. To check image size, I opened each photo, right clicked –> get info –> more info.

After finding out that the image sizes are all different, I looked for an application on Mac that could unify the image sizes. I kept the original folder with the unedited images, made a copy of the folder and titled it “edited_student_images”, and opened each image in

Preview –> tools –> adjust size –> fit to 320 x 320 pixels

I thought that this was sufficient to create images that were the same size, but I later found out that since the image sizes are scaled proportionally, they still showed up as different sizes on Jupyter Notebook. This issue was fixed by unchecking the “scale proportionally” box.

Now that I have a local file of uniform images of students and non-students, I can work to display the images in Jupyter Notebook.

I started with a sample code from the TensorFlow tutorial

It displays the file names for all of the images that I need, but not the images themselves

I changed the code to try and fix this issue, but then I got this error message

I later realized that the reason it displayed this error message is because there was a hidden file that wasn’t .jpeg. I manually deleted the .DS_Store file via the terminal (shown below). Tomorrow, I will research a way for the code to automatically detect the file types and only attempt to display images that are .jpeg so this issue doesn’t produce an error message next time.

Now, the code in Jupyter Notebook works correctly. The images are all vertically displayed in a row, and they are the same size.

DAY 3: RETRAINING AN IMAGE CLASSIFIER

I opened the image retraining guide via the terminal and ran the code which consisted of the following steps that I will be replicating on my facial recognition model:

• Select the TF2 SavedModel
• Resize images in the dataset for the module, and augment the photos
• Put a linear classifier on top of the feature_extractor_layer
• Start with non-trainable speed layer
• Train the model
• Test the model’s accuracy using the validation dataset
• Trained model can be saved for deployment to TF Serving or TF Lite (mobile)

The dataset setup step of the image retraining demonstration outputs an AttributeError.

After researching the error message online, the response was that “the specific function (tf.keras.preprocessing.image_dataset_from_directory) is not available under TensorFlow v2.1.x or v2.2.0 yet. It is only available with the tf-nightly builds and is existent in the source code of the master branch.”

Currently, I am on version 2.0.0 of tensorflow. I updated my tensorflow version to 2.3.0 to see if it would fix the error message.

Initial tensorflow version

After running “conda install tensorflow=2.3.0” in the terminal, it says that “All requested packages already installed” but the tensorflow version is still at 2.0.0

Error message:

PackagesNotFoundError: The following packages are not available from current channels:

  – tensorflow=2.5.0

Current channels:

  – https://repo.anaconda.com/pkgs/main/osx-64

  – https://repo.anaconda.com/pkgs/main/noarch

  – https://repo.anaconda.com/pkgs/r/osx-64

  – https://repo.anaconda.com/pkgs/r/noarch

I then tried “pip install tensorflow==2.5.0” and it returned this error message:

“pip install –upgrade tensorflow” returned this error message

The reason why it is giving error messages: the version of anaconda is not compatible with tensorflow 2.5.0

Solution: reinstall using Tensorflow 2.5 with GPU device (Python 3.9, Cuda 11.2.2 , Cudnn 8.1.1) Conda environment 

1. Install Anaconda with Python 3.9
2. conda create –name TF-2.5 python=3.9
3. conda activate TF-2.5
4. Install libraries

DAY 2: TENSORFLOW

The main objective for today was to follow the TensorFlow 2 quickstart code and the TensorFlow Convolutional Neural Network (CNN) tutorial. To successfully compile the code, I upgraded my TensorFlow version to 2.5 and installed matplotlib (using conda) for the image retraining documentation I will finish following tomorrow. The Jupyter Notebook for Retraining an Image Classifier will be especially useful because I will be following very similar steps when I retrain an existing model. I ran into an issue with the image retraining code – the “Setup” step resulted in a “ModuleNotFoundError: No module named ‘matplotlib'” even after I installed matplotlib and checked the version with “conda list | grep matplotlib” on my terminal. I am still in the process of devising a solution.

Update: The reason why it showed an error message is because I installed “matplotlib” under in “base” instead of “tf”. This was the same issue for “tensorflow_hub” too. After repeating the process for “tensorflow_hub”, the code now runs successfully without an error message.

• 

• 

• 

• 

DAY 1: HPCC IN AZURE

I started deploying HPCC in Azure after installing Azure Cli. Halfway through the process, I ran into an ECL error because the helm chart’s version didn’t match the image version. My first attempted solution involved deleting “mycluster” and deploying a matched version (rc4 instead of rc2), but I still received an error message. Ultimately, the issue was fixed by changing the helm chart version to 8.0.20 to match the docker image 8.0.20-1.

helm install mycluster hpcc/hpcc –set global.image.version=8.0.20-1 -f examples/azure/values-auto-azurefile.yaml

I also finished reading the Hands on Machine Learning book (chapters 10 and 14), so I am now ready to select which type of model I will work on.

• Artificial Neural Network
  • Siri, classifying images, recommending videos
  • Good for when you have lots of data
• Artificial neuron
  • Multiple binary (on/off) inputs and one binary output
  • artificial neuron activates the output when more than X number of inputs are active
• Identity function: if neuron A is activated, then neuron C is also activated
• Second network: neuron C is only activated when both A and B are activated
• Third network: neuron C is activated if A or B are activated (or both)
• Fourth network: neuron C is activated only if neuron A is activated and B is off
• Threshold logic unit
  • Inputs and outputs are numbers (instead of on/off binary values)
  • Each input connection is associated with a weight
• The Perceptron
  • ANN
  • Heaviside step function 
  • TLU outputs a positive class if the combo of the inputs is greater than the threshold value
    • Negative if it is less 
  • Neurons in each layer are connected to the previous layer: fully connected layer
  • Inputs are fed to input neurons
  • Bias neuron outputs 1 all the time
  • Connections help reduce the error
    • Fed one training instance at a time
      • Makes a prediction
    • For every output neuron that had the wrong prediction, it reinforces the connection with inputs that would have contributed to the correct prediction
• Multilayer Perceptron
  • One input layer, at least one TLU hidden layer, one output layer of TLUs
  • Deep neural network – when the ANN has many hidden layers
  • Backpropagation
    • Can find out how each connection weight and bias term should be tweaked to reduce the error
  • Handles one mini-batch of instances
    • Goes through the full training set multiple times
    • Each pass is an epoch
  • Mini batch is passed to the input layer, then sent to the hidden layer
  • Algorithm measures output error
  • Calculates how much each connection contributed to the error
  • Tweak connection weights
• Regression MLP
  • Predict a single value = single output neuron
  • Predict multiple values = one output neuron per output dimension
• Classification MLP
  • Binary classification → single output neuron
• Keras: build, train, and execute neural networks
  • Supports TensorFlow
• Build an image classifier
  • Load a dataset using Keras
    • Pixels represented as floats
    • Split into a training set, test set, and validation set
  • Create Sequential API
    • Single stack of layers connected sequentially
  • Build the first layer and add it to the model
  • Add a hidden layer
    • Manages its own weight
• Model can overfit the data, especially if you don’t have a lot of training data
• Compile the model
  • You can have a list of extra metrics to compute during training and evaluation 
• Training and evaluation
  • Fit ( ) method
  • Validation error is computed at the end of each epoch
  • Training error is computed during each epoch
• Callback
  • List of objects that Keras will call at the start and end
• For the hidden layers, use the same number of neurons in each layer
• Hyperparameters
  • Learning rate
    • Optimal learning rate is about half of the maximum learning rate
  • Optimizer 
  • Batch size
    • Large batch: training algorithm sees more instances per second
    • But may not generalize as well as a model trained with a small batch size

• 

• 

• 

• 

• 

• 

• Higher level neurons are based on the outputs of neighboring lower-level neurons
• Convolutional layer
  • Not fully connected
    • Only connected to neurons located within a small rectangle of the first layer
    • Faster for image processing
  • Simultaneously applies many trainable filters to the inputs so it can detect many features anywhere in the inputs
• CNN → good for image recognition
  • Processes the value of each pixel (color)
• Color channel: red, green, blue
• Greyscale images have 1 color channel
• In TensorFlow, each input image is represented as a 3D tensor of height, width, channels
  • Turn pixel intensity into floats (0 to 1)
  • Create filters
  • Apply filters to images
  • Plot the resulting feature map
• Convolutional layers use lots of RAM for memory
• CNN architectures stack a few convolutional layers, then a pooling layer, then more convoluted layers
  • Images get smaller as they are processed
• Using pre trained models from Keras
  • Make sure images are the right size
  • Scale inputs from 0 to 1 or whatever the model’s scale is
• Semantic segmentation
  • Each pixel is classified according to the class of the object (car, person, street, etc)
  • Doesn’t distinguish objects of the same class