Transfer Learning in PhysicsAI

Tutorial Level: Intermediate In this tutorial, you will be guided through the process of using transfer learning in PhysicsAI with a provided pre-trained model and datasets by taking two routes: using transfer learning and training a model from scratch.

Transfer learning in PhysicsAI enables the adaptation of a pre-trained model to a new objective by leveraging the weights from an existing model. This approach is highly effective when dealing with similar tasks or datasets, significantly reducing training time and improving model performance. Transfer learning involves shifting the objective to optimize new but similar datasets quickly by leveraging pre-existing knowledge. It serves as a benchmark for reaching optimization more efficiently.

The datasets used in this tutorial were generated by the Ohio State University Digital Design and Manufacturing Lab. You can see the data visualization here.

Before you begin, copy the file(s) used in this tutorial to your working directory.

Use the Pre-Trained Model

  1. Open HyperMesh.
  2. From the menu bar, click View > Ribbons > PhysicsAI to open the PhysicsAI ribbon.
  3. Create a project.
    1. From the PhysicsAI ribbon, select the Create Project tool.
      Figure 1.


    2. For Project Name, enter pre_trained_model.
    3. For Location, click Choose and select a save location for the project.
    4. Click OK.
  4. Import the model.
    1. From the PhysicsAI ribbon, select the Model Training tool.
      Figure 2.


      The Model Training dialog opens.
    2. Click Import Model to begin the process of importing a pre-trained model.
      The Model Import dialog opens.
    3. For Model Name, enter pre_trained_model.
    4. For Model to Import, browse and select pretrained_model.psmdl.
    5. Click OK.
      After importing, the pre_trained_model appears in the Train Model dialog.
    6. Click Close.
  5. Create datasets.
    1. From the PhysicsAI ribbon, select the Create Dataset tool.
      Figure 3.


      The Create Dataset dialog opens.
    2. For Dataset Name, enter TL_train.
    3. For File System, click and navigate to the TL_train folder.
    4. Select and transfer all of the files.
    5. Click OK.
      The dataset is extracted and the Datasets dialog opens.
    6. Create a secpmd dataset.
      1. Click to reopen the Create Dataset dialog.
      2. For name, enter TL_test.
      3. For File System, click and navigate to the TL_test folder.
      4. Select and transfer all of the files.
      5. Click OK.
      The dataset is extracted.
    7. Click Close in the Datasets dialog.
  6. Perform transfer learning.
    1. From the PhysicsAI ribbon, select the Manage ML Models tool.
      Figure 4.


      The Model Training dialog opens.
    2. Click Create Model.
      The Train Model dialog opens.
    3. For Model Name, enter TL_model.
    4. For Training Data, verify TL_train is selected.
    5. For Inputs, select cae.coord.
      This input setting defines the coordinate data used for training.
    6. For Vectors, select Equivalent_Stress_maximum_MPa_.
      This output setting specifies the type of stress data the model will predict.
    7. For epochs, enter 500 to define the training duration.
    8. Clear the Early stopping check box to allow the training process to complete the specified number of epochs.
    9. Select the Transfer Learning check box and click to define the Pretrained Model Path.
    10. Click Train.
    Figure 5.


    The Model Training dialog opens.
    Figure 6.


Conduct Training from Scratch

With the new dataset already imported, follow these steps to train a model from scratch. Training from scratch means starting the model training without any pre-existing weights, allowing the model to learn from the data provided from the beginning.

  1. Train model.
    1. From the PhysicsAI ribbon, select the Manage ML Models tool.
      Figure 7.


      The Model Training dialog opens.
    2. Click Create Model.
      The Train Model dialog opens.
    3. For Model Name, enter Scratch_model.
    4. For Training Data, verify TL_train is selected.
    5. For Inputs, select cae.coord.
      This input setting defines the coordinate data used for training.
    6. For Vectors, select Equivalent_Stress_maximum_MPa_.
      This output setting specifies the type of stress data the model will predict.
    7. For epochs, enter 500 to define the training duration.
    8. Clear the Early stopping check box to allow the training process to complete the specified number of epochs.
    9. Click Train.
      The Model Training dialog opens.
      Figure 8.


  2. Train model.
    1. From the PhysicsAI ribbon, select the Test ML Model tool.
      Figure 9.


      The Test Model dialog opens.
    2. In the Models area, select TL_model.
    3. In the Datasets area, select TL_test and click OK.
      The Model Testing dialog opens.
    4. Click Test Model to reopen the Test Model dialog.
    5. In the Models area, select pretrained_model.
    6. In the Datasets area, select TL_test and click OK.
    7. Close the Model Testing dialog.
Compare the testing results by looking at the Vector MAE (Mean Absolute Error) value. A lower MAE value signifies better performance and improved accuracy due to transfer learning.

Results and Conclusion

The transfer learning process significantly improves model accuracy compared to using the pre-trained model without transfer learning.

Initially, the pre-trained model may not predict outcomes accurately for the new dataset. However, after applying transfer learning, the model's predictions become much closer to the actual values. For instance, testing the pre-trained model on the TL_test dataset resulted in an MAE (Mean Absolute Error) of 16.639, while the TL_model achieved an MAE of 6.889, reflecting a 45% improvement.

Moreover, when comparing the transfer learning model to a model trained from scratch on the same smaller subset of data for an equal duration, the transfer learning model demonstrated a 35% enhancement in performance on average. This underscores the efficiency of transfer learning in utilizing pre-existing knowledge to boost performance on new but similar tasks.

Overall, transfer learning is advantageous as it facilitates faster convergence and superior performance without the need to start from scratch.

Comparison of Mean Absolute Error (MAE)

Model MAE (Pre-Trained) MAE (Transfer Learning) MAE (From Scratch)
MAE Comparison 16.639 6.889 8.095