Uncertainty-aware Models (PNN)

A Probabilistic Neural Net (PNN) Model trained on collected data to reflect the relationship between model inference performance and allocated resources, client demands, etc, with uncertainty.

Model Training Script Conda Environment Data

Collected Data Visualization Example

Model inference time vs CPU time slice limit vs # of clients

Model inference time vs CPU time slice limit vs GPU instance

Model inference time for various CPU limits, 1 clients.

Coverage: 0.9850889834442464

Model inference time for various CPU limits, 2 clients.

Coverage: 0.9784145580671055

Model inference time for various CPU limits, 3 clients.

Coverage: 0.9817625884270647

Model inference time for various CPU limits, 4 clients.

Coverage: 0.9850737962542044

Model inference time for various CPU limits, 5 clients.

Coverage: 0.9798594725424478