Apache SINGA vs ONNX Runtime

Apache SINGA is a free distributed deep-learning library for training deep-learning and machine-learning models. Its listed training mode is local, with data-parallel training across multiple GPUs on one node or across multiple nodes. GPU packages use CUDA and cuDNN, and Docker images are available for Nvidia GPUs. Installation options include pip, Docker, Conda, and source builds; pip documentation covers Python 3.9, 3.10, and 3.11. SINGA supports Python and C++, loads ONNX models, and can save models defined through its APIs in ONNX format. Optimizers listed include momentum-based stochastic gradient descent, Adam, RMSProp, and AdaGrad. Models can be queried in an RDBMS. The repository and Google Colab provide domain-specific models, including healthcare and science examples. It runs on Linux, macOS, Windows, and self-hosted environments, with on-premises deployment and Apache License 2.0 licensing.

ONNX Runtime is a free engine for machine-learning inference and training within existing technology stacks. It can run models from frameworks including PyTorch, TensorFlow/Keras, TFLite, and scikit-learn. For inference, it optimizes latency, throughput, memory use, and binary size through graph transformations, accelerator-aware partitioning, and optimized computation kernels. Its Execution Providers connect ONNX models to hardware acceleration libraries for CPUs, GPUs, FPGAs, and specialized NPUs. Listed options include NVIDIA CUDA and TensorRT, Intel OpenVINO, DirectML, Qualcomm QNN, Android NNAPI, Apple CoreML, and WebGPU. Deployment spans cloud servers, edge and mobile devices, and browsers; its web runtime supports browser models and its mobile runtime supports Android and iOS. Supported languages include Python, C#, C++, Java, JavaScript, and Rust. Developers can create smaller web or mobile packages by including only the operators and opsets their models need. The open-source runtime is available on Android, iOS, Linux, macOS, self-hosted environments, web, and Windows.