Apache SINGA vs tinygrad

Apache SINGA is a free distributed deep-learning library for training deep-learning and machine-learning models. Its listed training mode is local, with data-parallel training across multiple GPUs on one node or across multiple nodes. GPU packages use CUDA and cuDNN, and Docker images are available for Nvidia GPUs. Installation options include pip, Docker, Conda, and source builds; pip documentation covers Python 3.9, 3.10, and 3.11. SINGA supports Python and C++, loads ONNX models, and can save models defined through its APIs in ONNX format. Optimizers listed include momentum-based stochastic gradient descent, Adam, RMSProp, and AdaGrad. Models can be queried in an RDBMS. The repository and Google Colab provide domain-specific models, including healthcare and science examples. It runs on Linux, macOS, Windows, and self-hosted environments, with on-premises deployment and Apache License 2.0 licensing.

tinygrad is a deep learning framework for neural network training, bringing together a tensor library, compiler, JIT, and training tools. It supports forward and backward automatic differentiation, and tensor operations run lazily when a tensor is realized. Neural network classes, optimizers, and state loading and saving are included. TinyJit captures and replays kernels in a decorated function, and tensors can be distributed across multiple GPUs. Listed accelerators include OpenCL, CPU, Metal, CUDA, AMD, NV, Qualcomm, and WebGPU. The project describes use in openpilot to run a driving model on a Snapdragon 845 GPU. It supports Python and local training, with multiple deployment targets and safetensors or PyTorch weights through model-specific loaders. Installation instructions cover pip, though the project recommends installing from source. tinygrad is free under the MIT license. Its API is described as stable for a while, though the project is not yet version 1.0.