注意：Ollama官方容器不支持Jetson的GPU，需使用Step3的容器

Step1: check version

cat /etc/nv_tegra_release

Typical output:

# R36 (release), REVISION: 4.0, GCID: 37976356, BOARD: t23x
# DATE: 2025-05-09, 14:51:25, L4T 36.4.0-aarch64-64-Ubuntu

因此版本为36.4.0

Step2: 安装Container Toolkit

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
| sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
| sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
| sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update

sudo apt-get install -y nvidia-container-toolkit

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Source: https://hub.docker.com/r/ollama/ollama

Step3: 运行容器(keep alive 可选)

docker run --runtime nvidia -it -d --network=host -v ~/ollama:/ollama -e OLLAMA_MODELS=/ollama -e OLLAMA_KEEP_ALIVE=10000000 --name ollama dustynv/ollama:r36.4.0

Source: https://hub.docker.com/r/dustynv/ollama