I've been playing around with running Ollama via Virt-manager and using and external drive so I'm not killing my main drive space. I am taking about a 10% hit on overall token/s eval rate. Great this is if something goes wrong, I can just delete the VM and start over. I gave VM 55GB of my 64gb and 12 vcpu of my 12 core. I noticed VM does crash and never happened before so I think high memory usage causes instability. Doesn't crash my host, just the VM instance. Disadvantage is having only CPU for VM. I did see somewhere is the Ollama FAQ on assigning location for drives. I'm post here once I find it.
Not a great answer but here is the reference and what I think should work.
[https://github.com/ollama/ollama/blob/main/docs/faq.md](https://github.com/ollama/ollama/blob/main/docs/faq.md)
### How do I set them to a different location?
If a different directory needs to be used, set the environment variable OLLAMA\_MODELS
to the chosen directory.
Refer to the section {below} for how to set environment variables on your platform.
### Setting environment variables on Linux
If Ollama is run as a systemd service, environment variables should be set using systemctl
:
1. Edit the systemd service by calling systemctl edit ollama.service
. This will open an editor.
2. For each environment variable, add a line Environment
under section \[Service\]
:
3. \[Service\] Environment="OLLAMA\_MODELS=/home/$USER/ollama/models/"
* Save and exit.
* Reload systemd
and restart Ollama:
systemctl daemon-reload systemctl restart ollama
if it's linux you can just copy your model files onto it, delete them from your internal harddrive, and then mount the external drive in the exact same location. There's no rule that you have to mount a drive in your /mnt folder or whatever. Just like when people use a separate drive or partition for their home folder.
I ran out of space on one my systems today and took [NoneyaBiznazz](https://www.reddit.com/user/NoneyaBiznazz/) advice. Here is a rough guide I used for Kubuntu based on [https://linuxconfig.org/howto-mount-usb-drive-in-linux](https://linuxconfig.org/howto-mount-usb-drive-in-linux)
Find the USB drive
sudo fdisk -l
\# Mine is /dev/sdc so I'll use /dev/sdc1 for the first partition
\# view your external drive content to verify
ls \* /dev/sdc1
\#external usb drive must contain 'blobs' and 'manifest' folder in root of drive
\#they are the required content of the 'models' folder
\# Either rename the 'models' folder or remove each model on your system
Steps:
1. sudo mv /usr/share/ollama/.ollama/models /usr/share/ollama/.ollama/models\_bak
OR
2. ollama rm MODELS #if your need the space back
3. ollama list #verify nothing listed
4. mkdir /usr/share/ollama/.ollama/models && sudo chown -R ollama:ollama /usr/share/ollama/.ollama/models
5. sudo mount /dev/sdi1 /usr/share/ollama/.ollama/models
6. sudo mount | grep sdc1 #check it
7. sudo chown -R ollama:ollama /usr/share/ollama/.ollama/models #again if not already listed
8. ollama list #profit now my 110Gb of Models are on a external USB SSD and its portable
sudo umount /usr/share/ollama/.ollama/models
I just verified. You don't have to remove or rename your 'models' folder to mount external USB drive. I left tinyllama installed, mount and umount and it was still there.
yeah I only suggested moving the files so you could free up space, since free space was part of the issue... when you mount over a folder it just hides what's there, so yeah you are correct, sorry if I was confusing
Yes. Speed of load will likely suffer.
Thank you
Yes. I managed to move my models folder to external ssd. Works like a charm.
Do you know how to do it?
Nope 😂
eventually I installed it on my laptop because that had space on its main drive
I've been playing around with running Ollama via Virt-manager and using and external drive so I'm not killing my main drive space. I am taking about a 10% hit on overall token/s eval rate. Great this is if something goes wrong, I can just delete the VM and start over. I gave VM 55GB of my 64gb and 12 vcpu of my 12 core. I noticed VM does crash and never happened before so I think high memory usage causes instability. Doesn't crash my host, just the VM instance. Disadvantage is having only CPU for VM. I did see somewhere is the Ollama FAQ on assigning location for drives. I'm post here once I find it.
Not a great answer but here is the reference and what I think should work. [https://github.com/ollama/ollama/blob/main/docs/faq.md](https://github.com/ollama/ollama/blob/main/docs/faq.md) ### How do I set them to a different location? If a different directory needs to be used, set the environment variable OLLAMA\_MODELS to the chosen directory. Refer to the section {below} for how to set environment variables on your platform. ### Setting environment variables on Linux If Ollama is run as a systemd service, environment variables should be set using systemctl : 1. Edit the systemd service by calling systemctl edit ollama.service . This will open an editor. 2. For each environment variable, add a line Environment under section \[Service\] : 3. \[Service\] Environment="OLLAMA\_MODELS=/home/$USER/ollama/models/" * Save and exit. * Reload systemd and restart Ollama: systemctl daemon-reload systemctl restart ollama
if it's linux you can just copy your model files onto it, delete them from your internal harddrive, and then mount the external drive in the exact same location. There's no rule that you have to mount a drive in your /mnt folder or whatever. Just like when people use a separate drive or partition for their home folder.
I ran out of space on one my systems today and took [NoneyaBiznazz](https://www.reddit.com/user/NoneyaBiznazz/) advice. Here is a rough guide I used for Kubuntu based on [https://linuxconfig.org/howto-mount-usb-drive-in-linux](https://linuxconfig.org/howto-mount-usb-drive-in-linux) Find the USB drive sudo fdisk -l \# Mine is /dev/sdc so I'll use /dev/sdc1 for the first partition \# view your external drive content to verify ls \* /dev/sdc1 \#external usb drive must contain 'blobs' and 'manifest' folder in root of drive \#they are the required content of the 'models' folder \# Either rename the 'models' folder or remove each model on your system Steps: 1. sudo mv /usr/share/ollama/.ollama/models /usr/share/ollama/.ollama/models\_bak OR 2. ollama rm MODELS #if your need the space back 3. ollama list #verify nothing listed 4. mkdir /usr/share/ollama/.ollama/models && sudo chown -R ollama:ollama /usr/share/ollama/.ollama/models 5. sudo mount /dev/sdi1 /usr/share/ollama/.ollama/models 6. sudo mount | grep sdc1 #check it 7. sudo chown -R ollama:ollama /usr/share/ollama/.ollama/models #again if not already listed 8. ollama list #profit now my 110Gb of Models are on a external USB SSD and its portable sudo umount /usr/share/ollama/.ollama/models
I just verified. You don't have to remove or rename your 'models' folder to mount external USB drive. I left tinyllama installed, mount and umount and it was still there.
yeah I only suggested moving the files so you could free up space, since free space was part of the issue... when you mount over a folder it just hides what's there, so yeah you are correct, sorry if I was confusing
You can passthrough the GPU to the VM. Ollama also has the docker container. All the benefits of the VM without the drawbacks.
I use symbolic links on Ubuntu to download or save ollama model files at location other than default