It has a Intel Xeon E3-1225 V2, 20gb of ram, and a Strix GTX 970 with 4gb of VRAM. I've actually tried Mistral 7b and Decapoda Llama 7b, running them in Python with Huggingface's Transformers library (from local models)
SaintWacko
joined 1 year ago
Hm... Alright, I'll have to take another look at it. I kinda gave up, figuring my old server just didn't have the specs for it
Show as in I waited a few minutes and finally killed it when it didn't seem like it was going anywhere. And this was with the 7b model...
I tried doing that on my home server, but running it on the CPU is super slow, and the model won't fit on the GPU. Not sure what I'm doing wrong
The Tick
They're being useless, but what I do is use Proxmox and just install my stuff each in their own LXC
Nice! Glad to hear that works. I'll have to give it another go. I had spent the whole day trying to get Mullvad (without WireGuard) working, but it kept failing to create the tun device, so by the time I got it working with WireGuard I didn't really feel like trying to figure out the VLAN thing too lol.
Ugh, I wish I could be more help on that, but I couldn't get Mullvad to work that way either. I think what needs to be done is to use pfsense or something to create a virtual LAN, set the container running Mullvad to be the gateway on that network, then give each container a virtual network bridge connected to that virtual network. What I ended up doing was just installing Mullvad (through WireGuard) on the same container as qBitTorrent and telling qBitTorrent to use the virtual network device that Mullvad creates.
Fortunately, that's the only thing that really needs to run through it for me (I think your Real Debrid will need to as well). AFAIK, the *arr stuff doesn't need to be hidden.
As to getting things to talk to each other in containers, where were you having trouble? You should just be able to give all the *arr stuff the addresses where you reach the other ones. That may just be their IP address, or I run PiHole so I can have a local DNS and give them all their own hostnames.
Edit: I'm doing all this in Debian LXCs
I've been working on the same thing over the past month, with some minor differences. I skipped portainer and am just running LXCs on Proxmox, and built it from the beginning as a *arr/Plex box, so it has 4x4TB internal drives in ZFS RAID6, with the OS on an SSD. I still need to try out the TrueNAS thing, but I'm running a Minecraft server on it, and I just spent the better part of a day figuring out how to run Mullvad on it and force all my torrent traffic to use it.
Here, I'm going to scare all the front end devs "Make it pixel-perfect to the designs"
The EU privacy thing? What's that got to do with it?