this post was submitted on 31 May 2024
89 points (93.2% liked)

Technology

59390 readers
2896 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 7 points 5 months ago* (last edited 5 months ago) (1 children)

Having tried many different models on my machine and being a long-time GPT-4 user, I can say the self-hosted models are far more impressive in sheer power for their size. However, the good ones still require a GPU that most people nor teenagers can't afford.

Nonetheless, GPT-4 remains the most powerful and useful model, and it's not even a competition. Even Google's Gemini doesn't compare, in my experience.

The potential for misuse increases alongside usefulness and power. I wouldn't use Ollama or GPT-3.5 for my professional work because they're just not reliable enough. However, GPT-4, despite also having its useless moments, is almost essential.

The same holds true for scammers and malicious actors. GPT-4's voice will technically allow live, fluent conversations through a phone using a dynamic voice. That's the holy grail for scamcallers. OpenAI is right to want to eliminate as much abuse of their system as possible before releasing such a thing.

There is an argument to be made for not releasing such dangerous tools, but the counter is that someone malicious will inevitably release it someday. It's better to be prepared and understand these systems before that happens. At least i think thats what OpenAi believes, i am not sure what to think. How could i known they Arent malicious?

[–] [email protected] 1 points 5 months ago* (last edited 5 months ago) (1 children)

Saying you wouldn't use 'ollama or gpt3.5' is such a... I want to say uneducated statement? These are not two like terms

You're aware that ollama isn't an LLM? You're aware there are LLMs available via ollama that exceed gpt4s capabilities? I mean, you're right that you need an array of expensive gpus to run them effectively, but... Just comparing ollama to gpt-3.5 is like comparing an NCAA basketball star to the Harlem globe trotters. It's ridiculous at its face. A player compared to a team, for starters.

[–] [email protected] 2 points 5 months ago* (last edited 5 months ago) (1 children)

Correct, i kept it simple on purpose and could probably have worded it better.

It was a meant as a broader statement including “both publicly available free to download models like those based on the ollama architectures as well as free to acces proprietary llm’s like gpt3.5”

I personally tried variations of the vicuna, wizardLM and a few other models (mostly 30B, bigger was to slow) which are all based on ollama’s architecture but i consider those individual names to be less known.

Neither of these impressed me all that much. But of course this is a really fast changing industry. Looking at the hf leaderboard i don’t see any of the models i tried. Last time i checked was January.

I may also have an experience bias as i have become much more effective using gpt4 as a tool compared to when i just started to use it. This influences what I expect and how i write prompts for other models.

I’d be happy to try some new models that have since archived new levels. I am huge supporter for self-hosting digital tools and frankly i cant wait to stop funding ClosedAi

[–] [email protected] 1 points 5 months ago

Llama3-70b is probably the most general purpose capable open source

There are a bunch of contenders for specific purposes, like coding and stuff, though. I wanna say Mistral has a brand new enormous one that you'd need like 4 4090s to run smoothly.