this post was submitted on 08 Mar 2024
72 points (86.7% liked)

Technology

59148 readers
2703 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago
MODERATORS
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
72
AI Prompt Engineering Is Dead (developers.slashdot.org)
submitted 8 months ago by [email protected] to c/[email protected]
35 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 80 points 8 months ago* (last edited 8 months ago) (7 children)

This is dumb. Literally nothing has changed. Anyone who knows anything about LLM's knows that they've struggled with math more than almost every other discipline. It sounds counter intuitive for a computer to be shit at math, but this is because LLM's "intelligence" is through mimicry. They do not calculate math like a calculator. They calculate all responses based on a probability distribution constructed from billions of human text inputs. They are as smart, and as fallible, as wikipedia + reddit + twitter, etc, etc. They are as fallible as their constructing dataset.

Think about how ice cream sales correlate with drownings. There is no direct causality, but that won't stop an LLM from seeing the pattern or implying causality, because it has no real intelligence and doesn't know any better.

"Prompt engineering" is about understanding an LLM's strengths and weaknesses, and learning how to work with them to build out a context and efficiently achieve an end result, whatever that desired result may be. It's not dead, and it's not going anywhere as long as LLM's exist.

[–] [email protected] 28 points 8 months ago (1 children)

I really wish all of these companies racing to replace their existing software features and employees with LLMs understood this. So many applications are dependent on a response being 100% accurate for a very specific request as opposed to being 80% accurate for a wide variety of requests. "Based on training data, here's what a response to your input might look like" is pretty good for conversational language and image generation, but it sucks for anything requiring computation or expertise. Worst of all, it's so confidently wrong about things I might as well be back on Reddit.

[–] [email protected] 1 points 8 months ago* (last edited 8 months ago)

I really wish all of these companies racing to replace their existing software features and employees with LLMs understood this.

They totally understand it. And OpenAI has solved it. For example while researching The Ultimate Answer to Life the Universe and Everything, I asked it to calculate 6 by 9 in base 13 and got the correct answer - 42.

ChatGPT didn't use the LLM to calculate that. It only used the LLM understand an obscure and deliberately confusing chapter of the Hitchhiker's Guide book, to write and execute this python script.

# To calculate six by nine in base 13, we multiply the numbers in our standard decimal system and then convert the result to base 13.

# Calculate 6 * 9 in decimal
result_decimal = 6 * 9

# Convert the result to base 13
# The easiest approach is to use the divmod() function repeatedly to get the remainder (which corresponds to the base 13 digit) 
# and update the quotient for the next iteration until the quotient is 0.

def decimal_to_base_n(num, base):
    if num == 0:
        return "0"
    digits = []
    while num:
        num, remainder = divmod(num, base)
        digits.append(str(remainder))
    return ''.join(digits[::-1])

# Convert the decimal result to base 13
result_base_13 = decimal_to_base_n(result_decimal, 13)

result_base_13
[–] [email protected] 12 points 8 months ago* (last edited 8 months ago) (1 children)

It's not dead, and it's not going anywhere as long as LLM's exist.

Prompt engineering is about expressing your intent in a way that causes an LLM to come to the desired result. (which right now sometimes requires weird phrases, etc.)

It will go away as soon as LLMs get good at inferring intent. It might not be a single model, it may require some extra steps, etc., but there is nothing uniquely "human" about writing prompts.

Future systems could for example start asking questions more often, to clarify your intent better, and then use that as an input to the next stage of tweaking the prompt.

[–] [email protected] 0 points 8 months ago* (last edited 8 months ago)

Future systems could for example start asking questions more often

Current systems already do that. But they're expensive and it might be cheaper to have a human do it. Prompt engineering is very much a thing if you're working with high performance low memory consumption language models.

We're a long way from having smartphones with a couple terabytes of RAM and a few thousand GPU cores... but our phones can run basic models and they do. Some phones use a basic LLM for keyboard auto correct for example.

[–] [email protected] 3 points 8 months ago

You know, I had gotten frustrated using it because it wouldn't understand me, but now I'll use the approach to find out how it understands me

[–] [email protected] 1 points 8 months ago* (last edited 8 months ago)

If input.type = int; use calculator.

[–] [email protected] 1 points 8 months ago

Re math, enter functionary-able models

[–] [email protected] 0 points 8 months ago

I mean it's not really like humans are good at math either, we are good at making abstractions and following linear rules but we are slow and fallible. Digital computation is just near absolute the best method for doing math. LLMs are decent abstraction and general problem solving tho. They are not as creative as people but they are still pretty good! It's a step on the right direction for true agi. Honestly even when we have agi I doubt they will ever beat raw cpus in computation speed.