Testing GPT-4.0 accuracy rate on Japanese language

[mariodesu]

@WeebPotato

How does the “perfect +” impact the results? That the model just adheres more closely to your prompt?

“perfect” (without +) is the key word that actions the plugin transforming what follows into the “perfect” prompt, and I’d expect it to improve it by making it more logically coherent, but personally I can always improve it slightly and I accidentally discovered that you can just dialogate with GPT about things you don’t like of the result, I think this is useful because provided that the plugin itself works good you don’t risk of ending up with a contradictory or inaccurate prompt

[WeebPotato]

@mariodesu

“perfect” (without +) is the key word that actions the plugin transforming what follows into the “perfect” prompt, and I’d expect it to improve it by making it more logically coherent,

Does one see the “perfect” version of the prompt before its issued to the model?

@mariodesu

accidentally discovered that you can just dialogate with GPT about things you don’t like of the result,

You mean, comment on the final output from the model you get back? I’m a little lost on what you mean by “prompt” here .

[mariodesu]

First, I realized that I do need to specify an action in this particular context otherwise the system aka @Vanilla is not going to know that my first command refers to position 2 and not position 1. Second,since I always need a reference point to operate changes, those could be number position inside the chain (1st position, 2nd, etc.) and number value (1,4,5,2,3), so it would be:
“Add 0, subtract 2, subtract 2, add 2, add 2”

Wait, nothing implies sequentiality, does it? Or maybe it’s the sequence itself, which implies following a direction

[mariodesu]

@WeebPotato

Does one see the “perfect” version of the prompt before its issued to the model?

Yes indeed, this way you can operate. Note that I prompt the result in another chat and never in the same, to avoid confusion, if this makes sense

@WeebPotato

You mean, comment on the final output from the model you get back? I’m a little lost on what you mean by “prompt” here .

I’m sorry, with my previous clarification it should be understood now. Basically I have this chat that is solely for prompt engineering and I first put a simple request preceded by “perfect” and then I discuss with it about the results

[Vanilla]

@mariodesu

I believe that this is were the problem lies and I imagine that understanding this is why many experts are pushing for slowing down capacity augmentation, so that there is more time for interpretation.

@mariodesu

I think differently here, and I don’t know if I’m wrong because I’m ignoring some fundamental technical concept, but logically, you cannot be sure of not being tricked into believing it is aligned if there is not perfect interpretation. Your thoughts?

I don’t know about ignoring a fundamental concept, or anything, but I think its very focused on interpretation. I wouldn’t be surprised if there was some crazy math proof that one day emerged that there was a way to check for AI alignment without understanding it. Or maybe one day super intelligent AI will align super intelligent AI, working like a system of checks and balances. Theres also the problem of perfect alignment isn’t really well defined.

@mariodesu

Assuming that I don’t have to specify “don’t do nothing”, then I’d make it “swap to #2; swap to #3; swap to #4; swap to #5” (# = position)

Complier error in line 2 :

swap to #2 << “swap” function expected 2 numbers to be swapped and only 1 was provided.

@pm215

Hard mode - come up with a set of instructions that works regardless of which particular numbers are in the list

that was the original task, actually, but I was going to save pointing that out for later haha

[mariodesu]

@Vanilla

swap to #2 << “swap” function expected 2 numbers to be swapped and only 1 was provided.

oh okay this is an english comprehension problem
Let’s go with “swap 4 with 2, swap 5 with 3”

[mariodesu]

https://twitter.com/svpino/status/1668695130570903552?s=46&t=1u_Als_a2YkvrRZhAYPlZw

Can anyone explain the technical details of this tweet? I don’t understand what is a function call and why this new got4 update is supposed to be that good

[Vanilla]

@Vanilla

[1,4,5,2,3]

@mariodesu

oh okay this is an english comprehension problem
Let’s go with “swap 4 with 2, swap 5 with 3”

[1,2,3,4,5] ! Congrats! The problem has been solved. The next list is [5,4,3,2,1]

Your set of instructions yields the result: [3,2,5,4,1]

Oh no! It looks like you have a bug in your code. You need to make sure it works for any list of 5 numbers!

Also

@mariodesu

but logically, you cannot be sure of not being tricked into believing it is aligned if there is not perfect interpretation. Your thoughts?

Also, let me provide you with a thought experiment that kind of highlights where I think your thinking doesn’t apply. Lets say we have a petri dish with a whole bunch of cells in it that multiply. We have 20 cells in the dish that we check individually and none of them are the bad “B-CELL” that we don’t like. We know the B-CELL can’t emerge from another cell and can only come from other B-CELLS. We seal our petri dish in a way that we can be certain no B-CELL could enter and let our 20 cells multiply. 10 days later there are billions of cells (its a big dish ok). We are unable to check each of the cells individually to see if any of them are B-CELLs because there are too many.

Can we be sure our dish isn’t infected? The answer is yes. By being absolutely sure of the initial conditions and environment by which the culture grew under, we are able to say that there are no B-CELLs despite not being able to check them all.

In practice, being able to say you are sure of those things is the hard part of course. Its extremely hard, in fact. But, my point is that its not necessarily the case that “you don’t know something unless you see for yourself” in the math world. If it were, we wouldn’t be so sure that there are infinite primes.

[Vanilla]

@mariodesu

Can anyone explain the technical details of this tweet? I don’t understand what is a function call and why this new got4 update is supposed to be that good

To explain in very simple terms without using programming words:

1. The person wrote some pieces of code that do very specific tasks when given certain inputs.
2. He showed chatGPT a description of what the pieces of his code does.
3. He then gave it a problem that he wanted to solve written out in normal english.
4. ChatGPT read the prompt for the problem he wanted to solve and told him what pieces of code he needs to execute in what order with what inputs to get the answer to the question he wanted.
5. He then wrote another piece of code to basically listen to chatgpt’s suggestion and execute it automatically in the suggested manner. And he got out the right answer.

I think…someone can correct me if im wrong.

[WeebPotato]

@Vanilla

He then wrote another piece of code to basically listen to chatgpt’s suggestion and execute it automatically in the suggested manner. And he got out the right answer.

This is a thing which a certain lazy creative dev did with Siri’s voice recognition + ChatGPT. He would just verbalize his ideas for an API, tell ChatGPT to implement it, then create a pipeline which grabs the output of that and deploys on his server. Then, he would iterate on his ideas by showing back the implementation to ChatGPT (+ Siri to parse his intended changes) and changing specific elements of it (messaging, minor bug fixes, etc.).

[Vanilla]

[WeebPotato]

@mariodesu

for a good starting point on how to get into the habit of combining different tools A, B, C I recommend this article: Taco Bell Programming

In times immemorial on Unices we used to have simple tools which did stuff on certain predictable inputs and spit out predictable output. One would then build complex pipelines upon these tools.

[Kazzeon]

@Vanilla

Or maybe one day super intelligent AI will align super intelligent AI, working like a system of checks and balances.

Hell yeah, Sibyl System!

[yamitenshi]

@WeebPotato

In times immemorial on Unices we used to have simple tools which did stuff on certain predictable inputs and spit out predictable output. One would then build complex pipelines upon these tools.

We should apply these concepts to programming too. Sounds like a novel idea I’m sure nobody’s thought of before

[mariodesu]

@Vanilla

Oh no! It looks like you have a bug in your code. You need to make sure it works for any list of 5 numbers!

Just to clarify before I answer - you’re asking me for a line of command that is able to transform [3,2,5,4,1] AND any other [w,x,y,z] combination into [5,4,3,2,1]? Is it even possible to achieve with the same instruments I used before (swap x with y)? To solve this I need to know what I can work with, so please define some instructions first

@Vanilla

Can we be sure our dish isn’t infected? The answer is yes. By being absolutely sure of the initial conditions and environment by which the culture grew under, we are able to say that there are no B-CELLs despite not being able to check them all.

In this particular example the assumption is always correct but this reminds me a lot economics basics where they explain single economics phenomena by isolating them from all the rest and freezing variables…
in truth the case may be different:
a) someone put one or more B-CELLS into the petri, for obscure reasons (maybe the cell’s nature was unknown even to the operator), and no one realized
b) with growth in age/number they start developing new functions or behaving differently
c) new good cells may born or transform into B-CELLS for unknown reasons or no reason at all (this is the weirdest situation but it’s just how we evolved out of broth)

Random idea I had while listening to a podcast yesterday. What about developing and training an AI in a controlled setting, convincing it that there is an escape route and just watch if it tries to escape? The key point here would just be to make sure that the AI has zero knowledge and suspicion about the trap.

[Kazzeon]

@mariodesu

solve this I need to know what I can work with, so please define some instructions first

Didn’t he say you can use comparing if a number is less than another number or vice-versa?

@Vanilla

We know the B-CELL can’t emerge from another cell and can only come from other B-CELLS.

This means you’ve checked that b and c don’t happen, and a is just .

So saying, but what if they happen, goes against the whole point.

@mariodesu

convincing it that there is an escape route

Convincing it?

[mariodesu]

@Vanilla

To explain in very simple terms without using programming words:

1. The person wrote some pieces of code that do very specific tasks when given certain inputs.
2. He showed chatGPT a description of what the pieces of his code does.
3. He then gave it a problem that he wanted to solve written out in normal english.
4. ChatGPT read the prompt for the problem he wanted to solve and told him what pieces of code he needs to execute in what order with what inputs to get the answer to the question he wanted.
5. He then wrote another piece of code to basically listen to chatgpt’s suggestion and execute it automatically in the suggested manner. And he got out the right answer

This is exactly how I imagined making GPT-4 work out with math

[WeebPotato]

@mariodesu

Random idea I had while listening to a podcast yesterday. What about developing and training an AI in a controlled setting, convincing it that there is an escape route and just watch if it tries to escape? The key point here would just be to make sure that the AI has zero knowledge and suspicion about the trap.

I’m not sure how that would exactly work. Could you outline step-by-step how you envision it so we can help break it down?

The immediate issues I see here are

• the convincing part as suggested by @Kazzeon
• why would the model want to try to find an escape route?
• what trap?

[mariodesu]

@WeebPotato

This is a thing which a certain lazy creative dev did with Siri’s voice recognition + ChatGPT. He would just verbalize his ideas for an API, tell ChatGPT to implement it, then create a pipeline which grabs the output of that and deploys on his server. Then, he would iterate on his ideas by showing back the implementation to ChatGPT (+ Siri to parse his intended changes) and changing specific elements of it (messaging, minor bug fixes, etc.).

this sounds very interesting, I can’t wait to understand what it means haha

@WeebPotato

Taco Bell Programming

I have no clue about what those technical details mean and didn’t understand the line of code, but actually it transmitted a lot and I think I can better see what role is played by logic in programming now.

I’d like to understand this line:

I could have done the whole thing Taco Bell style if I had only manned up and broken out sed, but I pussied out and wrote some Python.

What’s wrong with Python?

[WeebPotato]

@mariodesu

What’s wrong with Python?

By default Python provides a lot more control over the code, better data handling, etc. In Shell/Bash it’s easy to write a script that just falls through without you knowing why and what happened. The danger of long chains of command-line calls like in the Taco Bell style example is that if the whole call fails at any point, you won’t know how/why/etc.