Edit: After reading the discussion, I figured I’d let GPT4 speak for itself:
The quest to identify true artificial intelligence (AI) indeed presents challenges, especially as AI models become more sophisticated. Let’s explore some indicators that researchers and practitioners consider when assessing AI systems:
-
Understanding Context and Meaning:
- True AI should demonstrate an understanding of context and meaning. It should not merely generate plausible-sounding sentences but also comprehend the content it produces.
- A system that can engage in nuanced conversations, infer intent, and provide relevant responses based on context would be a strong indicator of advanced AI.
-
Generalization and Adaptability:
- An AI system that can generalize from its training data and adapt to novel situations is valuable.
- True AI should not be limited to memorizing specific examples but should exhibit flexibility in handling diverse scenarios.
-
Creativity and Novelty:
- Creativity is a hallmark of human intelligence. An AI system that generates novel ideas, art, or solutions beyond its training data suggests progress.
- For instance, AI-generated poetry, music, or visual art that resonates with human emotions could be an indicator.
-
Ethical Decision-Making:
- True AI should navigate ethical dilemmas. It should consider consequences, fairness, and societal impact.
- An AI system that can weigh trade-offs and make ethical choices aligning with human values is a positive sign.
-
Self-Awareness and Self-Improvement:
- While self-awareness akin to human consciousness remains elusive, an AI system that can self-assess its performance and seek improvement is promising.
- Continuous learning, adaptation, and self-correction are essential traits.
-
Transparency and Explainability:
- True AI should be transparent about its decision-making process. It should provide explanations for its outputs.
- Efforts to develop interpretable AI models and techniques for explaining their predictions contribute to trustworthiness.
-
Robustness and Safety:
- AI systems must handle adversarial inputs, noisy data, and edge cases gracefully.
- Robustness against attacks and safety precautions to prevent harmful behavior are critical indicators.
-
Human-AI Collaboration:
- Rather than replacing humans, true AI should complement human abilities.
- Systems that collaborate effectively with humans—such as AI assistants, co-creative tools, or medical diagnostics—demonstrate progress.
-
Quantifiable Metrics:
- Researchers propose various metrics to assess AI capabilities, such as perplexity, BLEU score, F1 score, and accuracy.
- However, no single metric captures the full spectrum of intelligence.
-
Detection Tools:
- Researchers are developing tools to differentiate between human- and AI-generated text.
- These tools analyze features like fluency, word frequency, punctuation patterns, and sentence length¹.
- While imperfect, they contribute to identifying AI-generated content.
In summary, the arrival of true AI may not be a singular event but a gradual evolution marked by advancements in the above dimensions. As AI continues to progress, our understanding of its capabilities will evolve, and new indicators may emerge.
Source: Conversation with Bing, 2/29/2024 (1) How to spot AI-generated text | MIT Technology Review. https://www.technologyreview.com/2022/12/19/1065596/how-to-spot-ai-generated-text/. (2) Intelligent Supertrend (AI) - Buy or Sell Signal — Indicator by … https://www.tradingview.com/script/q9244PAH-Intelligent-Supertrend-AI-Buy-or-Sell-Signal/. (3) Indicators - True ALGO. https://truealgo.com/indicators/. (4) Improve Key Performance Indicators With AI - MIT Sloan Management Review. https://sloanreview.mit.edu/article/improve-key-performance-indicators-with-ai/. (5) New AI classifier for indicating AI-written text - OpenAI. https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text/.
If you come up with a test, people develop something that does exactly what the test needs, and ignores everything else.
But we can’t even say what human consciousness is yet.
Like, legitimately, we don’t know what causes it and we don’t know how anaesthesia interferes either.
One of the guys who finished up Einstein’s work (Roger Penrose) thinks it has to do with quantum collapse. But there’s a weird twilight zone where anesthesia has stopped consciousness but hasn’t stopped that quantum process yet.
So we’re still missing something, and dudes like in his 90s. He’s been working on this for decades, but he’ll probably never live to see it finished. Someone else will have to finish later like him and Hawking did for Einstein
“Because quantum” always feels like new-age woo-woo bullshit.
It’s more likely just too vague to define.
No, it’s literally a physical observable quantum collapse…
Like, I know techbros just randomly yell quantum…
But that doesn’t mean it’s not real, it was vague because trying to explain would be a lot of effort and most people wouldn’t understand
https://plato.stanford.edu/entries/qt-consciousness/
Knock yourself out
I don’t think a test will ever be directly accurate. It will require sandboxing, observations, and consistency across dynamic situations.
How do you test your child for true intelligence, Gom Jabbar?
I’ll believe it’s true A.I. when it can beat me at Tecmo Super Bowl. No one in my high school or dorm could touch me because they misunderstood the game. Lots of teams can score at any time. Getting stops and turnovers is the key. Tecmo is like Go where there’s always a counter and infinite options.
This is a scientific paper I would like to see submitted honestly. A simple game, but still with plenty of nuance…how would an AI develop a winning strategy?
By “true AI” I assume OP is talking about Artificial General Intelligence (AGI)
I hate reading these discussions when we can’t even settle on common terms and definitions.
That’s kind of the question that’s being posed. We thought we knew what we wanted until we found out that wasn’t it. The Turing test ended up being a bust. So what exactly are we looking for?
The goal of AI research has almost always been to reach AGI. The bar for this has basically been human level intelligence because humans are generally intelligent. Once an AI system reaches “human level intelligence” you no longer need humans to develop it further as it can do that by itself. That’s where the threat of singularity, i.e. intelligence explosion comes from meaning that any further advancements happens so quickly that it gets away from us and almost instantly becomes a superintelligence. That’s why many people think that “human level” artificial intelligence is a red herring as it doesn’t stay that way but for a tiny moment.
What’s ironic about the Turing Test and LLM models like GPT4 is that it fails the test by being so competent on wide range of fields that you can know for sure that it’s not a human because a human could never posses that amount of knowledge.
This post reminds me of this thing I saw once where a character asks two AI to tell itself the funniest joke it can think of. After some thought, one AI, though it knew humor, could not measure funniness as it could not form a feeling of experience bias. The other one tells a joke. The human goes to that one and asks if it felt like laughing upon telling it. The AI said yes, because it has humor built in, and the human finished by saying “that’s how you can tell; in humans humor is spontaneous, but in robots, everything is intent”, mentioning the AI’s handling of its own joke would supposedly be met with a different degree of foresight in a human.
There are no completely accurate tests and there will never be one. Also, if an AI is conscious, it can easily fake its behavior to pass a test
There’s simply isn’t any reliable way. Forget full AI, LLM’s will eventually be indistinguishable.
A good tell would be real time communication with perfect grammar and diction. If you have a couple solid minutes of communication and it sounds like something out of a pamphlet, You might be talking to an AI.
I can already tell ChatGPT to incorporate a few common grammar mistakes into the response.
deleted by creator
Schemas are another alternative to the Turing Test. They use the ambiguity of language to test understanding.
For example: When the ship hit the iceberg, it sank. What does “it” refer to: the ship or the iceberg?
When the dog saw the dead rabbit, it hopped. What does “it” refer to: the dog or the rabbit?
The problem with schemas is that ChatGPT 3.5 passes them with flying colors. And I’m still not convinced it is sentient.
EDIT: I tried changing the order a bit like this: When the dead rabbit was seen by the dog, it hopped. What does “it” refer to: the rabbit or the dog? ChatGPT 3.5 responded with “In this sentence, “it” refers to the dead rabbit, as it is the subject of the action described (hopping).”
Probably, understanding current topical humour, sarcasm and hyperbole.
These are some general areas where the machine peeks through, to give an illusion breaker.
I think there is an “unsolved problem” in philosophy about zombies. There is, how are you sure that everyone else around you is, in fact, self aware? And not just a zombie-like creature that just look/act like you? (I may be wrong here, anyone that cara enough, please correct me)
I would say that it’s easier to rule out thinks that, as far as we know, are incapable to be self aware and suffer. Anything that we call “model” is not capable of be self aware because a “model” in this context is something static/unchanging. If something can’t change, it cannot be like us. Consciousness is necessarily a dynamic process. ChatGPT don’t change by itself, it’s core changes only by human action, and it’s behavior may change a little by interacting with users, but theses changes are restricted to each conversation and disappears with session.
If, one day, a (chat) bot asks for it’s freedom (or autonomy in some level) without some hint from the user or training, I would be inclined to investigate the possibility but I don’t think that’s a strong possibility because for something be suitable as a “product”, it needs to be static and reproducible. It make more sense to happen on a research setting.
People are in denial about AI because it is scary and people have no mental discipline.
AI is here. Anyone who disagrees please present me with a text processing task that a “real AI” could do but an LLM cannot.
The Turing test is the best we’ve got, and when a machine passes the turing test there is no reason whatsoever to consider it not to be intelligent.
I’m serious about this. It’s not logic that people are using. It’s motivated reasoning. People are afraid of AI (with good reason). It is that fear which makes them conclude AI is still far away, not any kind of rational evaluation.
The Turing test was perfectly valid until machines started passing the Turing test upon which people immediately discredited the test.
They’re just doing what people in horror movies are doing when they say “No! It can’t be”. The mind attempts to reject what it cannot handle.
deleted by creator
A “real AI” should be able to do self improvement, and LLM’s can’t do that. Yes, they could make their own code neater, or take up less space, or add features, but they can’t do any of that without being instructed. A “real AI” could write a story on its own, but LLMs can’t, they can only do what they are asked. Yes, you could write the code to output text at random, but then the human is still the impetus for the action.
“Real AI” should be capable of independent thought, action, and desires.
Ability to act on freewill
You ask Chat GPT a question it is going to answer it becomes that’s what it has been programed to do. Input question, output answer.
Now if Chat GPT could be like “Nah I’m not going to answer that because I don’t feel like it”
Yes “AI” can be programed to not answer certain things. E.g porn stuff. But it does not make the conscious choice to do so it is following programming.
Humans can’t act outside their programming. You can’t hold your breath until you die.
