The OP tweet seems to be leaning pretty hard on the “AI bad” sentiment. If LLMs make academic knowledge more accessible to people that’s a good thing for the same reason what Aaron Swartz was doing was a good thing.
That would be good if they did that but that is not the intent of the org, the purpose of the tool, the expected or even available outcome.
It’s important to remember this data is not being scraped to make it available or presentable but to make a machine that echos human authography convincingly more convincingly.
On an extremely simplified level, it doesn’t want to answer 1+1=? with “2”, it wants to appear like a human confidently answering an arithmetic question, even if the exchange is “1+1=?” “yes, 2+3 does equal 9”
Obviously it can handle simple sums, this is an illustrative example
It demonstrably is already though. Paste a document in, then ask questions about its contents; the answer will typically take what’s written there into account. Ask about something you know is in a Wikipedia article that would have been part of its training data, same deal. If you think it can’t do this sort of thing, you can just try it yourself.
Obviously it can handle simple sums, this is an illustrative example
I am well aware that LLMs can struggle especially with reasoning tasks, and have a bad habit of making up answers in some situations. That’s not the same as being unable to correlate and recall information, which is the relevant task here. Search engines also use machine learning technology and have been able to do that to some extent for years. But with a search engine, even if it’s smart enough to figure out what you wanted and give you the correct link, that’s useless if the content behind the link is only available to institutions that pay thousands a year for the privilege.
Think about these three things in terms of what information they contain and their capacity to convey it:
A search engine
Dataset of pirated contents from behind academic paywalls
A LLM model file that has been trained on said pirated data
The latter two each have their pros and cons and would likely work better in combination with each other, but they both have an advantage over the search engine: they can tell you about the locked up data, and they can be used to combine the locked up data in novel ways.
On the whole, maybe LLMs do make these subjects more accessible in a way that’s a net-positive, but there are a lot of monied interests that make positive, transparent design choices unlikely. The companies that create and tweak these generalized models want to make a return in the long run. Consequently, they have deliberately made their products speak in authoritative, neutral tones to make them seem more correct, unbiased and trustworthy to people.
The problem is that LLMs ‘hallucinate’ details as an unavoidable consequence of their design. People can tell untruths as well, but if a person lies or misspeaks about a scientific study, they can be called out on it. An LLM cannot be held accountable in the same way, as it’s essentially a complex statistical prediction algorithm. Non-savvy users can easily be fed misinfo straight from the tap, and bad actors can easily generate correct-sounding misinformation to deliberately try and sway others.
ChatGPT completely fabricating authors, titles, and even (fake) links to studies is a known problem. Far too often, unsuspecting users take its output at face value and believe it to be correct because it sounds correct. This is bad, and part of the issue is marketing these models as though they’re intelligent. They’re very good at generating plausible responses, but this should never be construed as them being good at generating correct ones.
Just let anyone scrape it all for any reason. It’s science. Let it be free.
The OP tweet seems to be leaning pretty hard on the “AI bad” sentiment. If LLMs make academic knowledge more accessible to people that’s a good thing for the same reason what Aaron Swartz was doing was a good thing.
That would be good if they did that but that is not the intent of the org, the purpose of the tool, the expected or even available outcome.
It’s important to remember this data is not being scraped to make it available or presentable but to make a machine that echos human authography convincingly more convincingly.
On an extremely simplified level, it doesn’t want to answer 1+1=? with “2”, it wants to appear like a human confidently answering an arithmetic question, even if the exchange is “1+1=?” “yes, 2+3 does equal 9”
Obviously it can handle simple sums, this is an illustrative example
It demonstrably is already though. Paste a document in, then ask questions about its contents; the answer will typically take what’s written there into account. Ask about something you know is in a Wikipedia article that would have been part of its training data, same deal. If you think it can’t do this sort of thing, you can just try it yourself.
I am well aware that LLMs can struggle especially with reasoning tasks, and have a bad habit of making up answers in some situations. That’s not the same as being unable to correlate and recall information, which is the relevant task here. Search engines also use machine learning technology and have been able to do that to some extent for years. But with a search engine, even if it’s smart enough to figure out what you wanted and give you the correct link, that’s useless if the content behind the link is only available to institutions that pay thousands a year for the privilege.
Think about these three things in terms of what information they contain and their capacity to convey it:
A search engine
Dataset of pirated contents from behind academic paywalls
A LLM model file that has been trained on said pirated data
The latter two each have their pros and cons and would likely work better in combination with each other, but they both have an advantage over the search engine: they can tell you about the locked up data, and they can be used to combine the locked up data in novel ways.
the problem is you can’t take those weaknesses and call it “academic” - it’s a contradiction in terms.
When a real academic makes up answers its a problem, when chatgpt does it its part of the expectation.
i agree, my problem is that it wont
On the whole, maybe LLMs do make these subjects more accessible in a way that’s a net-positive, but there are a lot of monied interests that make positive, transparent design choices unlikely. The companies that create and tweak these generalized models want to make a return in the long run. Consequently, they have deliberately made their products speak in authoritative, neutral tones to make them seem more correct, unbiased and trustworthy to people.
The problem is that LLMs ‘hallucinate’ details as an unavoidable consequence of their design. People can tell untruths as well, but if a person lies or misspeaks about a scientific study, they can be called out on it. An LLM cannot be held accountable in the same way, as it’s essentially a complex statistical prediction algorithm. Non-savvy users can easily be fed misinfo straight from the tap, and bad actors can easily generate correct-sounding misinformation to deliberately try and sway others.
ChatGPT completely fabricating authors, titles, and even (fake) links to studies is a known problem. Far too often, unsuspecting users take its output at face value and believe it to be correct because it sounds correct. This is bad, and part of the issue is marketing these models as though they’re intelligent. They’re very good at generating plausible responses, but this should never be construed as them being good at generating correct ones.