ChatGPT Isn’t as Good at Coding as We Thought::undefined
You must log in or register to comment.
I’ve experimented a bit with chatGPT, asking it to create some fairly simple code snippets to interact with a new API I was messing with, and it straight up confabulated methods for the API based on extant methods from similar APIs. It was all very convincing, but if there’s no way of knowing that it’s just making things up, it’s literally worse than useless.
ChatGPT has been helpful in being an interactive rubber duck. I used it to help myself breakdown the technical problems that I need to solve and it helps to cut down time taken to complete a difficult ticket that usually take a couple of days of work to a couple of hours.
“just good enough to be dangerous”
I’ve had similar experiences with it telling me to call functions of third party libs that don’t exist. When you tell it “That function X does not exist” it says “I’m sorry, your right function X does not exist on library A. here is another example using function Y” then function Y doesn’t exist either.
I have found it useful in a limited scope, but I have found co-pilot to be much more of a daily time saver.
So? You should be testing the code it generates, a few lines of code at a time, and writing unit tests where appropriate.
By doing that, you’ll quickly identify anything that it does wrong, and you can just tell it “this line didn’t work”. Normally you don’t need to tell it how to fix it, but you can do that too.
Except that in code, you can write unit tests and have checks that it absolutely has to get precisely correct.
If you have to write the code and tests yourself… That’s just normal coding then
You don’t, you get it to write both the code and the tests. And you read both of them yourself. And you run them in a debugger to verify they do what you expect.
Yeah, that’s half the work of “normal coding” but it’s also half the work. Which is a pretty awesome boost to productivity.
But where it really boosts your productivity is with APIs that you aren’t very familiar with. ChatGPT is a hell of a lot better than Google for simple “what API can I use for X” questions.
You might have to rewrite all of it. The code and the tests.
Hell even the structure/outline it took might not be correct.
Yeah but I don’t. That’s the whole damn point.
I really suggest you guys try it.
It’s really really bad very often.
I asked it to write a bash script which simply read a couple inputs using readline and then ran a couple commands inside an if/else
It declared a variable that it never used. I pointed out the mistake and asked to remove the line. It simply renamed the variable.
I’d trust an unpaid intern more
Hah hah… What a dummy!
discreetly pushes his unused variables under desk with his foot
An unused variable!?
/clutches_pearls
I am using ChatGPT 4+ with the code interpreter to code c# scripts inside my Unity project and it works and to be unreliable, b it about a month ago when the code interpreter came out it became very useful. Like it rarely makes a compile error and when it does it often fixes it with no further issues. The code it writes is solid and it has even been able to write multiple, interacting scripts with singletons, etc to do all sorts of more complex things in Unity. It has saved my so much time I am blown away. Some of these scripts are 200-300 lines. Beyond 300 it seems to have many issues though al really only good for the smaller stuff, which is mostly what Unity tends to be.
It is also amazing as feeding it error logs and having it tell you the bits that matter and why. Everyone should be using it for this at a minimum.
I love it, but look forward to the day where it is in my Unity project editor and is able to see all and address all the ridiculous and mundane issues that consume far too much of my time and other developers time. Just finished implementing AssetBundles with it, which triggered many of my scripts needing to be updated, which it did in just a few seconds each. Amazing.
I always thought of chat gpt as a “companion tool” that isn’t meant to write good code by itself, but to help experienced programmers write good code (just like search engines and documentation)
I thought it was terrible. Is it worse than that?
It can be useful for basic coding or to answer questions like ‘Is there any way to do X thing in Javascript?’ I were talking about it with some classmates , they said the same. There was one program I was doing on my own with Js & Html (I’m still learning) and for relying to much on GPT without much knowledge I ended up “walking on circles” for 6 Hours without any progress. It is good for giving some information and sometimes finding a bug, but never, never use it as if it were capable of doing everything. It’s a tool, not a programmer.
I used it extensively to help me code my PHP for an art portfolio site. Briefly thought about using 11ty but needed to put something up quick after being laid off and I knew PHP.
For the most part it was good. It was really good at creating simple functions for me. My issue came when I asked it to build me a JS lightbox in Bootstrap. i wanted it to look a certain way so I had to edit my prompt multiple times because it would edit the code and “forget” my previous modification. Ended up using someone else’s JS code.
It was incredibly frustrating. It’s powerful, but still limited.
I don’t know how others are using chatGPT for coding, but I found I get the best results when starting small and iterate over the results few times. Like:
- write a function to make a GET request;
- write a function to handle this example JSON;
- write a function that combines the first two;
- etc etc
I use it mostly for Typescript, Bash and Clojure and results vary from good to OK (Clojure). The whole process is way faster if you use a tool like sGPT.
It’s like a rookie programmer or an intern at best. There has been times it has been really helpful though.
