Author Topic: AI Coding Assistants (Who uses them and which) (Read 9129 times)

KynloStephen66515 · « **on:** August 02, 2024, 02:06 PM »

Hey guys,

I'm curious to see who on the DC forum uses AI coding assistants in their IDEs or CLIs to enhance their daily coding workflow.

If you do use them, which one did you pick, and why?!

Ath · « **Reply #1 on:** August 02, 2024, 02:27 PM »

Simple: Nope. To me that's garbage in, garbage out.

KynloStephen66515 · « **Reply #2 on:** August 02, 2024, 02:45 PM »

Simple: Nope. To me that's garbage in, garbage out.
-Ath (August 02, 2024, 02:27 PM)

Not sure what you mean here.

Do you mean the quality of the code? The LLMs themselves? or am I missing something

Ath · « **Reply #3 on:** August 02, 2024, 04:23 PM »

Do you mean the quality of the code? The LLMs themselves? or am I missing something
-KynloStephen66515 (August 02, 2024, 02:45 PM)

Any example I've seen so far seems to be a garbage answer, unusable or totally unfit to what it's supposed to be tailored to. I'll wait and see what's left after the dust of this hype settles

KynloStephen66515 · « **Reply #4 on:** August 02, 2024, 04:50 PM »

Do you mean the quality of the code? The LLMs themselves? or am I missing something
-KynloStephen66515 (August 02, 2024, 02:45 PM)

Any example I've seen so far seems to be a garbage answer, unusable or totally unfit to what it's supposed to be tailored to. I'll wait and see what's left after the dust of this hype settles
-Ath (August 02, 2024, 04:23 PM)

Which assistants have you tried, because I personally know quite a few (and even work for one) that are actually incredible (especially for autocomplete, but also for quickly getting code snippets, answers, bug fixes, code smells, etc)

Shades · « **Reply #5 on:** August 03, 2024, 03:25 AM »

Which assistants have you tried, because I personally know quite a few (and even work for one) that are actually incredible (especially for autocomplete, but also for quickly getting code snippets, answers, bug fixes, code smells, etc)
-KynloStephen66515 (August 02, 2024, 04:50 PM)

Not assistants per se, but I have been using a tool: 'LM Studio' to run 8 LLM's locally. This tool provides an easy way to download LLMs, use one or more of those in the provided chat screen and allows you to run one or more models (at the same time) as a server, which you can access via an API that uses the same form as the OpenAI API.

Right now I'm most impressed with model 'bartowski\StableAI Instruct 3B'. It doesn't take up that much RAM and responds surprisingly well in CPU-only mode, even on a i3 10100F CPU. You can also set it to use the available GPU (NVidia/AMD) if that has enough memory to offload one or more models into. And it allows you to play with quite some model-specific settings for the LLM's you load into memory. LM Studio is freeware.

Sometimes I verify the results bij filling in the exact same prompt into ChatGPT (v3.5, I think that is the free one) and the locally running StableAI model. ChatGPT answers show up faster and usually have a lot more words to convey the same message.

Basic script generation works quite well in both, but ChatGPT can deal with a bit more complexity. Still, for my purposes, the StableAI model hasn't been too far off ChatGPT or too slow in comparison.

The thing I am looking for is a relative easy way to train the StableAI model I have with company-specific documentation, our script language and documentation portals. For that purpose, the open source tool 'LlamaIndex' appears to be very interesting.

Once I can train the LLM I have, turning my local AI instance into a proper personal AI Assistant shouldn't be too much of a problem.

Ath · « **Reply #6 on:** August 03, 2024, 03:36 AM »

Which assistants have you tried
-KynloStephen66515 (August 02, 2024, 04:50 PM)

None, opinion based on the crappy 'examples' / 'answers' that ppl posted in (another) forum.

Basically, I'm sitting out the storm, to see what eventually rolls out without the world crumbling to pieces. (Must be related to my age/experience and somewhat conservative approach to 'the latest hype')

KynloStephen66515 · « **Reply #7 on:** August 03, 2024, 06:07 AM »

Which assistants have you tried, because I personally know quite a few (and even work for one) that are actually incredible (especially for autocomplete, but also for quickly getting code snippets, answers, bug fixes, code smells, etc)
-KynloStephen66515 (August 02, 2024, 04:50 PM)

Not assistants per se, but I have been using a tool: 'LM Studio' to run 8 LLM's locally. This tool provides an easy way to download LLMs, use one or more of those in the provided chat screen and allows you to run one or more models (at the same time) as a server, which you can access via an API that uses the same form as the OpenAI API.

Right now I'm most impressed with model 'bartowski\StableAI Instruct 3B'. It doesn't take up that much RAM and responds surprisingly well in CPU-only mode, even on a i3 10100F CPU. You can also set it to use the available GPU (NVidia/AMD) if that has enough memory to offload one or more models into. And it allows you to play with quite some model-specific settings for the LLM's you load into memory. LM Studio is freeware.

Sometimes I verify the results bij filling in the exact same prompt into ChatGPT (v3.5, I think that is the free one) and the locally running StableAI model. ChatGPT answers show up faster and usually have a lot more words to convey the same message.

Basic script generation works quite well in both, but ChatGPT can deal with a bit more complexity. Still, for my purposes, the StableAI model hasn't been too far off ChatGPT or too slow in comparison.

The thing I am looking for is a relative easy way to train the StableAI model I have with company-specific documentation, our script language and documentation portals. For that purpose, the open source tool 'LlamaIndex' appears to be very interesting.

Once I can train the LLM I have, turning my local AI instance into a proper personal AI Assistant shouldn't be too much of a problem.
-Shades (August 03, 2024, 03:25 AM)

You might be better suited with RAG than a fine tuned model (much quicker to set-up, and vastly easier to keep up to date with ever changing information)

KynloStephen66515 · « **Reply #8 on:** August 03, 2024, 06:09 AM »

Which assistants have you tried
-KynloStephen66515 (August 02, 2024, 04:50 PM)
None, opinion based on the crappy 'examples' / 'answers' that ppl posted in (another) forum.

Basically, I'm sitting out the storm, to see what eventually rolls out without the world crumbling to pieces. (Must be related to my age/experience and somewhat conservative approach to 'the latest hype')
-Ath (August 03, 2024, 03:36 AM)

Well, if you want any suggestions for good tools, let me know as I have a list! (Unbiased and based off of personal experience with the tools in question) XD

anandcoral · « **Reply #9 on:** August 03, 2024, 06:56 AM »

I use ChatGPT directly, to get c# code snippets for my application features.
I do not use AI in ide.

Since I am not well versed with C#, I can vouch it gives me perfect codes, as required, most of the time.
I try to explain it more elaborately if it gives wrong answers. Sometimes I had to leave the query if I do not get what I want.

All the codes I get I use in my application and can say they are working as I need them.

Deozaan · « **Reply #10 on:** August 03, 2024, 06:16 PM »

Well, if you want any suggestions for good tools, let me know as I have a list! (Unbiased and based off of personal experience with the tools in question) XD
-KynloStephen66515 (August 03, 2024, 06:09 AM)

I'm interested in seeing a list, because my experience has left me with the impression that AI models aren't yet fully baked, and rarely give me truly useful results.

KynloStephen66515 · « **Reply #11 on:** August 03, 2024, 08:44 PM »

Well, if you want any suggestions for good tools, let me know as I have a list! (Unbiased and based off of personal experience with the tools in question) XD
-KynloStephen66515 (August 03, 2024, 06:09 AM)

I'm interested in seeing a list, because my experience has left me with the impression that AI models aren't yet fully baked, and rarely give me truly useful results.
-Deozaan (August 03, 2024, 06:16 PM)

Are you interested in IDE based plugins (think GitHub Copilot), or standalone/web-verions (ChatGPT, etc)?

Tuxman · « **Reply #12 on:** August 06, 2024, 05:24 AM »

Two of the main problems with "AI coding assistants" - whether they claim to be able to write a complete application or simply complete variable names based on what they boozed up on the internet last night - are these:

They violate licences. There are many examples (and probably a large number of unreported cases) of parts of code copied verbatim that were under a clear licence, but that clear licence is not part of what was copied.
They are writing rubbish. A work colleague likes to have work supposedly done by ChatGPT. I would need the time he needs to iron out the worst mistakes in the result to simply write what he wanted to achieve myself. Incidentally, this does not seem to be a ChatGPT-specific problem.

And I'm not even taking into account the mental embarrassment I would personally experience were I to be degraded from a developer to a supplicant to the computer...

anandcoral · « **Reply #13 on:** August 06, 2024, 07:06 AM »

Two of the main problems with "AI coding assistants" -
-Tuxman (August 06, 2024, 05:24 AM)

Never take what the seller insist.

All the big sellers of IT are throwing our own codes to us as labeled "AI". Yes this machine instructed program is reading our own open/closed sources and giving us back as "created by AI".

I do not think or use ChatGPT as some "intelligent" one. I take it as a a helper who helps me by "searching and sieving" my query.

kunkel321 · « **Reply #14 on:** August 06, 2024, 09:26 AM »

I'm not a professional coder. I just do AHK v2 for fun. Claude.ai has gotten pretty good. I use if frequently. I mostly used ChatGPT before that. GPT was often pretty good at laying out the logic for how the code should work (assuming I articulated my needs well). It would make code with so many syntax errors, though, that it was easier to just write the code myself. Claude is pretty good with syntax too though.

Tuxman · « **Reply #15 on:** August 06, 2024, 10:37 AM »

If you need to explain how the algorithm is supposed to work en detail, wouldn’t it be easier to just write the algorithm yourself either way?

Deozaan · « **Reply #16 on:** August 06, 2024, 12:17 PM »

Are you interested in IDE based plugins (think GitHub Copilot), or standalone/web-verions (ChatGPT, etc)?
-KynloStephen66515 (August 03, 2024, 08:44 PM)

I haven't tried any IDE-based plugins. I've just used web versions.

Renegade · « **Reply #17 on:** September 09, 2024, 07:17 PM »

I use ChatGPT and CoPilot directly in VS.

The code quality varies. Sometimes good and sometimes way off. But no matter what, you have to check it and verify it. Sometimes I can use it exactly as is, and other times I only need minor modifications. And of course, sometimes I throw it out and start over.

Shades · « **Reply #18 on:** September 10, 2024, 10:14 PM »

Found today another AI "toy" to play with.

It is called tabby and you'll find it on GitHub.

Tabby is indeed an assistant and one that you can self-host. No matter where your Linux, MacOS or Windows computer is running (on-prem/hybrid or cloud) it will work. Instructions to download and run the software are very easy. It will download 2 (smaller) LLM's from Qwen and StarCoder, if it doesn't find these on your system.

Currently I'm testing it with a computer based on a pre-ryzen AMD APU that AMD adjusted to fit on motherboards that support Ryzen 1st gen till 3th gen. That computer also has an old NVidia GeForce 1650 card which has (only) 4 GByte of VRAM on it. And yet, both LLM's fit in there. The website has a listing of which LLM's are supported and their requirements, including the required NVidia development code. It might all sound complicated, it really isn't.

Once you have it running, tabby becomes a server on your computer. Acces it by entering http://localhost:8080 in a browser on the computer that hosts tabby. Or use any other computer with a browser in your network to visit: http://<ip address tabby>:8080

You will be asked to create an admin account the first time you access it. Tabby comes with a community license for free (till 5 users). They also have subscription plans if that is more your thing.

My test machine could be considered crippled. However, tabby performs really well. Even on my testing clunker it is almost as fast as ChatGPT. This amazed me to no end. Sure, the models are small and I have had hardly any time to really check how useful the answers it provides truly are.

But the responsiveness and ease of working with the community version, that was a (very) pleasant surprise. So I thought to mention it here at DC.

Oh, while it comes with its own web interface, in there are links that point to ways on how to incorporate tabby into editors like VSCode. If that is more to your liking.

wraith808 · « **Reply #19 on:** September 17, 2024, 11:25 PM »

Do you mean the quality of the code? The LLMs themselves? or am I missing something
-KynloStephen66515 (August 02, 2024, 02:45 PM)

Any example I've seen so far seems to be a garbage answer, unusable or totally unfit to what it's supposed to be tailored to. I'll wait and see what's left after the dust of this hype settles
-Ath (August 02, 2024, 04:23 PM)

Not sure what you use. We use Copilot at work and it's been a wonder. It helps me not have to do scut work, and I use it in place of going to stack overflow also.

What I use it for:

Algorithms that I know but would have to look up the implementation.
Spotting inefficiencies in code
Unit Tests
Documentation
Explaining new or under documented code
Scaffolding new classes and POCO/Entities from examples
Generating test data

It really takes a lot of scut work out, and lets me concentrate on other work.

I also use one that's linked to our documentation to help take the load off of us having to do support for the API. We have the documentation, but putting it into a format that everyone can find what they're looking for (or even wants to find what they're looking for) is a challenge. Having them ask their question, and being able to link them to documentation/logs has really cut down on our support queues.

Two of the main problems with "AI coding assistants" - whether they claim to be able to write a complete application or simply complete variable names based on what they boozed up on the internet last night - are these:

They violate licences. There are many examples (and probably a large number of unreported cases) of parts of code copied verbatim that were under a clear licence, but that clear licence is not part of what was copied.
They are writing rubbish. A work colleague likes to have work supposedly done by ChatGPT. I would need the time he needs to iron out the worst mistakes in the result to simply write what he wanted to achieve myself. Incidentally, this does not seem to be a ChatGPT-specific problem.

And I'm not even taking into account the mental embarrassment I would personally experience were I to be degraded from a developer to a supplicant to the computer...
-Tuxman (August 06, 2024, 05:24 AM)

Incorrect. Github Copilot is set to filter out code that uses public sources- it's basing it on context within the app. I asked it for a particular algorithm, and had to massage the prompt because the code was getting filtered out.

If you need to explain how the algorithm is supposed to work en detail, wouldn’t it be easier to just write the algorithm yourself either way?
-Tuxman (August 06, 2024, 10:37 AM)

You don't really have to explain in detail, depending on the context and what you're trying to get. In one case, I just couldn't remember the type of algorithm I needed, and gave it a general idea of what I wanted to do (b-tree sort without modification or recursion) and it gave me that I wanted the Morris Traversal, and an example.