topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Thursday December 5, 2024, 7:47 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Last post Author Topic: AI Coding Assistants (Who uses them and which)  (Read 16466 times)

Shades

  • Member
  • Joined in 2006
  • **
  • Posts: 2,939
    • View Profile
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #25 on: October 18, 2024, 10:49 AM »
By the way, in this environment there is is still a i7 CPU 1st generation active as a database server. It has 32 GByte of RAM and SSDs. Yes, it is even more ancient that your second generation i7 system, but it doesn't feel slow in anything it needs to do. So my boss doesn't want to change it, mainly because of (grandfathered) Oracle licensing-costs and Oracle being slowly phased out here.

So, ancient computer or not, as long as you don't game on those computers, they are still quite useful. And energy costs are not that big of a concern here in this part of South America, where the whole country runs on hydro-power.

Deozaan

  • Charter Member
  • Joined in 2006
  • ***
  • Points: 1
  • Posts: 9,775
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #26 on: October 18, 2024, 11:25 PM »
I forgot to mention I have 16 GB RAM in this machine.

I tried it on CPU, CUDA 11 and 12 and Vulkan and I get similar results for all of them in a form looking very much like this:

Code: Text [Select]
  1. 11012.227 s   Starting...←[2m2024-10-19T03:54:38.253368Z←[0m ←[33m WARN←[0m ←[2mllama_cpp_server::supervisor←[0m←[2m:←[0m ←[2mcrates\llama-cpp-server\src\supervisor.rs←[0m←[2m:←[0m←[2m98:←[0m llama-server <embedding> exited with status code -1073741795, args: `Command { std: "D:\\Apps\\tabby_x86_64-windows-msvc\\llama-server.exe" "-m" "C:\\Users\\Deozaan\\.tabby\\models\\TabbyML\\Nomic-Embed-Text\\ggml\\model.gguf" "--cont-batching" "--port" "30888" "-np" "1" "--log-disable" "--ctx-size" "4096" "-ngl" "9999" "--embedding" "--ubatch-size" "4096", kill_on_drop: true }`
  2. ←[2m2024-10-19T03:54:38.253838Z←[0m ←[33m WARN←[0m ←[2mllama_cpp_server::supervisor←[0m←[2m:←[0m ←[2mcrates\llama-cpp-server\src\supervisor.rs←[0m←[2m:←[0m←[2m110:←[0m <embedding>: warning: not compiled with GPU offload support, --gpu-layers option will be ignored
  3. ←[2m2024-10-19T03:54:38.255312Z←[0m ←[33m WARN←[0m ←[2mllama_cpp_server::supervisor←[0m←[2m:←[0m ←[2mcrates\llama-cpp-server\src\supervisor.rs←[0m←[2m:←[0m←[2m110:←[0m <embedding>: warning: see main README.md for information on enabling GPU BLAS support

I guess I'll try LM Studio next.

Deozaan

  • Charter Member
  • Joined in 2006
  • ***
  • Points: 1
  • Posts: 9,775
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #27 on: October 18, 2024, 11:32 PM »
Hmmm... LM Studio won't work for me either. After poking around in the various tabs of the GUI it seems my CPU is the problem. 😭

LM Studio - LM Runtimes.png

Shades

  • Member
  • Joined in 2006
  • **
  • Posts: 2,939
    • View Profile
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #28 on: October 19, 2024, 12:40 AM »
Hmmm... LM Studio won't work for me either. After poking around in the various tabs of the GUI it seems my CPU is the problem. 😭

[ Invalid Attachment ]
Too bad about your CPU.

In all honesty, I did not expect that, as I have used both Tabby and LM studio on a computer with an AMD APU (9700), same GPU and 16 GB RAM. That APU was designed in the 'bulldozer'-era of AMD, but made to fit on the AM4 socket. That era may not be 2nd gen i7, but isn't that far off either. Tabby and LM Studio worked both on that CPU, but it struggled when more than 1 person was using the LLM functionality. Hence I moved it to the next best (or should I say worst) thing, which was the computer with the i3 10100F CPU. Since then, Tabby is "smooth" sailing" with 5 people accessing it.

Here in Paraguay it isn't that hard to find motherboards and CPUs from the 10 gen for cheap. It is also not really a problem to get 13th and 14th gen gear, but that is more expensive than what it costs in the U.S. or Europe. If I remember my parts prices correctly, that i3 computer would have costed about 425 to 450 USD, with the GPU as the most expensive part.

Perhaps it is an option to get/trade your computer (without GPU) for another older computer with a 4th/5th/6th gen CPU for cheap? Refurbished gear or something from a company that dumped their written-off gear at a computer thrift store? For getting your feet wet with LLMs/AI  that could be useful, while also not breaking the bank.

Deozaan

  • Charter Member
  • Joined in 2006
  • ***
  • Points: 1
  • Posts: 9,775
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #29 on: October 21, 2024, 03:21 AM »
I don't really have the budget for upgrading my PC. Maybe I should get a $15 microcontroller and play around with TinyML.

But that's not really an LLM, so I guess I'm better off just accessing some free tier of ChatGPT.

Shades

  • Member
  • Joined in 2006
  • **
  • Posts: 2,939
    • View Profile
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #30 on: October 21, 2024, 09:09 AM »
Don't stare yourself too blind on ChatGPT, because it isn't the "all-in-one" solution for everyone or every use-case.

So here are some (mainstream) alternatives:
- ChatGPT (pretty much on par with Gemini for my use-cases).
- PHind (did well with Powershell requests and Oracle requests I made).
- v0 (relatively new player, that looks promising. Bit of a nag about getting a subscription with them, but that is it).
- Gemini (pretty much on par with ChatGPT for my use-cases).
- CoPilot (no experience with, besides unconsciously activating it by misclicking in the Edge browser).

- Replicate (site that lets you try out many different models, some free, some at costs).

Deozaan

  • Charter Member
  • Joined in 2006
  • ***
  • Points: 1
  • Posts: 9,775
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #31 on: October 24, 2024, 11:55 AM »
I forgot to thank you once again for being so helpful.

So thanks! I had no idea about most of those. I'll check them out. :Thmbsup:

KynloStephen66515

  • Animated Giffer in Chief
  • Honorary Member
  • Joined in 2010
  • **
  • Posts: 3,759
    • View Profile
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #32 on: October 24, 2024, 04:35 PM »
For API Usage:
Mistral have an extremely generous free tier (1 billion tokens per month, 500,000/minute)
Gemini is free with a huge context window (2 million on pro, 1 million on flash models)
Groq (not Grok) is also free (but has heavy usage limits that might not suit everybody, but has EXTREMELY fast output speed)

Also try OpenWebUI, as it's capable of "plugging in" any model from the above list.  Another really good web-ui with a lot of extra features, that can also run locally, is LobeChat (They also have a hosted version)


I don't wanna shamelessly promote the company I work for (We have a very generous free tier for our VSCode/JetBrains/WebUI, AI coding assistant) so if you wanna know about that, drop me a DM here or discord and I'll let ya know (and can hook you up with a month of free Pro tier, too).

Deozaan

  • Charter Member
  • Joined in 2006
  • ***
  • Points: 1
  • Posts: 9,775
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #33 on: October 26, 2024, 05:55 PM »
Thanks Kynlo. While I was adding these to my bookmarks, I decided to try out LobeChat's hosted version. It went something like this (highly paraphrased):


Q: Does the onboard LED of the Raspberry Pi Pico W support PWM to adjust its brightness?

A: Yes, it does.

Q: Can you show me some example code that will fade the LED?

A: This code will do that for you.

[ 15 minutes trying to get the code to work ]

Q: The code isn't working. What might be wrong?

A: Try these troubleshooting steps.

[ 15 minutes trying all the troubleshooting steps ]

Q: Are you absolutely sure the onboard LED supports PWM?

A: My apologies. The onboard LED does not support PWM.

Q: Can you show me example code to simulate PWM to adjust the LED's brightness in software rather than hardware?

A: This code will do that for you.

[ It works! ]

Pretty helpful when it doesn't lead you on a wild goose chase with wrong information.


I'm surprised there hasn't been a movie made yet about a group of people with a robot/AI companion who all end up stuck in a cavern and they ask the robot/AI
to help them find a way out and it unintentionally states incorrect information as fact and based on that information (and future queries to the robot/AI that don't challenge the initial assumption) the humans travel through a treacherous path where many people are injured or die until finally someone asks "Are you sure this is the best course of action?" and then the robot/AI says "My apologies. We can just take the ladder near where we started." ;D

Shades

  • Member
  • Joined in 2006
  • **
  • Posts: 2,939
    • View Profile
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #34 on: October 26, 2024, 06:17 PM »
You could watch a TV-series, called 'Beacon 23'. Which plays in distant future, where humankind uses manned beacons for interstellar travel. And you get a hint of what life is like in such a beacon., which belong to corporations that have AI on those beacons themselves, and tend to provide a personal AI to the person manning the beacon.

Might sound boring for a sci-fi series, but it really isn't. Quite a bit more serious than your movie idea. Which I think would go over well  :D

KynloStephen66515

  • Animated Giffer in Chief
  • Honorary Member
  • Joined in 2010
  • **
  • Posts: 3,759
    • View Profile
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #35 on: October 27, 2024, 08:20 AM »
Thanks Kynlo. While I was adding these to my bookmarks, I decided to try out LobeChat's hosted version. It went something like this (highly paraphrased):


Q: Does the onboard LED of the Raspberry Pi Pico W support PWM to adjust its brightness?

A: Yes, it does.

Q: Can you show me some example code that will fade the LED?

A: This code will do that for you.

[ 15 minutes trying to get the code to work ]

Q: The code isn't working. What might be wrong?

A: Try these troubleshooting steps.

[ 15 minutes trying all the troubleshooting steps ]

Q: Are you absolutely sure the onboard LED supports PWM?

A: My apologies. The onboard LED does not support PWM.

Q: Can you show me example code to simulate PWM to adjust the LED's brightness in software rather than hardware?

A: This code will do that for you.

[ It works! ]

Pretty helpful when it doesn't lead you on a wild goose chase with wrong information.


I'm surprised there hasn't been a movie made yet about a group of people with a robot/AI companion who all end up stuck in a cavern and they ask the robot/AI
to help them find a way out and it unintentionally states incorrect information as fact and based on that information (and future queries to the robot/AI that don't challenge the initial assumption) the humans travel through a treacherous path where many people are injured or die until finally someone asks "Are you sure this is the best course of action?" and then the robot/AI says "My apologies. We can just take the ladder near where we started." ;D

What model did you use for this?

I'd highly suggest only using Claude-3.5-Sonnet-Latest, for coding-related work, as it's the only one that's actually any use.  GPT4o isn't terrible, but given that it has a knowledge cutoff in 2023, it has absolutely no idea about anything more recent.


This is the output I got using Claude (albeit, with a tuned system prompt for coding):

Yes, the Raspberry Pi Pico W's onboard LED (GPIO 25/"LED" pin) does support PWM! I'll show you both hardware PWM and software PWM approaches.

1. Hardware PWM Solution: Here's a complete example that fades the LED up and down using hardware PWM:

from machine import PWM, Pin
import time

# Setup the onboard LED with PWM
led_pwm = PWM(Pin("LED"))  # GPIO 25
led_pwm.freq(1000)  # Set PWM frequency to 1000Hz

def pulse_led():
    while True:
        # Fade in
        for duty in range(0, 65535, 1000):
            led_pwm.duty_u16(duty)
            time.sleep(0.01)
           
        # Fade out   
        for duty in range(65535, 0, -1000):
            led_pwm.duty_u16(duty)
            time.sleep(0.01)

try:
    pulse_led()
except:
    # Clean up PWM on exit
    led_pwm.deinit()

2. Software PWM Solution (if hardware PWM isn't working): This simulates PWM using software timing:

from machine import Pin
import time

led = Pin("LED", Pin.OUT)

def set_brightness(brightness):
    # brightness should be between 0 and 100
    cycle_time = 0.001  # 1ms cycle
    on_time = (brightness / 100.0) * cycle_time
    off_time = cycle_time - on_time
   
    led.value(1)
    time.sleep(on_time)
    led.value(0)
    time.sleep(off_time)

def pulse_led():
    while True:
        # Fade in
        for brightness in range(0, 101, 5):
            for _ in range(10):  # Multiple cycles per brightness level
                set_brightness(brightness)
       
        # Fade out
        for brightness in range(100, -1, -5):
            for _ in range(10):  # Multiple cycles per brightness level
                set_brightness(brightness)

try:
    pulse_led()
except:
    led.value(0)  # Turn off LED on exit

Key differences between the approaches:

Hardware PWM:
  • More efficient
  • Smoother fading
  • No CPU overhead
  • Uses dedicated hardware

Software PWM:
  • More CPU intensive
  • May show slight flickering
  • Works on any GPIO pin
  • Useful when hardware PWM isn't available


To run either example:

mpremote cp main.py :main.py

mpremote reset

The LED should start fading in and out in a continuous loop. The hardware PWM version will give you the smoothest results, but both approaches work to demonstrate LED brightness control.

Deozaan

  • Charter Member
  • Joined in 2006
  • ***
  • Points: 1
  • Posts: 9,775
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #36 on: October 27, 2024, 03:32 PM »
What model did you use for this?
-KynloStephen66515 (October 27, 2024, 08:20 AM)

When I created my account it asked me what I was interested in and I selected the option that appeared to be more about coding and development. It then suggested (and I accepted) a bunch of "assistants" which I assumed it would query all of or pick the best one for me based on my question. So I just started typing away (in the "Just Chat" section). :-[

Looking more into it, it seems to have created a variety of "assistants" for me on various topics but they are all set to use gpt-4o-mini, except for one which is set to use gpt-3.5-turbo-16k.

I'll try adjusting or creating one using Claude and see if that gives better results, but judging on your answer, it still gives you incorrect information. As far as I can tell the onboard LED of the Pico W does not actually support (hardware) PWM, because it's controlled by the WiFi module instead of the RP2040. But there's a big enough knowledge gap for me that I may be wrong about that. Maybe the fact that I'm using Arduino IDE (which I think is just C or C++) rather than MicroPython makes a difference in what hardware features the libraries I'm including use.

KynloStephen66515

  • Animated Giffer in Chief
  • Honorary Member
  • Joined in 2010
  • **
  • Posts: 3,759
    • View Profile
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #37 on: October 27, 2024, 06:30 PM »
Raspberry Pi Pico W's onboard LED

Not to stray too far off-topic, but you seem to be correct, as per: https://forums.raspb...ewtopic.php?t=358964 and https://forums.raspb...ewtopic.php?t=336836

GrumpyCoder

  • Supporting Member
  • Joined in 2015
  • **
  • Posts: 30
    • View Profile
    • https://grumpycoder.win/
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #38 on: December 02, 2024, 11:53 PM »
I've been using Codeium for quite some time now.
It can be used in the browser directly, as an extension for VSCode/VStudio, or through their own editor Windsurf.
Best of all, it's completely free. They do have paid subscriptions with more features, but I never felt that I needed to upgrade.

PS: I'm not affiliated to Codeium in any way.

KynloStephen66515

  • Animated Giffer in Chief
  • Honorary Member
  • Joined in 2010
  • **
  • Posts: 3,759
    • View Profile
    • Donate to Member
Re: AI Coding Assistants (Who uses them and which)
« Reply #39 on: December 04, 2024, 07:04 AM »
I've been using Codeium for quite some time now.
It can be used in the browser directly, as an extension for VSCode/VStudio, or through their own editor Windsurf.
Best of all, it's completely free. They do have paid subscriptions with more features, but I never felt that I needed to upgrade.

PS: I'm not affiliated to Codeium in any way.


I've been trying out Windsurf recently and it has huge potential to be a great product, but it's not quite there yet.  Between the constant issues with Codium hitting their usage caps with Anthropic, and tool_use errors, it still has some way to go.  That being said though, I have enjoyed it so far (even if the "oopsie" errors are half way through it editing a file it refuses to elaborate on so you don't know what it broke).

I do mainly use Cody AI (by Sourcegraph), which is the one I work for, mainly, but that doesn't mean I can't enjoy seeing what other tools are doing  :D ;D