| Author |
 Topic  |
|
|
Christof Schardt
Junior Member
20 Posts |
 Posted - May 17 2026 : 06:41:50 AM
|
I gave it a first try today.
I selected a 100-line function (part of a JSON parser) and asked it to flag any problematic code and annotate it inline.
It took 2.5 minutes to come back with a single finding: that NULL should be replaced with nullptr.
For comparison, I pasted the same code into the Claude chat window (via Mammouth), and within 2�3 seconds I had a well-structured list of 5 issues ranging from minor to serious, clearly explained and neatly presented.
Problem 1: The performance is unacceptable.
Problem 2: The quality of VA's AI is nowhere near that of an external AI.
The supposed benefit of a seamless, integrated workflow doesn't hold up if the underlying results are this far behind and take this much time.
Am I expecting too much from VA's built-in AI, or is this a known limitation?
Would it make sense, if in addition to the local model an interface to some user defined external LLM would be provided (for people who don't care about locality)? |
|
|
David_Kelly
Junior Member
12 Posts |
Posted - May 18 2026 : 07:14:28 AM
|
quote:
Originally posted by Christof Schardt
It took 2.5 minutes to come back with a single finding: that NULL should be replaced with nullptr.
Hi Christof,
could you please also tell us what hardware you used for the test?
Memory? Which GPU? GPU memory?
|
 |
|
|
Christof Schardt
Junior Member
20 Posts |
Posted - May 18 2026 : 2:15:57 PM
|
[/quote]
Memory? Which GPU? GPU memory?
[/quote]
AMD Ryzen 9 7900 12-Core 3.70 GHz
32,0 GB RAM
AMD Radeon(TM) Graphics (486 MB)
|
|
|
|
feline
Whole Tomato Software
United Kingdom
19379 Posts |
Posted - May 19 2026 : 12:31:26 PM
|
If you run task manager are you seeing any GPU usage?
Remember, running a local model isn't likely to match the performance of running on a chunk of a data centre. But still, this doesn't seem good. I am working to construct some useful test case examples to work on here.
As for using an external LLM, I have put in a feature request, to see what people make of it, but it has an obvious issue of removing the advantage of a purely local LLM, but if this is what people want to use, we aren't going to stop them.
case=166188 |
zen is the art of being at one with the two'ness |
|
|
| |
Topic |
|
New VA Intelligence
