Trainers showing off or the LLM showing off?I imagine 'AI' developers think people are more impressed by quantity, that quantity gives an illusion of quality.
I was surprised the smaller LLM's worked on Pi4, but they do get things a bit hotter.
I am not overclocking my Pi5, yet.
Yep, I found I can ask for the short answer and get less waffle.
I can usually get a short answer before the CPU redlines.
Using extra energy for waffle does not make sense.
Not sure why it needs so much thinking time to add 2+ 2?
Sometimes I don't need a chatty bot, just a calculator.
I know some humans I should preface with "Short answer please...."As you probably know, you can make a modelfile including a system message, telling the LLM to keep the answer short. That really works, better than with human Image may be NSFW.
Clik here to view.
I am one of them Image may be NSFW.
Clik here to view.
I keep finding CPUs with NPU on them.
Rockchip 3588, AMD Ryzen AI, Intel Core Ultra and now Snapdragon 8s.
Going into SBCs, Laptops, Desktops, Mobilephones.
Then there is all those Edge IoT with smaller NPU versions......
While the Pi5 uses it brain to the max, code is getting optimized.
Microsoft has been doing much in NN maths libraries for Windows11, ditto for Intel, AMD etc.
It might not have escaped the RPT engineers attention, so NPU coming to Pi next?
Need to do some more testing to see which LLM is the most "Trustworthy".
I had one give me "2 + 2 = 4 but that is not right because 2 is greater than 4".
Running these locally on Pi5 means more of us can test these things.
https://www.confident-ai.com/blog/llm-t ... strategies
Evaluate and Test...
https://github.com/confident-ai/deepeval
Will it work on a PI5?
Most Python based AI stuff seems to work fine on Pi's Image may be NSFW.
Clik here to view.
Statistics: Posted by Gavinmc42 — Fri Mar 29, 2024 12:55 am
