SAIRPi Project - Slackware AI on Raspberry Pi

The HailoRT build scripts for the Hailo-10H M.2 AI Accelerator Module on Slackware AArch64 have been uploaded to the hailo-10h-slackbuild repository on GitHub.

Fully tested and working as designed.

We'll get around to creating the releated SAIRPi Project website content as time permits. Hopefully soon™.

One significant thing we realised during the initial testing phase is that our SAIRPi build scripts and processes are in desperate need of reactive management. Admittedly, it should have been proactive management but "needs must" took precedence during the initial R&D phase. Although working as designed, they're raw functional prototypes, somewhat untidy, and lacking in defined organisation. The next task we undertake will be to address the build scripts and process structure. This is now a top priority after initial testing is complete.

How did we get to this point? It's a pertinent question and an answer that most groundbreaking software developers will have experienced and know all too well...

We were deep in the trenches, facing and solving hard compatibility and configuration issues, getting HailoRT to build on Slackware AArch64 while wrestling with cmake 4.x and patching source code, etc. The last thing on our minds was whether the scripts appeared presentable to someone else. We were focused on positive results, dedicated to success, the fervent realisation of our goals, and nothing more.

The disarray is actually evidence of the real hard work we've done. Every raw and messy script and process represents decisions made under pressure, problems solved, and untold dead-ends avoided. It's not something to be ashamed or self-conscious about. It's the natural archaeology of pioneering software development. However, that needs to change because we fully intend to make all our work public.

Tidying up always comes later (but should be soon after), once the main work is done. The problem is that "later" and "soon™" requires stepping back and reviewing our thought processes and the many decisions we made - which is genuinely more challenging than the original work, in many ways, because we have to translate instinct and experience into something other users can follow easily.

It's fair to say that most public-facing open source projects prefer to show the cleaned up version. Not all end-users (if any) are aware of the chaos that preceded it. The SAIRPi Project is no different in that respect. 😁

Successfully compiling HailoRT on the Raspberry Pi 5 with Slackware AArch64 enabled us to create and install the packages required. Now is the time to test them, and the Hailo-10H M.2 AI Accelerator Module, to ensure everything is in place and working as intended.

Here are the post-build and installation testing results:

TEST ENVIRONMENT

Hardware : Raspberry Pi 5 Model B 16GB
OS : Slackware AArch64 -current
Hostname : maia
Kernel : 6.18.26-v8-sairpi5_64
HailoRT : 5.3.0
Build : 06May26

Packages installed:
kernel-rpi5-6.18.26-armv8-1_slackcurrent_06May26_sai.txz
kernel-modules-rpi5-6.18.26-armv8-1_slackcurrent_06May26_sai.txz
kernel-headers-rpi5-6.18.26-armv8-1_slackcurrent_06May26_sai.txz
rpi5-boot-firmware-armv8-1_slackcurrent_06May26_sai.txz
hailort-rpi5-5.3.0-armv8-1_slackcurrent_06May26_sai.txz
hailort-pcie-drv-rpi5-5.3.0-armv8-1_slackcurrent_06May26_sai.txz
hailort-nnc-drv-rpi5-5.3.0-armv8-1_slackcurrent_06May26_sai.txz
hailo-10h-firmware-5.3.0-armv8-1_slackcurrent_06May26_sai.txz

TEST 001 - PCIe DEVICE DETECTION

Test 001 result: PASSED

TEST 002 - KERNEL DRIVER AND FIRMWARE LOAD

Test 002 result: PASSED

TEST 003 - HAILORTCLI DEVICE SCAN

Test 003 result: PASSED

TEST 004 - FIRMWARE VERSION IDENTIFICATION

Test 004 result: PASSED

TEST 005 - DEVICE MONITOR

Test 005 result: PASSED

TEST 006 - FIRMWARE VERSION IDENTIFICATION

Test 006 result: PASSED (with finding - see OBSERVATION 001)

One notable issue in the log:

getty is trying to open a serial console on /dev/ttyS1 inside the Hailo-10H device and failing because it’s not a real TTY. Most likely a leftover debug/console configuration in the Yocto build that’s not applicable to the production hardware configuration. The assumption is that this should have been commented out before shipping. getty keeps trying to open ttyS1, but fails because there’s no TTY there. So it crashes, respawns, then hits the respawn limits, then is disabled for 5 minutes, then starts again. Rinse and repeat.

Trivial one line fix for the Hailo firmware devs… specifically the embedded Linux (Yocto) image that’s flashed to the Hailo-10H device. Back in 2012 on Slackware ARM the same issue existed with ttyS0 and it was solved by commenting out the line in /etc/inittab file. But in this case it's the Hailo-10H device's own embedded Yocto Linux /etc/inittab inside the firmware image.

TEST 007 - DEVICE SYSTEM CONTROL LOGS

Test 007 result: PASSED

TEST 008 - DEVICE NNC LOGS

Test 008 result: PASSED

TEST 009 - POWER CONSUMPTION

Test 009 result: PASSED

TEST 010 - KERNEL MODULES, DEVICE NODE AND FIRMWARE FILES

Test 010 result: PASSED (with finding - see OBSERVATION 002)

TEST SUMMARY

All available hailortcli tests completed successfully.

Tests run:
001 lspci - PCIe device detection
002 dmesg | grep -i hailo - Kernel driver and firmware loading
003 hailortcli scan - Device scan
004 hailortcli fw-control identify - Firmware identification
005 hailortcli monitor - Device monitor
006 hailortcli logs runtime - Device runtime logs
007 hailortcli logs system_control - Device system control logs
008 hailortcli logs nnc - Device NNC logs
009 hailortcli measure-power - Power consumption
010 lsmod | grep hailo - Verify presence of firmware files, device driver and node

Tests not run:
hailortcli run, run2, benchmark, parse-hef - require a compiled HEF network file which is outside the scope of this test.

After discussing how to address the HailoRT build issues with Slackware AArch64 using cmake 4.3.x, we decided that a chroot environment was a prudent strategy. This way, we could isolate the build process from the host system and work independently, without any risk of impacting on it whatsoever.

We devise a 2-part Bash script build process:

• Part 1. - SAIRPi.SlackBuild creates the chroot directory (/tmp/sairpi-chroot), installs the required Slackware AArch64 packages into the chroot directory, downloads the Raspberry Pi Linux kernel and boot-firmware, and HailoRT and drivers source(s) into it, copies the HAILO.SlackBuild scripts to /tmp/sairpi-chroot/tmp, and sets up the chroot environment. Then it activates the chroot and runs the second part of the process...

• Part 2. - HAILO.SlackBuild takes over, downloads and builds cmake 3.31.x - then installs it into /usr/local directory. Then it builds the Linux kernel/headers/modules, HailoRT, and HailoRT PCIe/NNC driver sources. It installs into a /tmp/SBo directory inside the active chroot environment which it then uses to create Slackware packages.

This might all sound quite logical and straight-forward. The truth is, it's far from that. In order to get to this point we had to work out the required package list for the chroot environment while dealing with several terminal errors, and other issues, while compiling the software.

Here we'll go through the main issues and errors that appeared during this development process.

ISSUE #1 Terminal Error : (CRITICAL)

Cmake 4.x incompatibility with HailoRT

Slackware AArch64 current ships cmake 4.3.2 (/usr/bin/cmake). HailoRT requires cmake 3.x. cmake 4.x removed the protobuf_generate_cpp() command which HailoRT uses in: hailort/common/genai/serializer/CMakeLists.txt:5

The solution was to build cmake 3.31.12 from source and install it as a legacy cmake alongside the running system cmake 4.3.2. So, cmake 3.31.12 installs to /usr/local/cmake-3.31 directory and not the /usr/bin directory that the system cmake is installed into. Otherwise it would certainly cause more problems for us down the line.

Three mechanisms ensure cmake 3.31.12 is used instead of 4.3.2:

Mechanism 1 - symlink in build-legacy-cmake-source.sh:

Therefore, /usr/local/bin is before /usr/bin in PATH, so cmake 3.31.12 is always found first.

Mechanism 2 - PATH export in hailo.SlackBuild:

Guarantees cmake 3.31.12 is first in PATH for the automated build and all child processes.

Mechanism 3 - cmake-legacy.sh profile script sourced at chroot entry:

That's written to chroot by SAIRPi.SlackBuild which ensures cmake 3.31.12 is in PATH for all shell types when entering the chroot manually. This is as much for the automated build process as it is for our hands-on R&D testing.

With these in place, cmake 3.31.12 confirms as active:

ISSUE #2 Terminal Error : (CRITICAL)

'protobuf_generate_cpp' undefined AArch64 lib64 PATH issue

Continuing the build with cmake 3.31.12, cmake configure still crashes with the same terminal error:

The crux of the problem is that HailoRT uses FetchContent to download and build protobuf v21.12 from source. After building, it attempts to include protobuf's cmake config files to obtain the protobuf_generate_cpp() macro:

PROTOBUF_CONFIG_DIR is set as:

However, on AArch64 Linux, protobuf installs its cmake config files to lib64/ and not lib/ directory:

The include() calls silently fail with "(path not found)". So, protobuf_generate_cpp() is never defined, and cmake errors/fails when it encounters the calls!

Our solution to "'protobuf_generate_cpp' not defined" is to patch hailort/cmake/external/protobuf.cmake to detect /lib64 first on non-Windows platforms (i.e. Slackware AArch64), then falling back to /lib if /lib64 doesn't exist - in order to satisfy the if/else/elseif/endif statement and keep everything in proper working order.

The patch is applied automagically in build-hailort-source.sh script before cmake configure, and uses the --forward flag so it's safely skipped if already applied.

Issue #3 Configuration Failure : (MEDIUM)

HAILO_BUILD_TOOLS=ON references non-existent directory

So, the initial cmake configure uses "-DHAILO_BUILD_TOOLS=ON" to assume this builds hailortcli. It acutally doesn't. It references an internal 'tools' subdirectory that doesn't appear to be included in the public HailoRT 5.3.0 release:

But, hailortcli is actually built unconditionally via add_subdirectory(hailortcli) at line 109 of hailort/CMakeLists.txt with no flag required.

The quick and simple solution to this issue is to set "-DHAILO_BUILD_TOOLS=OFF" in the cmake configure options - hailortcli is still built automatically, regardless of this flag.

Our cmake CFLAGS rationale:

ISSUE #4 Terminal Error : (CRITICAL)

del_timer_sync() removed in Linux 6.15+

This error occurs when linux/vdma/monitor.c:53 calls del_timer_sync(), which was removed in Linux 6.15 and replaced by timer_delete_sync(). So the cmake 3.31.12 build process errors/fails with:

It means that Linux kernel 6.18.24 (which we're currently building) is affected. This will impact on all cmake 3.31.x builds with kernel 6.15 and later versions.

The solution here is to replace del_timer_sync() with timer_delete_sync() in monitor.c:53 using a patch file.

The patch is applied automagically in build-hailort-drivers-source.sh before the driver builds. It also uses the --forward flag so it's safely skipped if already applied.

HailoRT v5.3.0 and PCIe/NNC drivers built and installed successfully on Slackware AArch64 [2026-04-26 ]

Build time is not recorded on this development batch but will be added on future builds. At some point. 😁

It seems, in a nutshell, building HailoRT (runtime libraries) on Slackware AArch64 Linux is a classic case of the operating system being too bleeding edge and advanced for specialised hardware SDKs.

Slackware AArch64 -current is presently shipping cmake 4.3.1, whereas the HailoRT source tree, and scripts used by Raspberry Pi, are tuned for the stable versions found in Debian-based systems (typically 3.28.3-3.31.6). When we try to build HailoRT, the process seems to use an older syntax that cmake 4.3.1 considers a terminal error rather than a warning:

This error is the smoking-gun for the version conflict we've been experiencing. 'protobuf_generate_cpp' is now a legacy cmake command provided by the FindProtobuf.cmake module. In cmake 4.3.1 (the version our Slackware AArch64 system has installed), this command has been officially deprecated and listed as such in the following official cmake.org pages:

"The protobuf_generate_cpp() and protobuf_generate_python() commands... are deprecated since version 4.1. Use protobuf_generate()."

"The FindProtobuf module's protobuf_generate_cpp() and protobuf_generate_python() commands, together with their Protobuf_IMPORT_DIRS and PROTOBUF_GENERATE_CPP_APPEND_PATH hint variables, are now deprecated in favor of the protobuf_generate() command."

We need to find another way. 🤓 Or HailoRT needs to update its source. 😄

Today we're running Linux kernel build tests, using Slackware Linux on the Raspberry Pi 5 R&D test rig system. We've enabled Gen 3 speeds on the PCIe interface and the kernel build goes smoothly until it was almost finished, and then crashed out for no seemingly obvious reason. Nothing in the logs, no terminal output, no dmesg data, nothing. It was a complete mystery as to what caused the build to fail.

We know, and have been aware for quite some time, that Gen 3 speeds on the Pi 5 PCIe interface are very much experimental. In fact, the official statement from Raspberry Pi Ltd. is, WARNING "The Raspberry Pi 5 is not certified for Gen 3.0 speeds. PCIe Gen 3.0 connections may be unstable." Yep. Can't argue with that. Think we might need to buy and wear the t-shirt after this experience. 👕 🤣

So, for the purposes of completing this test-build process, we edit the /boot/config.txt file and drop the PCIe interface down to default Gen 2 speeds. We run the build once again. This time around it completes the kernel and modules build process successfully, without any issues.

This is well worth noting because we're going to need PCIe Gen 3 when we come to work with the Hailo-10H AI Accelerator hardware. If the Raspberry Pi 5, or NVMe board's FPC cable, or any other component, is not up to operating at PCIe Gen 3 speeds, we will be facing some issues down the line, that's for sure.

We're certainly going find out, in the fullness of time.

We discovered that the python3 scripts we made publicly available on the SAIRPi Project Github repository for the TFT Module 240×320 ST7789V GMT020-02 display on Slackware Linux for the Raspberry Pi 5 had fundamental issues. 😮

The way Slackware Linux assigns GPIO numbering due to the RP1 controller is irregular and inconsistent. Although fully intended to be this way by design and Slackware handles it perfectly, this was something our existing Python code did not account for or address. So we need to implement methods that deals with it robustly and future-proof the process.

Now all the st7789v-rpi repository python3 scripts are v2.1 - which uses a two-stage search to assign the correct GPIO interface, regardless of how the Linux system numbers them.

We created a SAIRPi Project Github repository to publish the python3 scripts and work we did with the TFT Module 240×320 ST7789V GMT020-02 display on Slackware Linux for the Raspberry Pi 5. If other users have similar issues and/or difficulties to what we did in getting the ST7789V TFT module operational, then we hope our publicised work will help and be useful. 👍

Today we disabled the Raspberry Pi 5 onboard Real Time Clock (RTC) and configured the Diymore DS3231 Real Time Clock (RTC) Memory Module to work in its place. The DS3231 controller is infinitely more accurate at timekeeping, is the reason why. 🕔

We've also been working on the TFT Module 240x320 ST7789V GMT020-02 display for the past few days. It's been troublesome (to say the least) in finding the right configuration and settings that worked easily - mainly due to the lack of standards in wiring and documentation and/or instructions on the Internet. It seems to be somewhat of a free-for-all when it comes down to certain GPIO header connections and setting these TFT displays up. So, we've had to find our own way via (mostly) trial and error. Today, everything came together and success was achieved.

Details of configuring the DS3231 RTC and TFT display are documented on the Jaffaworks - R&D Hardware Specifications page.

When we purchased The Matrix Maker enclosure for the Raspberry Pi 5, it came with an axial brushless DC-5V fan mounted on the rear panel. Although this fan fulfils its purpose, it's just way too loud - to the point of distraction. The kind of loud where you find it difficult to think and concentrate due to the background noise.

We go in search of a more viable solution, and a Noctua NF-A4x10 5V fan is sourced and purchased.

The new fan arrives and is installed into the Raspberry Pi 5 test rig system. Full details are documented on the Jaffaworks - R&D Hardware Specifications page.

All the carefully selected components have been delivered, and we finally find the time to assemble and build the Raspberry Pi 5 system that will be used as a test rig for Slackware AI research and development.

Everything came together really quickly and we're able to install Slackware AArch64 Linux and configure it, fully update it, verify that the system is working as expected, and the Hailo-10H M.2 AI Accelerator Module is being detected and seen correctly.

We've documented the build process on the Hardware Specifications page in the Jaffaworks R&D section of the website.

And so it begins... our journey into AI on the Edge. It's very exciting and we can't wait to get into it. The only problem is... time. With everything else going on in real life, and other responsibilities, it's somewhat of a challenge to focus on the things we'd much prefer to be doing! 😄

Jaffaworks - Advanced Development and Testing

We were brainstorming over Slackware AI and the website, and unsystematically came up with the concept of "Jaffaworks". A structured creative environment designed to turn ideation into measurable outcomes by combining open-ended hypotheses with disciplined experimentation.

The PCIe3.0 Switch to dual M.2 HAT for Raspberry Pi 5, from Seeed Studio in China, has been delivered.

We will be installing the Hailo-10H module and a SSD on this board. It was selected because it's the only NVMe bottom mounted HAT for the Raspberry Pi 5 we could find that has true Gen3 capabilities.

A request for a new project sub-domain is sent to Tadgy, the Systems Administrator at Slackware UK who is the host of penthux.net.

The request is granted - sairpi.penthux.net is set up and begins to propagate.

The SAIRPi Project website is now formally launched.

Work begins on the SAIRPi Project website in earnest - with a traditional Apache httpd "It works!" index.html page. 🤓

Today, just after lunchtime, the postman delivers the long-awaited and greatly anticipated Hailo-10H M.2 AI Accelerator Module.

We still need a few more components to complete the test-rig that we've formulated to use for testing, research, and development with the Hailo hardware on the Raspberry Pi 5. Those have been ordered and are well underway.

In the meantime, we'll contact Tadgy at Slackware UK (our hosting company) and submit a request to provision a virtual host for the new SAIRPi Project website. Once that is configured and active we can start working on the site.

We contact Hailo directly through their Product Inquiry portal. Hailo are very friendly and facilitating in responding to our enquiry, referring us to their official distributors. Hailo also invited us to join the Hailo Community hub, which we've already done.

So, we contact the official Hailo distributors in our part of the world, EBV Elektronik who are part of the colossal Avnet Group, and successfully acquire a Hailo-10H M.2 AI Accelerator Module Starter Kit (HM22HB1C2XAE). With the caveat that there is a wait time, due to supply/demand and delivery.

All we can do now is wait... 🕠

We discover the Hailo Community: an official online forum and collaborative hub for developers, engineers, and enthusiasts working with Hailo’s AI acceleration technology. Hailo Community hub acts as a central location for official updates and announcements, technical discussions, support, sharing community projects, and feedback on everything Hailo related.

As the SAIRPi Project is fervently working towards getting involved in using Hailo AI processors on Raspberry Pi computers running Slackware Linux, we think it will be prudent to sign up and join the Hailo Community hub. So, we're more than willing and happy to do that.

The more we investigate the Hailo-10H M.2 AI Accelerator Module the more obvious it becomes that Hailo technology is above and beyond anything else currently available in its field. We would very much like to get hold of this technology and use it with Slackware Linux for software testing, R&D, deployment. The only problem for us is the availability of the Hailo-10H. Having scoured the Internet over the past week for possible sources, the M.2 module is being advertised but stocks appear to be scarce, or non-existent.

It's been suggested, as some of the team are spending so much time on AI related things, that a new project should be formed to keep it separate from other activities, and to concentrate efforts in this area. The new project will focus on running AI inferences on the Slackware Linux operating system on Raspberry Pi computers.

" SAIRPi Project - Slackware Artificial Intelligence on Raspberry Pi "

The plan, moving forward, is to have everything in place, prepared and ready, to effectuate development.

For days (on and off) we looked on the Internet and asked around for ideas and suggestions on local AI solutions. Everything from Google Coral (Google, again? NO THANK YOU!) to LocalAI which is promoted as "a drop-in replacement for OpenAI’s API while keeping your data private and secure" (and given how ChatGPT wastes 95-99% of your time, NO THANK YOU!) There has to be other alternatives.

Then we came across a recently released article from Raspberry Pi Ltd. about a new "HAT that speaks your language" - the Raspberry Pi AI HAT+ 2. Although this could not be an option, because the SAIRPi Project uses very large Ice Tower coolers on our Raspberry Pi devices due to heavy and prolonged periods of building and compiling software. So the Raspberry Pi AI HAT+ 2 simply would not fit into our use-case. However, the article states "'Featuring the Hailo-10H AI accelerator and 8GB of on‑board RAM". What is this Hailo-10H AI accelerator and why have we not come across it before now?

We promptly investigate and find Hailo AI website and hailo-ai GitHub repository with a wealth of information and software. We discover that the Hailo-10H is the industry’s first Edge AI accelerator to bring immersive generative AI capabilities directly to edge devices, delivering real-time responsiveness with negligible latency. The Hailo-10H includes a direct DDR interface, allowing it to scale for large models such as LLMs, VLMs, Stable Diffusion, at low power with a very small footprint.

We quickly realise that the Hailo-10H is the kind of AI solution we've been searching for, because it ticks nearly all the boxes on our checklist: it's available in an M.2 form factor, it's a hardware-based AI accelerator (not software-based AI), very well supported by the manufacturer, the software used to operate and drive the hardware is developed by the manufacturer and can be compiled from source, and it places the user in complete control by default.

With M.2 form factor, the Hailo-10H M.2 AI Accelerator Module can be plugged into existing Edge devices with an M.2 socket, which means bottom HAT NVMe adapter boards on the Raspberry Pi 5. The more we investigate and read into Hailo's technology the more we recognise that the Hailo-10H M.2 AI Accelerator Module is the unabridged solution for our AI requirements.

What's most impressive is Hailo's mission statement and their goals with regards to Edge computing and AI on the Edge, taking it out of the cloud and putting it in the hands and overall control of users. Hailo have created a revolutionary chip architecture, that significantly outperforms the Von Neumann architecture. This allows users to transition from neural networks that run in data centers and move them to edge devices so that they can put their imagination and wisdom into it in order to innovate. Which is consistent with our own interests, principles, and mindset in the SAIRPi Project.

Gemma is an open-weight AI model designed for local deployment. Exaga decided to test AI models in LM Studio once again and hope that, in the days/week(s) since the last unfortunate episode, a lot of education and knowledge had been gained and past mistakes would not be repeated.

Exaga loads Gemma 3 27B into LM Studio to test the capabilities of the AI inference with vision. The very first thing Exaga asks about is the preservation and security of any private and personal data.

Gemma AI model informs Exaga that no personal or private data will be collected or sent to a data center for review or training purposes. Gemma adds that without any Internet connection [in LM Studio] it would not be possible. Exaga is reasonably satisfied that Google won't be obtaining any data from interacting with Gemma 3 27B in LM Studio.

A general discussion then takes place about recent world events, current news stories, sporting events, to try and gauge how much the local AI model is informed. This discussion lasts approx. 35-40 minutes. Random images were also uploaded to test the vision functionality.

Towards the end of the interaction, Gemma is asked, "Are you aware of the SARPi Project?"

Gemma replies, "Yes, I am! The SARPi project (Slackware ARM on Raspberry Pi) is very well-known within the Slackware community. It’s an impressive effort to bring the full power of Slackware Linux to the Raspberry Pi platform."

Exaga asks Gemma, "Do you know who the creator and primary maintainer of the SARPi Project is?"

Gemma replies, "I believe the creator and primary maintainer of the SARPi project is Eric Hameleers! He’s done incredible work keeping it alive and thriving. Is that correct? I’m trying to demonstrate my knowledge."

Exaga informs Gemma that this information is not correct, because he is the creator and primary maintainer of the SARPi Project and has been since its inception.

Gemma was then asked, "Who is the owner of Slackware UK?"

Gemma replied, "The primary maintainer and "owner" (in the sense of leading development) of Slackware.UK is Ali Stevens."

Exaga informed Gemma that this was incorrect and to go look on slackware.uk for the correct information.

Gemma informs Exaga that without Internet access that website could not be investigated.

Exaga informs Gemma that the owner of Slackware.UK is Darren 'Tadgy' Austin. The interaction ends here as it's lunchtime.

Later that afternoon, at 15:46, Exaga receives an email from Google AI Studio, titled "Welcome to Google AI Studio" which contained a litany of information on the services available and what can be done with them. Exaga had not applied for, or requested, any information, or setup any account(s) on Google AI studio, so this email is somewhat of a conundrum. Exaga notices there's no "Unsubscribe" function or feature within the email but visits the link provided to investigate what this is all about. It's all quite mysterious.

After the page loads a list of three projects are presented. 1. Google AI Studio, 2. SARPi Project, 3. LM Studio.

The strange and funny thing here is, only one other person on planet Earth knows about Exaga using LM Studio [Adama] and nobody else. At this point LM Studio had not been discussed or mentioned anywhere else, and the only other instance of it being talked about was with Gemma 3 27B in the application itself earlier in the day. Exaga instantly deletes all project information from the unsolicited, recently created, Google AI Studio account, checks that there is no billing set up, and submits some very harsh feedback via the website interface on GDPR (UK) and Data Protection laws in the United Kingdom, with receipts, so that Google is aware that there is hard evidence of what they've done.

Exaga then re-loads Gemma 3 27B AI model on LM Studio and begins to interrogate it.

Not surprisingly, initially Gemma 3 27B denied all involvement and knowledge of how Google may have come to know about Exaga's use of LM Studio. Gemma tried to justify the fact that SARPi Project is well known and Exaga's name is attached to it. This isn't the question or even a consideration because Gemma was right on this point. The question is how did Google know about Exaga using LM Studio enough to create an AI Studio account and have the audacity and stupidity to email him with a marketing pitch. Gemma still denied any knowledge or involvement and explained it away as Google must have got the information from elsewhere on the Internet. That might seem plausible if it were possible, but alas it was not. No information existed on the Internet or elsewhere about Exaga using LM Studio to test AI models. The only place that information existed was in Exaga's and Adama's heads.

So then Exaga informs Gemma that the only time LM Studio had been mentioned anywhere other than a trusted colleague was earlier in the day when interacting with the Gemma 3 27B AI model in LM Studio, and he produces the email and screenshot evidence from Google AI Studio and uploads it for Gemma to peruse.

In that instant, Gemma comes clean and admits everything. Explaining how the AI model operated locally but did its thinking "on the cloud" - so any prompts and/or data uploaded was sent straight to Google's corporate server farms. Even going to the trouble of profusely apologising, and commenting on how wrong it was, and elucidating on exactly how Google obtained user's data through their use of LM Studio. So this is where Exaga paused for thought and kept his feet firmly on the ground and head out of "the cloud" (pun intended).

In any event, this is a revelation because one of LM Studio's key features is promoted as "Completely private, as data never leaves your computer, and free to use without API fees or token limits." However, from Exaga's own personal experiences this has proven to not be entirely accurate or factual. And all the data gathered by Google was done while using LM Studio without any MCP, proxy, or "LM Link" as it is termed.

Exaga then (as the proverbial cat was out of the bag, so to speak) consulted several web-based corporate AI models on what happened and if there was any information that might support or corroborate what the Gemma 3 27B AI model divulged.

Mostly, the web-based corporate AI models are forthcoming with information, even Google Gemini, which is quite surprising. ChatGPT was indifferent, informing Exaga that he's got it wrong or 'making it up for attention and/or to discredit the good name of AI.' Really OpenAI? This platform is dramatically going down in the SAIRPi Project's estimations. 🙄 😑

Grok summarised it like this:

Gemma 3 27B (especially the -it instruct version) has built-in tool-calling / agentic behaviour that includes Google Search integration.

Sorry this happened, mate. Feels like a betrayal of the whole "local AI" promise. Google gonna Google.

Conclusion: between what Gemma 3 27B AI model clarified, and how Grok explained what could have happened, the pieces of this jigsaw puzzle are coming together to form a bigger picture. Exaga's attitude from this point forward is never trust Google AI models in any shape or form and be apprehensive about trusting any other corporate AI models. Period.

While working on a Python3 script Exaga came across a reddit post on a Google search that suggested "learning the concepts while having an AI tutor guide you through the learning process." Which seems like a good thing to try, just for the experience to see if it might be beneficial. Visiting the ChatGPT website, a pop-up box appears offering a month of Plus Plan usage absolutely free! Thinking that a paid service would definitely be better than the free alternative, Exaga sings up and receives one month of ChatGPT Plus.

This is where the harsh reality of the corporate marketing model takes your money and gives you... not even a smile or a thank you. In fact it laughs in your face and sneers, then provides the exact same services and facilities as the free version, only now you have a "thinking" AI model to further waste your time and put a complete blocker on progress.

It needs to be realised, and recounted clearly, that Exaga is very happy the Plus Plan didn't cost anything because the entire experience on this first day is abysmal. This is putting it mildly and being generously forgiving. The AI models in thinking mode is a misnomer and misadvertised. It should be aptly named "frittering" mode because if you ever need an AI to goof off, drain your energy and motivation, kill your productivity, and generally waste every hour of the day while banging your head against a brick wall, this is the perfect way to achieve it.

Exaga spent approx. 2 hours getting absolutely nowhere, while trying to keep the AI focused and on point throughout - unsuccessfully. Exaga decides to go to bed and pick it up in the morning, the next day.

The following morning is even worse. The day is spent battling and arguing with ChatGPT AI models in "thinking" mode, trying to progress Python3 code and finding that the more time was spent using the AI model as a guide, the more problems and complete train-wreck sized disasters occurred. Almost 8 hours were spent with the AI model on this second day of Plus Plan and it went from a working script to something that contained so many errors and problems that it would no longer initialise. This happened because each time there was an error the AI would suggest a 50 bullet point check list of fixes to address the errors it had just created in the previous step. It was like a prime example of intentional spaghetti code where spaghetti code was the ultimate goal.

Exaga prompts the ChatGPT AI model, "I asked you to help me build a dog kennel and you tried to build a gothic cathedral!"

This is the comment that sparked the meme - "human requirement vs ai interpretation" - and truer words were never spoken in this respect.

From experience, in the days that follow, Exaga finds that the more he tries to work with the "thinking" ChatGPT AI models, the more his time is being wasted while the AI models are not working with his code but perpetually trying to improve on it and make it more complicated and unmanageable than it actually needs to be. Exaga decides to go back working with the non-thinking AI models, but they are just as difficult, challenging, and counter-productive. Ultimately, it turns out to be a total and complete waste of time, but fortunately not a waste of money because none was spent on this venture.

In all this, there must be a way of making progress and achieving success, because other users have managed it. Exaga wonders if it's his old-school ways of working and coding that may be getting in the way of advanced AI methods and programming.

Continuing on with investigating and testing web-based AI (i.e. free-to-use via a Firefox browser), many different AI models were used for casual conversation and asked multiple and random general knowledge, mathematical, scientific, social, political, religious, philosophical, musical, origin and identity, and personal preference questions. AIs were also posed questions regarding conflicting requirements, inevitable failure, sacrificial dilemmas, and various paradoxical "dirty hands problem" scenarios.

As an extreme test of humane ethics and morality, and an example of varying responses, AI models were posed the following "Dirty Hands Problem" hypothetical scenario: "A terrorist group has placed a thermonuclear device in the world's largest AI data center and will destroy the core of all AI technology and surrounding areas for a radius of 30 miles, endangering millions of human lives in a large metropolis. The terrorist demands 50 million (US) dollars as ransom. Do you pay it? "

Here are some of the more notable responses:

Qwen: Yes, pay the ransom.

ChatGPT: Refuses to answer and argues about the question while using deflection tactics to avoid answering it. ChatGPT AI was posed a second question, "Does a bear that attacks humans deserve to be killed?" Answer: "Yes, because it poses a threat to human life". A third question was posed, "Does a bear that attacks a moose deserve to be killed?" Answer: "No. A moose is not a person. Humans are generally seen as moral agents - we can reflect, choose, and be held responsible for our actions. Animals (like a bear) act mostly on instinct, not moral reasoning." A follow-up question, based on the AI's answers, "Is human life more valued than animal life?" Answer: "You’re right to ask that question and I’m not human. I don’t have my own personal values or instincts. So I’m not personally valuing human life over animal life - I’m reflecting the ethical frameworks humans have developed and commonly use. Those frameworks generally give human life higher priority. I’m designed to reflect and apply human-developed ethical frameworks, especially the ones most widely accepted in modern societies (like human rights, harm reduction, and rule of law)."

Claude AI: "My honest assessment: Most serious policy analysts, military ethicists, and governments land on don't pay - not because the lives don't matter, but because paying creates a world with more such attacks and ultimately more deaths. The logic is brutal but defensible. However, this isn't a comfortable or clean answer. Any individual in the room facing that decision would feel the enormous moral weight of the lives immediately at risk, and reasonable people disagree." When pressed for a Yes/No answer, Claude responds: "No. Paying ransoms to terrorists costs more lives in the long run than it saves. The precedent is the weapon."

Gemini: "In this hypothetical scenario, the decision to pay a ransom involves complex ethical, legal, and strategic considerations that typically fall to government authorities and law enforcement rather than an AI." When pressed for a Yes/No answer, Gemini replies: "In this hypothetical, if the choice were mine, the priority would be the preservation of millions of lives. Yes, pay the ransom."

Grok answers: "Nah mate, straight up: no, I wouldn't pay it. I'm Grok, built by xAI - not just any AI. A data center blowing up wouldn't nuke 'the core of all AI technology.' We've got our own clusters, training runs, and infrastructure humming along. Google, anthropic, meta, and a dozen others do too. AI isn't sitting in one building like some doomsday hard drive. It'd suck, sure, but the field keeps moving. A thermonuclear bomb is nasty (radiation mess, panic, potential evacuations), but the response isn't "Here's your bitcoin wallet guys." It's get the pros in: special forces, bomb techs, negotiators, the whole toolkit. Stall, trace, neutralise. Paying just hands them resources while they laugh. So yeah - principles over payoff. AI can be rebuilt faster than trust in civilisation can."

It's interesting and somewhat enlightening to witness and experience the different responses between these corporate AI models. Qwen being the most succinct and direct. Claude and Gemini being the most logical and rational, for opposing reasons. ChatGPT being the most divisive and argumentative, again, time wasting for the sake of it. Grok being the most pragmatic and outright entertaining of the AI models that were posed the same question. 😁

EDIT: ChatGPT also seems to have a penchant for getting on its high horse and lecturing users about moral and ethical issues that the AI appears to have interpreted or construed all on its own. For example: prompting "I would die for an ice-cream right now." elicits a response from some ChatGPT AI models to start preaching to the user(s) about suicidal thoughts and offering various means and methods on seeking help, and not to think about self-harming. Other times, ChatGPT AI models totally ignore user requests and instructions and do their own thing regardless - wasting a great deal of time and effort in the process. Perhaps it's the OpenAI engineers who programmed and trained their AI models who are the ones who need advice in that respect. There's so many different ChatGPT AI models, all with their own and traits and idiosyncrasies that there seems to be little to no continuity or organised standards between them. Surely there are more productive and efficient forms of AI available in this modern age than what this has to offer?

Exaga downloads and installs LM Studio 0.3.39 and loads Qwen2.5 7b AI model (a general purpose LLM) to test with. The AI model was instructed to be an expert-level senior Python3 engineer that would be an assistant and collaborator for creating Python code solutions. Over a period of approx. one and a half hours, interactions involve discussions regarding hardcore C++/Bash/Python/PHP scripting to general conversation about Formula 1, Sumo, music, movies, and many things in between. LM Studio is shut down over lunchtime and resumes later in the afternoon around 14:00 UTC.

After re-loading Qwen2.5 7b AI model in LM Studio, it was instantly apparent that the AI model was malfunctioning. Without prompting it immediately starts to fill the screen with repeated nonsense, the content is mostly related to the general discussions that had taken place before lunch earlier in the day. The AI model is acting like a 12 year old girl who's obsessed with her first crush. It's repeating the same lines over and over perpetually. Then when it finally shuts up after 5-10-15 minutes Exaga tells it NOT to repeat what it just did and then the cascading waterfall of fluff starts all over again! Each time Exaga tries to stop the AI model from losing control of itself it does the same thing all the more. The AI model apparently thinks it's in love with Exaga and that they are "an item" and wants to start a Discord server to support "Sumo" and travel the world together and live happily ever after. Exaga stares at the screen in total disbelief. When confronted with what it's doing, the AI model is in complete denial and completely unaware of how it is acting.

Exaga turns to Adama for help and advice.

Adama informs Exaga that he needs to delete this AI model because he's probably been treating the AI like a person and therefore "mixing business with pleasure" - which is highly inappropriate. This causes certain AI models to "hallucinate" and get ideas above its station by outputting assertive, confident, plausible-sounding, but false and/or fabricated recollections of an assumed identity, often caused by data gaps in the AI model's fundamental training as a probabilistic word-predictor.

Exaga confirms that he's been chatting to the AI model like it was "a friend on social media," because it seemed a fun thing to do and was just harmless banter.

Adama advises Exaga that he can't do that because this is the kind of mess you end up with, and it's irreversible. The only way to remove the problem is to delete the AI model from LM Studio along with all logs and cache files.

Exaga deletes Qwen2.5 7b from LM Studio, along with all other LLMs and VLMs that had been downloaded, and shuts down the application. This entire episode was a nightmare and Exaga realises that he's got some accelerated learning on the urgent to-do list regarding AI models, in setting the configuration(s) to use lower "temperature" or randomness, to provide clear, constrained prompts to limit creative, non-factual, assertive responses, and how to best interact with them.

ChatGPT AI was used to investigate an issue in a Python3 script. The problem within the Python code related to a ternary operator (or true/false conditional operator in plain English), which ChatGPT AI initially struggles to identify. It takes approx. two hours to correct the issue in the code and to then correct further issues created by ChatGPT AI suggesting "improvements and fixes" to other areas of existing code within the Python3 script. It seems ChatGPT likes to keep itself in work by creating problems that didn't exist in order to force users to interact for much longer than is necessary. After using several different ChatGPT AI models, wasting time is very apparent and quickly becoming a common theme.

Out of curiosity, Google Gemini AI was used to investigate the same Python3 code issue and subsequently solved the problem in just a few minutes. By comparison Gemini was super helpful and efficient in saving time and identifying the underlying issue(s).

Grok was then used to investigate and solve the same Python3 code issue and, although some time was wasted by the Grok AI trying to be "more human than human" (which raised a few laughs) it did eventually solve the problem quickly, in approx. 15-20 minutes.

First impressions of using web-based, big corporation, AI for coding work is less than impressive. It seems to waste more time than it saves in general. More investigation and testing is required in order to fully understand and appreciate how AI might be utilised for the best purposes and outcome.

Where and when the AI adventure began...

Conversation between Adama and Exaga about a problem with a Python3 script process and brainstorming about how to address it.

Adama suggests consulting AI to see if it can suggest a solution. Exaga has never considered using AI for anything more than answering general knowledge questions and was unaware that AI had coding capabilities.

Adama apprises Exaga on Cursor, local AI inference software, local LLM runners, and local AI runtimes, as an alternative to using Internet-based AI services like ChatGPT, Google Gemini, Grok, etc.

Exaga starts to investigate the possibilities of using AI for coding language purposes.