Nick's Path to AI enlightenment

[NickK]

So I thought I'd put together a thread around my move towards a better professional understanding of AI. This is a complex and very varied landscape but I hope some of this provide inspiration to anyone else in the same unfortunate position with difficult job market.

So for a first post I'll split this into two - business and the technology. I'll be approaching both.

1. Business
So I went from engineering to presales, then to product with new market and existing products, startups to being accountable for 70+ as part of a very large 3y/2.7Bn programme and the last role basically building out cloud for a multi-national conglomerate at a family level of products.. so my interest here is not "what is AI" but more the business, including the obvious financial approaches, operational models and all the normal fun IT stuff. How do you estimate a AI development using models for example? Cost Structures. Best practices etc.
I have a fairly good idea by cross referencing my experience, the technical business management/finops experience for the cloud work etc with observations but I wanted to get a structured approach, have industry feedback and also get a badge on the CV that says growth mindset.
Approach here is to enroll and pay for a "certified" 6 week course from a recognised university. I don't get a degree but I do get a confirmation of attendance and a mark at the end of it from the assignments. I will also have some live lectures and some group working.

Obviously I can't provide the course, but I will confirm my observations.

We've already seen an ecosystem of AI.. AI as a service etc. Simply put, before the course has stated I'm already aware of:
a) Large company vendors generate a general model (lots of CPU time) based on industry available data. For example the models that AWS provide through Marketplace for Bedrock.
b) Smaller vendors then often produce specialisations of the models or additional models that integrate using a more specialised set of data (that may or may not be specific for your company).
c) Your company then has to expend more time and cost (let's call this development) with specialist skills to train/characterise and confirm acceptance of the resulting model integrations (more development).
d) Competition and data privacy laws and regulations then become part of the morality frameworks, along with the purpose and exposure of individual's data or affected.

I start on the 20th, so this will be and interesting course.. I finish late January.


2. Technology itself
I have a BSc Software Engineering (parallel and distributed computation) so I have some maths, I have A-level maths in the distant past and some maths used for performing image processing - specifically FFT, phase correlation (dot product) and 2/3D maths around spread function distributions in terms of probability. I also spent some time around PHD/Professor maths/physics for quantum computing and photonic cryptography - in short I can appreciate things like error, distributions etc but have also played a bit with randomness and Markov chains but not in any structured fashion. I've basically built maths for implementations from scratch based on scientific papers before. Including finite response and I've also (at school) built a backwards propagation neural network from a Scientific American article back in the early 1980s/1990s.
So with that - basically most devs that have done some maths should be able to follow along.

Approach here is:
* learn the Python programming language, this is going to be painful as it's not a language I like. I preferred R to be honest..
* learn the maths required - specifically calculus, linear algebra and probability theory.
* Lean some ML dev stack - Jupyter notepad, panda, bumpy, and maths plotting. Basically libraries and tools at a very basic level.
* Start with ML courses:
** Stanford Online - ML specialisation
** Andrej Karipathy - Neural Networks
** Stanford Online - Deep Learning specialisation

I'll them probably play with some Kaggle challenges.

Maths - I have some maths school books that specifically deal with calculus, linear algebra and probability theory - that should be a good start.

The rest is simply a case of building myself a virtual machine, installing linux etc and then hacking away as I have done before as the founder of a quantum hackathon team. Thankfully I had some team members with really good maths skills that previously worked in quantum.

So if you feel that you want to embrace the AI change in the workplace, and put a step in that direction. This thread may be a good way.

 

[NickK]

So on Calculus it depends a little on your level of maths but:
Recap:

. this is basic limits, derivatives and integrations but it doesn't tackle continuous functions.

I also have the following books
* Cambridge University Press - AS & A-level maths: Pure Mathematics 2 & 3 Coursebook. This covers algebra, differentiation, integration etc
* Cambridge University Press - AS & A-level maths: Probability & Statistics 1 Coursebook. This covers data, variation, probability, perms & combs, distributions

The idea here is to have a basic knowledge and vocab for communicating and understanding data scientists etc.

One of the best channels and learning is 3blue1brown - I can't explain how good this channel is (it can go really really advanced but this is is simple):

Update - I've gone through the 3B1B Calculus and derivatives and just starting the integration video. Tomorrow should see the end of that. Then I'll start on the probability etc. There's a lot of videos and they're really good as recap/slower pace than the books but I will also continue with the books.

 

[NickK]

What's the end result you're expecting of this? From my understanding on the career thread you have held some pretty senior / strategic positions... you seem to going pretty deep here into technology not the outcomes it drives?

Correct - senior focus. It has a couple of modules on AI, deep learning but also covers ethics and AI in organisations.

Learning deeper, helps understand more. I don't need to the nth degree but I'd like to develop something from principles thus know how that maps.

 

[NickK]

Overview of AI for managers @ Oxford:

Oxford Artificial Intelligence Programme | Saïd Business School

Understand the mechanics behind artificial intelligence, its potential and its challenges

www.sbs.ox.ac.uk

That’s the course I’ve been accepted on.

Unfortunately I’ve not had training budget at my disposal since 2019.. so this will be great.

 

[NickK]

An interesting podcast with the Chief Data Officer of BAE Systems - this has some great organisational observations (not surprisingly) but also highlights the need to focus on the problem and the where people cost has given way to energy costs (and reliance on power and networking that could/will be under threat as we move towards a war with russia).

Driven by Data: The Podcast

Driven by Data: the Podcast is designed for data enthusiasts to hear from the data & analytics industry’s thought leaders! Each week you will

orbitiongroup.com

 

[NickK]

Well .. you can't escape doing ML optical character recognition as the ML "hello world"..

 

[NickK]

So I've been getting back into the course over the last two days (although I have the lurgy). Rather than battle on this afternoon I decided on a simpler task to understand some of benefits and disadvantages of coding with ChatGPT models.

The task - create a Xcode macOS application to display a configurable guitar fretboard.

ChatGPT4 seemed to understand the prompt interactions better but still managed to give code that failed to check index limits resulting in a segmentation violation unless explicitly told not to.

The ChatGPT earlier model came up with a completely different code project, the code is completely different and the project itself is define differently - using the same text for prompts.

But it's been interesting - will continue tomorrow when the free credits reset.

 

[NickK]

Can ChatGPT code?

TL;DR - it would not replace a graduate joiner at the moment.

I chose a fiendishly simple application - tune a guitar through the audio coming from a USB interface on the Apple Mac. This has some intesting gotchas that are obvious but not obvious:
* application operating system interactions need authorisation - in this case the audio needs authorising by the user
* audio interactions using USB require higher QoS thread scheduling to prevent hangs etc, especially with 24bit 192kHz sample rate
* audio processing by default should use fast libraries to perform analysis (my target here in my head is the vDSP library to provide the maths for FFT)
* detecting the fundamental frequency of the guitar string in the noise of of the audio spectrum that the pick up hears - this includes both standing string wave, harmonics and the travelling wave+harmonics of the string pick that decays to leave the standing wave.
* multiple strings and frequency variations means there's not going to be a perfect frequency match so typically this would be "what accuracy would you consider being close to note X?".
* accuracy needs to be in the 0.01Hz region to detect 'cent' (ie detect each fret space divided into 1200 steps) to 0.1 cent.
* I indeed asked that it should use a notch filter to improve the detection, but also detect non-notched (ie inverse) for detecting fret buzz where strings are too close to the frets and rattle the strings.

So here's some experiences just using ChatGPT to code:

1. It didn't validate configurable parameters - so adding 1 string resulted in a core dump crash. I had to specifically tell it that the application should not crash due to unchecked array index values.
2. You have to be very particular in what you are telling it.. and it often talks about doing stuff but doesn't. It decided not to bother coding the notch filters or calculate the quality based on noise based on non-notched frequency, instead it decided to give me the power of the frequency. Even though it talked about notching - that doesn't exist, nor does notched-based quality measurements.
3. I had to tell it which frameworks and tools to use, it stops and says it needs something there but needs interactions to define the path it takes (ie use vDSP for FFT).
4. It wrote unsafe pointer code.. again this is crash material.
5. It wrote code that has concurrency issues - it attempts to run the audio code on the main GUI thread.. so that's has serious consequences in hangs etc.
6. It needed me to sort out the nuances of audio authorisation - both declaring the app uses the audio input and to add code to seek the users permission.
7. The resulting detection of notes is .. well.. lacking. It assumes the code it writes is correct and works. It doesn't create tests to demonstrate the code is correct and complies to the requirements given.

I'll give it a couple of years to get good.. because at the moment it's not going to get the coding job. It requires lots of knowledge of the coding, of the requirements, and of development processes to be a viable 'coder', even to suggest coding examples without due-diligence.

Still - it was fun.

 

[NickK]

What will be interesting is if the code languages themselves end up being generated by AI - specially to remove some of the 'human' language syntax restrictions.

I've heard that about CoPilot and GitHub is that ChatGPT is better and that's saying something!

 

[NickK]

Oh.. turns out it has a memory leak too 445GB of ram later

 

[NickK]

CoPilot is "ChatGPT", with the default using 4o but you can pick o1 and o1-mini as additional options, however, they are opening it up to more models, Claude 3.5 being one.

My experience so far is ChatGPT directly, I haven't used Claude (requires mobile number even with google login!) nor CoPilot.

When I have more time I'll look further afield. I did look at the google AI ecosystem architecture too but for now it's keeping it simple.

I have noted that people use "ChatGPT" even when the model is not.

 

[NickK]

So I'm now 3/6 modules completed with decent marks so far. It's also occurring to me I've learnt more as part of this course than I would if I'd simply hacked away using the more detailed Google AI training (and I've done the parallel neural network course) as you're forced to think through examples and describe the thinking/approaches etc.

It's been interesting hearing about AI changes that are rattling around in the UK legal sector. How healthcare in the US classified AI as a medical device for example and that gets used as a US regulatory reference at the moment.

A discussion on safety was an interesting one. Plenty to think about but most people think AI vs humans, rather than competitive AIs competing against each other AND humans. It's also starting to dawn on people that if you replace workers then the government has to charge more in corporation tax to support the missing taxation revenue.

Can AI replace humans - undoubtedly so. Will AI replace humans? Money and resources but mainly planetary resources will be an issue.

Now if AI deems humans as legacy, heads for the stars and leaves it's mess behind for humans to cleanup, that would not surprise me.

 

[NickK]

I’m now deep into Ethics of AI as part of the 5th module, currently upto my eyeballs going through privacy law (wider than GDPR).

The modules 1-3 have been quite technically deep but also wide. Talking to some fellow students at a meetup evening, it’s clear that everyone is saying “7-10 hours/week” is more like 21-30 hours in reality.
The modules 4 & 5 have moved away from technical (but you still need that nuanced understanding) toward organisational use, ethics and finally business cases etc.

i think M4 may be a weak essay from me - I had cold and didn’t want to take an extension or I’d be behind. Will find out next week.

I oddly feel I seem to be approaching subjects in the essays thay the next module is about.

 

[NickK]

An LLM is stochastic so it will generate different code each time, hence it makes it extremely difficult to maintain if you want to adjust the code over time.

 

[NickK]

An LLM in a generative capacity uses random noise.

An LLM without learning that is static is, as you’ve stated, results in a distribution. Ie as it doesn’t learn the paths don’t change.

Sorry I was thinking of agentic using things like chatgpt, a genai.

Surely the selling point is gen ai is dynamic.. and the reason for the painful fun.
Switching off (temp zero) means you may as well make code with an AI, then use that.

 

[NickK]

I’ll recheck my notes when I have chance.

I know about temp, p & k hyper parameters and for non-llm the explore/exploit ratio.

Noise being used in training input to desensitise (has a similar effect to downsizing).
Noise also being used in gen ai where the text input is also supplemented with random input to generate output.

I’m also aware of the llm process (use of vector maps etc).

The “agentic” process of feedback isn’t static compared to old agents. I can see the llm can be setup either way with 0 temp. True but most hype is people are pushing for both operating reinforcement but also starting to look at non zero temp (seems a bad idea to me)

I’ll have a more indepth look again next week as I want to write my own agents.

 

[NickK]

maybe this is interesting

How Nvidia uses GPT-4 to make AI better at Minecraft

The Voyager program can both devise new challenges in the game, and then continuously refine strategies for success. Sometimes it hallucinates non-existent items, though, such as an "acacia axe."

www.zdnet.com

It seems they had a chatgpt doing the code, then another chatgpt evaluating the original chat gpt.


which I guess is the only way that makes sense when chatgpt gives so much garbage code

So basically an adversarial network by the sounds of it. Not sure I'd want both the same vendor of model.

 

[NickK]

So the ethics for AI is done and submitted. I can see the issues etc but for some reason I found it difficult to get it down on paper.

 

[NickK]

Claude 3.5 is flawless so far, Technically its github copilot using the Claude 3.5 sonnet preview

Interesting that Claude 3.5 data privacy retains your data/code. Whilst Claude Pro does not unless explicitly told todo so. I did see a Samsung case where proprietary code was put into ChatGPT for debugging only to essentially make the code public domain.

Like all vendors, it will be interesting to see the changes that Trump may make w.r.t. AI and data access. I already see that high end AIs require mobile numbers for verification and we're starting to see blocks on VPN access to AI prompts.

 

[NickK]

So I’ve decided against my business case for air-sea salvage using drones.

I have another idea that is better suited:
* i have had direct experience
* impacts a wide market
* has a gov research paper request out right at this moment indicating my idea is a good one and relevant

I know what & so I have a plan for tomorrow to build the business case, leaving time to think on a third draft, and answering a second simple question on tuesday.