Attempting To Program Something Using LLMs With Zero Knowledge or Skill

[olePigeon]

I get Google Gemini for free at work. So I've been messing around with it. Currently I am attempting to make a tiny database program using THINK C version 6 on System 7.

I attempted to start with Codewarrior, but it was a complete failure. I then tried Symantec C++ and it was a bit better, but it was just constant syntax errors. So now I'm on THINK C v6 and it's going along somewhat smoothly.

I told it to limit its knowledge to a handful of period appropriate books. Even though I tell it to use C and not Pascal, it keeps trying to sneak Pascal in there and breaks the program. I think it gets confused on the code examples it's pulling from where they give both Pascal and C examples for the same thing.

Anyway, it's coming along surprisingly far given I have zero experience. I'm also learning a bit about how C works and the Macintosh Toolbox. I guess it's just sort of a nerdy sandbox. Just playing around.

 

[CC_333]

You may have landed on one of the relatively few genuinely helpful uses I can think of for LLMs.

I'd be lying if I didn't say I wasn't just a tiny bit curious, but I'm a Luddite who looks at this newfangled technology (and the companies behind it) very distrustfully, and I don't want to give them any of my money or information if I can help it.

If I didn't have so many qualms about it, I'd ask it to write me a stream player app for Mac OS 7 ,8 and 9 (plus a Carbonized version that can also run natively on Mac OS X 10.0.x-10.4.x) that's lighter and more purpose-built for streaming than iTunes and Sound Jam MP, easier to use than MpegDec, and can support at least a useful subset of modern security protocols and encryption ciphers so it can connect to encrypted streams. Kind of like a Spotify or Pandora app (I think someone actually has made such a thing for the former, so there's some precedent), but more platform agnostic where all you do is feed it a URL to a stream, and it simply plays it and nothing more.

Or better yet, how about a fully up to date Mozilla-like browser (kind of like Firefox, but for Classic Mac OS) running with at least basic HTML5 support, which has this facility built in?

c

 

[finkmac]

ok

 

[Durosity]

You may have landed on one of the relatively few genuinely helpful uses I can think of for LLMs.

I agree. I’ve always been awful with programming.. just my brain will not do it. But I have been using Claude to build plugins for my Home Automation system Indigo Domotics and it’s done an amazing job. I mean.. it’s a bit of a fight.. I feel like I’m in control of an autistic toddler who’s been given a multi pack of red bull.. but when properly controlled it’s doing great work.. it’s managed to integrate things I’d never thought possible!

 

[cheesestraws]

The problem is that, given a problem, writing the code to solve the problem isn't the hard bit. It's knowing whether it will break or not, and working out how, that's the hard bit.

 

[adespoton]

The problem is that, given a problem, writing the code to solve the problem isn't the hard bit. It's knowing whether it will break or not, and working out how, that's the hard bit.

Modifying code written by an LLM is also a challenge.

I've found one really good use for them: distilling large volumes of text into something actionable. Sometimes I get LLMs to script up something for me as well, but I make sure they do it in a manner where I can visually check all the components and easily identify the stuff it just makes up and fix it -- so it saves me time, but doesn't waste my time.

Actually, I lie: along with the distilling large volumes of text, LLMs are also good at doing a first code review and suggesting things I should fix in my code -- it'll make a bunch of stuff up that I can ignore, but it can also catch some obvious and not so obvious mistakes I've made, allowing me to fix them and learn from them.

 

[olePigeon]

I think it would be very useful for an accomplished programmer who knows what they're doing and has a large repository of code they've written. The LLM can can train off that repository and produce code that the programmer is accustomed to and would find it relatively easy to follow and debug.

Writing from scratch using whatever random stuff the LLM has stolen off the internet ... not so much.

 

[ArmorAlley]

I write SQL for a living (which is sort of code) and while I often find LLM-generated SQL impressive, you really need to know what the right answer is, because otherwise you can't judge how useful or correct the generated SQL is.
I have thought that LLMs might be great for someone who is already well-versed in the Assembly Language used to write software up to System 6, once it can be fed in appropriate texts, code and so on.

 

[adespoton]

I write SQL for a living (which is sort of code) and while I often find LLM-generated SQL impressive, you really need to know what the right answer is, because otherwise you can't judge how useful or correct the generated SQL is.
I have thought that LLMs might be great for someone who is already well-versed in the Assembly Language used to write software up to System 6, once it can be fed in appropriate texts, code and so on.

It can also same some time in disassembly analysis by identifying code blocks that are already well understood; however, I've found that there's a small enough body of work for pre-system 7 68k asm available that the current crop of LLMs have no clue what to do and keep on defaulting to OS X PPC/ObjC structures, and get confused about endianness and Pascal vs C structures. You'd need to do some very careful training to get a usable model there, I think. Post-2000 stuff? Yeah, it can save some time in basic scaffolding and in checking your work.

 

[olePigeon]

I write SQL for a living (which is sort of code) and while I often find LLM-generated SQL impressive, you really need to know what the right answer is, because otherwise you can't judge how useful or correct the generated SQL is.
I have thought that LLMs might be great for someone who is already well-versed in the Assembly Language used to write software up to System 6, once it can be fed in appropriate texts, code and so on.

I've been told by others that it does a great job commenting Assembly to make it easier to understand what's going on.

 

[olePigeon]

So I decided to start from scratch after a couple days of fiddling around. This time I set up parameters before I started and it's working a LOT better and producing compatible code way more often. It seems to be pulling its code almost exclusively from Julian Skidmore's "One Week Wonder" Blogpost which has a TON of posts with examples from programming in C that he'd done in the past. I really should write him an email and thank him. This program is 99% his code.

The parameters:

1. Use only C89/C90 standard for C programming.
2. Do not use Pascal.
3. Do not use Carbon.
4. Use only these following sources:
* The Macintosh C Programming Primer (Vol 1 & 2)
* Macintosh C Programming By Example
* Learn C on the Macintosh

 

[olePigeon]

OK. It's been nearly a month, and I've managed to accomplish what I suspect a competent programmer could have done in a weekend. Parts have been an immense struggle. The AI loves to hallucinate. I make a LOT of copies of code as I make short but stable progress. The problem I run into is that for some reason it will start to make random, arbitrary changes to other parts of the code completely unrelated to what I'm doing, completely breaking it. Even if I re-upload code from a stable build, it will insist that the random changes it made are exactly the same as the completely different code that I uploaded. I have to upload the same code file 3 or 4 tries to make the damn thing realize it's pulling completely random sh-t out of its ass instead of using the stable code I literally just uploaded as its source.

Absolutely infuriating.

In any event, I've finally managed to get to a major milestone in my app. For an extremely simple database program with less than a 1,000 lines of code, it has been a very slow grind. Holy cow. I don't think I'm learning much of anything. However, it's still really neat and fun to see an app I've always wanted to make come to fruition.

It continues to rely heavily on Mr. Skidmore's "One Week Wonder" blog. When it deviates, that's when it tends to mess everything up.

 

[David Cook]

Very interesting. I'm glad you're keeping us up to date.

I'm not surprised it has been difficult. I'm skeptical of LLMs for programming, for the same reasons other people have already mentioned in this thread. But, as a new way of searching and presenting findings, LLMs are wonderful. So, hearing about your progress is encouraging that there is still a way of making valuable use of LLMs in a programming environment.

I wonder if you incorporated additional Macintosh books and source code whether the results would improve.

 

[olePigeon]

So my app is now feature complete. This is the simple Reader, I do have an Editor, but since it's based off the Reader, I wanted to make sure the Reader app is mostly bug free before continuing off the same code base to make the Editor.

My app is an extremely simple database program with a very simple database file using custom delimiters. It is nearly identical to the Cracks & Numbers format, except that my SNDB format allows for a separate Cracks entry for each Version entry, and it also supports color text. I've also reimplemented a missing feature from Cracks & Numbers for support for patching applications, although not as cleaver. I figured since ResCompare already existed, just use that. Also, you can link serial number generators, hardware copy protection emulators, etc.

This was just something I've wanted to do since high school but never had the ability to do it, even if it doesn't really accomplish much more than what the existing C&N and S&K didn't already do.

At the very least, I had a lot of fun making the icons.

Feel free to run it through the ringer if you like. It was developed on an emulator, so there're probably some bugs I'd like to know about. I tested it on my IIfx and it seemed to run fine.

 

[olePigeon]

Very interesting. I'm glad you're keeping us up to date.

I'm not surprised it has been difficult. I'm skeptical of LLMs for programming, for the same reasons other people have already mentioned in this thread. But, as a new way of searching and presenting findings, LLMs are wonderful. So, hearing about your progress is encouraging that there is still a way of making valuable use of LLMs in a programming environment.

I wonder if you incorporated additional Macintosh books and source code whether the results would improve.

I see it as a potentially powerful tool for someone who is an experience programmer already. Maybe implemented as part of an IDE where it can make auto-complete suggestions using a code library of your own. That way the code will already be familiar (since it's based off code you wrote), and will have comments automagically generated in your own style so it's easy for someone else to understand what's going on. It could speed up coding pretty fast, but also provide suggestions for solutions you hadn't considered to a problem.

I think I would agree that it might improve the results, but also, it begins to hallucinate quickly. It doesn't matter how many times I tell it that it doesn't need #include <textutils.h> because it's included with the standard Mac library in THINK C, after a dozen or so prompts it will inevitably do it. Without fail. Only to respond, "You're right. You already told me this, but I included it anyway. I'll remove it."

 

[3lectr1cPPC]

I'm an amateur programmer, "learning as I go" and whatnot. I find LLMs helpful when I've got a bug I can't fix. I'll give it my code, ask it to tell me what's wrong, and I make sure I understand what the problem was rather than just blindly trusting anything. 9 times out of 10 it's one tiny syntax error that my eyes missed in the dozen times I scanned through the code myself.
That's about the limit of what I use it for though, troubleshooting my own code that I've writted. AI-generated code is problematic for a whole number of reasons which other posters here have laid out. But I think it can be a great learning tool to help inexperienced programmers, if they're using it correctly.