I compiled Mbed-TLS with Codewarrior 4 (attempting to make an HTTPS client)

[wanderingjew]

I have successfully compiled MbedTLS (or PolarSSL) for the classic Mac environment using Codewarrior Pro 4 on a machine running OS 8.6.

That's the clickbait. It doesn't _work_, oh no. I've been trying to connect to my server via HTTPS for a few weeks now, and I'm really, really tired of this project. I didn't think I would need to port the _entire_ MbedTLS library to make this thing work, but here I am.

The original idea for this was to build a 'client' of sorts for https://640by480.com/, a weird little 'instagram for digital cameras' thing I have. With the client you'd be able to login, upload photos, view pics, leave comments... basically instagram for the first digital camera. Except it'll run on a quadra or something. This of course means I need an SSL implementation to work with the API. So I started on this little project of trying to pull data into a text box from the API.
Mbedtls was written for C99 compilers, but my version of CodeWarrior only supports C89/C90. The transition required significant code modifications:

• Creating compatibility layers for modern C integer types
• Implementing 64-bit integer emulation
• Restructuring code to declare variables at block beginnings (C89 requirement)
• Addressing include path limitations in Mac’s un-*NIX-like file system

That last bit – addressing the path limitations – is a big one. You know how you can write #include "mbedtls/aes.h", and the compiler will pull in code from the aes.h file that’s in the mbedtls folder? You can’t do that on a Mac! There is a text-based file location sort of thing in the classic Mac OS, but I couldn’t find any way to use that in Codewarrior. The solution is basically to put all the files from Mbed-TLS into the project as a flat directory. Yes, Metroworks Codewarrior has an option for DOS/UNIX-like file paths when importing files, but I couldn’t figure out how to do that.

The biggest problem? C89 doesn’t support variadic macros or method overloading. 64-bit ints are completely unknown on this platform Holy hell this is annoying as shit. The mbedtls library uses 64-bit data types. int64_t, uint64_t, and the like. My compiler doesn’t know what those are. So I need to create them. Out of fucking thin air and structs, I guess. I made a shim library that does all of the 64-bit arithmetic, but that also means I need to port the code, and there's a lot of 64-bit math in the crypto libraries.
I’ve discovered a great plot hole in an Asimov short story. If you’re wondering how can the net amount of entropy of the universe be massively decreased, the answer isn’t to use a computer trillions of years in the future, the answer is to use a computer built thirty years ago.

The classic Mac OS has very little entropy, something required for high-quality randomness. This meant my SSL implementation gave the error code MBEDTLS_ERR_ENTROPY_SOURCE_FAILED. I created a custom entropy collection system that draws from multiple sources:

• System clock and tick counts at microsecond resolution
• Mouse movement tracking
• Memory states and allocation patterns
• Hardware timing variations
• Network packet timing with OTGetTimeStamp()
• TCP sequence numbers and connection statistics
• Time delays between user interactions
• The amount of time it takes for the screensaver to activate

All of these sources are combined and XORed together for a pool of randomness that’s sufficient for crypto operations. I wouldn’t exactly call this random, but it’s random enough to initialize the crypto subsystems in mbedtls. It works, but I make no guarantees about its security of this entropy function. This mbedtls implementation should be considered insecure. Not that it works, anyway.

CURRENT STATUS:
Right now the current state of this port is that portions of MbedTLS compile (not everything, it's just a minimal system), and I SHOULD be getting data from my server, but all I get is SSL handshake failures. Yes, I've tried to reconfigure my server to work with the ciphersuites I have. It didn't work.

I'm putting all of my work up in a Github for people to look at. I'm pretty burnt out on this project, and I would have loved to pull data over HTTPS for #Marchintosh, but... yeah this sucks. I'm not against picking it up again and making it work, but I think I need a few more eyes on this. I know @cy384 has done similar work, but with cross-compilation under Retro68, whereas I'm going for a full native compile.
So if you're interested, take a look LOOK AT THE GITHUB: https://github.com/bbenchoff/Milipixel-App THERE'S A .SIT FILE IN THERE SO GO NUTS

I'm a weird state of burnt out but still want to get this done so I encourage you to ask questions about this. Yes, it's TLS1.3, so wow, amazeballs.

 

[adespoton]

You mention it'll run on a Quadra. That means it's supposed to target 68040? I can't imagine TLS1.3 going back any further than that and being fast enough to be usable.

 

[cheesestraws]

64-bit ints are completely unknown on this platform

Implementing 64-bit integer emulation

You don't need to do this! 64-bit ints are not unknown at all. CWPro4 supports a chunk of what turned up in C99, just make sure strict ANSI compliance is turned off in project settings. It's often worth seeing whether something C99-y works, even though the compiler predates C99.



So you can use 'long long' and this will give you a 64-bit number. Try this. In CWPro4, in a SIOUX project, with strict ANSI compliance turned off:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

typedef long long int64_t;

int main(void)
{
    int64_t x;
    printf("size: %d\n", (int)sizeof(x));
    return 0;
}

gives me

size: 8

 

[Snial]

"Elliptic curves configured@Offering" is the bit that perked my ears up as I keep wondering if it's possible to ever provide a simplified BBS type interface, with encryption for 68KMLA that a 68K Mac can connect to. @wanderingjew, I understand you're trying to compile for something as old as a Quadra, but do you also intend to compile for PPC? I mean, Mac OS 8.6 is PPC only for a start.

@cheesestraws, so did stdint.h and stdbool.h exist then? if so, I guess adding:

typedef long long int64_t;
typedef unsigned long long uint64_t;

is all @wanderingjew needs to add.

 

[cheesestraws]

so did stdint.h and stdbool.h exist then? if so, I guess adding:

No, you will need to create your own stdint.h equivalent.

 

[wanderingjew]

I understand you're trying to compile for something as old as a Quadra, but do you also intend to compile for PPC? I mean, Mac OS 8.6 is PPC only for a start.

I'm compiling it as a FAT application on a beige G3. If it ever works, it'll probably run on an SE/30 -- anything less than that and I'm not confident there's the horsepower to do the encryption.

 

[joevt]

Using the "Activate C++ Compiler" option allows defining locals anywhere (including inside the for loop expression).

Use #include <cinttypes> to get the explicitly sized integer types such as int64_t.
This requires #include <string> for some reason. You may need to select the "Enable C++ Exceptions" option.
Also, you need to make the changes shown in the screenshot to make cinttypes compatible with the C++ compiler.
Use #define __STDC_FORMAT_MACROS to get the integer size format macros (such as PRIX64) for printf.
Use use namespace std; to use the integer types without having to include the name space qualifier std::

Put all the above in a prefix file so you don't need to change any of your existing source files.




Your screenshot shows boxes which might be newlines while classic Mac OS requires carriage returns. There is a "Map newlines to CR" option but I forget what it does. I suppose the documentation will tell you what the option is for. Without that option, newline in printf works as expected in the SIOUX console window. If you change the option, then you may need to change the MSL libraries to the NL variants.

 

[hauke]

I have successfully compiled MbedTLS (or PolarSSL) for the classic Mac environment using Codewarrior Pro 4 on a machine running OS 8.6.

Great job! Thanks for tackling this - I have been looking at embedded TLS off and on, but found the topic daunting.

I'm a weird state of burnt out but still want to get this done so I encourage you to ask questions about this. Yes, it's TLS1.3, so wow, amazeballs.

Since you asked:

Have you looked at WolfSSL? It claims to support 16/32 bit environments, and has been used by other teams.

Why did you decide to go with the 2.x branch of MbedTLS, instead of the current 3.x branch?

I noticed some of the files in the git repository have CR (Macintosh) file endings, and show up as single line. All VCS after CVS have been bad in dealing with different line ending conventions (or outright refusing to do so), so you need to do your own conversion before checking in.

 

[Snial]

Have you looked at WolfSSL? It claims to support 16/32 bit environments, and has been used by other teams.

Your "other teams" link is very interesting, since they're trying to get TLS 1.3 working on a DX2-66 running Windows 3.11. It's worth making a few comments on the blog post.

Firstly, they compiled for 16-bit code, but did they need to? WFW 3.11 can run 32-bit code can't it, even though the OS is mostly 16-bit? Perhaps they could have compiled it as a 32-bit DLL?

Secondly, the description for 16-bit far addressing is correct for 8086 mode, but incorrect for protected mode (see link [2] in the article). Protected mode can be described fairly easily though. Given a 16-bit segment value (in one of 4 segment registers) and a 16-bit offset (in one of 4 registers, or a direct value)

Address=DescriptorTable[segment].baseAddr + offset;

Where baseAddr is a 24-bit value (because 16-bit protected mode can only handle 16MB of physical memory) and the DescriptorTable is essentially read-only to user programs. I think that's useful to understand, because in protected mode you're forced to use an extra level of indirection. If you, for example, have a large, >64kB table of data, you can't compute an offset by shifting part of the address, instead you'd store a bunch of far-pointers:

gMyFarPointerTable[index>>kStructSizeBlockBits][index&(kStructSizeBlockMask)];

Which is astoundingly slow, because accessing your own far pointer table would usually trigger a descriptor table reload too!

Thirdly, because segmented, protected memory is chronically slow; it gives some plausibility that a significantly slower 68030 (perhaps 25MHz with FPU?) could also run the code adequately. E.g. instead of the above, it's just a straight array index.

 

[cluster_fsck]

I took a stab at this as well earlier this year, but focused on getting it building under Code Warrior 8 and running on PowerPC Macs running later versions of classic Mac OS (I'm running Mac OS 9.x). To avoid porting too much stuff, I used GUSI to handle some of the UNIX-y things that classic Mac OS lacks, but did start to re-write the net_sockets.c calls for Open Transport rather than just rely on GUSI. I got tests passing and the sample programs mostly working but it's definitely a slog and the resulting code is still pretty large, given all the stuff that's pulled in. There was complications with the latest Universal Headers and GUSI as well that I still need to sort out.

My goal in doing this was to see how far I could get with a simple Mastodon client for classic Mac OS that didn't rely on a proxy. On lower-spec PowerPC machines (one of mine is a 7100/80) the overhead with TLS processing is still very real. That then lead me down the pathway of turning all the networking-related code into a separate faceless background app (FBA) process, because those can use the Multiprocessing APIs on Mac OS 9 and for true pre-emptive scheduling and have functions to curry results back to cooperatively scheduled code (and, of course, most of the Mac OS Toolbox isn't safe to call from the MP APIs, but Open Transport and some File Manager calls are, so it's actually useful).

Anyway, apologies for the babble, but applaud your efforts and will check out your code when I get chance!

 

[wanderingjew]

UPDATE: THIS WORKS NOW. TLS 1.2 NATIVELY COMPILED

A bit more screwing around, and this works now. I'm connecting to my server and getting data from the API. Take this as proof of concept for a working port of Mbed-TLS. This is running on a beige G3 with OS 8.6, compiled under Codewarrior Pro 4.

The issues I was facing:

Cipher suite problem: The client was only sending TLS_EMPTY_RENEGOTIATION_INFO_SCSV without any actual cipher suites. Because it presented no cipher suites, it could not negotiate a handshake. The actual error here was a sort of C89/90 memory error, where my array of ciphersuites was not being stored as data to be sent. Do you know the difference between const and static? I do, now.

Elliptic Curve Configuration: This was similar; there was an error in the way the compiler read my array of curves. Const and static issue. Also, now I'm only supporting one curve, MBEDTLS_ECP_DP_SECP256R1, this is sufficient to complete a handshake.

Memory allocation failure: Once I had the cipher suite config, I was hit with another error, MBEDTLS_ERR_SSL_ALLOC_FAILED (-24192). This occurred when trying to add the elliptic curves to the ClientHello message. There are two ways to go about this; defining a gigantic buffer and calloc and freeing it into the SSLState struct, or by letting the config do that for you. By adding MBEDTLS_MEMORY_BUFFER_ALLOC_C to the config and adding mbedtls_memory_buffer_alloc_init() somewhere in the initialization, I got this to work. The Elliptic curves worked.

TLS Protocol Version: It works better with TLS 1.1, but this is really my first pass at this. Now that it works and I know what to look for, I can debug TLS 1.2

I'm still going to work on this until I can change the api.h to point to any API endpoint, and the JSON is printed to the text box. But this works now. I'll update this thread, and hopefully have something working up on Github later.

 

[adespoton]

This is awesome! Any chance you can compile the end result as a library other people can import into their own projects?

 

[wanderingjew]

IT WORKS This is JSON pulled off of my server at this endpoint: https://640by480.com/api/v1/posts



I'm going to clean this up and do a complete writeup, and eventually package this as a 'lightweight bare-bones project' for other people to hack on.

Few updates from the last time I updated -- I added a 'debug file' thing, where all the data is written to the textbox (32k limit), AND a text file in the same location as where the program is run from. This doesn't have a 32k limit, and it's easier to debug. The rest of the work was just getting the configs right and testing, testing, testing.

I'll spend a few days on this and have a proper github release.

 

[Snial]

IT WORKS This is JSON pulled off of my server at this endpoint: https://640by480.com/api/v1/posts

View attachment 85356

I'm going to clean this up and do a complete writeup, and eventually package this as a 'lightweight bare-bones project' for other people to hack on.

Few updates from the last time I updated -- I added a 'debug file' thing, where all the data is written to the textbox (32k limit), AND a text file in the same location as where the program is run from. This doesn't have a 32k limit, and it's easier to debug. The rest of the work was just getting the configs right and testing, testing, testing.

I'll spend a few days on this and have a proper github release.

Wow! Well done!

When you do, I wonder if it'll compile for my PB1400/166 under CodeWarrior 10 Gold Academic (PPC)?

 

[cheesestraws]

Good stuff. Well done!

 

[wanderingjew]

Here's the github link

Boy I wish we had an edit button here

 

[Trekintosh]

Here's the github link

Boy I wish we had an edit button here

We do. Bottom left of the post next to report

 

[joevt]

We do. Bottom left of the post next to report

But it has a time limit.

 

[Trekintosh]

But it has a time limit.

Oh, I comprehend now.

 

[joevt]

If we can't have an "edit", the next best thing would be an "append".