Join us on Discord!
You can help CodeWalrus stay online by donating here.

Hardest platforms to program/work for ?

Started by gameblabla, November 22, 2018, 04:23:50 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

gameblabla

I thought i would share my experience with some of the platforms i've had to deal with and the most horrible ones i had to work with.

Atari Jaguar

Why use a single lame Motorola CPU that is not even 32-bits while use two 32-bits RISC processors as co-processors ?
And it is just the beginning.
For example, the memory controller cannot execute code out of ROM : you must load your game's code in RAM in its entirety.
Then there's the unusual way the Atari Jaguar draws graphics : It is not a framebuffer nor is it tiled-based.
In fact, even when looking at the Remover's Library, it's not clear how the Jaguar draws graphics (scanline-based ?) and it chuggles when you attempt to draw over 256 sprites on-screen.
(This does include things like the map or tiles, at least if you're not going the framebuffer way)

I later learned that you must pull off crazy tricks to even reach 1000 sprites. (Such as Super Burnout)
Meanwhile, the PS1 and Saturn have no issues doing just that. Even the 3DO is much less trickier when it comes to CELs.

I saw some people suggest that the unusual architecture makes it more powerful than a PS1, Saturn or even a N64.
In my experience, this is laughable, as it is very hard to even achieve smooth 2D graphics without the Jaguar outright crash or become sluggish.

So yeah, this is the reason why Evil Australians is still not finished.
While i did manage to make it run at decent speed (finally) by dropping unused objects, i am encountering weird bugs and crashes related to memory.
I have to remind you that the very same C code works fine on other Motorola platforms like the Falcon with no issues.
I do not know if i will be able to solve this. I might release an early build to prove my point.

Sega Saturn

Similar to the 32x, it has two SH-2 processors but they only have one cache shared by the two. (unlike modern processors)
If you thought that would make your task hard enough then you will have to deal with the VDP, because it is not trivial to use !
If you use Sega's official tools, it is surprisingly complex to display a single sprite on-screen because you have to go through so many steps
to convert a single bitmap.
Seriously, even the so-called "new improved tools" were hard to use at the time.

Another flaw of the Saturn is the two RAM chip configuration.
It is divided into two different ones : a slow bigger one and a fast small one.
This means that if you need to use more than 1MB, then you basically can't do it easily. Good luck with that.
It also forces you to manually manage memory, as if dealing with two SH-2 processors was not enough !

The flaws would have been soften somewhat if the Saturn was a cartridge system like the 32x.
While it could in theory do that, all games so far use the CD-Rom format. And that means dealing with memory is even more painful,
because you also need to worry about slow disk access time and you can't cheat like you would do on carts by loading and unloading graphics from RAM.
This is the perhaps the reason why many arcade games on the Saturn require the 4MB cart :
because it is much more straightforward to deal with.

All of this makes it a horrible system to program.
On the plus side, there are new libraries since then though that now soften the blow somewhat.Still, this was enough for me to run away from it, not to mention there's no way to boot unofficial software through the CD drive.

Android

Why is it even on this list ? Well, it's simple.
Supporting the more than 10000+ different phones out there is a nightmare.
In some cases, you are required to use hacks and weird permissions to even use basic features like sound support.
In some other instances, updates to software like SDL fix weird issues on new Android versions but also introduces others on phones with crappy GPU drivers.

How do you deal with this ? The short answer is you can't.
Making a whole operating system around Java and requiring all the other programming languages to use the JNI (worthless piece of c btw) is a mistake.
And don't tell me to use Java for Android especially after Oracle sued Google over it.
It was a terrible idea to begin with and the implementation is even worse.

Getting something trivial to work on all phones might be child's play but it becomes more painful as your project grows in scope.
Add to that ads pay nothing but scraps as well as the Google Play environment discouraging paid apps in general (made worse by the numerous phones out there to support)
and you have a recipe for disaster.

Honestly, I've given up on it and it's just not worth it given that there are also millions of other apps out there.
You will never stand out among the crowd and censorship is rampant on the Google Play store anyway.
(I had gotten complains about one of my games, Rubby Bird, briefly mentioning feminists and nazis, really !)

iOS is somewhat better on the programming aspect but they also have other issues, like the fact that Apple forbids numerous apps over non-sensical reasons and
asking for a huge annual fee. No thanks.

Good examples

GBA : This is actually a very straightforward platform to work with, i've been very surprised by it.
The bitmap modes are especially easy to set up and are very suitable for 3D engines.
Sprites are a bit finicky to use though.
But other than that, it is relatively easy to get some work done.

Dreamcast : Kallistios is really a pleasure to use on the Dreamcast, all things considered. It's no Windows CE, but it is much better performance wise while
still being POSIX compliant. However, mastering the GPU does require a learning curse but if you wanna go the software renderer route, then it is very easy
to setup a framebuffer.
Did you work on any difficult platform at some point or another, whenever it is modern or old ? Share your thoughts and opinions here
  • Calculators owned: None (used to own an Nspire and TI-89)

Yuki

I heard Google is working on a new OS to replace Android because its design is just getting plain outdated and is a pain to support the new features that came out since, say, Android 2. Looking forward to it, just to see how better it'll be or if it's gonna replace Android at all.

Also I hope you guys will not have to deal directly with the Win32 API in straight, unmanaged C/C++. Kinda forgot about that, been years I haven't touched it, but I remember it was kinda weird. At least, it made .NET look like a walk in the park.
  • Calculators owned: TI-83+ (dead?), Casio Prizm (also dead???)
  • Consoles, mobile devices and vintage computers owned: A lot
Read Zarmina!
YUKI-CHAAAANNNN
In the beginning there was walrii. In the end there will be walrii. All hail our supreme leader :walrii: --Snektron

if you wanna throw money at me and/or CodeWalrus monthly it's here

Jean-Baptiste Boric

#2
PlayStation 2

My take on this is somewhat atypical because I once tried to port back NetBSD to it, so I did not care about all the insane Sony proprietary bits. Turns out the CPU is one of the most insane things about it : from what I can recall, it's a MIPS-III with most MIPS-IV instructions, a couple of MIPS-V instructions, 128 bits registers, 64 bits virtual addresses like MIPS64, 32 bits physical addresses like MIPS32, FPU units almost but not quite IEEE 754-compliant and a 64 byte cache line.

WHY?! :banghead:

It's technically a MIPS CPU, but that's the weirdest MIPS ever made by far. It's like they've decided to make a hybrid of five different MIPS ISA because they could and piled on more insanity on top of it...

I haven't even talked about things a general-purpose UNIX-like OS does not care about, but the heterogeneous computing with the PS1 CPU bolted-on (or the PowerPC core on later models) on the bus would've yielded even more insanity to exploit.

HP Prime G1

It's not hard because of the hardware (it's fairly standard), but the firmware' OS is quite odd from what I've heard (early versions actually relied on the RAM mirroring to boot, which screams sloppy programming to me), the flashing protocol is proprietary and the utilities are Windows-only, the calculator will be bricked if the beginning of the NAND gets corrupted or overwritten and the touch chip is proprietary. That makes for a very, very frustrating platform to work on for homebrews.

Hopefully the G2 will not be such a pain in the ass to work with.

Good examples

The NumWorks calculator: extremely standard and boring hardware/flashing protocol/debugging interfaces, thoroughly documented, the source code of the firmware is available. The only real downsides are the limited amount of RAM/Flash on an unmodified model, the frame-buffer not being memory-mapped and the inability to act as a USB host, but man it is easy to work with.

Quote from: Juju on November 22, 2018, 05:00:51 AM
I heard Google is working on a new OS to replace Android because its design is just getting plain outdated and is a pain to support the new features that came out since, say, Android 2. Looking forward to it, just to see how better it'll be or if it's gonna replace Android at all.

That's Google Fuchsia. They'll probably use it in an embedded product like Google Home or Google WiFi first, but one of the main development platform is the Google Pixelbook and they do have a nice looking, hardware-accelerated UI on it based on Flutter. I'm getting familiar with the microkernel Zircon (previously Magenta), which is the best designed micro-kernel I've seen in a long time (I've done my share of hacking and pull requests on MINIX3). So much that I'm itching to do things with it, if only I could scrape together enough free time for such an endeavor...

By the way, some guys at the NSA are interested by it at the cyber-security level. They've done an informal audit of the system to evaluate how things are going even though it's still deep into development and even made a presentation on their findings (https://youtu.be/Jov4dTnjm2o?list=PLbzoR-pLrL6rOT6m50HdJFYUHyvA9lurI&t=1430).

gameblabla

#3
Quote from: Juju on November 22, 2018, 05:00:51 AM
Also I hope you guys will not have to deal directly with the Win32 API in straight, unmanaged C/C++. Kinda forgot about that, been years I haven't touched it, but I remember it was kinda weird. At least, it made .NET look like a walk in the park.
Win32 rofl. But don't give Microsoft more reasons to push UWP down our throats D:
QuotePlayStation 2

My take on this is somewhat atypical because I once tried to port back NetBSD to it, so I did not care about all the insane Sony proprietary bits. Turns out the CPU is one of the most insane things about it : from what I can recall, it's a MIPS-III with most MIPS-IV instructions, a couple of MIPS-V instructions, 128 bits registers, 64 bits virtual addresses like MIPS64, 32 bits physical addresses like MIPS32, FPU units almost but not quite IEEE 754-compliant and a 64 byte cache line.

WHY?! :banghead:

It's technically a MIPS CPU, but that's the weirdest MIPS ever made by far. It's like they've decided to make a hybrid of five different MIPS ISA because they could and piled on more insanity on top of it...

I haven't even talked about things a general-purpose UNIX-like OS does not care about, but the heterogeneous computing with the PS1 CPU bolted-on (or the PowerPC core on later models) on the bus would've yielded even more insanity to exploit.
Ah, interesting. I've heard a lot of talk about the GS processor on the PS2 but not a hell lot about the CPU. I guess most people assumed it was just a boring old MIPS cpu. I mean in theory it should be possible to tell GCC to target some of the CPU's features right ? (or even create a new target but i think you guys went with the safe route C:)
QuoteHP Prime G1

It's not hard because of the hardware (it's fairly standard), but the firmware' OS is quite odd from what I've heard (early versions actually relied on the RAM mirroring to boot, which screams sloppy programming to me), the flashing protocol is proprietary and the utilities are Windows-only, the calculator will be bricked if the beginning of the NAND gets corrupted or overwritten and the touch chip is proprietary. That makes for a very, very frustrating platform to work on for homebrews.

Hopefully the G2 will not be such a pain in the ass to work with
Isn't  the G2 going to be more locked down though ? It's a shame that neither the Linux or NetBSD ports went anywhere.But touchscreen isn't that important on a calculator one could argue.

I saw some articles about Fushia but it's as early stages as Haiku OS it seems. At least they got rid of Java it seems.I don't really care for the microkernel design but as long as it is POSIX compliant and performs decently, it's all good.But then again, it might end up like Tizen and stay on smartwatches and all. (which kind of died because the license was not truly open source among other things)

Also, i think i should add the Philips CD-i to my list as well... Seriously, it's terrible to work with and the whole design is flawed.

The good thing is that it does support lots of colors on screen compared to other platforms at the time (like the SNES or Megadrive).

The bad things is that you must go through the OS's calls to do your work.
I mean, when you play a sound effect, do you expect it to be delayed by 4 seconds the next time you play it ?
It's not even consistent, it's completely random.

Philips came up with "libraries" (like Balboa) to somehow help developers... except said libraries is unnecessarily even complex than just using the OS's calls !

Also, everything has to be software rendered. The advantage is that it does make it more flexible, kind of like DOS PCs at the time.The bad thing is the CD-i's main CPU is too slow to deal with the load, forcing devs to rely on compiled sprites.
But it does have a CLUT 128 colors mode in which you can have 2 planes, which is used by most games.

Overall though, it honestly looks like my ports to it are going to be challenging... I still have yet to find a fix for the sound effect issue.
  • Calculators owned: None (used to own an Nspire and TI-89)

Jean-Baptiste Boric

#4
Quote from: gameblabla on December 06, 2018, 06:11:56 PMAh, interesting. I've heard a lot of talk about the GS processor on the PS2 but not a hell lot about the CPU. I guess most people assumed it was just a boring old MIPS cpu. I mean in theory it should be possible to tell GCC to target some of the CPU's features right ? (or even create a new target but i think you guys went with the safe route C:)
The toolchain situation for the Emotion Engine is a tad complicated. Back in the day, Sony released the sources of a patched GCC version alongside their Linux port, but it did not end up being merged upstream for reasons I do not know. You can treat it as a MIPS-III from the userland's point of view and get away with it (which NetBSD users did for a while), but that won't work for the kernel. Someone did add basic support for the EE to GCC without the proprietary insane bits shortly before I started to work on the resurrection, which was enough for me to reach the point where the kernel panicked because it did not found a root disk to mount, with almost no hexadecimal-coded instructions in the assembly source files.

Anyway, all of this is moot now as any interest in this port resurrection promptly died when Imagination Technologies announced the CI20. Which really was for the best, because the PlayStation 2's architecture is a bad fit for general-purpose computing and the MIPS world really needed an official Raspberry Pi-like equivalent.

Quote from: gameblabla on December 06, 2018, 06:11:56 PM
QuoteHopefully the G2 will not be such a pain in the ass to work with
Isn't  the G2 going to be more locked down though ? It's a shame that neither the Linux or NetBSD ports went anywhere.But touchscreen isn't that important on a calculator one could argue.
It gained the ability to enforce a lockdown with a signed bootcode at the SoC level and the official HP firmware appears to enforce an effective chain of trust. However, from what scant tips I've gathered, the bootcode should be unlocked and we should be able to run homebrew firmwares (patching the original firmware will be far harder and we should definitively not risk invoking the wrath of HP). I intend to explore the G2 once mine gets delivered during this Christmas.

The touchscreen itself is not needed for a Linux port, but there's only one USB port available. Having the ability to have a mouse/touchscreen without requiring an USB mouse (and possibly an USB hub to plug in more things) has its upsides.

That reminds me, some time ago I've managed to come up with a theoretical way to effectively lock-down the NumWorks calculator with a simple software update, at the cost of fusing off the debugging interface. The "nice" thing about it is that I also found a theoretical way to allow unofficial firmwares to still run unhindered yet still prevent them from compromising the root of trust and from operating the LEDs. So if it ever comes to that, there will be a glimmer of hope besides desoldering the SoC and soldering an unlocked one.

Quote from: gameblabla on December 06, 2018, 06:11:56 PMI saw some articles about Fushia but it's as early stages as Haiku OS it seems. At least they got rid of Java it seems.I don't really care for the microkernel design but as long as it is POSIX compliant and performs decently, it's all good.But then again, it might end up like Tizen and stay on smartwatches and all. (which kind of died because the license was not truly open source among other things)
Fuchsia is not POSIX compliant (but does offer a limited level of POSIX compatibility) and frankly I think the POSIX API has accumulated a lot of rust over the decades when directly used as the native kernel ABI of an operating system. However, crostini's approach to bring Linux containers to Chrome OS (which I am impatiently waiting for it to leave the dev channel for my Acer R13) decouples the host kernel from the Linux kernel running containers inside a VM, so Chrome OS switching from Linux to Fuchsia in the future would be completely transparent to the end-user (as long as they migrate Android support to crostini too).

On the other hand, I am not sure what will be the consequences of switching Android from Linux to Fuchsia will be in practice, beyond that upgrades at the lowest layers of the OS stack should be much easier to perform. I am however tepidly sure that Fuchsia won't disappear overnight (if anything, it should make for a wicked embedded OS).

Lionel Debroux

Switching Android from Linux to Fuchsia is apparently far from trivial, according to Kees Cook this summer (2018/07/28) on ##linux-hardened. I'm not aware of a no-logs policy on that chan, so straight from my logs:
Quote[18:48:21 CEST] <xtrWrithe>   hello, im using this kernel, it comes with grsec? or i will need to patch it manually?
[20:04:37 CEST] <Lionel_Debroux>   linux-hardened does not come with PaX/grsecurity.
[20:05:24 CEST] <Lionel_Debroux>   There used to be minipli's unofficial forward port of grsecurity to newer Linux 4.9 stable versions, but that stopped when KPTI was integrated to mainline: mixing the broad KPTI changes with MEMORY_UDEREF and KERNEXEC was too much, too hard work, IIUC.
[20:22:57 CEST] <xtrWrithe>   Lionel_Debroux: ty for reply, yes i saw that the other port is dapper's one, but as you say KPTI made it very hard to keep working and the 4.13 versiom doesnt work yet
[20:23:38 CEST] <xtrWrithe>   what should i focus on to prevent kernel exploits or at least mitiagte them a bit?
[20:37:45 CEST] <Lionel_Debroux>   smeso reimplemented something like MPROTECT, and several other protections, in the SARA LSM... but LSMs still aren't stackable, and on the anti-exploitation front, nothing comes remotely close to grsecurity.
[20:37:59 CEST] <Lionel_Debroux>   Individuals are out of luck, they can't buy grsecurity subscriptions on their own.
[20:39:55 CEST] <Lionel_Debroux>   If you're part of a small company, and can convince those who decide (or are one of the deciders and it doesn't cost too much), maybe you can become a customer of Open Source Security, Inc.
[20:41:54 CEST] <Lionel_Debroux>   If you're part of a large company... you're probably out of luck as well, as the subscription probably won't be very cheap, and making those who have the power to decide on such bills - and are usually of the dumb financial type - understand the value of grsecurity, no matter how huge it is, is usually a hopeless task.
[20:51:01 CEST] <Lionel_Debroux>   In the longer term... IIUC, Android's going to switch to a completely different kernel implementation with a Linux compatibility layer. Microsoft used the same approach for Win10's WSL.
[21:49:16 CEST] <kees>   Lionel_Debroux: Android leaving Linux is somewhere between "never" and "in decades maybe"
[21:49:44 CEST] <kees>   xtrWrithe: I recommend the latest upstream kernel, and the recommended settings linked from the /topic
[21:49:51 CEST] <kees>   (if you can't find a way to use grsecurity)
[21:53:26 CEST] <Lionel_Debroux>   You know the topic better than I do, but... that far in the future, really ? Why ?
[21:53:54 CEST] <kees>   Lionel_Debroux: because changing a kernel is hard. :)
[21:54:03 CEST] <Lionel_Debroux>   AFAIK, many Android apps, especially those written in higher-level languages, do not have strong dependencies on Linux.
[21:54:32 CEST] <kees>   Lionel_Debroux: true, but the system integrity is tightly bound to Linux (e.g. SELinux)
[21:55:05 CEST] <kees>   Lionel_Debroux: now, what I'd expect is for that kind of thing to happen in VERY small devices where memory and CPU resources are the tight spot
[21:55:29 CEST] <kees>   but on phones... multi-core CPUs, gigs of RAM, I really think it'll stay Linux for a long long time
[21:57:56 CEST] <Lionel_Debroux>   Yeah, Linux on very small devices is a lost cause. The tinification efforts weren't deemed useful enough (by the powers that be) to offset the (sometimes small, IIUC) maintenance increase.
[21:58:32 CEST] <Lionel_Debroux>   Higher performance and better compatibility are deemed more important than smaller footprint... or better security.
[22:00:29 CEST] <Lionel_Debroux>   It's the first time I'm reading "the system integrity is tightly bound to Linux (e.g. SELinux)" argument, which is indeed a good technical reason why moving away from the Linux kernel for some usages can be hard.

And agreed, Fuchsia should be able to manage to be part of our world for a while.
Member of the TI-Chess Team.
Co-maintainer of GCC4TI (GCC4TI online documentation), TIEmu and TILP.
Co-admin of TI-Planet.

Jean-Baptiste Boric

Android applications do not require Linux to work.* They do however require an Android runtime.

* unless they do, but that's another story.

BlackBerry 10 devices (which are based on the QNX microkernel) can run Android applications and I do not think they are running a full-blown Android operating system with its Linux kernel inside a VM on their phones to do that. Another, less pertinent example here would be the original ARC for Chromebooks, even if it turned out to be a technological dead-end. Alternative Android implementations do appear to exist too (Dalvik Turbo) but I do not know anything about them and especially if they require Linux.

I'm not suggesting this is something easy to do (far from it), but it's been done before and Google has the resources to make it happen once Fuchsia is ready for it.

Yuki

Yeah, that's pretty much what's happening. Most Android apps will never make any calls to the Linux kernel, they'll make calls to the Dalvik VM instead, which will make calls to the Linux kernel to do its job. It's pretty much the same thing as the Java VM, and to a lesser extent .NET: unless it uses some system-specific features, the same binary should work on every OS/architecture combination the VM has been ported to. So if Dalvik has been ported to QNX or Fuchsia or even Windows (I think it's pretty much what's happened with BlueStacks), its apps should work fine.
  • Calculators owned: TI-83+ (dead?), Casio Prizm (also dead???)
  • Consoles, mobile devices and vintage computers owned: A lot
Read Zarmina!
YUKI-CHAAAANNNN
In the beginning there was walrii. In the end there will be walrii. All hail our supreme leader :walrii: --Snektron

if you wanna throw money at me and/or CodeWalrus monthly it's here

Powered by EzPortal