Join us on Discord!
You can help CodeWalrus stay online by donating here.

Programming language support

Started by DarkestEx, November 04, 2015, 02:40:00 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

What languages should the microcat be programmed in?

C only
C and Basic (some easy dialekt - ZX spectrum like Basic maybe)
C and the best of Lua and Basic (fewer features but runs faster, allows for large and more optimized games)

DarkestEx

The microcat is going to be very modular and open. The BIOS will support loading different firmwares which can either be runtime environments for games written in (compiled-) scripting or VM based languages or C based games (similar to the Gamebuino). Changing the firmware will be easy and will be done using a graphical menu or the firmware that is currently running (if supported by it). The flash has an endurance of only 10^4 erase-write cycles and therefore writing C games will be supported but it's not recommended to change between them too often (yes, the whole flash gets overwritten every time you change the firmware). This is a reason to primarily focus on scripting languages / VM based games. Sure, the ARM processor does indeed support executing from RAM, but this is rather useless as we only have 128KB of it which is shared between the variables and the game code then.

Now, initially we wanted to port Lua over, but this might not be the best idea as Lua games would need to be loaded into RAM and take up most of it in the end.
Then we were thinking if Basic games would maybe fix this issue and they probably do, but Basic is less powerful than Lua.
So we were thinking about making a new language that is very similar in syntax to Lua but has no tables and only four different data types being nul, int16, int32 and an array of int16.
It would also have static functions and local and global variables. This langauge would in any case take up way less space than Lua would and it would be more powerful than Basic.
The only problem is, that we would have to write it first, but we were already planning time in for such a task if necessary.

So what do you think?
  • Calculators owned: TI-84+, Casio 101-S, RPN-Calc, Hewlett-Packard 100LX, Hewlett-Packard 95LX
  • Consoles, mobile devices and vintage computers owned: Original Commodore 64C, C64 DTV, Nintendo GameBoy Color, Nintendo GameCube, Xbox 360, PlayStation 2

Dream of Omnimaga

A compiled language would definitively be a must, as it won't take much space in the firmware. I would say that C or C++ would be better since it would make porting games easier.

But for the higher level interpreted language, it depends. If it's an established language, the main concern is how much space does the interpreter takes. Would it be better to implement a stripped down version of BASIC and xLIBC aimed directly at gaming? It would be smaller than a straight port of Lua and TI-BASIC. Speed-wise, we can deal with it. People wanting max speed can just use C or perhaps use third-party C routines.

So yeah, in my opinion, it should use either a stripped down version of TI-BASIC/Lua, modified to only have game-oriented commands to keep the interpreter file size to a minimum, or you should use an old BASIC interpreter that is small like on ZX Spectrum and stuff. But of course, making a brand new language would be time consuming.


I would say focus on getting the firmware done so it can launch C games, then worry about the interpreted language afterwards.

Or you could use Cumred's language but he hasn'T started working on it much IIRC so that might not be ideal for now.
  • Calculators owned: TI-82 Advanced Edition Python TI-84+ TI-84+CSE TI-84+CE TI-84+CEP TI-86 TI-89T cfx-9940GT fx-7400G+ fx 1.0+ fx-9750G+ fx-9860G fx-CG10 HP 49g+ HP 39g+ HP 39gs (bricked) HP 39gII HP Prime G1 HP Prime G2 Sharp EL-9600C
  • Consoles, mobile devices and vintage computers owned: Huawei P30 Lite, Moto G 5G, Nintendo 64 (broken), Playstation, Wii U

DarkestEx

#2
Quote from: DJ Omnimaga on November 04, 2015, 11:26:24 PM
A compiled language would definitively be a must, as it won't take much space in the firmware. I would say that C or C++ would be better since it would make porting games easier.

But for the higher level interpreted language, it depends. If it's an established language, the main concern is how much space does the interpreter takes. Would it be better to implement a stripped down version of BASIC and xLIBC aimed directly at gaming? It would be smaller than a straight port of Lua and TI-BASIC. Speed-wise, we can deal with it. People wanting max speed can just use C or perhaps use third-party C routines.

So yeah, in my opinion, it should use either a stripped down version of TI-BASIC/Lua, modified to only have game-oriented commands to keep the interpreter file size to a minimum, or you should use an old BASIC interpreter that is small like on ZX Spectrum and stuff. But of course, making a brand new language would be time consuming.


I would say focus on getting the firmware done so it can launch C games, then worry about the interpreted language afterwards.

Or you could use Cumred's language but he hasn'T started working on it much IIRC so that might not be ideal for now.
Well the BIOS is what will allow to launch C titles (C++ will work as well though it will not be officially supported). I will work on it some more when we have a PCB prototype as right now I cannot implement it any further because the hardware is in this aspect not compatible to what we will end up with (quite a few incompatibilities there).

My idea was to make an intermediate language between Lua and Basic which is indeed time consuming, but would be compiled to bytecode and therefore not be that slow.
For Lua, after testing the implementation that we would be using on a similar microcontroller, we decided not to use it as it will be too slow and takes up way too much RAM to be able to make proper games. In case that I would do my own language, I would look some more at Basic implementations and learn from them. This will certainly speed up development.
Also an own language has many features over a preexisting one that has to be fitted into the tiny space that we have.
Such as having a way smaller memory and Flash footprint, being faster and lightweighter than existing implementations and being specially optimized for the platform that it is running on.
The custom language would be able to reference shared libraries, be written and compiled to bytecode right on the device, using JavaScript inside your browser or as a standalone C application. It would also read directly off the SD card into a pipeline which would buffer the code so that it executes way faster.
Being similar to Basic it would have the known Lbl and Goto statements same as the known from Lua functions. It won't have tables or special strings (strings would be byte arrays).
Normal variables would be int16 and arrays would be int8 (the idea behind this is, that most of the time you won't have lists of long numbers but mostly sprites and simple values). Strings would be arrays. The general syntax would be borrowed from Lua though (if, then, while, end, for, etc.). Variables will always live inside their block and sub-block - no need for local. Declaring variables would work like this: var lol = 12; -- semicolon not required, as in lua

A simple program could look like this (I am open for suggestions, but I'd like to keep it syntactically close to Lua):

-- Simple number guessing program

var number = rand(1,100);  -- Create new variable called number and write a random number between 1 and 100 into it
var found = 0;   -- Has the value been found
var guess_no = 1;   -- Guess number
while guess_no <= 3 && found == 0 do   -- For loop from 1 to 3
    var guess = input_number();   -- Sample function that prompts the user to input a number
    if guess == number then
        found = 1;
        print("Found!");   -- Sample function to output a string
    elseif guess < number then
        print("Too low!");
        guess_no = guess_no + 1;   -- Increment number
    elseif guess > number then
        print("Too high!");
        guess_no = guess_no + 1;
    end   -- End of if
end   -- End of while


Would this language work for the people here? Please feel free to comment and/or vote in the poll.
  • Calculators owned: TI-84+, Casio 101-S, RPN-Calc, Hewlett-Packard 100LX, Hewlett-Packard 95LX
  • Consoles, mobile devices and vintage computers owned: Original Commodore 64C, C64 DTV, Nintendo GameBoy Color, Nintendo GameCube, Xbox 360, PlayStation 2

adekto

im personaly not of fan of reflashing the chip constantly apart from its slow and can break the device in some cases.

i dont know to much about the how the running off ram works, but in theory we got enugh ram to run bytecode (c or other compiled code) on there, i mean the gamebuino has to do this in only 32kb of flash +2kb ram

DarkestEx

Quote from: adekto on November 05, 2015, 12:41:15 PM
im personaly not of fan of reflashing the chip constantly apart from its slow and can break the device in some cases.

i dont know to much about the how the running off ram works, but in theory we got enugh ram to run bytecode (c or other compiled code) on there, i mean the gamebuino has to do this in only 32kb of flash +2kb ram
Yes the chip can corrupt during reflashing and it only has 10000 erase/write cycles before the flash reached the end of its durability. And yes, it will always take some time to verify it and write it.

Well RAM is always a problem, we can indeed run C code from RAM and that is what we are also heading to.
But I want to have a scripting language on board as well to have it a good platform for beginners.

The Gamebuino uses reflashing to load new games. The reason why 2KB RAM are enough is, that the display is smaller in resolution and only monochrome.
  • Calculators owned: TI-84+, Casio 101-S, RPN-Calc, Hewlett-Packard 100LX, Hewlett-Packard 95LX
  • Consoles, mobile devices and vintage computers owned: Original Commodore 64C, C64 DTV, Nintendo GameBoy Color, Nintendo GameCube, Xbox 360, PlayStation 2

Snektron

I guess you load the data from the SD via serial? Because that can make if work okay for an interpreted language. You'd only need to store the variables in ram, because the program code won't change anyway. The entire interpreter would have to be in ram though, i guess.
  • Calculators owned: TI-84+
Legends say if you spam more than DJ Omnimaga, you will become a walrus...


DarkestEx

Quote from: Cumred_Snektron on November 05, 2015, 12:57:26 PM
I guess you load the data from the SD via serial? Because that can make if work okay for an interpreted language. You'd only need to store the variables in ram, because the program code won't change anyway. The entire interpreter would have to be in ram though, i guess.

Right now I am talking about the custom language.
The SD card is connected using an HSMCI interface and not through hacky SPI, so its about 4 times faster than it would be with serial. I estimated the maximum speed possible to about 29MB/s.
The interpreter/VM is going to read bytecode that is produced by a compiler which takes the source code and turns it into bytecode.
The interpreter will read read directly off the SD card through a pipeline.
And the interpreter would be in flash of course.

For the C based one, it has to be fitted into RAM. Let's say a big game has 50KB of program size, then it will have about 40KB of RAM left which it can use for variables.
  • Calculators owned: TI-84+, Casio 101-S, RPN-Calc, Hewlett-Packard 100LX, Hewlett-Packard 95LX
  • Consoles, mobile devices and vintage computers owned: Original Commodore 64C, C64 DTV, Nintendo GameBoy Color, Nintendo GameCube, Xbox 360, PlayStation 2

novenary

#7
Quote from: adekto on November 05, 2015, 12:41:15 PM
im personaly not of fan of reflashing the chip constantly apart from its slow and can break the device in some cases.

i dont know to much about the how the running off ram works, but in theory we got enugh ram to run bytecode (c or other compiled code) on there, i mean the gamebuino has to do this in only 32kb of flash +2kb ram

Keep in mind that the Gamebuino uses an 8 bit CPU, you are using a 32 bit CPU so the amount of RAM you have is actually comparable (not completely equivalent since the architecture is obviously different, but comparable). The other issue on the gamebuino is indeed that it works by rewriting the internal flash as it uses an AVR chip which uses a pure Harvard architecture, it can only execute from flash, not RAM.

As I discussed with DarkestEx last night on Telegram, I think native code using a library stored in flash is the optimal solution for your use case.
You can probably fit a VM as well for your own custom language if you want it to be programmable on the system itself.
Another good idea would be to have a language programmable from a computer that compiles to C, this way you can leave all the hard part of actually compiling and optimizing the code to GCC, keeping the benefits of native code and also bringing a potentially friendlier language to the platform for beginners. This last solution is the best of both worlds in my opinion and it allows anyone to program in C or asm on the official firmware without any hassle since the derived language would use the same libraries.

Dream of Omnimaga

Quote from: DarkestEx on November 05, 2015, 01:04:35 PM
Quote from: Cumred_Snektron on November 05, 2015, 12:57:26 PM
I guess you load the data from the SD via serial? Because that can make if work okay for an interpreted language. You'd only need to store the variables in ram, because the program code won't change anyway. The entire interpreter would have to be in ram though, i guess.

Right now I am talking about the custom language.
The SD card is connected using an HSMCI interface and not through hacky SPI, so its about 4 times faster than it would be with serial. I estimated the maximum speed possible to about 29MB/s.
The interpreter/VM is going to read bytecode that is produced by a compiler which takes the source code and turns it into bytecode.
The interpreter will read read directly off the SD card through a pipeline.
And the interpreter would be in flash of course.

For the C based one, it has to be fitted into RAM. Let's say a big game has 50KB of program size, then it will have about 40KB of RAM left which it can use for variables.
But does the entire game including all songs have to be loaded into RAM before run time or can you unload some parts of the code/data during gameplay then load different stuff? What I mean is like Reuben Quest and Illusiat 13, where  map data remains in the flash, and when you need a particular map, you only load the set of maps it's in into RAM, load the map you want then remove the storage program from RAM.
  • Calculators owned: TI-82 Advanced Edition Python TI-84+ TI-84+CSE TI-84+CE TI-84+CEP TI-86 TI-89T cfx-9940GT fx-7400G+ fx 1.0+ fx-9750G+ fx-9860G fx-CG10 HP 49g+ HP 39g+ HP 39gs (bricked) HP 39gII HP Prime G1 HP Prime G2 Sharp EL-9600C
  • Consoles, mobile devices and vintage computers owned: Huawei P30 Lite, Moto G 5G, Nintendo 64 (broken), Playstation, Wii U

DarkestEx

Quote from: DJ Omnimaga on November 05, 2015, 08:09:07 PM
Quote from: DarkestEx on November 05, 2015, 01:04:35 PM
Quote from: Cumred_Snektron on November 05, 2015, 12:57:26 PM
I guess you load the data from the SD via serial? Because that can make if work okay for an interpreted language. You'd only need to store the variables in ram, because the program code won't change anyway. The entire interpreter would have to be in ram though, i guess.

Right now I am talking about the custom language.
The SD card is connected using an HSMCI interface and not through hacky SPI, so its about 4 times faster than it would be with serial. I estimated the maximum speed possible to about 29MB/s.
The interpreter/VM is going to read bytecode that is produced by a compiler which takes the source code and turns it into bytecode.
The interpreter will read read directly off the SD card through a pipeline.
And the interpreter would be in flash of course.

For the C based one, it has to be fitted into RAM. Let's say a big game has 50KB of program size, then it will have about 40KB of RAM left which it can use for variables.
But does the entire game including all songs have to be loaded into RAM before run time or can you unload some parts of the code/data during gameplay then load different stuff? What I mean is like Reuben Quest and Illusiat 13, where  map data remains in the flash, and when you need a particular map, you only load the set of maps it's in into RAM, load the map you want then remove the storage program from RAM.
Well you cannot unload code and music is fully streamed and does so almost take up no RAM at all. Sprites and maps have to be loaded into RAM and they can be loaded and unloaded at any point.

In the custom language the code itself doesn't take up any RAM at all, neitherless how big it is. Only sprites and maps that get loaded into RAM take up space there.
  • Calculators owned: TI-84+, Casio 101-S, RPN-Calc, Hewlett-Packard 100LX, Hewlett-Packard 95LX
  • Consoles, mobile devices and vintage computers owned: Original Commodore 64C, C64 DTV, Nintendo GameBoy Color, Nintendo GameCube, Xbox 360, PlayStation 2

Dream of Omnimaga

I see. That doesn't seem too bad then. I was worried that every game had to be 50KB total or less.
  • Calculators owned: TI-82 Advanced Edition Python TI-84+ TI-84+CSE TI-84+CE TI-84+CEP TI-86 TI-89T cfx-9940GT fx-7400G+ fx 1.0+ fx-9750G+ fx-9860G fx-CG10 HP 49g+ HP 39g+ HP 39gs (bricked) HP 39gII HP Prime G1 HP Prime G2 Sharp EL-9600C
  • Consoles, mobile devices and vintage computers owned: Huawei P30 Lite, Moto G 5G, Nintendo 64 (broken), Playstation, Wii U

DarkestEx

Quote from: DJ Omnimaga on November 05, 2015, 08:33:59 PM
I see. That doesn't seem too bad then. I was worried that every game had to be 50KB total or less.
No don't worry, programs can split the memory however they need it.
  • Calculators owned: TI-84+, Casio 101-S, RPN-Calc, Hewlett-Packard 100LX, Hewlett-Packard 95LX
  • Consoles, mobile devices and vintage computers owned: Original Commodore 64C, C64 DTV, Nintendo GameBoy Color, Nintendo GameCube, Xbox 360, PlayStation 2

DarkestEx

I have been looking into Lua some more and it just doesn't seem fast enough and has a way to high memory usage :(
So we were thinking again and we will make a runtime for games written in C. Which is basically a dynamic linker that links assemblies to the graphics and system API at run time.
Also I have been developing a new language together with some help of Cumred.
The language will be editable and compileable to bytecode (not machine code) right on the device or on a PC and will be executable on many different devices such as the microcat, on the PC, maybe on the Gamebuino and (very unlikely but probably possible) on KnightOS.
To make it easier to learn the language, it has a very similar syntax to Lua in many parts though it doesn't require things like "then" or "do" and it doesn't have tables. It does have functions and two different variables which are numbers (16 bit signed) and arrays (8 bit values). We have chosen to lower to lower the size of arrays as most arrays will just hold sprites, levels, maps or strings which are all 8 bit. We will have some functions to write and read 16 bit values to and from the arrays (two items, one is the lower and one the upper half of the 16 bit number).
The language should read the programs though a pipeline off the SD card and not require the full program to be loaded into RAM, as this would limit the maximum game size, which is something that we want to prevent. Programs can therefore be way larger than the RAM of the console as only a small sector of it gets streamed through the RAM at a time. The compiler will create a look up table of all functions and variables at the starts of the program and the functions and other blocks to increase the speed and prevent searching for functions.
The variables are created with "var" (for numbers) and "var[size]" (for arrays while size is the size of the array which can probably be set at run time) and they are garbage collected at the end of the block that they have been defined in. We handle this garbage collection using levels of nesting. If the nesting level of a variable is exited, it is discarded. To help with that, there is a "block" block which does nothing but allows to define variables that get discarded after the block and don't require dirty hacks such as a "if true" block.

Here is an example program:
-- Simple number guessing program

var number = rand(1,100);  -- Create new variable called number and write a random number between 1 and 100 into it
var found = 0;   -- Has the value been found
var guess_no = 1;   -- Guess number
while guess_no <= 3 && found == 0   -- For loop from 1 to 3
    var guess = input_number();   -- Sample function that prompts the user to input a number
    if guess == number
        found = 1;
        print("Found!");   -- Sample function to output a string
    elseif guess < number
        print("Too low!");
        guess_no = guess_no + 1;   -- Increment number
    elseif guess > number
        print("Too high!");
        guess_no = guess_no + 1;
    end   -- End of if
end   -- End of while
  • Calculators owned: TI-84+, Casio 101-S, RPN-Calc, Hewlett-Packard 100LX, Hewlett-Packard 95LX
  • Consoles, mobile devices and vintage computers owned: Original Commodore 64C, C64 DTV, Nintendo GameBoy Color, Nintendo GameCube, Xbox 360, PlayStation 2

bb010g

If you want a nice, small, efficient language that can dance with the bare metal and be pleasant to program, Forth is your language. Your bootloader may very well be programmed in a variant of Forth.

Even if you don't end up picking Forth for the Microcat, read this.
  • Calculators owned: HP 50g, Prime, 28S, 35S, Casio Prizm, dead Nspire CX CAS

Dream of Omnimaga

Maybe Cumred's language for the 84+CE could be based on this once this is finished?

Also I am fine with no Then instruction, as long as we have Else and End. Otherwise, we end up with a big mess of spaghetti code like on the TI-81 (If blocks could only contain 1 line of code unless you just called a sub-program) >.<
  • Calculators owned: TI-82 Advanced Edition Python TI-84+ TI-84+CSE TI-84+CE TI-84+CEP TI-86 TI-89T cfx-9940GT fx-7400G+ fx 1.0+ fx-9750G+ fx-9860G fx-CG10 HP 49g+ HP 39g+ HP 39gs (bricked) HP 39gII HP Prime G1 HP Prime G2 Sharp EL-9600C
  • Consoles, mobile devices and vintage computers owned: Huawei P30 Lite, Moto G 5G, Nintendo 64 (broken), Playstation, Wii U

Powered by EzPortal