September 30, 2020, 05:16:28 am

The shoutbox is currently out of service. Join us on Discord instead.
You can help CodeWalrus stay online by donating here.

proof-of-concept kernel for the TI nspire CX

Started by nspiredev500, February 27, 2020, 09:54:34 pm

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

nspiredev500

TLDR: I am currently trying to create an operating system for the ti nspire and just hit a (small) milestone, searching for testers in the future

In my current project, called simply OSExt for now (because it should extend the normal ti OS sometime in the future), I deviated from simply writing a program that I wanted to integrate with the OS to wanting to write a whole paralell OS kernel.
The whole kernel-idea started because I hated it when I tried to test a program and the calculator crashed, leaving me waiting for it to reboot, and with no clue why it even crashed in the first place.


This is a massive undertaking, and I may never finish it, but I now have a small testing-version finished, which can run a programm in userspace in its own virtual address space, where it can't do harm to the rest of the system if it goes wrong.
Now I need to implement a proper program loader, a way for the program to use files, allocate new memory, create new processes and communicate with them.
I want to use the newlib library, which is actually already used by ndless, so it shouldn't be that hard to port.
My goal is to make a complete POSIX system, so I can eventually port some open-source software to my os.

If someone wants to help me, or maybe volunteer to test it if it's more mature, I'd be happy. Because I only have an ti nspire CX CAS HW-AA, I can only test the rest in the emulator myself.
If you want to test it in firebird yourself: the github page or the .tns file


The current Roadmap:
  • implementing a way for programs to allocate memory
  • implementing a way for programs to access files, if they have the permission
  • implementing POSIX fork and exec to create new processes
  • implementing a message system and a pipe system for the programs to communicate
  • porting newlib
  • implementing a dynamic linker for shared libraries
  • making a lcd driver
  • making a keypad driver
  • making a touchpad driver
  • making a desktop environment with a file browser and the ability to execute programs and let them display to the screen
  • optimizing memory usage
long-term:
  • if I'm not dead already, implementing a usb storage driver (will certainly take much time for this one)
  • porting gcc and binutils, so I can develop the kernel on my calculator
  • if I'm somehow still alive, integrating it into nLoader to boot from it

I have this shell for debug output:


And here is my proof of going into userspace:


Here is the address space. All but the first and second entries are my kernel:


nspiredev500

Nothing groundbreaking, but it now runs a little userspace test on actual hardware.
Now I can start to implement file io, timeslices for processes, IPC and the userspace drivers.
Something a little more visual may be coming in the next weeks, maybe I'm starting with the lcd driver.

nspiredev500

It is now able to draw to the screen while the OS is running without any flickering. It does use a bit of performance though.
And weirdly, trying to turn that off while it is installed crashes the calculator, but not the emulator.
I also implemented a miniclock:

nspiredev500

I have now completed the timer drivers, which means I could now interrupt user mode programs, so nothing can just stay in an endless loop and use all the processor time.
I'm now going to start implementing the system calls for the usermode programs to use.
After that I have to port newlib, and then I can start writing user mode programs, and then I should have something more impressive to present.

Knucklesfan

April 08, 2020, 11:17:47 pm #4 Last Edit: April 08, 2020, 11:22:16 pm by Knucklesfan
This is extremely exciting! I am completely willing to help with testing, but I have an Nspire CX II running fw 5.0 so that might not work. Two questions: is this planned to completely replace the nspire firmware or just work alongside it? Is this planned to replace ndless in the future? Thank you for your work, please keep the project alive it's really exciting!
EDIT:
I just read the GitHub and it answered all my questions lol. Keep up the good work!

nspiredev500

I always didn't like the plain black background of the nspire homescreen, so I added basic background image support for windows bmp files.
Currently it just looks for black pixels and replaces them with pixels from your image, but it vanishes when a menu is opened, because it would colour the text, too.



You just have to put a file called "background.bmp" (.tns, but you don't see that on the calculator) in the documents folder.
I hope the walrus isn't copyrighted  :)

nspiredev500

Quote from: Knucklesfan on April 08, 2020, 11:17:47 pmbut I have an Nspire CX II running fw 5.0 so that might not work.
I want to port it to the CX II, but first I need a new Ndless version to do that.
It will probably be a while until the new version is out.

nspiredev500

small update: You can name bmp files "background2.bmp" to "background4.bmp" and get a slideshow of your images.
They change every 15 seconds currently.

nspiredev500

Now you can actually set the time instead of only seeing it. I made a small time setting screen, but I needed the touchpad for the arrow keys, and the touchpad driver turned out be a bit hard.

Now I will start working on the processes and the scheduler.

nspiredev500

I won't be able to work on this for the next 1.5 months, because I have my final exams in this time.
But after that, I have plenty of time.

nspiredev500

I am mostly done with my finals now, so I can start working on this project again.
I'm now working on the scheduler and some system calls to let user-mode code read and write files, allocate memory and use the framebuffer.
So soon I should have a graphical "Hello user-mode" working, instead of the firebird debugger showing "usr".
Then I want to make a basic input driver, so I can have a first early userspace console.

nspiredev500

I can now load ELF files, relocate them and execute them.
For now the execution is in kernel mode, but this enables me to split functionality into modules and search for them at the start, load them and run them.
This should make the startup faster if you don't need all features, and reduce the file size.

I will split the clock and the background image support into modules.
There will also be a battery charge module, as I was unsatisfied with the 25% reading the OS provides you from the beginning.

I also made a small configuration program that gets compiled with your system compiler.
It asks you what value options should have before the kernel gets compiled.
The options are documented in config/option_ descriptions.txt.

With that I also changed the compiler from my own arm-osext-gcc to a standard arm-none-eabi-gcc (like the one that is included with ndless), to make it easier to compile OSExt.
Now you only need the ndless-sdk and an arm-none-eabi toolchain.
Together this should allow you to actually use the configurability, but I will also release the precompiled modules and a few option configurations.
For more obscure combinations you would have to compile it yourself.

I'll make the 3 modules I talked about, make a small release and then get back to getting userspace working.

nspiredev500

June 07, 2020, 08:43:53 pm #12 Last Edit: June 07, 2020, 08:48:41 pm by nspiredev500 Reason: Added the archive as an attachment, seems to work again now
The module system is finished.
The clock is now a module, and the background image support too.
There is also a battery charge module, but for some reason it makes the calculator run quite slow and breaks the OS battery charge indicator  ??? .

The charset and background images now have to be put into /documents/osext, to reduce the clutter in the documents folder.

The modules have to be put in /documents/modules
At start OSExt tries to load all modules it finds, but there is also a module manager.
You can get to the module manager by pressing and holding ctrl and ,  .
It displays the currently loaded modules and the memory they occupy.
You can move the selection up and down by pressing the up and down arrows on the touchpad.
The first selection (empty at the start) is for entering a module name.
If you transferred a module to the calculator or uninstalled it, you can type it's name (without .elf.tns), hit enter and it gets loaded.
When you select a module name and hit pi, the module gets uninstalled.

lights0123

On the GitHub page, you mentioned:

Quote from: nspiredev500making a proper file brower, the integrated one is too slow

Do you know if it's possible to do async file I/O at all? I've written Rust bindings to ndless, and one thing I've been playing around with is its zero-cost language-level async/await support. I have basic support for the timer and keypad which lets you write code that compiles to a state machine without having to actually think about it, as if there were threads. Typically, in Linux, this is done with fcntl and O_NONBLOCK, but ndless doesn't have (expose?) that syscall. Do you know if there's any ability to do that? I don't know how far you're looking to write your OS--if that just can't be done with Nucleus RTOS, is there any way to talk to the flash chip directly any do some processing while waiting for the results, like a modern computer?

Basically, I'm trying to figure out how you're going to make a faster file browser, as I'd assume that it's I/O bound and you can't really speed it up.

nspiredev500

Quote from: lights0123 on June 09, 2020, 01:45:42 amBasically, I'm trying to figure out how you're going to make a faster file browser, as I'd assume that it's I/O bound and you can't really speed it up.
I don't think it's I/O bound, basically it only needs to iterate over the directory entries and stat them.
It should not take that long to do, even on the calculator.
I think it gets slower to open the more files you have (and i have quite a few), even if they are in directories that aren't open. (or that's just my perception, I don't want to delete all my files right now)
It also doesn't cache that information, and reloads every time you open the file explorer.
That is some room for improvement.
I ran a quick test on my calculator, just iterating over the documents folder and stating everything, took only a fraction of a second. I don't know why the document browser takes so long.
Maybe it actually reads the files until EOF to get the size?

Also I'm working on USB support by writing an EHCI driver, writing the device drivers on linux with libusb to test them and then port them to my project.
If I get USB mass storage drivers to work, I can actually do async file I/O. But that will take some time.

Quote from: lights0123 on June 09, 2020, 01:45:42 amI've written Rust bindings to ndless
That's cool, I thought about using Rust myself, and I even saw your project on github. My first memory manager didn't work, and I thought Rust could help with that. I ended up just doing a rewrite and it seems to work now, but for other things Rust could still be helpful. But I don't know enough about Cargo to do what I need. Maybe you can help me. I would need to compile the code like with GCC's -fPIC or -fPIE flag. It has to use the global offset table and link into my ELF file.
The global offset table is important, because it self-relocates to a higher base address, and will be mapped like this into every address space created.
I looked at your project now, and it looks great :thumbsup: . In your book you say "Your program will never segfault". Something like this was one of the motivations for my project.
Creating a user-mode wrapper around the OS and the hardware, so no program can crash or brick your calculator.
What timer do you use for your wrapper? I want to remain compatible to as many community projects as possible.
I use the first timer for my millisecond UNIX time, because the OS doesn't use it. If your Rust wrapper also uses the first timer, both cannot work at the same time.
But I can expose timer functions as syscalls, and detecting OSExt is also easy, so you could just do a runtime check.
I remember right now that I also have to make OSExt compatible with the lcd-compatibility feature of ndless  9_9.

Quote from: lights0123 on June 09, 2020, 01:45:42 amis there any way to talk to the flash chip directly
Ndless exposes a read_nand and write_nand syscall from Nucleus. But you would have to parse the filesystem youself with the information on Hackspire.
I also tried to read the flash chip directly via memory-mapped IO, but it doesn't work right now and I'm very carefull, because I don't want to brick my calculator.

Quote from: lights0123 on June 09, 2020, 01:45:42 amthis is done with fcntl and O_NONBLOCK
Nucleus writes "initializing POSIX layer" on the uart sometime during boot I think, so it should have the fcntl syscall.
The problem is to know which system call number it has and what arguments it takes.
You could assume normal linux arguments and just fuzz all the system calls in firebird until it works.

Powered by EzPortal