Skip to main content ↑ ↓
Topic: TurboSphere (Read 2437223 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

Re: TurboSphere

Reply #450 –
Realistically, I have no idea. I am going to try to write a full map engine in script, but I also fully expect a lot of edge cases to crop up when doing that. I expect that most of the issues and time spent porting will be related to the exact behaviour of the map engine.

Most everything not related to the map engine should be rather simple to port over. Most of the legacy API is rather simple to shim, and except for the graphics API most of TurboSphere's API is intended to just be extensions and minor modifications to the existing API. Even then, I've found the 1.x graphics API to map fairly nicely to the new graphics API.

Re integers versus floats, only software vertex geometry is stored in ints, so it should be quite simple to change it to use floats. Everything else, including transformation factors, script-side geometry, and GPU-side geometry is stored as floating point numbers.

Re: TurboSphere

Reply #451 –
Accuracy is not easy. Even in my SFML engine, I may have the API implemented so that it works but it's not always exactly the same behavior. So a lot of careful considerations have to go into the code you write in order for two games to play the same. It'd be odd not being able to beat a game due to some weirdness in collision, color, or script execution.

The last one, the script execution part would be the hardest. Sphere get's... weird... undefined behavior when you try to call map scripts from a person script called in a delay script while all that was wrapped in an eval. It's uh, it can be difficult. Some games may love that weirdness. True enough persist games call person and map scripts all the time, and if there was an edge case in which a script could not be called you could lose out on data or gameplay.
If you use code to help you code you can use less code to code. Also, I have approximate knowledge of many things.

Sphere-sfml here
Sphere Studio editor here

Re: TurboSphere

Reply #452 –
I'm not too concerned with 100% accuracy. Not yet, anyway. What I need most right now is just a working, mostly correct map engine.

Anyway, I've update TurboSphere to use the newest version of V8. I'm now building V8 with the TurboFan enabled, too. I'm working on rawfiles now, mostly so that I can start reading the Sphere filetypes in script. At that point I should be able to see what I really need to add to the rest of the engine (in particular the gfx API), and begin working on the stuff that the other plugins handled by calling OpenGL in the raw.

Alternatively, I may want plugins to be able to generate surfaces in native code through Sapphire when they open resources...well, we'll see how things look when we get there.

Also, the new V8 seems to stutter a LOT less. I can run unthrottled and not see any noticeable stuttering! Of course, I can't rely on V8 always working that nicely, I'm on a brand new MacBook and I suspect it would be a lot less smooth on my Core 2 Duo laptop.

I'm quite happy to see some new V8 stuff finally showing up. Finally, there is a way to shut down the engine at any time without just jumping ship and calling Exit()! Plus, they've exposed the platform API, so now it's up to the embedder to supply platform methods (like threading calls and atomics). They still provide a default platform for common ones (well...OS X, Windows, Linux, and Android), but that actually will make it a lot easier to port TurboSphere. Especially since I've been making a lot of the stuff V8 needs now on my own, for Sapphire and other plugins.

Which is kind of neat. To a certain extent, both TurboSphere and the V8 instance under it will use the TSPR. Platform integration for the win!

I also may have mentioned this before, but you can pass V8 flags through TurboSphere with the -8 option (or --v8flags). For instance, if you wanted to use the `let' keyword, you could call TurboSphere like this:
Code: [Select]
./turbosphere -8"--harmony --use-strict"
Or
Code: [Select]
./turbosphere --v8flags="--harmony --use-strict"

I've been printing TurboFan stats using some V8 flags like this. This is probably one of the more useful things I've put into the base engine!

Re: TurboSphere

Reply #453 –
The RawFile API with TypedArrays has been implemented...but it leaks memory uncontrollably.

Why, you may ask?
https://code.google.com/p/v8/issues/detail?id=3562

V8 offers no way of reading an ArrayBuffer from native without taking ownership of its backing memory. Fair enough, they want to be sure it doesn't get freed by the wrong side. This takes it out of the GC pool (that is, when the handle is destroyed, the backing memory is not deallocated. The handle can still be GC'ed).

So, it would make sense that you could give the memory back when were done reading from it. Except that you can't! Once you look at it, you own it and V8 refuses to deal with it again.

This is an issue particularly with RawFile.write().

So, then I figured, ugly as it would be, I could perhaps forcibly set the argument to be a new ArrayBuffer with the old contents in it. It would theoretically work, if the API was there, since I am using a backing store that was allocated with the proper allocator by V8.

And there is a method to create an ArrayBuffer from memory you already have...but the ArrayBuffer deals with it as being externalized then!

So, I opened that issue. I don't particularly expect it to be answered. There is an oft-requested feature of V8, to be able to see the memory without externalizing. I'm going to assume that by some fluke of its design, that's dangerous with an internalized ArrayBuffer. That's why I asked for a way to hand the memory back. It keeps strong ownership, but still allows you to read from an ArrayBuffer in native without making its backing memory un-GC-able. And strangely, no one seems to have ever asked for it this way, just to see the backing memory without externalizing. It seems more natural to me for V8's design, given my years of working with it.

In the mean time, I'm going to poke at a hidden member so that I can make an ArrayBuffer with preallocated memory, and then mark it as not External ;_;

Re: TurboSphere

Reply #454 –
I managed to get TurboSphere compiled and mostly working for Windows again.

There are some issues with the rawfiles. As you may have seen on that issue I filed with V8, I actually ended up modifying V8 to have a 'Reinternalize' method for ArrayBuffers. I may have forgotten to apply this change to V8. Because TurboSphere is fairly fault tolerant about loading plugins, it just doesn't load the new scriptfs plugin.

I've added native Win32 threading, atomic, and timing backends for the engine and Sapphire. I also removed some of the very old and terrible plugin loading code that was for WIndows only. This reminded me how long it's been since I've had TurboSphere working on Windows!

I also got Mesa 10.3 rc3 compiled for Windows. I need Mesa for OpenGL 3 on my Win32 dev machine, and I've pulled in the OpenGL 3 and OpenGL ES headers from Mesa into the platform code for TurboSphere on Windows. If you are wondering, my only Windows machine right now is a Dell D630. This machine is a blight upon humanity. I'm considering including a copy of Mesa with the next TurboSphere release to help with testing.

While I have Mesa at hand, I've begun adding support for quick screen caps and texture uploads when TS is using Mesa (since it's all in system memory in Mesa anyway), and the ability to override which OpenGL library Sapphire loads so that you could easily switch to using Mesa or something else. This would also come in handy on Linux with some of the buggier proprietary drivers, just to test behaviour easily.

Re: TurboSphere

Reply #455 –
Needless to say, I've been away for a while.

I was pretty annoyed at how the V8 devs are handling the ArrayBuffer issues that I and many other embedders have been having. Well, I kind of still am. I'm not impressed with how Google manages the project. Then, I hit some issues with not owning a decent Win32 machine, which made everything much harder than it needed to be.
So I took a break and started working on an IRC Client. Which was interesting, and it let me try out my networking library as well as write a much nicer monitor than the one in the C++ standard library. And do some GUI programming, which I still don't like.

I've updated TurboSphere to use the newest version of V8 again. It seems like the major API changes are really slowing down. Or maybe the way I'm using the API is just caught up in the backwaters of what hasn't changed. Either way, it was rather painless. I also reverted to using the slow, per-element setters for ArrayBuffers.

Which means that RawFiles work. I think (fingers crossed) that that was the only thing left the script-side Map Engine needs from native code, so I'm going to start working on the Map Engine once again.

Re: TurboSphere

Reply #456 –
Upgrading to Yosemite made some old ghosts show up again.

Every once in a while, I used to have HUGE latency when I started TurboSphere. A frame would take an entire second to render, even though we were getting solid framerates. Once I updated, I found that this was always happening.

The issue is that before sometimes, and now always, I was getting bad timing with the engine and the render queue. This is because I was using a timeout system rather than a monitor system. So, no problem, I actually just wrote a monitor implementation that I rather like, and it's been tested to work well in Kashyyyk.
Using it made the delay much better, but now we were always getting only about 100 FPS, no matter how little drawing we do. The more I try to push through the pipeline, the slower it gets. I've noted before that mutexes are slow in OS X for whatever reason, but it turns out that condition variables are slow like that multiplied by how many monitors are waiting on them.

So, I went back to an older idea I had, which isn't as nice as just using monitors, but turns out to work very well.

We can consider a call to FlipScreen() to not really be a call to clear the screen, but rather to be a procedural marker that we are finished with a frame's worth of drawing, and that everything following it is in a new frame. So, what if we rendered our frames into not one queue, but multiple queues? We can cycle through, skipping any that are being drawn to, one frame per queue. This can result in frames rendering out of order on extremely rare occasions, but this chance decreases the more buffers we use. The likelihood is inversely exponential (I don't properly know why that is, I just did a bunch of profiling). With only 3 buffers as I originally tried it happens just about every other frame. Using 16 frames, it basically never happens.
I do know that for it to happen, we have to change from rendering many frames quickly to suddenly taking a long time to render a frame. This rarely happens, and the difference needs to be more than the time it takes to draw a frame in OpenGL (which is usually an order of magnitude higher than how long it takes to draw a frame from script). Additionally, if we keep rendering quite quickly again, we will outrun the render thread again.
This leads to dropped frames, but there's no way to avoid that if you draw faster then the maximum screen refresh rate. Well, there are software ways, but in hardware you lose frames.
Fixing out-of-order frames simple. We just need to flush all pipeline buffers except the one currently being drawn from when we cross over it. But, as noted, this will very rarely happen. Most often, we will just lose as many frames as we have buffers. Which is disappointing, but it will keep the FPS smooth (whereas flushing all render queues can be slow, especially to be sure we don't flush out an rendering-in-progress queue), and if we are really outrunning the render thread that quickly we were going to lose a lot of frames anyway. Better to do it smoothly, I suppose.

Using multiple render queues that we cycle through, we are back up to the huge theoretical framerates I saw on good runs before, and we are now constantly running quite smoothly.

Re: TurboSphere

Reply #457 –
Out of curiosity, how difficult would it be to get TurboSphere to compile/work on Android?  I ask because, well, technically Android is a Linux distro, but I don't know enough about the Android API to know how feasible a port would be.  Having Sphere on my S5 would be awesome though. :-)

Re: TurboSphere

Reply #458 –
I have in the past gotten the engine and the necessary libraries to compile using the NDK before, but no graphics plugin. Everything is known to work, except that I'm using OpenGL instead of OpenGL ES.

On top of that, I'm using very new OpenGL. OpenGL ES is based on older version of OpenGL. Once ES 3 is out, it shouldn't be crazy hard.

Re: TurboSphere

Reply #459 –
We can now read maps and tilesets in script.

The biggest bottleneck? Pushing all the data to script. Because Google doesn't believe embedders can handle manual memory ownership changes.

I've also got a couple demos working. One is the Canvas particle demo that Neo posted a while ago. The other is the Emscripten Box2D demo. This is really cool, because we are using C++ that has been compiled to JS! It's pretty slow unfortunately, likely because V8 doesn't acknowledge ASM.js as a real thing.

The demo (using CubicVR and WebGL) is here: http://kripken.github.io/box2d.js/webgl_demo/box2d.html?10

It actually is almost bearable in TurboSphere, until the boxes hit the ground.

The demos can both be found in the games folder of the TurboSphere github repo: https://github.com/FlyingJester/TurboSphere/tree/master/bin/games
 

Re: TurboSphere

Reply #460 –
The map engine is going well.

You can see the script-side map engine here:

The compatibility layer and main engine:
https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/map_engine.js

Map implementation:
https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/map.js

Tile and Tileset implementation
https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/tileset.js

Any questions or comments are welcome.

My first thoughts on making the Map Engine in script are that this is the best way to do it, hands down. After many graphics API revisions and map engine attempts, I find this combination to really be my favorite. A smaller core graphics API designed for batch operations and persistent structures behind a JS map engine, with a wrapper for the 1.5/1.6 API.

I wish I had decided to put more of the engine in script a long time ago. Casiotone was right! And I was wrong!

You might notice I put some other scripts in the system/scripts/turbo directory. Most are designed to at least be helpful for other engines (notably the image.js, which shims the old graphics API for the new one). The for_each.js is a Mozilla script. It's not strictly necessary, but I include it in my system scripts that use Array.forEach just in case they ever get used in an engine with older JS.

I've also added a simple CoffeeScript demo, showing how to run CoffeeScript from TurboSphere.=

Re: TurboSphere

Reply #461 –
I just took a peek at the code, and, well... it seems so much easier to do it this way! Especially all the old compatibility get and set functions look like a breeze to implement. And I like how you put it all in Turbo.CurrentMap, very difficult to confuse what the heck that would be. :)

I'm going to take some time to look at the scripts more in-depth later, but I definitely like where you're going with this. Looks like the easiest and best way to implement the map engine, as well as make it extenable when you need it to be!

Re: TurboSphere

Reply #462 –
The thing I see about the map and tileset is it kinda looks a bit hackish still. I'd love to see more normal color conversion and data access abstraction going on just to make it a bit more readable.

But overall I think this is awesome! I might even feel inclined to do this to my SSFML version and just compare the results. The bonus is the map becomes extensible by the user if implemented in JS. That also means no more update and render scripts as we know it. You can get more flexibility in how the loops are used and in what order they are executed. It paves the way for infinite-scrolling map engines, Terraria like procedural generation, and even any other kind of map engine you desire whether it is isometric or platformer.

So, yeah, this is pretty cool.
If you use code to help you code you can use less code to code. Also, I have approximate knowledge of many things.

Sphere-sfml here
Sphere Studio editor here

Re: TurboSphere

Reply #463 –
Quote from: Radnen – on

The thing I see about the map and tileset is it kinda looks a bit hackish still. I'd love to see more normal color conversion...[/url]


Quite understandable. I realized that the handler for BPP > 32 could be unified almost perfectly with BPP = 32, so I did that.

Quote from: Radnen – on

...and data access abstraction going on just to make it a bit more readable.


I'm curious how you'd like the data access changed.


Yeah, a lot of stuff starts to look pretty silly if you can touch the Map Engine in script. Especially how map scripts are handled.

Re: TurboSphere

Reply #464 –
While I support migrating Sphere's formats to being processed in script, until we can definitively finish up the 1.x branch we need to bank putting this in vanilla under "sphere-futures" until either TurboSphere becomes the de facto Sphere engine or Sphere vanilla catches up in speed, power, and/or capability.