Skip to main content ↑ ↓
Topic: TurboSphere (Read 2437068 times) previous topic - next topic
0 Members and 4 Guests are viewing this topic.

Re: TurboSphere

Reply #465 –
Quote from: Flying Jester – on

Quote from: Radnen – on

...and data access abstraction going on just to make it a bit more readable.


I'm curious how you'd like the data access changed.


Real user-level IO reads bytes you tell it to read and manages it's own pointer. I see a lot of this going on:
Code: (javascript) [Select]

array[at++];


Which is kind of tedious and adds a layer of hard work. What if you forget an at++? Or you didn't concat the two correct bytes? Take this area of code for example,

Code: (javascript) [Select]

    this.animated_next = Turbo.dByteCat(array[at++], array[at++]);
    this.delay         = Turbo.dByteCat(array[at++], array[at++]);


It should totally become:
Code: (javascript) [Select]

    this.animated_next = ByteBuffer.read16();
    this.delay         = ByteBuffer.read16();

// ByteBuffer.js:
ByteBuffer.prototype.readInt = function() {
    return Turbo.dByteCat(this.array[this.at++], this.array[this.at++])
}


No need to hand-increment raw binary pointers, and no need to manually concatenate bytes each time an int or higher is observed. It cuts down in complexity, and increases readability, and provides less areas where there are issues. I use this API to read the files in C#, for example: C# binary reader.

edit:
renamed readInt to read16, for clearer picture these are 2-byte 'short' values being read and not 4 byte integers.
If you use code to help you code you can use less code to code. Also, I have approximate knowledge of many things.

Sphere-sfml here
Sphere Studio editor here

Re: TurboSphere

Reply #466 –
That's a very good point. I was thinking of it at more of a memory-mapping level than a stream level, but using stream semantics does make it a lot simpler and less error prone.

I added a simple stream/reader abstraction that covers arrays, ByteArrays/TypedArrays, and RawFiles, to that end.

Additionally, I added a simple scheme system for defining the binary file formats (well, chunks of them). This had always kind of bothered me, but at this point I think I can tackle it. I added a system/formats directory, which contains json files that define chunks of the Sphere binary formats, as well as a mechanism for loading them (or arbitrary schemes in general) and reading an object from a reader based on a scheme. This makes the construction of any given file usually little more than just defining the order of the chunks (often based on previous chunk data) and structuring the resulting data, which is much cleaner than how things were before.

Re: TurboSphere

Reply #467 –
Sweet! :)
If you use code to help you code you can use less code to code. Also, I have approximate knowledge of many things.

Sphere-sfml here
Sphere Studio editor here

Re: TurboSphere

Reply #468 –
It's only needed for two fields of a single Sphere filetype, and only for single precision, but I've added the ability to load binary fields of data as floating point numbers of arbitrary precision. To do this, I ported a Node.js module that handles ieee754 numbers.

A fellow on the v8-users group pointed me to a trick that Chrome uses for CanvasData. So even Google knows their API is almost useless in the real world!

This makes maps load instantly, since I don't need to make two allocations and three copies for every ArrayBuffer I fill in native.
With that, we can efficiently use readers for everything, and take advantage of the different widths of typed arrays for almost all data conversions.
So, from a 'how much works yet' standpoint, still where we were last time. But from both a speed and stability perspective, things are much better.

Another cool thing--V8 supports PowerPC now. This also means it has support for any of its targets in BE mode (theoretically, only tested for PPC I've heard). The IBM team did this, so I assume it mainly supports BE.
The main issue that remains is that TurboSphere uses modern OpenGL. Before Leopard, I am pretty sure that it won't work out of the box. Fortunately, this is something I have dealt with before. I use Mesa3D on my Windows box to fix the fact that it has Intel graphics, and I believe that the same could be done on older OS X.
So, if one was going to port TurboSphere to PowerPC OS X, here's what you would need to do:

  • Get Clang 2.9+ or GCC 4.7+ (which means not using XCode at all)

  • Compile V8 for native-endian PowerPC. Good luck.

  • Compile Mesa3D for PowerPC OS X. This should be pretty easy.

  • Fix any endian issues with Sapphire. This should be almost totally restricted to api.hpp and api.cpp.



I may dig out my old G3 iBook soon just for this.

Re: TurboSphere

Reply #469 –
Maps and Tilesets now fully load, and maps will fully draw all layers.

This brings up two very simple and effective optimizations I am going to add. The first one (which is necessary for the second) is that we can fairly simply make the tileset a texture atlas. This has three advantages:

  • We only perform a single allocation and upload for all the tiles from the RawFile

  • We only do a single upload to the GPU, and we never need to change textures within a single layer's rendering (we still will need to within the map for Spritesets)

  • This lets us put the entire layer into a single Shape



The last one is the most important. It lets us define the entire layer as a single mesh, and means that we only upload and render a quarter(ish) as many vertexes as we do now, perform a single texture binding, and only a single GLSL Uniform set for every layer. It also allows us (since recursive Groups aren't a thing yet) to put the entire layer and any sprites it has into a single Group. Since it's a part of the API to allow a render script per-layer, we can't always guarantee that the entire map could be one uninterrupted drawing operation. But this should make the maps draw as efficiently as possible.

Re: TurboSphere

Reply #470 –
Yes! I have been doing that with my SSFML. :)

There is one small caveat, however. You need to also be able to quickly update the source-rects on that texture atlas for tile animations. So nothing is going to be truly baked. You can either modify the texture-coords in the "layer mesh" or you can do what I did and redraw the animated tiles within the texture atlas if you treat the atlas as a 'render target'. I'm sure it's faster. I reckon, updating a small 16*16 sized tile graphic is far faster than supposedly changing large layer meshes for each layer that has a tile animation. I might have benchmarked this and noticed the texture atlas method winning over the mesh-updating method by alot, especially on maps with whole layers dedicated to animated tiles (such as water or swaying grass).

So, it's up to you, maybe you'll figure out something smarter? Also I hope you have a good algorithm for tile animation, you should never do more than a singular loop to animate all the tiles. That's why I redraw the atlas, it was faster in terms of algorithmic complexity O(n) than compared to updating the layers O(l*n2). I was able to get really fast animation performance so that whether you have 1 tile animating or a hundred, at different intervals, and at different lengths, would all update at the same speed. No matter if you have a million tiles or not, it always took the speed of refreshing the atlas and never the entire map.
If you use code to help you code you can use less code to code. Also, I have approximate knowledge of many things.

Sphere-sfml here
Sphere Studio editor here

Re: TurboSphere

Reply #471 –
I think what I'm going to try is putting all tiles that are not animated into a single shape, and all the animated tiles into their own shapes.

One thing that this makes clear is that TurboSphere's bottleneck right now is how many Shapes it can draw. We can push out about 200 per frame with no problem, but much past that and things begin to slow down past 60 FPS. That is unfortunate--we can push TONS of vertices through! I'll see if I can find why this is the issue.

Just looking, a part of it may be that for some reason (which I don't recall) I disabled the ability to push an entire Group through the pipeline from the main thread. Instead, all the Groups' Shapes and state changes are calculated on the main thread and then pushed through one by one, even though Groups can do this on their own from the render thread.

EDIT: Yes, that was a huge part of it. Changing the group to iterate its Shapes on the render thread almost doubled the framerate.

I managed to get about 25% higher framerate with something that looked unlikely to me. Apparently, checking and updating the draw mode every frame was putting a bigger strain on the system than I thought. I would guess that this just because any time on the render thread is prime real estate.

Switching to use a texture atlas and a single Shape fixes the performance issue totally on its own, though.

Re: TurboSphere

Reply #472 –
Quote from: Flying Jester – on

One thing that this makes clear is that TurboSphere's bottleneck right now is how many Shapes it can draw. We can push out about 200 per frame with no problem, but much past that and things begin to slow down past 60 FPS. That is unfortunate--we can push TONS of vertexes through! I'll see if I can find why this is the issue.


That's the key fundamental idea behind the sprite batch: draw all objects as a single contiguous vertex mesh as often as you can. Vertices are cool in that you can really set both their texCoords and coords to be anything, anywhere on the screen, supplied they keep the same source texture and draw mode. But sometimes in sprite batching you don't always win. Currently, in my SSFML, if you draw the same graphic a million times it's really fast, but if you draw in an A/B fashion, as in, if you swap between two totally different textures, it slows down a lot since it's committing the batch each time the texture changes. I guess my next idea to speed things up is to truly order the textures so that all like textures are placed down on their own 'layer', but then I worry about render order. It'd be nice to see just 2 commits of the sprite batch when drawing patterned blocks. Of course if texture A and B came from the same source texture, then this doesn't matter, as is the case with map layers, spritesets, fonts or really anything that has a texture atlas.
If you use code to help you code you can use less code to code. Also, I have approximate knowledge of many things.

Sphere-sfml here
Sphere Studio editor here

Re: TurboSphere

Reply #473 –
I was mostly surprised that there was such a higher overhead to non-optimized Groups and Shapes--but it's a lot better now. It's mostly similar to the issues I was having before, time on the render thread is a lot more precious than I originally thought.

I've added preliminary spriteset support. Curiously, just implementing spritesets the way that made the most sense to me, I implemented almost verbatim the Sphere spriteset_object's interface. Without knowing the actual history, just seeing the format docs, I'd guess that the spriteset interface and format has seen more love than most others.

Re: TurboSphere

Reply #474 –
I've been doing a little housekeeping with T5 and the engine proper.

Plugin loading is no longer a terrible mess! And T5 can now properly iterate over the filesystem, which also means I can nicely add the FS operations to script.

I turned on all warnings and error-on-warning for TurboSphere with Clang and GCC, and it's all still working! Hurray!

Re: TurboSphere

Reply #475 –
Getting closer to a working map engine. Spritesets are fully loaded (including legacy versions), and some of the basic person functionality exists.

Most importantly, TurboSphere FINALLY correctly reports errors in Require'd scripts! Now if only I could figure out the issues with the wrong line numbers on non-Windows platforms!

Very unfortunately, we will need to start using Googles HORRIBLY BUGGY gclient tools to build or even fetch V8 soon--even though the project moved to using git!
If you're curious how buggy they are, if you start it with SVN installed but no git, and then later install git, you will need a fresh checkout of gclient. If you do it the other way around, it will mangle your git config when it next starts. Better set everything up perfectly, 100% correctly, first time!

Re: TurboSphere

Reply #476 –
Sounds like V8 is becoming more trouble than it's worth.  Have you considered switching engines?  V8 really doesn't seem like it has the huge performance advantage it once had...

Re: TurboSphere

Reply #477 –
That is true.

I have seriously considered switching to Spidermonkey. V8 is still somewhat higher performance, but SM has occasionally been leapfrogging it. Additionally, SM has an interpreted backend, something that I appreciate more and more--plus it has a native UltraSparc port.

But I also know a bit about Mozilla's build system, Mach. When I was working there, everyone made fun of it. It is slow (compared to autotools), very difficult to configure (although much easier than autotools!), and prone to unnecessarily rebuilding objects. The saving grace is that it integrates quite well with autotools, SCons, and CMake, and is very reliable. The general consensus was that for something the size of XUL/Gecko, SM, and the supporting (mozilla updated) libraries, this was kind of inevitable.

I would guess this is also what is happening with Blink/Webkit, V8, and Google's versions of their libraries. Now that Google is the caretaker of their own WebKit fork (which is increasing becoming different from WebKit), having a unified build probably seems better. They just happened to choose the GYP WebKit build, and therefore the GYP V8 build system, all governed by gclient. Which would not be my first choice.

Re: TurboSphere

Reply #478 –
Why are you building the engine yourself anyway?  You mean to tell me Google doesn't have a minimal package with just the binaries and import libs/other bindings?  SM used to, don't know if Mozilla still does that either though...

Re: TurboSphere

Reply #479 –
I build the engine myself because all of Google's projects use it as a static library.

There is no official Google binaries because Google never uses it as a separate binary. Additionally, I enable a few things that are non-standard (the better debugger, GDB-JIT support, and internal UTF8).

Given the new paradigm for rapid releases of browsers, I don't think Mozilla does that anymore, either. They only barely make standalone Gecko releases anymore, for similar reasons.