Spherical forums

Sphere Development => Engine Development => Topic started by: Flying Jester on March 14, 2013, 01:42:25 pm

Title: TurboSphere
Post by: Flying Jester on March 14, 2013, 01:42:25 pm
Wow, it's nice to have some Spherical forums again!

For posterity:

About TurboSphere

TurboSphere is designed to resemble the Sphere RPG Engine. TurboSphere uses Mozilla SpiderMonkey for JavaScript compiling and execution. Graphics, audio, input, and various other functions available to scripts are provided via plugins. The default graphics plugin is hardware accelerated using OpenGL 3 or 4, and a script-based map engine that implements much of Sphere 1.5's API is included.

Why Use TurboSphere?

TurboSphere continues Sphere's legacy of providing massive power and functionality to script, while having a simple and effective API. It also continues Sphere's use of JavaScript, which is an easy to learn scripting language which can be used to easily code games.

TurboSphere does not attempt to duplicate the exact script functions of Sphere, although TurboSphere's functions are similar, and in general are intended to equal Sphere's.

In addition, TurboSphere boasts several improvements to the original Sphere RPG Engine. These include

- Extensible plugin interface
- No file system sandboxing
- Asynchronous rendering
- Generalized event-based input API
- Support for Mac OS X and Linux

Downloads


If you have Windows (or are using Linux and Wine), download this:
Latest Version (http://flyingjesterentertainment.webs.com/ts-0.3.5c.rar), stable release.

Other Downloads:
TenguDev Turbosphere (http://flyingjester.tengudev.com/), older stable releases.
SourceForg site (http://sourceforge.net/projects/turbosphere/), unstable and source releases.

There are no compiled binary downloads anymore. I do not use Windows on a regular basis, and my OS X machine is heavily contaminated with development libraries. If someone wants, I can doctor up some OS X 10.10, FreeBSD amd64, Linux amd64 or arm6, or Solaris UltraSparc IIIi binaries for them.

The Github repo is the main source code repository:
https://github.com/FlyingJester/TurboSphere/ (https://github.com/FlyingJester/TurboSphere/)

TurboSphere is expected to compile out of the box on Unix platforms with a 10.x release of Mesa or a modern Radeon or Nvidia graphics driver with OpenGL 4.x.

Resources

TurboSphere Function List (http://wiki.spheredev.org/User:Flying_Jester/TurboSphere_API)
Guide to Making TurboSphere Plugins (http://wiki.spheredev.org/User:Flying_Jester/Making_Plugins)
TurboSphere Plugin Architecture Description (http://wiki.spheredev.org/User:Flying_Jester/TurboSphere_Plugin_Roadmap)


TurboSphere Function List (https://github.com/FlyingJester/TurboSphere/wiki/Function-List)
Introduction to Galileo drawing API (https://github.com/FlyingJester/TurboSphere/wiki/Using-Galileo)

Is TurboSphere Cross Platform?

Yes. Sort of. It is largely cross platform, and all of its components should work on Windows, Linux, Mac OS X, FreeBSD, and Solaris. Additionally, it should work on 32 and 64 bit PC and Macs, as well as many other devices including ARM machines. However, it isn't tested very well anywhere but Mac OS X and Linux, and Linux testing could be better.

Highlights of Current Version

- Support for various gamepads, including XBox 360 controllers
- Partial Map Engine support
- Full audio support for sampled audio, no mod or midi
- Configurable game-function name and sgm file name
- Fast and flexible Galileo drawing API and scene graph
- Font, Windowstyle, Map, Spriteset, and Tileset support
Title: Re: TurboSphere
Post by: Flying Jester on March 18, 2013, 06:56:11 pm
A  guide for making TS plugins (http://wiki.spheredev.org/User:Flying_Jester/Making_Plugins) is up on the wiki. Of course, until I get TurboSphere working in Windows again (the plugin system is only cross platform in a structural way), you can only see it in action on Linux (and maybe Cygwin...?).
I have no idea if how I've implemented the plugins is genius or idiocy. But it's working, and is much better than the monolithic architecture I used up to 0.1.7, so I'm happy with it for now.

EDIT: the plugin system is working on Windows as well, now. I've got the bmpfont and graphicSDL plugins working, which is a very good sign. graphicSDL is by far the most complicated, and bmpfont requires graphicSDL, so I think it's just a matter of making the same changes to the rest of the plugins and compiling them.
Title: Re: TurboSphere
Post by: alpha123 on March 20, 2013, 03:45:39 pm

I have no idea if how I've implemented the plugins is genius or idiocy. But it's working, and is much better than the monolithic architecture I used up to 0.1.7, so I'm happy with it for now.

Any plugin system is better than a monolithic engine. :P

That's actually quite fantastic, I'm very happy that Sphere has plugins (especially since V8 doesn't support js-ctypes like I used in Sphere 1.7). That was on my wish list for 1.7, but the old architecture would make that a real headache.

Expect me to write some TurboSphere plugins for sure. :)

Long live TurboSphere!
Title: Re: TurboSphere
Post by: Flying Jester on March 20, 2013, 09:22:47 pm

Any plugin system is better than a monolithic engine. :P

Definitely true! It's already saving me time and effort adding features. I didn't expect it to, but it's also saving me a lot of time porting my changes back and forth between Windows and Linux.

Out of curiosity, what kind of plugins are you thinking of making?


Long live TurboSphere!

:)
Title: Re: TurboSphere
Post by: alpha123 on March 20, 2013, 11:53:18 pm

Out of curiosity, what kind of plugins are you thinking of making?

Things that I come up with for Sphere 1.7 (yes, I do intend to continue that thing), mostly. The first thing that came to my mind was SQLite. Also possibly a pathfinder plugin, as JS pathfinders tend to run out of memory (in my experience).
Title: Re: TurboSphere
Post by: Radnen on March 20, 2013, 11:55:50 pm

Things that I come up with for Sphere 1.7 (yes, I do intend to continue that thing), mostly. The first thing that came to my mind was SQLite. Also possibly a pathfinder plugin, as JS pathfinders tend to run out of memory (in my experience).


Those are nice to have for RPGs. +1 on SQLite, I've always wanted an easier way of storing values for items, creatures, and players. As plugins they would be appropriate too since you don't have to make these right away, and are optional anyways. It might turn out that your game doesn't need SQL or pathfinding, etc.
Title: Re: TurboSphere
Post by: Flying Jester on March 21, 2013, 02:49:26 am
I'd thought about a pathfinding plugin. Yes, a very reasonable thing to be a plugin indeed!

Also, 0.2.0 Windows binaries are up on sourceforge (https://sourceforge.net/projects/turbosphere/files/Bin-Win32/0.2.0/). It's got the plugin system all working on Windows, with even more plugins than I had working before on Linux. At this point the only thing that was lost in the transition to the plugins is the sound system, which I was rather dissatisfied with anyway.

EDIT:
The source is also up. It's got a couple minor issues on Linux+GCC+Make (you have to manually build the main libraries and copy them to the working directory), but it does actually compile. I've been spending more time on the Windows side recently, so the Linux-side build process is not quite up to what I'd like it to be. I'll get that sorted out in a bit. Nice to know now that it works on Windows too and has MSVC project files.
Title: Re: TurboSphere
Post by: alpha123 on March 21, 2013, 11:49:02 am
Have you considered CMake (http://www.cmake.org/) for the build process? It's quite an excellent tool in my experience. Basically, you write a few dead simple files and CMake can generate Makefiles, VS solutions, XCode whatever-xcode-calls-its-projects, and more.
Title: Re: TurboSphere
Post by: Flying Jester on March 21, 2013, 04:44:18 pm
If I was going to use a tool like that, I would probably choose scons. I've found cmake to be very difficult to get to work at all under Windows, and I've had problems getting pkgconfig for cmake working under Linux. From my experiences with Scons, it has always just worked and worked well.

As it is, I don't think I want to learn a new tool like that. I might as well just keep working on my makefile skills. I may have a look at making a scons file for TurboSphere at some point, if these difficulties persist.
Title: Re: TurboSphere
Post by: alpha123 on March 21, 2013, 06:20:11 pm

I've found cmake to be very difficult to get to work at all under Windows

Works great for me. Are you using a recent version?

Quote

As it is, I don't think I want to learn a new tool like that. I might as well just keep working on my makefile skills. I may have a look at making a scons file for TurboSphere at some point, if these difficulties persist.

The nice thing would be you don't have to maintain both a makefile and a VS solution, you could move to a different build system for Linux (like ninja (http://martine.github.com/ninja/)), support MinGW in addition to VS on Windows, etc. Even better you could enable/disable certain functionality depending on what libraries are present; this would be a great combination with plugins.
If you couldn't tell, I *highly* recommend using a "real" build system instead of hand-rolled makefiles; it's much nicer for maintainability. Make not necessarily CMake (although I've had great success with CMake on Windows; haven't tried Linux), but at least something. Scons would be fine as well.
Title: Re: TurboSphere
Post by: Flying Jester on March 21, 2013, 09:52:36 pm
Yes, I tried to use it late last year to make Fluidity, using the newest version of cmake at the time. I simply couldn't get it working on Windows, and it took a lot of effort to make it function on Linux, but I did get that working eventually.

I do like the idea of having a fuller build system, particularly now that the project is much more complex. But for now I think I will still work on fixing the makefiles. Once I need more than just SDL, freetype, and V8 (which means once I get to the audio plugin) I certainly may start using scons.

As it is, the makefiles do very almost work the same way they do on Linux as they do on Windows with MingW. If V8 could also be compiled with MingW, I would put a higher priority on making that work. You can actually compile the core Configmanager, Graphicalg, and Graphiccommon libraries using the DMC's cl.exe (making larger and slower binaries, though).
Title: Re: TurboSphere
Post by: N E O on March 22, 2013, 10:20:29 am
Do you use Code::Blocks for non-VS IDE on Windows? I vaguely remember it having some sort of Makefile generation capabilities in it.

Also, now that you have a GitHub account, why not mirror TS source on it?
Title: Re: TurboSphere
Post by: Flying Jester on March 22, 2013, 04:06:55 pm
Yes, I do use Code::Blocks, and I did use the cbp2make utility to generate the makefiles. But the thing is that cbp projects just aren't quite as advanced as Visual Studio Solutions (they are pretty much identical to a Visual Studio project), and cbp2make doesn't quite work perfectly with GCC > 4.5 (particularly when making libraries). The makefiles are actually more convenient than Code::Blocks alone could be.
Once I realized that CB wouldn't cut it alone, I relegated it to just handling the turbosphere.exe binary. That's all the turbosphere.cbp file (if I even included it) deals with.

Yes, I believe I shall put TurboSphere on GitHub.

EDIT:
And so I did.
https://github.com/FlyingJester/TurboSphere/ (https://github.com/FlyingJester/TurboSphere/)

Fixed build instructions for Linux. (https://github.com/FlyingJester/TurboSphere/blob/master/docs/INSTALL-Linux)

Oh yeah, and a little change log for 0.2.0:
- Added Plugin System for Windows.
- Lost sound support.
- Regained TTF functions.
- Fixed leaks in font.DrawText, Image.SetPixel, and RequireScript.
- Added Get- and SetClippingRectangle functions.
- Added File object (no members).
- Added RemoveFile and GetFileList functions.
- Fixed a couple crashes related to SetResolution.
- Changed JS-side exceptions to be of more descriptive types (primarily TypeErrors, but a few RangeErrors as well).

Known Issues:
- TTFFont.drawText leaks memory.
- On Linux, SetResolution still crashes when you give it negative resolutions.
- the game.sgm seems to need to have the right line endings for your platform (although Unix-style seems to usually work on Windows).
- TurboSphere crashes on Windows when it closes because the script ends and you only have one script loaded.
Title: Re: TurboSphere
Post by: Radnen on March 22, 2013, 06:54:33 pm
Do you have an API list anywhere? I'm curious to see how much you have implemented. And if I could perhaps get started with making some kind of game... I don't need sound right now, but I hope there's enough to create something interactive.
Title: Re: TurboSphere
Post by: Flying Jester on March 22, 2013, 09:40:09 pm
There's a mostly complete API doc in the docs directory included with TurboSphere, and on the wiki for it on SourceForge.

As it is, that game I released just after Halloween (The Unwelcome Guest, it's on RMN) was the first properly working TurboSphere game, and was completely playable except for the sound (at that time it would have worked completely had I replaced the .midi's with .wav's). To be fair, the sound didn't work on Linux with Sphere 1.5+wine for me, anyway!

I might as well give a little list of what works right here:

- FlipScreen and game()!
- Loading and drawing of image files (bmp, png, tga, and certain tiff), windowstyles, ttffonts, and Sphere rfn fonts.
- Creation of Image and Surface objects from files or from specs (w, h, color)
- Some of the graphic primitives: Rectangle, Circle, Line, Point, GradientRectangle, OutlinedCircle (Line is even optimized for drawing in the cardinal directions)
- Image[or_surface].getPixel and setPixel, zoomBlit, and the image.width and height members.
- The Majestic Map engine runs on it!
- Color objects and members.
- Setting and getting the ClippingRectangle, identical to Sphere 1.5
- font.drawText, getStringWidth, and drawZoomedText, GetSystemFont.
- ttffont.drawText, drawZoomedText, and ttf.size (which is writable), although the ttf functions are a little leaky right now.
- GetKey, and all key buffer and keyPressed-related functions. All the Key constants (these are listed in the API.txt and in the SourceForge Wiki for TS, there are some new keys recognized in TS)
IsMouseButtonPressed, the related constants (including support for up to 5 mouse buttons), GetMouseX and Y
GetTime and Delay (which actually idles the engine, so that you don't burn up the CPU like Sphere 1.x does)
GetScreenWidth and Height, SetResolution (use that last with caution! It can handle some garbage data, but some still slips through the cracks and causes hard crashes)
RequireScript (mostly, this is just a little buggy, particularly when the files don't have the right line endings for your platform)

...and more things I don't recall specifically. Once I get the File object fleshed out and get sound up again, you could actually make a full game.

I'm missing quite a lot of the advanced graphics features (especially related to surfaces), and some other stuff. If someone was making a game with TurboSphere, and asked for a specific 1.5 or 1.6 function to be added (or something that would be a good thing to have anyway), I probably would do that function next thing, to be honest. It would be very cool to see someone making a game with TurboSphere, and I would want to help as much as possible.

If you were going to make a game and really wanted sound, I could fairly easily make up a soundSDL plugin out of what I had before. It would be able to play wav and ogg sounds only, but it would work. I'm hesitant to do that and include it in TurboSphere proper because I want to make a new and better sound plugin, probably using BASS or IrrKlang.
Title: Re: TurboSphere
Post by: N E O on March 22, 2013, 10:07:27 pm
New and better sound please, kthxbai! ^_^
Title: Re: TurboSphere
Post by: Flying Jester on March 23, 2013, 12:49:53 am
My thoughts exactly. Except I usually don't think to myself 'kthxbai'.

Also, the makefiles have been fixed (on the github version), and TS can also be built using Scons now.
Title: Re: TurboSphere
Post by: Radnen on March 23, 2013, 09:19:35 pm
I'm going to use github to post any issues I find with it, so check there periodically. I'm going to try and experiment and use turbosphere. So, basically I'm going to beta test it's features!
Title: Re: TurboSphere
Post by: Flying Jester on March 23, 2013, 09:26:37 pm
Ah, I saw the issues you posted, as well. Much appreciated!

EDIT:

I've updated TurboSphere, btw: TurboSphere 0.2.1 (https://sourceforge.net/projects/turbosphere/files/Bin-Win32/0.2.1/).

The entire graphics plugin is built a bit differently now. I removed a lot of old and unnecessary functions that were used for testing before.

New:
- Surface.Line, Surface.Rectangle
- Numerous minor improvements to the graphics system.
- Circles are faster, and OutlinedCircles no longer have issues with colors of alpha<255
   
Fixed:

- Blitting two windowstyles on top of each other no longer causes the second blit to lose it's alpha channel (this was part of a deeper issue that was fixed, but only appeared in that circumstance with the functions existing so far)
- AreKeysLeft and GetKey perform much closer to Sphere 1.5
- RequireScript is less buggy
- No longer crashes on closing in Windows.
- SDL window no longer becomes unresponsive on Windows when the key buffer is not handled by the game.
Title: Re: TurboSphere
Post by: Radnen on March 24, 2013, 02:03:12 am
Amazing work! Just to let you know, I'm building up a unit test framework for testing all kinds of things in your engine.
Title: Re: TurboSphere
Post by: Flying Jester on March 24, 2013, 03:02:23 am
That's good to know. It's probably obvious that TurboSphere could do with a bit more testing, and some regressions have slipped past for quite a while.

Also, I posed this idea on Google+, but now that the forums have returned I'll put it here:

I'm planning on making the mouse buttons used a queue just like the keyboard. The same old mouse functions will still exist, which are an analog of the IsKeyPressed for the keyboard. But new functions, probably called GetMouseButton and AreMouseButtonsLeft will be added.

When I was first suggested this idea, I didn't really think it was all that necessary. But the more  thought about it, the more I realized that whenever I have to deal with the mouse in Sphere I spend a bit of time recreating this functionality. I think it would be beneficial to add it.
Title: Re: TurboSphere
Post by: Radnen on March 24, 2013, 08:45:48 pm
I was able to get the core radlib library to run under TS, but many of the more complicated features can't run. And I cannot fix them since it crashes straight-away without telling me why. For example, a simple syntax error will make it crash, or a function that wasn't defined. It's hard to debug this way.
Title: Re: TurboSphere
Post by: Flying Jester on March 24, 2013, 09:31:47 pm
That's partially because I'm still working on getting v8 3.15 for the windows build. As it is, the v8 library I include is fairly ancient (3.9), and within that it isn't compiled with JIT debugging enabled. The V8 internal error messages end up being somewhat useless right now.

One thing I do not know how to solve just yet is that issues with declarations before or in game() make V8 say that game is not a function.

It should tell you something about the crash, though (assuming you run it in a command prompt). And it shouldn't crash without either message from V8 or a message from TurboSphere.
Title: Re: TurboSphere
Post by: Radnen on March 24, 2013, 09:52:13 pm
Well, it just closes. It's a CTD type of crash, and it does not generate an output file. I put "breaks" (GetKey()) around the code I'm trying to test, but even then it will CTD between two code statements, especially stuff in other code files.
Title: Re: TurboSphere
Post by: Flying Jester on March 25, 2013, 12:00:16 am
Try running it from a command prompt. I never intended for errors to be logged to any files, just posted into a containing terminal window (and if I needed output to file, I'd just run turbosphere.exe >log.txt). It shouldn't crash the terminal.

The whole multiple scripts situation is a little odd right now. Each one is its own compiled script in V8, and it can crash without taking down all execution (which will undoubtedly cause unexpected things to happen). The error that crashed the script will still be logged to the terminal.

I'm trying to make it more like Sphere had it, without just turning RequireScript into a C-style include that will just dump the second script into the first. There surely are elegant solutions working just with V8. But I don't know exactly how to do that yet.
Title: Re: TurboSphere
Post by: Radnen on March 25, 2013, 12:34:52 am

I'm trying to make it more like Sphere had it, without just turning RequireScript into a C-style include that will just dump the second script into the first. There surely are elegant solutions working just with V8. But I don't know exactly how to do that yet.


No kidding, line error's are in the 1000's!

I don't know if this is a bug or feature, but you've made Sphere Objects [object Object] types rather than [object Image] etc. which Sphere did. So, it's harder to tell if an image is an image or some object.

Edit: Now I'm running into an issue with the console. I don't see the entire log. I'm loading hundreds of scripts, okay, so the console has a huge list of stuff, but only stops half-way before the screen pops up running. Then when an error occurs, nothing else is added to the console.

Code: [Select]

$ ./turbosphere.exe
Fixed plugins: 1
Fixed plugins as recorded: 1
Opening SGM file startup/game.sgm
Opening script startup/scripts/main.js
File is good.
Opening script startup/scripts/radlib/radlib.js
File is good.
Opening script startup/scripts/radlib/radextend.js
File is good.
No JS result will be passed
Opening script startup/scripts/radlib/animation.js
File is good.
JS Result will be passed.
function (x, y) {
        this.frames[this.frame].i.blitMask(x, y, this.color);
}
Opening script startup/scripts/radlib/assert.js
File is good.
No JS result will be passed
Opening script startup/scripts/radlib/audio.js
File is good.
No JS result will be passed
Opening script startup/scripts/radlib/colors.js
File is good.
No JS result will be passed
Opening script startup/scripts/radlib/debug.js
File is good.
Opening script startup/scripts/radlib/game.js
File is good.
Opening script startup/scripts/radlib/hooklist.js
File is good.
JS Result will be passed.
function (key) {
        var hook = List.get(this.hooks, key, "key");
        hook.paused = false;
}
No JS result will be passed
No JS result will be passed
Opening script startup/scripts/radlib/event.js
File is good.
JS Result will be passed.
function () {
        return this.parent.constructor.name;
}
Opening script startup/scripts/radlib/game.js
File is good.
Opening script startup/scripts/radlib/hooklist.js
File is good.
JS Result will be passed.
function (key) {
        var hook = List.get(this.hooks, key, "key");
        hook.paused = false;
}
No JS result will be passed
Opening script startup/scripts/radlib/hooklist.js
File is good.
JS Result will be passed.
function (key) {
        var hook = List.get(this.hooks, key, "key");
        hook.paused = false;
}
Opening script startup/scripts/radlib/inherit.js
File is good.
No JS result will be passed
Opening script startup/scripts/radlib/input.js
File is good.
JS Result will be passed.
function (button) {
        return IsJoystickButtonPressed(this.num, button);
}
Opening script startup/scripts/radlib/image.js
File is good.
JS Result will be passed.
function (x1, y1, radimg, x2, y2) {
        // setup positions:
        var top    = Math.max(y1, y2);
        var left   = Math.max(x1, x2);
        var right  = Math.min(x1 + this.width , x2 + radimg.width )-1;
        var bottom = Math.min(y1 + this.height, y2 + radimg.height)-1;

        // collision:
        var colA, colB;
        for (var y = top; y < bottom; ++y) {
                for (var x = left; x < right; ++x)
                {
                        colA = this.canvas.getPixel(x - x1, y - y1);
                        colB = radimg.canvas.getPixel(x - x2, y - y2);
                        if (colA.alpha > 0 && colB.alpha > 0) return true;
                }
        }

        return false;
}
Opening script startup/scripts/radlib/json2.js
File is good.
No JS result will be passed
Opening script startup/scripts/radlib/list.js
File is good.
Opening script startup/scripts/radlib/assert.js
File is good.
No JS result will be passed
Opening script startup/scripts/radlib/debug.js
File is good.
Opening script startup/scripts/radlib/game.js
File is good.
Opening script startup/scripts/radlib/hooklist.js
File is good.
JS Result will be passed.
function (key) {
        var hook = List.get(this.hooks, key, "key");
        hook.paused = false;
}
No JS result will be passed
No JS result will be passed
No JS result will be passed
Opening script startup/scripts/radlib/loader.js
File is good.
No JS result will be passed
Opening script startup/scripts/radlib/mouse.js
File is good.
No JS result will be passed
Opening script startup/scripts/radlib/path.js
File is good.
No JS result will be passed
Opening script startup/scripts/radlib/resources.js
File is good.
No JS result will be passed
Opening script startup/scripts/radlib/savefile.js
File is good.
Opening script startup/scripts/radlib/json2.js
File is good.
No JS result will be passed
JS Result will be passed.
function (filename) {
        if (!Assert.is(filename, "string")) { Debug.log("Filename not a string.", LIB_ERROR); return; }

        if (filename.indexOf(".") < 0) filename += ".sav";

        if (!Assert.fileExists("~/other/", filename)) {
                Debug.log(FormatString("File {?} doesn't exist!", filename), LIB_ERROR);
                return;
        }

        var file = OpenRawFile("~/other/" + filename);
        this.content = JSON.parse(CreateStrin



Edit2: Only when there's a script error hmm... Weird.
Title: Re: TurboSphere
Post by: Flying Jester on March 25, 2013, 01:05:35 am
Well, I do see one thing that I know is my fault. Normally the system that prints 'JS Result Will Be Passed' is only supposed to work on scripts embedded in maps. I'll have to change how that is handled, that for sure is causing a problem here. I believe that whenever it spits out a value from a script (and it looks like it's doing that with functions here? I've never seen that before!), whatever it spits out is lost. But I'm not sure of that, I haven't worked on that part of TurboSphere in about a year. Time to take another look at it!

I believe nothing else is added to the console after a certain point because the original script has thrown some sort of error. I just made up a new version of V8 3.15 with JIT debugging enabled for Windows, and I'll make a new release with both the new V8 included and I believe some changes that should fix this.

On another note, I think that there is builtin JSON in V8 by default, so you shouldn't need the json.js files (but don't quote me on that).

EDIT: Nevermind. Whenever 'result will be passed' is printed, it means that something was actually passed from the script. I removed the special handling a long time ago, apparently. I think that means here that whatever is being passed was not assigned to anything in script?
Title: Re: TurboSphere
Post by: Radnen on March 25, 2013, 01:23:29 am

On another note, I think that there is builtin JSON in V8 by default, so you shouldn't need the json.js files (but don't quote me on that).


I don't know.. I couldn't get foreach to work... and it was implemented in JS 1.6... I'm still using my foreach code! But we'll see... It's still good to have.

I got radgui to now work!

That means, I have now got radlib's debug system working (the interactive version, supplied by radgui). Yay, I can finally report errors! But there were many things it's lacking.

1. Surface.gradientRectangle()
2. Font.getHeight();
3. Accurate Font.getStringWidth()
4. Font/WindowStyle .get/.setColorMask()
5. Image.blitMask()
6. GetFileList()
7. GetDirectoryList()
8. GetNumJoystickButtons,
9. IsMapEngineRunning,
10. Font.wordWrapString(),
11. Windowstyles are not drawing their backgrounds?
12. Fot.drawTextBox()
13. JS Error (Esp error.stack)

Re the "result will be passed": It seems to be the last function/line in the file.

Code: [Select]

Opening script startup/scripts/radlib/hooklist.js
File is good.
JS Result will be passed.
function (key) {
        var hook = List.get(this.hooks, key, "key");
        hook.paused = false;
}


Equals:
Code: [Select]

HookList.prototype.unpause = function(key) {
var hook = List.get(this.hooks, key, "key");
hook.paused = false;
}
Title: Re: TurboSphere
Post by: Flying Jester on March 25, 2013, 01:34:45 am
GetFileList should be working (I did test it, and it seemed to work for me), and Font.getStringWidth should properly work (it has always seemed to for me?).
GetNumJoyStickButtons exists now (I haven't updated the binaries to reflect this just yet).
WindowStyles do not draw their backgrounds yet, this is true. At the time I made windowstyleSDL, I didn't even know they had backgrounds.

blitMasks don't exist yet. As part of generalizing the primitives and blitting features, they (along with surface.gradientRectangle and all other primitives, and font drawing) will be added. You can (or could, I haven't tested this in quite a long time) set the mask for TTFFonts just by accessing the mask member. But TTFFonts are a little leaky right now, and I don't know why.

JS Error seems to be quite possible in V8. DaVince asked me about it a while ago, and I know it can work. We found that in TurboSphere, the error object does exist, it just doesn't do anything. I need to do some more V8 research to get it working right.

I'm really rather sure V8 has a JSON builtin (I might need to do something to make it appear, but I know it exists). I don't know what the deal is with foreach'es, I
think I found them not to work before, as well.


Edit: Yes, the result is (and I say this with fear of using a grossly incorrect term) whatever was accumulated last in the script. I was under the impression that it normally meant that some expression went unused, but I'll bet it's quite possible that that's not true.

And btw, that's a beautiful sight there!
Title: Re: TurboSphere
Post by: Radnen on March 25, 2013, 01:44:26 am
In Windows, GetFileList returns an empty object.
Title: Re: TurboSphere
Post by: N E O on March 25, 2013, 01:29:36 pm
For some super useful cross-platform abstraction, I highly recommend the nall (and phoenix for GUI) libraries by byuu (http://byuu.org/phoenix). Even command-line apps benefit from it!
Title: Re: TurboSphere
Post by: Flying Jester on March 25, 2013, 06:47:21 pm
@Radnen: GetFileList() defaults to the save directory and I haven't given it the ability to move to another directory just yet. Within that limitation, it works fine for me. Filesystem access isn't done yet (and it's one of the things I'm working on at the moment).

@Neo: I've seen Phoenix before (and I always take Byuu's stuff very seriously). I'll have a look again
Also, are we ever getting the long lost [js] tags back?

EDIT: Just saw your other post. Sweet! It's been a long time since we had that!

EDIT EDIT: I've added a  TurboSphere Function List (http://wiki.spheredev.org/User:Flying_Jester/TurboSphere_API). The one of SourceForge is not entirely factual, but I rewrote it here while consulting the one and only true authority on what functions are in TurboSphere right now, the TurboSphere Source Code (tm).
Title: Re: TurboSphere
Post by: Flying Jester on March 26, 2013, 03:05:29 am
 Binaries for TurboSphere 0.2.2 are here! (https://sourceforge.net/projects/turbosphere/files/Bin-Win32/0.2.2/)

New:

- GetFileList works as expected now.
- WindowStyles now draw their backgrounds, and support all the background modes from Sphere 1.5.
- Fixed a GradientRectangle bug where the rectangle is one pixel narrower than specified.
- Fixed Line bugs where when the x's or y's are the same it is draw one pixel off.
- Added File.read, write, flush, and close. Long live T5!
- Fixed a couple bugs dealing with some primitives changing the clipping rectangle back to the whole screen if it was previously set as something else.
- WindowStyle loading and drawing is marginally faster.

Known Issues:
- Negative windowstyle.drawWindow widths or heights cause hard crashes.
- Passing a string with a '\' in it to filesystem functions (especially GetFileList) may cause errors (but hopefully not crashes).
- TTFFonts still leaky.

GetFileList is something I'm worried about. It has a good chance to cause crashes. But I've given it a bit of testing, and it seems pretty resilient so far.
Title: Re: TurboSphere
Post by: Flying Jester on March 29, 2013, 09:47:40 pm
64-bit Windows Binaries have arrived (http://www.sourceforge.net/projects/turbosphere/files/Bin-Win64).

This release (which is otherwise identical to win32 0.2.2) contains a couple more libraries that allow jpegs and tiffs to be used as images.
Title: Re: TurboSphere
Post by: Radnen on March 29, 2013, 11:19:06 pm
Nice, I think it might be even faster. Now, could you get started on the text functions?

Font.drawTextBox() and Font.wordWrapString(). If you first do the latter, the former would be easier. Also add Font.getHeight() and try to fix Font.getStringWidth(). I'll attach the font it wasn't quite working with.
Title: Re: TurboSphere
Post by: Flying Jester on March 30, 2013, 12:06:30 am
I've already got the backend for wordWrapString going. I've been working on the fonts in general lately.

I'm curious to see where getStringWidth isn't working right. At this point I kind of suspect it has to do with my implementation of rfn reading not being quite accurate to how the editor(s) create them (it was derived totally from the internal/font.rfn.txt doc and the system.rfn file).
Title: Re: TurboSphere
Post by: Flying Jester on March 31, 2013, 12:56:09 am
OK, I've got getStringWidth fixed. It was a silly mistake, and was only visible from script, which is why I didn't notice it while making up the wordWrapString guts.
Title: Re: TurboSphere
Post by: Flying Jester on April 01, 2013, 06:25:19 am
In lieu of all these problems, like my experience with getKeyString, I've made an executive decision. The use of V8 is not helping make TurboSphere fast, reliable, or stable. In fact, I have come to the realization that using a scripting language is just a limitation to game making. So, I will make TurboSphere use the FAME library (http://www.m68k.com/fame/) instead of V8 or even a JS engine at all. FAME lets us use a much more tried and true language for making games. That is, 68k assembly.

The 68k processor was used in the Sega Mega Drive (aka Genesis), Commodore Amiga (which was groundbreaking in terms of graphics and sound), and the Neo Geo, which is still in use today. To best facilitate the creation of games from the 16-bit and early 32-bit generation, the best tool would be the same one used in the games of that time period.

By simulating a machine using this processor, and exposing the host machine's graphics and sound hardware, games can be made with the elegance of the era Sphere is best at reproducing, and still provide the graphical horsepower of a modern computer.

To see a working proof of concept of the new TurboSphere:Link! (https://www.google.com/search?btnG=1&pws=0&q=April+Fool's+Day)
Title: Re: TurboSphere
Post by: Metallix on April 01, 2013, 07:40:51 am
I have a general question about turbo sphere. To be honest I did not follow the development of turbo sphere for a while, but now I realized it is really worth a try to do something with it. But I learned from the past that it is a bad idea to rely completely only on one engine. ( No update for Kyuus Sphere 2 for ages and I was really confident to continue my game with it :/ Currently it seems it was a mistake going so early in that direction )
So I think about writing a good stable code foundation for Aquatis in JS. It has some pros:

I can reuse some old code parts from the old demos.
If I keep my code clean enough, it will be easy to port to other engines using JS.


For the graphics layer in turbo sphere: How are the graphic plugins implemented? Is there still a GL implementation? Is it possible to boost performance by combining draw calls by drawing a lot of rectangles using the same texture? For that a drawTexturedRectangle function would be necessary, right? The low level implementation could then sort rectangles by texture and draw them all at once.
This would help with particle systems and self implemented map engines or similar stuff...

Or I got something completely wrong^^

Feels nice to have the forums back, btw :D


Haha, first login and I got fooled xD
Title: Re: TurboSphere
Post by: Flying Jester on April 01, 2013, 08:37:56 am
Hey Metallix! Long time no see.

There doesn't exist a proper GL back-end yet, although I do plan on making one (in conjunction with SDL 2.0, when that is finally released). It's very possible for one to be created, with the new plugin system.

Limiting the number of blits (and certain primitives) doesn't help much with speed in TurboSphere, unlike Sphere. Since I'm using SDL's 2D at the moment, the speed of any blitting is very much dependent on the number of pixels to be changed. There would be times that blitting onto a surface and then blitting that surface to the screen would be faster. For example, when there is a lot of overlap of the primitives or textures being blitted to the surface, and the entire group will need to be blitted again.

That might change later (and probably will with other graphics back-ends), but that's how it works right now.

And I would consider TurboSphere pretty safe to target, since by design it is meant to be very close to Sphere. Any Sphere game code should be pretty easy to use in TurboSphere, and the opposite should also generally be true. But at the moment, I would also say that if you intend to eventually TurboSphere, just keep using Sphere for now. The only things that are different (besides TurboSphere still lacking a majority of Sphere's functions) is that TurboSphere objects use true constructors. But even then the arguments to, for example, "new Color()" is the same as the arguments for "CreateColor()".
Title: Re: TurboSphere
Post by: alpha123 on April 01, 2013, 12:42:36 pm
lol at the F.A.M.E. thing, that was really quite funny.

BTW, since you mentioned that objects are created with new, e.g. new Color, does that mean their prototypes can also be extended?
Title: Re: TurboSphere
Post by: N E O on April 01, 2013, 12:50:03 pm
Re targeting - it's my expectation that TurboSphere will reach at least parity with Sphere v1.5 API- and performance-wise soon (I would hope midsummer at the latest, but this is essentially a volunteer effort by one person and real life does get in the way), and once that happens I will be more than happy to give it a proper go on Mac (Snow Leopard, mainly b/c Lion and later piss me right the fck off) and eventually recommend using it as the engine to package over basic v1.5.

Re TS API - you can make the API whatever you want, as long as there's a shim of some sort to existing API. The shim can even simply be a system script that gets packaged with the thing if you don't want to (or can't) build in the aliases easily.
Title: Re: TurboSphere
Post by: Radnen on April 01, 2013, 12:50:55 pm
Haha Jest, good one on the FAME library. :P
Title: Re: TurboSphere
Post by: Flying Jester on April 01, 2013, 07:22:35 pm

BTW, since you mentioned that objects are created with new, e.g. new Color, does that mean their prototypes can also be extended?


If you can't, I meant you to be able to.

Although now that I think about it, I do believe that for a long time I haven't had the tying logic for the constructor and the prototype (and so I think the prototype has some horribly mangled name in JS). It interfered with something I had when...something was introduced in V8, and I never put it back. Thanks for reminding me, it's really quite simple to do.
Title: Re: TurboSphere
Post by: Radnen on April 01, 2013, 07:24:32 pm
So, what happens if you do one of the following?

Code: (javascript) [Select]

var c = new Color(0, 0, 0);

Abort(typeof c) // ???

Abort(c instanceof Color) // ???

Abort(c.toString()) // ???
Title: Re: TurboSphere
Post by: Flying Jester on April 01, 2013, 08:14:33 pm
Bad things. Because without the logic that is needed to tie it all together, it's anarchy in there.

Code: (javascript) [Select]

var c = new Color(0, 0, 0);

Abort(typeof c) // object

Abort(c instanceof Color) // false

Abort(c.toString()) //  ReferenceError: [name of last variable created with 'new Color()'] is not defined.


So it's impossible to get at the prototype from JS right now.
Title: Re: TurboSphere
Post by: Metallix on April 02, 2013, 10:34:17 am
Ok ^^ against your recommendation I am going to play around with turbo sphere. I like the Delay( ms ) function and I think I do not need advanced graphic function for now. But I have problems running a simple script using the RequireScript function. I wonder if I am doing something wrong with the encoding of the scripts? The editor is showing me UTF_8. ( I am currently playing around with some alternative js editors... but that should not be an issue ) See the screenshot for all details. I double checked the path. The file is there. Both are encoded UTF_8. Am I missing something?
Title: Re: TurboSphere
Post by: Radnen on April 02, 2013, 12:19:00 pm
Metallix, are you using the latest version (0.2.2)? I had reported that error to the issue tracker, and I thought Jest fixed that.
Title: Re: TurboSphere
Post by: Flying Jester on April 02, 2013, 01:03:00 pm
That doesn't look like the newest version (which accidentally lost the turbocharger icon and just uses the default windows program icon).

I've seen that bug before, and it was recently fixed.

The newest binaries are on SourceForge right now: Win32 and Win64 Binaries (http://sourceforge.net/projects/turbosphere/files/)

TurboSphere should be able to handle UTF8, as well. It uses that internally when casting strings out of JS.

The Delay function is cool. It was the first thing I did in TurboSphere where Sphere doesn't really have the capability of doing the same thing.
Title: Re: TurboSphere
Post by: N E O on April 02, 2013, 01:46:01 pm
Yea, usually everyone just "rolls their own" Delay function in script, often not much more than the following:

Code: (javascript) [Select]

function Delay(ms) {var t = GetTime()+ms; while (GetTime()<t);} // uses Sphere-specific GetTime, no checking on `ms`


Using the Date object would allow greater portability across JS engines, however, so in the future if someone wants to RYO Delay function I'd recommend using that instead.
Title: Re: TurboSphere
Post by: Flying Jester on April 02, 2013, 01:47:01 pm
Well, in TurboSphere, the built in Delay function actually idles the engine and cedes the processor, so that it doesn't eat up all available CPU time. That's why I like it.
Title: Re: TurboSphere
Post by: alpha123 on April 02, 2013, 03:05:46 pm

Well, in TurboSphere, the built in Delay function actually idles the engine and cedes the processor, so that it doesn't eat up all available CPU time. That's why I like it.

Cool. That's rather handy.

Could you also implement DoesPersonExist and DoesFileExist? These are two more often-implemented but trivial functions that regular Sphere doesn't have. I implemented DoesFileExist for Sphere 1.7, and it appears Sphere 1.6 at least does have DoesPersonExist. I'm not sure about 1.5.
Title: Re: TurboSphere
Post by: N E O on April 02, 2013, 03:28:54 pm
To add to alpha123's request, another reason to have those two built-in is because doing them in script (particularly DoesPersonExist) has been known to cause multitudes of problems in the past.
Title: Re: TurboSphere
Post by: Metallix on April 02, 2013, 04:43:04 pm
Thanks! It works :) I thought I had the newest binaries, but I was wrong.
Title: Re: TurboSphere
Post by: Radnen on April 02, 2013, 07:38:05 pm
Yeah, I added DoesPersonExist to Sphere 1.6, or rather tung committed it for me since he had access to the then sourceforge version (but I remember he didn't like it originally since he believed what you can do in code should stay in code, and the interface between JS and the engine should only give you access to Sphere's features, and nothing else).

I'm guessing Jester will only add DoesPersonExist() when he starts the map engine, which would be some time now... Maybe.

@Jester: I don't know how hard it is to write plugins for your engine, but I'm guessing it's not so hard, if you want I could help with coding the map engine. I have emulated it's code once before in another project of mine (how players are handled / the command queue) So I could help you with that when the time comes. But I'm also going to be super busy for the next 33 weeks as I finish my college education.
Title: Re: TurboSphere
Post by: alpha123 on April 02, 2013, 11:32:14 pm

Yeah, I added DoesPersonExist to Sphere 1.6, or rather tung committed it for me since he had access to the then sourceforge version (but I remember he didn't like it originally since he believed what you can do in code should stay in code, and the interface between JS and the engine should only give you access to Sphere's features, and nothing else).

Well, the problem is doing DoesPersonExist in JS is very inefficient (basically GetPersonList().indexOf(person) > -1... I suppose you could monkey-punch CreatePerson and maintain your own person list, but that gets ugly).
DoesFileExist is a bit annoying to implement, and it turns out Sphere 1.6 (and probably prior) actually has that function in the C++, I just had to expose it to JS in 1.7.
Title: Re: TurboSphere
Post by: Flying Jester on April 03, 2013, 02:40:39 am
DoesFileExist is certainly something I can add.

I haven't begun the map engine yet. Well, sort of, I've played with the idea and made a couple prototypes. But I'm not calling either of them the base of the map engine. That's something I'm kind of putting off, since it will require some things from the graphics plugin in particular that I haven't quite gotten working to where I want them yet.

Writing a plugin for TurboSphere, as shown in some detail here (http://wiki.spheredev.org/User:Flying_Jester/Making_Plugins), is not especially difficult. Certainly no more so that it would be to write it anything that would be included inside the engine itself. I've even written up a couple little functions that make it a bit easier to talk to V8, and I've abstracted a little bit of the SDL-related code as well, to give the graphics plugin more control of the graphics.

The plugin API is still not quite done. I haven't worked out the final system for dependencies yet, just a few config file entries that can control the order of plugin loading. But it's pretty close to what I expect I will have in the end, and I certainly think every part that exists now will be a part of the final design.
Title: Re: TurboSphere
Post by: DaVince on April 03, 2013, 07:39:40 am
Since you've already made a Delay() function, could you add SetFrameRate() to that (and change FlipScreen() to use it too, of course)? It's actually my preferred method of controlling the frame rate because it simplifies the whole thing.
Title: Re: TurboSphere
Post by: Flying Jester on April 03, 2013, 09:34:25 am
Yes, that could be done.

I'd have to ask, though, since I don't normally do it that way: Does SetFrameRate just throttle the frame rate, or does it also try to catch up after lagging calls to FlipScreen?
Title: Re: TurboSphere
Post by: DaVince on April 04, 2013, 03:24:24 pm

Yes, that could be done.

I'd have to ask, though, since I don't normally do it that way: Does SetFrameRate just throttle the frame rate, or does it also try to catch up after lagging calls to FlipScreen?

I'm entirely not sure about this. But I always thought it made the engine wait an exact amount of time after each FlipScreen() call in order to get to n FlipScreen() calls per second. Maybe it just added a small delay in FlipScreen() itself, since only invoking that causes the framerate to actually throttle.
Title: Re: TurboSphere
Post by: Metallix on April 04, 2013, 05:21:39 pm
Somehow I would prefer having FlipScreen() not changed how it currently works, because it should just flip the screen buffer like it name suggests. So a FlipScreenDelay( ms ) seems much more appropriate. Also you can write that function as a system script instead of hardcode it in the engine.
Title: Re: TurboSphere
Post by: DaVince on April 04, 2013, 05:38:30 pm

Somehow I would prefer having FlipScreen() not changed how it currently works, because it should just flip the screen buffer like it name suggests. So a FlipScreenDelay( ms ) seems much more appropriate. Also you can write that function as a system script instead of hardcode it in the engine.

But the thing is, old Sphere had a SetFrameRate() which (I think?) directly affected FlipScreen() anyway, so if Jester were to add it in like this, it actually *would* be how it originally worked! That is, if it indeed did work like that.

Besides, if you never invoke SetFrameRate(), FlipScreen() just works like it always has: working as quickly as it can.
Title: Re: TurboSphere
Post by: Metallix on April 04, 2013, 05:51:57 pm
If that is the case I would agree on that solution. But -I think- the SetFrameRate was only affecting the MapEngine. I can't check it out currently...
Title: Re: TurboSphere
Post by: DaVince on April 04, 2013, 05:53:17 pm

If that is the case I would agree on that solution. But -I think- the SetFrameRate was only affecting the MapEngine. I can't check it out currently...

Nope, there was a separate function for that, SetMapEngineFrameRate().
Title: Re: TurboSphere
Post by: Metallix on April 04, 2013, 05:55:56 pm
Haha :) wow I think there are still things I can learn about the old Sphere ^^
Ok I have no objections then.
Title: Re: TurboSphere
Post by: Flying Jester on April 07, 2013, 12:46:28 pm
Alright, I have DoesFileExist, font.wordWrapString, and font.drawTextBox (including the escaped character parsing) working. I'll make a release in a bit, there's a couple things with DoesFileExist that I need to port back to Windows.

I'm also working on making the small(ish) change to let TurboSphere use V8>3.15. I've been holding pretty steady with 3.15, but I need to make this change if TurboSphere is ever going to use V8>3.15. Plus, this change (coupled with the new V8 function that needs it) will significantly reduce the overhead of parsing TurboSphere objects out of JS and back to the engine. It will also improve the speed of calling any object's member function that is defined by the engine or a plugin, and possibly of internal calls as well (it certainly won't hurt). So there's a lot of cases where it will provide at least a modest speed boost without even considering the performance upgrades to using newer V8.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 08, 2013, 12:55:25 pm
So I'm curious, is TurboSphere's ultimate goal to be a drop-in replacement for the current 1.x engine? That's the vibe I'm getting, but I want to make sure.

Oh, and I also have to ask: It won't go Turbo and deliberately introduce game-breaking glitches into my game out of jealousy, will it?
Title: Re: TurboSphere
Post by: Flying Jester on April 08, 2013, 01:03:01 pm
It is mostly a drop in replacement for the 1.x engines, yes.
At the same time, I've made a few minor changes that there had been talk of adding to the 1.x engine for ages (like using proper constructors for Sphere objects, for one).

It's not ready to be that just yet, though. You can theoretically make a full game with it right now, but for one it's missing a lot of the Sphere functions still (including the entire map engine and any sound or networking support).

If it does go Turbo and takes over your game, and possibly your computer and house and city, just open an issue on GitHub and I'll see about fixing the megalomaniac behaviour.
Title: Re: TurboSphere
Post by: N E O on April 08, 2013, 03:59:31 pm

So I'm curious, is TurboSphere's ultimate goal to be a drop-in replacement for the current 1.x engine? That's the vibe I'm getting, but I want to make sure.



It is mostly a drop in replacement for the 1.x engines, yes.
At the same time, I've made a few minor changes that there had been talk of adding to the 1.x engine for ages (like using proper constructors for Sphere objects, for one).


Once it reaches stable WRT replacing 1.5, I'd recommend having some console output (eg, "Warning: xyzFunctionality is TurboSphere specific, please consider rewriting code to be compatible with Sphere Engine vM.N" or something) whenever TS-specific API is used. Once it reaches 1.7 (the most recent official version currently being developed by ANYBODY) then we can talk about merging APIs if there's any deviation left at that time between vanilla 1.7 and TS 1.7.
Title: Re: TurboSphere
Post by: Flying Jester on April 08, 2013, 04:24:47 pm
Hmm, I should actually add info statements about such things, that's a good idea. Visibility being controlled by configuration, of course.
Title: Re: TurboSphere
Post by: Radnen on April 08, 2013, 08:01:38 pm
Saving up for another release? I do the same with my Sphere Studio. :) I really wanna try out the .wordWrapString() though. Did you look at the original source or did you try to re-implement it? Just wondering, because if you did not copy the original source, then I can't expect the same reliability. ;) I hope that your implementation may fix any bugs or issues that were present in the old function (though there are no bugs that I know of right now, which means, at the very least, it is your implementation that could have bugs in it).
Title: Re: TurboSphere
Post by: Flying Jester on April 09, 2013, 01:32:52 am
I reimplemented it.

I tested it fairly extensively, and it seems to work pretty well. At the moment, I know of no way to make it misbehave. I'll bet there are bugs to be found in it, though!

If you are really itching to try it, that change (and the addition of .drawTextBox) has been pushed to the github repository, which compiles using MSVC 2010 just fine for me (and it may work with cl.exe and scons using the MSVC command prompt too, although I haven't pushed the exact changes I've been using to do that just yet). You'd need to compile V8 from scratch, and get the SDL .libs, though.
Title: Re: TurboSphere
Post by: DaVince on April 09, 2013, 10:35:48 am
Well, someone has seen Wreck-it Ralph. :P

In any case, you're making a lot of progress now, and I am looking forward to the next release!

Question (since I haven't checked things out in a while): is a startup game still supported? I'm going to be trying my hand at writing a new one.
Title: Re: TurboSphere
Post by: Flying Jester on April 09, 2013, 12:49:18 pm
Not yet. The functions to switch games while the engine is running don'w exist yet. The only way to start a game other than the one in the startup folder is to start turbosphere(.exe) with the path to the game as an argument (I think I made that work the same way engine.exe does it).
Title: Re: TurboSphere
Post by: Fat Cerberus on April 09, 2013, 01:40:03 pm

Well, someone has seen Wreck-it Ralph. :P


Haha, indeed, and that movie was awesome. Every gamer owes it to themselves to see it at least once, if only for the nostalgia factor alone. The filmmakers really did their homework there. The fact that the story is great is just icing on the cake (no pun intended, I swear! :) )
Title: Re: TurboSphere
Post by: DaVince on April 09, 2013, 02:06:33 pm


Well, someone has seen Wreck-it Ralph. :P


Haha, indeed, and that movie was awesome. Every gamer owes it to themselves to see it at least once, if only for the nostalgia factor alone. The filmmakers really did their homework there. The fact that the story is great is just icing on the cake (no pun intended, I swear! :) )

I'm going a bit off-topic here, but all that movie is missing is Sonic saying "and that's no good!" when he had a perfect opportunity to.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 09, 2013, 02:41:40 pm
Yeah, we kind of hijacked the thread a bit there didn't we? Getting back on topic, any idea when TurboSphere will be at 1.5 levels of functionality? The current 1.x engine is pretty much a 10-year-old codebase, so it's be nice to use something a bit more modern :)
Title: Re: TurboSphere
Post by: Flying Jester on April 09, 2013, 06:43:32 pm
It'll be a while. I'm generally aiming for summer next year, but it might be later or sooner. Some weeks I get ten functions added in and the ability to read another Sphere filetype, sometimes a month passes with nothing to show.

If you want it sooner, bug folks to make up plugins for TurboSphere. That's 99% of what there is left to do, just flesh out the plugins.
Title: Re: TurboSphere
Post by: DaVince on April 09, 2013, 06:47:14 pm
I think I've found a reason to learn a bit more C++. But I'll wait until your API becomes stable (and until I've actually got the time to properly learn that stuff). :)
Title: Re: TurboSphere
Post by: Flying Jester on April 09, 2013, 07:21:28 pm
At the moment, I really only expect API changes to require minor changes to most plugins written right now.
Title: Re: TurboSphere
Post by: Metallix on April 11, 2013, 05:56:54 pm
I have a super small request:

Is it possible to have an image.blitMask() function? That would really help to create fade effects. That simple function alone would make me so much more happy ^^

Pleeeeeeeaaaaaaaaaseeee *-*
Title: Re: TurboSphere
Post by: alpha123 on April 11, 2013, 06:10:25 pm
If you could think of a nicer way to do blitting than all the zoom/rotate/mask blit combinations (http://wiki.spheredev.org/API:Functions#Blitting_.28drawing.29_images) that would be awesome (that part of Sphere's API kind of irritated me). Maybe a .zoom property on images? Since native prototypes are extendable in TurboSphere, the compatibility script could add the old methods.
Title: Re: TurboSphere
Post by: Flying Jester on April 11, 2013, 07:21:32 pm
Graphics masks are coming. But it's a big system, and I haven't quite fleshed out how I'm going to do it yet. I've been playing around with the idea for a while. I could certainly plan on adding them soon (or next).

As far as zoom/rotate/mask (and transform), at some point everything except a normal blit will use the same backend inside TurboSphere (which will be pretty much like transformBlit) with optimizations for when it would only need to do a zoom or rotate. I'm sure there could be a way to make it more unified in script, but I haven't thought of it yet.

The reason that rotate and transformBlits don't exist yet is that right now I'm not using OpenGL, which would make it much simpler and almost certainly much faster. At some point, I plan on replacing the graphics plugin with an OpenGL one.
Title: Re: TurboSphere
Post by: Metallix on April 14, 2013, 03:10:58 pm
Just a small bug report:
The Screenshot below shows four colored circles using FilledCircle( x, y, r, color );
It seems like it is broken for r = 5 and r = 6... I did not tried other values... r = 4 works fine.
Title: Re: TurboSphere
Post by: Flying Jester on April 14, 2013, 07:13:35 pm
Ah, that. It's actually a pretty common problem with circle drawing algorithms, Sphere's old editor does almost the same thing. I think I do know a good way to fix it now, though.
Title: Re: TurboSphere
Post by: Flying Jester on April 15, 2013, 06:49:19 am
TurboSphere 0.2.3 binaries are  released (https://sourceforge.net/projects/turbosphere/files/), Windows 32 (https://sourceforge.net/projects/turbosphere/files/Bin-Win32/0.2.3/) and 64 (https://sourceforge.net/projects/turbosphere/files/Bin-Win64/0.2.3/) bits.

I've mostly been sitting on this for a while, since I've been pretty busy lately.

This adds font.wordWrapString, font.drawTextBox, DoesFileExist, and upgrades V8 to its newest versiont (3.17). It also fixes a crash I found related to GetFileList and another very unlikely crash in WindowStyles.

Unfortunately, V8 is on the cusp of a huge change to the API. This is probably going to be the last upgrade of V8 I do for a while. Maybe. They might be fixing the weirdness where the only way to add weak reference callbacks is to make a handle persistent and then to change it back to being weak, and how handlescopes seem rather limited. I hope they change that, at least. In any case, they are refactoring the entire handle system, so all of the code I have now for wrapping objects for JS will be deprecated.

I also am distributing the libraries to read more image filetypes. You should be able to also use jpegs, tiffs, and...whatever libwebp can open. I actually don't know what that last one is for, but it came with the newer SDL_image, so I included it.

I also fixed quite a few one-time memory leaks, and a couple other minor leaks.

Internally, I also changed a couple things with plugin.h, which makes argument validation for plugins much less cumbersome.
Title: Re: TurboSphere
Post by: DaVince on April 16, 2013, 09:07:19 am
Good job! Wish I had time to test today.

Quote
... whatever libwebp can open ...

WebP images. (https://developers.google.com/speed/webp/)
Title: Re: TurboSphere
Post by: Metallix on April 16, 2013, 04:43:36 pm
Oh :/ looks like the new version lost its ability to render anything except fonts. Can you confirm that, or is it only on my machine?

And according to the console output GetSystemFont() always creates a new font. Is that a desired behavior?
Title: Re: TurboSphere
Post by: Flying Jester on April 16, 2013, 05:13:31 pm
It actually reopens the font's file and rereads it, too. Since you will be able to change the images in a font object like you can in Sphere, it's supposed to do that.

EDIT:

About not rendering other stuff, have you tried to, or is it just that the included test game doesn't render anything? Because the test game isn't supposed to.

It turns out I went and broke surface.blitSurface. And colors might be messed up. I didn't notice until after I made the release.
Title: Re: TurboSphere
Post by: Flying Jester on April 18, 2013, 09:48:43 pm
I've begun creating the OpenGL graphics plugin.

It feels good to make a new graphics plugin. The old one was a bit long in the tooth, having bits from all the way back when I started, then through the internal plugin API, and then the current plugin API.
Title: Re: TurboSphere
Post by: Metallix on April 19, 2013, 02:29:57 am
Cool news:)

About the rendering. I could not see rectangles and circles drawn in my game. Font was okay. But it will be solved with the new plugin anyway ;)
Title: Re: TurboSphere
Post by: Flying Jester on April 19, 2013, 04:45:06 am
It probably has to do with colors, surfaces, and images being broken. The only reason 0.2.3 doesn't segfault all the time is because TS_Color is such a simple structure that as long as the address that is claimed to be a TS_Color is given doesn't hit non allocated memory, anything can be read as a TS_Color. But all colors given as arguments are as good as random colors.

Pretty much, the graphics plugin in 0.2.3 is broken. It doesn't work, and when it does work, it was random chance that it didn't segfault. I'll make a 0.2.4 release soon (tomorrow?). It will have the last version of graphicSDL, I think.

And at this point, I'm a little sad. Using OpenGL takes some of the fun out of making the graphics plugin--it's so useful! I don't have to reimplement every algorithm for every graphics primitive! Count me among the OpenGL believers. About three hours work, and I've already reimplemented about half of graphicSDL, which took over a year to make. And it's hardware accelerated as well.
Title: Re: TurboSphere
Post by: alpha123 on April 19, 2013, 12:54:48 pm
Haha, that's great!

Maybe I'll try making an OpenVG (http://www.khronos.org/openvg/) (or MonkVG (https://github.com/micahpearlman/MonkVG)) plugin at some point for hardware-accelerated vector graphics. (Vector graphics games are something I've been interested in.)
Title: Re: TurboSphere
Post by: Flying Jester on April 21, 2013, 08:27:41 am
I've been thinking about vector graphics, too. It would be nice to have them, although I doubt they would be used as often as normal images.

Also, using OpenGL makes TurboSphere much more Turbo. I tried out a naive map engine system, which got about 500 FPS for a (16x64)x(16x64)x(1 layer) map at a resolution of 640x480. It gets over 2000 FPS with the same map and resolution using OpenGL, and I'm just using immediate mode--the map engine sounds like a great time for VBOs, particularly once more map layers are added.
Title: Re: TurboSphere
Post by: Flying Jester on April 22, 2013, 03:01:31 pm
The GL plugin is going quite well.

I originally thought that I would also have to completely replace any code in the other plugins that does blitting. While this is still a good idea, it's pretty simple to keep them the way they are and expose an SDL surface blitting function to them. In fact, I already did that with the graphicSDL plugin, although it was not necessary back then.

Already, I've got blit masking (for any blitting function that works so far), all the old blit functions I had with SDL plus transform and rotate blit, and all the old primitives plus polygons and triangles (and all the gradient versions of them), and all with wonderful hardware acceleration. I see now my impression that OpenGL was going to complicate things was completely incorrect.

Fortunately though, all the time I spent making the functions for graphicSDL actually wasn't wasted. At least some of it will still be used in the software rendering of the SDL_GL plugin for the surface objects.

I take back anything bad I said, directly or indirectly, about the OpenGL plugin for Sphere 1.5, as well.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 22, 2013, 03:23:54 pm
Does the OpenGL plugin for TS fix the off-by-one errors in 2X scaling mode? That's my only issue with it in 1.5, it causes one-pixel gaps to appear between adjacent objects.
Title: Re: TurboSphere
Post by: Flying Jester on April 22, 2013, 03:44:59 pm
It might, it might not? I will check. I remember that happening, and it was annoying. And if any such problems are found I can surely fix them.

It's of no code relation to the Sphere OpenGL drivers, so I'm going to say (before checking) that it doesn't.
Title: Re: TurboSphere
Post by: Flying Jester on April 24, 2013, 08:38:42 am
Oh, I misunderstood before. There are no scaling modes yet, but considering how I'm doing this I don't expect it to have that issue.

Going through the code, I found out that I've missed another memory alignment optimization to use with V8. All built in object member functions have easily had their function call overhead cut by three quarters, since we can now rely on their holder objects to be aligned in memory (and treat them as such).

I've actually got the entirety of the Image object and member functions finished now, including saving images (and surfaces) as BMPs. I'm working on getting PNG support for that as well, but saving PNGs isn't a part of SDL or SDL_Image, so that isn't nearly as easy. Ideally I'd like JPEG and TGA support too, but that's not as important.

And except for circles, all the old 2.2.1 primitives (plus a quite a few more) are working, and now all have their Gradient- and Outlined- companions. Once I unify my older surface code with the new OpenGL system and I get the other plugins working with OpenGL, I'll make a release. A big, beautiful, high-performance, amazing release.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 24, 2013, 08:57:47 am

Once I unify my older surface code with the new OpenGL system and I get the other plugins working with OpenGL, I'll make a release. A big, beautiful, high-performance, amazing release.


You mean one that I can port Spectacles over to? :-)  That'll be nice, finally being able to get off of the five-year-old 1.5 engine!
Title: Re: TurboSphere
Post by: Flying Jester on April 24, 2013, 09:16:55 am
Well...no.

There's no map engine yet. I am glad I waited to properly make one until I went OpenGL--I've got some ideas for how I can use all this modern technology, it should be very efficient.

But no map engine yet. Also, no sound either. But both will exist eventually (probably the map engine sooner, too).
Title: Re: TurboSphere
Post by: Fat Cerberus on April 24, 2013, 09:43:17 am
The sound I could do without; it's still pretty early in Specs' development process.  But yeah, the lack of a map engine is kind of a dealbreaker.  Oh well, no need to rush, better to take your time and do it right than push out buggy code as a release.  I should know, having had plenty of experience with that--I've been working on the Spectacles Saga since 2006 at least! ;-)  Well, not the programming, but other aspects of it (story, characters, etc.)

As for OpenGL: Isn't it nice that pretty much every processor has a GPU built in now? It's finally viable to make a 2D game that relies on hardware graphics acceleration without having to worry if it'll run on a low-end machine.  Even the cheap AMD APUs like the crappy E2 processor in my all-in-one desktop are worlds better than the onboard graphics of a few years ago. I can actually run 3D games at decent settings on it!
Title: Re: TurboSphere
Post by: Metallix on April 24, 2013, 10:19:18 am
Wooohoooo can't wait for that awesome release :D Even without mapengine, sound and circles it will be super nice to have something to work with. Proper transform/mask image support makes 90% of a game :D </hype>
Title: Re: TurboSphere
Post by: Flying Jester on April 24, 2013, 10:36:58 am
transform and masks now work...and are HARDWARE ACCELERATED. I'll probably get Circles (likely slow ones, but circles nonetheless) working before the release.

I agree, one thing that had been irking me about TurboSphere for a long time is the lack of transformBlits.

For the map engine, as with just about everything else, going hardware with the graphics has actually made it easier. Much easier. And I'm running out of other things to do, so I'm going to have to just go ahead and build it pretty soon!

On another note, I'm kind of thinking that it would be cool to have a way to set the default color mask for images and surfaces. In fact, I kind of already do it that way internally, except that the mask is always full. Just:

Code: (javascript) [Select]
image.setColorMask(color_object);
//and the corresponging
color_object = image.getColorMask();


It would be dead simple to do, and this already exists for fonts and such. So I think I shall add it.
Title: Re: TurboSphere
Post by: alpha123 on April 24, 2013, 02:50:41 pm
Awesome progress!

When do the network functions fit into your timeline? Those were always one of Sphere's strong points; not many other 2D engines have that functionality (LÓ¦VE doesn't have it, Flash can't do it without some server stuff).
Title: Re: TurboSphere
Post by: Flying Jester on April 24, 2013, 04:46:14 pm
I may do networking over this coming summer. I don't really have any set plan for adding networking just yet, but it does have the advantage of not needing relying in any way on other plugins, so it could be added any time.

I do know that I don't really plan on using SDL_net, unless I look at it and marvel at how useful it would be for TurboSphere.
Title: Re: TurboSphere
Post by: alpha123 on April 25, 2013, 12:53:49 pm
Hm, I just had this awesome thought: if you're using OpenGL for all the rendering, how hard would it be to expose a GLSL interface to JS and allow using shaders?
Title: Re: TurboSphere
Post by: Flying Jester on April 25, 2013, 01:55:11 pm
Simple. About as simple as inlining them in the TurboSphere or plugin source, actually. Since I'm not using Glut or Glew (or anything else like that), so I already am binding up all the necessary functions by hand for use in C++.

And funny, I was just thinking about doing it, too.

Partially because it's a cool idea (a very cool idea), but also because I highly doubt any other engine lets you do that. How often do you get to use custom OpenGL shaders in combination with a 2D, script-based game engine?

I also like the idea because it's simple to deal with any GLSL errors in the included shaders. I can fairly easily make it hard to screw things up with game-included shaders. It could possibly be the easiest way for someone to tinker with GLSL, actually.
Title: Re: TurboSphere
Post by: alpha123 on April 25, 2013, 03:22:59 pm
LOVE2D has that feature, and is actually one of the reasons I was thinking of switching. LOVE has a very fast scripting language (Lua; it can use LuaJIT) + OpenGL + shaders + physics, while Sphere has a slow scripting language (ancient SpiderMonkey) + fast software graphics + networking + a built-in map engine + a very nice API. TurboSphere will have a fast scripting language (V8 is close to LuaJIT) + OpenGL + shaders + networking + a very nice API, and eventually a built-in map engine. Hm, long-term a physics (Box2D, probably) plugin would be rather nice.

Incidentally, wanting to play with GLSL is what made me think of that. :)
Title: Re: TurboSphere
Post by: Flying Jester on April 25, 2013, 05:10:47 pm
I'll have to have a look at how it's done in LOVE2D. I have an idea how I would expose it, but I'm curious how it's done over there.

Also, all the plugins are using OpenGL now. Once I compile and test on Windows, I'll make a release. A lot of silliness on Windows with compiling plugins that relied on a compatible graphics plugin existing is now solved by using OpenGL, as well. Or it should be anyway, I haven't tried yet.

I upgraded the TTF Font plugin, too. It caches often used strings, which more than offsets the change to using fully blended font rendering. I figure if I don't pretty text, why am I using a TrueType font?
Title: Re: TurboSphere
Post by: DaVince on April 27, 2013, 02:34:29 pm
That screenshot is a mess. A beautiful mess. :)
Title: Re: TurboSphere
Post by: Flying Jester on April 27, 2013, 03:31:21 pm
Messes are the best way to test alpha blending.  ;D
Title: Re: TurboSphere
Post by: Fat Cerberus on April 27, 2013, 06:06:49 pm
TurboSphere supports TT fonts? Nice!
Title: Re: TurboSphere
Post by: Flying Jester on April 27, 2013, 10:11:58 pm
TrueType fonts were the first graphics I actually got working. True story.

The release may be a bit delayed. I'm removing all the immediate mode stuff I put in there to get things up and working (and am almost done with that), and a few bits broke when I ported it to Windows. I'll make a release of it just for the sake of having a working OpenGL plugin pack out there, but it has some flaws.

If you want to play with a working OpenGL plugin pack right now, the source is on GitHub and builds quite well in Linux and somewhat well in Windows. There are a couple complicated bits if you don't use Scons, and I think the Scons file is an older version that doesn't quite work out of the box on Windows (I should really fix that).

I am thinking that it might be useful to have an image.tileBlit function, that blits an image with a given x, y, (and maybe width and height), and repeats it a given number of times for the width and height (or just makes the dimensions the base dimensions * number of repeats). Mostly because I found that quite a useful thing to do with a modified Majestic, and it can be done very easily in hardware.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 28, 2013, 04:17:29 pm

I am thinking that it might be useful to have an image.tileBlit function, that blits an image with a given x, y, (and maybe width and height), and repeats it a given number of times for the width and height (or just makes the dimensions the base dimensions * number of repeats). Mostly because I found that quite a useful thing to do with a modified Majestic, and it can be done very easily in hardware.


Bending the texturing hardware to your will, huh? ;)
Title: Re: TurboSphere
Post by: Radnen on April 28, 2013, 04:38:03 pm

I am thinking that it might be useful to have an image.tileBlit function, that blits an image with a given x, y, (and maybe width and height), and repeats it a given number of times for the width and height (or just makes the dimensions the base dimensions * number of repeats). Mostly because I found that quite a useful thing to do with a modified Majestic, and it can be done very easily in hardware.


I see some limited use for it, but not for really for custom map engines. It has a lot to do with the fact that at one point many tiles would have changed, and so not always is one tile being repeated. I think the use of a sprite batcher is better, many modern game engines use one (XNA, pixi, etc): they increase the drawing FPS 1000% in some cases.

It has to do with the communication between the CPU/RAM and GPU/VRAM. Drawing hundreds of small images can be slower than batching them into 1 image and drawing portions of that image to screen. Here's the general algorithm:

1. "Draw" all images to screen. They aren't actually being drawn at this point.
2. For each object:
    a. Keep track of the image handle being used, don't store repeat handles.
    b. Keep track of the x/y locations of all drawn objects.
3. Take all the images and put them into one large batch. It's up to you how to create this 'texture atlas'.
4. Send it to the GPU, and perform source/destination blits. The source being the texture-packed location of the image drawn, and the destination being where the user wanted to put it.
5. See the magic at play.

That basic framework can really speed graphics up. Of course, there's more to it. If you introduce different blit modes, they have to also be passed to the sprite batcher (so in other words it changes the API for drawing GFX). If you want to use shaders on particular blits then they might as well just be sent to different sprite batches altogether (perhaps not even seeing a speedup).

An API would be something like this, (how XNA does it).
Code: (JS) [Select]

var sb = new SpriteBatch();
while (true) {
    sb.begin();

    sb.blit(image, x, y, red);
    sb.blit(image2, x2, y2, red);
    sb.blit(image3, x3, y3, red);

    sb.end();
   
    FlipScreen();
}


The sprite batch is *not* like drawing to a canvas and drawing that whole canvas to screen. It's just an organizational tool.
Title: Re: TurboSphere
Post by: alpha123 on April 28, 2013, 05:03:31 pm
Sprite batches would be so amazingly useful. I'm actually working on a custom map engine currently (for Tiled (http://www.mapeditor.org/) maps), and they would be excellent. Currently I'm drawing everything to one big image and blitting that (it's about 200 FPS on Sphere 1.6) which is about twice as fast as drawing every tile individually. However, the maps can get pretty big, and blitting enormous images like that can't be good for performance. So I was going to chunk the really big image into a few large ones, and blit them depending on the camera. This approach, however, would be better for performance and much easier to implement. (Although since I'd like the game to run on default Sphere 1.5+ (at least until TS is more stable/has networking functions) so I'd probably have to keep the chunking approach.)

I'd really like sprite batches in Sphere. LOVE2D has them, but I never looked into how they work. It looks insanely useful. Thanks for the explanation Radnen.
Title: Re: TurboSphere
Post by: Flying Jester on April 28, 2013, 06:46:58 pm

I see some limited use for it, but not for really for custom map engines. It has a lot to do with the fact that at one point many tiles would have changed, and so not always is one tile being repeated.


Oh definitely not specifically for maps. But it's something that I end up doing somewhat regularly, and it can be done very easily and quickly. In fact, not much higher overhead, if at all, than would be just blitting a single image normally.


I think the use of a sprite batcher is better, many modern game engines use one (XNA, pixi, etc): they increase the drawing FPS 1000% in some cases.

It has to do with the communication between the CPU/RAM and GPU/VRAM. Drawing hundreds of small images can be slower than batching them into 1 image and drawing portions of that image to screen. Here's the general algorithm:

1. "Draw" all images to screen. They aren't actually being drawn at this point.
2. For each object:
    a. Keep track of the image handle being used, don't store repeat handles.
    b. Keep track of the x/y locations of all drawn objects.


That's the complicated bit right there. The Sphere API is not well suited to this, it's pretty much a shadowy reflection of GL immediate mode, and doesn't control graphical element persistence. I could work out something to deal with it, but I'm just getting everything up and running right now.


3. Take all the images and put them into one large batch. It's up to you how to create this 'texture atlas'.
4. Send it to the GPU, and perform source/destination blits. The source being the texture-packed location of the image drawn, and the destination being where the user wanted to put it.
5. See the magic at play.

That basic framework can really speed graphics up. Of course, there's more to it. If you introduce different blit modes, they have to also be passed to the sprite batcher (so in other words it changes the API for drawing GFX). If you want to use shaders on particular blits then they might as well just be sent to different sprite batches altogether (perhaps not even seeing a speedup).

An API would be something like this, (how XNA does it).
Code: (JS) [Select]

var sb = new SpriteBatch();
while (true) {
    sb.begin();

    sb.blit(image, x, y, red);
    sb.blit(image2, x2, y2, red);
    sb.blit(image3, x3, y3, red);

    sb.end();
   
    FlipScreen();
}


The sprite batch is *not* like drawing to a canvas and drawing that whole canvas to screen. It's just an organizational tool.


If I introduced new parts to the API to handle this, it could be done very easily. It makes me a little wary that what you wrote looks suspiciously like immediate mode GL, though.

As far as blitting performance, at the moment TurboSphere is fairly evenly split between being CPU and GPU bound. Each image has an associated texture in VRam, and a blit just calls up that texture to be blit. I'll do some testing to see how the performance is between minimal large blits and many small blits.

The thing is, having all image blits occur close together won't do that much (or at least I don't think it will). I think a much larger performance gain could be granted by having some sort of batcher that stores a series of images and primitives and related coordinates that are modifiable (but hopefully won't all be modified every frame) and just calling the batcher to blit them all at some point in the game loop. It would even be simple to translate their positions all at once by some value. Having some form of graphical element persistence would make it much easier to use the faster and more modern parts of graphics hardware, and it's what I've been imagining for the new map engine as well.
Title: Re: TurboSphere
Post by: Radnen on April 28, 2013, 06:50:58 pm

Currently I'm drawing everything to one big image and blitting that (it's about 200 FPS on Sphere 1.6) which is about twice as fast as drawing every tile individually.


Well, that's not how a sprite batch works, but you might already know that. Just for clarification if you do 2000 calls before it's still going to do 2000 calls after. It doesn't turn 2000 into 1 (like in the section I quoted). The idea is that those 2000 calls came from different parts of memory / different image handles / context's. A sprite batch is one lump image, where all other images are dumped into. The GPU can do 20000+ draw calls easily if they all came from the same place. So the speedup is not from limiting draw calls! It's by doing perhaps even more draw calls than you did before, but all from one place. That is the trick.


I think a much larger performance gain could be granted by having some sort of batcher that stores a series of images and primitives and related coordinates that are modifiable (but hopefully won't all be modified every frame) and just calling the batcher to blit them all at some point in the game loop.


Exactly what I'd like to see! (What er, I've been talking about really!)
Title: Re: TurboSphere
Post by: Flying Jester on April 28, 2013, 07:42:15 pm


I think a much larger performance gain could be granted by having some sort of batcher that stores a series of images and primitives and related coordinates that are modifiable (but hopefully won't all be modified every frame) and just calling the batcher to blit them all at some point in the game loop.

Exactly what I'd like to see! (What er, I've been talking about really!)


OK, that's good. It looked more like what would happen with your example code is less state changes--which would increase performance, but not by much.
Title: Re: TurboSphere
Post by: alpha123 on April 28, 2013, 09:04:16 pm

Well, that's not how a sprite batch works, but you might already know that. Just for clarification if you do 2000 calls before it's still going to do 2000 calls after. It doesn't turn 2000 into 1 (like in the section I quoted). The idea is that those 2000 calls came from different parts of memory / different image handles / context's. A sprite batch is one lump image, where all other images are dumped into. The GPU can do 20000+ draw calls easily if they all came from the same place. So the speedup is not from limiting draw calls! It's by doing perhaps even more draw calls than you did before, but all from one place. That is the trick.

I didn't mean to imply that's how a sprite batch works; rather than that a sprite batch would be a better approach. So a sprite batch would work like I add every tile to the sprite batch, and then render each tile from there as needed? So I'm doing more calls with the sprite batch than with the single-huge-image-blit, but it's much faster with the sprite batch, correct? (Well, for one thing, it wouldn't be drawing 960x1920 images.....)
Title: Re: TurboSphere
Post by: Flying Jester on April 30, 2013, 10:51:33 pm
TurboSphere 0.2.4 has been released! (https://sourceforge.net/projects/turbosphere/files/Bin-Win32/0.2.4/)

Now with hardware accelerated graphics and V8 3.17. I'll go and update the wiki page I made for the API in a bit, there's a lot of new stuff, but the included test game shows off a lot of it.

I've also been working on fixing up the scons file to make it build from source easily on computers that are not my laptop. Using the exact github source, it very almost builds perfectly on my Arch machine and my Debian machine. I'll update the source again soon to make it work more easily.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 30, 2013, 11:18:34 pm
Tried to run it, got this output (had to run it from PowerShell just to get the output as the console window closed immediately when run from Explorer):

Code: [Select]
Fixed plugins: 1
Fixed plugins as recorded: 1
Opening Game.
Opening SGM file startup/game.sgm
Opening main script.
The main script is startup/scripts/Jest_Main.js
Opening script startup/scripts/Jest_Main.js
Getting number of plugins.
Getting number of plugins.
Plugin graphicSDL_GL.dll is open.
[SDL_GL] Video mode setup succeeded.
[SDL_GL] Error: GL_EXT_pixel_buffer_object is not present.
Title: Re: TurboSphere
Post by: Flying Jester on April 30, 2013, 11:29:39 pm
What is your graphics card?

I can remove that check and it will run fine (for now). I'm planning another release pretty soon, I guess one more thing to change.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 30, 2013, 11:39:18 pm
It's a G3 Core i7, so Intel HD Graphics 4000 I believe.  I've played Team Fortress 2 and Portal 2 on this machine at 1080p and it doesn't lag at all, so quite jarring for a 2D engine like TurboSphere to fail entirely.
Title: Re: TurboSphere
Post by: Flying Jester on April 30, 2013, 11:48:24 pm
Well...I've not tested on anything but NVidia cards (that's all I have unless I go all the way back to AGP cards), and from what I hear Intel HD doesn't have great OpenGL support, so it could possibly be missing that extension. I sort of doubt that, though, it might just be that the driver won't allow it to be accessed from the compatibility OpenGL context that SDL opens.

I'm guessing that TF2 and Portal 2 have code paths that don't require certain features when they can't be used. I haven't done that yet, and so if you don't have an extension TS needs (and it doesn't need any of them yet--and likely won't for quite some time), then it will just tell you so and quit. I put that check in because I'm using that extension in the unfinished map engine (perhaps I won't now, or at least include a safe setup for when it doesn't exist), and I forgot I left the check in the graphics plugin.
Title: Re: TurboSphere
Post by: Radnen on May 01, 2013, 01:03:56 am
Lord English: Are you running on Win8 or Win7? The drivers for Win8 may not work, for Win7 do you have the latest drivers?
Title: Re: TurboSphere
Post by: Fat Cerberus on May 01, 2013, 01:28:03 am
Windows 8.  Seems odd that the Win8 drivers would be worse than the Win7 ones, though...
Title: Re: TurboSphere
Post by: Metallix on May 03, 2013, 06:49:32 am
FYI: The new release is not working on my machine, too. ( win8 ) I got the same output.
Title: Re: TurboSphere
Post by: Radnen on May 03, 2013, 01:28:36 pm
The default program works in Win7, but when I have it load my Radlib library (that I fitted to work with TS) it crashes with a segmentation fault error. It was in the middle of requiring scripts. There were like 51 scripts it was trying to load.
Title: Re: TurboSphere
Post by: Flying Jester on May 03, 2013, 04:02:43 pm
Most likely it wasn't the number of scripts. It might have been some glitch with RequireScript, though. But I would bet it was some engine function that is not working right.

I might add that GetSystemWindowStyle causes crashes on Windows (but not Linux). I know what's wrong with it.
Title: Re: TurboSphere
Post by: alpha123 on May 03, 2013, 04:15:28 pm
I can't run in on my Windows 7 computer. I get the same output as Lord English.

This isn't a particularly new/good computer though (2009), and it has an integrated Intel GPU with somewhat poor drivers, so that may be the reason.
Title: Re: TurboSphere
Post by: Flying Jester on May 04, 2013, 02:13:19 am
Not that I think this will have a huge impact, but getting the newest drivers (and actual Intel ones, not the ones packaged with Windows) might make it work.

But it's possible that that GPU does not have the GL extension GL_EXT_pixel_buffer_object. In which case TurboSphere 0.2.4 will always refuse to work with it.
Title: Re: TurboSphere
Post by: Fat Cerberus on May 04, 2013, 03:12:19 am
Windows Update always gets the real Intel drivers automatically for me (annoying tray icon and all), so that's not the issue.  Of course, that may not be the latest version all the time, so I'd have to look into that...  I find it hard to believe the GPU wouldn't support it at all.  Integrated or not, the i7's GPU isn't a slouch by any means--the Windows benchmark gives it a 6.8 for gaming graphics!
Title: Re: TurboSphere
Post by: Metallix on May 05, 2013, 07:38:38 am
It is probably then an issue with intel on board GPUs? I have the same setup too...
Title: Re: TurboSphere
Post by: Flying Jester on May 06, 2013, 04:07:45 pm
I wouldn't really call it an 'Issue', though. I could also search for the ARB version of that extensions (which operates a little differently, but produces the same results), but we don't need that extension for anything that is happening in TurboSphere right now. I'm in a place with terrible Internet, but I'll put up a new and improved version of TurboSphere soon (this new one doesn't use immediate mode, although is really identical in performance). The new release will also include a more fleshed out plugin loading system that will be much more configurable (which is not really an issue yet, but if we ever had more than one primary graphics plugin, for instance, and different games used different plugins this would be necessary).

I'll also need to have a look at seeing if a TurboSphere configuring plugin for Sphere Studio would be useful. It's all just in plain text config files now. I could have a little code sharing here and use T5 to read and write the config files. I sort of assume that C# has some INI reading/writing capabilities in the core libraries, but it makes sense to only use on solution for files that both the engine and editor muck with to make sure they are readable on both sides.
Title: Re: TurboSphere
Post by: Flying Jester on May 09, 2013, 10:44:00 pm
Once again, TurboSphere has a map engine. I've worked out two different methods for drawing the map--an unoptimized draw, a more optimized RLE and chunking renderer (about 20% more efficient on a good day and with a good map for it), and I'm working on a much better buffer based renderer. The unoptimized renderer mainly exists to test out new features and for debugging. It works perfectly, and is so simple it pretty much has to work perfectly. The RLE/Chunking renderer is nice because it is faster, (it looks for contiguous chunks of the same tile to all draw at once, and failing those draws contiguous strips of the same tile all at once) and does not need any extensions (which apparently some drivers don't have? I'll have to research this a lot more, and see if I have any old Intel-GPU laptops around to test things on). The new engine doesn't handle much beyond the map and sprites yet, but all in good time. It looks like drawing a 480x320 screen of 16x16 tiles with 3 layers takes about 0.5 ms with the RLE/Chunking renderer, and calculating the strips and chunks is easily as fast as an unoptimized draw alone is, so that seems pretty good for a fallback method. The buffer based renderer will definitely be faster as well.

It is quite simple to compile the included scripts in the maps at load time with V8. I'm not sure this will actually be any faster than the way 1.5/1.6 is, but it seems like it should be. At the very least, this grants the embedded scripts the same speed benefits of stand alone script files.

I would also like to add the ability to set the map engine scripts using functions directly instead of strings. It seems much nicer that way, and it's easily done on the engine side.

About the Sphere filetypes...I don't understand why the tileset files (and embedded tilesets in maps) are the way they are. Unlike every other Sphere file format, the tiles are each listed in bitmap format, and then after all the tiles' images we begin with their headers (or I guess footers in this case). Why doesn't each info block follow or precede the tile's data? I don't know.
Title: Re: TurboSphere
Post by: Radnen on May 10, 2013, 01:46:59 am
+1 on the chunking, my Sphere Studio does that, and makes maps load and draw quite fast.

The tileset was quite strange to load for me. Did you come across the error in which a tiles name is stored after it's header? The name itself is not in the header (primarily because the header is fixed length and the string is not). The name comes after the header (between the current one and the next). The header also stores the length of the name after the obstructions count (it's a short), that leaves 20 bytes remaining rather than the 22 the docs say.

Header
name
Header
name

If the names are empty, then you can load them without the error, but if you load tiles that have names, well, then you come across the error, It's not in the latest docs, this I found out when I worked on it. I've been meaning to update the github docs, though.

Edit:
Here's how I saved a tileset in the Sphere Studio:
https://github.com/Radnen/spherestudio/blob/master/Sphere.Core/Tileset.cs#L170-186

and the new tileset spec:
https://github.com/sphere-group/sphere/blob/v1.6/sphere/docs/internal/tileset.rts.txt
Title: Re: TurboSphere
Post by: Flying Jester on May 10, 2013, 06:42:14 pm
Huh, I haven't come across any tiles with names stored (at least not interspersed with the headers). I'll have to add support for that. And by the 'latest' docs, you mean the ones that are dated January of 1999?  :P

If you want to make it so your editor can save them a new way (a version 2 of the spec), I'd gladly add support for it.

The way I'd like to see it:

Header
Image
Name/Strings

Header
Image
Name/Strings

...

Like all the other Sphere filetypes. Every string is 2 bytes of length info, and then characters for the specified length. It would be consistent for the names (stroke strings section) to follow the headers. It mainly bothers me that the headers are not interleaved with the data, every other filetype follows that format, and it makes things more complicated that it doesn't work that way.

If you are updating the docs, a note on what is meant by 'word', etc. would be nice. It became clear pretty quick when I started trying to read the files, but when I see 'word' I think '32 bits', since that's what the basic data types are in most languages these days, even in 64-bit systems.
Title: Re: TurboSphere
Post by: N E O on May 11, 2013, 01:01:22 pm
'word' is 16-bit, 'dword' is 32-bit, 'qword' is 64-bit. It's been this way for decades.
Title: Re: TurboSphere
Post by: Flying Jester on May 11, 2013, 05:07:00 pm
I'm now aware. DWord and QWord are very self explanatory if you know the word length.

I'm sure that if you are actually using the Sphere source that the filetype docs are much more useful.
Title: Re: TurboSphere
Post by: Harry Bo21 on May 11, 2013, 08:05:27 pm
Quote
Does the OpenGL plugin for TS fix the off-by-one errors in 2X scaling mode? That's my only issue with it in 1.5, it causes one-pixel gaps to appear between adjacent objects.


I noticed no-one answered this. Im pretty certain Kyuu fixed this at my request. He sent me a new .dll for it. I lost it a while back and someone posted another copy and im pretty certain im using it now on this PC so we can post it up here

It might be this one?

** EDIT ** and why am I a "newbie" lol

** FURTHER ** I was gonna say if you need PCs/Laptops to test on I have a bunch here.

PC on XP - pretty old, still has a parallel port which has been very useful for me
PC on Win 7 - About 4 years old - on AMD graphics
PC on Win 8 - 3 months old on Nvidea
Laptop on XP - Also about 7 years old ATI Radeon I think
Laptop on Win 8 - 2 years old on ATI Radeon

My friend and step dad have various Linux distributions also
Title: Re: TurboSphere
Post by: Flying Jester on May 11, 2013, 10:26:24 pm
It would be very cool if you could test the latest release from sourceforge on the NVidia laptop and also on an ATI laptop. I've ascertained that it definitely doesn't work with Intel graphics, but it should almost definitely work on NVidia computers. I don't know about ATI machines.

As far as TurboSphere on Linux goes...it works with scons, but you *do* need to build and install V8 manually (as in compile it and then copy it to `/usr/lib[64]/turbosphere` before using scons on TurboSphere). Directions for all that (except the manual copying) is in the docs folder.
Title: Re: TurboSphere
Post by: Flying Jester on May 16, 2013, 05:54:18 pm
Well, I was going to wait to make a release until I got the map engine working with VBOs (for speed)...but I did a little test.

Drawing using RLE-ing on the tiles, with a 2 layer map: 1300 FPS.
Drawing using RLE-ing on the tiles, with a 1 layer map: 1400 FPS.

Just looping and counting blank frames for the same amount of time: 2400 FPS.

So with that in mind that speed is very acceptable. Just the drawing is more than half as fast as...well, doing nothing at all.

The map engine is a lot like what I released back in 0.1.0, except that now it supports loading any map that the editor makes, and can actually draw multiple layers. I'm working on adding sprite support and animated tiles next.

I need to unpack my desktop to be able to compile on Windows, but I'll get to that this weekend likely.
Title: Re: TurboSphere
Post by: casiotone on May 20, 2013, 03:52:25 am
I spent a few hours last night getting this to compile on OS X. I haven't gotten it working correctly yet, as it crashes in the word-wrap function because of a memory issue. But it compiles! I'll have some patches for you later. The main issues were not including the Cocoa/OpenGL frameworks, and libv8 wasn't included when building configmanager.

edit: the memory issue is in TS_BMPFont::wordWrapString(const char *t, int w, int* num), a SIGABRT is thrown on one of the realloc calls.

Also T5_file::T5_file(const char*file) was failing to read any data from the settings files - I fixed this by seeking to the beginning of the file after opening. My guess is this is caused by the file being opened twice (once to check the file exists, and once to read it) but I'm not sure.
Title: Re: TurboSphere
Post by: N E O on May 20, 2013, 12:57:44 pm
Which version of OS X, 10.6 or one of the later ones?
Title: Re: TurboSphere
Post by: casiotone on May 20, 2013, 01:35:49 pm
10.8.2
Title: Re: TurboSphere
Post by: N E O on May 20, 2013, 02:13:21 pm
Is it possible to build w/the 10.6 SDK on your setup or is that more trouble than it's worth? My Macbook Pro is early 2006, so I can't update it later than the last 10.6 and I don't have a Hackintosh setup w/10.7 or 10.8 :(
Title: Re: TurboSphere
Post by: casiotone on May 20, 2013, 03:02:07 pm
I'm not sure it is possible, I've tried something similar before.

Here's a pull request with the changes I made, anyway:

https://github.com/FlyingJester/TurboSphere/pull/10 (https://github.com/FlyingJester/TurboSphere/pull/10)

At this point it gets to here:

(http://i.imgur.com/gxDVuOX.png)

(edit: sometimes it actually gets a bit further.)

Then receives a SIGABRT. Here's some of the logging output and a gdb backtrace:

Code: [Select]
[ttffontGL] DrawText Info: Binding string Do a barrel roll! to cache slot 0.
JS Result will be passed.
310
[ttffontGL] DrawText Info: Binding string Do a barrel roll! With TTF Fonts! to cache slot 1.
turbosphere(57676,0x7fff71b79180) malloc: *** error for object 0x1006528f0: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug

Program received signal SIGABRT, Aborted.
0x00007fff891f8212 in __pthread_kill ()
(gdb) backtrace
#0  0x00007fff891f8212 in __pthread_kill ()
#1  0x00007fff8aa2aaf4 in pthread_kill ()
#2  0x00007fff8aa6edce in abort ()
#3  0x00007fff8aa4a8a5 in szone_error ()
#4  0x00007fff8aa4bcc6 in tiny_free_list_remove_ptr ()
#5  0x00007fff8aa46092 in szone_realloc ()
#6  0x00007fff8aa4254a in malloc_zone_realloc ()
#7  0x00007fff8aa42cac in realloc ()
#8  0x00000001015ac1da in TS_BMPFont::wordWrapString (this=0x7fff5fbfe740, t=0x7fff5fbfe740 "??_?", w=1606412096, num=0x7fff5fbfe740) at bmpfont.cpp:377
#9  0x00000001015aeac3 in TS_BMPdrawTextBox (args=@0x7fff5fbfe774) at bmpfont.cpp:470
#10 0x0000000100041b73 in v8::internal::Builtin_HandleApiCall ()
#11 0x000010376540654e in ?? ()
#12 0x000010376542dd55 in ?? ()
#13 0x0000103765425367 in ?? ()
#14 0x0000103765412017 in ?? ()
#15 0x0000000100068e2d in v8::internal::Invoke ()
#16 0x000000010001fd73 in v8::Function::Call ()
#17 0x0000000100001be7 in SDL_main (argc=8563848, argv=0x100862000) at engine.cpp:146
#18 0x000000010000311a in -[SDLMain applicationDidFinishLaunching:] ()
#19 0x00007fff874e947a in _CFXNotificationPost ()
#20 0x00007fff87952846 in -[NSNotificationCenter postNotificationName:object:userInfo:] ()
#21 0x00007fff861b760d in -[NSApplication _postDidFinishNotification] ()
#22 0x00007fff861b7346 in -[NSApplication _sendFinishLaunchingNotification] ()
#23 0x00007fff861b4532 in -[NSApplication(NSAppleEventHandling) _handleAEOpenEvent:] ()
#24 0x00007fff861b412c in -[NSApplication(NSAppleEventHandling) _handleCoreEvent:withReplyEvent:] ()
#25 0x00007fff8796c12b in -[NSAppleEventManager dispatchRawAppleEvent:withRawReply:handlerRefCon:] ()
#26 0x00007fff8796bf8d in _NSAppleEventManagerGenericHandler ()
#27 0x00007fff86d35b48 in aeDispatchAppleEvent ()
#28 0x00007fff86d359a9 in dispatchEventAndSendReply ()
#29 0x00007fff86d35869 in aeProcessAppleEvent ()
#30 0x00007fff82a0d8e9 in AEProcessAppleEvent ()
#31 0x00007fff861b0916 in _DPSNextEvent ()
#32 0x00007fff861afed2 in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] ()
#33 0x00007fff861a7283 in -[NSApplication run] ()
#34 0x0000000100003725 in main ()
Title: Re: TurboSphere
Post by: Flying Jester on May 20, 2013, 06:25:01 pm
I'm in a place with terrible internet, or else I would push the fix for wordWrapString (that I did about a month ago), and all the other stuff I've changed. The problem is (or was) that the function BMPFont.addLine doesn't reassign one of the addresses that it reallocs.

This has hung around because realloc very rarely changes the address on Linux, but almost always does on Windows (and apparently OS X as well). It's just random chance when it doesn't crash. Since I do almost all the developing in Linux, I very rarely encounter it while I'm working on TurboSphere, so this bug has hung around a lot longer than I'd like.

Very awesome! I didn't expect to see this working even a little in OS X for a long time!

I'm also pleasantly surprised that plugin loading works on OS X. I didn't really write it with anything but Windows and Linux in mind.
Title: Re: TurboSphere
Post by: N E O on May 21, 2013, 01:58:44 pm
@casiotone - What's your tool chain on OS X? I want to make a note of whether I should use old XCode 3.2.x to build on 10.6 or use MacPorts gcc or something when I go back to my Macbook.
Title: Re: TurboSphere
Post by: Flying Jester on May 21, 2013, 06:05:12 pm
Without any knowledge of what it looks like on Mac (this might in fact be bad advice for OS X), I personally recommend GCC. Both TurboSphere and V8 compile quite well with it, and with pretty much any version of it (from 3.5 all the way to 4.5, at least I've tested those).
Title: Re: TurboSphere
Post by: N E O on May 22, 2013, 12:45:33 am
I think the issue is less whether I can and more whether I should. Keep in mind my limitation of using an older XCode is more a result of being on a version of OS X that doesn't support newer ones and that while I've had much success using the Macports gcc (especially when I was willing to spend almost 12 hours downloading Macports Qt4 to get phoenix to compile on Mac) there's an integration achieved when using XCode that is lacking without it.
Title: Re: TurboSphere
Post by: casiotone on May 22, 2013, 05:47:06 am

@casiotone - What's your tool chain on OS X? I want to make a note of whether I should use old XCode 3.2.x to build on 10.6 or use MacPorts gcc or something when I go back to my Macbook.

I use the XCode toolchain, but I'm not sure if that would be better or worse than gcc on 10.6... In theory, it should be better.
Title: Re: TurboSphere
Post by: Rukiri on May 27, 2013, 02:51:34 pm

Without any knowledge of what it looks like on Mac (this might in fact be bad advice for OS X), I personally recommend GCC. Both TurboSphere and V8 compile quite well with it, and with pretty much any version of it (from 3.5 all the way to 4.5, at least I've tested those).

What version of GCC are you using? The latest is GCC 4.8 which supports C++11 and other improvements.

I'm just curious, how well does turbo support 2.5D?  I generally would just use Unity for this but I'm just not a fan of C# or boo(even though I love python).
Title: Re: TurboSphere
Post by: Flying Jester on May 28, 2013, 08:53:34 pm
TurboSphere doesn't have anything more (and it's not done, so often less) than Sphere does. It does have more potential to, just because a 2.5D or 3D plugin could be made for it. I did have a ray-tracing 3D plugin working for a short time while I was working out the plugin API, but I never released it. It a software renderer (and was ray-tracing), so it was far, far too slow for realtime use. But even then, an OpenGL or DirectX 3D plugin was quite possible, and still is.

I do plan on adding very basic 3D/2.5D support to the core graphics plugin, mainly just primitives and textured polygons with no perspective, for effects like those from Castevania Symphony of the Night. I don't want to get into anything much more than that in the SDL-GL plugin pack, though. That sort of functionality is also something I'd much prefer to add to a sprite batcher, too. It would be much more elegant that way.

I was wrong before, use GCC 4.6 usually, although it did compile on 4.8 not too long ago and I'll bet it still does. I used to use C++11, but at the moment I'm not because when I moved to Scons I didn't add the switch to use it, and I started using adjacent string literal concatenation, which apparently doesn't work in C++11. That is, as far as I know, the only thing in TurboSphere that doesn't work with C++11.

The main problem with getting TurboSphere working from scratch is compiling V8. TurboSphere proper seems quite content to compile with just about any toolchain, it seems.
I would worry a little about things like XCode or MingW, because for everything that needs to work differently on Windows and Linux (GCC and MSVC), I check for _WIN32, and then assume GCC otherwise, and in only a couple places where something really is MSVC or GCC specific (and probably not always then) I check for MSVC or GCC.

It does compile with the Intel Compiler on Windows, though. But I can't distribute those binaries due to the license. And sometimes it's notably slower (particularly the software graphics use, like surfaces and surface primitives) because if I set it to use more than MMX vectorization it makes binaries that refuse to run on AMD processors, like mine (even if I just enable 3DNow!, which makes no sense from a technical standpoint).
Title: Re: TurboSphere
Post by: Flying Jester on May 30, 2013, 06:47:32 pm
TurboSphere on Git has been updated (https://github.com/FlyingJester/TurboSphere). It now matches my development machine's copy, so it also has the mostly restored (http://flyingjesterentertainment.webs.com/apps/blog/show/26424765-code-blocks-you-re-fired-) map engine. In addition, it no longer uses OpenGL immediate mode (although I doubt that is anything more than just aesthetic right now), more software primitives for surfaces, and the fix for drawTextBox crashing--I hope. That one's like a bad penny.
Title: Re: TurboSphere
Post by: Flying Jester on June 03, 2013, 07:27:07 pm
I'm taking a little break from the Map Engine to work on an audio plugin.

I was thinking that I would use irrKlang for the audio. I know it has a nice API and also then the editor could share the library with TurboSphere. But I don't think that there exists a 64-bit Linux version of the library, which is (for me personally, as a user of the engine I'm making) a deal breaker. I may end up making an irrKlang library later, either just for Windows and 32-bit Linux, or when 64-bit Linux binaries are released (which is quite possible, the irrKlang developers seem very active in Linux development).

But for now, I'm tentatively going with Bass (http://www.un4seen.com/), which has a very similar license to irrKlang, and also has the added bonus of supporting MIDI much more easily. I also generally rather like how the API looks. Bass also offers the ability to use many other file types through plugins (so does irrKlang, but not as many). I'm not sure if irrKlang can, but I know Bass can also encode audio using any installed system codec, and operate directly on CDs (which dovetails nicely with SDL's CD-drive functionality...at least in theory). I would someday like there to be tools in TurboSphere to run games and/or the engine off of CDs. Properly streaming sound from the game CD might not really be necessary, but it sounds like a nice way to take advantage of the CD media.

I'll be back to the Map Engine relatively soon. But the thing is, there's so much to it that a lot of the code I have to write doesn't tangibly change the functionality right away, and I need to write a lot of such code to get most things working since so many things rely on so many other things.
Title: Re: TurboSphere
Post by: N E O on June 03, 2013, 07:37:51 pm
BASS powers mudlord & kode54's User-Mode Soundfont Driver (https://github.com/mudlord/BASSMIDI-Driver), a software-only replacement for the Creative Soundfont Loader letting Windows users without SF-capable sound cards use Soundfonts for MIDI playback.

While the BASS license itself is significantly more restrictive than irrKlang's, it's a fantastic choice for temporary use.
Title: Re: TurboSphere
Post by: Flying Jester on June 03, 2013, 08:52:58 pm
I don't think it's too much more restrictive, but I haven't paid too much attention to how it works out for commercial use. I'm pretty much assuming that if anyone really wants to go commercial, they should write their own libogg based plugin and just use that (or something like that). It's easy to make a simple but inflexible audio plugin (I've done it before). It's hard to make a one that can measure up to how Sphere 1.x can handle audio.

And also either way, the sound plugin is a lot more freeform than the some of the others. An irrKlang plugin, a Bass plugin, an SDL_Mixer plugin, or even an Audiere plugin could all be drop in replacements for each other in most cases, unlike the plugins that use graphics which have to agree with the main graphics plugin.

I would definitely like to use irrKlang, but until there is a 64-bit Linux version I won't, because 99% of my development is on 64-bit Linux. And plus, I think that if you want to use MIDI files, you'll get to a lot sooner with a Bass plugin than you will with an irrKlang (combined with a different midi library) plugin.

Edit: Speaking of, I've got the new audioBASS plugin working just as well as the old, pre-plugin sound support did...except now it also supports MIDIs.

Edit Edit: And right off we also have external soundfont support, as well as per-midi font changing (including multiple fonts running at once, and combining fonts on the same sound). Not that that last bit is hugely important, but it's great that it is possible now. I'm looking for a good font to include with TurboSphere (maybe Chorium? I haven't checked the license for it yet).
Title: Re: TurboSphere
Post by: Flying Jester on June 06, 2013, 07:52:00 pm
I've updated TurboSphere on github (https://github.com/FlyingJester/TurboSphere) to include the BASS audio plugin (and updated the included docs accordingly). It includes the ability to use external soundfonts, and includes a demo font (Hollywood, which is terrible but tiny, only ~512k. But you can change the system.ini file, and replace it with anything you want). You can also play WAVs, OGGs, theoretically mod-music, and possibly a lot of other files (BASS can supposedly try to use system codecs if it can't open a file, but I haven't tested that). I also plan on adding general BASS plugin loading.

I don't have it working so that you can use the machine's system soundfont, because I can't test that yet. Fedora doesn't include a default MIDI soundfont, I haven't installed the stuff to make MIDI work at all on my Arch machine, and my Windows machine died recently (I'll cast res on it soon, once my mana recharges--aka, when I get enough money to buy the parts).

Right now all that has been added is:
Code: [Select]

new Sound(filename) //Sound
sound.Play([loop]);
sound.Stop();


This update also includes some of the new (actually restored after losing them when I switched to OpenGL) software surface primitives:

Code: [Select]

surface.Line(x1, y1, x2, y1, color);
surface.Rectangle(x, y, w, h, color);
surface.FilledCircle(x, y, r, color);
surface.OutlinedCircle(x, y, r, color);


I also slightly improved engine startup time and game loading time...which is probably overshadowed by now loading BASS and BASSmidi at startup.

To test the sounds with the new startup game, press M to play an ogg and N to switch and play a midi.
Title: Re: TurboSphere
Post by: Flying Jester on June 07, 2013, 10:28:24 pm
TurboSphere 0.3.0 has been released! (http://sourceforge.net/projects/turbosphere/files/Bin-Win32/0.3.0/)

This release doesn't need or worry about OpenGL extensions, so it should work on Intel GPUs now. It lacks the Map Engine (there's not much to look at yet), but includes all the new graphics improvements and the sound plugin.

Considering how little there is to do to finish the sound plugin (somehow I thought there were a lot more sound functions in Sphere), I may just finish it. And have a completely finished plugin for once!

I'll update the TS functions page on the wiki sometime soon. It needs it badly.

I've fixed up the source (as posted as 0.3.0 on sourceforge) quite significantly for non-Linux platforms, especially Windows. It should compile well from the scons file on Windows (this is the first win32 release I've made using scons to build it). It should also no longer have the heap corruption issues it used to have with font.drawTextBox on Windows and OS X. I may or may not make a win64 release soon...I need to get the win 7.1 SDK on my newly resurrected Windows machine.
Title: Re: TurboSphere
Post by: N E O on June 07, 2013, 10:40:48 pm
Visit the Sound API page (http://wiki.spheredev.org/API:Sound) to see just how many functions the Sphere Sound object really has ;) (note: does not include SoundEffect or SFXR APIs)
Title: Re: TurboSphere
Post by: Flying Jester on June 08, 2013, 12:03:31 am
Yeah, that's just what I was looking at.

Considering that a lot of the functions are shared (in some manner in the code, at least) with sound effects, that's not really a whole lot for a plugin (compared to, say, the graphics plugin or the map engine). And I'm also considering how much easier it is to make this now that I'm using something other than SDL_mixer.

I kind of plan on putting the SFXR into a separate plugin, or as a sub plugin to the sound plugin. Partially because if and when I switch to irrKlang (or something else, even, for instance if I or someone else wanted to make a totally free, even for commercial use sound plugin), then the SFXR plugin will be less affected.
Title: Re: TurboSphere
Post by: N E O on June 08, 2013, 03:01:07 pm
That's fine; I wouldn't have expected the SFXR stuff to be built directly into the Sound stuff even on vanilla Sphere.
Title: Re: TurboSphere
Post by: Metallix on June 10, 2013, 03:17:25 am
Yay! Finally I can continue using turbosphere :) the new gfx functions and intel support was quite essential for me.
Title: Re: TurboSphere
Post by: Fat Cerberus on June 10, 2013, 08:56:03 am
I'll test it on my i7 later, but it crashes on startup on my desktop (shows a blue triangle first, then crashes).  Admittedly this thing isn't the fastest thing ever, it's only a cheap AMD dual-core processor, but figured I'd throw it out there anyway in case you knew what was causing it.

Graphics card: Radeon HD 7340 (CPU graphics, chip is AMD E2-1800)
Title: Re: TurboSphere
Post by: Flying Jester on June 10, 2013, 06:10:19 pm
@Lord English: If you run it through a command prompt, does it tell you anything? I doubt it will be too useful to do that on Windows, but it might. Otherwise I'd need to try dissected startup scripts to see what is crashing it.

I'm working on 0.3.1 right now. I've been trying to get my old Breakout game to run on it. And as it turns out, there were only four functions missing, and a little problem with file reading.

Considering that most of my best games had to do with juggling or hand-ball, I think that the circle functions (OutlinedCircle, FilledCircle, and GradientCircle) are coming soon.
Title: Re: TurboSphere
Post by: Fat Cerberus on June 11, 2013, 03:11:12 pm
Results of running TurboSphere from PowerShell:

Code: [Select]
PS C:\users\bruce\desktop\turbosphere> .\turbosphere
Fixed plugins: 1
Fixed plugins as recorded: 1
Opening Game.
Opening SGM file startup/game.sgm
Opening main script.
The main script is startup/scripts/Jest_Main.js
[ConfigManager] Info: Successfully opened script startup/scripts/Jest_Main.js
Getting number of plugins.
Getting number of plugins.
Plugin audioBASS.dll is open.
[audioBASS] Info: loaded and initialized BASS.
[audioBASS] Info: Initialized BASSmidi.
Plugin bmpfontGL.dll is open.
Plugin getkeystring.dll is open.
Plugin inputSDL.dll is open.
[inputSDL] Info: Event state setup succeeded.
Plugin scriptfs.dll is open.
Plugin SDL_GL.dll is open.
[SDL_GL] Video mode setup succeeded.
[SDL_GL] Info: Using OpenGL version 4.2.11764 Compatibility Profile Context
Plugin ttffontGL.dll is open.
Plugin windowstyleGL.dll is open.
startup/fonts/DejaVuSans.ttf
PS C:\users\bruce\desktop\turbosphere> _
Title: Re: TurboSphere
Post by: Flying Jester on June 11, 2013, 06:14:30 pm
Huh...normally you get more info about crashes than that. At least a segfault message or something.

So it crashes when you see the first screen. Is it before or after you hit a key to go to the next screen?

EDIT: Expect graphics scaling in 0.3.1 (1x through 16x guaranteed). I may even get hq2x and/or Eagle2x working too, but no promises there.
Title: Re: TurboSphere
Post by: Fat Cerberus on June 11, 2013, 09:56:47 pm
I see a blue triangle after running it, then it immediately crashes.  No keypresses or anything.

Edit: Works like a charm on my i7, though. :)
Title: Re: TurboSphere
Post by: Flying Jester on June 13, 2013, 02:37:25 am
Interesting...

What does the command line say when you run it on your i7 machine?
Title: Re: TurboSphere
Post by: Fat Cerberus on June 13, 2013, 10:30:37 am
A bunch of stuff, I don't remember it all now and I'm not on that machine at the moment.  I remember more and more stuff being added to the console output while the game was running, though, JS events or something.  I'll have to check it out again later.

Edit: Just pulled out my laptop and tried it, output doesn't appear to be helpful.

Code: [Select]
Windows PowerShell
Copyright (C) 2012 Microsoft Corporation. All rights reserved.

PS C:\Users\Bruce> cd Desktop/TurboSphere
PS C:\Users\Bruce\Desktop\TurboSphere> ./turbosphere
Fixed plugins: 1
Fixed plugins as recorded: 1
Opening Game.
Opening SGM file startup/game.sgm
Opening main script.
The main script is startup/scripts/Jest_Main.js
[ConfigManager] Info: Successfully opened script startup/scripts/Jest_Main.js
Getting number of plugins.
Getting number of plugins.
Plugin audioBASS.dll is open.
[audioBASS] Info: loaded and initialized BASS.
[audioBASS] Info: Initialized BASSmidi.
Plugin bmpfontGL.dll is open.
Plugin getkeystring.dll is open.
Plugin inputSDL.dll is open.
[inputSDL] Info: Event state setup succeeded.
Plugin scriptfs.dll is open.
Plugin SDL_GL.dll is open.
[SDL_GL] Video mode setup succeeded.
[SDL_GL] Info: Using OpenGL version 4.0.0 - Build 9.17.10.2875
Plugin ttffontGL.dll is open.
Plugin windowstyleGL.dll is open.
startup/fonts/DejaVuSans.ttf
JS Result will be passed.
32
[ttffontGL] DrawText Info: Binding string Do a barrel roll! With TTF Fonts! to cache slot 0.
PS C:\Users\Bruce\Desktop\TurboSphere> _


The "JS Result will be passed" line didn't appear until I pressed a key on the blue triangle screen, the line immediately before it mentioning DejaVuSans.ttf is the last line displayed before it crashes on the AMD machine.  I don't think the console output is going to be too helpful here.  Honestly, if I could compile TurboSphere with VS, I could probably find the bug myself using the VS debugger, but since I don't think I can, I guess this one's up to you. :)
Title: Re: TurboSphere
Post by: Flying Jester on June 13, 2013, 02:34:41 pm
You probably could compile it using MSVC. The source for 0.3.0 on SourceForge has a Scons file that works with MSVC quite well (that's how I compiled it).

Considering that it crashes before it gets a chance to call the game function (the 'JS result will (or will not) be passed' means that it fully evaluated a script, in this case the main script without calling game(). So it's something else in the script, and there's not too much else. It pretty much has to be GetKey or maybe FlipScreen.

Out of curiosity, your AMD laptop doesn't have a bunch of crazy media buttons or something like that, does it? Or a button to toggle on and off the touchpad?
Title: Re: TurboSphere
Post by: Fat Cerberus on June 13, 2013, 03:00:14 pm
The AMD machine is a desktop, not a laptop.  Well, more specifically an all-in-one.  It does have an MS ergonomic keyboard attached with a bunch of media/other misc. keys, though...
Title: Re: TurboSphere
Post by: Flying Jester on June 13, 2013, 09:57:47 pm
Huh. I do know that on my older Toshiba laptop, when I have the touchpad toggled off using a button below the keyboard, it makes TurboSphere hang until it's toggled back on for several key functions (SDL reads some ridiculous scan code for that button, and for some reason there is blocking while the pad is toggled off). A similar thing happens when I mute my laptop using the media key, SDL constantly reads a key being pressed, although in that case no scan code is generated. In both cases, I have no idea how to fix it. I'm just letting it go until I move to SDL 2.0, or it becomes enough of a problem that I'll have to figure it out.

You could try the script with all the GetKey()'s commented out. That's the only thing between the screen being drawn and the script ending. Otherwise it might actually be something in game(), but I haven't seen that kind of thing cause this kind of problem before (something terrible in game() causing a crash before game() is called).
Title: Re: TurboSphere
Post by: Flying Jester on June 14, 2013, 10:48:06 pm
I've updated http://wiki.spheredev.org/User:Flying_Jester/TurboSphere_API (http://wiki.spheredev.org/User:Flying_Jester/TurboSphere_API), and now it is (as far as I know) fully up to date for TurboSphere 0.3.0, except for the sound functions.

EDIT: 0.3.0's sound API is now documented as well. (http://wiki.spheredev.org/User:Flying_Jester/TurboSphere_API#AudioBass)
Title: Re: TurboSphere
Post by: Flying Jester on June 19, 2013, 09:36:10 pm
I've updated TurboSphere to use V8 3.19. I thought this would be a much more difficult transition, since persistent handles are going away. And make no mistake, the only reason I could make the transition with only a dozen lines of code difference or so is because there are a couple of hacks in v8 now so that it can behave more like 3.17 until persistent handles are finally gone and regular handles are changed to take up the slack.

I'm not looking forward to that day. But I am very happy that I've worked out the plugin system the way I have. The changes to all the plugins to make them work with 3.19 were almost completely in plugin.h.

Title: Re: TurboSphere
Post by: Radnen on June 20, 2013, 06:20:57 am
That's good to hear. Would it mean making easier plugin creation?
Title: Re: TurboSphere
Post by: Flying Jester on June 20, 2013, 05:53:47 pm
Well, plugins aren't very hard to make. At least I don't think so. I do know that it is a lot easier with plugins than it used to be before plugins.

The good news is that (partially accidentally) almost everything that seems to change in V8 is covered by the plugin making tools in the source. One change in the source, and the plugins just need to be recompiled (which would have to happen anyway, they are linked to V8).

Also, TurboSphere is beginning to run some games with no modifications to the scripts! Which also shows how the old editor does not create spec TGA files (I don't think TGA's can actually have alpha information).
Title: Re: TurboSphere
Post by: DaVince on June 22, 2013, 07:44:09 am
Awesome.

Hey FJ, when you get to the map engine, I have a request... Could you add functionality to grab a defined area of the entire map as an image/surface? In current Sphere, grabbing the map is limited to just the visible area on the screen (unless you use some kind of hack).
Title: Re: TurboSphere
Post by: ninjasword on June 22, 2013, 11:07:11 am

In current Sphere, grabbing the map is limited to just the visible area on the screen (unless you use some kind of hack).


1. open the editor,
2. create a new project,
3. create a new map that is bigger than ur current screen resolution (tiles * tile size),
4. fill the default map layer with a recognizable pattern,
5. right click a layer on the left hand side of the map editor,
6. mouse over the context menu option: "Export"
7. select the option "Export all visible layers"
8. ???
9. profit:

http://i828.photobucket.com/albums/zz207/the_gimis/programmins/testMapPic.png

edit: added quote
Title: Re: TurboSphere
Post by: Fat Cerberus on June 22, 2013, 11:15:07 am
@ninjasword:
I think he meant doing it programmatically, i.e. in-game. To implement special effects or whatnot.
Title: Re: TurboSphere
Post by: DaVince on June 22, 2013, 11:35:47 am
Yes, programatically. Exporting the map image (which I already knew btw ;)) has disadvantages (re-export every time you change the map, have to do it for every map, the project's size gets bigger, you lose animations/the fact that you are running the map engine live...)

Being able to do it on-the-fly means we could create proper Mode7 effects, or play around with the rendered image in some other way.
Title: Re: TurboSphere
Post by: Radnen on June 22, 2013, 03:12:02 pm
There is a way through code, but it can get slow to do it per-frame, and it only works on a single layer, but can be used in a loop to capture the whole map:

Code: (javascript) [Select]

var surfaces = [];
var c = CreateColor(0, 0, 0, 180);
var i = GetNumTiles();

while (i--) {
var surf = GetTileImage(i).createSurface();
surfaces.unshift(surf);
}

var width = GetLayerWidth(layer);
var height = GetLayerHeight(layer);
var l_surf = CreateSurface(width << 4, height << 4, Colors.clear);

for (var y = 0; y < height; y++) {
for (var x = 0; x < width; x++) {
l_surf.blitSurface(surfaces[GetTile(x, y, layer)], x << 4, y << 4);
}
}

return l_surf;


A version exposed through the API would be much, much faster, I hope.

FJ: If you do start the map engine, make it so you can string them together. I've always wanted to have free-form maps like in Pokemon, but cannot do so in the current Sphere. Make it so you can string maps together!
Title: Re: TurboSphere
Post by: DaVince on June 22, 2013, 05:13:13 pm
Radnen: yup, that's the kind of solution I had in mind myself, and also the of thing I'd call hacky. It's strange it's not an engine feature, really.

And I definitely agree on the stringing maps together! Adding 50% to each map that's identical to the next map is hell to do. :P
Title: Re: TurboSphere
Post by: Flying Jester on June 22, 2013, 05:43:09 pm
I too have thought of having maps strung together. It's not difficult at all conceptually, and I do want the ability to do it (easily and cleanly, not through work-arounds in script).

Also, it would be quite simple to add a function that grabbed a portion of the map as an image or surface. Definitely as fast as actually drawing the map, definitely faster than from script.
Title: Re: TurboSphere
Post by: DaVince on June 22, 2013, 05:51:45 pm
That's great great great to hear!

By the way... The website is getting a bit old. Maybe just remove the release file from there and make it link to SF too? (While keeping the detailed description (btw update it!), of course... And I think it'd be a good idea to link to your blog from there too ;))
Title: Re: TurboSphere
Post by: Flying Jester on June 22, 2013, 09:22:51 pm
Oh yeah...I completely forgot about the tengu TurboSphere page! I actually have a much better site built that I meant to put up there, but I forgot!
Title: Re: TurboSphere
Post by: Flying Jester on June 27, 2013, 09:26:57 pm
I'm very almost done with 0.3.1. It includes much more fleshed out Sound and SoundEffect objects; Filled, Outlined, and Gradient Circles (with the option for antialiasing); a more robust File object; a better written sound plugin (which only really matters if you like reading the source code...), and a slightly less broken (but still not really usable) map engine plugin, graphics scaling (not perfectly scaled compared to unscaled, but I looked at Sphere's Unix GL driver, and it doesn't look like it uses a much different scaling method).

Pretty much now I need to fix MSVC so that I can actually compile. It seems that it breaks every time I need to install a new .NET redistributable! And I can't use MingW just yet, since they are still making it possible to build V8 as a dynamic library using MingW. I may give Clang a go (TS compiles with it in Linux, I know, and I assume the XCode build that almost worked right for CasioTone used LLVM).
Title: Re: TurboSphere
Post by: Flying Jester on July 07, 2013, 10:10:23 pm
I've uploaded the source for TS 0.3.1 (https://github.com/FlyingJester/TurboSphere). As soon as I get this terrible MSVC version conflict issue resolved on my laptop I'll be able to upload Win32 binaries. But after a close call with my HDD blowing up, I decided to upload what I have right now for safety's sake.

The biggest new thing is that 0.3.1 uses SDL 2.0 and V8 3.19. It was a much less painful process to switch over to SDL 2.0 than I thought it would be, but that's probably helped by the switch to OpenGL, since I was using SDL surfaces for what they really should have been used for at that point.

In addition, circles have been added as graphics primitives, all the old software primitives are now available for use on surfaces (and I fixed a very old bug where lines would not render properly onto surfaces when both X coordinates or both Y coordinates were identical), graphics plugin scaling is supported, the sound plugin is much more fleshed out (almost done, in fact), and several bugs with the file object have been resolved.

Using SDL 2.0 also seems to have doubled the base framerate, there's much less housekeeping overhead now. It should also help with some of the OpenGL compatibility issues.

Interestingly, the all of TurboSphere compiles (with very slight doctoring) using Google's NDK (which is an almost complete C/C++ toolchain for Android), it seems almost all the OpenGL code I wrote is also valid OpenGL ES, and SDL 2.0 has Android support. So there's not really a whole lot stopping this from working on Android, other than the fact that I have no idea exactly what I'm doing with that and just bumbling around with the NDK and Android SDK.
Title: Re: TurboSphere
Post by: Radnen on July 07, 2013, 10:26:34 pm
What's the framerate of the code here?
https://github.com/Radnen/sphere-sfml/blob/master/Engine/Engine/Startup/scripts/test.js

Just curious.
Title: Re: TurboSphere
Post by: N E O on July 08, 2013, 04:30:44 pm
Re future Android support - use RetroArch as an example for coming up with an Android GUI for loading projects into the engine.
Title: Re: TurboSphere
Post by: Flying Jester on July 09, 2013, 08:56:04 pm

What's the framerate of the code here?
https://github.com/Radnen/sphere-sfml/blob/master/Engine/Engine/Startup/scripts/test.js

Just curious.


Huh...not sure. It segfaults on that transformBlit! Which is weird, I've been transformBlitting all over the place and haven't seen that happen yet. I'll see if I can't find out why.

edit: Fixed that problem. Previously, you could try and load an image if a file existed with the same name but a different extension. Then trying to do anything with the image would cause a segfault.

I get (I think, this is hard to say for certain at this kind of rate) 125000 FPS average. <--that's running in Valgrind. I get 400,000 FPS normally.

Why, what kind of FPS do you get?
Title: Re: TurboSphere
Post by: Radnen on July 09, 2013, 09:51:15 pm
Not sure now, but damn nowhere near 400000, that can't be right. My tests had been around 11,000 fps, which is consistently 2,000 higher than SphereGL.

I get 9500 FPS on that demo, in Sphere I get 8600 FPS.

I can't even run a single test in your code without a segfault or some other strange glitch. May I consider unit testing? :P

Okay in v0.1.6 I get 3700FPS.
Title: Re: TurboSphere
Post by: Flying Jester on July 10, 2013, 04:53:52 am

Not sure now, but damn nowhere near 400000, that can't be right. My tests had been around 11,000 fps, which is consistently 2,000 higher than SphereGL.

I get 9500 FPS on that demo, in Sphere I get 8600 FPS.

I can't even run a single test in your code without a segfault or some other strange glitch. May I consider unit testing? :P


I do test. And I kind of don't, too. My tests are actually pretty limited, and I don't really end up running other folks' scripts very often. If you write scripts the way I do, it runs quite well, I guess. And it doesn't help that this is a pretty new graphics plugin--first to GL (which I can only test on NVidia cards), and now to SDL 2.0. No doubt there are a lot of new bugs, and a lot of old ones showing up as well.

Until I get the map engine going, it's just kind of up to me to write most of the tests it seems, since most games I could test need the map engine. I've been testing my old games, since I rarely used the map engine...but I wrote those. I end up fixing mostly problems that I have.

If you hit a crash, I'd really like to know what you did to make it happen!

Yeah, nevermind that FPS figure. I did the math wrong.

It's 1600 on my laptop. But I only get 1500 with SphereGL on my laptop (which is getting very old). So it sounds pretty comparable.
Title: Re: TurboSphere
Post by: Radnen on July 10, 2013, 05:04:54 am
I'm waiting for a new version. I wonder what exciting changes you made! :)

One version on my computer segfaults each time I enter it, but the v0.1.6 one worked.
Title: Re: TurboSphere
Post by: DaVince on July 14, 2013, 10:19:21 pm
Finally tried out TS 0.3.0 today (on Windows 7)... It displays some blue rectangles. When I press enter twice I get to the animated testing environment with the music and everything, but then it crashes after a few seconds, showing Windows' "turbosphere.exe no longer works" dialog. The terminal that opens in the background also locks up, but doesn't seem to be showing any special error messages...

This is on a system that has an Intel Sandybridge 3000 GPU and also an NVidia GT540M (with latest drivers). Not entirely sure which one it used for the test, and if it's the cause of the crash.
Title: Re: TurboSphere
Post by: Fat Cerberus on July 14, 2013, 10:31:20 pm

This is on a system that has an Intel Sandybridge 3000 GPU and also an NVidia GT540M (with latest drivers). Not entirely sure which one it used for the test, and if it's the cause of the crash.


If you have a dedicated nVidia GPU, it would have used that.  The way Sandy Bridge is set up, the on-chip GPU is automatically disabled if you have a dedicated graphics card, they can't be used at the same time.  I don't remember if they changed that with Ivy Bridge...
Title: Re: TurboSphere
Post by: Flying Jester on July 14, 2013, 11:58:15 pm
Well, that's not a great sign, particularly since I've been working on something all new for the graphics plugin (that I probably won't actually end up using!).

You can try commenting out the fn.drawTextBox, that used to (still does?) have some problems on Windows, and usually does not cause immediate crashes, but crashes after a few seconds. I've actually fixed a couple of possible issues with it in what will become 0.3.1.

It does kind of make me wonder that you are getting an OpenGL 4 context, too. I've never tested that, although I expect that if TS 0.3.0 works in the slightest that's not the issue--it would crash pretty much at startup.


In other news, I may or may not end up using it, but I've written a new version of the graphics plugin that runs the software drawing routines in a separate thread. It runs pretty much identically fast as the non-threaded version most of the time, but when I added the functions to draw Majestic 2D maps (and didn't call any functions that would force the thread to finish operations for until it was already done), it removed the lag to do the internal blits completely (about 200ms--though that was a very low stress test).

I've got it prioritizing drawing operations for when a surface is properly needed (such as a blit tot he screen or a call to getPixel), correctly walking operation dependencies, and varying the CPU time ceded based on how many operations remain to do (so that it doesn't burn up my dual core, mostly). I'm surprised how easy it was to do it, but I must admit I worry about an increase in complex and hard to diagnose bugs it would introduce--I've already been a bit stumped by a crash it causes in the Nvidia glx core, although the crash is very deterministic--I need to find out what exactly is causing it.
Title: Re: TurboSphere
Post by: DaVince on July 15, 2013, 06:22:03 am


This is on a system that has an Intel Sandybridge 3000 GPU and also an NVidia GT540M (with latest drivers). Not entirely sure which one it used for the test, and if it's the cause of the crash.


If you have a dedicated nVidia GPU, it would have used that.  The way Sandy Bridge is set up, the on-chip GPU is automatically disabled if you have a dedicated graphics card, they can't be used at the same time.  I don't remember if they changed that with Ivy Bridge...

Well, no... Both run, the display output goes through the Intel GPU, and NVidia's output is actually passed to that. The laptop can switch between which GPU to use for every piece of software I run. It's an option in the software, even. NVidia calls this Optimus technology.

The only issue is, I updated my drivers yesterday, and ever since then the option "run with GPU..." has disappeared from my right-click context menu. I think it was running through the NVidia GPU since I set that one as the default one to use, but I didn't know if that setting was still the same after the update. That's why I said "not sure". (Note: it was still set to use the NVidia graphics.)

I'll play around with the graphics settings for a bit.

EDIT: whelp! It crashes, regardless of whether I set the preferred GPU to NVidia or integrated graphics.
Title: Re: TurboSphere
Post by: DaVince on July 15, 2013, 07:22:31 am
FJ, try the attached test. For me, the following happens:

I see the text and press on a key.
I press 1.
It crashes.

Maybe it has issues loading some image formats? Since that's the only difference in the screens once you press 1, 2 or 3... In any case, it doesn't seem very graphic driver related since it crashes with both the Intel and NVidia card. (Though it hangs for a second before it crashes on the NVidia one while it's instant on the Intel one.)

On another note, TS crashes if I drag game.sgm onto the executable. I need to make it a startup game before it will run.
Title: Re: TurboSphere
Post by: Fat Cerberus on July 15, 2013, 10:13:10 am
@DaVince
That's weird, I seem to remember reading that you couldn't use the SB GPU at all if you had dedicated graphics hardware, unless you disabled the latter in BIOS.  Maybe I misread and that only applies to the QuickSync feature (HW accelerated video transcoding).  Now you have me curious, I'll have to go look it up again...
Title: Re: TurboSphere
Post by: DaVince on July 15, 2013, 10:16:59 am
Well, I have a notebook. It seems nonsensical to me to have discrete graphics AND dedicated graphics when one is specifically intended to save on battery and the other one is specifically intended for when you need the GPU power. Although you *can* disable the discrete graphics in the BIOS if it supports it (mine doesn't).

Also, I've successfully been selecting and switching between which GPU to use for what software. But in TS's case, it doesn't seem to run on either.
Title: Re: TurboSphere
Post by: Fat Cerberus on July 15, 2013, 10:29:06 am
Huh, just looked it up and there are conflicting reports; apparently it depends on the motherboard.  Some will disable the Intel graphics when a card is plugged in, others will even allow them to work simultaneously, i.e. for multi-monitors.  Good to know!
Title: Re: TurboSphere
Post by: Flying Jester on July 15, 2013, 09:46:54 pm

On another note, TS crashes if I drag game.sgm onto the executable. I need to make it a startup game before it will run.


I designed it to work with the editor--I didn't realize Sphere took the SGM. TurboSphere takes a directory, which is what the old editor passes the engine. I can easily add the ability to start with an SGM as well, though.

That test you posted doesn't segfault on Linux--GetSystemWindowStyle has been known to crash on Windows. But on Linux, it works, and there's another issue with WindowStyle.drawWindow that I'll take a look at. It complains that argument 2 is not an integer...which it is. Very interesting. EDIT: Fixed that issue.

Also, you are using LoadImage. That is not defined by default in TurboSphere--the image objects are properly undefined in your script. LoadImage has been replaced with new Image()--same thing with all the other Sphere objects, too.

Did you try removing the drawTextBox from the Jest_Main.js script in the included test game? I doubt it is the GPU, I've actually tested 0.3.0 with an AMD integrated GPU, and the only strange thing was that the affine transformations went form opposite corners--which will, whether it is properly a bug or not, be fixed in 0.3.1. And I've removed all the stuff that used to cause the problem of only NVidia and theoretically certain ATI cards being able to run TS.

You can try running the...performance monitor? It's something in the 'computer--*right click*->manage' menu, and you can see what processes run on what GPU--I can see which of my SLI cards runs TurboSphere from it.

A lot of these crashes seem to be things that happen more often or only in Windows. Which is not unexpected, I rarely use Windows, and so I don't debug on it very often. I'll need to have a look in Windows to get a lot of this sorted out.
Title: Re: TurboSphere
Post by: Flying Jester on July 17, 2013, 11:59:01 pm
Well, I wanted to get 0.3.1 out a while ago. But Visual Studio 2010 is horrendously broken on my Windows machine. It can't even uninstall itself, after a couple minutes of trying a dialog pops up that simply says 'The operation cannot be completed', same as if I try to run VCexpress. If I try and invoke it from the command line, it's like half of its standard library is simply gone. I then tried to use MSVC 2008, just in the interest of getting the code compiled at all. But that hit the snag that V8 can no longer be compiled with it, and unlike before with the Scons file for V8 you can't compile V8 without having a fully operational Visual Studio--unless you have the professional version which contains the tools to compile an sln file from the command line.

So I managed to get V8 3.19 compiled with MingW (which wasn't too bad, actually) using a friend's machine (which has MingW working, I can never seem to get it running right), ran DLLtool on it to get an MSVC compatible .lib file.

Then I learned that whatever is breaking my MSVC 2010's standard library is doing the same to 2008 (a little, anyway). I no longer have (or somehow can't use) the header files that contain the functions I need for the plugin interface or to compile T5 on Windows! And neither the 2008 or 2010 installer seems to be able to repair this.

So I guess I'll give in and also install MSVC 2012--if it can install on my machine. This is a good example of why I rarely make binary releases for Windows.
Title: Re: TurboSphere
Post by: Radnen on July 18, 2013, 01:43:12 am
Honestly, you have got to be prodding with the wrong sticks to screw up a MSVC install. I could try installing it with my vs2012 if you want. I just need to know the steps: a 'compilers guide'.

BTW, how much of the engine do you have finished?
Title: Re: TurboSphere
Post by: Flying Jester on July 18, 2013, 04:25:00 am
I don't think it is MSVC that properly broke, I think it has something to do with .NET (although I can't explain the missing or broken header files that way). This started happening after I had to re-install .NET for a game.

The 'compilers guide' is pretty simple--but I need to upload the source for 0.3.1 first (I can get on that tomorrow, the internet is too slow where I am now to do so), the GitHub source contains most but not all the changes so far.

1. Compile V8 using their instructions (I include instructions for downloading it using SVN, just be sure the branch is 3.19) and put the headers in the TurboSphere root. The command to get the right version is
Code: [Select]
svn checkout http://v8.googlecode.com/svn/branches/3.19 v8

2. Get SDL2, SDL2_image, and SDL2_ttf. Put the headers for all three in a folder called 'SDL' at the TurboSphere root.
3. Get Bass and BassMidi, put the headers in the 'plugins/audioBASS/include' directory.
4. Using the MSVC command prompt, enter
Code: [Select]
scons --install_libs=y --build_plugins=all buildplugins libinstall


All libraries, DLL and LIB files, go in the TurboSphere root directory. Bear in mind this might not work right out of the gate, I haven't tested the source for 0.3.1 on Windows yet.

As far as how much is finished, without going through everything and finding hard numbers:

Graphics: 80%
Audio: 80%
File I/O: 50%
WindowStyles: 90%
BMPFonts: 80%
TTFFonts: 70%
Map Engine: 5%
Title: Re: TurboSphere
Post by: Radnen on July 18, 2013, 05:01:12 am
Hmm, I don't want to sound rude, but my theory is correct, C#/Mono type languages do seem to make you more productive. I have more implemented than you for having put less time into it. That includes time spent making test cases for the API as-I-go. These tests are simple and pretty much just run to make sure I have 2 things:

1. The item has been implemented. ie: I'm not missing something small. When say, 'fonts' are 100% done I mean they are 100% done. But not necessarily 100% tested.
2. For some items, to test if the item works. I could not possibly do this for everything. But for some things I just do basic get/set tests and sanity checks. Most common one: does toString() return "[object correct_name_here]"? Why I care about that above so many other things I can't really say...

But I have to say, the C++/SDL/v8 learning curve must have been tremendous! And you'll have a native cross platform app that may indeed be faster than mine when all is said and done. 0_0 What I hope to achieve at the very least is a standard to compare engines against. Perhaps not for speed or for cross-platform-ness, but at least as an implementation guide. I think C# is way easier to read (and write, I mean header files? Come on!) than C++, but then again it may just be due to the fact I spend so much time in it.

If Jurassic were mono-compatible... Ah that'd be the day. It's not too far off. It's just missing a single feature that the mono team hasn't implemented yet. It's really not an OS limitation but a mono-implementation one. <sigh>.
Title: Re: TurboSphere
Post by: Fat Cerberus on July 18, 2013, 11:56:04 am
@Radnen
Which feature is missing for Mono compatibility, out of curiosity?

But yeah, I have to agree with you here, C# is much better for productivity. I was practically raised on C++ but every time I try to go back now and write a native app I give up in disgust.  Mostly due to the unwieldy header file management you have to do, it's just unbelievably clunky to work with.
Title: Re: TurboSphere
Post by: Radnen on July 18, 2013, 01:02:41 pm

@Radnen
Which feature is missing for Mono compatibility, out of curiosity?


System.Reflection.Emit.DynamicMethod dynamicMethod
Title: Re: TurboSphere
Post by: Flying Jester on July 18, 2013, 10:12:56 pm

Hmm, I don't want to sound rude, but my theory is correct, C#/Mono type languages do seem to make you more productive. I have more implemented than you for having put less time into it.


No, I totally agree. I knew there were easier ways to do this that would also get done faster. I chose C/C++ because of the promise of ultimate speed and efficiency. But I always knew the price was that it would take far more time and effort to achieve that.

My reasoning (that I still stand behind, in the long run) is that I should spend more time making the engine, since that extra effort will eventually be seen as higher performance and lower resource requirements.


But I have to say, the C++/SDL/v8 learning curve must have been tremendous! And you'll have a native cross platform app that may indeed be faster than mine when all is said and done. 0_0 What I hope to achieve at the very least is a standard to compare engines against. Perhaps not for speed or for cross-platform-ness, but at least as an implementation guide.


Yes, about the first six months of making TurboSphere were just teaching myself how to program in C++ (and relearn what little C I knew before)--and V8 was a jump into the deep end for C++!

I have no doubt that a C# based engine will be easier to make, in the early stages be far less buggy than a native-code engine, and be finished sooner. But I didn't (and still don't completely) trust C#, as a Microsoft language. I was burned when the old VB was killed off, I was burned when Delphi became what it is now. It's nothing against the languages--and it seems like C# is certainly here to stay. It just feels odd to me to write anything in .NET. But that is just a personal reservation.


But yeah, I have to agree with you here, C# is much better for productivity. I was practically raised on C++ but every time I try to go back now and write a native app I give up in disgust.  Mostly due to the unwieldy header file management you have to do, it's just unbelievably clunky to work with.


Mostly I feel this way about MSVC and .lib files, or exp or def...I'm not even sure why there are three choices! I wish it was like GCC with fully PIC and fully public symbols. And to be fair, there are many things I really wouldn't want to use C or C++ for.
Title: Re: TurboSphere
Post by: Fat Cerberus on July 18, 2013, 10:30:28 pm

But I didn't (and still don't completely) trust C#, as a Microsoft language. I was burned when the old VB was killed off, I was burned when Delphi became what it is now. It's nothing against the languages--and it seems like C# is certainly here to stay. It just feels odd to me to write anything in .NET. But that is just a personal reservation.


To be fair with this though, C# has one advantage that neither of those other two can claim: It's standardized by ECMA, just as JS is.  It's not going anywhere.  Given that standardization, it's very difficult to call it a "Microsoft language" with a straight face, any more than you can call Blu-ray a "Sony format".
Title: Re: TurboSphere
Post by: Radnen on July 18, 2013, 10:32:03 pm


But I didn't (and still don't completely) trust C#, as a Microsoft language. I was burned when the old VB was killed off, I was burned when Delphi became what it is now. It's nothing against the languages--and it seems like C# is certainly here to stay. It just feels odd to me to write anything in .NET. But that is just a personal reservation.


To be fair with this though, C# has one advantage that neither of those other two can claim: It's standardized by ECMA, just as JS is.  It's not going anywhere.  Given that standardization, it's very difficult to call it a "Microsoft language" with a straight face, any more than you can call Blu-ray a "Sony format".


Hence: Mono. :)
Title: Re: TurboSphere
Post by: Flying Jester on July 18, 2013, 10:40:19 pm
OK, the Git source has been updated!

I haven't seen any serious show stopping bugs associated with the threaded graphics plugin, so I've included it and set it to the default (although the old graphics plugin is still included). It is faster sometimes (especially if you script with it in mind), and I've never seen it be slower.

The SDL2 libraries need are at:

SDL2 (http://www.libsdl.org/tmp/download-2.0.php)
SDL2_image (http://www.libsdl.org/tmp/SDL_image/)
SDL2_ttf (http://www.libsdl.org/tmp/SDL_ttf/)

The dev libraries for ttf and image should come with the dynamic library for freetype and the necessary format libraries, respectively.

I know C# has been standardized. That doesn't stop Microsoft (or anyone else, actually) from adding nonstandard extensions, even proprietary ones. And if Microsoft stopped supporting it (which seems less and less likely, granted) then there would be much less momentum for the language. I doubt it would die, but it may be a dark time.

At the same time, I think that Basic has also been standardized, and Delphi is (or was? I don't even know if it is still being developed or supported) a dialect of Pascal.

All that is just conjecture, and highly unlikely conjecture at that. That's not really why I don't use C#. I just don't prefer .NET. For things that are very unsuitable for native code I usually use Java because of how cross-platform it is.
Title: Re: TurboSphere
Post by: Radnen on July 18, 2013, 10:51:36 pm

For things that are very unsuitable for native code I usually use Java because of how cross-platform it is.


I agree with you there.

Will you have latest Windows binaries anytime soon?
Title: Re: TurboSphere
Post by: Flying Jester on July 19, 2013, 06:27:06 am

Will you have latest Windows binaries anytime soon?


Maybe. I've installed  MSVC 2010 on a friend's computer, maybe tomorrow or this weekend I'll get it compiled.
Title: Re: TurboSphere
Post by: Flying Jester on July 20, 2013, 09:37:46 pm
Alright, I actually fixed MSVC (through a lot of reinstalling .NET and redistributables).

And I see what you mean by TurboSphere being very crashy. It's not like that in Linux, it's pretty stable there. Which is kind of to be expected, I've spent a long time using only Linux and it seems that I've written a lot of code that is less MSVC friendly than I thought.

I see a couple major things I need to solve before I can make a new release for Windows. Image loading is broken, it actually crashes during an SDL_image call (the main one, actually, IMG_Load()). Considering that it works like a charm on Linux, and SDL2 is still in the RC stage, that one might not be my fault. I'll keep trying to figure it out anyway.

Second, and I knew this would be an issue, the map engine (which primarily consists of map-related resource loading functions) relied on being strongly linked to the graphics plugin. That works great with GCC-style libraries, since I just load the graphics plugin first and its symbols are all available for any plugins loaded after it. Not so with MSVC.
That's not MSVC's fault, either though. I've been thinking of making this work nicer, by DLopening the functions I need (or LoadLibrary'ing, as the case may be). That would be a better solution anyway.
Title: Re: TurboSphere
Post by: Flying Jester on July 21, 2013, 10:36:05 pm
Image loading issue fixed. It was my fault, I needed to update libpng, libtif, and libjpeg (my code for loading SDL_image was flawed, it did not test if we got the loading libraries we asked for, only if an internal error had occurred in IMG_Init).

I am now seeing the same crash as DaVince did with 0.3.0 (which works fine for me, strangely). Now I'm just trying to get a proper debug build compiled, I've never properly had to do that with Scons before.
Title: Re: TurboSphere
Post by: Flying Jester on July 24, 2013, 05:28:39 am
I've got it narrowed down a lot. I think it has to do with a bad matrix setup in OpenGL that is ignored by the NVidia OpenGL library, but causes problems with the Windows library.

But of course, there is a snag. My main dev machine, my laptop, is dying. I ordered the new hard drive it needs. Once I get that, I'll have to reinstall MSVC (which won't be nearly as bad as fixing a bad install!), and then I can be up and running again.

This is what I get for trusting six year old, frankensteined computers.
Title: Re: TurboSphere
Post by: Flying Jester on August 05, 2013, 07:08:41 pm
Well, I fixed my laptop with a new harddrive and shiny new Arch Linux. And lost my old windows disk.

I'm going to just continue developing in Linux until I get back to university at the end of the month, there I can use the internet on my desktop (which has Windows). It's exceedingly annoying to try and fix these problems with no internet--particularly since getting a new download of V8 alone will take several hours where I am now.

On the other hand, I'm working on making a binary distribution for Linux. I've managed to weed out a lot of dependencies from the libraries I use, and I think I can make a universal Linux binary set.
Title: Re: TurboSphere
Post by: Flying Jester on August 20, 2013, 10:41:59 pm
I've added the ability to point TurboSphere at an SGM instead of only a directory containing an SGM, and to change games while it is running. I've also added GetSystemTTFFont(), surface.triangle(), font.getImage(), and a lot more functions to access the map-related resources. I've also improved background surface operations (many more functions are now set up to operate in the background than before). Once I get Wine installed, I'll resume work on the map engine.
Title: Re: TurboSphere
Post by: Flying Jester on September 03, 2013, 09:01:51 pm
Fonts now use texture atlases for all drawing. I didn't do this before because it had little effect on my ancient NVidia laptop. But on my new AMD machine, it vastly improved performance of the test script. I'm excited to see how fast texture atlases plus texture caching in buffers will be!
Title: Re: TurboSphere
Post by: Radnen on September 03, 2013, 09:02:45 pm
Sweet, I'm happy you've added texture atlases. :)
Title: Re: TurboSphere
Post by: Fat Cerberus on September 03, 2013, 09:17:09 pm
Alright, I'll bite, this has been bugging me for a while... What exactly is a texture atlas?
Title: Re: TurboSphere
Post by: Radnen on September 03, 2013, 09:21:04 pm
It's two things:

1. A large texture usually in a power of 2 format (but for 2D games it does not matter).
2. A list of rectangular bounds for each image.

Drawing an image means drawing portions of the one large texture to screen. This is really fast since you are sending one texture to the GPU even if you draw hundreds of unique images from it. Imagine using it for each character in a font or image in a spriteset, or tileset... Now you see the speedup. ;)
Title: Re: TurboSphere
Post by: Fat Cerberus on September 03, 2013, 09:50:42 pm
Ah, so it's basically just a fancy word for a sprite sheet.  I guess it threw me off seeing the term "texture atlas" used for this concept, seeing as my very first exposure to sprite/tilesets was with RPG Maker, which uses the "single image tileset" as its native format (compared to, say, Sphere's formats which store the images independently).  So because of that when I see the word "tileset" or "spriteset" I automatically imagine it as a single image, without needing a separate term for it.
Title: Re: TurboSphere
Post by: Flying Jester on September 04, 2013, 01:22:38 am
Apparently a lot of what I am doing is done in software for 8000m series NVidia cards, though. It was faster to just cache things than to use atlases before.

But it should be even better when I set up string caching.

The fewer GL state changes (and calls in general), the faster. I'm not sure how you've been doing it, but I've been uploading the textures first thing and just leaving them all resident in graphics memory until shutdown or when the JS-side object is freed. But still, this means only one on-GPU memory access, and many fewer state changes.

EDIT: As it turns out, even if you try and change the state to a new texture, as long as it is the same as the previous texture there is only a miniscule performance hit (as opposed to the monumental one of actually changing textures). At least with the AMD fglrx OpenGL library, but given the abysmal quality of fglrx I'd be shocked if this wasn't totally standard.

On the one hand, I've been playing texture binding fast and loose in TurboSphere and never unbind a texture, just bind a new one when I need a texture (unsafe in theory--I've never been bit by it, but it is faster). I imagine this may not be the case if I actually unbound each texture at the end of each procedure. But still! Why was I never told about this!?
Title: Re: TurboSphere
Post by: Flying Jester on September 07, 2013, 08:38:00 am
So here's a question.

If I added vector-graphics support, would it be useful? I'm mainly thinking about rotating and zooming images. It would certainly not be used nearly as much as raster images, but I can think of cases where I ended using a bunch of procedurally generated graphics to replace a raster image, and a vector image would be at least as useful.
Title: Re: TurboSphere
Post by: N E O on September 07, 2013, 12:54:45 pm
By vector graphics support, do you mean engine API to load vector formats like SVG and blit/transform/whatever?
Title: Re: TurboSphere
Post by: Flying Jester on September 07, 2013, 08:53:00 pm
Yes.

Probably just using the same API as images, but without the ability to manipulate the data in any way (at least to begin with), not even turning to surfaces or images. Mainly it would be blitting, rotate and zoom blit, and blitmasks.
Title: Re: TurboSphere
Post by: Flying Jester on September 09, 2013, 08:52:38 am
I will soon update the V8 used by TurboSphere to 3.21. Once I do, I believe I will begin the arduous process of backporting about six months of work to Windows. I need to work out why there are so very many crashes in Windows right now (or at least there were last time I checked, but I doubt that changed), and almost none in Linux. A very frightening state of affairs, but considering that TurboSphere was once quite stable on Windows certainly not an impossible task. Plus, one of the most significant problems on Windows I was able to directly trace to SDL2, which has finally been properly released in between then and now.

The new version of V8 will have the advantage of being able to seed the builtin random number generator (and redefine the entropy source!), and the new version also contains the functions needed to gracefully and elegantly change games while running, and to properly handle termination on errors (instead of only terminating the specific script that threw the error, as TurboSphere does now). Hopefully the release will support startup games--I've already modified the engine to allow for it, I just need to have the right functions from V8 to do it.

I must say that in the space between 3.14 and 3.21 the API of V8 has been vastly improved. Many utility functions for embedders have been added, and it's beginning to resemble the functionality that (from what I gather) SpiderMonkey's API has.

I've added Abort and Exit. I don't know why those weren't some of the first functions I added...I guess I was too excited by using SDL and SDL_ttf, and wanted to see something more interesting.
I've also added the rawfile and bytearray functions, and added a couple utility functions to T5, several TTF functions, more sound functions (and a reworked sound plugin), proper RequireScript and EvaluateScript functionality, reconfigurable game() and game.sgm names via config file, the ability to point TurboSphere at a file to be used as the game.sgm in addition to a folder holding a game.sgm, fixed most crashes relating to trying to open unavailable resources, and a bunch of graphics primitives. I'd need to go through the current TS API page on the wiki to know exactly what is new.
Title: Re: TurboSphere
Post by: Flying Jester on September 11, 2013, 10:29:41 am
RawFile and ByteArray are now fully implemented, complete with MD5-hashing.
Title: Re: TurboSphere
Post by: Flying Jester on September 13, 2013, 07:30:43 am
I'm still waiting for V8 3.21 to be branched. It seems the new version will finally support bitwise negation! It also looks like more work has been done to get V8 to compile with Mingw, and compilation with MingW64 has been achieved. I'm excited!

All MapEngine related resources can now be loaded and accessed through script. Tilesets use texture atlases. Scripts embedded in maps are now read and compiled by V8 upon loading the map. So the MapEngine is back to the level of functionality that it had when CodeBlocks bugged out on me over the summer.
Title: Re: TurboSphere
Post by: Flying Jester on September 17, 2013, 08:51:39 am
Images and surfaces can now also be saved as PNGs and TGAs (with or without RLE compression). Looks good. (https://github.com/FlyingJester/TurboSphere/blob/master/bin/Release/startup/images/LOL.png)

I've update TurboSphere on GitHub to reflect all the new stuff. It won't work for Windows just yet, I still need to backport the additions to T5 and the scriptfs plugin first. But it works quite nicely on Linux (and theoretically OS X).

https://github.com/FlyingJester/turbosphere (https://github.com/FlyingJester/turbosphere)
Title: Re: TurboSphere
Post by: DaVince on September 17, 2013, 03:42:13 pm
Awesome work, dude. :) Really need to play around with this more, but alas, time...
Title: Re: TurboSphere
Post by: Flying Jester on September 19, 2013, 12:22:14 pm
I'm working on a GUI configuration and information tool.

I'd like to use Byuu's Phoenix, but it seems any guide whatsoever on compiling it has vanished since Byuu's site was reborn. Just guess what seems to always pop up when I try to google it! (http://forums.spheredev.org/index.php?topic=13.30). I'd really like to use C or C++, because that would make it quite easy to list the script functions that a given plugin provides, as well as use T5 to read and write the config changes (which woudl automatically ensure that TurboSphere and plugins could read the resulting configuration.

I may end up creating a new shared library with Java bindings to call from the plugins, but I'd rather not do that--this doesn't really deserve a new library, although I wouldn't mind adding Java bindings to T5. It makes me wish I knew more about GUI programming in a language other than Java or Visual Basic.
Title: Re: TurboSphere
Post by: N E O on September 19, 2013, 01:48:11 pm
It wasn't Wayback Machined? Boo.

I'll come up with a quick Windows MinGW+MSYS compilation guide shortly. I pretty much just copy the same Makefile and Makefile.win I made between projects since I came up with one that works.

Update: here's a thread I just wrote (http://forums.spheredev.org/index.php/topic,209.0.html) :) also has a link to byuu's latest projects' sources.
Title: Re: TurboSphere
Post by: Flying Jester on September 19, 2013, 02:52:47 pm
I'll have to give that a try.

I don't think I mentioned this, but on Linux x86_64 you can place the AMD libm shared library in the same directory as the TurboSphere binary, and TurboSphere will automatically use AMD's libm for its software graphics math instead of the system libm. It still works perfectly fine without amdlibm (you'd never even know it could use it unless you looked at the source), but I have found a slight speed increase from using it on my AMD laptop.
Title: Re: TurboSphere
Post by: Flying Jester on September 20, 2013, 04:51:37 pm
Interesting things happen when you being using integer division, rarely-used TGA features, and OpenGL glReadPixel together.

(https://lh3.googleusercontent.com/-yovqWD-of6Y/Ujy1Amn1aTI/AAAAAAAAAWQ/jj6Opfh3884/w426-h266/0113_09_20_12_47_11_0317.png)
Title: Re: TurboSphere
Post by: N E O on September 20, 2013, 05:52:01 pm
Whoa.
Title: Re: TurboSphere
Post by: Flying Jester on September 20, 2013, 07:11:03 pm
That's what happens when all TGA-reading applications don't follow the spec for origin specification (so it's read back swapped horizontally, rather than vertically), and you use the run-length from vertically-flipped rows to encode the image.

In other news, screenshots now work (I fixed that craziness), and can save as PNG, BMP, and TGA files (the format used is configurable).
TS now has masking for BMPFonts, screen clipping rectangles, and preliminary blend mode support.
Title: Re: TurboSphere
Post by: Flying Jester on September 21, 2013, 03:38:56 pm
TurboSphere can now run Tung's startup game (with a couple minor graphical errors).

I've added a system for plugins to add to the script Surface prototype directly through the SDL_GL_threaded plugin (which allowed for surface.drawText, the last thing I needed to run Tung's startup game).
Title: Re: TurboSphere
Post by: Radnen on September 21, 2013, 04:11:53 pm
I too get graphical errors on tung's startup game and I don't know exactly what's wrong. Not the one in your screenshot, but a different one that changes the background color of the text on surfaces (surface.drawText works in all cases but this one).
Title: Re: TurboSphere
Post by: Flying Jester on September 21, 2013, 04:28:17 pm
Looks like a blendmode/alpha channel problem. I have a couple of those kinds of problems still too.

Usually the fix in SDL is changing the surface creation flags, and being sure to always set a blend mode on surface creation.

EDIT: Just fixed a similar problem in TurboSphere. Lovely blend modes.

That first screenshot actually was rendering almost right, except for the outline of the scroll bar. It seems that Tung's game gets just a little confused if there is no games folder.
Title: Re: TurboSphere
Post by: Flying Jester on September 28, 2013, 08:11:43 pm
I like FLTK.
Going with C/C++ was the right thing to do for the config tool. The configuration program can read out what functions, variables, and types a plugin provides for script, as well as what functions it provides for other plugins to use and what plugins it depends on.

Plus, I can read and write the config files with the same library that is used in the engine.
Title: Re: TurboSphere
Post by: DaVince on September 29, 2013, 01:42:54 pm
Very nice indeed, good work. Sounds useful, especially if it could download dependencies for you.

(...But add some padding on the tabs and "plugin information" pane, dude! And make that Refresh list take up the full width you have left. And fix the fact that the padding in the list is bigger on the left side than it is on the top! :P)
Title: Re: TurboSphere
Post by: Flying Jester on September 29, 2013, 06:52:37 pm
It could download dependencies, yes. But I kind of think that should be the job of a dedicated game manager--the plugins are dependencies of games, or of another plugin, or are explicitly installed. Like how packages and groups work for package managers like Yum, Pacman, and Aptitude. But in the end, I don't download libpng because I love having libpng, I download it so that programs can read png images. I think it should be game-centric. I just might be working on a way for games to specify what plugins they really need, too...

On the other hand, it may be good to have a utility like this to handle plugin dependencies. The game manager could be engine-independent, and then tell the configuration utility what it is doing. The config utility could then, say, change settings for the game to handle, for instance, engine differences (like TurboSphere using constructors instead of functions for engine objects by including additional scripts, or the opposite for a game designed for TurboSphere but being run in another engine), download plugins if necessary, etc.

It would be cool to have the plugins' function and variable lists act as a sort of 'provides' list (like in Yum), so you could use alternative plugins if you wanted, or if you already had a plugin that provided the functions you wanted installed but the game specified a different plugin by default you could just use the plugin you already have.

But that's a ways off. Once one of the Sphere Next Generation engines is done, I may just look into this kind of utility to make it easier to use any of the engines you want. Of course, that would mean all the engine devs would have to come together and agree on standards for these things...or one of us could just blaze forward, and the others would be inclined to adopt it. I wouldn't mind that, I don't especially like creating standards myself.

After all this work with T5, I looked at some of the code that is in the engine's support libraries. And I don't like what I see! It's some of the first C++ I ever wrote, and the quality and methods are about what you'd expect given that. The configmanager needs serious work (not that it doesn't work, it's just not pretty, clean, or particularly logical), and I need to finally put all the graphical algorithms inside graphicalg. That's what its' there for!

I know the interface isn't too nice right now...but that doesn't cause it to segfault, (whereas opening a file with T5 before calling T5_Init does, for  example), so I haven't done much for it yet :)
Title: Re: TurboSphere
Post by: Flying Jester on October 02, 2013, 05:01:49 pm
 I have created an RPM of the version V8 that TurboSphere uses. (http://flyingjesterentertainment.webs.com/v8-3.19.18-1.fc19.x86_64.rpm) I got really, really sick of rebuilding V8 because it takes one million years and is tricky enough that I usually mess it up (and then have to wait another million years to recompile it).

I must warn anyone who uses it: I make no guarantee about this package, except that it MAY break any other application that uses V8 as a shared library (this DOES NOT include Chrome or Chromium). The version of V8 provided by Fedora is 3.14, which is over a year old. In fact, it's older than TurboSphere. I suspect the reason is that almost no other programs use V8 as a shared library except node.js, which uses 3.14. Which certainly explains that, doesn't it?

Fortunately, node.js was packaged well enough that it my V8 rpm and they are recognized as being in conflict by the package manager.

Edit:
A T5 rpm (http://flyingjesterentertainment.webs.com/t5-0.8.5-1.fc19.x86_64.rpm) is also up. I'll probably just end up making a linux binary release soon.
Title: Re: TurboSphere
Post by: Flying Jester on October 07, 2013, 08:48:59 pm
I've replaced the SGM-parsing code in TurboSphere--which was the oldest code I've written that was in TurboSphere! SGM-parsing now uses T5, which means it can much more easily parse in any non-standard fields in the SGM file.

I'm working on finally getting a proof-of-concept plugin that uses a language other than C or C++ for its backend. I know it is possible, but I'd really like to get a C# plugin going...and that requires me to learn the intricacies of linking .NET and CLI code and still exposing C-linkage functions.


And just guess what the third result is on Google when you search for "C linkage in C++/CLI". I seem to be making a bit of a name for myself...so long as you look for the same anwsers I do.
Title: Re: TurboSphere
Post by: Metallix on October 08, 2013, 05:27:19 pm
Hey, nice to see the progress here. Since some weeks I am again a bit more active, playing around with javascript and turbosphere and/or html5. I could not find any clue in the docs: Can I draw an image/surface on an image/surface in turbosphere? Or is this still work in progress? This would really help me creating a small js based map engine ;)
Title: Re: TurboSphere
Post by: Flying Jester on October 08, 2013, 07:20:06 pm
Only if you have Linux :(

I just got Windows on a computer after a long time, so the Windows binary releases are getting pretty old. In addition, the last Windows release was really buggy, but was fine in Linux.

I'm working on porting all the new stuff back to Windows, which includes suface.blitSurface support. And rawfiles, bytearrays, a MapEngine on par with TS's 0.1.0 plus spriteset and basic person support, more graphics primitives including circles, a proper EvaluateScript and RequireScript, GetGameList, basic BlendModes (REPLACE, RGB_ONLY, ALPHA_ONLY, and BLEND), and a lot of other stuff. I recommend either waiting, or, if you have Linux, compiling from the source on github.

The function list (http://wiki.spheredev.org/User:Flying_Jester/TurboSphere_API) is generally up to date for the latest Windows release.
Title: Re: TurboSphere
Post by: N E O on October 09, 2013, 02:46:44 pm
Hey dude, once you make some more progress with the TS map engine can you go to the Developing a Sphere-compatible engine wiki page (http://wiki.spheredev.org/Developing_a_Sphere-compatible_engine) and add some more of your experiences to it?
Title: Re: TurboSphere
Post by: Flying Jester on October 09, 2013, 04:39:01 pm
Yes, I certainly can! Most of what needs to be done are things that I need to test to find the correct behaviour--which is exactly what that page needs. It would be good for all involved to have more of that information fleshed out, there's no need to rediscover these things again and again.

I can easily add a bit about what I've got so far. Especially for spritesets, it took me a minute to make sense of them on the engine-side, mostly based on the file spec.
Title: Re: TurboSphere
Post by: Metallix on October 09, 2013, 05:04:59 pm
Ok, I will wait for a proper windows release :) Thanks for the info!
Title: Re: TurboSphere
Post by: Flying Jester on October 10, 2013, 08:31:22 pm
I'll just put this here.

If you're curious what happened to the missing versions between 0.3.1 and 0.3.5, they were released silently as source on github, and only worked (reliably) on Linux.

Nifty Things Realistically Planned for TurboSphere, Version 0.3.6:

Fat Binary Plugins. C# editor plugins and C/C++ TurboSphere plugins, and for your convenience, all in the same DLL. I think I now know how to do this.

Separate all the threadable surface functions into surface-thread operations. The only non-threadable surface functions I can think of are converting to an image, saving, and blitting (another kind of converting to image).

A C#-based proof-of-concept plugin.

A FLTK- (or Phoenix-...we'll see what happens) based graphical configuration utility. It can already read what functions and variables a plugin contains for use script.

Tung's startup game will function correctly!
Title: Re: TurboSphere
Post by: Flying Jester on October 13, 2013, 01:38:40 am
Testers Needed!

I present you with a still slightly buggy version of TurboSphere for Windows. Get it at http://flyingjesterentertainment.webs.com/ts-0.3.5a.zip (http://flyingjesterentertainment.webs.com/ts-0.3.5a.zip).

All those promises in the post above are not in this build (except for partially the last one). I need to just stabilize things on Windows first.

It needs a lot of testing. Like, a lot. The included test game shows two problems currently. The first is that the surface worker thread never enters independent mode, so it doesn't actually work in the background, only when needed right away. Which defeats a part of the purpose--it can still perform reorderable operations out of order, but it cannot also work constantly to ready surfaces ahead of time. The second problem is that image saving isn't working due to issues with linking to libpng from SDL_Image. This causes a crash at the end of the test game (where it would normally save the MJ map it draws).

You can hit 'j' to see the Majestic Map Engine in action in TurboSphere, comma and period to see the circles at different radius sizes, M and N to switch back and forth between MIDI and OGG playback, and Y to test the Abort() function.

I recommend running from the command prompt to see all output. You can specify the SGM or directory to find the SGM file to run as a parameter. If it properly crashes for you at any point, please let me know. Ideally with information about what you were doing, and what function triggered the crash or just an example script that crashes it.

Alternatively, you can post an issue on the GitHub page.
Title: Re: TurboSphere
Post by: Flying Jester on October 15, 2013, 05:56:49 am
I fixed the bug that causes crashes on saving an image or taking a screenshot. The quick fix was to disable PNG saving, since I need to be more sure about linking to the same version of libpng as SDL2_Image to do things properly here.

I also fixed a bug (stroke bad feature) that stops TS 0.3.5a from working on some Intel GPUs. Seriously, Intel has bad OpenGL support compared to NVidia and AMD. They don't support a lot of fairly standard features. I'll make a new release with that and a fix for the screenshot-crashing bug tomorrow or the next day.
Title: Re: TurboSphere
Post by: Flying Jester on October 18, 2013, 01:01:30 pm
I've got PNG saving working on Windows, using the same libpng that SDL2_Image uses. Turns out that it's easier to make a .lib file to go with a lone .dll than I thought.

I'm still working on the threading issue, but I've got a good idea what the problem is and why it is happening.

Also, I need to remove the GetLocalName function from the test script, or just fix that function. It's a lot buggier than I thought at first.
Title: Re: TurboSphere
Post by: DaVince on October 20, 2013, 11:46:05 am
Hmm, the program crashes almost right away for me. First time I started it I heard some music, but after not even a second I get a runtime error. Subsequent runs crash almost right away.

Here's the not likely to be helpful nonsense it outputs.
Code: [Select]
Probleemhandtekening:
  Gebeurtenisnaam van probleem: APPCRASH
  Naam van de toepassing: turbosphere.exe
  Versie van toepassing: 0.0.0.0
  Tijdstempel van toepassing: 525a2ed1
  Naam van foutmodule: v8.dll
  Versie van foutmodule: 0.0.0.0
  Tijdstempel van foutmodule: 5258cbe5
  Uitzonderingscode: c0000005
  Uitzonderingsmarge: 0014957a
  Versie van besturingssysteem: 6.1.7601.2.1.0.768.3
  Landinstelling-id: 1043
  Aanvullende informatie 1: 0a9e
  Aanvullende informatie 2: 0a9e372d3b4ad19135b953a78882e789
  Aanvullende informatie 3: 0a9e
  Aanvullende informatie 4: 0a9e372d3b4ad19135b953a78882e789


Something with v8 once again, hmm.

Also see attached screenshot in case you can notice something wrong there. (Note: pic reduced to 256 colours)
Title: Re: TurboSphere
Post by: Flying Jester on October 20, 2013, 09:16:24 pm
OK, that's something that I haven't seen yet. The supposed cause being V8 means it's because of a call from script, so not directly related to the surface thread issues.

I actually think this is related to the call to GetLocalName, since that hasn't shown up in this shot, but everything else has. Plus I've been having issues with that on Windows. I'll try and fix that, and make a new release with that and all the other new fixes in it in a bit.

Title: Re: TurboSphere
Post by: Metallix on October 27, 2013, 12:40:35 pm
For me everything seemed okay, until it crashes on ESC in that second phase of the test. Seems like the save path contains some invalid characters (win 8.1, 64 bit).
Also when I checked the code it seems that you can only enter that second phase by pressing Q. But it somehow goes to that mode when I move my mouse outside of the game window...
Title: Re: TurboSphere
Post by: Flying Jester on October 27, 2013, 06:25:56 pm
That is strange that you see invlid characters in the path...but either way, it would have crashed since it isn't properly linked to libpng.

The main loop exits after 10,000 frames, and records how long it took to draw.

EDIT: I've updated the TurboSphere API (http://wiki.spheredev.org/User:Flying_Jester/TurboSphere_API) page to more accurately reflect 0.3.5a.
Title: Re: TurboSphere
Post by: Flying Jester on October 28, 2013, 12:33:53 am
OK, TurboSphere 0.3.5 is out (http://flyingjesterentertainment.webs.com/ts-0.3.5b.zip). It fixes the problems with the surface thread not locking, which fixed a lot of the freezes I was getting. It also very almost quits gracefully.

BEWARE: You will usually need to kill this version from the task manager to close the command window. It hits deadlock when it tries to fully exit. As you can probably tell, the issue is not totally resolved, but I am working on fully fixing this.

I believe that both the crashes that DaVince and Metallix were getting are fixed (I was only able to reproduce the problem Metallix had, and it is fixed for me, but I believe I know what was happening for DaVince and fixed that as well).

@Metallix: That's two errors at once. The garbage-path name error is fixed, the other no longer causes a full crash.
Title: Re: TurboSphere
Post by: Flying Jester on October 30, 2013, 04:12:23 pm
I believe I have found a way to allow accessing specific bytes of a ByteArray using index values (and more generally accessing data in script-side objects using array-style access in arbitrary ways). I plan on incorporating this addition in the next release.
Title: Re: TurboSphere
Post by: DaVince on October 30, 2013, 05:22:01 pm

I believe that both the crashes that DaVince and Metallix were getting are fixed (I was only able to reproduce the problem Metallix had, and it is fixed for me, but I believe I know what was happening for DaVince and fixed that as well).

Cool. I'll try it out when I'm on Windows. (Could be a while; could not be. We'll see :) )
Title: Re: TurboSphere
Post by: Flying Jester on October 30, 2013, 07:54:36 pm
@DaVince: No worries, I hope it works better for you this time. Ideally perfectly, but I doubt that given it crashes for me in certain circumstances.

I've made a change that will be a part of the next release:

(http://flyingjesterentertainment.webs.com/errorhandling.png)

Exceptions in script will be reported with the correct line number now (that alone is a vast improvement) and in a popup box. Not only is this shiny new eye candy compared to how Sphere does it (and much, much shinier than directing all output to stdout and stderr like TurboSphere did), but the text in the box is selectable. You can just highlight and copy the error, line number, everything and then paste it anywhere you want. It shouldn't be annoying either, since the popup gets focus, and any key press will make it disappear. If you didn't look at the screen, it would seem to act like Sphere 1.5.

I'd like to take this further, and add a 'go to line in source' button that would open an editor to look at the line in the script that threw the exception, but that is much more work, there are two editors I would have to consider (I can't use Radnen's editor on Linux), and I don't think that the original editor has this kind of function built in. So, oh well, a nice thing to dream about.
Title: Re: TurboSphere
Post by: DaVince on October 31, 2013, 11:43:03 am
Nice! Any particular reason you didn't use an actual editbox, though? It's not visually clear that you can copy and paste the text as it is. Also, is that third line indented or centered? Maybe you could remove start-of-line spacing...

Quote
any key press will make it disappear

I'd rather have just enter/space trigger the OK button and have everything else do nothing. Makes it not mess up when I try ctrl+C or when trying to select text using the keyboard.
Title: Re: TurboSphere
Post by: Flying Jester on October 31, 2013, 02:22:59 pm
It's more clear on Windows (or so I am told), and even on other Linux set ups. It's an SDL2 feature. That's a pure X11 box, it doesn't look that way for most folks even on Linux (my WM+X set up is sort of hand rolled, fluxbox with a customized theme). To me, that's clearly selectable since the background is dark gray and not silver. I'll see if I can't make it more clear (and see what it looks like on Windows).

You can easily select the text and copy, nothing unusual, it's just that it is set to give the button focus when it pops up.

The third line is exactly what is in the source when V8 gets it. So here, there are two horizontal tabs in it, just like the actual line from Jest_Main.js (that name should appear as the script name, it doesn't here because of reasons). The message box will always be sized to hold the line. I could remove whitespace and/or split it to keep it a reasonable size. It actually might have a maximum size, I'm not sure.

And of course, I can make all this configurable. In fact, I'm especially tempted to since it is an engine feature, not a plugin feature.

EDIT:
It works more like you described now, DaVince. Except for seeming more selectable, that'll have to wait until it gets ported to Windows (next release, since this really shouldn't add any new issues on Windows) so I know what it looks like there.
It also actually displays the script name now. That was the result of one of the oldest outstanding issues in TurboSphere, fixed now.
I actually like how it looks now a lot better. Thanks for the feedback!

(http://flyingjesterentertainment.webs.com/messageboxTS.png)

Title: Re: TurboSphere
Post by: Flying Jester on November 09, 2013, 11:58:45 pm
TurboSphere 0.3.5c is released! (http://flyingjesterentertainment.webs.com/ts-0.3.5c.rar)

There are three things that are improved in this version:

1. Fixed a bug where repeated calls to GetLocalName caused crashes (and bad return values of GetLocalName). This fixes most of the intermediate crashes I saw (and all crashes not at startup or exit).
2. Fixed the evil, evil bug causing TS to either crash outright or deadlock upon exiting. This fixes all crashes I was getting on exit.
3. Errors now report the correct line number.

As it turns out, SDL2's messagebox API is not entirely consistent between X11 and Win32, and the text is NOT selectable in Windows. But errors are reported in a much nicer way now.

As far as the crashing and/or deadlocking at exit...I'm shocked by how much of the standard library is not thread safe. But this did lead me to uncover quite a few 'bugs' that weren't apparent in the engine's handling of plugin lifetimes. I feel a lot more confident in the engine's handling of plugins now. TurboSphere will someday soon be able to change games while running--the only bit missing is the upgrade to a newer V8 that will let me gracefully restart V8 (due to new v8::Isolate code and a couple debug features) and not drastically change the architecture of the TurboSphere engine proper.
Title: Re: TurboSphere
Post by: Radnen on November 10, 2013, 12:32:44 am
You make me want to work on my sfml variant again. :)

Neat progress btw, I love the idea of using a msgbox for error messages. I can't easily do that in SphereSFML without adding some nasty dependencies, so if I were to do that it would be more of a custom message popup box. But errors still won't be selectable.
Title: Re: TurboSphere
Post by: Flying Jester on November 10, 2013, 09:42:32 pm
It's not too tough to add what TS has (on Windows and in C++ at least, just a small Win32 call). But to make the text selectable seems to be a ridiculous amount of work without bringing in GTK, QT, FLTK, or something similar, which I do NOT want to do with the main engine.

I've commited all the new changes to TurboSphere on Github. It's good to see the engine being at least very close to as stable on Windows as it is on Linux. I'm still getting some weird crashes related to V8 calls before script execution starts, but I'm not too worried. Either the cause will become clear in time, or the change to a newer V8 will make it clear (or just fix it without me knowing why).

I've been working on the map engine (again, I know...I keep saying, it's one of the most sophistocated and complicated parts of Sphere), and I've made some decisions that I think will make it much easier to develop it for TurboSphere--easier for me, anyway. I'm seperating out the functionality into a tiling and display engine (this will probably be the basis for sprite batches later, and be more tied to the graphics plugin), an entity and person engine (which will have stronger ties to the input plugin), and a physics and obstruction engine (possibly pathfinding, and exposing obstruction/hit detection features to script). I'd like to at least make it possible for other plugins and scripts to call just these parts of the map engine without needing to invoke the whole thing. The biggest reason I want to do this, really, is that it will help me make sense of the whole thing and build a cleaner architecture and implementation. It's too overwhelming to tackle the whole map engine as one piece!

I also like the idea because it will make the whole system less monolithic. You would, for instance (and purely theoretically at this point), only need to replace the physics and obstruction engine to have the map engine work for sidescrollers, or (very very theoretically) only replace the display and tiling engine to make it 2.5D (hardware 3D).
Title: Re: TurboSphere
Post by: Radnen on November 11, 2013, 12:27:39 am
I like the fact you split the functionality up. I did the same with my engine but did not expose its functionality in code.  Splitting up the tiling I think is a great way of making it flexible. I want to add an isomorphic engine to my tiler. But right now I've been focused on emulating default sphere. It gets hard when trying to get the systems to agree after you,  say teleport to another map. Even vanilla sphere gets it wrong. I know from having done the ld comp and noticing all kinds of strange behavior with how maps load depending on how you call the teleport code.

Good ideas, Good progress!
Title: Re: TurboSphere
Post by: Flying Jester on November 11, 2013, 03:12:49 am
If you are still interested in developing sphere-sfml (no judging from me about any hiatuses! I've spent longer stretches away from TurboSphere than I'd like to admit), maybe we should make a new 'Feature Request' thread, but more new-engine oriented. Of course, for a lot of new features, editor support would likely be important as well.
Title: Re: TurboSphere
Post by: N E O on November 11, 2013, 02:44:07 pm
Once again, the map engine seems to prove the biggest obstacle to progression of Sphere. I personally hate how monolithic vanilla Sphere's implementation is myself; it doesn't even split the updating and rendering to separate threads!

I still dread how much work it would actually take to make a usable map engine in the Web Sphere variants :/
Title: Re: TurboSphere
Post by: Fat Cerberus on November 12, 2013, 11:09:50 am

Once again, the map engine seems to prove the biggest obstacle to progression of Sphere. I personally hate how monolithic vanilla Sphere's implementation is myself; it doesn't even split the updating and rendering to separate threads!

I still dread how much work it would actually take to make a usable map engine in the Web Sphere variants :/


To be fair, splitting updating and rendering into separate threads tends to be harder than it sounds.  You don't want to render in the middle of an update, which means you have to synchronize constantly, losing most of the benefits of multithreading.  From a compartmentalizing standpoint it makes sense, but it most cases it doesn't actually improve performance and may in fact hurt it--especially in a multi-core environment where the constant synchronization bottlenecks both cores.
Title: Re: TurboSphere
Post by: Flying Jester on November 14, 2013, 01:07:49 am
I'm not really sure how much threading would add to the update and rendering threads. I haven't tried, mind you. But it's complicated enough for me already.

Conceptually, using modern OpenGL or DIrectX would achieve something similar to threading the render script. And that's what I plan to do, and why the map display engine will also function as a sprite (or any kind of drawing, really) batch engine.

The configuration utility for TurboSphere is up on github (https://github.com/flyingjester/tsconf). It requires T5 (included in TurboSphere) and FLTK 1.3. I'll put win32 binaries up once I actually make them.

Currently it only handles the video configuration (just fullscreen and video scale) and some TS specific configurations. The TS-specific stuff you probably will never want to change--it's the default sgm name, the startup folder (that you may actually want to change for releasing games--but TS doesn't care about that value yet), and the "game" function name used in script.
Title: Re: TurboSphere
Post by: Radnen on November 14, 2013, 01:40:17 am
So I forgot, are you going to store graphics options with the game files?

I want to propose a standardization for a .sg2m or .sg2 or whatever format that is a second version of the sgm. Built precisely to hold this kind of data:
1. Author
2. Description
3. W x H
4. 2x or 1x (scale)
5. v-mode
6. language
7. etc

But notice my engine can't have the same v-modes your engine has, or another engine uses a different language. So, we should use XML or some kind of non-binary (scalable) data type that can store these properties. Kinda like this:

Code: [Select]

<game>
    <details>
        <name>The Awesome Game!</name>
        // ...
    </details>
    <sound>
        <volume>100 or 1 or 255</volume>
        // ...
    </sound>
    <graphics>
        <scale>2</scale>
        // ...
    </graphics>
    <input>
        <up_key>42</up_key>
        // ...
    </input>
    <networking>
        <use>F or False or 0</use>
        // ...
    </networking>
</game>

(notice the above is a sketch)

Then we can expose methods through code so your game can modify it's own data. This way you can put custom stuff in there. Or you can modify existing stuff, thereby making it possible for games to do interesting things to their own sgm (without the engine wiping it).

I know we tried this once before, but we didn't take into consideration the configuration options stored separately into the sgm. (Which is why I propose an sgm 2.0 type format).
Title: Re: TurboSphere
Post by: Flying Jester on November 14, 2013, 02:06:15 am
I'm a little unsure if I really want XML in particular. I would like to use an INI-with-sections format (which TurboSphere already uses), but a more modern markup language may be better. I'll have to think about what I really want for that. XML in C++ is not all that much fun--but it could certainly be worth it. Maybe BML or SGML...or XML, I'm not really sure what the best option would be yet. It is hard to go completely wrong with XML.

The way I worked out the per-game scale and filter idea in was for each game to optionally have its own engine.ini and system.ini files whose values would override the main ones. But that is fairly ugly--it's designed to be simple if you were a Sphere 1.x user, not to be a good idea on its own merits. And I think that I removed the code in TurboSphere that checked for this at some point. TurboSphere gets some of this information from the engine.ini file (note that the one included with TurboSphere has stuff Sphere wouldn't use), but this is NOT an ideal solution. Just what organically happened as I developed TurboSphere.

I'm all for a new S2GM (Sphere 2 Game Meta?) format. I'll give this whole thing some thought.

As far as values to hold, here's what I'm thinking:

1. Author
2. Description
3. WxH
4. Scale
5. v-mode (what is this?)
6. launguage
7. Known usuable Sphere version
8. Things to disable (plugins, networking, maybe other stuff)

It would be good for the possiblity of engine-specific and version-specific settings and setting overrides.
Title: Re: TurboSphere
Post by: Radnen on November 14, 2013, 05:03:28 am
5. v-mode: in Sphere you could choose a filter. I think that's a better word, haha. (though I don't think sphere-sfml will have this anytime soon, but it is possible via shaders (and be nice and quick too)).

I wanted xml or the like so it could grow naturally. If we lock it down to a "this must be this" format like aegis did for all other sphere types then we lose the ability for each engine (namely yours and mine) to add their own unique things to it without breaking it for the other engine if you so choose to switch. :) So a modable, organic settings file for each game.
Title: Re: TurboSphere
Post by: N E O on November 14, 2013, 01:25:20 pm
For SGM v2 I'd personally like to have the extension .sph or .sphere on the game file. XML, BML, or INI format is fine by me but it should stay consistent and DEFINITELY have the SGM format version tagged in the file somewhere for later extension.

I'll start an official Future of Sphere thread for discussion.

(edit: done (http://forums.spheredev.org/index.php/topic,1099.0.html))
Title: Re: TurboSphere
Post by: DaVince on November 14, 2013, 03:56:17 pm
Interesting idea. I also like INI more for this; it's easier to read and modify even for non-techies. Maybe look at dosbox.conf for a nice example of a well-constructed INI file.

Regarding v-mode, I would name it video-filter instead. v-mode is too reminiscent of the v-blanking or v-sync functions, if you ask me. Also, a fullscreen option of course. :)
Title: Re: TurboSphere
Post by: Flying Jester on November 14, 2013, 07:17:54 pm
Just thinking about it, I would like to expose things like the resolution and scale values in the SGM file to script, both for reading and writing, so they could be edited and saved in game (saving your resolution and scale is pretty standard for games, after all). Same for v-sync, and probably the filter as well.

I will probably do this by extending the 'game' object (the kind returned by GetGameList) to have these values in read/write mode, and adding some function like GetCurrentGame() that returns the current game.
Title: Re: TurboSphere
Post by: Flying Jester on November 17, 2013, 03:54:00 am
I have fixed what I consider the longest standing and most severe flaw in TurboSphere.

I don't know if anyone has seen this happen, but TurboSphere does NOT validate engine-defined object types. You can segfault 0.3.5 by passing anything as a parameter for a Color, Image, Surface, etc., that is not what TurboSphere expects. Except for Colors, actually, since they are so small that most any object passed that has associated C++ data can be read as a Color.

But doing that sort of thing would cause a segfault 99% of the time. Or at least it did. It doesn't now.
Title: Re: TurboSphere
Post by: Flying Jester on November 18, 2013, 03:00:58 am
I've improved the graphics plugin performance. I've removed two of the mutexes (the ones that control how the surface rendering thread operates) and replaced them with atomics for signalling. This improves performance by about 10%, and fixes the recent issues I've had with slow startup on Windows (as in 500 ms before, not even noticeable now).

I could really improve performance by caching the hardware textures that surfaces generate when you blit them, and reusing them if the surface has not been modified since the last blit. The simplest way I can think of to do that would be for the surface thread to mark the surfaces as 'dirty' if they are the destination of any operation it performs, and the function that blits surfaces to the screen generate a new hardware texture for the surface if it is marked as 'dirty' and then unmark it, and to just use the surface's cached hardware texture if it is unmarked.
Title: Re: TurboSphere
Post by: DaVince on November 18, 2013, 06:36:11 am
Good job, I remember it freezing for half a second on Windows!

On a technical note, I have no clue what mutexes and atomics are, and why they affect the performance like they do. Think you can summarize or link to good articles explaining these?
Title: Re: TurboSphere
Post by: Fat Cerberus on November 18, 2013, 08:11:15 am
Yeah, mutexes are meant to be used for cross-process signaling (e.g. a lot of apps create a mutex when running that their installer checks for and prevents installation if it exists), so they have a good amount of overhead. So it's good that you fixed that.

@DaVince:
Mutexes and such are used to synchronize operations between different threads and/or processes that need to manage a shared resource (for example, rendering and updating code in a game). "Mutex" is actually short for "mutual exclusion", which is a pretty fair description of what it does, actually. :)
Title: Re: TurboSphere
Post by: DaVince on November 18, 2013, 09:49:59 am
Thanks for the explanation. Sounds useful for the right situation. :)
Title: Re: TurboSphere
Post by: Flying Jester on November 18, 2013, 02:46:21 pm
One of the biggest 'issues' with mutexes is that locking and unlocking them is slow(ish). Which is why you need to be very efficient with thread synchronization.

Mutexes are great for protecting a shared resource--they symbolize it, if you have locked the mutex than you have locked every other thread out of the resource, and can do whatever you want to it without worrying about other threads meddling with it.
But they aren't so great for signalling (and also not explicitly made to do so in the way I was using them). They work fine for it, but this is where that whole slowish thing comes in. If you profile TurboSphere, you will see that over half of its time is spent locking and unlocking mutexes. But I've managed to reduce that to about a third (even with the surface thread skewing this--most of what it does is wait for surface operations to be queued, just locking and unlocking mutexes with nothing better to do).

I've changed it so that instead of a mutex that controls whether or not the surface thread should do whatever or work on a specific surface, there is an atomic variable (atomic, as in I can guarantee that I can write or read it from multiple threads and nothing horrible will happen). The atomic is just a flag that the surface thread checks, and the main thread sets and unsets.

There is one last mutex I think I could replace with an atomic. It would probably improve performance even more (this is half of why blitting surfaces directly to screen in slow in TurboSphere), but replacing it would require some changes to the structure of the surface thread and supporting functions in the main thread. I plan on doing it, but it will take serious work. Threading requires a lot more consideration and concentration than most other things I've done. Which is why there are copious comments in the threading code in TurboSphere (which is probably why in turn it worked almost perfectly the first try!).
Title: Re: TurboSphere
Post by: Radnen on November 18, 2013, 03:20:03 pm
I'm always one for getting your design done right before you optimize (so a good design that's initially slow is a good thing). So it's good to see you optimizing right about now (non-trivially of course).

About the surface caching: I do that too! Whenever you modify the surface a changed flag is set to true. What it does is delay the processing of the surface drawing until you actually do one of three things: convert to image, blit to screen, and save to file (which sets the flag back to false).
Title: Re: TurboSphere
Post by: Flying Jester on November 18, 2013, 03:33:12 pm
I don't delay the processing per se. The surface thread just goes along working on surface operations in the order they come in, at its own pace. When a certain surface is needed, the surface thread goes through all the operations where the requested surface is the destination, and checks the source surface. If the source surface has older pending operations where it is the destination, those operations are performed first, then the original operation on the requested surface.

But the only time a surface is needed is for blitting, saving, and creating an image. At one point I removed the calls for requiring the surface from all the primitives, but those changes aren't in TurboSphere anymore (I lost my laptop's HDD, and it was over the summer when I had such bad internet that it was a serious ordeal to commit changes).

In TurboSphere it's more like out-of-order, lazy computation than delayed computation.

In my experience, requiring a surface usually flushes out almost all of the pending operations (forcing them to be completed right then and there). It's just a how things work in practice. But the system could be used in a way that lets you do a ton of surface work in the background, fully parallel to the main thread. The test game that comes with TurboSphere does this with the Majestic Map Engine (that's what can cause the engine to hang on startup).

Mainly what caching would entail is just adding a texture to the userdata of the surface, and a flag that determines if the surface needs to be pushed to the texture again. Any operation on the surface will unset the flag, and a blit, save, or conversion will first push the new surface data to the texture.
Title: Re: TurboSphere
Post by: Fat Cerberus on November 18, 2013, 05:54:31 pm
The thing with mutexes though, at least under Windows, is that they are cross-process. This is a big part of why they are slow. It also means, if one process creates a mutex, another can't use the same name or it'll just get the existing mutex back (which is bad if more than one instance of TS is running).  It's been a long time since I've written any multithreaded code, but I was almost positive there is a process-local alternative to mutexes (semaphores or something? IDR) that is faster and won't trample over other engine instances.

Not sure what you're getting at with the atomics though.  To me that sounds suspiciously like DCLO, which can still cause race conditions under preemptive threading, even with an atomic flag variable.
Title: Re: TurboSphere
Post by: Flying Jester on November 18, 2013, 06:09:35 pm
I don't think that is a feature of semaphores. Semaphores are like Mutexes, but don't explode if they are modified by a different thread, and can be locked and/or unlocked a number of times. They are evil, complicated, almost never the right solution, and evil.

The SDL2-exposed mutexes definitely aren't inter-process by name, or at least are mangled in some way to avoid it.

The atomics aren't guarding anything. They are sending signals that change how another thread will operate. Even if they were always wrong, TurboSphere would run fine, just with more pauses or slightly slower in general.
The way it's built right now, the worst thing that can happen is that the surface thread will do one extra operation before working on a specific surface.

You can, theoretically, replace almost all mutexes with atomic flags. But if you do replace data-guarding mutexes, there be dragons. I'm just replacing a signalling mutex with an atomic.

...DCLO? (http://en.wikipedia.org/wiki/DLCO)
Title: Re: TurboSphere
Post by: Fat Cerberus on November 18, 2013, 09:31:03 pm
Okay, phew. I was worried you were using the atomics for synchronization, which as you said, is asking for trouble (on a sidenote, "There be dragons" is priceless and never gets old :)).  And as for this...


...DCLO? (http://en.wikipedia.org/wiki/DLCO)


...I was referring to what is known as "double-checked locking optimization" (look THAT one up on Wikipedia, it's there), where you check for a locking condition first, and only if that condition is met do you lock the mutex, otherwise you continue as usual.  Bad enough as is, but I've seen some pretty terrible DCL implementations that simply use a bare variable as a guard, assuming the fact it's atomic is enough (which as we know, it isn't. It really, really isn't).  So yeah, good to know you're not doing that!

On a sidenote, all this progress on TurboSphere is making me want to try making an engine in C++ again.  It's been years since I've done that and the best I ever came up with was a crappy software-rendering engine that could only blit and draw rectangles, with some other gimmicky stuff thrown in like a universal map engine (meant to be extensible but I didn't really have the know-how to pull that off) and such.  I will say though that my time-based update system was pretty awesome.  That allowed the game to run at any framerate, even hundreds of thousands of FPS, and still update everything at a constant rate, completely decoupling the updating and rendering routines without even having to throttle updates.  (The downsides being: to achieve that kind of precision I had to use QueryPerformanceCounter, a facility I'm not sure is available in anything other than Windows; and double-precision was required for almost all gameplay variables).  But yeah, ultimately it was a mess; that, combined with how easy it is to program with Sphere and JS, hasn't exactly made me want to go back and try again. Maybe some day...
Title: Re: TurboSphere
Post by: Flying Jester on November 19, 2013, 12:17:45 am
I found my solution to ending up with a mess when I program from my years with TurboSphere. Compartmentalize as absolutely far as possible. Not only does it encourage good considerations for code reuse, but it allows you to refactor or replace most pieces fairly easily in case they get out of hand.

The plugin system and the four supporting libraries (sort of five) were my first experiment with that style, and I'm extremely happy with how it turned out. In fact, I've replaced the largest plugin, the graphics plugin (which I consider to be bordering on too largely scoped) twice now with few problems.

Originally, I started TurboSphere with the intention that I would write C++ just this once, so that I wouldn't have to ever again. It didn't really turn out that way...I actually have grown to like C++ and C, despite their slightly-screwed-upness.
Title: Re: TurboSphere
Post by: Fat Cerberus on November 19, 2013, 12:33:22 am
I don't really mind C++ as a language, even with all its quirks, and unlike a lot of people I don't consider the fact it allows you to shoot yourself in the foot a bad thing, as it means you can do some pretty evil stuff when needed (there is still such a thing as "necessary evil" :) ).  No, the biggest turnoff for me is its awful clunky header file system.  Headers were okay in C where the only thing you could expose were functions, variables, and the occasional struct) but in something as complex as C++ it's horrifying.  Class declarations have to include all private variables (unless you use something like the pImpl idiom, which hurts performance and requires more work to maintain) to avoid segfaults, which has the nasty side-effect of introducing extra dependencies for anything including the header--and that doesn't even begin to scratch the surface of the tip of that iceberg.

.NET does a much better job of managing dependencies.  All you have to do is add the DLL as a reference and (optionally) import the proper namespace(s) and you're good to go.  No messing with headers or import libs or anything stupid like that.

...and yeah, I think I've ranted long enough. :-X
Title: Re: TurboSphere
Post by: Flying Jester on November 19, 2013, 08:01:42 pm
I don't mind headers too much. They are like a promise that the functions will exist at runtime.
There are ways around declaring private data and functions in the headers without PIMPL, but they are not worth effort in my opinion.

I've Update the TurboSphere API page (http://wiki.spheredev.org/User:Flying_Jester/TurboSphere_API) with all the functions I feel comfortable saying are in TurboSphere. Some of them are buggy--I don't entirely trust the rawfile and bytearray objects in particular, and I haven't extensively tested joystick support. I also haven't fully tested the key constants since the change to SDL2, but I haven't seen a case of them being wrong yet.

Known bug: Saving to PNG will probably cause slightly annoying hanging in Windows. That's because there is a horrendous mix of MS tools, make, scons, and cmake involved for me to share the same libpng shared library with SDL2 on Windows. It works on Linux for me, but only on one machine (the other also hangs, even on Linux).

Saving to TGAs is my recommendation when file size is of concern, BMP otherwise. I've tested the TGA implementation fairly well, and the files are fully readable in feh, Windows Image Viewer, GIMP, and gPicView. I'm satisfied with that.

Also, GarbageCollect() finally actually does something since 0.3.5. Not much, but it does send a signal to V8 to do some GC'ing.
Title: Re: TurboSphere
Post by: Flying Jester on November 24, 2013, 01:53:15 am
The sprite batcher is coming along. It automatically sorts new textures into texture atlases to limit texture changes. It can read new textures from surfaces or images.

This will be a pretty slick way to draw certain persistent parts of games simply and quickly, and be the graphical backend for the map engine.
Title: Re: TurboSphere
Post by: Radnen on November 24, 2013, 02:54:39 am
Now that, that will put "turbo" in turbosphere. :)
Title: Re: TurboSphere
Post by: Flying Jester on November 24, 2013, 07:00:52 am
Yes, it will! The biggest thing I can see about the Sphere API that limits performance is that it pretty much looks like a direct translation of array drawing in OpenGL. Which isn't exactly slow, but it's certainly not as fast as using buffers.

The biggest limit on performance in TurboSphere's graphics plugin right now is switching textures. I might just set it up to use the texture-combining part of the sprite batcher for all images/textures.
Title: Re: TurboSphere
Post by: Flying Jester on November 24, 2013, 10:28:22 pm
I've put up a draft of what the SpriteBatch API will look like (http://wiki.spheredev.org/User:Flying_Jester/SpriteBatch_API). I'd appreciate any feedback on it. I've never defined an out-facing API like this before.

What do you guys think? Is the fewer-functions-with-more-options approach better or worse than the old Sphere approach? Should I make it more like the old Sphere API, or is this a better idea?

Anyway, screeny time again!
Texture sorting/fitting and loading spritebatch textures from both surfaces (the red X) and images (the Sphere logo). Loading textures from objects has almost identical overhead to blitting the object.
Title: Re: TurboSphere
Post by: N E O on November 25, 2013, 02:01:49 pm
I'm a little foggy on it; can you explain what SpriteBatch is and what it would be used for?
Title: Re: TurboSphere
Post by: Flying Jester on November 25, 2013, 02:26:29 pm
The sprite batcher is a way to specify a list of graphics operations (image blits and primitives). You then just call its draw operation, and all the operations in its list are done. Blitting images and drawing primitives this way is faster (by a significant amount).

It removes the function call overhead, makes texture atlases, lowers the number of GL state changes (by a lot) because the GL state is known for all the operations in the list and what order they will occur in, and uploads most of the information to the GPU only once.

You could use it for HUDs, menus, even possibly special effects. It is, however, best suited for map and tiling engines, and it will be the graphical backend for the TurboSphere map engine.

On a side note: TurboSphere now has hardware occlusion support. Occluded drawing (things that will be overlapped by something else before FlipScreen is called) is skipped when the hardware supports this. You could see up to a 10% performance improvement in certain cases. Such as that last screenshot I posted. And it's not just limited to images with uninterrupted, total opaqueness--notice the less-than full alpha in the red surfaces.
Title: Re: TurboSphere
Post by: DaVince on December 01, 2013, 11:59:16 am
Sprite batching looks like a clever technique. The API looks a little bit confusing at first, though.

You say "'tex' is the index of the texture to draw", but how do we know this index? Does spritebatch.addTexture() return this index to you? Do we have any better ways to find it, like passing the actual texture file as an argument so it can find (or otherwise add) this image in the array?

Why do addTexture and pushTexture look so different? They seem to do very different things, but their naming is very close to each other, which will only cause confusion. One adds a texture into the buffer, and one adds a drawing operation to the queue, right?

Also, is there any reason you're not overloading anything? You could easily just have an add() function that accepts Image, Surface, TextureOperation or PrimitiveOperation object. Or is this a big no-no to do?
Title: Re: TurboSphere
Post by: Flying Jester on December 01, 2013, 09:03:45 pm

Sprite batching looks like a clever technique. The API looks a little bit confusing at first, though.

You say "'tex' is the index of the texture to draw", but how do we know this index? Does spritebatch.addTexture() return this index to you? Do we have any better ways to find it, like passing the actual texture file as an argument so it can find (or otherwise add) this image in the array?

Why do addTexture and pushTexture look so different? They seem to do very different things, but their naming is very close to each other, which will only cause confusion. One adds a texture into the buffer, and one adds a drawing operation to the queue, right?


Yes, addTexture adds a new texture to the atlas, pushTexture adds a new operation to the spritebatch. I could certainly make it so that addTexture returns an index. The trouble with passing an image into pushTexture is that, while it is quite possible to do when dealing with textures already added to the spritebatch, it could get messy if you pass in any arbitrary image.


Also, is there any reason you're not overloading anything? You could easily just have an add() function that accepts Image, Surface, TextureOperation or PrimitiveOperation object. Or is this a big no-no to do?


I certainly could overload it, no reason not to. As in, have objects for each kind of operation. That's actually a better idea.

On a side note, over the weekend I fixed bugs in AreKeysPressed and GetKey. It was reporting 'phantom' key presses, and GetKey always missed the first key press when the engine started. It's fixed in the Git repo now.
Title: Re: TurboSphere
Post by: Flying Jester on December 05, 2013, 07:49:41 pm
I've added a much better guide to compiling TurboSphere to the Read me on GitHub (https://github.com/FlyingJester/TurboSphere#compiling).

I've been playing around with more advanced OpenGL in TurboSphere. I've almost got it working with OpenGL 3.2 (which is only eight years old...new OpenGL like 3.3, 4.x expects a lot more from the programmer), and have most of the pointer-based drawing code--which only slightly better than raw immediate mode--removed. This change will probably be merged and released for Windows early in the new year.

I've also confirmed the latest release to work with Mesa 3D on Windows. This is excellent if you have an Intel integrated GPU, which have spotty native OpenGL support.

That will also mark the addition of script-exposed Shaders. I'm not entirely sure if I want to make the default shaders baked-in to the executable or not...but I probably won't, since it will save a lot of time debugging if I don't have to recompile the whole graphics plugin to test changes to shaders.

So, since we've discussed shaders in Sphere engines before, here's a question for anyone.

What would you like the exposed shader API to look like?
Title: Re: TurboSphere
Post by: Radnen on December 05, 2013, 10:31:57 pm
The shader API is a hard one to implement and actually stems from your blitting model (API). In traditional Sphere blitting you would do this:
Code: [Select]

Image.blit(x, y[, shader]);
Image.zoomBlit(x, y, z, [, shader]);


But if you do blits differently:
Code: [Select]

Image.shader = shader;
Image.blit(x, y);


Then it get's easier.

Furthermore, as for the shader itself, we have to set and get properties from it, create new instances of it, etc. So by the way you are doing things it would look like:
Code: [Select]

var shader = new Shader('path_to_shader.glsl');
shader.setProperty(0, Image.texture);
shader.setProperty(1, 5.0);
//...


Then you can have a new /shaders/ folder as one of the many standard folders used by Sphere games.
Title: Re: TurboSphere
Post by: Flying Jester on December 06, 2013, 12:03:16 am
I definitely wouldn't want it to be a parameter for drawing operations. It should be, in some way, a part of a per-object and/or global (default) state.

Hmm...as far as per-image shaders, it would also be good to be able to set the current shader for all drawing. Perhaps as well to specify different default shaders for textured and un-textured drawing?

For how a shader would look out of script proper, I've been looking at Byuu's Quark (http://board.byuu.org/viewtopic.php?f=10&t=4152) for instpiration and guidance. I like the idea of a simple manifest file that specifies shader files that compose the shader program. It could also be used to specify certain whether or not to supply certain uniforms to the shader.

What about on the GLSL side? Again, I've been looking at Quark, but I'm not sure about having variable output size. I would like having most of the uniforms that are exposedm, though.

EDIT:
I have to add, this recent bit about of first working on the MapEngine, then the SpriteBatch, and now upgrading the OpenGL version and adding shaders isn't just some bout of lack of concentration or me having a very short attention span.
These are all related issues and concepts. The SpriteBatch is a backend for the MapEngine, which will automate a lot of graphical portions (and several optimizations) for the MapEngine, as well as provide this feature to script. But it works like modern OpenGL does--more like the map engine, less like the normal drawing API of Sphere. And when I switched to OpenGL, I made certain assumptions and decisions of how to do it that really only facilatated the drawing API, which works a lot like OpenGL immediate mode. Which was deprecated in OpenGL eight years ago.

It will probably be best to just get this all done in one clean pass. Remove the deprecated immediate-mode code by switching to OpenGL buffers and arrays, add shaders, and implement a generic texture atlasing and OpenGL array generator. This will make the SpriteBatcher much more efficient, give TurboSphere a much more modern OpenGL implementation, which will also be better suited to the SpriteBatcher, which will make the map engine easier to implement. It's just layers of me discovering what should be done before I work on what I originally planned on doing.

I'm trying to fix fundamental shortcomings and limitations of the current implementation of TurboSphere.
Title: Re: TurboSphere
Post by: N E O on December 06, 2013, 11:02:58 pm
I recommend shaders be global instead of per-object. Imagine the performance hit of even 20 spritesets onscreen with 20 different shaders applied!

A shader could be applied to MapEngine to globally affect all maps within that running map engine, with a ChangeMapShader (or similarly named) function added to change the map engine's shader on the fly. It would take care of most of the use cases for the ColorMatrix object (ie, applying environmental visual transformation effects like fog or dawn/dusk, etc).

A shader could be applied to global spriteset rendering, say to desaturate all the sprites on the screen but not the map during a Shakespeare-like aside or soliloquy or narration and then resaturate them afterward; for doing it per sprite you might as well transform a person's spriteset like before to prevent the hit on performance.
Title: Re: TurboSphere
Post by: Radnen on December 06, 2013, 11:44:05 pm
Yeah, the spriteset level would be best. I wanted per image as because I was thinking of effects like invisibility, and various other powerups get unique shaders for only specific sprites on a map. I also de see the use for global shaders like bloom, color, or fog shaders. I also see shaders for lamplight and other 2d type light sources as well. So a good shader API has to affect all layers of shader use.
Title: Re: TurboSphere
Post by: Flying Jester on December 07, 2013, 02:03:55 am
And whole-scene effects like fish-eye, stereograph 3D, and--most exciting to me--simple lighting effects.

Switching shaders doesn't have too huge a performance penalty from what I've seen, although it's not exactly cheap either. I probably wouldn't want to have them be different for every image, but per-SpriteBatch might be good. That would be comparable to per-spriteset or per-map shaders. And it would be just that on the backend.
Title: Re: TurboSphere
Post by: Flying Jester on December 10, 2013, 05:40:17 am
Behold, TurboSphere rendered using OpenGL Shading Language 1.5. The images and surfaces use buffers instead of pure pointers. Still not full vertex arrays (what I am working towards), though. I needed shading working first.

I'll need to redo all the drawing code. Fortunately, most plugins don't do a lot of it except the SDL_GL plugin. And that one uses the TS_Image object pretty exclusively for textures, so it's quick to modify.
And besides, I want to reduce the redundant primitive functions in SDL_GL anyway.

I've been thinking, and...
It would be trivial to add basic compositing. As in, whole-scene shading that takes the output at the end of a FlipScreen, and applies a shader to it, then draws it to the screen proper. That would possibly be the best way to implement filters and bloom or HDR-like effects. Plus, it would make grabbing the screen and saving screenshots a lot more efficient, although at a slight hit to performance (about one ms per frame, plus time to run a scene shader). I could very easily make it toggle-able at runtime, and disable it when no whole-scene shaders are in use, or even enable it for a single frame to grab a screenshot, and then turn it off again.

That would also make it simple to redirect the video output of TurboSphere to another window or output. I have no idea the use of that, but it would be...something. It would NOT be the best solution for recording, but it would still be more efficient for it than the way things are now.
Title: Re: TurboSphere
Post by: DaVince on December 10, 2013, 01:58:01 pm
Woo, good work. :)


That would also make it simple to redirect the video output of TurboSphere to another window or output. I have no idea the use of that, but it would be...something. It would NOT be the best solution for recording, but it would still be more efficient for it than the way things are now.

I like this idea. Just imagining running games from a launcher, and they're displayed in the launcher window...
Title: Re: TurboSphere
Post by: Flying Jester on December 11, 2013, 01:02:55 am
Compositing is working now. Thing is, I know that you can pass handles to GL contexts around...I just have no idea how it is done.

Fun fact: GL framebuffers don't use the global ortho and projection settings. The colour echoing was unrelated.

Pic just for fun, it's fixed now.
Title: Re: TurboSphere
Post by: Fat Cerberus on December 11, 2013, 09:54:11 am
So all this talk about shaders has me curious. I know shaders are used in 3D games for special effects, but what would be some applications of shaders in a 2D game? I'm straining, but for the life of me I can't imagine any (non-gimmicky) use cases for them.
Title: Re: TurboSphere
Post by: Radnen on December 11, 2013, 11:28:43 am

So all this talk about shaders has me curious. I know shaders are used in 3D games for special effects, but what would be some applications of shaders in a 2D game? I'm straining, but for the life of me I can't imagine any (non-gimmicky) use cases for them.


Fast efficient cave lighting, Bloom lighting, gray-scaling the screen for flashbacks, particle effects, advanced GUI objects (like the health globes of Diablo II), water, ice, invisibility cloaks, fire, a "speedup" or "warpspeed" effect, and lot's more. :P
Title: Re: TurboSphere
Post by: Flying Jester on December 11, 2013, 01:56:10 pm
You can use them for any kind of color or geometric transformation, any at all. Like, if you wanted hue shifts, or to go black and white, or to add some kind of ripple effect (water, or a screen transition, or anything else).

In those screenshots I posted, they are being used for color masks, defining the screen coordinates, and coloring primitives. If you think of the parameters to, say, Rectangle(), the shaders are defining what those values mean. You could just change one line, and the x coord would mean the y. Or the red color channel. Or rotation, or whether or not to do alpha blending, or whatever. The default shaders I wrote just imitate the fixed function pipeline and allow you to use raster coordinates.

Considering an only slightly gimmicky use....

Take the default fragment shader (https://github.com/FlyingJester/TurboSphere/blob/master/bin/Release/system/shaders/system.fs). If, say, you wanted to lower the lighting, and make it look like night. You could use a different shader that looks like this:

Code: [Select]

#version 120

void main(void){
        gl_FragColor = gl_Color/vec4(0.5, 0.5, 0.8, 1.0);
}


If you imagine a color to be a vector of four floating point numbers, each representing the luminosity of a channel (1.0 is like 255, 0.0 is like 0 in 32-bit color), this would cut the red and green channels in half during drawing. So it would look blueish. The point being, it would be pretty slow to post process these kinds of effects (even the trivial example above!) in software. And also much less elegant.

And besides, if you want to use OpenGL that isn't ten years old, you need to use shaders. I'm setting it up so that if you don't want to learn GLSL, you don't have to, either. The default shaders work the way it did previously, you can just pretend it isn't there and you will be fine. But you can write your own shaders and switch between them if you want to.

EDIT:

I've added hardware image cloning, and an alternative software method that will be used when it can't be done in hardware. Makes image cloning much faster, but also much more elegant (only a single function call, no buffers for pixels in system memory). The extension it uses is very common, and really should exist for your GPU if you have OpenGL 3 or greater (guaranteed to exist if your card is OpenGL 4.3 or greater).
Title: Re: TurboSphere
Post by: Fat Cerberus on December 11, 2013, 11:28:03 pm
How do I find out what version of OpenGL my card supports? I just built a machine with an AMD A10-6800K CPU so I'm curious what level of support the integrated GPU will give me.
Title: Re: TurboSphere
Post by: Flying Jester on December 12, 2013, 12:23:53 am
You can check out GPU Caps Viewer, that can tell you detailed things about your GPU. In linux land, there is always glxinfo.

I can tell you that an A10 supports at least 4.2. My laptop's A10 supports 4.3.
Title: Re: TurboSphere
Post by: Mooch on December 17, 2013, 07:03:04 am
Just reading through the TurboSphere API for the first time. I like it, it somehow seems more tightly-plotted than the Sphere API, if that makes sense. Also, I like the, um, how to describe it, formatting style(?) you use. Like, "var font = new Font("filename.extension")" instead of "var font = LoadFont("name.ext")" That's the way most programming languages do it, it's more intuitive for people coming from other programming languages.

I noticed that some of TS's API functions, however, have fewer parameters than the comparable Sphere ones. Just curious if those were deliberate choices and there's different ways of setting such options, or if you just haven't gotten around to adding those features yet. For example, the Sphere Polygon primitive has [invert] as an optional parameter, the TS doesn't.

Oh, also, are there system default fonts/windowstyles/arrows/etc. in TurboSphere? I'm guessing not, 'cause I'm guessing, as the name implies, TS is meant to be fast, and having default stuff like that would probably slow down execution. Just curious.

I've been trying to convert to Linux lately. I'm definitely gonna use TurboSphere as my primary Sphere engine in Linux. Though I'll probably wait 'til the first full release. I've got enough problems programming in the relatively-stable Sphere 1.5. I'm not brave enough to use an alpha anything just yet :p
Title: Re: TurboSphere
Post by: Flying Jester on December 17, 2013, 08:28:32 am
I've been trying to streamline the API, although I really haven't done too much in that respect. Most things that are missing (especially the map engine) are missing because they just haven't been added yet.

I have, however, tried to generalize the API more, and to make it look more like JS. For instance, the proper constructors for engine objects, some overloaded functions like `Surface`, etc.

Defaults can actually improve execution time. Since more is known about them, certain optimizations can be made that would be expensive or possibly counter-productive if applied to any arbitrary situation.
GetSystemFont, GetSystemTTFFont, GetSystemWindowStyle all exist. GetSystem(Up|Down)Arrow has been added to the bleeding edge, but not the last release. The API document is about 95% of what exists in TurboSphere. The other 5% is some stuff I forgot to put in, and some stuff that is secret  8)

For the polygon's invert, that's something I do want to add (it's actually the same technique that makes the 'complex' primitive). It's just that it is something much easier to do with Sphere (software/non-buffered hardware primitives, monolithic engine) than with TurboSphere. I'll get around to most of the Sphere API eventually, I've just partially focused on adding things that make other things possible so far (i.e., Image.transformBlit makes Image.rotateBlit possible by recreating it in script, so Image.transformBlit is more important to add). By the same token, you could use Polygon to do the same thing as an inverted Polygon, so it's lower priority than a lot of other stuff.
Title: Re: TurboSphere
Post by: Radnen on December 21, 2013, 10:23:41 pm
Hey, what is the overall time for TS to run this, I'm just curious:

Code: (javascript) [Select]

function Alert(text) {
    Rectangle(0, 0, GetScreenWidth(), GetScreenHeight(), CreateColor(0, 0, 0, 0));
    sys_font.drawText(0, 0, text);
    FlipScreen();
    GetKey();
}

function TestRawSpeed() {
    var t = GetTime();
    for (var i = 0; i < 10000000; ++i) { }
    t = GetTime() - t;
    Alert(t);
}


Sphere: 564ms
SSFML: 19ms (29.5x faster)
Title: Re: TurboSphere
Post by: Flying Jester on December 22, 2013, 02:09:46 pm
I get 20-21ms in TurboSphere on my Laptop. For comparison, Sphere 1.5 on my laptop gets 2771 (on Wine, mind you).

Also, TurboSphere now comes with a command-line SPK unpacker. I'll probably end up packaging it with tsconf, since I'd like to give it an FLTK graphical frontend. I'd also like to make it into a full SPK editor, but that's a ways off.

To use the spkeditor:
Code: [Select]

./spkeditor
options:
-x      Sets to extract mode. No other modes yet, unfortunately.
--if=[] Specify SPK File
--of=[] Specify Target Directory
Title: Re: TurboSphere
Post by: Radnen on December 22, 2013, 04:00:26 pm
Sweet, it seems for that example TS is 131 times faster. v8 is really fast!! :)

Hey so, do you have a good Windows port ready?
Title: Re: TurboSphere
Post by: Flying Jester on December 22, 2013, 04:32:49 pm
I can't really do that until the new year. I need my desktop's Windows install (as well as its MSVC) to do that, and I'm on holiday out of town right (and until the end of the first week of January). I'll actually probably be out of touch during that time, too.

Of course, you could try compiling it (https://github.com/flyingjester/turbosphere#how-to-compile-turbosphere), too. I've added that guide that details what the entirety of compiling TurboSphere, including compiling all of its dependencies, on Linux and Windows x86 and amd64. But given I haven't tried compiling the source on Windows in about a month, it probably will have some issues that need to be resolved (I know at least the SPK code will, I haven't added the Win32 code to create directories yet). And then, of course, it will probably still have a few bugs that will need to be ironed out.

I'd like to get tsconf working properly with shaders while I'm away, and ideally add the FLTK frontend to my spk unpacker. And file adding and removing from spks. And get TS fully working in OpenGL 3.2. Well, some of that will probably happen. When I have no internet, I tend to be forced to actually work on things.

Also, I tested, and V8 isn't actually optimizing the loop away (also, very strangely, running a loop over 256k times seems to make the V8 hi-optimizer cut out for the loop's condition+iterator!). Makes me wonder, does Jurassic optimize the loop away?
Title: Re: TurboSphere
Post by: Radnen on December 22, 2013, 05:01:14 pm
Jurassic has very few optimizations in it, sadly. It most likely doesn't optimize the loop, besides running it (with boxing) as if it were a .NET for loop. In C# .NET I get 3ms. So pure .NET is still much faster.

Actually, I'm not sure TS and Jurassic are going to be much different speed-wise. When Jurassic compiles JS to CIL, the .NET compiler does optimizations of it's own too. That said there are certain JS to .NET optimizations not being made, like static casts and static function generation.

For example: if the var of a for loop was only ever going to be treated as an int, then treat it as an int rather than a double. In .NET a double for that for loop example, runs at 11ms, which is considerably slower than the 3ms of the int, which is closer to the speed of the compiled version. I'm surprised yor v8 version isn't much faster (in the single digits). But ultimately I'm surprised how dang slow Sphere handles that. I would have thought - even for an older interpreted language - a simple for loop would be pretty fast (like a hundred ms, not 500 to 2000 depending on the computer).

BTW I'm running this on a rig with a 3.3ghz i5 processor.
Title: Re: TurboSphere
Post by: Flying Jester on December 27, 2013, 04:13:35 pm
I'm running a 2.4 GHz A10 (I'm going to guess this is somewhat similar to a 2.0 GHz i5?). And I am using Wine, although the only two things that seems to slow down with Sphere is startup time and the non-GL drivers.

I've got the FLTK frontend, SPK creation (not direct editing yet, though), and xz decompression as an alternative to zlib (although such a file must be made by hand right now) all working. I'm thinking it would have been nice if Chad Austin had left more reserved space in the SPK header--or any at all in the file headers themselves! It would be really nifty if you could specify the compression technique on a per-file basis.

I've also got TurboSphere set up to unpack SPKs into temp directories to run games from them. Ideally I'd use an in-memory fs for that, but that'll take more work.

I've exposed putting raw data into audio streams through ByteArrays. This means you could use JS to make any kind of sound-chip emulator (or any kind of effect, theoretically).
Title: Re: TurboSphere
Post by: N E O on December 27, 2013, 04:45:14 pm

I've exposed putting raw data into audio streams through ByteArrays. This means you could use JS to make any kind of sound-chip emulator (or any kind of effect, theoretically).


You mean like my nwavedit demo (https://drive.google.com/folderview?id=0B-5bW82jK5Z5aGY2VFVBZkFEdjg&usp=sharing&tid=0B-5bW82jK5Z5UGZaS0c1OFJZZDA) or like my SN76489 emulator (http://www.luxatom.com/vst-psg.php)? ;)
Title: Re: TurboSphere
Post by: Flying Jester on December 27, 2013, 05:21:41 pm
Precisely. Well, ideally precisely.

I'm having some trouble getting it to work perfectly, but that's an issue with me not quite being sure how this part of Bass works. I've got it reading sound data from wav files fine, with the data being pushed by JS rather than through a direct Sound object. Theoretically, you just need to push PCM sample data to it. I think it could also use FFT data, or ogg files, or anything it can play normally, but I'd expect PCM to be the simplest to craft in JS.

It needs more tweaking, but it's a pretty simple thing to add. And I think it has a chance of being quite useful. You could even stream music off the network with this (that's probably the easiest thing to do with it right now).

While I'm here (internet is very poor in rural Alaska), here's what the new TurboSphere Shader API looks like:

Code: [Select]

//Shader object:
shader.source //A string representing the shader's source. You can modify this, but changes only take effect if you subsequently make a new shader program using this object, or call shader.compile().
shader.name //A number representing the OpenGL name of the shader. Read-only.
shader.compile() //Compiles the shader. This is automatically done with ShaderProgram(), but you can check the validity of the shader using this. It throws JS errors.

//Creating shaders.

var f = new FragmentShader(source);
//Creates a fragment shader from source.

var v = new VertexShader(source);
//Creates a vertex shader from source.

//Compiling GLSL programs

var p = new ShaderProgram(frag, vert);
//Creates a shader program from the shader objects 'frag' and 'vert'.

var p = new ShaderProgram(file);
//Loads a shader program from a 'shade' file.
//See https://github.com/FlyingJester/TurboSphere/blob/master/bin/Release/system/shaders/system.shade
//And also https://github.com/FlyingJester/TurboSphere/tree/master/bin/Release/system/shaders
//For what that looks like.

//Setting and getting shaders

UseShader(p);
//Sets 'p' to be the current GLSL shader.

UseCompositeShader(p);
//Sets 'p' to be the current GLSL shader used for compositing (when compositing is enabled).

var p = GetSystemShader();
//Gets the system shader program. This is the one used by default.

var p = GetSystemCompositeShader();
//Gets the system shader used for compositing. This is probably, though not necessarily, the same program as the system shader.

var p = LoadSystemShader(file);
//Loads the .shade file 'file' from the system/shaders directory.


I'm still working on setting vertex attributes and uniforms from script. See https://github.com/FlyingJester/TurboSphere/blob/master/bin/Release/system/shaders/system.shade (https://github.com/FlyingJester/TurboSphere/blob/master/bin/Release/system/shaders/system.shade) and https://github.com/FlyingJester/TurboSphere/tree/master/bin/Release/system/shaders (https://github.com/FlyingJester/TurboSphere/tree/master/bin/Release/system/shaders) for what .shade files look like. There will be more parts to them, once you can set uniforms and vertex attributes, use geometry shaders, etc., from script.
Title: Re: TurboSphere
Post by: N E O on December 27, 2013, 07:31:07 pm
I could've sworn one of FBNil's last SoundEffect API changes was the ability to craft a sound from raw sample data...
Title: Re: TurboSphere
Post by: Fat Cerberus on December 27, 2013, 10:21:08 pm

I'm running a 2.4 GHz A10 (I'm going to guess this is somewhat similar to a 2.0 GHz i5?). And I am using Wine, although the only two things that seems to slow down with Sphere is startup time and the non-GL drivers.


The A10 is roughly equivalent to an i5, yes.  The machine I just built has a 4.1 GHz(!) A10-6800K and I can tell you that thing is fast.  The i7 is better for anything excessively CPU-bound (like video transcoding), but I've found that the AMD chips have better iGPUs and are less expensive, so I tend to gravitate towards them.
Title: Re: TurboSphere
Post by: Flying Jester on December 30, 2013, 09:45:20 pm
If you want to use OpenGL, AMD integrated graphics is lightyears ahead of Intel. Mostly because Intel seems to think that you don't actually need to implement the entire core OpenGL profile in order to claim compliance, so they are missing rather useful features (like proper vertex arrays as late as SandyBridge).

I've updated TurboSphere on github. The shader objects are exposed and updated the default shaders to no longer rely on deprecated projection stack variables. I've made the surface thread properly sleep when not doing anything, so TS doesn't burn down two cores at idle. I've also tweaked the V8 usage, which removes some overhead (it was causing bumpy CPU usage, even at idle, keeping stackframe info valid in case of fatal internal errors). This puts simple rendering loops to the technical maximum for TurboSphere, 2000 FPS. Until I fix the rest of the OpenGL usage, that is the hard limit on FPS (at least on AMD machines, haven't tested on NVidia in a while).
Title: Re: TurboSphere
Post by: Fat Cerberus on December 31, 2013, 01:31:57 am
I suppose it helps that AMD has the entire ATI legacy behind them. Intel seem to build their graphics hardware in-house and thus only really have their dismal motherboard graphics (which, it should be noted, was also notorious for not implementing stuff--in Direct3D, no less!) to build on.
Title: Re: TurboSphere
Post by: Flying Jester on January 03, 2014, 08:39:04 pm
If you ever try to set up AMD GPUs on Linux, you can see the ATI legacy alive and well. A vast majority of their tools and drivers are copyrighted AMD, but branded as ATI still. The only case I can see of a Unix tool being rebranded as AMD beyond something very superficial is the new amdcccle, which 'replaced' aticonfig. But it can't do a lot of the things that aticonfig can, and so you still need aticonfig (mainly for generating the default x.org file). AMD doesn't seem to have really built too much new stuff, just improved older ATI work (at least in software, to be sure).

Anyways, I've exposed screen width and height as uniforms to shaders. That means the default shaders don't use the deprecated projection stack. Only the addition of vertex attribute definition, and they will be fully OpenGL 4.1 (at least) compliant.

Edit:
It seems like the best changes always result in the least interesting screenshots. Anyway, TurboSphere, fully OpenGL 3.3 compliant. That, along with the minor change to the V8 usage, has made things pretty Turbo!
I have to rewrite small parts of just about every graphical function, but it's a good opportunity to make the primitives less redundant, and to fix some of the stuff left over from back when TS used software rendering (I'm thinking of the WindowStyle plugin mainly).

Kind of funny, TurboSphere went through software rendering, immediate mode, collected pointers, and now uses vertex arrays and fully buffered graphics data uploads. That's an awful lot like the general history of graphics rendering API techniques, right there. And I can say with certainty, that each change improved both speed and elegance.
Title: Re: TurboSphere
Post by: DaVince on January 04, 2014, 07:12:33 pm
Awesome! Good work. I bet you've learned a lot about all of these rendering types, too. ;)
Title: Re: TurboSphere
Post by: Flying Jester on January 05, 2014, 04:43:14 pm
I have learned a lot. I would certainly say I've learned enough to know why it's done the way it is now!

And it certainly hasn't been wasted time, either. The things I learned from software rendering are still quite applicable to the surface code directly--not to mention that it was a valuable learning experience about data transformation in general. The immediate mode and pointer-buffer coding was useful because it taught me about how some things should look when you are using modern, flexible (i.e., write it all yourself) OpenGL, and because you can see where a lot of more subtle bottlenecks are more easily that way.

Apparently, it's also OpenGL 4.3 compliant, I'm just not really sure how many computers out there can do 4.3 (for instance, my laptop can do 4.3, but my much more powerful desktop can only do 4.1). I think I'll leave it using 3.3, since that's pretty common.
Title: Re: TurboSphere
Post by: Flying Jester on January 06, 2014, 05:51:01 pm
Attached is a copy of the TurboSphere SPK (no gui) extractor, win32 binaries.
That copy of T5.dll in the same directory as turbosphere.exe will make older TurboSphere binaries very angry, though.

I've updated the Sphere system scripts that are distributed with TurboSphere to actually run on TS with no extra modifications. I've also added the long-fabled compatibility script (sphere15.js, a system script) that makes the new TurboSphere objects constructable using the old Create* and Load* functions.

I should have the FLTK (gui) SPK extractor win32 binaries up tomorrow or the next day, and some new, slightly unstable TurboSphere win32 binaries up a few days after that.

New stuff for the upcoming (0.3.6?) TurboSphere:
* Typed Arrays finally enabled!
* Fully OpenGL 3.3/4.3 compliant code (which is really fast!)
* Shaders exposed to script
* Many graphics functions will be missing :( I need to fix a lot of them up to work with OpenGL 3.3
* Running games from SPK files
* Fully updated system scripts for TurboSphere
* New Sphere 1.5 compatibility script included as a system script
* Fixed PNG saving
Title: Re: TurboSphere
Post by: Radnen on January 23, 2014, 01:48:50 am
So, I have no idea how fast TS is now, but the version on my computer is very slow. Showing some strings and a windowstyles brought it to 270fps. But even then - for that version - I was expecting better performance. I really want to know how much you've improved it by, and I'm sure it's going to be quite a bit! See, I thought you were mulling over a few fps here or there like I am, but it seems you have really revamped your graphics backend.

Also, have you started the map engine yet?
Title: Re: TurboSphere
Post by: Flying Jester on January 23, 2014, 02:12:38 am
What version do you have? All the 0.3.5 on up are pretty similar (and about 80% performance of what it is now, although initially loading images takes a little longer now than it used to).

The test script get's about 1000 fps on my laptop (showing two windowstyles, blitting three surfaces without converting them to images, playing a midi, writing several strings using both TTF and BMP fonts, doing zoomed, masked, and rotated blits). WindowStyles should actually be medium speed, they do some silly things, but only have a single draw per component by using greater-than-one texture coordinates. They will, however, get a bigger boost from the new graphics backend and fleshed out spritebatcher (atlasing their components makes perfect sense).

If it is 0.3.5, you can try disabling compositing in engine.ini (set it to zero). That kills performance on AMD machines, I noticed, though not as much on NVidia. Setting the scale has a small effect (TurboSphere in 0.3.5 is either cpu limited or fill limited, depending on what the scripts do). If you aren't getting over 1000 fps on the test script with a Radeon 8750, GeForce 550 ti, or better, I'd like to know more about what is happening there. I get over 1000 fps on my 550, right around 1000 on my 8750.

The new graphics backend is still only about a 20% improvement of the final 0.3.5. I'm adding it because, for one, it uses OpenGL that isn't nine years old, and for two it makes the sprite batcher (which will power the map engine in particular) much, much faster.

You can open maps and spritesets in TurboSphere (a spriteset sprite is rendered through the Sphere API in the 0.3.5 test script), and calling MapEngine will draw the map (but slowly, it's just a test of map loading). I have all the code to load maps and spritesets, and some code that I will use as reference for the actual map structure. But really, it's just prototype stuff. Nothing optimized, nothing really the way I want it to be, just feeling out what it should be like.

I have kind of been stuck with the backend, though. Mostly because fully updated Arch Linux can't use proprietary AMD drivers at the moment (it has Wayland compatibility in the X server, which crashes fglrx). So since the weekend I've only been able to use Mesa drivers. Which are slow.
Title: Re: TurboSphere
Post by: Radnen on January 23, 2014, 02:24:55 am
It might be veryold, I don't see a version number anywhere - not even in a readme. It's definitely 0.1.* though.

I'll try a 0.3.* version.

Dang. I was just about to finish a download and the file's server goes offline. Welp.
Title: Re: TurboSphere
Post by: Flying Jester on January 23, 2014, 02:32:50 am
Does it have an SDL_GL_Threaded.dll plugin in the `plugin` folder, and does the executable have the actual turbocharger logo as its icon?

Without the threaded graphics plugin, it's pre 0.3.x. With the turbocharger icon, it's either extremely new (the threaded plugin will indicate this), or extremely old.

The latest release is on SourceForge (http://sourceforge.net/projects/turbosphere/). It's not really what I even have as stable on GitHub (which is still well behind what I currently have), but it should be relatively close.
Title: Re: TurboSphere
Post by: Radnen on January 23, 2014, 02:43:36 am
0.3.0 does not work. Well, it does for a few seconds but crashes straightaway. And I can't seem to download anything else from sourceforge.
Title: Re: TurboSphere
Post by: Flying Jester on January 23, 2014, 03:01:02 am
There is also a copy on FJE at flyingjesterentertainment.webs.com/ts-0.3.5c.rar (http://flyingjesterentertainment.webs.com/ts-0.3.5c.rar).

Looking at 0.3.5c (the latest Windows release), I've been really terrible about making binary releases. That's a pretty old build, from just before I started using shaders.

I haven't really been making binary releases that often anymore because most of the changes are not visible. It's similar to when I was building the plugin system. Lots and lots of coding, much of it untestable until it's all done. And in the end, it all looks the same as when started. It will mean that the final product will be faster and more stable, and be finished sooner. But the change, in and of itself, has no noticeable effect.

In this case, it's actually worse from an outside perspective. All that works graphically in the bleeding edge branch of TurboSphere on GitHub, and I mean all that works, is image.blit and Rectangle. But oh-me-oh-my-oh, image.blit and Rectanlge are as absolutely efficient as they can be. Now I need to finish the automatic texture atlasing in the sprite batcher, and image.blit will be done. Then I can make all the other primitives work just like Rectangle, and make the changes for the rest of image and surface drawing functions.

Then I will have TurboSphere doing all the things it did before I started. That's why I haven't made any releases in a while. You could see what I have now, but you would wonder where the entire Sphere API went, and why calling the wrong function segfaults it.
Title: Re: TurboSphere
Post by: Radnen on January 23, 2014, 04:57:32 am
In 0.3.5 I get this error: glGenBuffers is not present in OpenGL Library.

And The 7-zip log was full of issues when extracting your .rar file. It still managed to extract the files though.
Title: Re: TurboSphere
Post by: Flying Jester on January 23, 2014, 05:26:50 am
That's probably because it was packed using unrar.

Are you using an Intel graphics card? I know that NVidia and AMD drivers have glGenBuffers. That's a pretty basic 3.x function, and almost always present in 2.x. I would say that it is necessary, but if you don't have an Intel card, I'm not really sure what to say about that.

EDIT: Does it say anything about what version of OpenGL it detected (usually appended to a vendor string)?
Title: Re: TurboSphere
Post by: Radnen on January 23, 2014, 05:43:48 am
Dude, I have an Nvidia 550Ti, it supports open GL 3.x I have had no issues with GL anytime in the past. (check out the cpu-z by clicking the link in my signature). It only did that in the plugin mapengineGL.DLL, I removed the offending file and now it says:

Quote

Uncaught ReferenceError: GetSystemFont is not defined
Error occured in script TS Test at line 30:
var fn = GetSystemFont();


I have to admit that is certainly a very informative error message!
Title: Re: TurboSphere
Post by: Flying Jester on January 23, 2014, 06:21:26 am
That's odd that glGenVertexArrays isn't found, I test all my Windows builds using an NVidia GTX 550 ti!

That's bmpfont error very strange...I actually don't know why that is happening. The only reason that should happen is if bmpfontGL.dll wasn't being loaded right. Does it mention that plugin on startup in the terminal?

It'd have been more informative if I had realized that it wasn't reporting script names properly before I released 0.3.5c.
Title: Re: TurboSphere
Post by: Radnen on January 23, 2014, 06:33:11 am
Quote

$ ./turbosphere.exe
game.sgm is the sgmname. We are running in / mode.
[Engine] The main script is startup/scripts/Jest_Main.js
Plugin SDL_GL_threaded.dll is not open.
Error: 1114
Plugin audioBASS.dll is open.
Plugins:        1
Plugins:        1
Plugin bmpfontGL.dll is not open.
Error: 998
Plugin inputSDL.dll is open.
Plugins:        2
Plugins:        2
Plugin networkTS.dll is open.
Plugins:        3
Plugins:        3
Plugin scriptfs.dll is open.
Plugins:        4
Plugins:        4
Plugin ttffontGL.dll is not open.
Error: 127
Plugin windowstyleGL.dll is open.
Plugins:        5
Plugins:        5

Running Script.


Game function done running.


Context exited.

Plugins:        5
Plugin Closed:  0
Plugin Closed:  1
Plugin Closed:  2
Plugin Closed:  3
Plugin Closed:  4
All Plugins Closed.
Fatal Error
Uncaught ReferenceError: GetSystemFont is not defined
Error occured in script TS Test at line 30:
var fn = GetSystemFont();
Segmentation fault


So that's weird.
Title: Re: TurboSphere
Post by: Flying Jester on January 23, 2014, 07:05:33 am
Well, that explains it. No BMPFontGL, no GetSystemFont. And no SDL_GL_Threaded (wtf?), not much of anything. And you say the rar had a bunch of errors...I'll see about repackaging it as a zip or tar.gz or something.

EDIT: Here it is as a 7z: http://flyingjesterentertainment.webs.com/ts-0.3.5c-win32.7z (http://flyingjesterentertainment.webs.com/ts-0.3.5c-win32.7z). Windows has some serious issues with zips made on Linux, so I'm kind of limited. This 7z is a mirror of the SourceForge download.
Title: Re: TurboSphere
Post by: Fat Cerberus on January 23, 2014, 11:45:32 am
Zip is a standard format that's been around forever. Unless Linux is using some exotic compression method (readable: not deflate) then every zip utility known to man, including PKZIP, should be able to extract it. Unless you're using some of the enhanced zip features first introduced in winzip (e.g. 64-bit)?
Title: Re: TurboSphere
Post by: Radnen on January 23, 2014, 11:46:51 am
7-zip says it cannot open that as an archive. I wonder if others can? Maybe there was an issue with either your upload of it or my download. I do commonly see a "no data received" error when trying the download link. Then sometimes it works and takes forever to download (though I have a fairly fast connection). Weird.

Edit:
for your original archive, 7-zip gave me these errors:
Quote

0 Unexpected End of Archive
1 <Filepath>
2 CRC failed in 0.3.5c\zlib1.dll'. File is broken.


I presume other files were broken too.
Title: Re: TurboSphere
Post by: Flying Jester on January 23, 2014, 04:59:28 pm
???

What is happening here?

I would get it if it was just FJE that was being weird, that host is fairly bad about downloads, but SourceForge too? I can download all three, and they all can be extracted fine over here. That should work.

@Lord English: It's Windows that is being non-standard. WinRAR, unzip, 7z, and every other utility can open them fine, but the built-in zip utility in Windows throws a fit about zips made in Linux. Or pretty much zips made with anything but itself.
Title: Re: TurboSphere
Post by: Radnen on January 23, 2014, 05:46:41 pm
No matter what browser or what host I use (SourceForge or your site), or archive I pick 0.3.5c cannot be extracted. :/

I am downloading files correctly. I just tried downloading some archives of mine and they all work and extract just fine. So, I don't know exactly what is happening with your files. I wonder if it is anti-virus related?
Title: Re: TurboSphere
Post by: Flying Jester on January 23, 2014, 06:03:57 pm
It really might be. All the executables and libraries are UPX compressed, and the engine executable has a 'hacked' icon since I have no idea how to use scons to give my executable one. And it doesn't help that there are a pile of libraries and a couple executables that the anti-virus program has almost certainly never seen before.
Title: Re: TurboSphere
Post by: N E O on January 23, 2014, 06:07:45 pm
Re app icons - you need a properly formatted resource file for Windows executables to have a custom icon. My sample phoenix app (https://github.com/apollolux/hello-phoenix) has one you can copy from.
Title: Re: TurboSphere
Post by: Flying Jester on January 23, 2014, 06:28:09 pm
But how would I use that with Scons? I've used resource files to give programs icons before, that's how the older releases had the turbocharger icon, but I have no idea how to use them with Scons, they just seem to be ignored.

I didn't try all that hard, mind you. I had used the resource hacker before, so I knew how simple it was to do it that way.

EDIT @Radnan: You could try using the bmpfontGL.dll, SDL_GL_threaded.dll, and ttfffont.dll from the 7z copy with the rest of the rar copy. But that is a pretty jagged solution.
Title: Re: TurboSphere
Post by: Flying Jester on January 26, 2014, 03:29:07 am
OK, I'm going to compile the source from GitHub for Windows. It has some of the improvements from the bleeding edge--particularly fully modern shaders, command-line configuration setting, and a slightly newer V8 version. Then I'll begin working on getting the bleeding edge ported over to Windows.

@Radnen, Re no glGenBuffers: Are you using the NVidia branded driver or the one that Windows installs automatically? The NVidia one has a new OpenGL library, and I'm not sure that the MS distribution of the drivers has an updated OpenGL library from the Windows default one (which is OpenGL 1.1). That's the only reason I could think of that a 560 ti would not be capable of vertex buffers. I don't check against extension lists, I check for exposed functions in the OpenGL library itself.

The other possible issue is that your card or drivers or OpenGL library won't open a 3.1 context. Without that, TS ends up with a 1.1 context, which will cause this kind of issue. I find this a suspect reason, though, since you have a 560 ti, and my 550 ti's can do it. The driver for a more advanced card should have at least as high an OpenGL context capability as older cards. Or so I would assume?
Title: Re: TurboSphere
Post by: Radnen on January 26, 2014, 04:56:05 am
I run Dolphin emulator in OpenGL, I would know if I didn't have a 3.* context. :/ I'm using official nVidia drivers. It could be that the DLL was messed up in the unzipping? But then again it wouldn't even load if that were the case. I really don't know why it said that.

I downloaded a tool called OpenGL Extensions Viewer and it says I indeed have access to glGenBuffer. I get a 100% feature test pass on my gfx card. Now, what's interesting is my on-board graphics is only OpenGL 1.1 compliant. Are you sure TS is using the correct rendering device? There is a difference between choosing the first device and choosing a supporting device.
Title: Re: TurboSphere
Post by: Flying Jester on January 26, 2014, 07:14:35 am
TS uses whatever SDL2 uses. On my laptop with dual, dissimilar GPUs, it uses whichever one the drivers use for display 0 in GLX. I assume it is similar in Windows, whichever GPU WGL will give it as a default. SDL2 does have more advanced support for choosing version of OpenGL, which I use to specify a 3.1 context. I don't know if it will try using other devices if the first one can't do it.

All a file needs in order to be successfully loaded by TurboSphere as a plugin is a valid executable library header that specifies the symbols Init, GetFunctions, GetFunctionNames, GetVariables, and GetVariableNames. It will, of course, fail quite spectacularly if those symbol definitions are lies.

This is too perfect of an error for it to be corruption. I'm going to assume I made some typo in the GL funciton loading code somewhere, and it's saying that glGenBuffers is missing when it's really something else completely. It would still have to be something a 550 has that a 560 doesn't. Quite possibly the NVidia texture cloning extension I use, which apparently has been superseded by a a core function, so an older card would have it but a newer card might not. Originally, I assumed that, since my AMD 8750, 8550, NVidia 8800 and 550 ti all had the NVidia variant, and only the 8550 had the GL variant, I could safely use the NVidia version. I recently changed that to try and get the NV function, then if that fails the GL function, then fall back to software if that fails. I don't think that change made it into 0.3.5c.
Title: Re: TurboSphere
Post by: N E O on January 28, 2014, 09:18:42 am
SDL uses the first device instead of the default device. byuu ran into a similar problem when writing multi-monitor support recently into higan. If I were on my laptop I'd link that thread.
Title: Re: TurboSphere
Post by: Flying Jester on February 07, 2014, 03:12:16 am
I've almost finished updating TurboSphere to use V8 3.24. At first I was very annoyed at how much the API has changed, but I'm actually glad it did.

A few things that make embedding it easier in TurboSphere:

- The C++ representation of a JS function no longer returns anything. Instead, the argument object is passed in has a new member, which is set to give the function a return value. This makes exporting the functions from libraries with C linkage much easier in MSVC.
- Everything cares about isolates now. This would normally be used to manage different scripts being executed at once--as in, multiple web pages running their own scripts. But it will make it much easier to have TurboSphere manage the lifetime of JS objects when games are changed.
- Everything is UTF8 instead of allowing ascii. This doesn't really help embed it, it just makes things nicer since I wanted to use UTF8 all the time from the start. So now, TurboSphere fully supports UTF8. I could also make it support UTF16 with just a single switch, but I'm not really sure anyone uses or wants UTF16.
- Persistent handles are impossible to use, but locals and handles work much, much better. Which is good, since persistent handles are a flawed thing.
Title: Re: TurboSphere
Post by: Flying Jester on February 12, 2014, 06:24:34 pm
The changes were big enough that it pushed me to revise the plugin tools. Which is shaping up nicely (https://gist.github.com/FlyingJester/8966643).

I don't know when I will make a Windows release. I was considering just wrapping up what was on Github and releasing it for Windows, but that actually hits a bug in the old version of V8 that only occurs on Windows (there's a simple workaround...but blah!). So I think I'm just going to keep on pluggin away in Linux, and once I get new V8 all settled in, and the new graphics backend all stabled-up, then I will port it all to Windows. I did backport the missing stuff to build the SPK editor already, so there isn't anything completely missing from either platform at the moment.

And just to be clear, the plugin tools are NOT required to actually make a plugin. You could write a plugin without them at all (getkeystring doesn't use them, for instance). I'm rewriting them to make them more suited toward the current version of V8, and to fix several issues that became apparent with the old implementation. I'm much happier with how it looks now.
Title: Re: TurboSphere
Post by: Flying Jester on February 13, 2014, 05:11:12 pm
I'm also cleaning up the build process. With the new Sconstruct, if you wanted to make a plugin, you would have two options:

1. Go rogue and do everything yourself. Always an option :D

2. Add a directory to the TurboSphere source, in plugins/. Then, if you have a sconscript or a makefile in that directory, just running scons on TurboSphere will attempt to build your plugin as part of the build process. This has the advantage of guaranteeing V8 and T5 linkage that is consistent with TurboSphere, automatically directing the preprocessor to the plugin tools directory, and installing your plugin and any libs you are building yourself automatically (to bin/Release/plugin and /usr/lib64/turbosphere on Linux, and to bin/Release/plugins and bin/Release on Windows).
Title: Re: TurboSphere
Post by: Radnen on February 13, 2014, 06:22:19 pm
Man, just to break up the monotony of triple posts I'm going to say good work on the plugin interface. Making plugins easier to create is always a plus, and adding it to the build routine is always nice too.

For my Sphere Editor, I made two libraries: the library for the editor and the library for the Sphere components and plugin handling. In this way I can instead make a new blank project that you can check out to make a plugin. However I have yet to do that, but that is theoretically where I'm going with it. In that way you don't need the full code library of the editor to make a plugin, just a small subset of the codebase.
Title: Re: TurboSphere
Post by: N E O on February 14, 2014, 07:39:14 pm
I'm getting an APPCRASH when running TurboSphere 0.3.5c (newly downloaded). Problem details as follows:

Quote

Problem signature:
  Problem Event Name:   APPCRASH
  Application Name:   turbosphere.exe
  Application Version:   0.0.0.0
  Application Timestamp:   527f07aa
  Fault Module Name:   v8.dll
  Fault Module Version:   0.0.0.0
  Fault Module Timestamp:   5258cbe5
  Exception Code:   c0000005
  Exception Offset:   0014957a
  OS Version:   6.1.7601.2.1.0.274.10
  Locale ID:   1033
  Additional Information 1:   0a9e
  Additional Information 2:   0a9e372d3b4ad19135b953a78882e789
  Additional Information 3:   0a9e
  Additional Information 4:   0a9e372d3b4ad19135b953a78882e789


The console gets to the [ bmpfontGL ] test with Font.drawTextBox then crashes. The actual engine window seems to have everything the startup tries to draw, though. Let me know if this was a thing that's fixed in current sources, though, or if there's a way I can get you more info for it, thanks!

Windows Server R2 Enterprise SP1.
Title: Re: TurboSphere
Post by: Flying Jester on February 14, 2014, 07:45:32 pm
Huh. That's not something I've seen with 0.3.5c before. But I am quite used to drawTextBox causing crashes.

font.drawTextBox uses black magic. In very certain circumstances, I write to an address I know I shouldn't, but using glibc it always works and MSVC it usually works, and it makes things much simpler. It's a very bad idea, I just haven't gotten around to fixing it yet. That and I can't make sense of the madness that is the code for drawTextBox, so I need to just rewrite it, really.

Try commenting out the call to drawTextBox in the test script, if it runs fine then it's the non-standard stuff in drawTextBox that is failing.

Can I see the output on terminal for it failing?

EDIT: At long last, my old dream of having functions with full specified JS call signatures, and object-orienting the C++ representation of JS types is realized with the new plugin tools.
This is how I imagined the plugins to work all along. I just didn't know enough at the time to make it work that way.
Title: Re: TurboSphere
Post by: N E O on February 14, 2014, 10:46:31 pm
The original output I'm getting:

Quote

[ConfigManager] Info: Fixed plugins: 1
[ConfigManager] Info: Fixed plugins as recorded: 1
game.sgm is the sgmname. We are running in / mode.
[ConfigManager] Info: Opening SGM file startup/game.sgm
[Engine] The main script is startup/scripts/Jest_Main.js
[ConfigManager] Info: Successfully opened script startup/scripts/Jest_Main.js
Plugin SDL_GL_threaded.dll is open.
[SDL_GL] Info: STND was 0. Now it is 0.
[SDL_GL] Info: STND was 1. Now it is 1.
[SDL_GL] Info: Using OpenGL version 3.3.0
Plugins:        1
Plugins:        1
Plugin audioBASS.dll is open.
[audioBASS] Info: Initialized BASSmidi as bass plugin 274726912.
[audioBASS] Info: loaded and initialized BASS.
Plugins:        2
Plugins:        2
Plugin bmpfontGL.dll is open.
Plugins:        3
Plugins:        3
Plugin getkeystring.dll is open.
Plugins:        4
Plugins:        4
Plugin inputSDL.dll is open.
[inputSDL] Info: Event state setup succeeded.
Plugins:        5
Plugins:        5
Plugin mapengineGL.dll is open.
Plugins:        6
Plugins:        6
Plugin networkTS.dll is open.
Plugins:        7
Plugins:        7
Plugin scriptfs.dll is open.
Plugins:        8
Plugins:        8
Plugin ttffontGL.dll is open.
Plugins:        9
Plugins:        9
Plugin windowstyleGL.dll is open.
Plugins:        10
Plugins:        10
[ttffontGL] DrawText Info: Binding string Do a barrel roll! to cache slot 0.
[ConfigManager] Info: Successfully opened script startup/scripts/MJ-12.js

Running Script.

Opened a file as a stream.
Opened a file as a stream.
Spriteset is version 3
This is just a test: 8
[ttffontGL] DrawText Info: Binding string Press Q to just give up. to cache slot 1.
[ttffontGL] DrawText Info: Binding string Do a barrel roll! With TTF Fonts! The mjMap's width is 1024 and its height is 1024 to cache slot 2.
[ttffontGL] DrawText Info: Binding string Attack.ogg's length is 59944, Spider's length is 87272 to cache slot 3.
[bmpfontGL] TS_wordWrapString Info: String This is a text box. These things gave me a bit of trouble, but now they work. I tried to make them lightning fast, which made it hard to debug any problems. But some solid reasoning and carefully spent blood, sweat, and tears, and now it works just pretty good. Sometiems cuts the lines a little early, but overall OK. cached to slot 1.


Commenting out drawTextBox doesn't resolve it, unfortunately. I thought it was the GetMouseX/Y calls, since every so often it runs long enough for me to hear a sound until I move the mouse, but even leaving the mouse alone didn't fix it.

I also optimized the GetScreenWidth/Height calls out of the main while loop in my copy to make sure those didn't affect it.

...
What is the following line supposed to signify and why is that value so high?
Quote

[audioBASS] Info: Initialized BASSmidi as bass plugin 274726912.
Title: Re: TurboSphere
Post by: Flying Jester on February 15, 2014, 03:47:56 am
The bassMidi value is a unique plugin ID that is chosen by Bass. So long as it isn't 0xFFFFFFFF or 0, the number is fine.

Are there any calls to GetSystemWindowstyle? Those are sometimes buggy on Windows (I thought I fixed, that, though).
Title: Re: TurboSphere
Post by: N E O on February 15, 2014, 03:03:13 pm

Are there any calls to GetSystemWindowstyle? Those are sometimes buggy on Windows (I thought I fixed, that, though).


Only one and that's called way near the beginning of game(). Commenting drawWindowStyle in the loop doesn't fix the crash.
Title: Re: TurboSphere
Post by: Flying Jester on February 15, 2014, 04:24:13 pm
Well, those are the functions I know caused issues before.

Does replacing the entirety of Jest_Main.js with

Code: [Select]

while(true){
FlipScreen();
GetKey();
}


stop the crash?
Title: Re: TurboSphere
Post by: N E O on February 17, 2014, 03:48:05 pm
Kinda; I get the following unexpected output when I define the game() function as the above:

Quote

TS_CallGame error: game is not a function.
Game function done running.


Your short while loop is explicitly in function game() {...} so I don't know why that message should even appear. No APPCRASH, though :/
Title: Re: TurboSphere
Post by: Flying Jester on February 17, 2014, 04:30:53 pm
...Oh yeah. I changed it so that `game` is checked first.

Code: [Select]

function game(){
  while(true){
    FlipScreen();
    GetKey();
  }
}


EDIT:
(http://flyingjesterentertainment.webs.com/returning.png)

The new plugin tools up and running. Updating the graphics plugin is the big task still to be done.

The new plugin tools and the new usage of V8 in the engine makes TurboSphere able to start different games and restart the current game at runtime. Having looked through Sphere's source, it looks like that sort of thing was a lot easier back when JS was interpreted. And V8 wasn't really meant to do this anyway. Chrome puts each page in its own process, and never reuses a V8 instance. But the new requirement of Isolates and increases 'security' checks (page identity in Chrome, game instance in TurboSphere), and much, much better identity of handles makes it much simpler to do.

Also note that V8 doesn't need or accept the flags to enable typed arrays anymore. That's because they are standard now!
Title: Re: TurboSphere
Post by: Flying Jester on February 21, 2014, 03:35:00 pm
The new version is still coming along. I'm putting the new graphics core into a new graphics plugin, using the new plugin tools. Everything new and shiny!

The new graphics plugin will actually be callable from other plugins a lot like the Sphere video drivers are in Sphere. I needed to see that in action, writing FJ-GL, to see the advantages that a more detached API could provide. The API will be closer to the metal than Sphere's (the plugins still must agree on OpenGL), but it will be more useful than what I had before.

I've added the ability to dynamically switch between multi-threaded and single-threaded surface rendering. It doesn't really have a good use, but I'm working on adding more threading, and it will be important in other places and other threads to be able to do that. While I'm at it, I've been moving the surface renderer over from my own homemade concurrent queue (which uses a mutices) to use the Intel Thread Building Blocks or the Microsoft Concurrency Library (both options available at compile time), which are atomic spin-lock based (faster, easy to remove busy-waits). Obviously TBB is necessary on Linux, and MSCL is included with MSVC so that's easier on Windows. They are, in the case of concurrent container classes, API compatible.

I've also worked out a file type for scripts that contain embedded V8 compiler data. It's basically IFF-style, lacking a resource map, with specific magic numbers for script text and V8 data. It also hijacks a reserved field to store V8 version data (this is important!). It's extremely simple to read this kind of file, check if there is V8 data in it, and if not generate and append V8 compiler data to it. This speeds up subsequent compilation of large and complex scripts (this is a part of how V8's snapshot works). I'm using this for the new graphics plugin, which defines a couple functions in embedded scripts.

On a different note:
For laughs, I tried to compile TurboSphere for Raspbian on the Raspberry Pi. It looks like all it needs is more OpenGL ES support. SDL2 has code just for the RPi, Bass has an ARM-Linux version that works on Raspbian, and V8 has ARM and Linux as primary targets. TS already has some OpenGL ES support, added during my cursory looks at Android. I've been adding space for it now as I work on the new graphics plugin.
Title: Re: TurboSphere
Post by: N E O on February 22, 2014, 03:03:38 pm
Whoa. That's hardcore modularization, dude!

I look forward to trying out an updated Windows build with Flappy Sphere :)
Title: Re: TurboSphere
Post by: Flying Jester on February 23, 2014, 05:26:32 pm
0.3.5c crashed even with the empty game function? That version never crashed for me at all.
Title: Re: TurboSphere
Post by: Flying Jester on March 03, 2014, 08:17:47 pm
Here, a Python script that creates basic script IFF files out of normal JS files for embedding JIT, parser, or any other engine data into.

Type ID 23 is used for raw scripts. Type ID 24 will be for zlib-compressed scripts. I'm planning on using type 127 for TurboSphere V8 data. Any other engine could also extend these files with embedded engine data, assuming they don't have ID collisions.

Bear in mind this is only for embedded engine scripts right now. This is exactly the script that creates the IFF that holds all the embedded scripts for the new graphics plugin.

My change of heart on the whole embedded scripts issue was because, after some testing, I found out (again) how costly the JS-to-C++ interface is. I still don't really like having scripts built into the engine, but it does improve performance where it cuts down on the number of native-to-script edges.
Title: Re: TurboSphere
Post by: Radnen on March 03, 2014, 10:50:25 pm

My change of heart on the whole embedded scripts issue was because, after some testing, I found out (again) how costly the JS-to-C++ interface is.


That bridge of C++ or in my case C# to JS interface is so costly that all V8 based C# wrappers I found were just too slow. The authors of those libraries say they utilize the full power of v8 and are lightning fast. But in actuality when you are running a game that loads images then calls their blit methods, it turns out to be really slow. They say it's fast since math operations and user-created objects are fast, but any native interface-exposed objects all take insane hits.

Jurassic is faster than V8 at this because it is literally C#. You code in JS, but it compiles as C#, or rather CLI, so there is no such thing as a JS-to-C# interface to worry about. It takes the same time to use a wrapped native method as it does to create your own method, theoretically.

The only reason in Jurassic why a C# exposed object is slightly slower than an in-engine object has to do with the fact that JS takes Number for it's number types whereas in C# we have int, float, double, short, byte, etc. and these things take a casting hit. Furthermore, more optimizations could be made to look up methods and things faster with better caching. Also, Jurassic doesn't do what V8 is really good at: identifying static types in a dynamic language. I did a test and in C# a for loop with a double as an incrementor is 10 times slower than an int. In the compiled Jurassic JS it seems to use doubles as the incrementor rather than knowing to use an int and stick with that. If Jurassic could identify these static hotspots it could do very well, potentially.

It also helps if I could find a way to compile the JS to file and load compiled JS straight up. It would help in only one way: faster loading times. But that's hardly a concern right now.
Title: Re: TurboSphere
Post by: Flying Jester on March 04, 2014, 12:55:15 am
V8 can't export compiled JS. It can export parser data and some heap data, which is what I will save in the IFF files.

The JS-C++ boundary is expensive, but not hugely so. If I can limit it to one call per statement, the performance is pretty good.
Title: Re: TurboSphere
Post by: Flying Jester on March 05, 2014, 03:14:52 pm
I've added the ability to pass flags to V8 on the command line. Some of the flags, such as strict mode, the new Harmony 'var' implications, and the new Harmony numeric literals (which include using 0b### for binary literals) also have config flags now, available in the game.tsgm.

The syntax is:

Code: [Select]


#turbosphere --v8Option
#for instance:

./turbosphere --v8harmony_numeric_literals
#Now you can use 0b to specify a binary literal.

#or:

./turbosphere --v8max_inlined_source_size=1000 --v8type_info_threshold=50 --v8no-harmony_symbols
# disable harmony private names, require half of a function's parameters to have a stabilized type before using the high-optimizer, and allow functions up to 1000 parsed lines to be inlined.



You could use this to optimize V8's inner workings for your game, to specify language options, or help with profiling and tracking performance. V8 can even write debug and profiler info to a file.

For now, to avoid Sphere overwriting any extra engine options, my stance is that games that are aware of TurboSphere should have a tsgm file in addition to an sgm file. The sphere 1.5/1.6 compatibility system script will be automatically included if TurboSphere is run on an sgm file that does not have a version field.

The tsgm can just be a direct copy of an sgm. No extra options are needed.
Title: Re: TurboSphere
Post by: Rahkiin on March 13, 2014, 06:05:46 pm
@FlyingJester

I just updated my V8 to 3.24.40 and I discovered that they changed a lot of API again. HandleScopes cant return (Close is gone), instead use EscapableHandleScope. Also String::New is gone, and everything needs isolate as parameter (Undefined(), Null(), FunctionTemplate::New(), everything). There is probably more. I think they dropped some Persistent stuff too.

Just sayin'


// Rahkiin
Title: Re: TurboSphere
Post by: Flying Jester on March 13, 2014, 09:54:06 pm
I know, right?

I worked out a new set of plugin tools (https://gist.github.com/FlyingJester/9540769) to work with V8 3.24.

Persistents still work, but it's very difficult to make a persistent back into a handle. Did you know that you don't normally need handle scopes at all? Every script callback is implicitly in its own handle scope, and its return value (now a part of the argument parameter) is automatically passed out of it. The whole String::New thing is a good thing. There's no such thing as `plain text', and they have finally made V8 fully UTF-8 aware.

They have been talking about this for a long time. Someday soon (they've got almost all the infrastructure in place now), Persistents will be removed altogether. The whole concept of a persistent is a little strange, and they were kind of a mistake to have at all.
Title: Re: TurboSphere
Post by: Rahkiin on March 13, 2014, 10:51:57 pm
About the persistensts: they have been splitting them up. There is PersistentBase, and both Persistent and UniquePersistent inherit from it. The latter has some unique id stuff so you can use them in collections I think. The first one allows traits for copying, so you can actually set them. Then you can also use PersistentBase, I think. But Persistent is still there and I use it, and need it.
There is now also Eternal: it makes an object persistent for ever. Pretty cool. Using it for FunctionTemplate cache.

HandleScope was never clear to me. I am just gonna try it out. I am also going to spread Isolate everywhere and eliminate Isolate::GetCurrent() so i can have multiple isolates and contexts. There are EscapableHandleScope, that have an Escape method, like HandleScope has Close. It is just so annoying that the only source of information about what we should actually do, is the mailing list archive.

The String::NewFromUtf8(isolate,str) is indeed good (as it is good that all New() etc now require an isolate), but it is so long and messes up my code width :P. Null(isolate) and Undefined(isolate) is also less nice than Null().

I made all my tests run again. It was fun xD

Did you know they will also get rid of Handle, and only keep Local?
Title: Re: TurboSphere
Post by: Rahkiin on March 13, 2014, 10:57:10 pm
Now we are talking about it:

Is there a way to do V8 stuff like creating Locals without creating a Context::Scope within scope?
Currently I think I must, and it means I have this awful piece of code that allows users of my framework to execute code within the context scope. https://github.com/joskuijpers/L8Framework/blob/master/L8Framework/Source/L8Runtime.mm#L112

Title: Re: TurboSphere
Post by: Flying Jester on March 14, 2014, 12:07:42 am
You need at least one context, context scope, and isolate. You always have, it's just that there used to be one that was implicit to initializing V8. Now you need to do it all explicitly.

Also, FYI, the Isolate::GetCurrent() is deprecated, although not marked as such. That's going to be fun to work around, isn't it?
Title: Re: TurboSphere
Post by: Rahkiin on March 14, 2014, 12:21:05 pm

Also, FYI, the Isolate::GetCurrent() is deprecated, although not marked as such. That's going to be fun to work around, isn't it?


Blagh yes. I can eliminate it everywhere, except in all my [L8Runtime current*] spots, where I actually ask for current isolate, current context...

Do you follow the mailing list?
Title: Re: TurboSphere
Post by: Flying Jester on March 17, 2014, 10:53:03 pm
I read both the V8 Users and V8 Dev mailing lists all the time. It'd be really hard to use V8 and not!
Title: Re: TurboSphere
Post by: Rahkiin on March 18, 2014, 12:12:11 am
Hehe yeah. I follow it since a day or two now. I used to do some trial&error and Google the ML archives & issuelists. And #v8 on freenode.net.

In my last code of L8, I eliminated every possible use of Isolate::GetCurrent() and GetCurrentContext(). I pass around Isolates everywhere. And the new L8VirtualMachine wraps an Isolate. I can also have multiple VirtualMachines at the same time, as long as each one is in a different thread (did not test this xD).
There is a way to avoid using Context::Scope. Context::Scope() does context->Enter(), and ~Scope() runs context->Exit(). You can call those yourself too. If you only ever have 1 context in 1 isolate, you could just run context->Enter() and be done with it ;)
Title: Re: TurboSphere
Post by: Flying Jester on March 18, 2014, 03:00:10 am
I actually only use context and context scopes in a single file in TurboSphere, only right when the engine is started. You don't really need to manage them at all if you just pass isolates around.
Title: Re: TurboSphere
Post by: Rahkiin on March 18, 2014, 06:24:05 pm
I still have to figure out when exactly it is necessary. But yes, V8 is much easier to use when the whole system is not event based and has a single entry point :P
Title: Re: TurboSphere
Post by: Flying Jester on March 18, 2014, 06:42:51 pm
The only point at which you need to wrangle contexts and context scopes is if you allow for multiple games running at once.

Although even then, as I found out by compiling TurboSphere's engine binary as a shared library when I was trying to get it working on Android, you can get away with the context handling being extremely simple. In that case you could even have it spawn a new thread for every open game, every open game would run in its own thread and have its own context and context scope.
That would be slightly wasteful, since I could theoretically have all the games run in different contexts in the same scope, and even share the threaded backends in plugins. But it would be so much more complicated.
Title: Re: TurboSphere
Post by: Rahkiin on March 18, 2014, 09:10:00 pm
You might be confusing your terms here:

You can't run contexts over multiple threads. Not without locking the isolate, which means there is no performance increase. You might want to run an Isolate per game though, and one Isolate per thread.

As my L8Framework is designed not only to be used for my Sphere runtime, it is a bit more complicated :P At least for 1 isolate and 1 context it works just fine :)
Title: Re: TurboSphere
Post by: Flying Jester on March 18, 2014, 09:19:30 pm
I wouldn't run contexts over multiple threads. The backends are clean of any V8 usage. The graphics backend is a mostly a combination of what I learned trying and failing at making OpenSVD and the core functionality of FJ-GL, for instance.

I would have one context scope, one context and one isolate per game. I mean that I could have a single set of worker threads that were used by all games, and all games shared the same work queues in them. I like the idea a lot, but oh-me-oh-my-oh would it be a lot of work to implement.

As it is, one context and one context scope. They only need any manipulation when a game starts or stops.
Title: Re: TurboSphere
Post by: Rahkiin on March 18, 2014, 09:23:35 pm
But then there is always the most important question: Will it be used?

Who plays 2 games at the same time on the same machine?
Title: Re: TurboSphere
Post by: Flying Jester on March 18, 2014, 09:27:58 pm
That's a big reason why I am not going to make it work that way.

Of course, I'd like to think of TurboSphere as more of a desktop JS toolkit, something akin to Node.JS but with a greater focus on multimedia usage. If that kind of usage is properly realized, then such features would become more useful.
Title: Re: TurboSphere
Post by: alpha123 on April 18, 2014, 01:08:30 am
Er, I often play two Sphere games at the same time on the same machine to test networked games. Also I've been thinking of/slowly working on a Sphere Debugger that runs as a game and attaches to another running game.
Title: Re: TurboSphere
Post by: Flying Jester on April 18, 2014, 01:38:09 am
I was speaking of adding server-client type facilities that specifically address multiple games running in one instance. I'm just going to leave it be, one game per engine instance, for the foreseeable future.

It's true that it does happen (I do the same thing, testing server-client functions without worrying about the network, using two machines, etc.). But having two games share the same backend (more like  how a web browser works) would be tremendously complicated. Simple in theory--but it would be a LOT of work to implement, and would cause at least some overhead when only one game is running.

Running two instances of the engine at once is still quite possible, though. That should always be possible.

Considering a debugger, it would be fairly simple to make a rudimentary piping plugin for TurboSphere that would let a second instance examine the JS stack on another instance.
But V8 already has GDB debugging integration. Or so they say--TurboSphere is always distributed with a V8 library that has that enabled. But there is, as with almost everything Google does, no documentation on how it works or how to use it.
Title: Re: TurboSphere
Post by: alpha123 on April 18, 2014, 03:18:09 pm

I was speaking of adding server-client type facilities that specifically address multiple games running in one instance. I'm just going to leave it be, one game per engine instance, for the foreseeable future.

Ah, like web browser tabs for Sphere games? Yeah, I can't see that being terribly useful. I can think of some cool stuff involving minigames inside e.g. a larger RPG, but there are other ways to do that involving much less effort.

Quote

It's true that it does happen (I do the same thing, testing server-client functions without worrying about the network, using two machines, etc.). But having two games share the same backend (more like  how a web browser works) would be tremendously complicated. Simple in theory--but it would be a LOT of work to implement, and would cause at least some overhead when only one game is running.

Better to work on other things then, and maybe revist this in a few years after everything else is done. =P

Quote

Running two instances of the engine at once is still quite possible, though. That should always be possible.

Good, I was getting a little worried there. :)

Quote

Considering a debugger, it would be fairly simple to make a rudimentary piping plugin for TurboSphere that would let a second instance examine the JS stack on another instance.
But V8 already has GDB debugging integration. Or so they say--TurboSphere is always distributed with a V8 library that has that enabled. But there is, as with almost everything Google does, no documentation on how it works or how to use it.

That would be awesome, because my current solution is extremely hackish. It involves recompiling the code to add instrumentation and then using the network functions to push to the debugger whenever something changes. Suffice it to say, it doesn't work currently. :P
Title: Re: TurboSphere
Post by: Rahkiin on April 18, 2014, 06:57:57 pm
There is the D8, the debugger for V8. Works just fine. You just connect to the app and you can set breakpoints, do traces etc.

Also, multiple isolates is not that hard, the problem will be the multiple opengl contexts I think.
Title: Re: TurboSphere
Post by: Flying Jester on April 23, 2014, 04:30:54 am
It's more difficult than having discrete processes for multiple games. This would introduce layers of complexity I am not prepared to manage.
Title: Re: TurboSphere
Post by: Flying Jester on May 31, 2014, 12:18:11 am
Given the general silence around here lately, I'm going to post a status update, since I have been working on TurboSphere, day after day, week after week.

The new graphics API (Galileo), which has been outlined as a part of the new Spher 2.0 API (Pegasus) is now functioning. There are horrendous issues still with texturing anything but a triangle, but it is working. It's threaded, allowing pipelining and asynchronous drawing. I've added a new version of FlipScreen in addition to the traditional version. This new version doesn't clear the screen. This lets you do partial redraws if you want, or merely draw on top of the previous scene. TurboSphere supports several additions to the new graphics API, which make it simpler to do things the old Sphere 1.x way (for instance, you can still do a direct blit of an image or surface, and load an image directly instead of loading a file as a surface and then converting to Image).

The old Surface threader has been improved, as I have properly learnt a lot more about lockless programming. It now is basically a garbage-collected heap of operations, with links based on per-surface operations. This lets it check, extremely quickly, if a surface is dirty or clean, lets it do out of order, per-Surface operations without interrupting operation queueing, and even means more surface operation threads could be run (although I'm keeping it to a single worker thread for now).

I've added the ability for the audio plugin to dynamically load any appropriate proprietary Bass library plugins from the root directory of TurboSphere. I also made the midi plugin optional (but distributed with TS by default). The main purpose of this is to add additional codecs for playback.

I've fixed about a million minor and aesthetic bugs with game loading. Ideally this is to make the game loading API cleaner and work properly, but I still haven't done that. There is a magic hook in TS to force it to work, but this only works if all the loaded plugins support it. According to the plugin API, it's not really a TS plugin if it doesn't work, but until now there was no actual test to see if a plugin was meaningfully supporting the Close() operation (and indeed, a few of my own standard plugins had issues).

The InputSDL2 plugin now supports Android-based touch interfaces as pointing devices--if only the graphics plugin supported GL ES! The actual engine binary can be compiled as a shared library, which means you could, theoretically, call it with JNI-bindings, and run TS on Android. I just haven't tried that, mostly since I know that the graphics plugin needs full OpenGL 3.1 at least, and it isn't even fully working there yet.

Rahkiin and I have been working out an API for querying extensions to the core API, and TurboSphere now implements it. This is especially important for an engine like TurboSphere, where your game might need to know what plugins exist, specifically by asking what capabilities are present. But other engines could also use it to say whether or not a dynamically loaded library could be found at runtime or not (for instance, a Windows engine probably shouldn't demand you install Bonjour to work at all--but if you have it installed, it would be cool to use it).
Title: Re: TurboSphere
Post by: DaVince on June 02, 2014, 12:03:29 pm
That's a lot of progress, nice work! :)
Title: Re: TurboSphere
Post by: Flying Jester on June 09, 2014, 08:30:36 pm
And now it almost--very almost--compiles on OS X. Mostly I'm adding alternative implementations for the Linux bits that are GNU, rather than early X/Open or SYSV bits. And figuring out the standard programming and deployment guidelines for OS X, too.

If anyone is curious about how the world looks to TurboSphere, at compile-time and run-time (especially gfx and threading) configuration, here is what is going on:

At compile time, there are two kinds of machines: Unix and Windows.
Or at least, there are machines were we have common Unix headers (we specifically look for the SYSV versions, although we also check in the X/Open locations), and there are machines with the MSVC runtime library. This isn't dependent on the compiler.
There are two kinds of compilers: GNU compilers (these include Clang and MingW) and MSVC compilers (this includes the Intel Compiler and formerly DMC). This just affects the specific switches we give the compilers.
OS X is a special case of Unix--while we normally go through some configuration to check for header locations and versions, we just use the OS X standard locations when we are running on an OS that identifies itself as Darwin.

At run time, it's a little different. TurboSphere has a composite architecture--that is, there are a series of interfaces in the code, and for each one we select a certain implementation to use for that interface. Most of the time the interfaces are to abstract Windows from Unix, although not always. For instance, when you call  Delay() from script it becomes a Win32 call in Windows, an X/Open call (usleep2) in Linux, and a Mach kernel call in OS X.
There are also a couple places with multiple libraries that can be used. For instance, the new drawing API can use the Intel Thread Building blocks concurrent queue implementation, or the Microsoft implementation, or (if neither are available) simply a handmade queue wrapped-in-a-mutex. Similarly, when available we use the Intel TBB atomics, but lacking those we use C++11 atomics. Lacking even those, we fall back to SDL2's spinlock atomics.

At run time, TurboSphere recognized three kinds of machines: Machines with OpenGL 3.2, machines with OpenGL 2.0, and machines with OpenGL ES 2. We don't really support pure GL 2.0, we require certain ARB extensions for a some functions (specifically non-pot textures, vertex buffers, and vertex arrays), but we do get along without shaders and use the older initialization system. In GL ES, we remove some calls, and again rely on the GL driver being more than just a core implementation.
TurboSphere does not use any extra OpenGL headers, only the very core GL/gl.h, which isn't much. When we need more advanced functions--so basically everything but generating GL textures--we use run-time loaded function pointers. This has some advantages over just using headers--we never blindly try to use something that isn't there--and over extension querying, which (more often than you'd think) under-reports the implementation. Extension querying also is especially unreliable if you aren't using the default GL version and profile, which TS almost never does.
Title: Re: TurboSphere
Post by: Flying Jester on June 11, 2014, 10:45:19 pm
The TurboSphere source just got a new coat of paint (https://github.com/FlyingJester/TurboSphere/tree/refactor). Or at least I'm staging things for it. I've made the directory tree much more appealing, and I'm working on making the build system much cleaner. The first is a simple task, the second is not so bad since most plugins need their scons files rewritten anyway with the new version of V8 and the new use of SConscripts.

The new build setup will make it simpler to build install TurboSphere, taking the number of commands to build and install from three to two--one to build the entire source, and one to install or package depending on the platform. The new build setup also puts each plugin in a configuration jail, only giving the ability to modify the global configuration state to the core build files that actually need that ability, which set the build up for everything else.
It also means that the build files will actually be in the same place as what they build or what they configure for. Which is much nicer!

Edit: The new build system is mostly working, just a couple little bugs. The way it works, for configuration the main Scons file simply calls the scons script from where it wants to import includes and libraries, and those are returned in an environment that extends the old one. And most importantly, you don't have to keep hammering Scons over and over again to get the build to complete, it just does it all with a single command.

I really need to find a convenient way to pass data from Scons/Python to make, so that I can have better integration of make-based plugins. Not that I have any, but I'd like the environment to be as welcoming to anyone who wanted to make a plugin as possible.
Title: Re: TurboSphere
Post by: Harry Bo21 on June 19, 2014, 05:26:45 pm
Your such a legend Jester :)

Its sounding awesome. Cant wait to start building projects with your baby :)
Title: Re: TurboSphere
Post by: Flying Jester on June 20, 2014, 01:17:07 am
Thanks! Don't call me legend until it gets released :D There is still lots to do, and right now the engine can only just (almost) draw triangles again.
Title: Re: TurboSphere
Post by: Harry Bo21 on June 20, 2014, 02:41:23 am
lol, you're a legend in my eyes :)
Title: Re: TurboSphere
Post by: Flying Jester on July 06, 2014, 08:22:06 pm
I've added an alternative to using the MSVC or Intel TBB concurrent queues. Now, the Sapphire graphics plugin can use Unix file descriptors set up as a pipe for threadsafe communication. It's still preferred to use Intel TBB on Unix systems when possible, but setting up TBB can be a pain. Having the option to use Unix file descriptors is just so that this is not vitally necessary anymore. It's also nice, because now there are native threadsafe communications methods on both Windows and Posix systems.

Using a Unix pipe is the default on OS X only, but still an option on Linux and, when using MingW or Cygwin, I suppose it would also work on Windows.

A major reason to use pipes instead of a std::queue wrapped in a mutex (std::mutex or pthread mutex) is that mutexes on OS X are abysmally slow. Using some simple testing (https://gist.github.com/FlyingJester/f05a231a58fb6c5744a7), pipes are up to four times faster (https://gist.github.com/FlyingJester/596679572afdc72f26ae) than mutexes. Of course, Intel's TBB concurrent_queue is still twice as fast as a pipe, and proper lockless concurrent queues are fastest on all the platforms I've tested. But it's nice to have a reliable, reasonable bring-up backend that works on a huge number of platforms out of the box. This would even work with some pretty archaic compilers, since it basically doesn't need the STL, or even pthreads (or any userspace synchronization primitives at all).

As the above gist might suggest, there is also a std::queue wrapped in a std::mutex backend available for the TurboSphere plugin tools. Pthread mutexes are used when there is no std::mutex available, too.
Title: Re: TurboSphere
Post by: Flying Jester on July 08, 2014, 09:08:23 pm
TurboSphere compiles on OS X now. The main engine works fine, including plugin loading and starting games. Unfortunately, only a couple plugins work just yet.

I've updated the SourceForge (http://sourceforge.net/projects/turbosphere/) page, including uploads of the current sources (which, as always, are available on the github page (https://github.com/FlyingJester/TurboSphere/tree/galileo)).

I've also began rewriting T5. An outline of what the new T5 will offer is given in a post on the new FJE site (http://flyingjester.site11.com/FJ-blog.html?id=6). When this is done, TurboSphere will be able to natively and transparently load files to and from archives (zip, tar, etc.) with or without compression (gz/zlib, xz). This will let TS properly run games out of SPKs, or out of any supported archive with or without compression (zip, tar.gz, tar.xz, cpio.bz2, etc).
You'll also be able to save and open files with compression transparently from script, and access files in archives transparently.

The Sphere File object will be updated, too. The new T5 will let you save the key/val files as either their current INI-style or JSON. I'm going to look into other formats (notably BML and XML).
Title: Re: TurboSphere
Post by: Flying Jester on July 13, 2014, 07:26:12 pm
TS now works as well on OS X as it does on Linux. There are still synchro issues, so it can't draw textures yet. Once those are fixed, it will be able to fully use the Galileo API, which means that Sapphire will have all the old primitives that SDL_GL_Threaded did, but in an extensible way.

It's also paying off that I previously added multiple backends for basic engine features. Since the system library setup is different in OS X than in other Unices, the old code that located and dlopen'ed SDL2 can't find it on OS X. The engine proper used this to display the error popups. Fortunately, I also added a basic terminal backend that it would fall back to if the SDL2 backend failed.
Title: Re: TurboSphere
Post by: Flying Jester on July 14, 2014, 04:47:27 am
Now with texturing, windowing, surfaces, images, and primitives.

Some quick stats on the new graphics plugin, all taken on a late 2013 MacBook Pro:

* 4000 FlipScreens per second. This is a hard limit, just like the old 2000 FlipScreens per second in SDL_GL_Threaded.
* 16,000 primitives per second with generic Unix pipes. 32,000 with a concurrent queue (Intel TBB on Unix, MSVC concurrency namespace on Windows)
* 45,000 triangles per second

Given that I'm using OpenGL 3.3 or OpenGL 4.1 depending on the platform, all my drawing in totally asynchronous (particularly from resource uploading and downloading), and all my drawing uses full Vertex Arrays, this should be a powerful enough architecture for a long time. The new graphics API makes it clear what will and what won't be fast, too. It makes graphics object-oriented.

Sapphire also exposes functions similar to Sphere 1.x video drivers, which makes using it from another plugin easy. I'm going to rewrite the old plugins to use this new API. This will also mean that you could use the standard plugins with a different graphics plugin, so long as the exposed interface is the same.

This has taken a long time, but I'm properly proud of Sapphire+Galileo. It fixes all the things that bothered me about SDL_GL_Threaded, about how plugins worked together, and about using the old drawing API with new OpenGL. It replaces all of the APIs I proposed but had reservations about, too (SpriteBatcher, Shader), with a unified graphics API.
Title: Re: TurboSphere
Post by: Radnen on July 14, 2014, 08:01:04 pm
This is amazing! I know you want to make a better more modern version of Sphere but you had no real direction until now. Your previous direction was more along the lines of "this is cool" and educational at best. But now it seems you've past that and are exhibiting a kind of mastery over the craft. Ok, maybe a bit much, but you get the idea.

Having a well laid out API and looking back at what you thought worked and didn't work is great and this project will be all the better. Plus, you've got it working on Mac! Holy cow!

SSFML has been on the back-burner for some time and it may never achieve multi-plat status (yet), so in the meantime I'm eagerly awaiting each new thing you do. :)

Also, I'll just go say it here: Our newer engines are using compilation approaches to JS and I think it's about time I update Sphere Studio with a fun toy: JSMin. I think it will increase the load/execution times of games by a small degree especially if you are like me and have hundreds of files with only a dozen or two lines of code. Having one giant JSMin'd file would be for the best. Also, I don't know about V8, but Jurassic has to 'spin up' each new file it sees (I think), so that just adds more overhead to it already opening dozens of files.
Title: Re: TurboSphere
Post by: Flying Jester on July 15, 2014, 02:49:15 am
Hmm, wouldn't minifying the source make accurate backtraces impossible?

And yes, until I started working on Sapphire, TurboSphere was very much the victim of 'Ooh, Shiny!' development. That had the advantage of laying a lot of foundations early, but it also meant that very little was fully fleshed out.
Title: Re: TurboSphere
Post by: Radnen on July 15, 2014, 09:32:24 am

Hmm, wouldn't minifying the source make accurate backtraces impossible?


Yeah, but it's more for release than for development.
Title: Re: TurboSphere
Post by: Flying Jester on July 15, 2014, 03:03:18 pm
Ah, that makes sense. Actually, having proper debug and release settings could be quite nice, there are a few things I can think of that would be well suited for that.

Since I can't poke SDL2 directly from the engine in good faith on OS X, not knowing enough about the system library setup, I wrote a quick Cocoa messagebox backend (almost certainly using the same backend as SDL2 does).

The error is also dumped to the terminal, since you can't select the text in this dialog, unlike the X11 dialog SDL2 provides on Linux.

Theoretically, there is a way to actually make the script's path a clickable link on OS X. That would be really nice!
Title: Re: TurboSphere
Post by: N E O on July 15, 2014, 03:39:40 pm
Two questions:

1. What's the earliest version of OS X that Mac TS supports/will support? Snow Leopard would be SUPER nice, but if there's an inherent limitation like "64-bit only" I'd be fine with Lion. I highly doubt the ability to backport certain features to current Mac Sphere anyways :/

2. Given the use of V8, is there going to be some sort of API for attaching console output somewhere? I know that in my branch of Web Sphere I plan on making heavy use of the JS console for messages from creation functions and such when certain debug flags are turned on, and I would love for a similar functionality on native apps outputting, say, to a given log file or a third-party debugger or something.
Title: Re: TurboSphere
Post by: Flying Jester on July 15, 2014, 11:10:42 pm
TurboSphere has a soft requirement of OpenGL 3.3 or 4.1 (theoretically, I think you could change the config to only need 3.1? Maybe even a 2.x release, but I can't test that). It will surely work on OS X Lion, I don't really know about Snow Leopard. I mention the OpenGL requirement, since I think that's the limiting factor.

You can always try to compile it on Snow Leopard, the Galileo branch is where I've got it compiling on OS X.

The console API would be simple to implement. I don't have any plans for it in particular, but it sounds like a good idea for a new plugin.
Title: Re: TurboSphere
Post by: Flying Jester on July 17, 2014, 04:28:30 pm
For those on OS X, you can test out a build here:

https://sourceforge.net/projects/turbosphere/files/bin-Mac/0.4.0/ (https://sourceforge.net/projects/turbosphere/files/bin-Mac/0.4.0/)

This build requests an OpenGL 3.3 context if an OpenGL 4.1 context is unavailable.

I'd be curious to see which versions of OS X it properly runs on. Unfortunately, I only have two Macs to test on, and they have quite disparate versions of OS X on them (Jaguar and Mavericks). I can only say that I know it works on Mavericks.

You need to run it from the terminal, with the extracted folder as your working directory.

If you are on Linux, you can see if the build is currently broken or not by checking the Travis CI page for TurboSphere:

https://travis-ci.org/FlyingJester/TurboSphere (https://travis-ci.org/FlyingJester/TurboSphere)

If you are using as ancient an Ubuntu or Debian as Travis, and can't be bothered with things like having a C++11 or C11 compatible compiler, my first suggestion is to use a distro with a rolling release. My more reasonable suggestion is to set 'TRAVIS' to 'true' before building:

Code: (bash) [Select]

export TRAVIS=true
scons


Note that at this precise moment, the plugins are NOT compiled in the Travis build. Very related, case-insensitive filesystems are a scourge upon the world that will destroy us all someday, and I curse Apple for having case-sensitivity be optional for HFS+, but have the factory setup be case-INSENSITIVE. Hopefully someday Apple will use a not-hack filesystem in their OS.
Title: Re: TurboSphere
Post by: Flying Jester on July 19, 2014, 10:53:40 pm
A couple of super quick demos:

https://gist.github.com/FlyingJester/067068bb6cc358282324 (https://gist.github.com/FlyingJester/067068bb6cc358282324)


https://gist.github.com/FlyingJester/675b58454e8738c0527c (https://gist.github.com/FlyingJester/675b58454e8738c0527c)


Both of these use a not-quite final API, I'm having issues with JS Accessors in V8, so everything must be done with member functions.

We get easily 2000 FPS drawing just a single image. I tried filling the screen with Sphere logos, drawing about 1000 of them. We still get 600 FPS. It's not linear, we can draw more and more shapes with lower and lower overhead. Even better, using Galileo Groups, we can draw those 1000 logos with a single JS call.
Title: Re: TurboSphere
Post by: DaVince on July 21, 2014, 07:38:13 pm
If the forum had a "like post" function, I'd be using that rather than clutter up the thread with an extra post, but I just have to let you know how much I like how you're progressing right now. Once I get back to making games, I'll be sure to do it with TS. :)
Title: Re: TurboSphere
Post by: Flying Jester on July 22, 2014, 02:56:19 am
Thanks, that's good to hear!

I've re-added PNG and BMP saving from SDL_GL_Threaded into Sapphire. This time, I'm doing it much smarter. We load libpng at runtime (without any exceptional circumstances, it's possible for TurboSphere to not need it), and we avoid many allocations that I previously had. We also avoid the infinite loop that a failed PNG save incurred previously--that's why it was disabled by default in SDL_GL_Threaded--and now we fail semi-gracefully.

I must admit, I didn't really understand the libpng API when I used it in SDL_GL_Threaded. And it's still a pretty dense library, but at the very least Sapphire uses it much more intelligently.

I plan on eventually adding JPEG, TGA, GIF, and TIFF saving.

The old idea of adding mouse click events in an analagous API to GetKey/AreKeysLeft has been added to the input plugin. I've done it in such a way that it would be trivial to add another kind of input with the same API (like GetGamepadButtons, or GetTouches, perhaps). Also, they basically share the same code but abstract interpreting the event itself--which is good since this API is really not fault tolerant at all, it's extremely important that behaviour is absolutely correct.

I haven't forgotten about the Sphere 2.0 API, these are just stepping-stone API changes I'm making. The new API is somewhere in between Sphere 1.x and Sphere 2.0. I kind of expect a lot of TurboSphere's natively exposed API to stay similar to how it is now, and most of the actual Sphere 2.0 API to be implemented in script from on top of what I am working on right now.
Title: Re: TurboSphere
Post by: Rahkiin on July 23, 2014, 05:40:30 pm
Wow this is awesome!

Makes me reconsider my mac engine: I could just fork/pullrequest TS and write plugins... currently i only use my engine to experiment anyways. I don't see myself writing a graphics engine as well as yours!

If i find the time, ill run it on my mac tomorrow.

Feature request: V8 remote debugging (allowing it. Can use d8 to debug).

I might cease working on the engine and continue on my IDE instead and a builtin debugger is a must-have!

Keep the work up!

// Rahkiin
Title: Re: TurboSphere
Post by: Flying Jester on July 23, 2014, 06:17:06 pm
Thanks!

It would be awesome to have another developer! I've really tried to make the plugin-writing experience as friendly as I can...but I'm the only one who has ever used it! Getting more feedback would be extremely helpful.

I see a lot of ancient and half-finished documentation on the V8 remote debugger system. I'd have to take a close look. Perhaps I should create a 'Debug' plugin, one that adds console writing, remote debugging, etc.
Title: Re: TurboSphere
Post by: Rahkiin on July 23, 2014, 06:46:00 pm
You actually only ned to enable debugging, add a flag to wait for debugger, and handle debug messages (depends on how you do synchronization). I did it toally in 10 lines or less.

The actual debugging can be done using d8, or writing your own program that uses the debug protocol, that still works.
Title: Re: TurboSphere
Post by: Flying Jester on July 27, 2014, 04:58:34 am
Debugging is enabled on the Windows and OS X builds of V8 I distribute. So I guess I'd just need to actually push debug messages.

I'ev added JPEG and TGA saving (again, done better than in SDL_GL_Threaded), as well as cleaning up PNG saving. libpng is SO much nicer than libjpeg (and API clones). Making the important structs have totally hidden implementations makes loading at runtime possible. I wish libjpeg did this, particularly since I'd like to be able to try to link to libjpeg-turbo, libjpeg/simd, and moz-jpeg explicitly before defaulting to whatever generic libjpeg the system has. Otherwise, I'd need to put fairly large amounts of libjpeg inside TS to be able to load it properly at runtime. So now it's a compile-time flag.
Title: Re: TurboSphere
Post by: Flying Jester on July 28, 2014, 04:07:19 am
I've written a new BMPFont plugin. It's designed so that it can also be used by external tools (like a TTF-RFN converter), in addition to being used as a plugin.

The new plugin uses Sapphire's API for extending Surfaces. The following script demonstrates using BMP fonts with Surfaces in Sapphire:

Code: (javascript) [Select]

RequireSystemScript("colors.js");

var s = new Surface(64, 64, Black);
var Fonty = GetSystemFont();

s.drawText(Fonty, 0, 0, "Sphere");
s.drawText(Fonty, 4, 16, "Fonts!");

var TextImage = new Image(s);
var DefaultShader = GetDefaultShaderProgram();

var Vertices = [new Vertex(0, 0),   new Vertex(64, 0),  new Vertex(64, 64), new Vertex(0,  64) ];

var TextShape = new Shape(Vertices, TextImage);
var TextGroup = new Group(TextShape, DefaultShader);


The new BMPFont plugin can handle fonts with up to 2^32 glyphs, in Sphere RFN format version 1 or 2. It can print UTF-8 (v1 and v2) and ASCII strings. Obviously, we'd need to make some unicode RFN fonts to really take advantage of that, but now it's not an inherent limitation.

I plan on making a new TTF-RFN converter that can handle full unicode codepoints that uses this plugin's rfn API as its backend, in combination with SDL_ttf (or perhaps freetype directly?).
Title: Re: TurboSphere
Post by: N E O on July 28, 2014, 05:40:16 pm
Re Unicode RFN fonts - I remember coming across problems when trying to convert a TTF with 1000s of characters (might have been Arial Unicode MS, I don't quite remember) using the vanilla editor's conversion function, so I'm not entirely sure if the vanilla engine was ever able to handle such large RFNs in the first place. This would be a huge step forward in the I18N aspect of game dev in Sphere.
Title: Re: TurboSphere
Post by: Flying Jester on July 28, 2014, 06:11:00 pm
There are two issues with Unicode support: An RFN can only have 2^16 chars. You need more than 24 bits to represent all possible UTF code points, and UT8 version 1 in particular can theoretically have significantly more. Since the vanilla tools zero out all reserved data blocks, I read 2 extra bytes, and use them as the most significant bytes for checking the number of chars. So no vanilla RFN file will ever read as having more than 2^16 chars, and backwards compatibility is maintained.

The other issue is that the engine needs to be Unicode aware, and convert characters from Unicode strings to a series of codepoints. This is the part that I don't believe that Sphere 1.x does. I haven't actually checked, mind you, but I've never seen it done, and I seem to recall any extended chars being represented by the same certain char number (127?). TurboSphere parses Unicode strings and converts them to a series of code points, and skips codepoints that the font doesn't contain. Ideally, I would do something different (like the hex code dumps that you see in Unix a lot, or just 'char not found' characters).

One issue with that is that a RFN font must necessarily be at least 256+(32*largest_code_point) of any font. So if you imported an ascii font, but then added, say, code point 2002 (nbs), the font must be at least 292 KB plus however big the font was anyway. On the bright side, it would compress very well  ;D That's not properly a limitation, though. It's just a minor inconvenience.
Title: Re: TurboSphere
Post by: casiotone on July 30, 2014, 04:30:48 pm

One issue with that is that a RFN font must necessarily be at least 256+(32*largest_code_point) of any font. So if you imported an ascii font, but then added, say, code point 2002 (nbs), the font must be at least 292 KB plus however big the font was anyway. On the bright side, it would compress very well  ;D That's not properly a limitation, though. It's just a minor inconvenience.

You should consider just bumping the version number and including a new table in the format with the character ranges that the font contains. e.g. an array of (int32, int32). Then your font could have simply 0x00 0xFF 0x2002 0x2002 as the range table and only include 256 glyphs. Wouldn't be too hard to implement.
Title: Re: TurboSphere
Post by: Flying Jester on July 31, 2014, 12:18:58 am
I could. I'm not too keen on implementing any new formats until they have editor support, though.

I've update TurboSphere to use the new version of T5. Which means that it can use either INI or JSON for its configuration files!
It's also the first step to transparent compression and archive support.
Title: Re: TurboSphere
Post by: Flying Jester on August 02, 2014, 03:23:21 am
Working on Surface.transformBlitSurface (with a screenshot of RunnerGuy ;D ). It's almost done, except for some rounding errors (as illustrated by the occasional holes). I also need to add the side-juggling code, so that it actually uses the longest sides first.

Note that my implementation does not use affine transformation, but instead uses triple bresenhams with UV mapping. Because of this, the entire quad is rendered as a smooth surface (as though it existed in 3D space), instead of as two triangles as in Sphere 1.x.
Since all surface manipulations are done on separate thread, and surface operations can be performed out-of-order, we can do things like this much easier. Even so, we can draw 1920x1080 pixels of surface this way in under 10 ms. This is easily the slowest surface operation, but even my naive implementation (which lacks the third error-breaking pass) can draw very fast--over 100 fps of full HD resolution!
Title: Re: TurboSphere
Post by: Flying Jester on August 08, 2014, 01:02:54 am
I'm still stumped by how I can't set accessors in new V8--I'm doing it the same way I was before, and the accessor API is not exactly rich, so I'm not sure what else I can do.

I've put in a workaround in the plugin tools. Each JSObj prototype has a vector of accessors associated with it, and they are simply applied to each created object when it is wrapped.
Title: Re: TurboSphere
Post by: Radnen on August 08, 2014, 02:21:09 am
I have seen artifacts like in the image above. It is due to rounding, but more importantly it is due to the direction the blit occurs. What should happen is that you 'lock in' the target surface so that the pixels are accessed one at a time, and then you guess them in from the source surface (the image data). If you start from the source surface (image data) and 'guess onto' the target surface you'll get the default source color showing through where the rounding made a mistake.

Now, while the image may not have those holes anymore you'll start to run into an accuracy problem. It turns out though, our eye on large images or grainy textures will never notice you took adjacent colors to fill in the gaps. So we are safe there. Notice how this method does not care about rounding, in fact the 'bad' rounding only aims to enhance what you see.

edit: I should also add that's as basic as it get's, you can of course add filtering on top of that to increase image quality. I only explained a nearest neighbor approach.
Title: Re: TurboSphere
Post by: Flying Jester on August 08, 2014, 04:32:17 am

I have seen artifacts like in the image above. It is due to rounding, but more importantly it is due to the direction the blit occurs. What should happen is that you 'lock in' the target surface so that the pixels are accessed one at a time, and then you guess them in from the source surface (the image data). If you start from the source surface (image data) and 'guess onto' the target surface you'll get the default source color showing through where the rounding made a mistake.


I am well aware. But I'm not doing it either way. I'm drawing lines from corner 0 to 1 and then 3 to 2 with a constant distance between each point, and then drawing raster lines between equivalent points on those lines. I have two issues going on in that shot: I am just assuming that the first of the two guides is longer (certainly not always true) and using its length to determine how many raster lines to draw (I need to actually sort the sides so that I order them from largest to smallest and change the resulting UV limits and granularity for each guide and raster line), and two, I'm using rounded coordinates for the raster lines. This makes lines look nice when you want to actually draw a line primitive. Not as much when you want to ensure coverage.

This method is slower, to be sure, but it provides high quality texturing, and very easily avoids affine distortion.
Title: Re: TurboSphere
Post by: Flying Jester on August 08, 2014, 04:33:50 pm
So, here's a question:

Is there a good reason to make framerate changes be totally in-order?

Currently, FPS throttling is done statelessly. Once you call SetFrameRate, the framerate is changed atomically and immediately--no matter what the current relationship is between the render thread and the engine thread.

So, let's say you draw some frames (just worrying about actual FlipScreens in the pipeline), and you've set the FPS to 30 beforehand:

Code: [Select]

Draw Queue:
[Frame 0|Frame 1|Frame 2|Frame 3]
Engine:
[Frame 4]

So right now, we've sent four frames to the draw queue that aren't drawn yet. The engine has no idea, it's working on the fifth frame, and doesn't care what's happening in the render thread (or that the render thread is behind). All those frames will be drawn at 30 FPS (30-ish ms between each one).

Code: [Select]

[Frame 0 | 30 ms interval | Frame 1 | 30 ms interval | Frame 2 | 30 ms interval | Frame 3]


Then you change the frame rate to 60 FPS.

Ideally (theoretical ideally), if you then submitted two more frames, it would look like this:

Code: [Select]

[Frame 0 | 30 ms interval | Frame 1 | 30 ms interval | Frame 2 | 30 ms interval | Frame 3 | FPS Change Operation | 16 ms interval | Frame 4 | 16 ms interval | Frame 5]


Right now, what would happen is this:

Code: [Select]

The second you change the FPS:
[Frame 0 | 16 ms interval | Frame 1 | 16 ms interval | Frame 2 | 16 ms interval | Frame 3]
Then, with two more frames:
[Frame 0 | 16 ms interval | Frame 1 | 16 ms interval | Frame 2 | 16 ms interval | Frame 3 | 16 ms interval | Frame 4 | 16 ms interval | Frame 5]


And so, in once sense, you've interfered with past events. But you've also performed the action immediately, and applied it based on real time rather than on synchronous timings.

This is actually really different from how Sphere and other totally synchronous engines would do it--you can affect the FPS of frames you submitted in the past! This sounds like kind of a bad idea, because it introduces a certain amount of unpredictable behaviour (how many, if any, frames have their resulting intervals changed). But on the other hand, it is much lighter weight than sending a message through the render queue to make it happen in-order, and I also can't especially think of a reason why you would want to be 100% sure that it executed in order.
Title: Re: TurboSphere
Post by: Radnen on August 08, 2014, 11:59:24 pm
It's hard to say, you need to see how well that works when controlling a game. Maybe the rendering looks nice, but does the gameplay feel the same to what's going on? Synchronous engines like Sphere are unfortunately the only guaranteed way to make sure the game loop feels right when playing the game.

I read an interesting article on CPU's (the death of Moore's Law) and Video Games get the brunt of the damage because they rely heavily on monolithic synchronous game loops; especially on consoles or handhelds. There's very little in a game you can truly make threaded. If you can somehow make an asynchronous game engine that also feels right when playing, then you would have solved a fundamental gaming problem plaguing game engines like the Unreal engine.

I'd say do some tests.

But I also say that if you used a variable timestep it might not make a difference since it would feel the same.
Title: Re: TurboSphere
Post by: Flying Jester on August 11, 2014, 04:47:31 pm
I would like to think that I do, in fact, have a partial solution to the issue of feedback in asynchronous engines in TurboSphere.

And that is that TurboSphere is kind of semi-synchronous now.

It works so that the render thread or engine thread 'float' back and forth--one can be ahead of the other--although they can be synced at any time. The real trick I've been seeing is that you always perform syncs on the opposite thread than the one you want to have wait. If you want to change the framerate, you need change the input rate. This lets the framerate stay floating, which is what we really want. Since you know that it will normalize to whatever input you give it (ignoring stutter), you can control it from the engine's main thread.

We'll see what happens when I start doing full scene specifications, but I am going to try something similar. A feeder thread that is throttled, and a render thread that floats freely. The feeder thread is basically zero overhead (much less than the render thread, at least), so the same logic should apply. The engine is already designed to handle what happens if the render thread outruns the engine proper, and the entirety of the render system in Sapphire is designed to be thread safe, so another layer of abstraction for feeding the renderer will be no problem. The number one blocker here is making graphics Groups a fully recursive data structure, which means I need to learn more about multi-pass shaders.

In unrelated news: The Sphere 1.x compatibility script for Images is coming along, as is the primitives script. The translation from the old API to the new one is actually pretty simple, it's mostly just giving each Image object a corresponding Shape and Group, and having separate shapes for each primitive.
Which outlines the difference between the old API and the new one--in Sphere 1.x, you give objects temporary states. In Sphere 2.0, both assets and states are unique and individual objects that are used in combination. An image doesn't inherently `blit', it's a texture, a `shape' is just a collection of points with properties, and a `group' is a series of transformations. You need all three to properly make something happen, they are all persistent but mutable, and you can use them in any combination.
Title: Re: TurboSphere
Post by: Harry Bo21 on August 11, 2014, 06:58:56 pm
As usual I am very impressed bro, Keep up the good work :)

Cant wait to start using this :)
Title: Re: TurboSphere
Post by: Flying Jester on August 11, 2014, 08:07:11 pm
Thanks, that's great to hear!

By the way, I can go pretty far with back compatiblity, or I can not. I'm curious how much you guys want.

I am planning on doing most of Images (Blit, ZoomBlit, RotateBlit, TransformBlit), and some primitives (rects, triangles, lines, points, and circles). Those are all pretty simple to do, it won't take too long to get at least a reasonably close facsimile of the Sphere 1.x API working. After that, I was just going to work totally on the new system. Surfaces are mostly unchanged (the spec is closer to Sphere 1.x than what TurboSphere has ever supported in the past). Everything else will be done, for the most part, as Sphere 1.x API first, and then as Sphere 2.0.

Regarding the map engine, I do want to implement the Sphere 1.5 map engine. But I'm planning on doing it in script, using the new API from the engine to implement it.
Title: Re: TurboSphere
Post by: Radnen on August 11, 2014, 08:49:38 pm

Regarding the map engine, I do want to implement the Sphere 1.5 map engine. But I'm planning on doing it in script, using the new API from the engine to implement it.


This, this would be neat to see. My Sphere engine has a fast map engine, but it's done in C# and not JS so it has zero extensibility (like the old engine). It turns out you do not need a very fast map engine, but one that is fast enough. I still need to implement the new proposed Sphere shape/group graphics API (was it called pegasus or something like that)? If so I'll see how Jurassic fares in rendering a real time map engine in JS (it shouldn't be that bad, because rendering had been the bottleneck all along).

It also turns out that a games graphics engine is the most crucial thing that needs to be fast. Heck, having V8 for the JS is too much speed since game logic in reality - even for very large engines - is peanuts. But, I will say physics is crucial too and would indeed benefit being more hardcoded into the engine than soft-coded into the script layer (for that V8 would be preferable). That's why I might expose a physics API in my engine rather than provide a script based one. (Box2D for C# (https://code.google.com/p/box2dx/))

This project is coming along nicely, however. What progress!
Title: Re: TurboSphere
Post by: Flying Jester on August 11, 2014, 08:56:52 pm
Given how large the MapEngine API is, I suspect that in almost all cases a pure script map engine will be at least as fast. If you're willing to extend the native interface a little to accommodate it (I am), I see no reason why it won't actually be faster.


This, this would be neat to see. My Sphere engine has a fast map engine, but it's done in C# and not JS so it has zero extensibility (like the old engine). It turns out you do not need a very fast map engine, but one that is fast enough. I still need to implement the new proposed Sphere shape/group graphics API (was it called pegasus or something like that)?


Pegasus is the Sphere 2.0 API, Galileo is the Shape/Group API. I've left the Shape/Group API very barebones (TurboSphere actually extends it by letting you specify OpenGL shaders for Groups).

Right now, Pegasus is a part of the Sphere-Group repo, and Galileo only exists in the TS subdirectory. I actually need to update the spec. Using it a little, there a couple differences with how it turned out versus how I wrote it.
Title: Re: TurboSphere
Post by: Rahkiin on August 13, 2014, 10:59:57 am
Hi y'all.

@FlyingJester Good work on TS!

Regarding the FPS throttling problem you stated earlier:
If the game uses the FPS to do its physics and timing (every frame walking 1px, or whatever), the frame rate must be predictable.
But as Radnen noted, using FPS as timing is very bad: especially when there is some lag caused by another program or the engine itself.
So we would need some JS-accessible timing functionality for the gameloop.

I have been out of game engine spheredev for a couple of months, so excuse me if above does not make sense at all :P
Title: Re: TurboSphere
Post by: Flying Jester on August 13, 2014, 01:03:24 pm
Well, I would generally say that it's a better idea to record the delta-time between two events and use that. We should be assuming that, no matter what, it's possible that a section of a game could lag out, and run well below 60 or even 30 FPS. Then you need to decide if you want to be Donkey Kong 64 and adjust your speed, damage, etc. factors, or Castlevania 64 and just slow down. That's totally up to the game, we just need to provide the tools to do these things.

I was having some serious issues with stutter, even when using delta-time and not frame throttling. I've discovered the problem, and its an easy fix.

The main way to get timing in Sphere is GetTime(). This has millisecond precision. The issue is that, with a simple test, Sapphire runs approaching an order of magnitude faster than a single millisecond per frame. That, combined with the inherent stutter of controlling the maximum frame float between the render and engine threads meant that it was basically impossible to make anything operate smoothly. The difference of a millisecond or two is huge when you normally are getting <1 millisecond delta time!

A simple fix is to either change GetTime() to use higher precision, or to add a new, higher precision timing function. It's simple to get microsecond precision on every platform I know of, but I'd like to make a new function that returns seconds as a floating point, which as high a precision as possible. This will usually get nanosecond precision, but it can higher (OS X and GNU/Hurd can return a value based on FSB plus some offset), and occasionally lower (Solaris can only go to tenth-of-a-microsecond in userspace). Making it a floating point and guaranteeing at least millisecond precision would be a very safe choice.
Title: Re: TurboSphere
Post by: N E O on August 13, 2014, 02:26:58 pm
I'd recommend rewriting GetTime so passing a parameter (maybe bool asFloatingPoint, default=false for existing behavior) still returns milliseconds but with the proposed finer grain decimal expansion. That way there's no breakage to existing scripts with GetTime (of which there are a LOT) and it's a simple change in script to get higher precision than full milliseconds.

I'd love to see GetTime return 40526.7275932589... milliseconds ;)
Title: Re: TurboSphere
Post by: Flying Jester on August 13, 2014, 03:06:49 pm
I was planning on leaving GetTime as it is, and adding a new function (GetSeconds(), perhaps?) that returns a floating point representation of the seconds to whatever precision is possible/reasonable (at least as good as GetTime()/1000).
Title: Re: TurboSphere
Post by: Flying Jester on August 17, 2014, 09:51:21 pm
Group rotation and non-triangle/poly shapes working!

Plus, since I decided to just work around the issues I'm having with V8, it's all being set using member variables.

I've got frame-throttling working in script quite well. I may just put it in the engine that way.

Using updated SDL2, TurboSphere can fully use Retina displays, without disturbing the coordinate system. The only case where this changes anything is with line-based primitives and points. In those cases, they truly are a single pixel in size (which I think is very pretty, but it would be unexpected if you only tested your game on other platforms). The biggest gain is that nearest-neighbor interpolated images actually look pretty nice.
It's kind of strange...the edgy, crisp-but-jagged look of Playstation 1 and Sega Saturn graphics, combined with a ridiculously high resolution. The resolution is the antialiasing.
I can honestly say I've never seen any game (or even any program) that has this look.

I'll post a shot of Retina-mode when I get a proper scene going that shows it off.
Title: Re: TurboSphere
Post by: Flying Jester on August 20, 2014, 05:14:03 pm
I definitely made the right decision by changing the Input event handling code. Adding scroll-wheel support was only adding 3 lines (plus adding the constants)!
It would be pretty simple to make writing and using user-defined event filters and translators (usually one-line functions) possible from script.
That would be nice because the native hardcoded filters (keys, clicks, joystick buttons, and scrolls) would still benefit from the low overhead of being totally native (which could really matter), but script-defined translators and filters would allow wide-open access to the underlying system to script.

Here's a picture of retina mode. Not the best example (no proper textures), but it looks very nice in motion, and the 1px lines are sooo sharp.

Title: Re: TurboSphere
Post by: Fat Cerberus on August 20, 2014, 06:17:21 pm
Isn't the idea of Retina that the PPI is so high you can't make out individual pixels?  Seems like that could cause such 1px lines to become invisible if the viewing distance is too large.
Title: Re: TurboSphere
Post by: Flying Jester on August 21, 2014, 02:36:34 pm
The idea is that something like this (primitives with 1px definition) would approach the look of true vector graphics. Hence the Asteroids theme. You can't make out the aliasing. Well, yes you can, but ideally you wouldn't be able to.
Title: Re: TurboSphere
Post by: Rahkiin on August 23, 2014, 08:45:29 am
The retina thing is why I once proposed to use floating point coords and sizes instead of integer. It does not matter for JS, and storage is the same too, but integer math might be a bit slower.
The advantage is that you can keep 1pt = 1px on non-retina, and on retina, 1pt=2px, like Apple does. also, you can then do 0.5pt = 1px on retina: making it possible to do interpolation and smaller graphics but also keeping non-retina games to work on retina displays.

I think that is what we should be aiming for.
Title: Re: TurboSphere
Post by: Fat Cerberus on August 26, 2014, 01:58:16 pm
So realistically, once TurboSphere is ready for primetime, how much work would I be looking at to port a full-scale game like Spectacles from Sphere 1.x to TurboSphere?  I know you've said you'll have some sort of shim to ease the transition, but with all these changes to how things work under the hood, it seems like porting issues could still easily crop up just due to differing implementation, even if the legacy API is preserved.
Title: Re: TurboSphere
Post by: Flying Jester on August 26, 2014, 06:59:29 pm
Realistically, I have no idea. I am going to try to write a full map engine in script, but I also fully expect a lot of edge cases to crop up when doing that. I expect that most of the issues and time spent porting will be related to the exact behaviour of the map engine.

Most everything not related to the map engine should be rather simple to port over. Most of the legacy API is rather simple to shim, and except for the graphics API most of TurboSphere's API is intended to just be extensions and minor modifications to the existing API. Even then, I've found the 1.x graphics API to map fairly nicely to the new graphics API.

Re integers versus floats, only software vertex geometry is stored in ints, so it should be quite simple to change it to use floats. Everything else, including transformation factors, script-side geometry, and GPU-side geometry is stored as floating point numbers.
Title: Re: TurboSphere
Post by: Radnen on August 26, 2014, 08:08:07 pm
Accuracy is not easy. Even in my SFML engine, I may have the API implemented so that it works but it's not always exactly the same behavior. So a lot of careful considerations have to go into the code you write in order for two games to play the same. It'd be odd not being able to beat a game due to some weirdness in collision, color, or script execution.

The last one, the script execution part would be the hardest. Sphere get's... weird... undefined behavior when you try to call map scripts from a person script called in a delay script while all that was wrapped in an eval. It's uh, it can be difficult. Some games may love that weirdness. True enough persist games call person and map scripts all the time, and if there was an edge case in which a script could not be called you could lose out on data or gameplay.
Title: Re: TurboSphere
Post by: Flying Jester on September 12, 2014, 07:33:41 pm
I'm not too concerned with 100% accuracy. Not yet, anyway. What I need most right now is just a working, mostly correct map engine.

Anyway, I've update TurboSphere to use the newest version of V8. I'm now building V8 with the TurboFan enabled, too. I'm working on rawfiles now, mostly so that I can start reading the Sphere filetypes in script. At that point I should be able to see what I really need to add to the rest of the engine (in particular the gfx API), and begin working on the stuff that the other plugins handled by calling OpenGL in the raw.

Alternatively, I may want plugins to be able to generate surfaces in native code through Sapphire when they open resources...well, we'll see how things look when we get there.

Also, the new V8 seems to stutter a LOT less. I can run unthrottled and not see any noticeable stuttering! Of course, I can't rely on V8 always working that nicely, I'm on a brand new MacBook and I suspect it would be a lot less smooth on my Core 2 Duo laptop.

I'm quite happy to see some new V8 stuff finally showing up. Finally, there is a way to shut down the engine at any time without just jumping ship and calling Exit()! Plus, they've exposed the platform API, so now it's up to the embedder to supply platform methods (like threading calls and atomics). They still provide a default platform for common ones (well...OS X, Windows, Linux, and Android), but that actually will make it a lot easier to port TurboSphere. Especially since I've been making a lot of the stuff V8 needs now on my own, for Sapphire and other plugins.

Which is kind of neat. To a certain extent, both TurboSphere and the V8 instance under it will use the TSPR. Platform integration for the win!

I also may have mentioned this before, but you can pass V8 flags through TurboSphere with the -8 option (or --v8flags). For instance, if you wanted to use the `let' keyword, you could call TurboSphere like this:
Code: [Select]
./turbosphere -8"--harmony --use-strict"
Or
Code: [Select]
./turbosphere --v8flags="--harmony --use-strict"

I've been printing TurboFan stats using some V8 flags like this. This is probably one of the more useful things I've put into the base engine!
Title: Re: TurboSphere
Post by: Flying Jester on September 14, 2014, 10:50:54 pm
The RawFile API with TypedArrays has been implemented...but it leaks memory uncontrollably.

Why, you may ask?
https://code.google.com/p/v8/issues/detail?id=3562 (https://code.google.com/p/v8/issues/detail?id=3562)

V8 offers no way of reading an ArrayBuffer from native without taking ownership of its backing memory. Fair enough, they want to be sure it doesn't get freed by the wrong side. This takes it out of the GC pool (that is, when the handle is destroyed, the backing memory is not deallocated. The handle can still be GC'ed).

So, it would make sense that you could give the memory back when were done reading from it. Except that you can't! Once you look at it, you own it and V8 refuses to deal with it again.

This is an issue particularly with RawFile.write().

So, then I figured, ugly as it would be, I could perhaps forcibly set the argument to be a new ArrayBuffer with the old contents in it. It would theoretically work, if the API was there, since I am using a backing store that was allocated with the proper allocator by V8.

And there is a method to create an ArrayBuffer from memory you already have...but the ArrayBuffer deals with it as being externalized then!

So, I opened that issue. I don't particularly expect it to be answered. There is an oft-requested feature of V8, to be able to see the memory without externalizing. I'm going to assume that by some fluke of its design, that's dangerous with an internalized ArrayBuffer. That's why I asked for a way to hand the memory back. It keeps strong ownership, but still allows you to read from an ArrayBuffer in native without making its backing memory un-GC-able. And strangely, no one seems to have ever asked for it this way, just to see the backing memory without externalizing. It seems more natural to me for V8's design, given my years of working with it.

In the mean time, I'm going to poke at a hidden member so that I can make an ArrayBuffer with preallocated memory, and then mark it as not External ;_;
Title: Re: TurboSphere
Post by: Flying Jester on September 20, 2014, 11:28:57 pm
I managed to get TurboSphere compiled and mostly working for Windows again.

There are some issues with the rawfiles. As you may have seen on that issue I filed with V8, I actually ended up modifying V8 to have a 'Reinternalize' method for ArrayBuffers. I may have forgotten to apply this change to V8. Because TurboSphere is fairly fault tolerant about loading plugins, it just doesn't load the new scriptfs plugin.

I've added native Win32 threading, atomic, and timing backends for the engine and Sapphire. I also removed some of the very old and terrible plugin loading code that was for WIndows only. This reminded me how long it's been since I've had TurboSphere working on Windows!

I also got Mesa 10.3 rc3 compiled for Windows. I need Mesa for OpenGL 3 on my Win32 dev machine, and I've pulled in the OpenGL 3 and OpenGL ES headers from Mesa into the platform code for TurboSphere on Windows. If you are wondering, my only Windows machine right now is a Dell D630. This machine is a blight upon humanity. I'm considering including a copy of Mesa with the next TurboSphere release to help with testing.

While I have Mesa at hand, I've begun adding support for quick screen caps and texture uploads when TS is using Mesa (since it's all in system memory in Mesa anyway), and the ability to override which OpenGL library Sapphire loads so that you could easily switch to using Mesa or something else. This would also come in handy on Linux with some of the buggier proprietary drivers, just to test behaviour easily.
Title: Re: TurboSphere
Post by: Flying Jester on November 10, 2014, 07:23:12 pm
Needless to say, I've been away for a while.

I was pretty annoyed at how the V8 devs are handling the ArrayBuffer issues that I and many other embedders have been having. Well, I kind of still am. I'm not impressed with how Google manages the project. Then, I hit some issues with not owning a decent Win32 machine, which made everything much harder than it needed to be.
So I took a break and started working on an IRC Client (https://sourceforge.net/projects/kashyyyk/). Which was interesting, and it let me try out my networking library as well as write a much nicer monitor than the one in the C++ standard library. And do some GUI programming, which I still don't like.

I've updated TurboSphere to use the newest version of V8 again. It seems like the major API changes are really slowing down. Or maybe the way I'm using the API is just caught up in the backwaters of what hasn't changed. Either way, it was rather painless. I also reverted to using the slow, per-element setters for ArrayBuffers.

Which means that RawFiles work. I think (fingers crossed) that that was the only thing left the script-side Map Engine needs from native code, so I'm going to start working on the Map Engine once again.
Title: Re: TurboSphere
Post by: Flying Jester on November 11, 2014, 10:40:41 pm
Upgrading to Yosemite made some old ghosts show up again.

Every once in a while, I used to have HUGE latency when I started TurboSphere. A frame would take an entire second to render, even though we were getting solid framerates. Once I updated, I found that this was always happening.

The issue is that before sometimes, and now always, I was getting bad timing with the engine and the render queue. This is because I was using a timeout system rather than a monitor system. So, no problem, I actually just wrote a monitor implementation that I rather like, and it's been tested to work well in Kashyyyk.
Using it made the delay much better, but now we were always getting only about 100 FPS, no matter how little drawing we do. The more I try to push through the pipeline, the slower it gets. I've noted before that mutexes are slow in OS X for whatever reason, but it turns out that condition variables are slow like that multiplied by how many monitors are waiting on them.

So, I went back to an older idea I had, which isn't as nice as just using monitors, but turns out to work very well.

We can consider a call to FlipScreen() to not really be a call to clear the screen, but rather to be a procedural marker that we are finished with a frame's worth of drawing, and that everything following it is in a new frame. So, what if we rendered our frames into not one queue, but multiple queues? We can cycle through, skipping any that are being drawn to, one frame per queue. This can result in frames rendering out of order on extremely rare occasions, but this chance decreases the more buffers we use. The likelihood is inversely exponential (I don't properly know why that is, I just did a bunch of profiling). With only 3 buffers as I originally tried it happens just about every other frame. Using 16 frames, it basically never happens.
I do know that for it to happen, we have to change from rendering many frames quickly to suddenly taking a long time to render a frame. This rarely happens, and the difference needs to be more than the time it takes to draw a frame in OpenGL (which is usually an order of magnitude higher than how long it takes to draw a frame from script). Additionally, if we keep rendering quite quickly again, we will outrun the render thread again.
This leads to dropped frames, but there's no way to avoid that if you draw faster then the maximum screen refresh rate. Well, there are software ways, but in hardware you lose frames.
Fixing out-of-order frames simple. We just need to flush all pipeline buffers except the one currently being drawn from when we cross over it. But, as noted, this will very rarely happen. Most often, we will just lose as many frames as we have buffers. Which is disappointing, but it will keep the FPS smooth (whereas flushing all render queues can be slow, especially to be sure we don't flush out an rendering-in-progress queue), and if we are really outrunning the render thread that quickly we were going to lose a lot of frames anyway. Better to do it smoothly, I suppose.

Using multiple render queues that we cycle through, we are back up to the huge theoretical framerates I saw on good runs before, and we are now constantly running quite smoothly.
Title: Re: TurboSphere
Post by: Fat Cerberus on November 12, 2014, 11:47:09 am
Out of curiosity, how difficult would it be to get TurboSphere to compile/work on Android?  I ask because, well, technically Android is a Linux distro, but I don't know enough about the Android API to know how feasible a port would be.  Having Sphere on my S5 would be awesome though. :-)
Title: Re: TurboSphere
Post by: Flying Jester on November 12, 2014, 06:43:56 pm
I have in the past gotten the engine and the necessary libraries to compile using the NDK before, but no graphics plugin. Everything is known to work, except that I'm using OpenGL instead of OpenGL ES.

On top of that, I'm using very new OpenGL. OpenGL ES is based on older version of OpenGL. Once ES 3 is out, it shouldn't be crazy hard.
Title: Re: TurboSphere
Post by: Flying Jester on November 17, 2014, 05:18:12 am
We can now read maps and tilesets in script.

The biggest bottleneck? Pushing all the data to script. Because Google doesn't believe embedders can handle manual memory ownership changes.

I've also got a couple demos working. One is the Canvas particle demo that Neo posted a while ago. The other is the Emscripten Box2D demo. This is really cool, because we are using C++ that has been compiled to JS! It's pretty slow unfortunately, likely because V8 doesn't acknowledge ASM.js as a real thing.

The demo (using CubicVR and WebGL) is here: http://kripken.github.io/box2d.js/webgl_demo/box2d.html?10 (http://kripken.github.io/box2d.js/webgl_demo/box2d.html?10)

It actually is almost bearable in TurboSphere, until the boxes hit the ground.

The demos can both be found in the games folder of the TurboSphere github repo: https://github.com/FlyingJester/TurboSphere/tree/master/bin/games (https://github.com/FlyingJester/TurboSphere/tree/master/bin/games)
Title: Re: TurboSphere
Post by: Flying Jester on November 18, 2014, 12:10:02 am
The map engine is going well.

You can see the script-side map engine here:

The compatibility layer and main engine:
https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/map_engine.js (https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/map_engine.js)

Map implementation:
https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/map.js (https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/map.js)

Tile and Tileset implementation
https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/tileset.js (https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/tileset.js)

Any questions or comments are welcome.

My first thoughts on making the Map Engine in script are that this is the best way to do it, hands down. After many graphics API revisions and map engine attempts, I find this combination to really be my favorite. A smaller core graphics API designed for batch operations and persistent structures behind a JS map engine, with a wrapper for the 1.5/1.6 API.

I wish I had decided to put more of the engine in script a long time ago. Casiotone was right! And I was wrong!

You might notice I put some other scripts in the system/scripts/turbo directory. Most are designed to at least be helpful for other engines (notably the image.js, which shims the old graphics API for the new one). The for_each.js is a Mozilla script. It's not strictly necessary, but I include it in my system scripts that use Array.forEach just in case they ever get used in an engine with older JS.

I've also added a simple CoffeeScript demo, showing how to run CoffeeScript from TurboSphere.=
Title: Re: TurboSphere
Post by: DaVince on November 18, 2014, 05:06:29 pm
I just took a peek at the code, and, well... it seems so much easier to do it this way! Especially all the old compatibility get and set functions look like a breeze to implement. And I like how you put it all in Turbo.CurrentMap, very difficult to confuse what the heck that would be. :)

I'm going to take some time to look at the scripts more in-depth later, but I definitely like where you're going with this. Looks like the easiest and best way to implement the map engine, as well as make it extenable when you need it to be!
Title: Re: TurboSphere
Post by: Radnen on November 19, 2014, 12:15:40 am
The thing I see about the map and tileset is it kinda looks a bit hackish (https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/tileset.js#L161-L177) still. I'd love to see more normal color conversion and data access abstraction going on just to make it a bit more readable.

But overall I think this is awesome! I might even feel inclined to do this to my SSFML version and just compare the results. The bonus is the map becomes extensible by the user if implemented in JS. That also means no more update and render scripts as we know it. You can get more flexibility in how the loops are used and in what order they are executed. It paves the way for infinite-scrolling map engines, Terraria like procedural generation, and even any other kind of map engine you desire whether it is isometric or platformer.

So, yeah, this is pretty cool.
Title: Re: TurboSphere
Post by: Flying Jester on November 19, 2014, 05:58:30 am

The thing I see about the map and tileset is it kinda looks a bit hackish (https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/tileset.js#L161-L177) still. I'd love to see more normal color conversion...[/url]


Quite understandable. I realized that the handler for BPP > 32 could be unified almost perfectly with BPP = 32, so I did that.


...and data access abstraction going on just to make it a bit more readable.


I'm curious how you'd like the data access changed.


Yeah, a lot of stuff starts to look pretty silly if you can touch the Map Engine in script. Especially how map scripts are handled.
Title: Re: TurboSphere
Post by: N E O on November 19, 2014, 09:45:37 am
While I support migrating Sphere's formats to being processed in script, until we can definitively finish up the 1.x branch we need to bank putting this in vanilla under "sphere-futures" until either TurboSphere becomes the de facto Sphere engine or Sphere vanilla catches up in speed, power, and/or capability.
Title: Re: TurboSphere
Post by: Radnen on November 19, 2014, 09:16:32 pm


...and data access abstraction going on just to make it a bit more readable.


I'm curious how you'd like the data access changed.


Real user-level IO reads bytes you tell it to read and manages it's own pointer. I see a lot of this going on:
Code: (javascript) [Select]

array[at++];


Which is kind of tedious and adds a layer of hard work. What if you forget an at++? Or you didn't concat the two correct bytes? Take this area of code for example,

Code: (javascript) [Select]

    this.animated_next = Turbo.dByteCat(array[at++], array[at++]);
    this.delay         = Turbo.dByteCat(array[at++], array[at++]);


It should totally become:
Code: (javascript) [Select]

    this.animated_next = ByteBuffer.read16();
    this.delay         = ByteBuffer.read16();

// ByteBuffer.js:
ByteBuffer.prototype.readInt = function() {
    return Turbo.dByteCat(this.array[this.at++], this.array[this.at++])
}


No need to hand-increment raw binary pointers, and no need to manually concatenate bytes each time an int or higher is observed. It cuts down in complexity, and increases readability, and provides less areas where there are issues. I use this API to read the files in C#, for example: C# binary reader (http://msdn.microsoft.com/en-us/library/system.io.binaryreader%28v=vs.110%29.aspx).

edit:
renamed readInt to read16, for clearer picture these are 2-byte 'short' values being read and not 4 byte integers.
Title: Re: TurboSphere
Post by: Flying Jester on November 21, 2014, 07:46:52 pm
That's a very good point. I was thinking of it at more of a memory-mapping level than a stream level, but using stream semantics does make it a lot simpler and less error prone.

I added a simple stream/reader abstraction that covers arrays, ByteArrays/TypedArrays, and RawFiles, to that end.

Additionally, I added a simple scheme system for defining the binary file formats (well, chunks of them). This had always kind of bothered me, but at this point I think I can tackle it. I added a system/formats directory, which contains json files that define chunks of the Sphere binary formats, as well as a mechanism for loading them (or arbitrary schemes in general) and reading an object from a reader based on a scheme. This makes the construction of any given file usually little more than just defining the order of the chunks (often based on previous chunk data) and structuring the resulting data, which is much cleaner than how things were before.
Title: Re: TurboSphere
Post by: Radnen on November 22, 2014, 01:13:17 am
Sweet! :)
Title: Re: TurboSphere
Post by: Flying Jester on November 26, 2014, 06:14:49 pm
It's only needed for two fields of a single Sphere filetype, and only for single precision, but I've added the ability to load binary fields of data as floating point numbers of arbitrary precision. To do this, I ported a Node.js module that handles ieee754 numbers.

A fellow on the v8-users group pointed me to a trick (https://groups.google.com/forum/#!topic/v8-users/iiZr67iAfU0) that Chrome uses for CanvasData. So even Google knows their API is almost useless in the real world!

This makes maps load instantly, since I don't need to make two allocations and three copies for every ArrayBuffer I fill in native.
With that, we can efficiently use readers for everything, and take advantage of the different widths of typed arrays for almost all data conversions.
So, from a 'how much works yet' standpoint, still where we were last time. But from both a speed and stability perspective, things are much better.

Another cool thing--V8 supports PowerPC now. This also means it has support for any of its targets in BE mode (theoretically, only tested for PPC I've heard). The IBM team did this, so I assume it mainly supports BE.
The main issue that remains is that TurboSphere uses modern OpenGL. Before Leopard, I am pretty sure that it won't work out of the box. Fortunately, this is something I have dealt with before. I use Mesa3D on my Windows box to fix the fact that it has Intel graphics, and I believe that the same could be done on older OS X.
So, if one was going to port TurboSphere to PowerPC OS X, here's what you would need to do:


I may dig out my old G3 iBook soon just for this.
Title: Re: TurboSphere
Post by: Flying Jester on November 29, 2014, 05:11:02 am
Maps and Tilesets now fully load, and maps will fully draw all layers.

This brings up two very simple and effective optimizations I am going to add. The first one (which is necessary for the second) is that we can fairly simply make the tileset a texture atlas. This has three advantages:


The last one is the most important. It lets us define the entire layer as a single mesh, and means that we only upload and render a quarter(ish) as many vertexes as we do now, perform a single texture binding, and only a single GLSL Uniform set for every layer. It also allows us (since recursive Groups aren't a thing yet) to put the entire layer and any sprites it has into a single Group. Since it's a part of the API to allow a render script per-layer, we can't always guarantee that the entire map could be one uninterrupted drawing operation. But this should make the maps draw as efficiently as possible.
Title: Re: TurboSphere
Post by: Radnen on November 29, 2014, 05:25:19 am
Yes! I have been doing that with my SSFML. :)

There is one small caveat, however. You need to also be able to quickly update the source-rects on that texture atlas for tile animations. So nothing is going to be truly baked. You can either modify the texture-coords in the "layer mesh" or you can do what I did and redraw the animated tiles within the texture atlas if you treat the atlas as a 'render target'. I'm sure it's faster. I reckon, updating a small 16*16 sized tile graphic is far faster than supposedly changing large layer meshes for each layer that has a tile animation. I might have benchmarked this and noticed the texture atlas method winning over the mesh-updating method by alot, especially on maps with whole layers dedicated to animated tiles (such as water or swaying grass).

So, it's up to you, maybe you'll figure out something smarter? Also I hope you have a good algorithm for tile animation, you should never do more than a singular loop to animate all the tiles. That's why I redraw the atlas, it was faster in terms of algorithmic complexity O(n) than compared to updating the layers O(l*n2). I was able to get really fast animation performance so that whether you have 1 tile animating or a hundred, at different intervals, and at different lengths, would all update at the same speed. No matter if you have a million tiles or not, it always took the speed of refreshing the atlas and never the entire map.
Title: Re: TurboSphere
Post by: Flying Jester on December 02, 2014, 08:00:35 pm
I think what I'm going to try is putting all tiles that are not animated into a single shape, and all the animated tiles into their own shapes.

One thing that this makes clear is that TurboSphere's bottleneck right now is how many Shapes it can draw. We can push out about 200 per frame with no problem, but much past that and things begin to slow down past 60 FPS. That is unfortunate--we can push TONS of vertices through! I'll see if I can find why this is the issue.

Just looking, a part of it may be that for some reason (which I don't recall) I disabled the ability to push an entire Group through the pipeline from the main thread. Instead, all the Groups' Shapes and state changes are calculated on the main thread and then pushed through one by one, even though Groups can do this on their own from the render thread.

EDIT: Yes, that was a huge part of it. Changing the group to iterate its Shapes on the render thread almost doubled the framerate.

I managed to get about 25% higher framerate with something that looked unlikely to me. Apparently, checking and updating the draw mode every frame was putting a bigger strain on the system than I thought. I would guess that this just because any time on the render thread is prime real estate.

Switching to use a texture atlas and a single Shape fixes the performance issue totally on its own, though.
Title: Re: TurboSphere
Post by: Radnen on December 02, 2014, 11:27:38 pm

One thing that this makes clear is that TurboSphere's bottleneck right now is how many Shapes it can draw. We can push out about 200 per frame with no problem, but much past that and things begin to slow down past 60 FPS. That is unfortunate--we can push TONS of vertexes through! I'll see if I can find why this is the issue.


That's the key fundamental idea behind the sprite batch: draw all objects as a single contiguous vertex mesh as often as you can. Vertices are cool in that you can really set both their texCoords and coords to be anything, anywhere on the screen, supplied they keep the same source texture and draw mode. But sometimes in sprite batching you don't always win. Currently, in my SSFML, if you draw the same graphic a million times it's really fast, but if you draw in an A/B fashion, as in, if you swap between two totally different textures, it slows down a lot since it's committing the batch each time the texture changes. I guess my next idea to speed things up is to truly order the textures so that all like textures are placed down on their own 'layer', but then I worry about render order. It'd be nice to see just 2 commits of the sprite batch when drawing patterned blocks. Of course if texture A and B came from the same source texture, then this doesn't matter, as is the case with map layers, spritesets, fonts or really anything that has a texture atlas.
Title: Re: TurboSphere
Post by: Flying Jester on December 09, 2014, 06:54:17 pm
I was mostly surprised that there was such a higher overhead to non-optimized Groups and Shapes--but it's a lot better now. It's mostly similar to the issues I was having before, time on the render thread is a lot more precious than I originally thought.

I've added preliminary spriteset support. Curiously, just implementing spritesets the way that made the most sense to me, I implemented almost verbatim the Sphere spriteset_object's interface. Without knowing the actual history, just seeing the format docs, I'd guess that the spriteset interface and format has seen more love than most others.
Title: Re: TurboSphere
Post by: Flying Jester on December 17, 2014, 06:03:51 pm
I've been doing a little housekeeping with T5 and the engine proper.

Plugin loading is no longer a terrible mess! And T5 can now properly iterate over the filesystem, which also means I can nicely add the FS operations to script.

I turned on all warnings and error-on-warning for TurboSphere with Clang and GCC, and it's all still working! Hurray!
Title: Re: TurboSphere
Post by: Flying Jester on January 06, 2015, 02:37:44 pm
Getting closer to a working map engine. Spritesets are fully loaded (including legacy versions), and some of the basic person functionality exists.

Most importantly, TurboSphere FINALLY correctly reports errors in Require'd scripts! Now if only I could figure out the issues with the wrong line numbers on non-Windows platforms!

Very unfortunately, we will need to start using Googles HORRIBLY BUGGY gclient tools to build or even fetch V8 soon--even though the project moved to using git!
If you're curious how buggy they are, if you start it with SVN installed but no git, and then later install git, you will need a fresh checkout of gclient. If you do it the other way around, it will mangle your git config when it next starts. Better set everything up perfectly, 100% correctly, first time!
Title: Re: TurboSphere
Post by: Fat Cerberus on January 06, 2015, 02:46:28 pm
Sounds like V8 is becoming more trouble than it's worth.  Have you considered switching engines?  V8 really doesn't seem like it has the huge performance advantage it once had...
Title: Re: TurboSphere
Post by: Flying Jester on January 06, 2015, 05:42:36 pm
That is true.

I have seriously considered switching to Spidermonkey. V8 is still somewhat higher performance, but SM has occasionally been leapfrogging it. Additionally, SM has an interpreted backend, something that I appreciate more and more--plus it has a native UltraSparc port.

But I also know a bit about Mozilla's build system, Mach. When I was working there, everyone made fun of it. It is slow (compared to autotools), very difficult to configure (although much easier than autotools!), and prone to unnecessarily rebuilding objects. The saving grace is that it integrates quite well with autotools, SCons, and CMake, and is very reliable. The general consensus was that for something the size of XUL/Gecko, SM, and the supporting (mozilla updated) libraries, this was kind of inevitable.

I would guess this is also what is happening with Blink/Webkit, V8, and Google's versions of their libraries. Now that Google is the caretaker of their own WebKit fork (which is increasing becoming different from WebKit), having a unified build probably seems better. They just happened to choose the GYP WebKit build, and therefore the GYP V8 build system, all governed by gclient. Which would not be my first choice.
Title: Re: TurboSphere
Post by: Fat Cerberus on January 06, 2015, 05:48:22 pm
Why are you building the engine yourself anyway?  You mean to tell me Google doesn't have a minimal package with just the binaries and import libs/other bindings?  SM used to, don't know if Mozilla still does that either though...
Title: Re: TurboSphere
Post by: Flying Jester on January 06, 2015, 06:05:14 pm
I build the engine myself because all of Google's projects use it as a static library.

There is no official Google binaries because Google never uses it as a separate binary. Additionally, I enable a few things that are non-standard (the better debugger, GDB-JIT support, and internal UTF8).

Given the new paradigm for rapid releases of browsers, I don't think Mozilla does that anymore, either. They only barely make standalone Gecko releases anymore, for similar reasons.
Title: Re: TurboSphere
Post by: Flying Jester on January 08, 2015, 08:33:44 pm
I fixed changing the Image of a Shape.

Trying to use this uncovered a HUGE problem with modifying Images and how they are managed. It's all fixed now, and I'm glad I caught the issue so soon.

Now, Images are properly maintained until their Image object and all Shapes that use the Image are destroyed, and changing the Image of a Shape using

Code: [Select]

some_shape.image = new Image("image.png");

correctly removes the reference for the old image.

Previously code like that would ALWAYS destroy the underlying GL texture of the old image, which would either cause that image to be garbage or blank, or (most perplexingly) the next call to `new Image' would create a new GL texture of the same name. This made the old Images that were replaced hold the new texture, because the old image used the old GL texture name, which was now the new image's texture, and had been overwritten by the new Image!

That took the last two days to figure out. It was fun, but also quite a nice challenge..
Title: Re: TurboSphere
Post by: Flying Jester on January 16, 2015, 06:31:01 pm
I just found out that V8 moved repos just a little bit ago (November 2014) on Github without much notice.

The new build system is even more broken. It refuses to directly build with anything but a prebuilt compiler that it insists on downloading (which is almost 400 MB). I finally got all that working, and I find that V8 has lost the ability to be directly compiled into a shared library. Legend has it there is a new switch to enable this, but it does not seem to work.

TurboSphere literally cannot use V8 as a static library.

In addition, running the included tests for V8 show it has actually regressed in performance since the older version I've been pulling, by almost 8%. It seems that TurboFan and CrankShaft are in some kind of weird conflict, particularly on non-jsm scripts.

I wonder how hard it would be to compile SpiderMonkey...
Title: Re: TurboSphere
Post by: mezzoEmrys on January 21, 2015, 11:49:07 pm
If you do switch to SpiderMonkey, would we get the benefit of being able to use the ES6 fat arrow notation?
Title: Re: TurboSphere
Post by: Flying Jester on January 22, 2015, 12:56:32 am
That would happen eventually anyway. To be honest, I'm not too keen on a lot of ES6. A lot of it sounds like things that people who don't really understand JavaScript think it needs. With the exception of a few small bits, like 'let' and string templates, I haven't really kept up too much on what engines can do what.

Maybe I will use SpiderMonkey. It's actually got a decent speed advantage right now (apparently TurboFan's teething problems are fairly significant), and looking at the docs, a lot of what I've needed to do by hand before is more ingrained in SpiderMonkey.

Plus, it can actually manage its memory, rather than just assuming the process will be terminated before too much garbage builds up like V8 does. And its got proper multi-threading support. And a real interpreter to fall back on, so it works on all platforms out of the box.

I remember I said a long time ago that I wanted to use V8 because I trusted it would stay significantly faster than SpiderMonkey. That kind of hasn't been the case in the last year or two.
Title: Re: TurboSphere
Post by: Flying Jester on January 23, 2015, 05:40:10 pm
I've got a new build of the engine (including the configmanager) using SpiderMonkey. No plugins yet (that's the hard part!), but it can run scripts at least. I guess years of using V8 taught me something about JS engines.

I noticed right away that SpiderMonkey has much better control of garbage collection (including a 'zealous' mode and a 'debug' mode), which is nice. For all I saw, V8 basically had no garbage collector at all.
I also noticed that SpiderMonkey is much more advanced with regard to multithreading and proper threaded use. I can easily imagine adding asynchronous callbacks with SpiderMonkey, whereas this was going to be a huge pain with V8.

SpiderMonkey actually makes me think it wouldn't be so hard to have multiple games run simultaneously from a single instance of TurboSphere. I can't think of a good reason to do this, though, except that it would be cool. And it would prove a certain cleanness of the engine's design.

Of course, coming up with a working engine build so quickly was vastly helped by SpiderMonkey actually being documented. "Read the Source" is not documentation. Not even close.

The flip side is that SpiderMonkey is a bit more verbose than V8 is. I'm OK with this so far, though. It's very consistent, whereas V8 has a rather irregular API.

I also took the opportunity to add multi-context tools to the configmanager, as well as cleaning out a lot of crap from it.
Title: Re: TurboSphere
Post by: Radnen on January 23, 2015, 08:54:05 pm
YOu sound skeptical and/or resistant in your diction, why is that? Is it performance or the fact you had been such a fan of V8 for so long? To me it seems like a slam-dunk win over V8. Is it developer fatigue (:P)?
Title: Re: TurboSphere
Post by: Flying Jester on January 23, 2015, 10:33:16 pm
It's actually really exciting to be working on this. I am surprised by how very much of my knowledge of V8 is helping here. Getting quick results with such huge changes is really satisfying!

It's a little disappointing to switch away from V8 after so long, so I guess I am constantly thinking of things like, "OK, but will SpiderMonkey let me down with using ArrayBuffers like V8 did?", or "Will SpiderMonkey's API change constantly now?"

That's also why I'm taking everything that seems better with a grain of salt. I am bracing myself to be let down, and to switch back to V8. If this proves disastrously difficult, then that's what I will do. At the very least, many of my long standing gripes with V8 seem to not be issues with SpiderMonkey.

So I guess I'm just speaking with the same kind of ambivalence I feel over the decision. It looks great on the surface, who knows what waits below.
Title: Re: TurboSphere
Post by: Flying Jester on January 25, 2015, 08:01:42 pm
https://github.com/FlyingJester/TurboSphere/tree/SM (https://github.com/FlyingJester/TurboSphere/tree/SM)

The SpiderMonkey branch is now live on Github.

No plugins working yet, but full error reporting and some of the engine proper's functions work. Error are also reported with the correct line number now, which never worked right in V8!
Title: Re: TurboSphere
Post by: Flying Jester on January 30, 2015, 03:55:12 pm
I've updated the plugin tools to support SpiderMonkey. Still a few issues, but things are shaping up.

SpiderMonkey has a much nicer concept of a JS class. In V8, you just make a template and then a constructor which is no different in concept than any other native call. In SpiderMonkey, you actually register a JS class with the engine.

While the V8 way could work fairly well, too, in the last year or so V8 has been moving away from all that, though. Chrome apparently doesn't properly use even those templates, instead defining all properties to an object whenever one is created, and V8 seems to be moving that way, too. Very logical for a prototype based language -_-
Title: Re: TurboSphere
Post by: Fat Cerberus on January 30, 2015, 04:18:39 pm
I kind of get the feeling V8 was never meant to be embedded.  It's definitely pretty clear that it's written to cater to Chrome and everything else is secondary.  Spidermonkey on the other hand has always been designed for embedding.  There's official documentation, it's self-contained in a dynamic library, and you used to even be able to get the standalone binaries for linking (don't know if this is still the case).
Title: Re: TurboSphere
Post by: Flying Jester on February 02, 2015, 06:55:31 pm
The ScriptFS plugin works now. When it comes to defining JS classes, SM is miles ahead of V8.
Sapphire will be a bigger challenge, but it really shouldn't be that bad. A lot of what I've had to do for it so far is removing abstractions I made that are now redundant with SpiderMonkey's API.

@Lord English: I totally agree. V8 is very hostile to non-Chrome needs.

As evident how, on several occasions, I tried to provide small patches that added the missing ArrayBuffer functionality to V8 in different ways, and every time was given a different reason why it was bad. That was a long running issue I had, and so did many other embedders.

SpiderMonkey just has the functions I need for that. Because they are a useful thing for a JS engine to have, whether Firefox specifically needs them or not.

I can also now properly add an ArrayBufferToSurface function. This was quite literally not possible with V8.
Title: Re: TurboSphere
Post by: Radnen on February 04, 2015, 11:04:06 pm

I can also now properly add an ArrayBufferToSurface function. This was quite literally not possible with V8.


I don't get it, is it because this is an ECMA6 Array Buffer going to a Sphere Surface, and on the C++ side you didn't have access to the array buffer or it's contents?

If that's the case then SM is definitely a huge win since I'm pretty sure you're intending to use ArrayBuffers quite thoroughly throughout, like for vertex maps and such.
Title: Re: TurboSphere
Post by: Flying Jester on February 04, 2015, 11:32:27 pm


I can also now properly add an ArrayBufferToSurface function. This was quite literally not possible with V8.


I don't get it, is it because this is an ECMA6 Array Buffer going to a Sphere Surface, and on the C++ side you didn't have access to the array buffer or it's contents?


It's because ArrayBuffers are single-use in V8. In V8, once you get the address of underlying memory of an ArrayBuffer, that ArrayBuffer can not be GC'ed anymore. Even worse is that until recently you could only get the underlying address once. You've either got to leak it and let script keep using it, or free it with no way to tell script that it's not usable anymore! Both totally unacceptable--either I've got a memory leak, or I can cause a segfault from script? How is this a useful API?

With SM, I can just get the ArrayBuffer's underlying memory buffer address without making un-GC'able. I don't see why this isn't the case for V8. It works fine in SM, and I've even hacked the same functionality into V8 a couple times. It works fine, it's how the API should work, but the V8 authors have some deep-seated fear of this.

This way it is simple to read a portion of a file representing a bitmap (like in a Sphere tileset, spriteset, or font), and totally from script, pass the resulting ArrayBuffer directly to an image. Fairly close to the efficiency of native, but still leaving it possible to modify the bitmap from script.

You can still get the contents, but since you can't just do a memcpy/std::copy from the native side, you have to actually iterate the ArrayBuffer using a JS array or a TypedArray, even in native. I tried to do that, since it's the 'correct' way with V8, but it is just horrifically slow. It's not an option for anything but a trivial example. That's why I am so disappointed with V8 ArrayBuffers, a lot of the advantages that having native memory available to script could have are made impossible by how V8 handles them. So it's technically possible, but it would actually be an ArrayToSurface function, applying the ArrayBuffer to a normal JS Array or iterating a TypedArray like a normal JS Array. This is unacceptably slow and complex. A map with 32 tiles, each 16x16, and a 4-layer playing field of 64x64 tiles, takes about 2 seconds to read this way on my Mac Book, it should be much, much less!


If that's the case then SM is definitely a huge win since I'm pretty sure you're intending to use ArrayBuffers quite thoroughly throughout, like for vertex maps and such.


ArrayBuffers are key to how I want to do I/O with TurboSphere, almost exactly how Sphere uses ByteArrays. It also allows for a lot of shortcuts when reading files (like above, read a bitmap from a file in one operation, turn it into a Surface with one operation, with only a single ArrayBuffer being passed around). So they aren't really much different from how ByteArrays are in Sphere 1.5, but they are obviously native to JS, and I intend to do much more I/O in script.
Title: Re: TurboSphere
Post by: Radnen on February 05, 2015, 02:16:08 am

It's because ArrayBuffers are single-use in V8. In V8, once you get the address of underlying memory of an ArrayBuffer, that ArrayBuffer can not be GC'ed anymore. Even worse is that until recently you could only get the underlying address once. You've either got to leak it and let script keep using it, or free it with no way to tell script that it's not usable anymore! Both totally unacceptable--either I've got a memory leak, or I can cause a segfault from script? How is this a useful API?


Ouch, wonder how Chrome handles that? I almost don't wanna know...
Title: Re: TurboSphere
Post by: Flying Jester on February 07, 2015, 08:44:24 pm
I don't want to say that I know for certain, but I suspect the answer is related to how each instance of V8 in Chrome has its own process...

Sapphire now compiles. There are still a few bugs to work out, but right now file reading and (mostly) graphics are working again, this time with SpiderMonkey.
SpiderMonkey provides a way to add a callback when a value is set by script, without acting as a setter. It is just a callback that fires on the set, and can optionally modify the value that is being applied. V8 used to call this an Interceptor. Used to. They are either deprecated, or fully removed now.

This lets us easily have the Shape array of a Group or the Vertex array of a Shape be accessible as a proper JS Array from script, and any changes to that script be applied in native without extra calls from script or polling from native. This required extra caching of the JS array by native and full getters and setters with V8.

One thing that caught me off guard about SM is that it crashes if you try to set an exception with another exception pending, before returning to script. This isn't really an issue, but it's different than how I worked with V8. Normally you wouldn't want to do this anyway, but consider this example:

When we check that the arguments of a function are the correct type and that the argument array is the correct length, we set an error if they aren't. In functions with variable arguments, I previously just ignored the set exception for the function signatures that didn't match. In SM, this actually makes no difference if I also indicate that there is no pending exception from the native function (the flag for exceptions can be true or false regardless of whether there is a set exception). If I set the pending exception a second time (like if another signature check fails), then BAM! Crash.

Fortunately, I built a suitable capability into the plugin tools previously. The signature-checking function can have a flag set to indicate we should not set an exception if the sigs don't match. This was primarily intended for when there are optional arguments on the end, but it works just as well here.
Title: Re: TurboSphere
Post by: Fat Cerberus on February 07, 2015, 08:45:16 pm
Just clarify for me: What is Sapphire exactly?  Is that just the codename for TS, or...?
Title: Re: TurboSphere
Post by: Flying Jester on February 07, 2015, 09:07:47 pm
Sapphire is the graphics plugin. It replaced the old SDL_GL_Threaded plugin last summer (which was a fork of SDL_GL, a fork of SDL_gfx).

It provides the primitives API, Image/Surface loading and API, and screen/window related functions.

The Map Engine for TurboSphere requires only ArrayBuffers, RawFile functions compatible with Sphere 1.5, input functions compatible with Sphere 1.5, and a Sapphire-compatible graphics API. That's why I focused primary on the ScriptFS plugin and Sapphire for the first SM-based plugins. InputSDL will be next!

The API for Sapphire is on the Sphere-Group/Pegasus repo, and is called Galileo. It's still a TurboSphere-named extension, mainly because nothing else supports it (yet?). I also wrote a mostly complete Sphere 1.5-compatible wrapper for it once.
Title: Re: TurboSphere
Post by: Radnen on February 07, 2015, 09:12:13 pm
I too would like to implement that API in SSFML.

We can say "Pegasus Ready" Sphere Engine. lol
Title: Re: TurboSphere
Post by: Flying Jester on February 08, 2015, 05:16:29 am
It works. 100% functional parity with V8 TurboSphere. Except that I messed up the default UV attributes for Vertices, so Shapes with 4 Vertices and no explicit UV coordinates end up upside down.

Just ignore that :P

(http://rpgmaker.net/media/content/users/23741/locker/spidermonkey_turbosphere.png)
Title: Re: TurboSphere
Post by: DaVince on February 08, 2015, 08:03:40 pm
Wow, awesome. Looks like the dude is just sunbasking to me.
Title: Re: TurboSphere
Post by: Flying Jester on February 09, 2015, 08:41:17 pm
I fixed the default UV attributes, as well as added the finalizers for Sapphire.

I've put the current list of all functions, objects, and types on the TurboSphere github wiki (https://github.com/FlyingJester/TurboSphere/wiki/Function-List). With the exception of ShaderProgram objects, it is 100% accurate.

I plan on adding a small tutorial for the new graphics API, as well as a bit of explanation for the plugin tools and plugin API.
Title: Re: TurboSphere
Post by: Flying Jester on February 11, 2015, 11:30:25 pm
I've exposed the old Surface methods for primitives again.

I've also created a page on the TurboSphere github wiki for  using the Galileo API (https://github.com/FlyingJester/TurboSphere/wiki/Using-Galileo). It's not even close to complete, but it's good to at least have something.
Title: Re: TurboSphere
Post by: Flying Jester on February 12, 2015, 04:00:48 pm
Font loading now works, totally in script.

(http://rpgmaker.net/media/content/users/23741/locker/text_demo.png)

That shot also shows that SetRenderScript works.

I'm curious how well the font system from the Turbo runtime works in minisphere or Sphere 1.5. Without an in-engine way to read directly from a file to a surface or image from script, loading would be a lot slower, but it should be pretty close in speed to a native solution when running.

I'm not sure if Lord English or Radnen are aware of this, but I have been working on some JSON schemes that represent the Sphere filetypes: https://github.com/sphere-group/formats (https://github.com/sphere-group/formats). I use these to make it easier to read files from script. I have no idea if they would be helpful for the other engines, but they are there.
Title: Re: TurboSphere
Post by: Fat Cerberus on February 12, 2015, 05:03:50 pm
I did see those, yeah.  Wasn't sure how exactly to utilize them, though.  Honestly Sphere's formats are simple enough anyway that it's pretty trivial to write loading logic (in any language) just by reading the format specs.  That's how I did the RFN loading in minisphere, by referring to this:

https://github.com/sphere-group/sphere-doc/blob/master/internal/font.rfn.txt

As I found out though, sometimes the docs are misleading--for example, the font spec above says the pixels in a v2 font are RGBA, when in reality the actual representation is ABGR.
Title: Re: TurboSphere
Post by: Flying Jester on February 12, 2015, 05:11:37 pm
I've been writing them using the internal docs from 1.6 as a reference.

What I do in TurboSphere is parse each component so that a script in the Turbo runtime, format.js (https://github.com/FlyingJester/TurboSphere/blob/master/bin/system/scripts/turbo/format.js) can create objects from a RawFile and these schemes.

As an example:

Code: [Select]


Turbo.FontScheme = Turbo.LoadSystemScheme("font.json");

var font_header = Turbo.ReadBinaryObject(stream, Turbo.FontScheme.header);

// Now font_header has all the properties from the font.json's `header' array, loaded from the ByteArray reader `stream'.

if(font_header.signature != Turbo.FontScheme.signature)
    throw "Bad signature. Should be " + Turbo.FontScheme.signature + " instead of " + font_header.signature;

var characters = new Array(font_header.num_characters);
//...



It's mostly just to automate loading from a specific binary layout into a JS object with specified property names. If you aren't loading the files in script, or don't have a good way to load JSON in native, it won't really help much.
Title: Re: TurboSphere
Post by: Fat Cerberus on February 12, 2015, 05:24:08 pm
Honestly I'm not actually sure how slow Duktape is.  In debug mode it's abysmal, taking in excess of 1-2 seconds just to get through all the RequireScript calls in Specs, but in release mode the delay completely disappears.  I can't fully test until I can get minisphere running the Specs battle engine, then I might consider moving asset loading into script, as in TurboSphere.  For now though, I just want to try to replicate the Sphere 1.5 API natively, look at the results, and then decide where to go from there.
Title: Re: TurboSphere
Post by: N E O on February 13, 2015, 05:09:52 pm
@FJ - is that Turbo.LoadSystemScheme essentially a LoadJSONFile function?
Title: Re: TurboSphere
Post by: Flying Jester on February 13, 2015, 05:38:10 pm
@NEO: Yes, specifically it loads a JSON file from the system/formats folder. I included a polyfill for JSON with the Turbo runtime too, just in case.
Title: Re: TurboSphere
Post by: Flying Jester on February 16, 2015, 01:36:10 am
Sound now works again!
I've written a new, very simple audio plugin that uses libsndfile and OpenAL. That means no more of the weird licensing issues that seem to be endemic of every single all-in-one solution for sound (except for Audiere--no wonder people still try to use it!). It also is much, MUCH more cross-platform-capable than the old BASS plugin was.

I need to flesh out the rest of the sound system, but having used OpenAL a bit, it's really quite a nice API. A little old-fashioned perhaps, but extremely predictable and simple to use. It also is designed for 3D sound, so it will be relatively simple to add that later. I also need to add audio streaming, we only support fully buffering the files just yet.

WindowStyles fully work as well. Just as with Fonts, though, the old functions from Sphere 1.5 are gone, and now they can only be pre-rendered to surfaces. But they fully load and render.

So, this won't really be of much interest to anyone but me, but TurboSphere fully compiles using Solaris Studio on UltraSparc and Clang's standard library. I get a really nasty segfault on startup, likely because of the incompatibility of Clang's library and Solaris Studio, but a new release of Solaris Studio is supposedly coming out soon that includes a C++11-capable standard library (the compiler is already C++11 capable). In any case, this means I am very close to getting a Solaris 10 UltraSparc version compiled.

Which no one really has any reason to care about but me...I guess once it works it would be proof that TS is big-endian capable and works on generic Unices.

I also have the issue of Mesa3D not initializing correctly on my Solaris machines. But I think that a OpenGL 1.2/OpenGL ES 2 backend for Sapphire would be beneficial, both here and for Android and iOS. Yet another thing to add to the wishlist.
Title: Re: TurboSphere
Post by: Flying Jester on February 19, 2015, 06:31:11 pm
I've added basic networking support using the libfjnet library I wrote for Kashyyyk. Everything but listening is there.

I also fleshed out the Sound api a bit more.

Edit:
ListenOnPort works. So now 100% of the network API works, although using ArrayBuffers rather than ByteArrays. I included a script in the Turbo runtime (system/scripts/turbo) to make ByteArray-compatible objects from ArrayBuffers. But there's really no reason to use ByteArrays in an engine that has ArrayBuffers and TypedArrays.
Title: Re: TurboSphere
Post by: Flying Jester on February 23, 2015, 02:41:35 am
I hit a couple issues with listening on ports. I'm not quite sure where the actual bug is, maybe in the Network plugin, maybe in the plugin tools, perhaps even in SpiderMonkey.

A fairly annoying part is that Sphere's API is very close, but not the same as the BSD socket API. ListenOnPort makes sense--but for some reason, you don't `accept' a new connected socket out of that listening socket, you just use that same socket as the connection. It makes the script-side nicer, but only if you only need a single connection to the server side.

But I have been working on adding a Poll() API (sometimes similar APIs are known as Select, as in lua). Basically, just wait for an event from a group of sockets. This is already a part of libfjnet, and in reality is uses Select on Windows or ancient Unix (theoretically), Poll on Linux or more modern Unix such as Solaris, or KQueue on FreeBSD or BSD derived systems such as OS X.

On a side note, the KQueue/KEvent API is just miles ahead of Linux's Poll, even the more advanced Epoll system. Kernel APIs don't need to be abstracted for no good reason. KEvent and KQueue are very nice to work with. And Select is just kind of this sad little half-baked idea compared to either of them--I guess it made sense back in the 1990s. But I suppose this all makes sense, almost every one stole their TCP/IP stacks from BSD originally, so BSD has the most mature implementation.

I've worked out having a single Shape belong to multiple Groups. This was always supposed to be possible, but there were a few issues when using more than a single Shader.
Title: Re: TurboSphere
Post by: Flying Jester on March 06, 2015, 02:27:40 am
I've updated to the newest version of SpiderMonkey. That's the first upgrade, and to be fair, the API changed.

But it was small and simple changes.

I had to replace this
Code: [Select]
 JS::RootedObject obj(ctx, JS_NewObjectWithGivenProto(ctx, &clazz, proto, global)); 

with this
Code: [Select]
 JS::RootedObject obj(ctx, JS_NewObject(ctx, &clazz)); 
JS_SetPrototype(ctx, obj, proto);


SM's api, from a pure ABI standpoint, may change fairly constantly, but conceptually it certainly looks more stable than V8's.

***

I also have been working on WindowStyles. As with many of the old APIs, it's difficult to properly compare the performance of TurboSphere with Sphere 1.5, SphereSFML, or MiniSphere directly. The new API enforces caching by enforcing persistence of objects and state. In the Turbo runtime, drawing a WindowStyle no longer looks like this

Code: [Select]
 // In the initialization
var window_style = LoadWindowStyle("windowstyle.rws");
// In the drawing loop...
window_style.drawWindow(16, 16, 128, 128);


Instead, it looks like this

Code: [Select]
 // In the initialization
var window_style = new WindowStyle("windowstyle.rws");
var window = window_style.createWindow(16, 16, 128, 128);
// In the drawing loop...
window.draw();


The WindowStyle.createWindow() function generates a Group that contains the specified window, all cached into a new Surface and uploaded as an Image for convenience.
This is equivalent to drawing a WindowStyle to an Image and then only drawing that.
Title: Re: TurboSphere
Post by: Fat Cerberus on March 06, 2015, 02:41:19 am
Don't worry, you'll probably be able to compare them soon enough. ;)  minisphere is damn close at this point to being a drop-in replacement for Sphere 1.5--in fact, my compatibility at this point is higher even than SphereSFML due to using the original Sphere 1.x engine source as reference.  I might have mentioned this before, but once I'm happy with 1.x compatibility I'm going to branch the minisphere repo and work on a more Pegasus-y version of the engine.

I'm assuming the TS API is meant to be Pegasus also?

On a side note, I've actually been trying to get Aquatis to run in minisphere as the final compatibility test but with all the German text in the scripts, Duktape doesn't want to parse them.  I keep getting "char decode failure", preventing script compilation.  I'm assuming it's trying to interpret the accented characters as UTF-8 and failing...
Title: Re: TurboSphere
Post by: Flying Jester on March 06, 2015, 03:00:10 am

Don't worry, you'll probably be able to compare them soon enough. ;)  minisphere is damn close at this point to being a drop-in replacement for Sphere 1.5--in fact, my compatibility at this point is higher even than SphereSFML due to using the original Sphere 1.x engine source as reference.  I might have mentioned this before, but once I'm happy with 1.x compatibility I'm going to branch the minisphere repo and work on a more Pegasus-y version of the engine.


That would be cool! The new API naively looks like a lot of work to use, but a lot of it just makes it more work to write things that are less efficient. Case in point, it becomes super annoying to redraw an entire WindowStyle window every frame, an operation that can bring several engines to their knees. At the same time, making the state persistent and caching the result become simple.


I'm assuming the TS API is meant to be Pegasus also?


This part is a part of the purely JS Turbo runtime. Ideally, any Pegasus-compliant engine would be able to use the Turbo runtime without any issue.
Some parts of the runtime are also things that should properly be a part of Pegasus, and that I have implemented in script. A Pegasus compliant engine could still use it, it would just overwrite those exposed components with its own.

If this is to become a part of Pegasus, it isn't in the spec yet :)


On a side note, I've actually been trying to get Aquatis to run in minisphere as the final compatibility test but with all the German text in the scripts, Duktape doesn't want to parse them.  I keep getting "char decode failure", preventing script compilation.  I'm assuming it's trying to interpret the accented characters as UTF-8 and failing...


That assumes Aquatis is really UTF-8, or UTF at all. Many european languages used to use codepages (oh, the glory days!), and from what I've seen I believe that the Sphere editor uses them--as an example, Sphere fonts are kind of wonky if you expect them to really be UCS-indexed. Using ascii for an extended codepage (or any codepage, really) will almost always make invalid UTF strings (unless you are very lucky: "Bush hid the facts" type scenarios)
Title: Re: TurboSphere
Post by: Fat Cerberus on March 06, 2015, 03:08:41 am
That's what I meant, the Aquatis scripts aren't actually UTF-8 but Duktape is trying to parse the files as though they are and getting hung up on the extended characters.  Not sure if that's a bug or not, maybe I should report it to the Duktape devs.
Title: Re: TurboSphere
Post by: Flying Jester on March 06, 2015, 03:23:51 am
I would consider it a bug of the old editor.

As much fun as I think codepages are (I really do enjoy working with them), I think that it would be more worth everyone's time to convert old CP strings to UTF-8 strings in scripts and resources.

Actually, a UTF converter for the resource formats may be a good idea.
Title: Re: TurboSphere
Post by: Radnen on March 06, 2015, 06:00:13 pm

in fact, my compatibility at this point is higher even than SphereSFML due to using the original Sphere 1.x engine source as reference.


Hey, I use the source as reference too! Especially for people and the command queue. I might not have as many data types loading just yet since I only focused on the most recent versions of each filetype, and windowstyles aren't even anywhere near full support. But still, I think with what I got there it's pretty dang close.
Title: Re: TurboSphere
Post by: Fat Cerberus on March 06, 2015, 06:10:01 pm
That's interesting, the command queue is one of the few persons-related things I didn't need a reference for.

Where I do tend to refer to it is to hammer out little subtleties, like how edge scripts work.  See, instinct would be to implement those working identically to triggers, right?  But no, it turns out Sphere will keep calling the edge script continuously as long as the input person is past the edge.  Non-intuitive, but for full compatibility the behavior has to be replicated.
Title: Re: TurboSphere
Post by: N E O on March 07, 2015, 03:35:51 pm

Instead, it looks like this

Code: (javascript) [Select]
 // In the initialization
var window_style = new WindowStyle("windowstyle.rws");
var window = window_style.createWindow(16, 16, 128, 128);
// In the drawing loop...
window.draw();


The WindowStyle.createWindow() function generates a Group that contains the specified window, all cached into a new Surface and uploaded as an Image for convenience.
This is equivalent to drawing a WindowStyle to an Image and then only drawing that.



If the API isn't yet finalized, can you also add window_style.createWindow(width, height) and window.draw(x, y) ?
Title: Re: TurboSphere
Post by: Flying Jester on March 07, 2015, 08:04:15 pm
You can always change a group's X and Y by directly setting it's `x' and `y' values:

Code: [Select]

// In the initialization
var window_style = new WindowStyle("windowstyle.rws");
var window = window_style.createWindow(16, 16, 128, 128);
// In the drawing loop...
// Move it somewhere...
window.x = GetMouseX();
window.y = GetMouseY();
// Now it draws at the mouse's coordinates
window.draw();


One major drawback of Sapphire's current implementation is that changing the properties repeatedly in between multiple draws within a single frame can cause certain combinations to not draw correctly.
That's because, while drawing is pipelined, state changes such as the texture, coordinates, and rotations, are executed immediately. I can change this, but it results in a fairly decent increase in latency.

I decided that it would be better to just have multiple Groups, one for each distinct 'thing' to draw each frame, since Groups are rather lightweight compared to Shapes or Images. Plus, that kind of practice makes the object model more logical. Each Group represents some group of Shapes that is a part of your current scene.
Title: Re: TurboSphere
Post by: Flying Jester on March 12, 2015, 04:16:10 am
I finally fixed map obstructions.

The issue is that I was reading the obstruction data after reading all headers. So, like this:

Code: [Select]

for(var i in tiles){
    tiles[i].loadHeader(stream);
}

for(var i in tiles){
    tiles[i].loadObstructions(stream);
}


That's totally not right. I had misread the documentation (or just interpreted it differently). In reality, it's like this:

Code: [Select]


for(var i in tiles){
    tiles[i].loadHeader(stream);
    tiles[i].loadObstructions(stream);
}


This was made MUCH more difficult to spot because the three test maps I usually use--Radnen's 'old school' map, the 'Astral' map from Trial and Error, and an early battlefield map from Attack!--all  exhibit almost correct behaviour when I read the obstructions this way. I had long assumed I was not performing segment intersections correctly.
On the bright side, my line intersection code is rather nice now :)

Tile animations are next. I have a few different ideas on how to handle these.
Title: Re: TurboSphere
Post by: Fat Cerberus on March 12, 2015, 11:33:43 am
How did you finally figure it out?  Did you look at minisphere's source or...?
Title: Re: TurboSphere
Post by: Flying Jester on March 12, 2015, 01:14:13 pm
I took a break and worked on other projects. When I came back, I added some debug code to draw the obstructions on the tiles themselves. When I did, I accidentally changed the order to be correct.
Title: Re: TurboSphere
Post by: Fat Cerberus on March 12, 2015, 01:20:31 pm
The only thing I can't figure out is how it went unnoticed for so long.  The way you were doing it, wouldn't you just end up getting back garbage tile headers after reading the first tile with any obstruction data?
Title: Re: TurboSphere
Post by: Flying Jester on March 12, 2015, 02:04:19 pm
I haven't yet implemented much that would use the headers. I was trying to just do obstructions first, and my test maps just happened to make almost-valid looking obstructions using their header data that way. That was why it took so long to figure out--it looked like I had obstruction data, but I was misinterpreting it or having issues with collision detection.
Title: Re: TurboSphere
Post by: Radnen on March 12, 2015, 08:02:43 pm

I took a break and worked on other projects. When I came back, I added some debug code to draw the obstructions on the tiles themselves. When I did, I accidentally changed the order to be correct.


I did something similar and found out much the same way! Haha.
Title: Re: TurboSphere
Post by: N E O on March 15, 2015, 12:44:20 am
If I had ever finished the web version of map parsing, I probably would've come across the same problem; thanks for confirming the correct behavior!
Title: Re: TurboSphere
Post by: Fat Cerberus on March 15, 2015, 03:42:53 am
The funny thing about this is I got it right the first time--I don't even recall finding the RTS doc to be ambiguous about it.  I guess it just seemed natural to me that the order would go header-obstructions-header-obstructions (especially since the header includes a "has obstructions" flag) that the uncollated order never occurred to me.
Title: Re: TurboSphere
Post by: Radnen on March 15, 2015, 04:31:24 am

The funny thing about this is I got it right the first time--I don't even recall finding the RTS doc to be ambiguous about it.  I guess it just seemed natural to me that the order would go header-obstructions-header-obstructions (especially since the header includes a "has obstructions" flag) that the uncollated order never occurred to me.


Well, other files code aren't like that, such as spritesets or windowstyles, but tiles are different. That said I found out about this very early on, after looking at some weird values returning back. I guess I didn't use maps quite like FJ's. :P
Title: Re: TurboSphere
Post by: Flying Jester on March 15, 2015, 04:47:47 am
I guess I didn't use maps quite like FJ's. :P


This. Maps other than the ones I test with would have made this easy to see.
Title: Re: TurboSphere
Post by: Flying Jester on March 31, 2015, 12:44:14 am
I've been using some features I've added but not extensively tested, particularly the Turbo Runtime font system. I've begun to examine how I could implement a streaming Surface system.

Right now I am attempting to simply add a blit method to Surfaces that has as little overhead as possible. The default would simply be to upload the surface every single time it will be drawn, or every time the surface is modified.
However, OpenGL has some extensions and some newer core features to make this more efficient. Some of them are Mapped Buffers and Pixel Buffers. I'm going to test out the performance and complexity of these solutions.

Ideally, I don't want to have a Surface.blit() method. I want to make a simple way to bind a Surface to an Image such that changes to the Surface are reflected in the Image automatically. I don't exactly know what I want this to look like just yet, although simply having a function or new constructor for an Image that creates an Image with a bound Surface is certainly an appealing option right now.

If this performs well enough, I may change the Turbo Runtime map engine to use this solution for layers. It would certainly make animations a bit simpler.
Title: Re: TurboSphere
Post by: Fat Cerberus on March 31, 2015, 01:16:13 am
I don't know how closely you looked at the minisphere source, but in minisphere surfaces and images are the same object internally.  In fact, if I didn't have to stay compatible with Sphere 1.x  I wouldn't even have both--everything would be an image.  Seems like it would be much cleaner (and perhaps easier to optimize) without the distinction.
Title: Re: TurboSphere
Post by: Flying Jester on March 31, 2015, 01:59:58 am
TurboSphere used to be that way, too.

Using hardware acceleration for everything, it makes sense to have both software representations of an image and a hardware representation. They have vastly different performance characteristics.

It is extremely taxing to update an image every single frame. I want to use this to limit the overall number of different Images, primarily due to GC concerns. In the case of just a simple Image, it makes no sense to keep the the software representation. If you are going to modify an image many times before use, it makes sense to keep it all in software and only touch the hardware when you actually need to draw it.

I could put that last part in the engine proper, for instance only upload the Surface once it needs to be blit, but that difference already exists in the Sphere API. That separation is the difference between Surface and Image, which is even stronger in TurboSphere since you simply cannot blit a Surface. The new idea I have would more over be a more optimized way to update an existing Image. It would still be better to just use Images and Surface wherever possible, but in a very few cases I am seeing if it would be more efficient to do it this way.

An alternative is to do what SDL does, and what I suspect Allegro mostly does, which is to draw everything on a single texture and just upload that one texture every frame. But I do not want to do that, since I want to take advantage of the GPU's geometry calculations. Even on simple stuff that equates to a memcpy(), I found an entire order of magnitude improvement when I started doing this in TurboSphere.

Another solution to the problem I am tackling right now would be to add an 'update' method to Image. But if I know that the Image will be updated often, I want to see if using PBO's or mapped buffers would be a better solution.
Title: Re: TurboSphere
Post by: Fat Cerberus on March 31, 2015, 02:09:45 am
As far as I know all hardware bitmaps in Allegro are implemented as FBOs, which means drawing to a surface in minisphere is theoretically just as fast as drawing to the backbuffer.  As such it didn't really make sense to have separate types internally--there's no performance hit, and it helps keep the size of the engine down.  KISS. :)

No offense, but at times at times I feel like the TurboSphere project suffers from a very bad case of overengineering.  Maybe it's just that I enjoy refactoring, but I'd rather implement something basic that works first, and if it proves to be inadequate down the road I can always change it.  That's what source control is for!
Title: Re: TurboSphere
Post by: Flying Jester on March 31, 2015, 02:22:45 am
That doesn't really make sense to me. An FBO is used to take the result of a drawing operation and put it into a texture rather than to a context. Do you mean PBO?

Even using buffer objects, it is slow to rebind them and all cards have a very limited number in comparison to the number of possible textures.

If you want to reach out and touch hardware, the separation of Image and Surface makes sense. It's true that it could all be wrapped up in the engine, but I believe that developers are quite capable of understanding and using Surfaces and Images responsibly, since the difference is simple--draw Surfaces rarely, modify Images rarely. In some engines like TurboSphere the difference is an important distinctions. In other engines it might not be, but it is then a relatively harmless addition.

EDIT:
I'm curious why you say that TurboSphere is overengineered. I certainly agree that it has a lot of NIH in it, and that the new drawing system is somewhat complex.
Title: Re: TurboSphere
Post by: Fat Cerberus on March 31, 2015, 02:25:59 am
https://www.allegro.cc/manual/5/al_set_target_bitmap

The Allegro docs mention FBOs.  No PBOs to speak of.
Title: Re: TurboSphere
Post by: Flying Jester on March 31, 2015, 02:31:25 am
Huh. If you want to modify what an FBO contains--for instance, to tell what the value a particular pixel is--you would still need to download it from the video card. If you modified a certain pixel, you could surely do that by a drawing operation, but as far as direct access and especially reading results into software, you face the same issues you would just using GL textures.
Title: Re: TurboSphere
Post by: Fat Cerberus on March 31, 2015, 02:31:44 am

I'm curious why you say that TurboSphere is overengineered. I certainly agree that it has a lot of NIH in it, and that the new drawing system is somewhat complex.


I don't know, I guess because I'm of the KISS/YAGNI school of development, seeing you overthink all the new additions and agonize over how best to implement them makes me roll my eyes.  Just call it difference of opinion I guess.  I like to see results quickly and let the program develop mostly organically; I'm not afraid of a little refactoring if things start to get too hairy.
Title: Re: TurboSphere
Post by: Fat Cerberus on March 31, 2015, 02:34:50 am

Huh. If you want to modify what an FBO contains--for instance, to tell what the value a particular pixel is--you would still need to download it from the video card. If you modified a certain pixel, you could surely do that by a drawing operation, but as far as direct access and especially reading results into software, you face the same issues you would just using GL textures.


That's the beauty of it though--the way Allegro is set up you very rarely have to cross the hard/software barrier.  The only time I ever lock a bitmap is to load an image from disk, otherwise I let Allegro do it all in hardware.  It's awesome that way.
Title: Re: TurboSphere
Post by: Flying Jester on March 31, 2015, 02:47:07 am


Huh. If you want to modify what an FBO contains--for instance, to tell what the value a particular pixel is--you would still need to download it from the video card. If you modified a certain pixel, you could surely do that by a drawing operation, but as far as direct access and especially reading results into software, you face the same issues you would just using GL textures.


That's the beauty of it though--the way Allegro is set up you very rarely have to cross the hard/software barrier.  The only time I ever lock a bitmap is to load an image from disk, otherwise I let Allegro do it all in hardware.  It's awesome that way.


When you remove blit for Surface's, this is mostly how it works anyway. A Surface is used only sparingly in TurboSphere, mostly when you want to touch pixels very specifically and see the results directly. If you just load an Image directly, it works that way entirely.

The reason I want to do anything in software at that point, except for retrieving data, is that some operations don't lend themselves well to hardware acceleration. I found this when doing gradient munching squares, many bezier curves, and some procedurally generated effects. I had much better performance simple drawing directly in software.

I like this difference being exposed to script.



I'm curious why you say that TurboSphere is overengineered. I certainly agree that it has a lot of NIH in it, and that the new drawing system is somewhat complex.


I don't know, I guess because I'm of the KISS/YAGNI school of development, seeing you overthink all the new additions and agonize over how best to implement them makes me roll my eyes.  Just call it difference of opinion I guess.  I like to see results quickly and let the program develop mostly organically; I'm not afraid of a little refactoring if things start to get too hairy.


Fair enough :)

I like to test different implementations and try different solutions for every problem. Most of it probably is not really needed, and any of the possible solutions are suitable. But I like to have my own personal results in hand, as well as a few different attempts with different methods before I properly commit to something. And I like to talk through what I'm doing and share my findings along the way. Sometimes being able to look what I wrote and know that others will see it helps me see it differently myself.

I do take any additions to the native API very seriously, though. I am much happier with the generality and minimalism of TurboSphere's native API than Sphere's API. Since I implemented the Galileo API, almost all of my additions since have been to solve a particular problem I had trying to write demos.
Title: Re: TurboSphere
Post by: Fat Cerberus on March 31, 2015, 03:01:11 am
Kind of off topic, but perhaps relevant to the "talking things through" point - Have you looked at my commit history at all?  I like to tell little mini-stories with my commit descriptions (as opposed to just a commit title as most people do), making it quite interesting to go back and read through them as it basically tells the story of the project's development.  In fact that's my metric for when to commit--if it's not a substantial enough edit to warrant more than a title, I hold off on committing.

Basically I can have a dialogue with my future self through my commit messages.  It's kind of neat.
Title: Re: TurboSphere
Post by: Fat Cerberus on March 31, 2015, 03:26:34 am
Getting the train back on track after that digression, I highly recommend you implement GetExtensions(), as minisphere does.  It'd be useful to have this API standardized, and two engines implementing the same function goes a long way towards that.  minisphere 1.0.x returns, for example, the following array of strings: [ "sphere-legacy", "minisphere", "frameskip-api" ]

Ideally this should only return native engine functionality.  So TurboSphere could probably return [ "sphere-legacy", "pegasus" ], but if the legacy functions were instead provided externally by a script, "sphere-legacy" would be left out.
Title: Re: TurboSphere
Post by: Flying Jester on March 31, 2015, 07:42:53 am
Oh, I recall the GetExtensions function :) The two main reasons I haven't had the urge to implement it is that I would need some way to share the array of strings between plugins, and also to the Turbo system scripts.
Title: Re: TurboSphere
Post by: Fat Cerberus on March 31, 2015, 09:13:08 am
I don't think the Turbo scripts should be able to affect it, honestly.  Plugins yes, but the idea is that a script calls GetExtensions to determine what the engine supports in native so it can adapt its functionality accordingly.  The Turbo runtime itself, for instance, could call it and look for the "pegasus" string and if it isn't found, fail to initialize.

The only reason I would say Turbo runtime should be able to add an extension string is if the engine is automatically eval-ing the system scripts on behalf of the game, but if so that's frankly a bit tacky and I would consider cutting that out.
Title: Re: TurboSphere
Post by: Flying Jester on April 02, 2015, 07:21:11 pm
I've been considering having TS compile some of the Turbo scripts if a game does not identify version, or identifies needing version 1. This way, when the game runs it gets as close to having a Sphere 1.5-like environment as its going to get with TS out of the box.

I went ahead and made a second kind of socket for TS. So, to accept a connection, you now do this:

Code: (javascript) [Select]

  var listener = new ListeningSocket(8192); //A constructor, otherwise a synonym for ListenOnPort
  while(true){
    var new_socket = listener.accept(); // Will return null if no incoming connections
    if(new_socket){
      // Wait until we actually have something to read.
      while(!new_socket.getPendingReadSize()) Delay(10);

      var buffer = new_socket.read(new_socket.getPendingReadSize());

      // Write out whatever we receive, just to prove it actually happened.
      var logfile = new RawFile("out.log", true);
      logfile.write(buffer);
      logfile.close();
      new_socket.close();
     
      break;
    }
    Delay(10);
  }


Then, if you were going to connect, it would be much like in Sphere 1.5:

Code: (javascript) [Select]

  var socket = new Socket("127.0.0.1", 8192);

  if(!socket.isConnected()) throw "Are you sure the server is running?";

  // The `.buffer' is because ByteArrays are actually UInt8Array's under the hood. This won't be necessary at some point in the future.
  socket.write(CreateByteArrayFromString("Hello, server!\n").buffer);
 
  while(socket.isConnected()) Delay(10); // The server function should drop us pretty quickly.

  Abort("Success!");



This makes the implementation much simpler on the back end compared to how Sphere 1.5 does it. Given the small number of networked games for Sphere, and how simple it is to change them to use this method if desired, I don't really see too much of a need to provide Sphere 1.5 compatibility here. Maybe if I get really ambitions I will add a ListenOnPort function in script to do it.

I have not pushed the Poll/Select/KQueue function out yet because it currently doesn't properly wake if one of the sockets is a listening socket that has an incoming connection on Windows or on OS X. So basically I need to make it actually work with Select and KQueue :)

I have included a simple web server in the default games to demonstrate all this with TurboSphere. I need to fix an issue with rawfile sizes before that works perfectly, but it demonstrates networking is working properly. Once Poll() works properly, then TS will really be ready for networked games!
Title: Re: TurboSphere
Post by: Fat Cerberus on April 02, 2015, 08:07:32 pm
Why don't you just implement the socket API the way minisphere has it?  This way it's fully backwards compatible and you still get the accept functionality.
Title: Re: TurboSphere
Post by: Flying Jester on April 02, 2015, 09:13:34 pm
How does Minisphere do it?

I did it this way because this is the simplest way for me to implement the native functionality. It wouldn't be too hard to write a ListenOnPort in script that emulates Sphere 1.5.

The reason I want to actually have sockets and listening sockets different is that it gets complicated when someone listens again on the same port. It's kind of wasteful to close the socket and then open up a new one right away. Particularly when you want to sync a list of sockets between script and native for a Poll/Select function.

In native, it's true that technically when you are listening on a socket it's a fully fledged socket. But it's nonsensical to actually use it like a socket--writing to it or reading from it doesn't make sense to do. They are less similar than a FD and socket. I think it's not a bad thing for that dissimilarity to be present, in some form, in script.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 02, 2015, 09:34:56 pm
Signature for ListenOnPort in minisphere:

Code: [Select]
ListenOnPort(port[, backlog])


If backlog is not specified, the socket behaves as in Sphere 1.5.  Otherwise, it persists as a listener and you call .acceptNext() on it to accept connections.  If you try to read or write to it directly, the engine will throw an exception.
Title: Re: TurboSphere
Post by: Flying Jester on April 02, 2015, 09:36:54 pm

If backlog is not specified, the socket behaves as in Sphere 1.5.


This is the complex part, and the part I am not going to put into the engine proper. It will be fine to have this kind of thing be only available from ListenOnPort. Any TurboSphere-aware program will call the constructor instead of ListenOnPort, and any Sphere 1.5 game won't use actual constructors.

I certainly could add backlog support, but I don't especially think it's a useful value to adjust from script. Setting it to 16 should really be enough for any game.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 02, 2015, 09:44:06 pm
Oh, of course. :). I didn't expect you to build that into the engine proper, I was just giving you a tip for how to implement ListenOnPort() when you get around to that, to maintain backwards compatibility.
Title: Re: TurboSphere
Post by: Flying Jester on April 02, 2015, 09:54:51 pm
Yeah, I was mostly just working out the native TS API right now.

Oh yeah, and I forgot April Fools! Insert some skit about TurboSphere being Solaris UltraSparc only, or switching to be written PHP...I was busy and couldn't flesh out my ideas for it in time!

I solemnly swear to have a joke ready in time for next year ;D
Title: Re: TurboSphere
Post by: Fat Cerberus on April 02, 2015, 10:02:32 pm
Honestly I almost forgot about it myself.  The only reason I remembered was because I ended up posting 1.0.3 that day anyway and when I went to update the release date I said "oh yeah, it's April fools!" :D
Title: Re: TurboSphere
Post by: Fat Cerberus on April 03, 2015, 01:01:38 am

I certainly could add backlog support, but I don't especially think it's a useful value to adjust from script. Setting it to 16 should really be enough for any game.


Yeah, I don't expect most games to really care about the backlog size much, but keep in mind my other options:
1) Create a whole new set of APIs to create "new" BSD-style sockets, e.g. CreateSocket(), Socket:bind(), etc.
2) Make the optional second parameter to ListenOnPort a boolean, true for BSD style, false for Sphere 1.5 style

My KISS philosophy and goal of keeping minisphere small, both in executable size and codebase precludes the first option, but since I'm adding a new parameter to a function's signature anyway, it makes sense to try to get as much value out of the API change as possible.  The second option would be rather limiting in that regard.  So I arrived at my third option: The optional parameter specifies the size of the backlog.  Even if most games never need to set it to anything other than 16, the option is there.  I'm more forward-thinking than I look. :)
Title: Re: TurboSphere
Post by: Flying Jester on April 06, 2015, 01:40:59 am
So, I've update the Galileo (https://github.com/sphere-group/pegasus/blob/master/api/ts/galileo.js) specification in Pegasus to correlate to how it works in TurboSphere. Based on actual performance characteristics, many of the qualities of Shapes now exist in Groups, and a stubbed up Shader system exists. Despite the word 'Shader', the spec is shading-language agnostic, and doesn't even require a programmable pipeline. In fact, TurboSphere doesn't have one yet, since I haven't needed it yet.

None of this is new in TurboSphere, but the Pegasus docs didn't reflect this until now.
Feedback is welcome! I haven't really had a need for too much more than what's in it now, even when writing the map engine.

On a side note about Pegasus, I'm not too savvy on its whole dependency system. It's a nice idea, but I believe it is a complex solution to a simple problem.

I've also rewritten the FileSystem plugin to be less horrible. It worked well enough, it just looked awful. It's now the same format as Cinnamon and the other simpler plugins are, with a [Plugin].cpp/hpp holding the plugin API code and script.cpp/hpp holding the bindings. I forget why I made the old scriptfs plugin a different way. I must have been under some form of possession or something when I thought up how its source was laid out.

I also got a new laptop. Well, new to me. I began seeing about making a new Windows release. And I remember now why developing in Windows is not my favorite thing to do :P
Title: Re: TurboSphere
Post by: Flying Jester on April 09, 2015, 03:11:52 pm
I may not have mentioned this when I first implemented it, but I did recently fix it again and I thought it might be worth mentioning.

You can pass in actual functions to `SetUpdateScript' and `SetRenderScript', and just about any script-setting function. I think it's pretty ugly to pass in strings to evaluate to those functions in the first place, and I would assume the reason strings were used was that it was easier/the only way in the ancient SpiderMonkey of pre-2000 that the API was originally developed for (especially given how ridiculous the code to call `game()' is in Sphere).

This is also important to note because the scoping in TurboSphere's SetUpdateScript/SetRenderScript is slightly different than it is in Sphere 1.5. It's now the same as if passing in `new Function(str)' was used, rather than the scoping rules of eval. To get the old scoping rules you can just as easily use an anonymous function.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 09, 2015, 03:14:40 pm
I do remember you bringing this up a while back, and I think Radnen implemented the same in SSFML.  I should probably allow this in minisphere as well.  I think that would be the first change that might warrant a bump to v1.1. :P
Title: Re: TurboSphere
Post by: Radnen on April 09, 2015, 06:31:52 pm

I do remember you bringing this up a while back, and I think Radnen implemented the same in SSFML.  I should probably allow this in minisphere as well.  I think that would be the first change that might warrant a bump to v1.1. :P


Yep. In fact passing a function was the only fastest method, until I added the ability to store pre-compiled scripts in Jurassic... then you could do an either/or kind of situation. Though I wonder if the JIT compiler actually optimizes functions passed in better than compiling from essentially a new source utilizing the same context.

I need to benchmark it, but I wonder if:
Code: (javascript) [Select]

//Is this faster
SetUpdateScript(func);

// than this?
SetUpdateScript("func()");
Title: Re: TurboSphere
Post by: Fat Cerberus on April 09, 2015, 06:35:53 pm
In minisphere I can guarantee the former would be faster.  The engine tends to stutter in the lithonite demo because lithonite abuses delay scripts and Duktape takes a bit to compile, even if it's only a single line of code.  Don't know about Jurassic though--the JIT may even things out.
Title: Re: TurboSphere
Post by: Flying Jester on April 09, 2015, 08:12:05 pm


I do remember you bringing this up a while back, and I think Radnen implemented the same in SSFML.  I should probably allow this in minisphere as well.  I think that would be the first change that might warrant a bump to v1.1. :P


Yep. In fact passing a function was the only fastest method, until I added the ability to store pre-compiled scripts in Jurassic... then you could do an either/or kind of situation. Though I wonder if the JIT compiler actually optimizes functions passed in better than compiling from essentially a new source utilizing the same context.

I need to benchmark it, but I wonder if:
Code: (javascript) [Select]

//Is this faster
SetUpdateScript(func);

// than this?
SetUpdateScript("func()");



It's really close-as-makes-no-difference in SM. In the second case, the parser can generate the bytecode so fast it doesn't really matter, and such a simple call still mostly relies on the functions it calls to be optimized, which would be necessary in the first case anyway.

You could theoretically get a tiny boost the second way since the parser in SM doesn't kick in (and the low optimizer in V8, a little slower in this instance), but small parses like that are lightning fast in both SM and V8 since they are very common on web pages. I would be surprised if, in any non-contrived usage, performance was notably different in SM, V8, or JSC or any other web-oriented JS Engine. I believe that V8 even caches small string literals that are compiled, and attempts to reuse the parse info when doing things like this.

I mainly advocate the first case because it is so much nicer, and encourages better practices.

It's good that it's also faster in non-web-oriented engines!
Title: Re: TurboSphere
Post by: Flying Jester on April 23, 2015, 05:55:22 pm
I've almost got the OpenGL 2.1 renderer working in Sapphire. This will let it work on almost all machines I own, most importantly my Windows laptop and my Solaris machines that have no Mesa drivers (which is usually slower, but bah). I've only merged in the changes that affect the implementation of the default OpenGL 3.3/4.1 renderer, though.

Now I need to read up on a couple ancient OpenGL 2.1 functions...I've basically forgotten all that I knew about it! I also need to remember how exactly GLSL 1.1 worked ;_; But it'll be worth it.

EDIT:
So I got the OpenGL 2.1 renderer fully working, sans shaders. It's seriously vertex-limited compared to the 4.4 renderer. It can manage a full 60 FPS on a map, but it begins to spend most of its time pushing vertices.
In general, the with the 4.4 renderer TurboSphere uses an almost constant 7% CPU to draw a map at 60 FPS. With the 2.1 renderer, we need 20% to draw the map since it has so many vertices. It's still usable, and allows Sapphire to run on older hardware. And it still uses the same asynchronous renderer that the 4.4 renderer uses, so even when I started pushing way too many vertices to the renderer to blow down the FPS (4 64x64 maps at once saturates the CPU, 8 brings my new MacBook down to 20 FPS), the FPS is still smooth, although low.

I would suspect that the 2.1 renderer actually makes better use of the asynchronous rendering system, since it's so much more CPU-intensive. The game can run regardless of the hundreds, sometimes thousands of GL calls that 2.1 generates per frame.
Title: Re: TurboSphere
Post by: DaVince on April 24, 2015, 02:50:48 am
So it supports both GL versions now? Kickass job, FJ. :)
Title: Re: TurboSphere
Post by: Fat Cerberus on April 29, 2015, 12:32:01 am
So I was perusing the TurboSphere source, and I have to say, you make pretty creative use of preprocessor macros.

Code: (cpp) [Select]
    #define ENSURE_PROP(NAME, VALUATOR, WRAPPER)\
    if(has_##NAME){\
        JS::RootedValue out_value(ctx);\
        JS_GetProperty(ctx, that, #NAME, &out_value);\
        if(!VALUATOR){\
            return false;\
        }\
        WRAPPER\
    }
   
    ENSURE_PROP(x, true, vertex.x = out_value.toNumber();)
    ENSURE_PROP(y, true, vertex.y = out_value.toNumber();)
    ENSURE_PROP(u, out_value.isNumber(), vertex.u = out_value.toNumber();)
    ENSURE_PROP(v, out_value.isNumber(), vertex.v = out_value.toNumber();)
    ENSURE_PROP(color, out_value.isObject() && color_proto.instanceOf(ctx, out_value, nullptr),
        {TS_Color *color = color_proto.unsafeUnwrap(out_value.toObjectOrNull());
        vertex.color = TS_Color(color->red, color->green, color->blue, color->alpha);}
    )
   
    #undef ENSURE_PROP


I think somebody misses their closures. ;)
Title: Re: TurboSphere
Post by: Flying Jester on April 29, 2015, 07:02:24 pm
It's a simple way to avoid a ton of typing. It is also less error prone than copypasta. The naive alternative would be to type out:

Code: (c++) [Select]

    if(has_x){
        JS::RootedValue out_value(ctx);
        JS_GetProperty(ctx, that, "x", &out_value);
        vertex.x = out_value.toNumber();
    }
    if(has_y){
        JS::RootedValue out_value(ctx);
        JS_GetProperty(ctx, that, "y", &out_value);
        vertex.y = out_value.toNumber();
    }
    if(has_u){
        JS::RootedValue out_value(ctx);
        JS_GetProperty(ctx, that, "u", &out_value);
        if(!out_value.isNumber()){
            return false;
        }
        vertex.u = out_value.toNumber();
    }
    if(has_v){
        JS::RootedValue out_value(ctx);
        JS_GetProperty(ctx, that, "v", &out_value);
        if(!out_value.isNumber()){
            return false;
        }
        vertex.v = out_value.toNumber();
    }

    if(has_color){
        JS::RootedValue out_value(ctx);
        JS_GetProperty(ctx, that, "color", &out_value);
        if(!(out_value.isObject() && color_proto.instanceOf(ctx, out_value, nullptr))){
            return false;
        }
        TS_Color *color = color_proto.unsafeUnwrap(out_value.toObjectOrNull());
        vertex.color = TS_Color(color->red, color->green, color->blue, color->alpha);
    }


I saw this kind of thing a lot in the Sphere source, and just sort of ignored it. It's something I kind of learned to appreciate at Mozilla.
I mean, I could do something with templates, but some times that just ends up being really complex. Sometimes the most readable, simplest way to do it is just a macro. I don't think this warrants anything more complex.

Anyways...
I've updated TurboSphere to use SpiderMonkey 40. Which meant a total of about 20 lines of changes, mostly to Sapphire and how it handles Group and Shape assignments. Compared to how V8's API changes, SpiderMonkey is extremely stable.

I also modified the Delay function to perform GC'ing, if more than 10 milliseconds of delay is requested, and if we have not GC'ed in the last second.

SM 40 also uses much less memory...we went from 40 MB running the test map down to 23 MB! I'm really impressed, that's much less than I thought. It's nice to have a JS engine that can actually manage its own memory for once.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 29, 2015, 07:42:22 pm
GC'ing on Delay() is a good idea, I'll have to implement that now. :D

Agreed on the rationale behind the macros, although I will generally try to factor it into a function if possible when boilerplate starts to pile up too much.  Ive had too many instances where a syntax error in a commonly used macro causes the compiler to spew out a hundred useless errors that don't help me to find the problem at all (a missing close brace in a macro can be a nightmare).  You'd think things like macros would be mostly write-once... But I'm a refactoring fiend so those are about as stable as the rest of my code. :P
Title: Re: TurboSphere
Post by: Fat Cerberus on April 30, 2015, 12:47:58 am
So in Sapphire, is there a reason you use a triangle strip instead of a fan for Shapes with >4 vertices?  I've debated switching minisphere to always use fans (which would of course then be incompatible with TS) because with a tri-strip, the following:

Code: (javascript) [Select]
var numPoints = 8;
for (var i = 0; i < numPoints; ++i) {
var phi = 2 * Math.PI * i / numPoints;
var x = Math.cos(phi) * 32;
var y = Math.sin(phi) * 32;
vertices.push(new Vertex(x, y));
}
var shape = new Shape(vertices, image);


...produces an odd crescent shape instead of an octagon. ???
Title: Re: TurboSphere
Post by: Flying Jester on April 30, 2015, 02:04:17 am
I used strips because it's easier to make a composite shape with them, as shown in the map engine. That's much more difficult/impossible with a triangle fan. Using triangle strip, you can still emulate the triangle fan behaviour:

Code: (JavaScript) [Select]

    let numCorners = 8;
    let vertices = new Array(numCorners*2);  
    for (let i = 0, e = 0; i < numCorners*2; ++i) {
        if(!(i%2)){
            vertices[i] = {x:32, y:0};
        }
        else{
            let phi = 2 * Math.PI * (i>>1) / numCorners;
            vertices[i] = {x:Math.cos(phi) * 32, y:Math.sin(phi) * 32};
        }
    }


I tend to prefer strips because you can still emulate fans, and OpenGL 3/4 is more VAO-limited than vertex limited (so pushing more vertices is better than pushing multiple VAO's, which a Shape represents).

Perhaps a way to switch between the two behaviours? They are both useful in different situations. It would be very easy to make this changeable in Sapphire.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 30, 2015, 02:24:43 am
Maybe leave Shape with its current behavior, and then have specialized constructors that always use one method or the other?  Or do the same thing, but rename Shape to AutoShape.  That would violate the current Galileo spec though...

It's funny you bring up VAOs though, because in the last hour I implemented vertex buffer support for Shapes in minisphere.  It really improves performance by a lot for shapes with lots of vertices.  To test it out I had it make a fan with 1,000,000(!) vertices and it still ran at 60fps on my i3 (took a while to upload all the vertices though...).  Drawing the same shape from a software buffer dropped me to about 20fps.

See, this is the kind of thing that happens when you leave me to my own devices for long: Shit gets done, but not without a ton of goofing off in the process. :P
Title: Re: TurboSphere
Post by: Flying Jester on April 30, 2015, 02:38:48 am

It's funny you bring up VAOs though, because in the last hour I implemented vertex buffer support for Shapes in minisphere.  It really improves performance by a lot for shapes with lots of vertices.  To test it out I had it make a fan with 1,000,000(!) vertices and it still ran at 60fps on my i3 (took a while to upload all the vertices though...).  Drawing the same shape from a software buffer dropped me to about 20fps.


VAO!=VBO.

A VAO (Vertex Array Object) requires OpenGL >= 3.2 (or extensions in older versions), and encapsulates an entire set of vertices and bound buffers. A Vertex Buffer/VBO is just a way to upload data more quickly. You sill need to do all the binding manually, which is where a VAO is faster. A VAO allows these binds to done by the driver/GPU, and allows binding with implicit knowledge of the current state.

Assuming you are using the same divisions I do in Sapphire, VAOs make Groups with many Shapes faster, while VBOs just make the things faster for the total number of vertices, regardless of organization.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 30, 2015, 02:46:16 am
Well I'm actually not sure what Allegro is doing internally.  I just know I called al_create_vertex_buffer()  :)  Low-level stuff isn't really my scene, I'll let the middleware deal with the intricacies. :P. Either way, I was smart enough to fall back on al_draw_prim() if the vbuf can't be created.

That said, does D3D even have an equivalent to VAOs?  Way back when I used to do DX programming I remember reading about "vertex buffers" in the docs, but I don't really know how those map to the OpenGL world.
Title: Re: TurboSphere
Post by: Flying Jester on April 30, 2015, 02:54:59 am
I really don't know. I've never actually dealt with any of the Direct* APIs. I've always managed with SDL+OpenGL.
I would assume that a D3D Vertex Buffer is either the analogue of an OpenGL VBO or VAO.

What you could do is see try making 100000 shapes with 4 vertices, and 4 shapes with 100000 vertices (or similarly different numbers). With Vertex Arrays, I can push many more Shapes, but a similar total number of vertices as without them.

It's also important to note that most GPU's will optimize for shapes that have duplicate vertices, mostly though occlusion and culling. You've got to push unique or semi-unique vertices through to see real performance.
Title: Re: TurboSphere
Post by: Fat Cerberus on April 30, 2015, 09:12:05 am

It's also important to note that most GPU's will optimize for shapes that have duplicate vertices, mostly though occlusion and culling. You've got to push unique or semi-unique vertices through to see real performance.


That doesn't really surprise me.  But of course, the GPU is naturally in a better position to make that optimization if it has all the vertex data on hand, already uploaded.  Otherwise I'm still pushing in all those redundant vertices every frame, making the optimization near worthless.

Honestly all this VBO, VAO, FBO stuff... We could be rid of it entirely if PCs would go to a unified memory architecture, the way consoles have been moving lately.  The GPU/CPU boundary is starting to feel more and more like a relic all the time.  Especially when you consider that most CPUs now have a GPU on-die!
Title: Re: TurboSphere
Post by: Fat Cerberus on April 30, 2015, 10:49:34 am
So I ended up doing this in minisphere to create a specific type of shape:

Code: (javascript) [Select]
var shape = new Shape(verts, texture, SHAPE_TRIANGLE_FAN);


The third parameter is optional and defaults to SHAPE_AUTO (meaning Sapphire's behavior). This avoids creating a bloated type hierarchy for such a simple primitive, and seems very Sphere-like anyway. :)
Title: Re: TurboSphere
Post by: Flying Jester on May 02, 2015, 12:23:14 am
I have a DirectSound version of the TS audio plugin working, as well as optional 32-bit floating point audio when using OpenAL.
I intend to make the OpenAL and DirectSound versions selectable on Windows.

I should note that I'm still warming up for a Windows release, but right now that is limited to getting the audio plugin (which I use in an unrelated project), and libyyymonitor and libfjnet (from Kashyyyk) working on Windows for now.

And boy...I forgot how silly the MSVC export model is :)
Title: Re: TurboSphere
Post by: Flying Jester on May 05, 2015, 09:11:26 pm
TurboSphere compiles on Windows now. It crashes in SpiderMonkey when trying to run a script, but it compiles and does not have any linkage issues.

In my adventures, I created a tool (https://github.com/FlyingJester/dumpbincleanse) that can help make import libraries for Visual C++ when you don't have one.

Ideally, I would compile everything with MingW or Cygwin, but doing so is sometimes broken for SM. This is one of those times. I am using libsndfile, SDL2, and SDL2_image compiled with MingW/GCC 4.9.

I would prefer not to use MSVC simply because it is a total trainwreck of a compiler. It has partial support of almost every standard, but not complete or mostly complete support for any of them, not even C++03. It looks to me like it has a gimpy parser. For instance:



I'm not saying that GCC or Clang are perfect. And Sun Studio is at best a curiosity (Compiler-level C++14 syntax support, but still no working C++03-compliant standard library, for instance). They have huge flaws too. I'm just surprised to see that, after three major releases since I last left it, MSVC still has so many issues like these.
Title: Re: TurboSphere
Post by: Fat Cerberus on May 05, 2015, 09:26:27 pm
I guess I haven't had any real complaints because I use all C.  MSVC's C99 support is similarly abysmal, but at least there it has enough to get by for cross-platform work.  I'll take my sized ints and my C++ style comments and that's all I need... ;)

Honestly though, C++ is just a bloated mess of a language.  It's like if someone took C and then asked a Family Feud survey what features they would want to add to it, and then went and implemented every single one.  Even more so with the latest standards.  And that's just the language--let's not even get into the beast that is the C++ standard library.

C on the other hand gives you a primitive set of tools and doesn't hold your hand at all.  If you should happen to stab yourself with a screwdriver, that's on you.  It's still a better situation, in my opinion, than the huge box of Binford power tools C++ gives you.  MOORREEE POOWEEERRR! :P

But I digress.  Carry on!
Title: Re: TurboSphere
Post by: Flying Jester on May 06, 2015, 03:44:44 pm
Clang, or MingW would be better for cross platform work. They are actually cross-platform, and Clang and MingW are as powerful, if not more so than MSVC. Particularly since the C compiler in MSVC is considered legacy and was at one point even deprecated.

I've added clipping rectangles to Groups.

Code: (JavaScript) [Select]
var g = new Group(...)
g.clipX = 10;
g.clipW = GetScreenWidth()-20;

// Now when g is draw, it is clipped relative to the screen.
g.draw();


I'm also working up to having actual shaders. Internally, it's fully implemented, but a lot needs to be done for script to get them properly.
Title: Re: TurboSphere
Post by: Flying Jester on May 06, 2015, 08:14:28 pm
I've fixed the text drawing system. And again again, SpiderMonkey is proving better for TurboSphere than V8.

Previously, I was doing all sorts of tricks to limit the number of Images generated. Each Image has an OpenGL Texture, and its lifetime is controlled by the JS engine. In fairly simple tests, repeatedly making up new string Images and then losing them would crash since the GL textures were never released. In some cases, I even could crash OS X by doing this. Sidenote, no laughing at Mac OS X for this. I have found similar ways to crash Windows and FreeBSD+Linux--or at least X11 and Mesa--just as easily and just as reliably. I've yet to crash Solaris, though :D

SpiderMonkey, however, can keep up with even very large numbers of Images being created. It also ramps up GC'ing when the total number of objects suddenly increases, which works very well in this case. This means it's fine to just make a new image every time you want to draw new text to the screen!
Title: Re: TurboSphere
Post by: Fat Cerberus on May 06, 2015, 08:45:37 pm
I'd be curious what the results of such a test would be in minisphere.  Duktape's garbage collector is scary efficient.  Ever since I added logging for images, if the image doesn't go away immediately after it goes out of scope, it's gone within 30 seconds.  It blew me away, honestly.
Title: Re: TurboSphere
Post by: Radnen on May 06, 2015, 09:16:41 pm

... if the image doesn't go away immediately after it goes out of scope, it's gone within 30 seconds.  It blew me away, honestly.


Or is it rather lazy? Because anything from time, to memory thresholds can cause GC to kick on. I personally don't mind if junk data stays behind for a long time even in GC'd languages since we have so much RAM these days. I think it's only an issue if you get pretty bad memory leaks.
Title: Re: TurboSphere
Post by: Fat Cerberus on May 06, 2015, 09:21:13 pm
There can be no contention for memory--say the game has been idle a while and suddenly creates a temporary image or two.  Those are finalized within the minute.  Every time.

But yes, with ram being what it is memory isn't really an issue.  The issue is when you're managing more than just memory--in Jester's case, hardware textures.  If the JS engine holds onto scarce resources too long, it can cause issues.
Title: Re: TurboSphere
Post by: Flying Jester on May 07, 2015, 01:35:14 am
If the JS engine holds onto scarce resources too long, it can cause issues.


I wish the old forums were archived. The original Sphere 2.0 project, headed by Kyuu, started out using V8. When I started TurboSphere, he cautioned me that V8 was unsuitable for precisely this reason. He was right.

I've done a little experimentation, and SM seems to free things that have a very small number of handles very quickly. I think it might depend if all those handles have a single root. I have yet to see a single Image that is lost exist for more than a second, and it is almost instant if I am allocating many objects (which is why I think the GC ramps up with allocations).

I suspect this kind of behaviour has to do with Firefox's reputation of using too much memory. Even though it uses less than Chrome now, and most memory usage nowadays is from using ad-block and its ilk. Mozilla is very sensitive about memory usage now, and it makes sense their JS engine is very efficient at collecting garbage.
Title: Re: TurboSphere
Post by: Fat Cerberus on May 07, 2015, 02:54:33 am
I wonder if SM is using refcounting as its primary GC method.  I know Duktape does, and as a result 90% of the time the object will be collected the moment the last reference goes out of scope.  It's really only in difficult cases (circular refs) that things tend to stick around longer.
Title: Re: TurboSphere
Post by: Flying Jester on May 07, 2015, 12:31:03 pm
I believe that SM uses a mark+sweep incremental GC, but I know that they want to change it to be an exact generational GC.

V8 supposedly has a generational stop-the-world GC. You're always reminded when talking to the devs that you should not count on it to run, ever.

...anyways...
So, I enabled the extra warnings flag for SpiderMonkey. It's actually really useful. Stuff like this is printed now:

Code: [Select]

[Engine] Strict Warning in file ./system/scripts/turbo/tileset.js line 198
reference to undefined property i.tex_coords.x1
[Engine] Strict Warning in file startup/scripts/lobby.js line 134
assignment to undeclared variable changed
[Engine] Strict Warning in file ./system/scripts/turbo/font.js line 161
reference to undefined property this.string_cache_size

Title: Re: TurboSphere
Post by: Flying Jester on May 12, 2015, 12:51:11 pm
Everything but Sapphire is working under Windows. Sapphire is having some weird issue where trying to LoadLibrary the binary results in a 193 error. That usually means you are trying to load 64-bit code on a 32-bit machine. But dumpbin claims it is 32-bit, and I do not have a cross compiler on my Windows machine!

A walk through the debugger even shows that Sapphire is partially loaded, and after loading all of the libraries it depends on, they are then unloaded and Sapphire itself is unloaded. If this was GCC-land, I'd guess that one of its dependencies was 64-bit (it is also a LOT easier to check there). But MSVC import libraries prevent this at link-time. Even with my handrolled tool I used to link against mingw libraries, I still examined the dumpbin headers manually, and everything is 32-bit here.

There is a tool to help check dependency and library loading issues, called gflags. Unfortunately, it's considered deprecated by Microsoft and has been removed from the Windows SDK, and there is no real alternative. I'll have to dig about in my old SDK disks to try and find a copy.

I've also got permission to use the Power by Mozilla (https://www.mozilla.org/en-US/about/powered-by/) logo with TurboSphere! :D
Title: Re: TurboSphere
Post by: Fat Cerberus on May 12, 2015, 03:04:57 pm
Okay, real talk: how difficult is it to build Spidermonkey?  Last time I used it Mozilla provided pre-built binaries, which was great.  Now apparently I'd have to build the entirely of Gecko, which is... Daunting, to say the least.
Title: Re: TurboSphere
Post by: Flying Jester on May 12, 2015, 03:11:34 pm
It is super easy on Unix, but a little difficult on Windows.

You don't need to fully build Gecko, although you do need to clone it and run its configure script.

Here is how I do it:



Now your headers are wherever jsapi.h is in js. There is no strong distinction between the Gecko-SpiderMonkey interface and the general embedding interface, so anything in the dist directory is fair game (and any exported symbols in general). You should have built (lib)mozjs-(version).(so/dylib/dll) somewhere in the dist directory.

You will need the mozglue and nspr(4) libraries on Windows, and icu as well. You cannot fully disable icu as a dependency, but I disable a couple features of it so that I don't need all three icu libs.
Title: Re: TurboSphere
Post by: Fat Cerberus on May 12, 2015, 03:16:06 pm
Ugh, those dependencies hurt.  What happened to the days when you just had to distribute js32.dll and call it a day? :P
Title: Re: TurboSphere
Post by: Flying Jester on May 12, 2015, 03:20:19 pm
That was when you didn't want off-thread GC'ing and asynchronous operations, UTF-8, or JIT.

There is a project to remove the hard dependency on NSPR from SpiderMonkey. This is mostly because SM only makes use of a small portion of the NSPR.

All things considered, two libraries built as a part of the build and supplied with it, as well as ICU (your choice, system or private build) are not bad at all.
The number of dlls in the same directory as a program means nothing.
Title: Re: TurboSphere
Post by: Fat Cerberus on May 12, 2015, 03:25:28 pm
Hey what can I say, I like my self-contained executables. :)  (plus  I'm OCD so there's that) Mainly I want to avoid a situation like with Sphere 1.x, where the engine and editor are in the same directory and you get to guess through trial and error which DLLs you need to distribute and which can be left out.
Title: Re: TurboSphere
Post by: Flying Jester on May 12, 2015, 04:07:15 pm
I totally agree, but I base what a dependency is on more than just the number of files. I consider Allegro a very heavy dependency. It handles audio, image loading, video, has an entire software renderer...

The same is true of SDL (although less so), and similarly you can compile things out. Someday I want to be rid of SDL2 completely, and call into WGL/GLX/EGL/CGL myself (I've done it before, they have extremely similar semantics). The main thing that stops me now is input handling, which is horrible everywhere except for Cocoa and Android. X input is a total cluster, and Win32 is ridiculously verbose about it, without even offering the kind of flexibility you would think would come with it.

Perhaps one day I will distribute a subset of SDL2, just the input handler imported into a TurboSphere plugin.

I'd rather have three very small dependencies than one all-mighty one. Like using libsndfile and OpenAL, as opposed to just SDL2's audio--why would a sound driver know anything about file types? Why would your graphics library know anything about sound? In particular, why use an audio library when your OS already has one that you're sound library is calling into? That's why I am using DirectSound itself, not OpenAL, on Windows. OpenAL is a part of CoreAudio on OS X, but I do want an ALSA and an OSS backend for the audio plugin, too. OpenAL at least also does 3D audio mixing, which would be very complex using any other backend.

ICU is different in some ways. There is no UTF-8 handling on Windows or some Unixes. Want UTF-8? You need to do it yourself, or use ICU which does it itself. NSPR is silly, though, since Win32 and Posix both have their own threading libraries. Only writing two backends is simple, and I know from experience that Win32 and pthreads are very similar.
Title: Re: TurboSphere
Post by: Fat Cerberus on May 12, 2015, 04:25:42 pm
Bloat is relative, though.  I cant consider Allegro to be bloat in my case since I touch pretty much every part of it at some point or another.  Plus having a consistent API for everything makes it less likely you're going to make mistakes.  You have no idea how many times I've almost accidentally passed a Duktape context pointer to some completely unrelated library function out of habit.

Also, since I now build Allegro myself, I've compiled out the stuff I don't use (notably, the video codecs and DirectX support).  So I'm very familiar with the concept of customizing my dependencies. :) one of the perks of using open source libraries with liberal licenses. :D

Anyway, I think I've derailed this thread enough.  Carry on! :P
Title: Re: TurboSphere
Post by: Flying Jester on May 12, 2015, 05:30:26 pm
So I figured out the Sapphire issue. If a shared does not expose a shared library main in Windows, no problem, it's not a COM or similar library. But this is an issue when the library is loaded from another shared library that has been LoadLibrary'd. You need to perform a certain call of LoadLibraryEx to get around this.

Excellent error reporting in Win32, as usual :P
Title: Re: TurboSphere
Post by: Fat Cerberus on May 23, 2015, 10:15:09 pm

I would prefer not to use MSVC simply because it is a total trainwreck of a compiler. It has partial support of almost every standard, but not complete or mostly complete support for any of them, not even C++03. It looks to me like it has a gimpy parser. For instance:


  • constexpr functions work, but they literally compile an object file and run its code. There is no JIT for it, like GCC, Clang, and even Sun Studio have. Using network shares like I do, this can quickly make some translation units prohibitively expensive to compile.

  • constexpr expressions simply do not work. This is a basic part of C++11, and unless you know that the code is fully compiled and run to get constexpr function results, it seems bizarre that constexpr functions work and not expressions.

  • inline array and struct initializers for class members are not accepted. The compiler knows what they are, but tells you not to use them.

  • You cannot put immediate initializations of members in template classes, even POD template classes. Since each POD template class instance is effectively a unique class, this is fine in GCC and Clang. Sun Studio tells me it will have it as a future feature. I doubt that. It works in non-template POD structs, strangely.

  • It takes sometimes hundreds of parsing passes. So you only see one or two parsing errors at a time. It's impossible for any compiler to parse C or C++ in only one pass, but this is ridiculous.

  • In certain circumstances, the old C++03 trick of putting a space between closing '>' of nested templates is still necessary.




Just wanted to revisit this because I recently read that VS 2015 is supposed to include support for C++11 and C++14.  How complete that support is (the RC is out now), I don't know, but worth pointing out at least.
Title: Re: TurboSphere
Post by: Flying Jester on May 24, 2015, 04:29:10 pm
That's a long time after GCC and Clang, though. C++11 was basically finished in 2008, and GCC and Clang have had support (in some way, under the name GNU++0x) for even longer.

I understand that they will have longer lead-up time, given they don't release as often. But they are trailing by the better part of a decade when it comes to standards support, and they will probably never fully support C99 or C11. Up to and including VS2010, you couldn't even mix code and data declarations.

I've also read that (http://blogs.msdn.com/b/vcblog/archive/2014/11/17/c-11-14-17-features-in-vs-2015-preview.aspx) many of my complaints about compatibility are still unaddressed. That post has a lot of back-patting. "Partial Compliance" can also be called "Not Compliant". constexpr still doesn't work right, you still can't have variadic templates (which are very useful!), almost all the new initialization list uses are still unsupported (there's kind of no way to fake that if your compiler doesn't have it, too), and still no UTF-8 literals.

It's better, but from a standards compliance viewpoint, it's still way behind GCC and Clang. It's not a good feeling when you have to change a simple, clear, and concise statement into a more awkward one when you port to Windows.
Title: Re: TurboSphere
Post by: Fat Cerberus on May 24, 2015, 05:16:37 pm
To be honest I don't find much of C++ to be "concise and clear".  It often feels like C++ code is too verbose to me.  Especially once templates start getting involved.  auto goes a long way towards addressing it, but IMO it's not enough.

Edit: That said, the C99 support seems to be improving as well.  snprintf() is included now, eliminating the need for #define snprintf _snprintf hackery.  Honestly the biggest thing that keeps me using MSVC isn't so much the compiler as the IDE.  In my opinion nothing beats it in that department.  The UNIX Way(tm) seems to be to either do everything from the command line or alternatively, to write scripts to do everything.  If I had to do that on a regular basis, I think I'd go crazy.  I think I've said this before, but I much prefer to spend my time writing code for the app I'm making than for my build tools.
Title: Re: TurboSphere
Post by: Flying Jester on May 25, 2015, 04:45:06 am

To be honest I don't find much of C++ to be "concise and clear".  It often feels like C++ code is too verbose to me.  Especially once templates start getting involved.  auto goes a long way towards addressing it, but IMO it's not enough.

Edit: That said, the C99 support seems to be improving as well.  snprintf() is included now, eliminating the need for #define snprintf _snprintf hackery.  Honestly the biggest thing that keeps me using MSVC isn't so much the compiler as the IDE.  In my opinion nothing beats it in that department.  The UNIX Way(tm) seems to be to either do everything from the command line or alternatively, to write scripts to do everything.  If I had to do that on a regular basis, I think I'd go crazy.  I think I've said this before, but I much prefer to spend my time writing code for the app I'm making than for my build tools.


It's funny, I feel the opposite.
Once you really grok them, templates are very good. I seriously wish C had templates. They are extremely powerful.
The IDE I find is super slow. Most of my Windows development is almost totally done on a Dell D630. It's super underpowered, but Code::Blocks, Codelite, and Flare (when it doesn't crash =P ) work fine on it, when MSVC doesn't. I understand that it is a good IDE, especially with debugging--lldb is bad!--but it does take a huge amount of resources in comparison to other IDEs.

It doesn't help that CMD.exe is horrendous when compared to any other terminal. I hear it is getting an upgrade in Windows 10, but again I suspect it will be far behind the time compared to Bash, Asn, and the Korn Shell.

I only suffer through this because most folks use Windows.
Title: Re: TurboSphere
Post by: Fat Cerberus on May 25, 2015, 07:55:30 am
I won't deny MSVC is slow (you basically need an i7 to get any decent performance out of it), but yes, it's mostly the debugger that I find invaluable.  You can see literally EVERYTHING at a glance, at a level I haven't seen in any other IDE.  The only thing I wish it had?  Time travel. :P. Nothing is worse than hitting a breakpoint and realizing it would be so much simpler to find a bug if you could only step backwards through the execution path...

As for templates, I don't know.  I see the appeal, but I guess I view them with contempt because they're basically a way to mimic duck typing in what is otherwise a strongly (VERY strongly) typed language.  For classes this usually makes perfect sense (vector<> et al say hi, also C# generics), but when I start seeing templates being used to make function overloads and such that's when I scoff.  Too much template code starts to feel to me like Mad Libs as applied to programming. :P
Title: Re: TurboSphere
Post by: Fat Cerberus on June 02, 2015, 02:23:23 am
So I just had a question as regards the Galileo implementation in TurboSphere.  I kind of rushed the minisphere implementation so I want to make sure I get it right this time.  In TS, what exactly happens when you call group.draw()?  In minisphere the Group is rendered to the backbuffer immediately like any other primitive (the shapes themselves are batched, of course).  I'm thinking this isn't what TurboSphere does, though, since I've seen you mention "asynchronous" in reference to the graphics system.  I assume it's equivalent to this (Async() in minisphere defers until the next FlipScreen):

Code: (javascript) [Select]
Async(function() {
    group.renderAllShapes(); // fictitious method, renders Shapes to backbuffer
});


Is this correct?  If so, I'll make the necessary changes under the hood for v1.2.
Title: Re: TurboSphere
Post by: Flying Jester on June 02, 2015, 04:31:22 am
TurboSphere's entire graphics system inherently is asynchronous, and in a fairly different way than what you describe. Long story short, you don't need to do it asynchronously. For TurboSphere (and conceptually for Galileo), FlipScreen simply means you have finished calling the draw operations for a single frame and are beginning the next frame.

Sapphire is based on a set of queues (originally 5 queues, now 3), each representing a frame. Right after you call FlipScreen, so the start of a frame, a queue is chosen and cleared. Any calls to group.draw() then push the groups drawing operation into that queue.
No actual drawing is done on the main thread. A rendering thread chooses a queue to 'draw', and performs all the operations on the queue until an operation occurs that flags the end of a frame (FlipScreen() pushes such an operation). The rendering thread then clears the screen and chooses another queue to draw from.

Considering there are only three queues, and we lock so that we will not draw and render from the same thread at once, this means that we always pick the only queue that is not currently in use. Under perfect conditions, the rendering and engine thread would just cycle through all three queues in order. That's not normally what happens, though.
If the rendering thread takes a long time rendering (which is pretty common), the engine thread will just bounce back and forth between the two remaining queues, clearing and rewriting them.

There have been a couple interesting things I've noticed about this system. For instance, it's vitally important that Sapphire use an odd number of queues. But it has worked quite well. The engine can run at crazy 'framerates', even with large amounts of drawing. The rendering stays responsive since only a few frames can be backed up, and we intrinsically drop late frames without ever having to consciously consider how and when to do so.
Title: Re: TurboSphere
Post by: Fat Cerberus on June 02, 2015, 08:54:17 am
Huh.  That's pretty clever, actually.  You basically get frameskipping for free because nothing is ever drawn for late frames, not even to the backbuffer.  The thing that concerns me though is, don't you get frames drawn out of order?  Since you imply the queues are picked essentially at random by the render thread...
Title: Re: TurboSphere
Post by: Flying Jester on June 02, 2015, 12:23:51 pm
Out of order rendering can happen, but it's quite rare in practice. You can only have every third frame be out of order with three queues (and only two with five). It requires the engine thread be running notably faster than the rendering thread, and then suddenly slow down to less than half the speed.

In the worst case scenario with three queues, you would get two frames in a row and then one back. But the chances of that happening more than once in row are very low, since the engine thread would need to be slowing down and speeding up at a highly specific rate relative to the rendering queue.

Far more common is the engine thread running at more than twice the render thread's speed and consistently overwriting multiple frames before they are drawn.
Title: Re: TurboSphere
Post by: Flying Jester on December 22, 2015, 05:14:04 pm
I've been thinking of why TurboSphere didn't really live up to what I wanted it to be, and how I would redo it. I think there are the main failings of TurboSphere:



I think that because of these three issues, TurboSphere sort of failed, despite having a basically complete API and good stability. But there were some good points. I think a few really key features to TurboSphere are:



I still like the concept behind Sphere and TurboSphere (and minisphere and all the other engines). But I have a new plan for what a modern Sphere-related engine will look like.

The New Plan

I want to fix the main failings of TurboSphere, while still retaining the good points. The new effort is called UltraSphere (https://github.com/FlyingJester/UltraSphere). UltraSphere still uses JS with SpiderMonkey, and is mostly a rewrite of the native components, but will use the same Turbo Runtime as TurboSphere did. It also features a similar plugin system, although it is slightly more object-oriented--slightly. Mostly a plugin's interface matches the lifetime of a context now, which is a big improvement. But the engine is still rich in script and sparse in native. I think that I was really on to something with pulling absolutely as much out of native as possible, and I want to continue this.

However, maintaining acceptable performance (particularly in regards to memory usage), while using JS as much as possible and also maintaining real compatibility with Sphere 1.5/1.6 is just not feasible. I tried, and I can tell you it's not a good idea. So, I have decided that UltraSphere will not be directly compatible with Sphere 1.5 or TurboSphere. This will be particularly notable with regards to loading images/surfaces, loading audio files, drawing primitives and images, and input handling. Everything that the Turbo Runtime already supports (Map Engine, Fonts, WindowStyles) will remain compatible as much as it already is. I actually plan on extending some Sphere 1.5 compatibility with regards to drawing.

With regards to releases, it was never really a goal of TurboSphere to work on Windows. It would have been nice, but this wasn't where I put my effort. This is not really a good idea if I want anyone except myself to actually use it. I intend on really making UltraSphere work on Windows, and I will make actual releases (like I used to with TurboSphere, long, long ago). This is important for two main reasons. One is that most people run Windows. But I could make a case for "Best engine for OS X and Solaris!" for TurboSphere. Just as important is that there are no existing editors for anything but Windows, and since I still use the Sphere formats this is a serious issue. I could change formats, but I don't really see the need to do so. I think much more beneficial is to just run on Windows and be able to use Radnen's editor or the 1.5 editor.

Finally, I built TurboSphere with a sort of "If you build it, they will come" philosophy. I suppose I intended to ride on the advertising the original Sphere engine would give me. But it's clear to me now that Sphere has almost no discoverability anymore, either. We do not promote the engine actively, and in places like RMN where the engine is known it's almost solely considered to be a relic of a bygone era, similar to ika or verge. (Hint to LordEnglish: Get minisphere on RMN! :) ). I need to actually make real demonstrations and not be so reclusive with the engine if I want any kind of actual usage. Riding on the coattails of the original engine buys any new engine very little.

So, on to how UltraSphere is actually like TurboSphere.

One part of TurboSphere that worked really well was Sapphire (the graphics plugin) and the Galileo API. I was able to get extremely good performance with Sapphire, cycling through asynchronous drawing queues combined with the object-oriented Galileo API which mirrored actual GPU resource usage. I plan on basically copying a lot of the renderer code, and Galileo will mostly stay as it is. However! I have, in my travels, discovered efficient means to draw software overlays on top of OpenGL windows on every platform I care about, including Windows. On an unknown platform this could even be implemented using pure OpenGL, although it would be less efficient. So I plan on making a few more changes to the graphics system. I will add a software overlay. It will be an ArrayBuffer of uint32's, an RGBA framebuffer that will be drawn over the current Galileo scene. It will be totally up to script how to utilize this, but it will alleviate a lot of the issues with using Galileo in the same way one would utilize the Sphere 1.5/1.6 drawing API. This will also make for a clean distinction between procedural drawing and the object-oriented Galileo method. Both are useful in certain circumstances, but the characteristics are quite different. It is important that they not be conflated either in the API or implementation.

I additionally don't want to implement "LoadImage" or "LoadSound" or their similar constructors the way I did before. I had long thought that it would be more elegant to implement a "DecodeImage" and "DecodeSound" system, where decoders for formats are exposed, and it's then up to script to upload those images and audio to surfaces and sounds. An Image and Sound object would just have memory buffers holding the raw data. The decode functions would return ArrayBuffers (or possibly act directly on existing ArrayBuffers). It would be pretty simple to implement a LoadImage and LoadSound on top of these functions. But I think it's important that they are seperated at an engine-level.

The most important part of all of this is that I don't want to keep with the Sphere 1.5/1.6 API. We have just a handful of games that utilize it, and it has some limitations I want to go beyond. My new plan of course allows for shims to be written--just being a JS engine makes that possible. But I will not write them. I know many people here have said it here before, but I agree that to really become a modern 2D engine, we need to look past the 1.5/1.6 API. UltraSphere is certainly less Sphere-like on the surface than even TurboSphere was. But I'm trying to keep the takeaways that I learned from Sphere. I still want to give JavaScript full control of the engine--even the event loop. I still want to provide a good set of graphics and audio features. But the long-fabled Sphere 2.0, this is not.
Title: Re: TurboSphere
Post by: Fat Cerberus on December 23, 2015, 10:46:52 am
Honestly the only reason minisphere ended up as a Sphere 1.x replacement is because it acted as a guide for me, something to work towards, which I later built on.  Originally, I wanted to make a whole new API which was more idiomatic for modern JS--but incompatible with Sphere 1.x--but realized early on that had I done so, it would have been a disaster; my aversion to planning ahead would have turned it into an unfocused, inconsistent mess.  Writing to the 1.x API first let me build something which had long been proven to work, and work well, then once that was finished I was able to build on it with Sphere 2.0 stuff.

UltraSphere sounds like it could be awesome.  The focus on ArrayBuffers worries me though; I've always said Sphere's main strength was that, despite its use of a full programming language and the engine's overall hands-off nature (as you say, the JS controls everything, including the event loop), it is deceptively approachable; this was further proven to me further the other day by my cousin, who has very little programming background, got interested in learning JS thanks to minisphere.  UltraSphere's planned more-low-level approach using buffers and such would lose that accessibility, I think.  This is a big reason I consider the Sphere 1.x API (however it's implemented) to be instrumental in any Sphere engine regardless of its aging facade: It offers something for new users to latch onto while still being reasonably powerful, and once they become proficient they can transition over to the more advanced stuff like Galileo, Audialis, etc. which require more discipline to use properly.

All that said, I wish you luck with UltraSphere and hope it turns into something great.  Honestly though?  I think the biggest thing that killed TurboSphere as a project wasn't anything technical, but more the lack of focus.  Somewhere along the lines I feel like the project lost its focus and didn't really know what it wanted to be.  Which, keep in mind, could kill UltraSphere as well, due to the second-system effect.  Something to keep in mind.
Title: Re: TurboSphere
Post by: Fat Cerberus on December 24, 2015, 12:04:57 pm
Regarding RMN: I checked it out real quick, but I'm not sure how to go about getting minisphere on there.  And they already have a section for Sphere; would minisphere fall under that category instead of being considered a separate engine?  I guess I would have to sign up for their forums and make some posts about it.

I've already done some advertisement of minisphere on the Allegro forums, but it seems Allegro is largely in the same boat as us, with a dwindling userbase (still much healthier than us though... :() and the library itself considered something of a relic, even in spite of Allegro 5 being a complete rewrite and modernization of the API.  So I don't think it gained me much.  RMN would be a better bet, and might even do something to get Allegro back in the spotlight as well.
Title: Re: TurboSphere
Post by: Flying Jester on December 24, 2015, 01:13:56 pm
On RMN, there are a number of RPG Maker clone engines that are listed as games (even though they are not games theselves). You could do that, although that never really struck me as a great solution.

Alternatively, you could add Spectactles and ask Kentona to add minisphere so that you can list Spectacle's engine properly. He generally wants there to be at least one game for an engine before he puts it on the list, and you could use it as a showcase for minisphere, as well.

But also, I think that waiting around and suggesting minisphere when people as "what's the best engine for xyz?" questions could be a good idea. There are a lot of people just starting out on RMN, and even if their games don't really go anywhere most of the time (don't we all have large libraries of games we only barely start to make? :P ) it would give minisphere some listed games on RMN, improving its presence.

Those were pretty much my plans with TurboSphere once I started a good looking game using it, but I always held back because once TurboSphere was actually useful since it wasn't for Windows anymore.
Title: Re: TurboSphere
Post by: N E O on December 24, 2015, 02:18:50 pm
Ideally I'd like to see existing complete Sphere games like Beaker's, Radnen's, sdhawk's, Tengu's, etc stuff and certain tech demos like kamatsu's destructive terrain demo confirmed working on minisphere, and minisphere updated if there's missing or lacking functionality, so that we have a decent amount of results we can point to when demonstrating Sphere's capabilities.

This was kind of the point of the sphere-games repo and my stickied call for devs to provide demos and games, where there would be free complete games distributed with the engine/editor that people could just "pick up and play"  and/or learn from; I dropped the ball on finishing Flappy Sphere even though I had at least 80% of the thing done, though, and I apologize for that.
Title: Re: TurboSphere
Post by: Fat Cerberus on December 24, 2015, 02:25:31 pm

Ideally I'd like to see existing complete Sphere games like Beaker's, Radnen's, sdhawk's, Tengu's, etc stuff and certain tech demos like kamatsu's destructive terrain demo confirmed working on minisphere, and minisphere updated if there's missing or lacking functionality, so that we have a decent amount of results we can point to when demonstrating Sphere's capabilities.


After Christmas I'll look through the download repo again and test more stuff there.  A big hurdle for a lot of Sphere games has been minisphere's lack of mp3 support, and converting them all to Vorbis (and performing the requisite find-and-replace in the game scripts) is tedious, particularly with large games like Aquatis.  I want to remedy that, but since Allegro is unlikely to ever support it natively I'll have to do it myself using Audialis internally.  Anyone know of a liberally-licensed (zlib/MIT/BSD/etc) mp3 decoder library?
Title: Re: TurboSphere
Post by: Flying Jester on December 24, 2015, 02:33:40 pm
All the MP3 decoders or frameworks that handle MP3 that I know of are GPL.

Maybe hook into Windows Media Foundation on Windows and GStreamer elsewhere?
Title: Re: TurboSphere
Post by: Fat Cerberus on December 24, 2015, 10:03:15 pm

All the MP3 decoders or frameworks that handle MP3 that I know of are GPL.


Indeed, some quick Googling corroborates this.  That explains why Allegro doesn't support it, actually: it's zlib licensed and including a GPL decoder would make it impossible to keep such a liberal license.
Title: Re: TurboSphere
Post by: Flying Jester on December 24, 2015, 10:14:26 pm
It's also why libsndfile doesn't support MP3. TurboSphere only supported it way back in the day when I was using BASS, before I wised up.
Title: Re: TurboSphere
Post by: FBnil on December 25, 2015, 02:54:34 pm
Note that you now have to compete with http://div-arena.co.uk/ and https://love2d.org/
they support many platforms to reach a high amount of users.

Also about the deep-programming thing: Someone will write a library eventually that just plays a sound, when I want it, without me having to worry about the buffering. But it is nice to know that you can, say, delay and dim the left channel to have a sound swooshing from the right...

BTW, have you seen this? The commodore 64 impossible mission in a browser: http://impossible-mission.krissz.hu/
Title: Re: TurboSphere
Post by: DaVince on December 29, 2015, 07:44:50 pm

I've been thinking of why TurboSphere didn't really live up to what I wanted it to be, and how I would redo it. I think there are the main failings of TurboSphere:


  • Murky Concepts of Compatibility

  • No Real Releases

  • No Discoverability




I'm a bit late, but I'd like to respond to this too.

As a pure user of Sphere (and only occasionally so), what I looked for in TurboSphere was a stable new version of Sphere. So I agree the compatibility let me down, but more importantly, it always kept crashing in different ways until I basically grew tired of it myself. And then I grew out of Sphere for a while again. This is a rather harsh truth, I suppose. Maybe it's a lot more stable now than it's used to be, but at this point it's a little too late. (Note that I definitely think your efforts with TS were very helpful and I was definitely enthusiastic about it early on until it kinda changed into something I was no longer familiar with, though!)

Now there's minisphere, and it became surprisingly stable, compact and elegant surprisingly quickly, and I don't even feel like anyone should be trying to create yet another version of Sphere. No offense to your plans at all, of course - I just personally feel this way. To me, the "solution" of modernizing the Sphere engine itself has more or less arrived, and at this point it might be better to focus development on that instead (as well as advertising it to the world).

UltraSphere still really sounds cool, but to me at least it feels like it might do better as a whole new engine that happens to borrow concepts from good ol' Sphere. :)
Title: Re: TurboSphere
Post by: Flying Jester on December 29, 2015, 08:16:53 pm
I totally agree. I didn't really support Windows at all, and almost all the crashes were because I simply didn't test my Windows builds.

And a big part of the issues with TurboSphere were because I was torn between making a new Sphere and making something altogether new.

I chose the name UltraSphere to relate it to TurboSphere, but in reality it shares the new Turbo aspects much, much more than the Sphere aspects.

Maybe I should rethink the name. Plus, it would be nice to keep the "Turbo" prefix :)
Title: Re: TurboSphere
Post by: Fat Cerberus on December 29, 2015, 09:39:18 pm
Maybe just call it the Turbo engine?  The vast majority of commercial engines seem to go for one-word names (Source, Rage, Build), so it would fit right in.  The name doesn't seem to be taken either, surprisingly. :)
Title: Re: TurboSphere
Post by: DaVince on December 29, 2015, 11:46:54 pm
I agree; I rather like the name "Turbo". :)