Author Topic: Sphere v2 API discussion  (Read 4899 times)

Offline Rhuan

  • Verified
  • Medium Poster
  • *
  • Posts: 180
    • View Profile
Re: Sphere v2 API discussion
« Reply #90 on: July 11, 2017, 04:20:27 pm »
edit: Dammit, that shader bug bit me after all.  Galileo was using the wrong shader when rendering to a surface because of this:
Quote
I have a feeling there might be a bug with the shader management because Allegro tracks them per-bitmap whereas my code tracks the current shader globally.
At least you've found the issue.

Online Fat Cerberus

  • miniSphere Developer
  • Global Moderator
  • Kitty-Eating Cow
  • *****
  • Posts: 2364
  • *MUNCH*
    • View Profile
Re: Sphere v2 API discussion
« Reply #91 on: July 11, 2017, 05:56:20 pm »
For now I fixed it by just unconditionally setting the shader for every draw operation, but the graphics driver isn't guaranteed to optimize that (i.e. by ignoring a request to bind the same shader), so to avoid dragging performance down, a more proper fix would be to track the active shader per-surface.  On the plus side, switching shaders doesn't seem to be nearly as expensive as setting the render target, likely because doing so involves Allegro creating an FBO for the texture each time.
miniSphere 4.7.1 - Cell compiler - SSj debugger
Forum Thread | GitHub Repo

Offline Rhuan

  • Verified
  • Medium Poster
  • *
  • Posts: 180
    • View Profile
Re: Sphere v2 API discussion
« Reply #92 on: July 12, 2017, 03:32:26 am »
What did the extra set shader commands do to the speed?

Separate question - if you have Dispatch Update and Render scripts and you want to terminate them without using Sphere.exit() how do you do it? (I tried setting variables to store their tokens then using Dispatch.cancel(token)
Code: [Select]
var tokens =[];
tokens[0] = Dispatch.onRender(drawing);
tokens[1] = Dispatch.onUpdate(updating);


//then within the function "updating" on meeting a ertain condition
Dispatch.cancel(tokens[0]);
Dispatch.cancel(tokens[1]);

But I got a hard crash:

Application Specific Information:
abort() called
*** error for object 0x7fefd985d5d0: pointer being freed was not allocated

Thread 4 Crashed:
0   libsystem_kernel.dylib           0x00007fffac2d9d42 __pthread_kill + 10
1   libsystem_pthread.dylib          0x00007fffac3c7457 pthread_kill + 90
2   libsystem_c.dylib                0x00007fffac23f420 abort + 129
3   libsystem_malloc.dylib           0x00007fffac32efe7 free + 530
4   sphereRun                        0x000000010ab2911c async_cancel + 206 (async.c:67)
5   sphereRun                        0x000000010ab3ceb7 js_Dispatch_cancel + 26 (pegasus.c:1429)
6   sphereRun                        0x000000010ab1c8bf duk__handle_call_inner + 675 (duk_js_call.c:1617)
7   sphereRun                        0x000000010ab1e3c1 duk__js_execute_bytecode_inner + 2373 (duk_js_executor.c:4533)
8   sphereRun                        0x000000010ab1d672 duk_js_execute_bytecode + 150 (duk_js_executor.c:2404)
9   sphereRun                        0x000000010ab1c988 duk__handle_call_inner + 876 (duk_js_call.c:1573)
10  sphereRun                        0x000000010ab48685 script_run + 129 (script.c:177)
11  sphereRun                        0x000000010ab2930f async_run_jobs + 105 (async.c:125)
12  sphereRun                        0x000000010ab480db screen_flip + 1549 (screen.c:383)
13  sphereRun                        0x000000010ab41d31 duk_safe_event_loop + 55 (pegasus.c:900)
14  sphereRun                        0x000000010aae9add duk_handle_safe_call + 610 (duk_js_call.c:2145)
15  sphereRun                        0x000000010aae7fb8 dukrub_safe_call + 34 (duk_rubber.c:64)
16  sphereRun                        0x000000010ab41ce5 pegasus_run + 38 (pegasus.c:696)
17  sphereRun                        0x000000010ab3276a _al_mangled_main + 3177 (main.c:280)
18  sphereRun                        0x000000010ab7db7a +[AllegroAppDelegate app_main:] + 23
19  com.apple.Foundation             0x00007fff98454b3d __NSThread__start__ + 1243
20  libsystem_pthread.dylib          0x00007fffac3c493b _pthread_body + 180
21  libsystem_pthread.dylib          0x00007fffac3c4887 _pthread_start + 286
22  libsystem_pthread.dylib          0x00007fffac3c408d thread_start + 13

Online Fat Cerberus

  • miniSphere Developer
  • Global Moderator
  • Kitty-Eating Cow
  • *****
  • Posts: 2364
  • *MUNCH*
    • View Profile
Re: Sphere v2 API discussion
« Reply #93 on: July 12, 2017, 09:59:29 am »
It hurt the framerate quite a bit on my GTX 1070 - from ~215 FPS down to around 120.  That's still better than the 87 I was getting before, but not all that great either.

I tried a different hack where the shader is forcibly set right after a render target change and that improved the situation to around 160 FPS.  I'm not sure why I'm not getting the full 215 with that hack - your code is only drawing to the screen during the main frame loop as far as I can tell, so the render target should never change.
miniSphere 4.7.1 - Cell compiler - SSj debugger
Forum Thread | GitHub Repo

Online Fat Cerberus

  • miniSphere Developer
  • Global Moderator
  • Kitty-Eating Cow
  • *****
  • Posts: 2364
  • *MUNCH*
    • View Profile
Re: Sphere v2 API discussion
« Reply #94 on: July 12, 2017, 10:04:05 am »
Separate question - if you have Dispatch Update and Render scripts and you want to terminate them without using Sphere.exit() how do you do it? (I tried setting variables to store their tokens then using Dispatch.cancel(token)
Code: [Select]
var tokens =[];
tokens[0] = Dispatch.onRender(drawing);
tokens[1] = Dispatch.onUpdate(updating);


//then within the function "updating" on meeting a ertain condition
Dispatch.cancel(tokens[0]);
Dispatch.cancel(tokens[1]);

But I got a hard crash:

Application Specific Information:
abort() called
*** error for object 0x7fefd985d5d0: pointer being freed was not allocated

Thread 4 Crashed:
0   libsystem_kernel.dylib           0x00007fffac2d9d42 __pthread_kill + 10
1   libsystem_pthread.dylib          0x00007fffac3c7457 pthread_kill + 90
2   libsystem_c.dylib                0x00007fffac23f420 abort + 129
3   libsystem_malloc.dylib           0x00007fffac32efe7 free + 530
4   sphereRun                        0x000000010ab2911c async_cancel + 206 (async.c:67)
5   sphereRun                        0x000000010ab3ceb7 js_Dispatch_cancel + 26 (pegasus.c:1429)
6   sphereRun                        0x000000010ab1c8bf duk__handle_call_inner + 675 (duk_js_call.c:1617)
7   sphereRun                        0x000000010ab1e3c1 duk__js_execute_bytecode_inner + 2373 (duk_js_executor.c:4533)
8   sphereRun                        0x000000010ab1d672 duk_js_execute_bytecode + 150 (duk_js_executor.c:2404)
9   sphereRun                        0x000000010ab1c988 duk__handle_call_inner + 876 (duk_js_call.c:1573)
10  sphereRun                        0x000000010ab48685 script_run + 129 (script.c:177)
11  sphereRun                        0x000000010ab2930f async_run_jobs + 105 (async.c:125)
12  sphereRun                        0x000000010ab480db screen_flip + 1549 (screen.c:383)
13  sphereRun                        0x000000010ab41d31 duk_safe_event_loop + 55 (pegasus.c:900)
14  sphereRun                        0x000000010aae9add duk_handle_safe_call + 610 (duk_js_call.c:2145)
15  sphereRun                        0x000000010aae7fb8 dukrub_safe_call + 34 (duk_rubber.c:64)
16  sphereRun                        0x000000010ab41ce5 pegasus_run + 38 (pegasus.c:696)
17  sphereRun                        0x000000010ab3276a _al_mangled_main + 3177 (main.c:280)
18  sphereRun                        0x000000010ab7db7a +[AllegroAppDelegate app_main:] + 23
19  com.apple.Foundation             0x00007fff98454b3d __NSThread__start__ + 1243
20  libsystem_pthread.dylib          0x00007fffac3c493b _pthread_body + 180
21  libsystem_pthread.dylib          0x00007fffac3c4887 _pthread_start + 286
22  libsystem_pthread.dylib          0x00007fffac3c408d thread_start + 13

You're doing the right thing, so that crash is definitely a bug (well, any hard crash is a bug, really, a sandboxed engine should never crash no matter what stupid thing you do).  What jumps out at me is that async_run_jobs() is on the stack; what it looks like is that I didn't account for a job being cancelled in the middle of actually running it and the engine removing it from the queue while it's still executing causes it to blow up.
miniSphere 4.7.1 - Cell compiler - SSj debugger
Forum Thread | GitHub Repo

Online Fat Cerberus

  • miniSphere Developer
  • Global Moderator
  • Kitty-Eating Cow
  • *****
  • Posts: 2364
  • *MUNCH*
    • View Profile
Re: Sphere v2 API discussion
« Reply #95 on: July 12, 2017, 12:04:33 pm »
I can reproduce the crash using this code:
Code: (javascript) [Select]
var updateToken = Dispatch.onUpdate(blowUp);
var renderToken = Dispatch.onRender(blowUp);
function blowUp()
{
Dispatch.cancel(updateToken);
Dispatch.cancel(renderToken);
}

That should be enough to come up with a fix.  Thanks for reporting it. :D

edit: The crash doesn’t happen in a debug build.  What fun...
« Last Edit: July 12, 2017, 01:14:57 pm by Fat Cerberus »
miniSphere 4.7.1 - Cell compiler - SSj debugger
Forum Thread | GitHub Repo

Online Fat Cerberus

  • miniSphere Developer
  • Global Moderator
  • Kitty-Eating Cow
  • *****
  • Posts: 2364
  • *MUNCH*
    • View Profile
Re: Sphere v2 API discussion
« Reply #96 on: July 13, 2017, 12:47:46 pm »
I just added a method to the socket object, .connectTo(), that allows you to reuse a socket after it's been closed.  That's convenient not only for reestablishing dropped connections, but also if an online game wants to change servers on the fly or something, it can use the same socket object, which is handy for encapsulation (e.g. if your connection manager doesn't have access to all the places that might be holding onto a reference to the socket, swapping in a brand new object is not an option).
miniSphere 4.7.1 - Cell compiler - SSj debugger
Forum Thread | GitHub Repo

Offline Rhuan

  • Verified
  • Medium Poster
  • *
  • Posts: 180
    • View Profile
Re: Sphere v2 API discussion
« Reply #97 on: July 15, 2017, 07:01:15 am »
Documentation related point - the v2 API effectively expects a far more intimate knowledge of "normal" JS functions/objects.

With v1 you could basically use the Math object, the various standard data types and Sphere functions. with v2 there's a high chance of finding yourself needed:
- Date
- TextDecoder
- JSON

And maybe others, - I know the v2 api is still WIP (and I've probably contributed to the length of time it will be WIP with my various comments...) but for documentation/tutorial purposes the built in JS objects that are likely to be needed should perhaps be introduced somewhere.

Alternate note, FS.readFile(filename) - could this just return a string instead of an arraybuffer? As a short cut function where there is likely no plan to write the data back out to file having it in array buffer format seems superfluous.

Am I doing something wrong or is this how I'm meant to load a JSON file:
Code: [Select]
var data = JSON.parse(new TextDecoder().decode(FS.readFile(name + ".json")));
« Last Edit: July 15, 2017, 07:05:00 am by Rhuan »

Online Fat Cerberus

  • miniSphere Developer
  • Global Moderator
  • Kitty-Eating Cow
  • *****
  • Posts: 2364
  • *MUNCH*
    • View Profile
Re: Sphere v2 API discussion
« Reply #98 on: July 15, 2017, 10:46:29 am »
To be fair, JSON very quickly became a common thing to find in a Sphere 1.x developer's toolbox.  json2.js was one of most popular Sphere scripts back in the day. :)

It's true that more knowledge of JS built-ins is needed, but making the v2 API more idiomatic was indeed one of my goals.  The C-like setup of the Sphere v1 API is (debatably) more beginner-friendly, but at the cost of increasing the barrier to entry for more experienced JS developers.  On the other hand, if you're like me and get introduced to JavaScript through Sphere, learning it through Sphere v1 hides too much complexity to be an effective teacher.  For example I didn't understand closures for many years because I took the v1 API at face value.

As for FS.readFile() - I guess I could make it return a string.  The reason for returning a buffer was to allow using the function with binary data, but as shown by the DataReader class, it makes much more sense in that situation to just use a full FileStream.  I can say from experience that any time I've ever used FS.readFile it's always been paired with a TextDecoder step.

I don't understand the reason to need to use Date though, at least any more than you'd use it in Sphere v1?
miniSphere 4.7.1 - Cell compiler - SSj debugger
Forum Thread | GitHub Repo

Offline Rhuan

  • Verified
  • Medium Poster
  • *
  • Posts: 180
    • View Profile
Re: Sphere v2 API discussion
« Reply #99 on: July 15, 2017, 11:08:29 am »
To be fair, JSON very quickly became a common thing to find in a Sphere 1.x developer's toolbox.  json2.js was one of most popular Sphere scripts back in the day. :)

It's true that more knowledge of JS built-ins is needed, but making the v2 API more idiomatic was indeed one of my goals.  The C-like setup of the Sphere v1 API is (debatably) more beginner-friendly, but at the cost of increasing the barrier to entry for more experienced JS developers.  On the other hand, if you're like me and get introduced to JavaScript through Sphere, learning it through Sphere v1 hides too much complexity to be an effective teacher.  For example I didn't understand closures for many years because I took the v1 API at face value.

As for FS.readFile() - I guess I could make it return a string.  The reason for returning a buffer was to allow using the function with binary data, but as shown by the DataReader class, it makes much more sense in that situation to just use a full FileStream.  I can say from experience that any time I've ever used FS.readFile it's always been paired with a TextDecoder step.

I don't understand the reason to need to use Date though, at least any more than you'd use it in Sphere v1?
Needing Date is for when you want to lock movement speeds or other actions to set time lengths rather than locking them to the frame rate. (as v1 had GetTime but v2 only has Sphere.now which gives you frame counts).

I'm not really saying I want to change the style of the api just saying that there are points around this that should be reflected in documentation or tutorials.

Online Fat Cerberus

  • miniSphere Developer
  • Global Moderator
  • Kitty-Eating Cow
  • *****
  • Posts: 2364
  • *MUNCH*
    • View Profile
Re: Sphere v2 API discussion
« Reply #100 on: July 15, 2017, 11:25:50 am »
Needing Date is for when you want to lock movement speeds or other actions to set time lengths rather than locking them to the frame rate. (as v1 had GetTime but v2 only has Sphere.now which gives you frame counts).

I'm not really saying I want to change the style of the api just saying that there are points around this that should be reflected in documentation or tutorials.

Yeah, switching to frame counts was a deliberate design choice.  I've done a lot of experiments in the past with time-based updates, and they do work well for getting framerate independence.  The problem with them arises when your game starts to lag; if the framerate drops low enough to actually slow you down (i.e. frameskip is not enough), you're still updating entities at the same rate which can lead to all kinds of issues: Entities getting stuck in walls, teleporting, FP rounding errors building up over time... just all sorts of things that cause glitchy frustration for the player and eventually require workarounds to fix.  So you can sync your game to Date.now() if you know what you're doing, but it's a bit of an advanced technique so I like that there's a learning curve in that particular case.

But yeah, I don't disagree about the documentation.  I thought maybe it was a waste of time documenting built-in objects that are much better documented elsewhere (e.g. MDN), but it might be good to document the JSON functions for example, since they're very useful in save systems, among others.  Plus the manifest format is JSON.
miniSphere 4.7.1 - Cell compiler - SSj debugger
Forum Thread | GitHub Repo

Online Fat Cerberus

  • miniSphere Developer
  • Global Moderator
  • Kitty-Eating Cow
  • *****
  • Posts: 2364
  • *MUNCH*
    • View Profile
Re: Sphere v2 API discussion
« Reply #101 on: July 15, 2017, 11:30:43 am »
Okay, FS.readFile will return a string in 4.7.  FS.writeFile accepts one, as well.
https://github.com/fatcerberus/minisphere/commit/dd8ecefac4b1e24ab78120a38d5ebf4863890a18
miniSphere 4.7.1 - Cell compiler - SSj debugger
Forum Thread | GitHub Repo

Online Fat Cerberus

  • miniSphere Developer
  • Global Moderator
  • Kitty-Eating Cow
  • *****
  • Posts: 2364
  • *MUNCH*
    • View Profile
Re: Sphere v2 API discussion
« Reply #102 on: July 16, 2017, 02:42:50 am »
Separate question - if you have Dispatch Update and Render scripts and you want to terminate them without using Sphere.exit() how do you do it? (I tried setting variables to store their tokens then using Dispatch.cancel(token)
Code: [Select]
var tokens =[];
tokens[0] = Dispatch.onRender(drawing);
tokens[1] = Dispatch.onUpdate(updating);


//then within the function "updating" on meeting a ertain condition
Dispatch.cancel(tokens[0]);
Dispatch.cancel(tokens[1]);

But I got a hard crash:
<snip>

This bug is now fixed, along with several other glitches related to the same root cause.  Cancellation should work properly now, even when called from a running job.

Only a few remaining things to implement for miniSphere 4.7, most importantly fully v1-compatible spriteset handling:
https://github.com/fatcerberus/minisphere/milestone/38

Changes so far:
https://github.com/fatcerberus/minisphere/blob/master/CHANGELOG.md

Not bad for 11 days' work!
miniSphere 4.7.1 - Cell compiler - SSj debugger
Forum Thread | GitHub Repo

Online Fat Cerberus

  • miniSphere Developer
  • Global Moderator
  • Kitty-Eating Cow
  • *****
  • Posts: 2364
  • *MUNCH*
    • View Profile
Re: Sphere v2 API discussion
« Reply #103 on: July 20, 2017, 01:55:01 pm »
@Rhuan: I was able to get your ring demo to top out at ~160 FPS in mode 0.  At one point I had it over 220, but that was only with the shader-switching bug intact (and therefore no output).  The speed appears to be limited by the fact that, when rendering using Galileo, the Sphere backbuffer necessarily uses a different shader from the real GL backbuffer.  So switching render targets on every flip also changes the shader and causes slowdown.  This problem doesn't exist for pure v1 code, as in that case both surfaces will always be using the default Allegro shader and no switching takes place.

Setting the GL backbuffer to always use the same shader as the Sphere backbuffer won't work either, for obvious reasons.

All that said, Galileo rendering is still a lot faster than it was in 4.6.0, and when dealing with a large number of vertices Galileo is definitely the clear winner.

edit: Experimented with different numbers of rings, and with 10k rings both v1 and v2 are about equal (which is to say, abysmal), while with only 100 rings Galileo pulls ahead by a pretty wide margin (900+ FPS vs 770 on my laptop).  So in the end I'm pretty happy with the performance, even if I can't quite reach that nice 215 fps with 1k rings I had before.  Super high framerate doesn't mean much if you can't actually see anything :)
« Last Edit: July 20, 2017, 02:21:55 pm by Fat Cerberus »
miniSphere 4.7.1 - Cell compiler - SSj debugger
Forum Thread | GitHub Repo

Offline Rhuan

  • Verified
  • Medium Poster
  • *
  • Posts: 180
    • View Profile
Re: Sphere v2 API discussion
« Reply #104 on: July 20, 2017, 02:55:30 pm »
Hmm would be nice if there was a way to avoid the shader switching but I guess we at least have to switch for the back buffer.

I'm really looking forward to seeing this new improved speed though, sound like great work.