Anything I can do to help debug the issue? I guess I should be running Valgrind and compiling with debugging symbols too, huh?

Edit: by the way, I also made an #engine-dev channel on the new Discord server where we could possibly do this in a more real-time fashion.

@Eggbertx Yep, I just stumbled into that one last night actually, it's a nasty double free + use-after-free caused by:
https://github.com/fatcerberus/minisphere/blob/v4.8.4/src/minisphere/pegasus.c#L626

The backbuffer is exposed in JS as screen, however it's registered into the API without calling image_ref() so once the Surface finalizer runs it frees the last reference.  Then miniSphere goes to free it again on shutdown (or switching games) and bam, crash.  Luckily it's a trivial fix.

See how useful Valgrind is?

Hmm thinking about it I'd be happy doing the image manipulation with an image object that was simply:

Code: [Select]

{
  width:num_pixels,
  height:num_pixels,
  data:array_of_pixels
}

The array could just be 1d but with each element being a color object (or equivalent - just needs the rgba values)

Don't need a 2d construct as can work fine with specified width to tell you which row you're on (could have a helper function that converts coordinates to an index location); as I'd want to combine images having a more complex interface would probably make it harder.

The key things I'd need which we don't have at the moment are methods to:
a) convert png/jpeg files to width + height + pixel array
b) save a width + height + pixel array in such a way that reading it in as an ArrayBuffer and then calling new Texture on it in miniSphere will work.

I probably could implement all of that in JS but it may be slightly painful.


EDIT: Hmm, lets see where I can get to with this starting point: https://github.com/arian/pngjs

To get this going I need a zlib.inflate function I can call, I've found three different zlib.js scripts but I can't get any of them to run they all need either node.js api features or browser api feactures and I can't manage to hack those requirements out of them easily; probably could if I spent a long time at it but this (i.e. cobbling together png support with pure JS) feels like excessive effort for something that could probably be more easily implemented in C.

Posted this on the discord but worth saying here too, for anyone building chakra core on linux/macos to test the latest version of minisphere use:
./build.sh --no-icu -j=4
Optionally you can also add --static to make it build static.

If you call the build script with no options it:
a) insists on linking dynamically to libicu (which is only needed for facilities that minisphere doesn't use)
b) builds with only one thread (so takes ages to build)
c) produces a dynamic library

Note, if you build with --static but without the --no-icu it will build a static library linked to a dynamic version of icu which seems silly to me.

If for some reason you want to include icu you can also use --embed-icu this makes the build script download icu and then it links to it statically so you don't have to worry about it. I can't find a way to fully static link without using this feature.

Good to know about that -j option.  On Windows with MSVC it's a 2-3 minute build while Linux was taking 15-20 minutes for me and I couldn't figure out why.

Finally got breakpoints fully working in SSj.  I couldn't get it to honor breakpoints in a .mjs, JsDiagSetBreakpoint() kept returning an error code.  I thought it was a bug in ChakraCore not letting breakpoints work in modules but it turns out I was just tracking script IDs wrong (trying to set the breakpoint in the wrong script).

Only one thing still doesn't work in SSj: the examine command.  With the Duktape engines you can use it to see all properties on an object and their attributes:

Code: [Select]

#/game_modules/console.js:86 [anon]
(ssj) x Sphere.Game
ew-c  name                "Spectacles: Bruce's Story"
ew-c  version             1
ew-c  saveID              "fatCerberus.spectacles"
ew-c  author              "Fat Cerberus"
ew-c  summary             "Follow Scott Starcross in his quest to stop the Primus from destroying both worlds."
ew-c  resolution          "320x200"
ew-c  main                "@/scripts/main.js"
ew-c  disableAnimations   false
ew-c  disableBattles      false
ew-c  disableSplash       false
ew-c  disableTalking      false
ew-c  disableTitleScreen  false

I haven't implemented object inspection in JSAL yet though, so in the 5.0 alpha examine just acts like a normal eval:

Code: [Select]

#/game_modules/console.js:86 [anonymous function]
(ssj) x Sphere.Game
eval() = {...}

I'd like to be able to build miniSphere from the command line updating an xcode project for even new release is super tedious.

Unfortunately though a few tweaks to the makefile you provide got it to build it just hangs when it opens - the same source built with my xcode project works (well execute game still doesn't work but otherwise it seems good).

@Rhuan I added self-path logic for macOS:
https://github.com/fatcerberus/minisphere/commit/1545ecceb0c1ec4fdcf5a3b0fff733e9263e94d5

One less thing you have to patch

Did some more testing and I think I've identified a clear issue  in miniSphere with chakra - the api functions are just slower to execute; some are worse than others - a particular culprit is new VertexList which takes 50% longer in the chakra version which will be noticeable at run time as it's already a slow function.

In a duktape render function you could just about afford to call new VertexList 50 times per frame and you'd still have a little breathing room for something else - though not much. In a CC render function 35 calls of new VertexList is about your limit per frame.

Initially on doing this new round of tests I thought it may be particularly an issue with new VertexList but I checked some other things for comparison and the issue is more prevalent.

So as another example Rectangle from the sphere v1 api takes 4x longer in its CC implementation than in the duktape version - I note however that it's sufficiently quick that that should rarely be an issue. (new VertexList takes over 20 times as long to run as Rectangle in both versions).

Even IsAnyKeyPressed() takes over twice as long in the chakra version than in the Duktape version.

Other thoughts I think my theory about garbage collection issues earlier was totally wrong - I think it's all down to the slower api function execution times; as for why they're slower I don't know, my guess at first was it would relate to some inefficiency in the way you collect values with the various jsal_require functions but the fact that even a parameterless function like IsAnyKeyPressed was so much slower highly confused me there.

Note: all speed comparisons mentioned above were done using 20,000 repetitions or more and repeated several times

If I'm going to hazard a guess, I'd surmise that crossing the JS<->C boundary is inherently expensive.  In Duktape a native call is super cheap; in fact, other things being equal, it's probably faster to call an API function than an equivalent JS function just because there's less overhead.  Duktape has to set up a stack frame and push the parameters (and then pop them on return), but that's it, otherwise it's just a straight C function call.  A JS call involves a lot more bookkeeping including setting up a new bytecode executor instance, etc.

In an engine like Chakra the story is different: JS is a very dynamic language and optimizations rely on making educated guesses about what the code is going to do based on things like static analysis, observation of the code at runtime, what have you.  Once you call into native code though, all bets are off, the call is essentially a black box from the point of view of the engine and pretty much anything could happen.  JS values might be modified, types changed in unexpected ways, etc.

So yeah, my hypothesis is that making so many API calls causes a lot of optimizations and/or JIT'd code to be thrown away, which is expensive for the much the same reason as a branch prediction failure is expensive on a modern CPU.

I can't see why the API calls should result in JIT'd code being thrown - from my understanding normally that should only happen if variables/parameters used within the JIT'd code unexpectedly change types.

Additionally if it was a general issue with native calls I'd expect there to be a flat increase instead different functions have different increases e.g.

Rectangle takes 4 times longer under chakra
new VertexBuffer takes 1.5 times as long under chakra
IsAnyKeyPressed takes 1.66 times as long
new Shape takes 4.5 times as long
Shape.draw() takes 2.5 times as long

And these ratios are consistent implying something specific about the internal workings of these functions that causes the time increase.

I note that in most cases these functions are still short enough that they'll rarely be a problem anyway despite the increase; there are some exceptions though such as new VertexBuffer which was just barely ok for frequent use under duktape and now really isn't under chakra.

I've done some more detailed tests and written the results out to some tables - see below. These timings are consistent (note that new VertexBuffer is intentionally only called 20,000 times compared with the 200,000 for the others due to its comparative excessive length.

I feel like there has to be some kind of pattern underpinning this but I don't know what.

Times with ChrakaCore miniSphere.

Event	TimeTaken	Number	time for 1	in 8 milliseconds
shape.draw	942	200000	0.00471	1,698.51
new Shape	1414	200000	0.00707	1,131.54
new VertexList	5042	20000	0.2521	31.73
Rectangle	1757	200000	0.008785	910.64
IsAnyKeyPressed	175	200000	0.000875	9,142.86
CreateColor	1691	200000	0.008455	946.19
Color	5	200000	0.000025	320,000.00

Times with Duktape miniSphere.

Event	TimeTaken	Number	time for 1	in 8 milliseconds
shape.draw	420	200000	0.0021	3,809.52
new Shape	319	200000	0.001595	5,015.67
new VertexList	2866	20000	0.1433	55.83
Rectangle	469	200000	0.002345	3,411.51
IsAnyKeyPressed	91	200000	0.000455	17,582.42
CreateColor	337	200000	0.001685	4,747.77
new Color	0	200000	0	n/a

new Color() is ridiculously cheaper than CreateColor()... why?

@Rhuan

Color constructor:
https://github.com/fatcerberus/minisphere/blob/master/src/minisphere/pegasus.c#L1483-L1508

CreateColor:
https://github.com/fatcerberus/minisphere/blob/master/src/minisphere/vanilla.c#L1905-L1926

Notice anything that might cause the latter to be significantly slower?

For anyone who's already downloaded 4.8.6: I just reuploaded the release because I found out I forgot to include Sphere Studio in the installer!